Does EDC conform to the Dataspace Protocol?

[VladimirAlexiev]

@jimmarino, @paullatzelsperger, cc @tmberthold, @nkapetanas, @robert-david

#4242 and #3305 talk about payload that:

• Uses the edc: <https://w3id.org/edc/v0.0.1/ns/> namespace. That namespace still doesn't resolve (it redirects to the general EDC documentation page)
• Uses it through @vocab, which practice is discouraged by the JSON-LD creators since it blindly maps any key "foo" (even misspelled ones) to that namespace
• Doesn't use DCAT, at least not at the top level
• Uses props like contenttype that are not the same as DCAT props with the same meaning

We've seen similar payload in the Sovity dataspace that extends EDC and that we use:

• https://github.com/sovity/edc-ce/blob/integration/mds22/extensions/broker-server-postgres-flyway-jooq/src/main/resources/db/testdata/V2_1__PoC_Test_Data.sql#L24
• Inspection in the browser shows similar stuff (and even a distinct JSON field called jsonLd that carries the JSON-LD of the overall message as a string: this doesn't conform to JSON-LD specs).

But the Dataspace Protocol specifies the use of DCAT, eg see

• https://docs.internationaldataspaces.org/ids-knowledgebase/v/dataspace-protocol/catalog/catalog.protocol
• https://docs.internationaldataspaces.org/ids-knowledgebase/v/dataspace-protocol/catalog/catalog.protocol#id-3.2-ack-dataset

Can you point us to specs and examples that show whether EDC payload conforms to the Dataspace Protocol (DCAT), and how edc: classes and props are used together with

[jimmarino]

No implementation can "conform" to DSP because a TCK has not been published at Eclipse. This is a work in progress and EDC does intend to pass the TCK when it is available.

Also, note that DSP profiles DCAT and ODRL, which will imply a subset of those specifications.

[VladimirAlexiev]

Ok @jimmarino , I understand that DSP uses particular subsets of DCAT and ODRL.
But my question is whether EDC uses the same profiles, and what is the role of edc: classes and properties in this process.

Eg can you show the payload of a Catalog response, and how dcat: and edc: terms are used together?

[jimmarino]

I'm not sure what you are asking. However, if you want an example of a catalog response message, look at the specification or capture a request off the wire.

We are not using Json-LD for semantic/RDF purposes. The EDC is specifically designed to sidestep the black hole of semantic data by simply storing asset properties in expanded form. In other words, EDC can convey data that has associated semantic meaning, but it is not a design goal to somehow process it other than through Json-Ld expansion. Users can write extensions or custom code to handle semantic data if they want.

[VladimirAlexiev]

@jimmarino which specification?

[tmberthold]

Hello, since I was made aware of this, here are some notes:

• On the DSP side between two connectors, the catalog response DSP "specification" can be found here https://docs.internationaldataspaces.org/ids-knowledgebase/v/dataspace-protocol/catalog/catalog.protocol#id-3.1-ack-catalog
• On the EDC side, APIs, such as the Management-API, can of course be used to trigger such a catalog query DSP-flow from one connector to another connector. This can also be the same connector for test/dev purposes, which should still work up to this point (but no further).

Please contact us for detailed questions for specific connectors.

[skarampatakis]

Hi, adding to the conversation two example data, one for a catalog asset as provided on the connector and then it's jsonld representation which is also part of the former. We are using the Sovity CaaS connector which is currently in version 8.1.0 which is if I am not wrong aligned with edc-core in version 0.2.1.3

response.json

{
        "endpoint": "https://connector1/api/dsp",
        "participantId": "https://connector1",
        "asset": {
            "assetId": "test01-1.0",
            "connectorEndpoint": "https://connector1/api/dsp",
            "participantId": "https://connector1",
            "title": "test01",
            "creatorOrganizationName": "sovity GmbH",
            "language": "https://w3id.org/idsa/code/EN",
            "description": "# Hello World\nThis is a test dataset to use the data transfer on the sovity connector",
            "descriptionShortText": "Hello World This is a test dataset to use the data transfer on the sovity connector",
            "isOwnConnector": false,
            "publisherHomepage": "https://example.com",
            "licenseUrl": "https://example.com/license",
            "version": "1.0",
            "keywords": [
                "keyword",
                "concept"
            ],
            "mediaType": "custom/type",
            "landingPageUrl": "https://example.com/documentation",
            "httpDatasourceHintsProxyMethod": false,
            "httpDatasourceHintsProxyPath": false,
            "httpDatasourceHintsProxyQueryParams": false,
            "httpDatasourceHintsProxyBody": false,
            "dataCategory": null,
            "dataSubcategory": null,
            "dataModel": null,
            "geoReferenceMethod": null,
            "transportMode": null,
            "sovereignLegalName": null,
            "geoLocation": null,
            "nutsLocations": [],
            "dataSampleUrls": [],
            "referenceFileUrls": [],
            "referenceFilesDescription": null,
            "conditionsForUse": null,
            "dataUpdateFrequency": null,
            "temporalCoverageFrom": null,
            "temporalCoverageToInclusive": null,
            "assetJsonLd": "{\"@id\":\"test01-1.0\",\"@type\":\"https://w3id.org/edc/v0.0.1/ns/Asset\",\"https://w3id.org/edc/v0.0.1/ns/properties\":{\"http://purl.org/dc/terms/license\":\"https://example.com/license\",\"http://www.w3.org/ns/dcat#keyword\":[\"keyword\",\"concept\"],\"https://semantic.sovity.io/dcat-ext#httpDatasourceHintsProxyBody\":\"false\",\"https://semantic.sovity.io/dcat-ext#httpDatasourceHintsProxyPath\":\"false\",\"http://purl.org/dc/terms/language\":\"https://w3id.org/idsa/code/EN\",\"http://purl.org/dc/terms/description\":\"# Hello World\\nThis is a test dataset to use the data transfer on the sovity connector\",\"http://purl.org/dc/terms/publisher\":{\"http://xmlns.com/foaf/0.1/homepage\":\"https://example.com\"},\"https://semantic.sovity.io/dcat-ext#httpDatasourceHintsProxyMethod\":\"false\",\"http://www.w3.org/ns/dcat#version\":\"1.0\",\"http://www.w3.org/ns/dcat#distribution\":{\"http://www.w3.org/ns/dcat#mediaType\":\"custom/type\"},\"http://purl.org/dc/terms/creator\":{\"http://xmlns.com/foaf/0.1/name\":\"sovity GmbH\"},\"http://www.w3.org/ns/dcat#landingPage\":\"https://example.com/documentation\",\"http://purl.org/dc/terms/title\":\"test01\",\"https://w3id.org/edc/v0.0.1/ns/id\":\"test01-1.0\",\"https://semantic.sovity.io/dcat-ext#httpDatasourceHintsProxyQueryParams\":\"false\"}}",
            "customJsonAsString": null,
            "customJsonLdAsString": "{}",
            "privateCustomJsonAsString": null,
            "privateCustomJsonLdAsString": "{}"
        },
        "contractOffers": [
            {
                "contractOfferId": "someId",
                "policy": {
                    "policyJsonLd": "{\"@id\":\"someid2\",\"@type\":\"http://www.w3.org/ns/odrl/2/Set\",\"http://www.w3.org/ns/odrl/2/permission\":[{\"http://www.w3.org/ns/odrl/2/target\":\"test01-1.0\",\"http://www.w3.org/ns/odrl/2/action\":{\"http://www.w3.org/ns/odrl/2/type\":\"USE\"},\"http://www.w3.org/ns/odrl/2/constraint\":[{\"http://www.w3.org/ns/odrl/2/leftOperand\":{\"@value\":\"ALWAYS_TRUE\"},\"http://www.w3.org/ns/odrl/2/operator\":[{\"@id\":\"http://www.w3.org/ns/odrl/2/eq\"}],\"http://www.w3.org/ns/odrl/2/rightOperand\":{\"@value\":\"true\"}}]}],\"http://www.w3.org/ns/odrl/2/prohibition\":[],\"http://www.w3.org/ns/odrl/2/obligation\":[],\"http://www.w3.org/ns/odrl/2/target\":\"test01-1.0\"}",
                    "constraints": [
                        {
                            "left": "ALWAYS_TRUE",
                            "operator": "EQ",
                            "right": {
                                "type": "STRING",
                                "value": "true"
                            }
                        }
                    ],
                    "errors": []
                }
            }
        ]
    }

assetJsonLd unescaped and formatted

{
   "@id":"test01-1.0",
   "@type":"https://w3id.org/edc/v0.0.1/ns/Asset",
   "https://w3id.org/edc/v0.0.1/ns/properties":{
      "http://purl.org/dc/terms/license":"https://example.com/license",
      "http://www.w3.org/ns/dcat#keyword":[
         "keyword",
         "concept"
      ],
      "https://semantic.sovity.io/dcat-ext#httpDatasourceHintsProxyBody":"false",
      "https://semantic.sovity.io/dcat-ext#httpDatasourceHintsProxyPath":"false",
      "http://purl.org/dc/terms/language":"https://w3id.org/idsa/code/EN",
      "http://purl.org/dc/terms/description":"# Hello World\nThis is a test dataset to use the data transfer on the sovity connector",
      "http://purl.org/dc/terms/publisher":{
         "http://xmlns.com/foaf/0.1/homepage":"https://example.com"
      },
      "https://semantic.sovity.io/dcat-ext#httpDatasourceHintsProxyMethod":"false",
      "http://www.w3.org/ns/dcat#version":"1.0",
      "http://www.w3.org/ns/dcat#distribution":{
         "http://www.w3.org/ns/dcat#mediaType":"custom/type"
      },
      "http://purl.org/dc/terms/creator":{
         "http://xmlns.com/foaf/0.1/name":"sovity GmbH"
      },
      "http://www.w3.org/ns/dcat#landingPage":"https://example.com/documentation",
      "http://purl.org/dc/terms/title":"test01",
      "https://w3id.org/edc/v0.0.1/ns/id":"test01-1.0",
      "https://semantic.sovity.io/dcat-ext#httpDatasourceHintsProxyQueryParams":"false"
   }
}

[VladimirAlexiev]

@tmberthold Can you provide some examples of DCAT JSON-LD exchanged by EDC/Sovity?
@skarampatakis shared above some payload from the test EDC/Sovity instance you set for us, and I don't see DCAT.
Also, the "assetJsonLd" string goes against all W3C specs about exchanging JSON-LD.

[tmberthold]

Hi - as previously mentioned, the examples for the DSP-messages can be found in the DSP-specification:
https://docs.internationaldataspaces.org/ids-knowledgebase/v/dataspace-protocol/catalog/catalog.protocol#id-3.1-ack-catalog

E.g. concrete example here:
https://github.com/International-Data-Spaces-Association/ids-specification/blob/main/catalog/message/example/catalog.json

E.g. the schema for it here:
https://github.com/International-Data-Spaces-Association/ids-specification/blob/main/catalog/message/schema/catalog-schema.json

[skarampatakis]

Hi @tmberthold , which request to the connector would generate a message similar to the catalog.json message? I grabbed the above mentioned examples from the response of the endpoint api/management/wrapper/ui/pages/catalog-page/data-offers but I suspect this does not use the dsp, as it is mentioning the ui wrapper.

[tmberthold]

For this you need to understand the difference between DSP-messages (between Connectors) and the returns of the APIs (e.g. Management-API or our API-Wrapper in this case). I.e. you are now looking for a message that is sent between two Connectors to communicate with each other on a standardized level -> DSP-messages. The other thing are interactions and responses from APIs of the Connector the user or whatever systems can interact with, which may require DSP-messages communication beforehand before returning the API-response between Connectors to get the catalog or whatever from the other connector.

By calling the API you mentioned for example, an exchange of DSP-messages is triggered between Connectors, but you do not get the DSP-messages back as a response of the API you called, but rather what the API provides as useful for it's use-case.

Here the introduction might be useful again:
https://docs.internationaldataspaces.org/ids-knowledgebase/v/dataspace-protocol#introduction

(Please correct me if I'm wrong or explaining it too confusingly.)

[skarampatakis]

I get it that those are different layers, but according to the above mentioned document, my understanding is that there should be the https:///catalog/request endpoint where I can POST a request to get info about the catalog(ues) on a catalog service (part of the connector) Is this correct? In such case, what would be the proper path when we are using the edc connector? From the UI I get that the connector endpoint is https://example.host/api/dsp

https://docs.internationaldataspaces.org/ids-knowledgebase/v/dataspace-protocol/catalog/catalog.binding.https#id-2.1-the-catalog-request-endpoint-provider-side

[paullatzelsperger]

quick reminder to @skarampatakis and @tmberthold that this is the EDC repository, and discussions should be strictly limited to EDC topics.
The response.json above looks like it does not come from an EDC API, so it is of no relevance here.

I believe the original question of "Does EDC conform to the Dataspace Protocol?" was answered, so I will mark it such.

Furthermore, I believe the discussion has veered far enough off the topic that we can close it.

Be advised that questions about downstream projects and statements bordering on advertisements will not be tolerated.

[VladimirAlexiev]

statements bordering on advertisements will not be tolerated

@jimmarino This is uncalled for. We're just trying to make ends meet here.

[jimmarino]

@VladimirAlexiev, what is uncalled for? As @paullatzelsperger mentioned, it is a strict policy set by the project committers that we do not allow commercial advertisements in our discussions. This is an open source project, not a billboard.

[paullatzelsperger]

specifically

Please contact us for detailed questions for specific connectors.

could be read as "if you have questions about one of our products, please call our sales department" and i'd rather err on the side of caution, especially since neither the discussion nor the actual connector are relevant to this discussion.

In any event, this discussion is closed. No point in arguing further.