Discussing a rework for conditional imports

[pietercolpaert]

Problems

The member extraction algorithm resolved a part of the use cases for having conditional imports in the spec. However, two problems imports solve still remain valid:

Problem 1: Fragmenting on external paths

A property is being used as a tree:path parameter, but it is not part of the actual member. Therefore, when the

Example of when this might be useful:

• Geospatially fragmenting sensor observations, while the actual sensor’s data is available in another collection
• Geospatially fragmentation departures of trains, when the geospatial location can be found based on the station’s geospatial location managed in another collection

Problem 2: importing streams

One should be able to register on a pubsub stream when the node or collection is going to update at a certain moment through that stream.

Solution

Conditional imports

A conditional import can be defined as follows - it defines an extra HTTP request that can be done if from the current page it must follow the link if it needs graph patterns that match the path.

ex:N1 tree:import [
    tree:path ( lc:departureStop gsp:asWKT );
    tree:import <stops.ttl> 
] .

Import Stream

Also tree:importStream was defined which allows one the subscribe on a websockets stream or an SSE stream.

Questions

Conditional imports: Is this design still adequate to the use case? Is the term import correctly chosen?

ImportStream: what can be expected from that stream? Should we constrain it to only view descriptions so that it always updates the full set of members? Or should we also allow to it being defined on a subset? Should we also define it to describe its updates through activity streams, as members can also be deleted?

[pietercolpaert]

From the call of 2023-11-08:

• Conditional imports disallow more hypermedia controls later. However, through the data portal on the first landing page of your server, you should be able to find the gsp:asWKT property as part of the shape of the other collection. Maybe the conditional imports are not needed after all?
• ImportStreams only is about updating and depends on the type of updates you have in your collection. Maybe we should consider linking a collection to an LDES, and make importStream only viable on an LDES?
• Could be related to transactions in LDES as well: Distributed transactions in LDES SEMICeu/LinkedDataEventStreams#46 - still needs to be standardized there

That way, imports could be entirely removed from the TREE specification.

[pietercolpaert]

From the call of 2024-06-05:

• ImportStreams should go to LDES indeed
• Conditional Imports: should be solved on the discovery (DCAT) level

Conditional imports should go: they don’t work on mixed collections/streams

Actions:

1. Remove the imports section all together
2. Open a work item to describe interconnectedness/dependencies between datasets in the discovery spec that is being split off
3. ImportStreams, maybe based on VOCALS, should become a work item in LDES