Roadmap details: Multiple graphs, scalability

[jakob-beetz]

Hi @marcelotto,

thank you so much for this awesome project, great ideas, extensive documentation and making this available.

I came across your work during initial design sketches for the development of a domain-specific, collaborative and federated vocabulary editor for a niche domain (building and construction) where statement-level provenance and version tracking is a fundamental requirement. Basically, there are modelling vocabularies for national building classifications, concepts, properties and relations that come from the ancient STEP EXPRESS ISO 10303 world along with open but niche definitions of meta vocabularies for provenance tracking of such models that allow groups of experts to model their sub-domains independently ... long story. [^1]

After having been down the rabbit hole of reefication for this a number of years back and looking into approaches like TimBLs Delta [^2], I thought "RDF-star is the hammer that was invented exactly for this nail".

I am really excited about your general approach, the notion of speech acts and the PROV-O and DCAT versioning and really hope this goes forward.

Being old and an architect by education, my toolbox in this domain is somewhat limited and the elixir wrench is not (yet?) in there. I would seriously consider taking up the chance to look into this.

Here are some questions for which I would greatly appreciate your thoughts on:

• With Rdf/SPARQL-star standardization still in progress, how "future-proof" is the current pathway? You mention in the docs that e.g. the introduction of multiple Named Graphs in a future version will probably break backwards-compatibility. But the general direction seems solid enough to migrate in that direction, does it not?

• are you aware of similar approaches and teams working in this direction?

• Fuseki and OxiGraph as quadstore backends are great, any other adapters (GraphDB) in the making? How would you approach auth? SOLID ? LDP?

• What would be the effort to incorporate this into non-elixir tool chains, creating bindings, HTTP/REST API (see earlier discussions here)? The proposed approaches and rtc: and og: vocabularies of course are technology-agnostic and the transactions could be implemented in other ecosystems, right?

Thank you very much for your time and effort!
Jakob

[^1] DOI:10.1201/9781482266665-35

[^2] Delta: an ontology for the distribution of differences between RDF graphs

[marcelotto]

Thank you for your interest in Ontogen and your thoughtful questions. I'm glad to hear that the project aligns with your needs.

• With Rdf/SPARQL-star standardization still in progress, how "future-proof" is the current pathway? You mention in the docs that e.g. the introduction of multiple Named Graphs in a future version will probably break backwards-compatibility. But the general direction seems solid enough to migrate in that direction, does it not?

The ongoing standardization within RDF 1.2 is indeed a critical point. Some of the approaches currently under discussion, which deviate significantly from "classic" RDF-star, are somewhat concerning. However, given the relatively wide adoption of "classic" RDF-star implementations in numerous triple stores and datasets created with them, I can't imagine a complete break. In my opinion, doing so would only lead to various "RDF-star" implementations.

Regarding the expected breaking changes mentioned in the article: while there are no plans to change the general versioning model, supporting versioning of datasets with multiple graphs will require tracking the graph to which the propositions/compounds belong. This will become part of the hash and thus their URIs, changing all hashes in the histories. However, a potential general migration strategy is emerging with another major planned development during the next year funded by NLnet. Until this is available, I would be cautious about productive use if you're not prepared to manually migrate future histories or remain on an old version.

• are you aware of similar approaches and teams working in this direction?

Some projects are mentioned in this thread: https://x.com/fkleedorfer/status/1665982697779363840
However, there's no further information available about the only RDF-star based system mentioned there, starvers.

I was also informed via DM by an employee of a major tech company that they are working on something very similar. Although this company has already released several popular OSS projects, they couldn't confirm if this is also planned for this project.

• Fuseki and OxiGraph as quadstore backends are great, any other adapters (GraphDB) in the making? How would you approach auth? SOLID ? LDP?

More adapters are definitely planned. However, not before support for multi-graph datasets is implemented (also planned as part of the next funding).

• What would be the effort to incorporate this into non-elixir tool chains, creating bindings, HTTP/REST API (see earlier discussions here)?

Currently, outside of projects running on the BEAM VM (i.e., Elixir, Erlang, etc.), integration is primarily possible through the CLI. In the short term, this will be improved through support for IO streams, allowing for more efficient and flexible data exchange between Ontogen and other systems. In the long term, an HTTP interface is indeed planned to be provided for these purposes (however, this is not yet part of the next planned funding year). Until this is planned in more detail, I can't give recommendations on auth.

The proposed approaches and rtc: and og: vocabularies of course are technology-agnostic and the transactions could be implemented in other ecosystems, right?

Correct. The implemented versioning model described in the articles can also be implemented in other ecosystems using these vocabularies.

As soon as the final approval of the funding for the development of the next year is through, I will make the agreed roadmap public. The corresponding process is in the final step and should be completed, hopefully successfully, in the next few days. 🤞

[jakob-beetz]

Thank you for the quick, extensive and very helpful reply!
That gives a lot of hope for the future and I will continue to look to invest in this approach. I will certainly make a few experiments using the CLI to do git-style versioning using ontogen, but will also look further into the conceptual challenges doing multi-graph rdf-star versioning this way. Compounds of Compounds from with dcat seem to be endlessly nestable.
It's a pity that apart from pyoxigraph wrappers no rdf-star tooling is available in python , but again this might be the chance to look closer at rdf.ex and this ecosystem.

As soon as I have a MVP of our research prototype, I will come back with some reports.

Really keeping my 🤞 for the next funding round, this would be so beneficial.

Thank you for your time!

[marcelotto]

I look forward to hear about your experiences as you experiment with Ontogen.

I wanted to let you know that we've just released version 0.1.3, which addresses an issue with timeouts on larger queries.

However, I should also point out some current limitations that I've recently added to the README. See here: https://github.com/ontogen/cli?tab=readme-ov-file#current-limitations
These limitations are important to keep in mind as you explore the system. I'm actively working on addressing these in future versions.

On this note, I'm thrilled to announce that the funding for the next development phase has been approved! 🥳 I'll be publishing a public roadmap soon.

[marcelotto]

If you're considering learning Elixir to work with RDF.ex, you'll gain much more than just a tool for RDF processing. You'll be joining a vibrant community and discovering a very powerful technology stack. 🌈🌟

According to the recent StackOverflow developer survey, Elixir is currently the second most admired programming language, and Phoenix (its web framework) is the most admired web framework.

To get an overview of what this stack has to offer, I recommend watching this talk: https://www.youtube.com/watch?v=wWRDysIpMFQ

Here's an excellent, free course to begin your Elixir journey: https://exercism.org/tracks/elixir

[marcelotto]

Here's a blog post on the roadmap: https://ontogen.io/blog/2024/08/22/roadmap.html