-
Notifications
You must be signed in to change notification settings - Fork 3
FAQ
The main purpose of this application is to provide a way to explore and learn about linked data resources from within the browser. It allows you to work with RDF data in multiple serialisation formats and run federated queries over one or more remote or local data sources. More details are available from the wiki homepage.
LD-Explorer runs 100% on the client and if you use it to bring in too much data, you will eventually flood the client's memory and your browser will either crash or become unresponsive. To help mitigate against this, currently many of the views within the app will truncate results to a small number of records and many of the built in SPARQL queries have a default LIMIT set in order to protect your browser from crashing. You also have the ability to halt any queries that are taking a long time to run or that are pulling down too much data.
That said, data is consumed in a streaming fashion so you can still use this app to query potentially massive datasets.
Not every conversation about data needs to be about scale - consider tools such as MS Excel, or MS Word - neither of these will scale to billions of pages / sheets but both are still indispensable tools. LD-Explorer is aimed more towards people who want to make considerations about their data model, rather than the amount of data.
It's also worth noting that Linked Data techniques can be made to scale, but such considerations are generally made after value has been proven on a smaller scale - LD-Explorer can still be the tool that helps you prove that value, even if it isn't the tool that eventually provides the solution.
The catalog exists to provide quick access to data sources, and is populated from a JSON file within the source code. If you believe you have an example of a linked open data source that would be valuable to include in this list, suggest the addition via an issue. If instead you want to add your own bespoke or internal data sources that perhaps are only relevant to yourself, you are free to fork LD Explorer for your own purposes.
With the exception of settings utilising the browsers local storage, LD-Explorer currently persists data only in memory, meaning you lose all of your data if you refresh your browser. There are a number of reasons for this.
- Regarding persisting the linked data itself - the inability to persist is very much by design. The nature of data and how we need to handle it changes completely once we start persisting - it is at that point where we need to start concerning ourselves with how we handle the data, potentially slowing down the pace of development. As this is a prototype and the aim is to prove value fast, we cannot afford to lose that pace. For this reason, no data is persisted outside of memory.
- If the thing you want to persist is instead a SPARQL query or a new exploration page that you feel would be of value to the majority of users, you are free to raise an issue and make the suggestion.
The aim of this prototype was not to make it "feature rich" or to add in every conceivable feature we could think of. We wanted to put in only what we believe are the features required to meet the core aims of the tool.
Currently, LD Explorer only allows access to data that is available openly. There is no technical reason that such features could not be included (they are supported by Comunica but they were considered out of scope for this prototype.
These questions all related specifically to Linked Data.
Named graphs are supported, however the various pre-written SPARQL queries you will find within the application generally only search the default graph. You can still import data that uses named graphs and can still run bespoke SPARQL queries to interrogate that data, plus there is a specific page within the explore section which will retrieve all the named graphs within your data sources.
The app itself is really just a user interface over the top of several existing open source libraries within the JavaScript linked data ecosystem, mainly the following.
- Comunica - a knowledge-base querying framework which provides the querying functionality in the app.
- N3 - an RDF utility library used to power the local data source functionality among other things.
- RDF/JS - the Javacript RDF specification.
- SPARQL.js - a SPARQL parser for JavaScript. This is used under the hood in Comunica and for checking syntax on custom SPARQL queries.
Aside from this, the app is built using SvelteKit and Tailwind and TypeScript, with the user interface mostly being built from the ICDS component libary.
© Crown Copyright GCHQ 2024 - This content is licensed under the Open Government License 3.0