Skip to content

[ENH][BEP028] Specification update for BEP028 BIDS-Prov #2099

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 74 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
3b64a2b
Adding dir at dataset level
bclenet Mar 18, 2025
45f6af4
Adding dir at any level
bclenet Mar 18, 2025
4c315ce
Adding rule for dir at dataset level
bclenet Mar 18, 2025
b490c91
Adding objects: JSONLD extension + top level dir
bclenet Mar 18, 2025
f2a67a6
Adding object : prov entity
bclenet Mar 18, 2025
d9bb7b0
Add suffixes in object.suffixes
bclenet Mar 18, 2025
64e2c65
GeneratedByProv and SidecarGeneratedBy metadata
bclenet Mar 18, 2025
acc6af0
prov entitiy and provenance json/jsonld files
bclenet Mar 19, 2025
34fafe1
provenance json/jsonld files inside new rules file
bclenet Mar 19, 2025
f73b668
Merge branch 'master' into BEP028_schema
bclenet Apr 10, 2025
a1fbb8a
[SCHEMA] removing provenance subdir level description
bclenet Apr 10, 2025
6622d08
Yaml lint
bclenet Apr 10, 2025
b05b950
[SPEC] first draft for BEP028 BIDS-Prov
bclenet Apr 10, 2025
9ae1649
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 10, 2025
f572174
Adapting markdown to mkdocs
bclenet Apr 22, 2025
ee0d30d
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 22, 2025
b619de1
Broken paragraph links
bclenet Apr 23, 2025
74722b1
Citing BIRS Provenance + removing 'BIDS-Prov' from spec
bclenet Apr 23, 2025
34411ba
Into JSON only provenance files
bclenet Apr 25, 2025
248d2b8
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 25, 2025
99faa51
Into JSON only provenance files
bclenet Apr 25, 2025
ac7aff4
Into JSON only provenance files
bclenet Apr 25, 2025
6289722
Merge branch 'master' into BEP028_schema
bclenet Apr 25, 2025
3fef3dc
Merge branch 'spec-refac' into BEP028_spec
bclenet Apr 25, 2025
03026d1
Flags for markdown code blocks
bclenet Apr 25, 2025
a3dd018
Merge branch 'master' into BEP028_spec
bclenet Apr 25, 2025
0202806
Typo in provenance.md
bclenet Apr 25, 2025
58055b7
Remark check
bclenet May 6, 2025
a3cef66
Removing non latin content
bclenet May 7, 2025
c9a65b7
Merge branch 'master' into BEP028_spec
bclenet May 13, 2025
4f89eff
Removing inline description of activities inside dataset_description
bclenet May 15, 2025
3d45ac9
Update description for suffixes
bclenet May 15, 2025
ae91f1d
Adding provenance contexte
bclenet May 15, 2025
197e173
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 15, 2025
fdd60f2
GeneratedByProv only contain uri(s)
bclenet May 20, 2025
710f008
Merge branch 'master' into BEP028_schema
bclenet May 20, 2025
93c75e1
Change name for GeneratedBy in JSON sidecars
bclenet May 21, 2025
2b8ad75
Merge branch 'master' into BEP028_schema
bclenet May 21, 2025
0211151
Adding Provenance records in the schema: Activities, Entities, Softwa…
bclenet May 21, 2025
b2ccd67
Adding metadata info for provenance records
bclenet May 21, 2025
46a7ea8
Change provenance suffixes definitions
bclenet May 21, 2025
b153ad0
Merge branch 'master' into BEP028_spec
bclenet May 21, 2025
f219c11
Refactoring provenance spec
bclenet May 21, 2025
29b2166
Refactoring provenance spec
bclenet May 21, 2025
c02c227
Adding ActedOnBehalfOf
bclenet May 23, 2025
a6c6943
Codespell
bclenet May 23, 2025
0598f8b
Merge branch 'master' into BEP028_spec
bclenet May 27, 2025
ded0420
Provenance datatype + removing base suffix
bclenet Jun 3, 2025
aabf81f
Merge branch 'master' into BEP028_schema
bclenet Jun 3, 2025
2f2b6a2
Merge branch 'master' into BEP028_spec
bclenet Jun 3, 2025
a61fe5a
Update examples
bclenet Jun 3, 2025
27459f2
Remark & latin-phrases
bclenet Jun 3, 2025
6bfbb50
Remark & latin-phrases
bclenet Jun 3, 2025
06ff6bc
Merge branch 'master' into BEP028_schema
bclenet Jun 10, 2025
2597af3
Merge branch 'master' into BEP028_spec
bclenet Jun 10, 2025
0dc7b6a
Merge branch 'BEP028_schema' into BEP028_spec
bclenet Jun 10, 2025
5ca65af
Latin + Yaml lint
bclenet Jun 10, 2025
d62fc27
Digest and Type added as metadata for Entity records
bclenet Jun 10, 2025
1fbc042
Using macros for records files
bclenet Jun 17, 2025
b24deb9
Using macros to define provenance in sidecar JSON and dataset_descrip…
bclenet Jun 18, 2025
9fae7b6
YML lint + codespell
bclenet Jun 18, 2025
053f6b1
Issue with provenance spec link
bclenet Jun 18, 2025
208131b
Merge branch 'master' into BEP028_spec
bclenet Jun 18, 2025
792c617
Provenance of derived dataset
bclenet Jun 18, 2025
e9796f8
Link issues
bclenet Jun 18, 2025
f5ea386
Entity Ids are BIDS URIs
bclenet Jun 19, 2025
c4e926d
Remove GeneratedByProv -> integrated into GeneratedBy for dataset lev…
bclenet Jun 19, 2025
0a0368f
Entities allowed in prov-*_ent.json files
bclenet Jun 19, 2025
a851f8b
Remark
bclenet Jun 19, 2025
2052b37
Entities allowed in prov-*_ent.json files
bclenet Jun 19, 2025
8415096
Removing GeneratedByProv
bclenet Jun 19, 2025
ab8ccaf
GeneratedBy field for Entities allow arrays
bclenet Jun 20, 2025
482c440
fMRIPrep example
bclenet Jun 24, 2025
be27631
Markdown lint
bclenet Jul 1, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ nav:
- Data summary files: modality-agnostic-files/data-summary-files.md
- Phenotypic and assessment data: modality-agnostic-files/phenotypic-and-assessment-data.md
- Code: modality-agnostic-files/code.md
- Provenance: modality-agnostic-files/provenance.md
- Modality specific files:
- Magnetic Resonance Imaging: modality-specific-files/magnetic-resonance-imaging-data.md
- Magnetoencephalography: modality-specific-files/magnetoencephalography.md
Expand Down
9 changes: 5 additions & 4 deletions src/common-principles.md
Original file line number Diff line number Diff line change
Expand Up @@ -355,10 +355,11 @@ Derivatives can be stored/distributed in two ways:
it is RECOMMENDED to use the format `<pipeline>-<variant>` in cases where
it is anticipated that the same pipeline will output more than one variant
(for example, `AFNI-blurring` and `AFNI-noblurring`).
For the sake of consistency, the subdirectory name SHOULD be
the `GeneratedBy.Name` field in `dataset_description.json`,
optionally followed by a hyphen and a suffix (see
[Derived dataset and pipeline description][derived-dataset-description]).
For the sake of consistency, the subdirectory name MUST be
the `Name` field of the first `GeneratedBy` object in `dataset_description.json`,
optionally followed by a hyphen and a suffix.
That is, if a directory `<dataset>/derivatives/<pipeline>[-<variant>]/` is used, the first
`GeneratedBy` object should have a `Name` of `<pipeline>`.

Example of derivatives with one directory per pipeline:

Expand Down
6 changes: 6 additions & 0 deletions src/introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,12 @@ For example:

- (publication forthcoming)

### Other extensions specific publications

#### Provenance

- (publication forthcoming)

### Research Resource Identifier (RRID)

BIDS has also a
Expand Down
20 changes: 1 addition & 19 deletions src/modality-agnostic-files/dataset-description.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ A guide for using macros can be found at
-->
{{ MACROS___render_text("objects.files.dataset_description.description") }}

Every dataset MUST include this file with the following fields:
Every BIDS dataset MUST include this file with the following fields:

<!-- This block generates a metadata table.
The definitions of these fields can be found in
Expand Down Expand Up @@ -44,17 +44,6 @@ and a guide for using macros can be found at
}
) }}

Each object in the `GeneratedBy` array includes the following REQUIRED, RECOMMENDED
and OPTIONAL keys:

<!-- This block generates a table describing subfields within a metadata field.
The definitions of these fields can be found in
src/schema/objects/metadata.yaml
and a guide for using macros can be found at
https://github.com/bids-standard/bids-specification/blob/master/macros_doc.md
-->
{{ MACROS___make_subobject_table("metadata.GeneratedBy.items") }}

Example:

```JSON
Expand Down Expand Up @@ -101,8 +90,6 @@ Example:
}
```

### Derived dataset and pipeline description

As for any BIDS dataset, a `dataset_description.json` file MUST be found at the
top level of every derived dataset:
`<dataset>/derivatives/<pipeline_name>/dataset_description.json`.
Expand All @@ -122,11 +109,6 @@ and a guide for using macros can be found at
}
) }}

If a derived dataset is stored as a subdirectory of the raw dataset, then the `Name` field
of the first `GeneratedBy` object MUST be a substring of the derived dataset directory name.
That is, in a directory `<dataset>/derivatives/<pipeline>[-<variant>]/`, the first
`GeneratedBy` object should have a `Name` of `<pipeline>`.

Example:

```JSON
Expand Down
48 changes: 48 additions & 0 deletions src/modality-agnostic-files/provenance-context.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
{
"@context": {
"prov" : "http://www.w3.org/ns/prov#",
"xsd": "http://www.w3.org/2001/XMLSchema#",
"RRID": "http://scicrunch.org/resolver/",
"Id": "@id",
"Type": "@type",
"Label": "http://www.w3.org/2000/01/rdf-schema#label",
"StartedAtTime" : {
"@id": "prov:startedAtTime",
"@type": "http://www.w3.org/2001/XMLSchema#dateTime"
},
"EndedAtTime" : {
"@id": "prov:endedAtTime",
"@type": "xsd:dateTime"
},
"GeneratedBy" : {
"@id": "prov:wasGeneratedBy",
"@type": "@id"
},
"AttributedTo" : {
"@id": "prov:wasAttributedTo",
"@type": "@id"
},
"AssociatedWith" : {
"@id": "prov:wasAssociatedWith",
"@type": "@id"
},
"InformedBy" : {
"@id": "prov:wasInformedBy",
"@type": "@id"
},
"Used" : {
"@id": "prov:used",
"@type": "@id"
},
"ActedOnBehalfOf" : {
"@id": "prov:actedOnBehalfOf",
"@type": "@id"
},
"Entities": "prov:Entity",
"Entity": "prov:Entity",
"Activities": "prov:Activity",
"Activity": "prov:Activity",
"Software": "prov:SoftwareAgent",
"Atlocation": "prov:atLocation"
}
}
Loading
Loading