Investigate relationship information about data on nwb file

[filippomc]

https://celltypes.brain-map.org/experiment/electrophysiology/643625553

we should find relationships between stimuli and sweeps and more.
Think that from an experimentalist point of view

• investigate/ask the pynwb team what we can do to find relationships

[pgleeson]

Info re updating Allen datasets to NWBv2: NeurodataWithoutBorders/pynwb#275

[filippomc]

That's great to have, I'd had been more happy with some conversion tool though. I'm afraid that after we are loading the file in legacy mode we must read the file differently. I'll add a card about exploring this.
Are you interested in supporting these legacy files in the long term anyway?

[filippomc]

One thing we want to have is the relationship between stimuli and responses.

[filippomc]

After some investigation on nwb format and pynwb apis it's still unclear if we can find a reliable way to relate data.
In general, nwb2 is constructed to allow for any kind of experiment and does not enforce any experiment style. It's a tool box where everyone can take what fits his own needs.
In general, the relationship can be in time and space.
The main source of time relationships are timestamps and intervals, with epochs and trials, bot how to interpret them seems left to human comprehension more than a standardized way.

Let's take this experiment from Allen's brain observatory: https://pynwb.readthedocs.io/en/stable/tutorials/domain/brain_observatory.html#sphx-glr-tutorials-domain-brain-observatory-py.
An animal was shown sequence of images and the running speed was recorded. The running speed is associated to the epochs, but how each epoch is related to the stimuli?
The relationship with the stimuli seems implicit with the number of epochs. Every epoch a different stimulus? Can we say that two events happening at the same time are related?

[filippomc]

The relationships have been investigated together with the sample files (see https://docs.google.com/spreadsheets/d/1BK7rIA3XoKHV2HhkwoMgq5E6iBGyoQ26LlTiAWT8O4w/edit#gid=0).

Some relevant sources of relationships are:

• Sweep table -> time series (1:N, N time series for each sweep number)
• Current voltage time series -> electrode (N:1)
• Electrode -> electrode group (N:1)
• Epoch -> time series (0:N)
• ROI -> time series (N:1)
• ROI -> imaging plane (plane segmentation)
• (*) -> Device (The device is referenced in many different objects)
• OptogeneticSeries -> Stimulus site
• LFP -> ElectricalSeries (1: N)

Every specific relationship represent an experiment model which may be represented with its own specificity. We may design a specific interface for each one, starting from widely used features (e.g. sweep tables).
More in general, we may provide a "link" to the related object in the detail view which can allow to browse inside the model.
Another option could be to use a network graph view to show and represent the relationships between objects (For instance, https://visjs.org/examples/network/basicUsage.html). @tarelli @pgleeson opinions?