@oruebel @bendichter and I discussed this today. I'd like to invite @h-mayorquin, @ehennestad, @VisLab, and the broader NWB community to weigh in.
NWBEP001 proposed storing EventsTable objects in a top-level /events group, alongside /intervals, acquisition, /units, etc. Based on later feedback from @oruebel (I believe), in #645, I expanded this to three locations:
/acquisition — events from the acquisition system (Neuralynx .nev, Open Ephys event channels, TTL edges)./processing/<module> — events derived by user processing (filtering, detection, or annotation)./events — primary experimental event tables ready for downstream analysis.
That three-way layout was my call, and I no longer think it's right. The /events vs /processing boundary is subjective and will produce inconsistent files across labs. I would like to revisit this and reach consensus on an approach before release.
Option 1: /events only. (Original proposal.) @bendichter said: Events are like time intervals and should be stored at the top-level like /intervals. /acquisition is for continuous recorded "signals", and discrete events are always processed or annotations.
Option 2: /acquisition and /processing, no /events. @oruebel said: Events are like TimeSeries and split naturally along acquisition/processing line. /intervals is different because intervals are session-level metadata.
Option 3: All three. Rejected for the reason above.
(@bendichter, @oruebel - please correct me if I'm misstating)
I prefer Option 1, though I disagree with @bendichter's premise that events are never raw. Many acquisition systems (e.g., Neuralynx, Open Ephys, Plexon) write event data (licks, rewards, stimulus times, trial markers, sync pulses) directly as lists of timestamps, which I consider raw. My reasons for Option 1:
- Having one place to store and discover event data (
/events) is worth not having the explicit, path-based labeling of raw vs processed events. If desired, such provenance could be stored in the EventsTable description or a new attribute like "source".
- Events are conceptually and structurally similar to time intervals in
/intervals and spike times in /units, and they would be used in similar ways during analysis. Therefore, they should be organized in a similar way.
- Users already report confusion that NWB organizes by processing stage rather than data modality (Pierre et al., 2024, sections "How should different data types be stored?" and "Data access pain points").
Note: EventsTable inherits from DynamicTable, so the schema permits placement under /acquisition and /processing regardless. The question is which location the schema recommends.
Please let me know your thoughts by the end of the week. I'd like to release NWB 2.10.0 ASAP. Thanks, and apologies for the compressed timeline.
@oruebel @bendichter and I discussed this today. I'd like to invite @h-mayorquin, @ehennestad, @VisLab, and the broader NWB community to weigh in.
NWBEP001 proposed storing
EventsTableobjects in a top-level/eventsgroup, alongside/intervals,acquisition,/units, etc. Based on later feedback from @oruebel (I believe), in #645, I expanded this to three locations:/acquisition— events from the acquisition system (Neuralynx .nev, Open Ephys event channels, TTL edges)./processing/<module>— events derived by user processing (filtering, detection, or annotation)./events— primary experimental event tables ready for downstream analysis.That three-way layout was my call, and I no longer think it's right. The
/eventsvs/processingboundary is subjective and will produce inconsistent files across labs. I would like to revisit this and reach consensus on an approach before release.Option 1:
/eventsonly. (Original proposal.) @bendichter said: Events are like time intervals and should be stored at the top-level like/intervals./acquisitionis for continuous recorded "signals", and discrete events are always processed or annotations.Option 2:
/acquisitionand/processing, no/events. @oruebel said: Events are likeTimeSeriesand split naturally along acquisition/processing line./intervalsis different because intervals are session-level metadata.Option 3: All three. Rejected for the reason above.
(@bendichter, @oruebel - please correct me if I'm misstating)
I prefer Option 1, though I disagree with @bendichter's premise that events are never raw. Many acquisition systems (e.g., Neuralynx, Open Ephys, Plexon) write event data (licks, rewards, stimulus times, trial markers, sync pulses) directly as lists of timestamps, which I consider raw. My reasons for Option 1:
/events) is worth not having the explicit, path-based labeling of raw vs processed events. If desired, such provenance could be stored in theEventsTabledescription or a new attribute like "source"./intervalsand spike times in/units, and they would be used in similar ways during analysis. Therefore, they should be organized in a similar way.Note:
EventsTableinherits fromDynamicTable, so the schema permits placement under/acquisitionand/processingregardless. The question is which location the schema recommends.Please let me know your thoughts by the end of the week. I'd like to release NWB 2.10.0 ASAP. Thanks, and apologies for the compressed timeline.