Converting Wide-field calcium imaging

[mehrdadramezani]

I need help converting preprocessed dF/F vectors from wide-field calcium imaging experiments. We no longer have the raw data, and our dataset format (.mat) doesn't match any of the existing formats.
I've tried using CNMFE and EXTRACT, but as expected, I got errors because our files were not processed with these ROI analysis pipelines.
Additionally, we're facing issues with converting Intan recordings. When attempting to convert the .rhd file without the imaging data listed, the process stalls at the "Data Entry/Source data" step. After clicking "Next," a window appears saying "Getting metadata for source data. Please wait..." but then it disappears after 1-2 minutes without progressing or showing an error.
Any assistance would be greatly appreciated!

[CodyCBakerPhD]

Hi there - it sounds like you're trying to use the NWB GUIDE to convert your data to NWB but running into a few problems...

All currently supported formats can be viewed not only in the app itself but also in the ecosystem format support table from the documentation

our dataset format (.mat) doesn't match any of the existing formats.

.mat is a generic suffix for any file saved from MATLAB-based software. It does not specify what content or structure exists within that file, and thus is not a true data format in and of itself

I've tried using CNMFE and EXTRACT, but as expected, I got errors because our files were not processed with these ROI analysis pipelines.

If you don't know what segmentation algorithm was used to create that .mat file, then you were right to try CNMFE and EXTRACT (the two MATLAB based methods we currently support) but it sounds like neither of these worked because a different (possibly custom) segmentation method was used

In this case, the next step for converting the segmentation data to NWB would be to use PyNWB or MatNWB APIs to (a) read your data from the .mat file and then (b) write it to NWB by following either of these tutorials: PyNWB tutorial or MatNWB tutorial

Additionally, we're facing issues with converting Intan recordings. When attempting to convert the .rhd file without the imaging data listed,

I am a tad confused by this - what did you use the Intan recording system for? That is normally for extracellular electrophysiology probes (though the analog/digital board can be used in any experiment) and is independent of imaging techniques

Maybe I need more context about your experiments to understand what you need?

the process stalls at the "Data Entry/Source data" step. After clicking "Next," a window appears saying "Getting metadata for source data. Please wait..." but then it disappears after 1-2 minutes without progressing or showing an error.

This is interesting... stalling out should not occur, especially with a timeout of the window

To thoroughly debug this, we may need to have access to your data to try to reproduce on our end - are you able to share a small amount of this data over Google Drive or Dropbox (or anything similar)?

[CodyCBakerPhD]

Also @mehrdadramezani, could you briefly check and report what version of the GUIDE you are using? (It should be displayed in the top left corner)

[mehrdadramezani]

Hi Cody,

Thank you for your detailed response. I should have provided a better explanation of the dataset we are trying to convert.
Our dataset is from a multimodal study published in 2021. In this study, we performed simultaneous electrophysiology using flexible electrodes (with the Intan data acquisition setup) and wide-field calcium imaging.

We have the raw .rhd files from the Intan setup that can be converted using NWB.
Unfortunately, we don't have access to the original Tiff files from the wide-field calcium imaging. These were preprocessed and saved as .mat files. Each .mat file contains dF/F and baseline fluorescence signals from 1-hour sessions. Each frame (128 x 128 pixels) is flattened into 1-D vectors (16,384 x 1). We have not done any segmentation and our preprocessing was limited to converting the fluorescence signals from each pixel into dF/F values for subsequent analyses in the paper.

We are happy to share data from one of the sessions for thorough debugging on your end. Please feel free to contact me for further coordination of sharing the data via Google Drive, Dropbox, or any similar platform.

We are using NWB GUIDE version 0.0.15.

[CodyCBakerPhD]

Thank you for your detailed response. I should have provided a better explanation of the dataset we are trying to convert.
Our dataset is from a multimodal study published in 2021. In this study, we performed simultaneous electrophysiology using flexible electrodes (with the Intan data acquisition setup) and wide-field calcium imaging.

Cool, that sounds interesting!

We have the raw .rhd files from the Intan setup that can be converted using NWB.

Great to hear; that is exactly a situation the GUIDE is developed for

Unfortunately, we don't have access to the original Tiff files from the wide-field calcium imaging. These were preprocessed and saved as .mat files. Each .mat file contains dF/F and baseline fluorescence signals from 1-hour sessions. Each frame (128 x 128 pixels) is flattened into 1-D vectors (16,384 x 1).

Unfortunately, what you are describing is an ad-hoc format (.mat simply being a generic container of items) and neither the GUIDE nor NeuroConv offer base interfaces for that data

Which means that your best bet for combining these two data streams would be to use the GUIDE to create initial NWB files (with all the file metadata too) for the Intan data stream, then append the file with custom PyNWB or MatNWB code as I linked above

In this case, the next step for converting the segmentation data to NWB would be to use PyNWB or MatNWB APIs to (a) read your data from the .mat file and then (b) write it to NWB by following either of these tutorials: PyNWB tutorial or MatNWB tutorial

I would recommend trying this out on a single session first to get a feel for how the workflow will go