Neuron Morphology / Reconstruction embedding in NWB

[smestern]

I was interested in embedding neuron morphology information in an NWB. We have a number of intracellular whole-cell patch-clamp recordings (in NWB) with paired post-hoc neuron morphology reconstruction (+microscopy images). I was interested in packing in the neuron morphology data (currently in SWC with its respective intracellular recordings, keeping everything linked and tidy. The pairs of intracellular data + morphology may be useful for biophysical modellers or cell typing in the future.

The SWC data itself is simply a table of node-ids (e.g., dendrite, axon, soma), and coordinates. It should be easy to embed as a dynamic table. Or I could embed a link to the swc as an external file.

I'd like to open the discussion to determine the preferred standardized way to embed the data.

[smestern]

If anyone is curious. I have tried prototyping out the inclusion under /processing/morphology as suggested by @oruebel in our slack discussion. This would look something like:

#load the nwb file
nwb_file = pynwb.NWBHDF5IO(nwb, 'a')
nwb_data = nwb_file.read()

#load the swc file. its actually just a space delimited text file, so we can use pandas to read it in
swc_data = pandas.read_csv(swc_file, sep=' ', header=None, names=['Index', 'Type', 'X', 'Y', 'Z', 'Radius', 'Parent'])
#make a new DynamicTable to hold the swc data
index_col = VectorData(name='Index', description='Index of the SWC point', data=swc_data['Index'].values)
type_col = VectorData(name='Type', description='Type of the SWC point', data=swc_data['Type'].values)
x_col = VectorData(name='X', description='X coordinate of the SWC point, in microns', data=swc_data['X'].values)
y_col = VectorData(name='Y', description='Y coordinate of the SWC point, in microns', data=swc_data['Y'].values)
z_col = VectorData(name='Z', description='Z coordinate of the SWC point, in microns', data=swc_data['Z'].values)
radius_col = VectorData(name='Radius', description='Radius of the SWC point, in microns', data=swc_data['Radius'].values)
parent_col = VectorData(name='Parent', description='Index of the parent SWC point, -1 if there is no parent', data=swc_data['Parent'].values)
swc_table = DynamicTable(name='SWC', description='SWC data from '+ os.path.basename(swc_file), columns=[index_col, type_col, x_col, y_col, z_col, radius_col, parent_col])

nwb_data.create_processing_module(name='morphology', description='Module to hold morphology data')
#add to the processing subdir
nwb_data.processing['morphology'].add(swc_table)
#save the new nwb file for recording keeping, we will save it in a new directory so we don't accidentally overwrite the original nwbs
out_file = os.path.join(NWB_OUT_DIR, os.path.basename(nwb))
with NWBHDF5IO(out_file, mode="w") as export_io:
    export_io.export(src_io=nwb_file, nwbfile=nwb_data)


nwb_file.close()
#test read
with pynwb.NWBHDF5IO(out_file, 'r') as io:
    nwb_data = io.read()
    print(nwb_data.processing['morphology']['SWC'])
    print(f"X col:{nwb_data.processing['morphology']['SWC']['X'].data[:10]}")

This embeds the data as a dynamictable titled SWC under /processing/morphology. However a better table name may be needed. Likely an extension would be beneficial to standardize it.
It does round trip:

SWC hdmf.common.table.DynamicTable at 0x2564374450000
Fields:
  colnames: ['Index' 'Type' 'X' 'Y' 'Z' 'Radius' 'Parent']
  columns: (
    Index <class 'hdmf.common.table.VectorData'>,
    Type <class 'hdmf.common.table.VectorData'>,
    X <class 'hdmf.common.table.VectorData'>,
    Y <class 'hdmf.common.table.VectorData'>,
    Z <class 'hdmf.common.table.VectorData'>,
    Radius <class 'hdmf.common.table.VectorData'>,
    Parent <class 'hdmf.common.table.VectorData'>
  )
  description: SWC data from M05_SP_A1_C04_Goettingen_HEKA_Cell04.swc
  id: id <class 'hdmf.common.table.ElementIdentifiers'>

X col:[522.82 522.89 522.89 522.96 522.96 523.03 523.03 523.17 523.17 523.24]