Run spike sorting on multiple files from the same session

[PiezaKake]

Feature you'd like to see:

Run Spike sorting on multiple files with the same configuration from the same session

Additional Context

Hi, I was wondering if it is possible to run spike sorting on multiple files with the same configuration.
I will soon start recording with open ephys, and my plan was to record multiple files in each session. Is it possible to 'feed' all files from a single session into kilosort for a single spike sorting output, insead of running spike sorting for each file individually?

[jacobpennington]

There isn't currently a way to do this through Kilosort, but it will be available in the future. The utility I was adding for it still needs some work.

[jacobpennington]

Actually, not as much work as I thought. There's a working version in the latest commit, but like I said I haven't done much testing with it yet. If you want to try it, you can install using the developer instructions at the bottom of the read me and pull the latest changes. This is the minimal code needed:

from kilosort import run_kilosort
from kilosort.io import BinaryFileGroup, load_probe

# Also add whatever else you need to set for your data, like sampling rate
settings = {'n_chan_bin': 385}
probe = load_probe(probe_path)
# Recommended: set filename to the directory containing the data files.
# It can be any path that exists, but you do need to provide a filename still.
filename = data_dir
# Use the file_object argument
file_obj = BinaryFileGroup.from_filenames(list_of_filenames, n_chan_bin)
_ = run_kilosort(settings=settings, filename=data_dir, probe=probe, file_object=file_obj)

[PiezaKake]

Thanks, I will give this try, but do you have any idea when the final add-on will be available? I just wanted to check whether it will be available by the time I start the data analysis

[jacobpennington]

I don't know, we have other priorities for development right now.

[mshallow]

Hi, I'm not sure if @PiezaKake ever tested this after asking about the feature, but I have been trying to run batches of files from one session and have run into an issue that appears to arise when the sorting tries to transition from one file to another. The error is somewhere in spikedetect.py and comes when it tries to pad and switch between files.
The error is similar to the error posted in issue #739 and I tried some of the fixes that were suggested in that thread, and they don't appear to solve the problem when kilosort is being run on multiple files.
I have tried changing the tmax value, which if its small enough appears to solve the issue, but as far as I can tell, this is because it's only making it through the amount of time in one of the two files.
The other issue that I have encountered trying to use the batch processing, is that it doesn't output a kilosort4.log file even if it appears to have finished sorting. I can get log files if I sort the individual files in the gui but if I try to run a batch even with a "list" of files that is only one of the files, it does not generate a log file.

[jacobpennington]

Okay, thanks for catching that! I'll try to work on this soon.