Converting continuous.dat into SpikeGLX flat binary about analysis-tools HOT 4 CLOSED

Hi Guido! Hopefully you saw Pavel's response in the Neuropixels Slack channel, but I'll add some more info here so it's archived:

• The only formatting difference between .bin and .dat files is the extension. They are both flat binary files of signed 16-bit integers. You should be able to point pykilosort to the .dat file and have it work automatically. If you're having trouble with this, let us know.

• There is a difference in the scaling that's applied by the recording software before writing. This doesn't matter for spike sorting, but it will be important for loading the data for further analysis. SpikeGLX writes the raw ADC values from the Neuropixels probe, so converting to microvolts requires knowledge of the gain that was used. Open Ephys scales all samples to have a "bitVolts" value of 0.195, regardless of the amplifier gain.

• The analysis-tools repo is no longer being maintained. Instead, we recommend using open-ephys-python-tools or open-ephys-matlab-tools, which we'll be keeping up to date. We just updated the analysis-tools README with this info.

from analysis-tools.

Hi Josh, thanks so much for your answer! Great that both raw data files are already flat binary, I should have known but I got thrown off by the existence of the .continuous files that the OpenEphys outputs in other cases. The scaling is also very good to know, thanks. There is one thing remaining: pykilosort also requires the SpikeGLX .meta file to run. I can probably write a conversion script that converts the OpenEphys metadata XML file into the meta data from SpikeGLX, but do you know whether such a conversion script already exists?

from analysis-tools.

A notebook for generating a .meta file from an Open Ephys recording directory is available here: https://www.dropbox.com/s/72dbum80z43zkh8/meta_file_generator.ipynb?dl=0

We plan to integrate these functions into open-ephys-python-tools eventually, but for now you'll just have to run the notebook (editing cells where appropriate).

At the Allen Institute, we have been running pykilosort via SpikeInterface, which can seamlessly read data from both SpikeGLX and Open Ephys. I assume you're using the IBL pipeline for pre/post-processing, but I'd recommend checking out SpikeInterface when you have time.

from analysis-tools.

Thanks Josh, spikeinterface is exactly what I needed, it works great!

from analysis-tools.