Efficient and reproducible pipelines for spike sorting large-scale electrophysiology data
- Allen Institute for Neural Dynamics, United States
Figures
Figure 1
Challenges of scaling electrophysiological recordings.
(a) Multi-shank Neuropixels probe overlaid on a mouse brain, with zoomed-in region showing the spatial footprint of a typical spike waveform (from Steinmetz and Ye, 2022). Waveforms from individual electrodes and spatiotemporal footprint (sampled by a hypothetical Neuropixels 2.0 probe) are shown on the right. The approximate scale of the spike (50 μm × 2 ms) necessitates dense sampling in both space and time. Scaling up the number of recorded neurons requires increasing the number of electrodes that are in close physical proximity to neurons. (b) Times required to run preprocessing, spike sorting, and automated curation on 2-hr recordings with different probe configurations. Assuming no parallelization across machines, a recording with six Neuropixels 1.0 probes (384 channels each) would take more than 2 days to process. A recording with six Neuropixels 2.0 Quad Base probes (1536 channels each), which recently became commercially available, would take over 1 week. Parallelization is essential to complete processing in under 24 hr after data collection.
Figure 2
Spike sorting pipeline overview.
Raw electrophysiology data from multiple probes (top) is ingested by the Job Dispatch step, which coordinates parallelization of downstream processing. All steps are run in parallel until Result Collection. Pipeline outputs (including Figurl interactive visualizations, metrics stored in JSON format, PNG-formatted images, and a Neurodata Without Borders file) are shown at the bottom. Each step includes an estimate of run time (per hour of recording) and required computing resources (CPU, GPU, and RAM). The pipeline is encapsulated in a Nextflow workflow (green background), and individual steps are implemented in SpikeInterface (action potential logo). Interlocking brick icons indicate processing algorithms that can be easily substituted. For the spike sorting step, run time is calculated for Kilosort4. See Table 1 for detailed run times and cost estimates.
Figure 3
Outputs of the visualization and quality control step.
(a) Screenshot of interactive visualization of raw and preprocessed data (view on web). (b) Screenshot of Sortingview GUI, used for inspecting and curating spike sorting outputs (view on web). (c) Quality control plot showing power spectra of continuously recorded electrical signals at three timepoints during the session. (d) ‘Drift map’ of spike locations over time, used to assess the overall stability of the recording. (e) Histograms and scatter plot of key unit-specific quality metrics.
Figure 4
Spike sorting pipeline usage at the Allen Institute for Neural Dynamics.
(a) Sessions processed by the pipeline each week between February 2024 and August 2025. Color indicates the spike sorter used for each pipeline run. (b) Cumulative sessions processed by the pipeline, split by spike sorter. (c) Cumulative probes processed by the pipeline (many sessions include data from multiple probes). (d) Cumulative units detected by the pipeline, with the subset of neural units indicated in yellow and units passing default quality metric thresholds in green. Units are considered neural if the UnitRefine classifier does not label them as ‘noise‘ (see Curation section for more details).
Figure 5
Benchmarking pipeline overview.
The Job Dispatch step accepts N sessions as input, each of which may contain data from multiple probes. Next, the Hybrid Generation step injects T ground-truth spike templates for each session and probe (default = 10), with IT randomized iterations (default = 5). Hybrid data are then processed in parallel through M Spike Sorting Cases, each of which sorts data using a different set of algorithms or parameters. The sorting results are then collected and compared by the Hybrid Evaluation step, which outputs figures and metrics for analyzing the performance of each case.
Figure 6
Example hybrid evaluation.
(a) 100 ms snippet of traces, showing ground-truth hybrid spikes and respective waveforms in blue if found by both sorters, red if found by Kilosort2.5 only, and purple if found by Kilosort4 only. (b) Spike rasters of all hybrid spikes in the recording with the same color scheme as panel a. Gray spikes are those from the original recording. Note that spikes not detected by any sorter are not visible in the plot. (c) Fraction of false positive spikes found by either sorter (a.u. = arbitrary units). (d) Fraction of false negative spikes. False positive and false negative spikes are calculated using 10 s time bins.
Figure 7 with 2 supplements
Benchmarking spike sorting algorithms.
(a) Accuracy, precision, and recall for all ground-truth units for Kilosort2.5 (red) and Kilosort4 (purple) for Neuropixels 1.0 (top panels) and 2.0 (bottom panels) probes. (b) Scatter plots for accuracy, precision, and recall for ground-truth units matched by both Kilosort2.5 (x axis) and Kilosort4 (y axis) for Neuropixels 1.0 (top panels) and 2.0 (bottom panels) probes. (c) Distributions of refractory period contamination for all units found by both sorters. (d) Histograms of presence ratio values for all units found by both sorters. In panels c and d, data from both probes are combined and include all units that were matched to a ground-truth hybrid unit (accuracy ≥0.2—Kilosort2.5: 3046 matches; Kilosort4: 3652 matches).
Figure 7—figure supplement 1
Spike sorter performance as a function of unit signal-to-noise ratio.
(a) Accuracy, precision, and recall for every hybrid unit added to the Neuropixels 1.0. Lines indicate the average value at each signal-to-noise level. (b) Same as a, but for Neuropixels 2.0 datasets.
Figure 7—figure supplement 2
Kilosort run times.
(a) Run times of Kilosort2.5 (red) and Kilosort4 (purple) on Neuropixels 1.0 datasets. Run times are relative to real-time (xRT). (b) Same as a, but for Neuropixels 2.0 datasets. Note that for Neuropixels 1.0, spike sorting was run on the full 384-channel recording, while it was split by shank (96 channels each) for Neuropixels 2.0, which is why the latter probe has faster execution times.
Figure 8 with 1 supplement
Lossy compression benchmarks.
(a) Accuracy, precision, and recall for all ground-truth units for lossless and lossy (dark green) WavPack compression (lighter greens as BPS decreases, that is, compression ratio increases) for Neuropixels 1.0 (top panels) and 2.0 (bottom panels) probes. The dashed red line shows the curves for Kilosort2.5 for reference. (b) Median accuracy, precision, and recall as a function of compression ratio for both probe types (top row: Neuropixels 1.0; bottom row: Neuropixels 2.0). (c) Scatter plots for accuracy, precision, and recall for ground-truth units matched by both the lossless (x axis) and BPS = 3 (y axis), for both probe types. (d) Distributions of refractory period contamination for all units found at each compression ratio. (e) Histograms of presence ratio values for all units found at each compression ratio. In panels d and e, data from both probes are combined and include all units that were matched with accuracy ≥0.2 to a ground-truth hybrid unit (Lossless: 3612 matches; BPS = 3: 3513 matches; BPS = 2.5: 3475 matches; BPS = 2.25: 3405 matches).
Figure 8—figure supplement 1
Spike sorter performance as a function of unit signal-to-noise ratio.
(a) Accuracy, precision, and recall for every hybrid unit added to the Neuropixels 1.0. Lines indicate the average value at each signal-to-noise level. (b) Same as a, but for Neuropixels 2.0 datasets.
Tables
Table 1
Resources, run time, and cost for all steps in the spike sorting pipeline.
Values are based on a 1-hr recording with six Neuropixels probes. Values for non-parallel steps (Job Dispatch, NWB Packaging, and Result Collection) are normalized by the number of probes. For steps that run in parallel, run times and estimated costs are reported as the mean ± standard deviation for probes in the same session. *Run time per hour of recording with one 384-channel probe. **Cost per hour of recording with one 384-channel probe. Costs are estimated using the prices of the reported instance from https://aws.amazon.com/ec2/pricing/on-demand/ as of April 30, 2025.
| Step | Parallel? | CPUs | GPUs | RAM (GB) | Run time* (hr) | Instance | Cost** (USD) |
|---|---|---|---|---|---|---|---|
| Job dispatch | N | 4 | 0 | 32 | 0.05 | r5a.xlarge | 0.011 |
| Preprocessing | Y | 16 | 0 | 64 | 0.84 ± 0.19 | m5a.4xlarge | 0.58 |
| Spike sorting (KS4) | Y | 16 | 1 | 64 | 2.7 ± 0.12 | g4dn.4xlarge | 3.24 |
| Postprocessing | Y | 16 | 0 | 128 | 1 ± 0.17 | r5a.4xlarge | 0.9 |
| Curation | Y | 4 | 0 | 128 | 0.1 ± 0.1 | r5a.xlarge | 0.022 |
| Visualization/QC | N | 8 | 0 | 64 | 1.02 ± 0.12 | r5a.2xlarge | 0.46 |
| NWB packaging | N | 8 | 0 | 64 | 0.38 | r5a.2xlarge | 0.17 |
| Result collection | N | 8 | 0 | 64 | 0.16 | r5a.2xlarge | 0.072 |
| Total | - | - | - | - | 6.25 | 5.554 |
Table 2
Benchmarking pipeline run times.
The number of jobs and effective run time in hours for the two benchmarking applications (Kilosort2.5 vs. Kilosort4, Lossless vs. Lossy) on data from two types of Neuropixels probes (NP1 and NP2). The ‘distributed’ execution refers to our Nextflow implementation, while the ‘serial’ execution has all jobs running sequentially on an individual workstation. The run time for serial execution is extrapolated from the distributed run times, and was not actually measured.
| Benchmark application | Probe type | Execution | # Jobs | Effective run time (hr) |
|---|---|---|---|---|
| Kilosort2.5 vs. Kilosort4 | NP1 | Distributed | 398 | 14 |
| Kilosort2.5 vs. Kilosort4 | NP1 | Serial | 398 | 870 |
| Kilosort2.5 vs. Kilosort4 | NP2 | Distributed | 662 | 12 |
| Kilosort2.5 vs. Kilosort4 | NP2 | Serial | 662 | 846 |
| Lossless vs. Lossy | NP1 | Distributed | 578 | 18 |
| Lossless vs. Lossy | NP1 | Serial | 578 | 1590 |
| Lossless vs. Lossy | NP2 | Distributed | 962 | 16 |
| Lossless vs. Lossy | NP2 | Serial | 962 | 1684 |
Additional files
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Efficient and reproducible pipelines for spike sorting large-scale electrophysiology data
eLife 15:RP110170.
https://doi.org/10.7554/eLife.110170.3
