Waveform detection by deep learning reveals multi-area spindles that are selectively modulated by memory load

  1. Maryam H Mofrad
  2. Greydon Gilmore
  3. Dominik Koller
  4. Seyed M Mirsattari
  5. Jorge G Burneo
  6. David A Steven
  7. Ali R Khan
  8. Ana Suller Marti
  9. Lyle Muller  Is a corresponding author
  1. Department of Mathematics, Western University, Canada
  2. Brain and Mind Institute, Western University, Canada
  3. Department of Biomedical Engineering, Western University, Canada
  4. Advanced Concepts Team, European Space Agency, Netherlands
  5. Department of Clinical Neurological Sciences, Schulich School of Medicine and Dentistry, Western University, Canada
  6. Department of Medical Imaging, Schulich School of Medicine and Dentistry, Western University, Canada
  7. Department of Medical Biophysics, Schulich School of Medicine and Dentistry, Western University, Canada
  8. Department of Psychology, Western University, Canada
  9. Department of Epidemiology and Biostatistics, Schulich School of Medicine and Dentistry, Western University, Canada

Abstract

Sleep is generally considered to be a state of large-scale synchrony across thalamus and neocortex; however, recent work has challenged this idea by reporting isolated sleep rhythms such as slow oscillations and spindles. What is the spatial scale of sleep rhythms? To answer this question, we adapted deep learning algorithms initially developed for detecting earthquakes and gravitational waves in high-noise settings for analysis of neural recordings in sleep. We then studied sleep spindles in non-human primate electrocorticography (ECoG), human electroencephalogram (EEG), and clinical intracranial electroencephalogram (iEEG) recordings in the human. Within each recording type, we find widespread spindles occur much more frequently than previously reported. We then analyzed the spatiotemporal patterns of these large-scale, multi-area spindles and, in the EEG recordings, how spindle patterns change following a visual memory task. Our results reveal a potential role for widespread, multi-area spindles in consolidation of memories in networks widely distributed across primate cortex.

Editor's evaluation

This article provides compelling evidence that deep convolutional networks can detect repeating patterns in biological data better than existing methods, in the presence of noise, biological or otherwise. In analyses of data acquired from the brains of primates using various modalities, the authors show that spindles in cortex have a wider spatial distribution that previously thought. Applications of the proposed approach in other settings may lead to novel findings about the distribution of transient oscillatory patterns in the brain.

https://doi.org/10.7554/eLife.75769.sa0

eLife digest

The brain processes memories as we sleep, generating rhythms of electrical activity called ‘sleep spindles’. Sleep spindles were long thought to be a state where the entire brain was fully synchronized by this rhythm. This was based on EEG recordings, short for electroencephalogram, a technique that uses electrodes on the scalp to measure electrical activity in the outermost layer of the brain, the cortex. But more recent intracranial recordings of people undergoing brain surgery have challenged this idea and suggested that sleep spindles may not be a state of global brain synchronization, but rather localised to specific areas.

Mofrad et al. sought to clarify the extent to which spindles co-occur at multiple sites in the brain, which could shed light on how networks of neurons coordinate memory storage during sleep. To analyse highly variable brain wave recordings, Mofrad et al. adapted deep learning algorithms initially developed for detecting earthquakes and gravitational waves. The resulting algorithm, designed to more sensitively detect spindles amongst other brain activity, was then applied to a range of sleep recordings from humans and macaque monkeys.

The analyses revealed that widespread and complex patterns of spindle rhythms, spanning multiple areas in the cortex of the brain, actually appear much more frequently than previously thought. This finding was consistent across all the recordings analysed, even recordings under the skull, which provide the clearest window into brain circuits. Further analyses found that these multi-area spindles occurred more often in sleep after people had completed tasks that required holding many visual scenes in memory, as opposed to control conditions with fewer visual scenes.

In summary, Mofrad et al. show that neuroscientists had previously not appreciated the complex and dynamic patterns in this sleep rhythm. These patterns in sleep spindles may be able to adapt based on the demands needed for memory storage, and this will be the subject of future work. Moreover, the findings support the idea that sleep spindles help coordinate the consolidation of memories in brain circuits that stretch across the cortex. Understanding this mechanism may provide insights into how memory falters in aging and sleep-related diseases, such as Alzheimer’s disease. Lastly, the algorithm developed by Mofrad et al. stands to be a useful tool for analysing other rhythmic waveforms in noisy recordings.

Introduction

Consolidation of long-term memories requires precise coordination of pre- and postsynaptic spikes across neocortex. New memories are transferred from hippocampus to neocortex for long-term storage (McClelland et al., 1995; Rasch and Born, 2007), where interconnections within a sparse, distributed neuron group are strengthened until their activity becomes hippocampus-independent (Frankland and Bontempi, 2005). Computational studies have identified neural oscillations as a potential mechanism to regulate synaptic plasticity (Masquelier et al., 2009; Song et al., 2000) and create precise spike timing (Cassenaer and Laurent, 2007; Muller et al., 2011). Further, experiments have shown that the sleep ‘spindle’ oscillation influences spiking activity (Contreras and Steriade, 1995; Kandel and Buzsáki, 1997; Peyrache et al., 2011) and causally contributes to sleep-dependent consolidation of long-term memory (Mednick et al., 2013). It remains unclear, however, precisely how this rhythm can coordinate activity across areas in neocortex for synaptic plasticity and long-term storage to occur.

While early recordings in anesthetized animals (Andersen et al., 1967; Contreras et al., 1996) and human electroencephalogram (EEG) (Achermann and Borbély, 1998) indicated that sleep spindles generally occur across a wide area in cortex, creating a state of large-scale synchrony (Sejnowski and Destexhe, 2000; Steriade, 2003), recent work in intracranial recordings from human clinical patients has challenged this idea by reporting isolated, ‘local’ sleep spindles (Andrillon et al., 2011; Nir et al., 2011; Piantoni et al., 2017; Sarasso et al., 2014, but see Frauscher et al., 2015). Because spindles are intrinsically related to sleep-dependent consolidation of long-term memory (Clemens et al., 2005; Gais et al., 2002; Mednick et al., 2013), this difference in reported spatial extent of the spindle raises an important question for the organization of engrams established through sleep-dependent memory consolidation. Recent evidence using cFos mapping in animal models suggests these engrams are distributed widely across brain areas (Kitamura et al., 2017; Roy et al., 2019 n.d.), which is consistent with previous imaging evidence in the human (Brodt and Gais, 2021; Wheeler et al., 2000). Taking these points together, we reasoned that widespread, multi-area spindles may occur more often than previously reported in primate and human cortex. If this were the case, these widespread spindles could provide the mechanism needed to link populations distributed widely across the cortex for sleep-dependent memory consolidation.

One potential mechanism is provided by previous work on spindles in intracranial electrocorticography (ECoG) recordings in human clinical patients, where these oscillations were found to be organized into a wave rotating across the cortex (see Video 1 in Muller et al., 2016). Based on their speed of propagation (2–5 m/s), which matches the axonal conduction speeds of long-range white matter fibers in cortex, it was identified that this rotating wave organization could precisely align spikes across areas separated by long distances in cortex to create the conditions necessary for both synaptic strengthening and weakening to occur. With this previously identified mechanism in mind, we thus hypothesized that widespread, multi-area spindles might be a critical missing link in understanding how networks widely distributed across cortex are modulated during sleep.

Reliably detecting individual spindles in noisy sleep recordings, however, is challenging. Spindle oscillation amplitudes differ across regions in cortex (Frauscher et al., 2015). Furthermore, oscillation amplitudes may differ significantly across recording sites simply due to variation in electrode properties (Kappenman and Luck, 2010; Nelson and Pouget, 2010). For these reasons, we reasoned that bandpass filtering followed by an amplitude threshold (AT), which is a technique common across methods for spindle detection (Warby et al., 2014), may only detect the largest-amplitude events, potentially leading to an underestimation of spatial extent. To address this question, we adapted deep learning algorithms initially developed for detecting earthquakes (Perol et al., 2018) and gravitational waves (George and Huerta, 2018) in high-noise settings to analysis of neural recordings in sleep. These convolutional neural networks (CNNs) are relatively general to the type of noise in each recording, provided there is enough training data and a set of high-quality marked events. Because obtaining many high-quality marked spindle events is itself difficult, however, as sleep recordings are in general manually scored by experts (a process that is both expensive and subjective), we introduce here a careful, two-step computational approach. First, we use a signal-to-noise ratio (SNR) algorithm (Muller et al., 2016) to generate a set of high-quality marked spindles for training the CNN. The SNR algorithm, which is closely related to the constant false alarm rate (CFAR) method used in radar (Richards, 2005), detects many ‘true’ spindles while minimizing false detections. This property makes the SNR algorithm an excellent method for generating a high-quality training dataset and, in addition, for providing a second check on results from the CNN model on the subset of spindles detected by this more conservative approach. We then use the trained CNN to detect a comprehensive set of spindles in sleep recordings. To test this approach, we studied sleep spindles in macaque non-human primate (NHP) ECoG, human electroencephalogram (EEG), and, finally, clinical intracranial electroencephalogram (iEEG) recordings, which provide a window into the circuits of the human brain at one of the highest spatial resolutions possible (Lachaux et al., 2012; Mukamel and Fried, 2012). This two-step approach results in a subject-specific model, adapted to the noise encountered in each recording type and the specific sleep waveforms in each individual, that can more sensitively detect a range of clearly formed large- and small-amplitude spindles in the sleep recordings. Finally, at each point in the analysis, we also return to the subset of spindles detected by the SNR algorithm to validate results obtained from the CNN.

Our approach reveals that the spatial extent of spindles, defined here in terms of co-occurrence across electrode sites within the same 500 ms detection window, is widely distributed over a broad range of cortex. In particular, multi-area spindles are much more frequent than previously estimated by AT approaches, which tend to select only the highest-amplitude spindles and could miss events that transiently fall below threshold. Importantly, while we apply our approach to very different datasets (ECoG, EEG, and iEEG) in this work, the comparisons we make are always between the spatial extent of spindles detected by our CNN approach and AT methods within an individual recording type. These results provide strong evidence that widespread, multi-area spindles may have been underestimated in previous work. This finding, which clearly emerges consistently across all recording types, is not affected by differences in spatial sampling of different electrode types, as we always restrict comparisons within a single type of electrode. In human sleep EEG after low- (L-VM) and high-load visual memory (H-VM) tasks, our method also detects an increase in regional and multi-area spindles uniquely following an H-VM task. Finally, we note that spindle co-occurrence does not imply zero-lag synchrony across recording sites, with all sites reaching positive (or negative) peaks in potential at the same point in time. Further spatiotemporal analysis of the sleep EEG recording reveals that the multi-area spindles are organized into rotating waves that are also modulated by the memory task. Taken together, these results reveal a sophisticated spatiotemporal organization of sleep spindles in the primate brain, both in co-occurrence and in phase organization, that has previously gone unappreciated. These results thus provide substantial insight into the spatiotemporal organization of sleep spindles in the primate brain, during normal sleep and also following memory tasks.

Results

Sleep recordings from both human and NHP were obtained from electrodes ranging from traditional scalp EEG to invasive intracranial EEG electrodes (Figure 1a). We trained subject-specific CNN models over high-quality training datasets generated by the SNR algorithm. The SNR algorithm robustly detects spindles ranging from high to lower amplitudes (Figure 1—figure supplement 1), providing a good training set for the CNN. To verify the quality of spindles detected by our CNN model (Figure 1c), we first computed average power spectral densities (PSDs) over spindle and non-spindle windows. The average PSD of detected spindle events shows an increase in the 11–15 Hz spindle frequency range (red lines, Figure 1b), while non-spindle events do not show a corresponding increase (black lines, Figure 1b). Spindles detected by the CNN are well formed, consistent with standard morphology (Loomis et al., 1935; Newton Harvey et al., 1937; Silber et al., 2007; Figure 1d), and in agreement with previously observed durations (average ± SEM: 0.69±0.004 s, NHP ECoG; 0.87±0.006 s, EEG; 0.77±0.009 s, iEEG) (Fernandez and Lüthi, 2020; Takeuchi et al., 2016; Warby et al., 2014). To further validate spindles detected by the CNN, we designed a time-shifted averaging approach for application to recordings with only a 1 Hz highpass filter applied (thus excluding any potential effects from lowpass filtering). To do this, we collected signals from detected spindles, filtered at a 1 Hz highpass, time-aligned the events to the largest positive value within the detected window (corresponding to a positive oscillation peak), and then computed the average across aligned events. With this approach, the average over detected spindles exhibited clear 11–15 Hz oscillatory structure (black line, Figure 1—figure supplement 2), while no oscillatory structure is observed when averaging over time-matched randomly selected non-spindle activity (dashed red line, Figure 1—figure supplement 2). This result demonstrates that spindles detected by the CNN exhibit the correct structure even in a mostly raw, unprocessed signal with no lowpass filtering applied, while non-spindle activities only exhibit a peak due to the alignment to the central peak in the window, with a decay consistent with the autocorrelation time present in the 1 Hz filtered signal. We then compared the average number of spindles per minute (Figure 1—figure supplement 3a) and the distribution of peak Fourier amplitudes in the 9–18 Hz band for spindle events detected by the CNN and AT approach (Figure 1—figure supplement 3b). In the intracranial recordings (ECoG and iEEG), AT detects a subset of spindles that are significantly higher amplitude than those detected by the CNN (p<0.02, NHP ECoG recordings; p<1 × 10–12; iEEG recordings, one-sided Wilcoxon signed-rank test; n.s. in EEG), consistent with the expectation that AT will preferentially select the largest amplitude events. The CNN, however, detects a broader set of spindles and can find well-formed spindles that are both large and small in amplitude (Figure 1—figure supplement 4). This improved resolution allows us to study the spatial extent of spindles in an approximately amplitude-invariant manner. Furthermore, to understand more generally the performance of the CNN and AT approaches under different types of noise and in the presence of artifacts, we conducted a detailed simulation study using surrogate data with systematically varying noise characteristics or rate of artifacts (Supplementary file 1 and Figure 1—figure supplements 5 and 6). Finally, we used the pattern of activation of the feature map and gradient map to study the underlying mechanism by which the trained CNN detects sleep spindle oscillations (Figure 1—figure supplements 7 and 8) and we evaluated the choice of architecture tailored with respect to the duration of rhythmic activity (Figure 1—figure supplement 9).

Figure 1 with 12 supplements see all
Electrophysiology, architecture of the convolutional neural network (CNN) model, and detected spindles.

(a) Electrode placement of multichannel electrocorticography (ECoG) recordings of two macaques (top), high-density scalp electroencephalogram (EEG) used for recordings after low- and high-load visual memory tasks (middle), and example intracranial electroencephalogram (iEEG) contacts in a human clinical patient (bottom). (b) Average power spectral density estimate for spindle windows detected by the CNN model (red) and matched non-spindle windows (black), illustrating the nearly 10 dB increase within the 11–15 Hz spindle band in non-human primate (NHP) ECoG recordings (top), human EEG recordings (middle), and human iEEG recordings (bottom). Power at line noise frequency omitted for clarity. (c) The architecture of the CNN model developed for spindle detection. (d) Examples of detected spindles by the CNN model (red) in NHP ECoG recordings (top), human EEG recordings (middle), and human iEEG recordings (bottom).

What is the spatial extent of spindle oscillations across cortex? To answer this question, we studied the distribution of spindle co-occurrence across electrodes in the sleep recordings. We defined three classes of spindles based on co-occurrence across recording sites: local (1–2 sites), regional (3–10 sites), and multi-area (more than 10 sites). We noted that our CNN approach detected many spindles with electrode sites distributed widely across the cortex (Figure 2a). By taking into account the unique cortical regions sampled by electrodes in each individual (9 on average, ranging from 7 to 12 cortical regions, Supplementary file 2), we verified that these were indeed multi-area spindle events (Figure 2b) that happen on average across 60% of recorded cortical regions. Considering the different spatial sampling across subjects, we also confirmed a significant increase (ranging from 40% to 70%) in cortical region participation at the subject level in multi-area spindles with respect to the local spindles (Figure 2—figure supplement 1a). We then compared spindles detected by the CNN and AT approaches. To do this, we first computed the ratio of spindles detected by the CNN and AT for all classes. This comparison revealed that multi-area spindles were systematically detected approximately 1.5 (ECoG) to 10 (iEEG) times more often with the CNN than with the AT (Figure 2c and Figure 2—figure supplement 1b). Across all recordings, the increase in the multi-area spindles detected by the CNN was significantly greater than in the local spindles (p<1 × 10–3, NHP ECoG recordings; p<1 × 10–5, EEG recordings; p<0.02, iEEG recordings, one-sided Wilcoxon signed-rank test; similar results for the local-regional comparison, p<0.02, EEG recordings; p<0.01, iEEG recordings, one-sided Wilcoxon signed-rank test, n.s. in NHP ECoG). Importantly, iEEG has the highest spatial resolution across the recording types studied here (Mukamel and Fried, 2012) and also exhibits the largest increase in multi-area spindles detected by the CNN versus the AT. It is important to note, as well, that in the ECoG dataset the CNN approach detects fewer local and regional spindles than the AT (Figure 2c). This effect was primarily due to estimated threshold varying widely across electrodes in one subject, which in turn caused more detections of local and regional spindles in the AT. Next, we computed spindle participation at the level of cortical lobes (frontal, temporal, parietal, occipital) and cortical systems (executive, limbic, visual, auditory, somatosensory) and detected a significant increase in multi-lobe and multi-system spindles across all recordings (Figure 2—figure supplement 2). Taken together, these results demonstrate that spindles appear much more widespread across cortex when detected using our approximately amplitude-invariant deep learning approach.

Figure 2 with 2 supplements see all
Distribution of the extent of spindles detected by convolutional neural network (CNN) and amplitude-thresholding (AT) approaches.

(a) An example of a widespread, multi-area spindle with electrode sites distributed widely across the cortex. Filled gray circles indicate electrode contacts in gray matter. (b) Plotted is the percentage of unique recorded cortical regions with spindles detected by the CNN in the local versus multi-area case across all subjects in the intracranial electroencephalogram (iEEG) recordings (average ± SEM; n = 389445 for local, n = 28407 for multi-area; p < 1 × 10-10, iEEG recordings, local versus multi-area, one-sided Wilcoxon signed-rank test). Results were similar at the level of individual subjects (Figure 2—figure supplement 1a). (c) Plotted are the ratios of spindles detected by the CNN and AT in non-human primate (NHP) electrocorticography (ECoG) recordings (left, n = 13), human electroencephalogram (EEG) recordings (middle, n = 32), and iEEG recordings (right, n = 89) in local (1–2 sites), regional (3–10 sites), and multi-area (more than 10 sites) spindle classes (average ± SEM in all cases; p > 0.1, NHP ECoG recordings; p < 0.02, EEG recordings; p < 0.01, iEEG recordings, local versus regional comparison, one-sided Wilcoxon signed-rank test; p < 1 x 10-3, NHP ECoG recordings; p < 1 x 10-5, EEG recordings; p < 0.02, iEEG recordings, local versus multi-area comparison, one-sided Wilcoxon signed-rank test). Across recordings, the increase in regional and multi-area spindles detected by the CNN is significantly larger than for the local spindles (except local versus regional in the NHP ECoG).

The organization of spindles across the cortex is thus neither fully local nor fully global: the co-occurrence patterns of this sleep rhythm contain a mixture of local and widespread events. If this is the case, how can pre-sleep memory engagement impact this distribution? To answer this question, we further studied the human EEG dataset, which had the unique feature of testing sleep after tasks with varying memory loads. Briefly, before nap EEG recordings, subjects completed a task in which five novel outdoor scenes (H-VM) or two novel outdoor scenes (L-VM) were required to be held in working memory for 6 s (Figure 3a). After the delay period, subjects were then presented with a subsequent visual scene and asked whether it belonged to the previously presented set. In each case (H-VM and L-VM), trials were balanced so that the same total number of visual scenes was presented before sleep. An increase in spindle density after memory tasks and its relationship with memory consolidation is well established (Clemens et al., 2005; Dang-Vu et al., 2008; Gais et al., 2002; Schabus et al., 2007; Schabus et al., 2004); however, the effect of memory tasks on co-occurrence remains unknown. Considering the potential circuit mechanism for spindles to link activity in neuron groups distributed across multiple areas in cortex through long-range excitatory connections (Muller et al., 2016), we then hypothesized that sleep following H-VM tasks would exhibit more multi-area spindles and a larger spatial extent. To test this hypothesis, we first confirmed that amplitudes of detected spindles did not differ across L-VM and H-VM conditions (p>0.77, Wilcoxon signed-rank test). We then defined a ‘multi-electrode’ spindle rate, which considers spindles occurring simultaneously on several electrodes as a single event. Importantly, this multi-electrode spindle rate is distinct from the ‘single-electrode’ spindle rate computed previously, where spindles occurring simultaneously across multiple electrodes are not recognized as the same event. We next computed the multi-electrode rate for local, regional, and multi-area spindles after L-VM and H-VM tasks. Both regional and multi-area spindles appeared more often after H-VM than L-VM (p<0.038, regional spindles; p<0.026, multi-area spindles; one-sided paired-sample Wilcoxon signed-rank test; average ± SEM multi-electrode rates 0.77±0.10 [0.79±0.11], 0.82±0.09 [0.99±0.13], and 0.38±0.02 [0.62±0.08] for local, regional, and multi-area spindles, respectively, in L-VM [H-VM in square brackets]) as detected by the CNN model, consistent with our hypothesis (Figure 3b), while local spindles did not appear more frequently (p>0.34, same test). Similarly, the largest increases following H-VM versus L-VM were observed in the subset of multi-area spindles detected by the more-conservative SNR approach (Figure 3—figure supplement 1a); however, no increase in multi-area spindles was observed with the AT algorithm (Figure 3—figure supplement 1b). The CNN model and SNR approach thus provide clearly converging evidence that an increase in distributed spindles appears following H-VM tasks, a change that is not detected by the AT approach. These results not only validate the performance of the CNN approach in contrast to amplitude-based approaches, but also clearly demonstrate that this approach is able to find qualitatively new results providing insight into the process of human memory consolidation. Lastly, we divided EEG electrodes based on their cortical lobe (Figure 3—figure supplement 2a) and studied the change in density of spindles in frontal, occipital, and parietal lobes in low and high visual memory conditions. To do this, in each cortical lobe, we computed the percentage of electrode sites with spindles within the detected windows by the CNN model. Interestingly, we observed a significant increase in the electrode participation during spindles in H-VM versus L-VM across cortical lobes, with the largest increase in the occipital lobe and lowest in the frontal lobe (Figure 3—figure supplement 2b).

Figure 3 with 2 supplements see all
Impact of visual memory load on multi-electrode sleep spindle occurrence.

(a) Schematic representation of low- and high-load visual memory tasks. (b) Multi-electrode spindle rate (average number of spindles detected per minute across the array) in high versus low visual memory condition. Spindles are grouped into local (left), regional (middle), and multi-area (right) classes as detected by the convolutional neural network (CNN) model. A significant increase in the number of spindles among subjects can be observed in multi-area and regional spindles as opposed to local spindles (p>0.34, local spindles; p<0.038, regional spindles; p<0.02, multi-area spindles; one-sided paired-sample Wilcoxon signed-rank test).

We then studied the spindles detected by our CNN approach in the EEG dataset further, by applying techniques previously developed to study the spatiotemporal organization of spindles across electrodes during individual oscillation cycles (Muller et al., 2016; Muller et al., 2014). We first computed the average organization into rotating waves traveling from temporal, to parietal, and on to frontal lobe (denoted ‘TPF waves’) and waves rotating in the opposite direction (first temporal, then frontal, and then parietal lobe, denoted ‘TFP waves’), over all spindle events in H-VM and L-VM conditions. We observed a significant shift toward TFP waves in the H-VM versus L-VM condition (p>0.50, TPF direction; p<0.003, TFP direction; one-sided paired-sample Wilcoxon signed-rank test) (Figure 4a). This increase in TFP waves under conditions of high memory load disappears when we restrict the analysis to local spindles (Figure 4b), consistent with the idea that these TFP waves may be related to multi-area spindles (p>0.42, TPF direction; p>0.63, TFP direction; one-sided paired-sample Wilcoxon signed-rank test). We then repeated the analysis over all electrodes during multi-area spindles and again observed a significant shift in the average TFP rotating waves (p>0.40, TPF direction; p<0.007, TFP direction; one-sided paired-sample Wilcoxon signed-rank test) (Figure 4c and Video 1). These results demonstrate that multi-area spindles detected by the CNN model exhibit a clear rotating wave pattern which increases in the TFP direction under conditions of high memory load.

Figure 4 with 1 supplement see all
Impact of visual memory load on rotating waves.

(a) Average TPF (gray) and TFP (red) rotation directions computed over all spindle activities detected by the convolutional neural network (CNN) model in high versus low visual memory condition. A significant increase in the TFP direction was observed as opposed to the TPF direction in the high visual memory conditions (one-sided paired-sample Wilcoxon signed-rank test). An outlier point (low-load visual memory [L-VM], high-load visual memory [H-VM]): (0.48,0.51) in the TFP direction was omitted for the sake of visualization. (b) Average TPF rotation direction (gray) and TFP rotation direction (red) computed over just local spindles. No significant increase was observed in both directions (one-sided paired-sample Wilcoxon signed-rank test). (c) Finally, average TPF rotation direction (gray) and TFP rotation direction (red) computed over all electrodes during multi-area spindles. The increase in TFP directions became significant in high visual memory conditions in multi-area spindles (one-sided paired-sample Wilcoxon signed-rank test) which verifies that the increase is driven by the multi-area spindles. An outlier point (L-VM, H-VM): (0.51,0.56) in the TPF direction was omitted for the sake of visualization.

Video 1
Rotating waves in multi-area spindles.

An example of a rotating wave in TFP direction during a multi-area spindle detected by the convolutional neural network (CNN) model in the electroencephalogram (EEG) recording. Z-score of bandpass filtered (here 9–18 Hz) signals are plotted in falsecolor in a lateral view of the scalp EEG (where frontal, temporal, and parietal lobes are, respectively, located on the right-hand side, the bottom center and top center).

Discussion

These results may tie the increase in multi-area spindles to the neural circuit mechanism that we have previously identified could play a role in consolidating memories across distributed networks in cortex through synaptic plasticity (Muller et al., 2016). Synaptic plasticity occurs through spike time-dependent plasticity (STDP) (Bi and Poo, 1998; Markram et al., 1997), for which presynaptic vesicle release and postsynaptic spiking must occur with a precision of a few milliseconds (Magee and Johnston, 1997). While it has become increasingly clear that sleep spindles play an active and causal role in sleep-dependent memory consolidation (Aton et al., 2014; Clemens et al., 2005; Eschenko et al., 2006; Gais et al., 2002; Mednick et al., 2013; Rasch and Born, 2013), it remains unclear how these oscillations coordinate activity across areas to shape neocortical assemblies distributed over long distances (Klinzing et al., 2019). In previous work, we studied the spatiotemporal dynamics of the sleep spindle oscillation in intracranial ECoG recordings from human clinical patients, and we found that – instead of being synchronized with zero delay throughout the cortex – sleep spindles are often organized into rotating waves traveling across the cortex in a preferred direction (Muller et al., 2016). Because these waves travel at the same speed as axonal conduction across long-range white matter fiber networks in cortex, the offsets of activity across areas could precisely align spikes across areas to create the conditions necessary for bi-directional synaptic plasticity (Figure 5) – either for creating strong links between assemblies distributed widely across cortex or for downscaling connections to maintain synaptic homeostasis (Crunelli et al., 2018; Klinzing et al., 2019; Tononi and Cirelli, 2014). Importantly, previous theoretical work has identified the relative phase of sending and receiving populations as an important factor in determining the balance toward potentiation or depotentiation by STDP during these rhythms (Muller et al., 2011) and future computational analyses could study these offsets in detail. At the circuit level, thalamocortical circuits may set the rhythm for spindle oscillations in cortex (Clascá et al., 2012; Destexhe and Sejnowski, 2001), which are then shaped into waves by long-range corticocortical connections (and their axonal time delays). Understanding the network mechanism for this interplay between thalamocortical and corticocortical connections is thus an important subject for future computational analyses and network models.

Rotating waves during multi-area sleep spindles provide a mechanism for linking local neuronal populations distributed across cortex.

(Left) Spindles that appear across multiple areas are often organized into rotating waves in human cortex. (Right) Phase offsets between cortical regions emerging during rotating waves correspond to axonal conduction delays of white matter fibers and can provide a mechanism to align spikes between cell populations distributed widely across cortex.

This mechanism places the spatial extent of spindles across cortex, and how this extent changes under different memory conditions, as a critical point in understanding the neural process of sleep-dependent memory consolidation. The spatial extent of spindles we reported in this work provides a potential mechanism by which long-range excitatory connections between distant populations in cortex could be strengthened during memory consolidation in sleep. Based on this mechanism, we then hypothesized that large, multi-area spindles may exhibit an increase following H-VM tasks. Consistent with this additional hypothesis, both the CNN and SNR methods detect an increase in multi-area spindles and rotating waves uniquely following H-VM tasks. This increase in multi-area spindles was further associated with an increase in waves traveling in the temporal → frontal → parietal (TFP) direction. Interestingly, these TFP waves are opposite to the dominant rotation direction observed in previous work (Muller et al., 2016), potentially reflecting increased top-down influence from higher cortical areas following the high-load memory condition. These present results clearly indicate that different memory conditions can modulate the extent and spatiotemporal organization of sleep spindles across cortex; however, future analyses of intracranial recordings at very high spatial and temporal resolution during memory tasks will be needed to fully understand the spatiotemporal dynamics reported here and their connection to the process of sleep-dependent memory consolidation.

Previous work has found that spindles can occur broadly across the cortex with low measures of synchrony in EEG and magnetoencephalography (Dehghani et al., 2011a; Dehghani et al., 2011b, Dehghani et al., 2010), and also in iEEG recordings (Frauscher et al., 2015). The results reported in our work may provide insight into the underlying mechanism for these previous findings: because traveling waves introduce systematic phase offsets across sites, traveling waves during multi-area spindles can account for increases in spindle power across broad regions of cortex that also show low synchrony (as measured, for example, by correlation between electrode sites in Frauscher et al., 2015). The results reported in this work highlight the importance of distinguishing between the extent of spindle occurrence (as with detecting individual spindle events in this work) and measures of synchrony. Further comparisons focusing on spatiotemporal dynamics and spindle synchrony, for example with simultaneous EEG and iEEG recordings, will be important for future work.

Taken together, these results provide considerable and convergent evidence from both human and NHP sleep that (1) the spatial extent of sleep spindles was previously underestimated by AT approaches (which tend to select only the highest-amplitude events) (Figure 2), (2) this spatial extent can be modulated by the specific memory conditions prior to sleep (Figure 3), and (3) increased spatial extent in sleep spindles following H-VM tasks is also associated with rotating waves traveling in a specific direction across cortex (Figure 4). To analyze these sleep recordings, we adapted newly developed deep learning approaches for detecting rhythmic events in high-noise data (George and Huerta, 2018; Perol et al., 2018). The detection process involves two steps: first, we use a simple algorithm (here, the SNR approach) to detect a subset of high-quality examples that can be used for training the CNN, and second, we use the CNN to detect events throughout the recording. The fact that this two-step training approach works well on recordings with very different electrode types and spatial sampling (ranging from scalp EEG to invasive intracranial depth electrodes, Figure 1—figure supplement 3a) demonstrates promise of this computational tool for analysis of other rhythmic waveforms that may be of interest in high-noise biological recordings.

We believe this two-step approach represents a methodological advance, coupling a constrained initial detection step with a CNN model that can detect a comprehensive set of events in noisy neural recordings. Our toolbox for this two-step training protocol is available online (http://github.com/mullerlab/spindlecnn), with detailed documentation for applications to new neural rhythms and general timeseries data across biology. We believe this technique can provide a first step in addressing an important methodological consideration in analysis of sleep: how can we make population-level statements about a set of neural events detected algorithmically in the absence of a ground truth, without relying on arbitrarily defined thresholds? This technique could be useful throughout biology where questions such as this arise. At the same time, however, we must emphasize that, while the CNN provided a robust method to sensitively detect a comprehensive set of spindles in the sleep recordings studied here, care must be taken both to understand the mechanisms underlying the selections by deep learning algorithms in each case and to validate results on well-constructed controls. Importantly, future work to expand these methods to new sleep phenomena or other biological rhythms should carefully consider the control analyses developed here (e.g., Figure 1—figure supplements 2 and 10), along with the analyses developed to understand the features selected by the convolutional model after training (Figure 1—figure supplements 7 and 8) and the choice of CNN architecture (Figure 1—figure supplement 9). In particular, the control analyses using the subset of spindles detected by the SNR algorithm represents a useful strategy to validate findings from the CNN, as the SNR method is both highly interpretable and has well-controllable statistical performance (as inspired from the constant false alarm rate technique in radar). We thus believe that this set of methods can inspire future well-controlled studies utilizing open-access data that are increasingly available for computational analysis of neural dynamics in intracranial recordings (Boran et al., 2020; Frauscher et al., 2018; Nejedly et al., 2020; Stolk et al., 2018; von Ellenrieder et al., 2020).

Materials and methods

Recordings

Request a detailed protocol

We studied performance of the CNN model across three sleep datasets that include full recordings annotated as sleep without excluding REM states obtained from electrodes ranging from traditional scalp EEG to invasive intracranial depth electrodes. These datasets represent recordings from very different electrode types, which vary widely in resolution and SNR. Training the CNN model in the same way over these very different recordings demonstrates the generality of the framework developed here; further, these results also represent a cross-species comparison of sleep-rhythm dynamics in NHP and human neocortex.

The first dataset contains ECoG recording from most of the lateral cortex in two macaques during natural sleeping conditions (Yanagawa et al., 2013). Recordings were obtained from 128 electrodes in both monkeys and sampled at 1 kHz by a Cerebus data acquisition system (Blackrock Microsystems, Salt Lake City, UT). Sleep state was determined by the degree of spatial synchronization in slow wave oscillations and a significant increase in delta power was reported in sleep condition versus waking activity (Chauvette et al., 2011; Dang-Vu et al., 2005; Destexhe et al., 1999; Murphy et al., 2009). This dataset was recorded and distributed by Laboratory for Adaptive Intelligence, BSI, RIKEN, and was made freely available at http://neurotycho.org/anesthesia-and-sleep-task.

The second dataset contains high-density scalp EEG recording from 20 healthy participants (Mei et al., 2018). Each participant participated in two separate sessions and completed an H-VM and L-VM task. The recordings were obtained during naps following the visual memory tasks from a 64-electrode EEG skull cap and sampled at 1 kHz. Sleep state was manually assessed by an expert for stage 2 NREM sleep. Ultimately, sleep recordings that did not reach stage 2 sleep or were too noisy were excluded from the study. Under these criteria, four subjects were excluded (subjects 12, 20, 26, and 27). In addition, the recordings were common average referenced to remove large artifacts with potentially non-neural origin. These recordings were made freely available at the Open Science Framework through the link https://osf.io/chav7.

The last dataset contains iEEG recordings from five epileptic patients in the Epilepsy Monitoring Unit (EMU) at London Health Sciences Centre (LHSC). Patients were implanted using depth electrodes for the sole purpose of surgical evaluation. Informed consent was collected from the patients in accordance with local Research Ethics Board (REB) guidelines. Each patient was implanted with 9–15 iEEG electrodes located across the cortex with up to 10 contacts in gray or white matter. The iEEG signals were recorded continuously for a duration of 7–14 days for the purpose of seizure localization. We used clinically annotated sleep onsets and studied half an hour recording starting from the beginning of the sleep/nap cycles in electrode contacts located within gray matter.

SNR measure for sleep spindle detection

Request a detailed protocol

To specify a subset of spindles required to train our CNN model, we implemented a modified version of SNR algorithm (Muller et al., 2016). This algorithm, which is inspired by the adaptive, CFAR technique in radar, was used to detect narrow-band rhythmic activities. We measure the ratio of power within the frequency band of interest (here, 9–18 Hz) to power in the rest of the spectrum (1–100 Hz bandpass, with band-stop at 9–18 Hz) at each electrode. The SNR measure is computed over a sliding window of time (500 ms) and produces an estimate of how power in the frequency band of interest compares to total power in the recording, taking into account the noise on individual electrodes. We then used the SNR algorithm to produce high-quality training samples for the CNN model. To do this, we reduced the probability of false positives by setting the threshold to the 99th percentile of the SNR distribution, thus detecting only the activity patterns that have the highest unique power concentration in the spindle frequency range. We additionally required the SNR algorithm to only include activities with a duration between 0.5 and 3 s, consistent with the duration of sleep spindles. The detected windows are then used for training the CNN model.

To additionally verify performance of the SNR algorithm, we implemented this approach over 1 s recordings of a 90 by 90 array of local field potentials (LFPs) generated by a spiking network model of cortical activity in the awake state (Davis et al., 2021), which does not contain the thalamic reticular loops and thalamocortical projections needed to generate sleep spindles. This model, composed of several million neurons with biologically realistic synaptic connectivity (and several thousand synapses per cell), creates realistic ongoing activity patterns consistent with the well-studied asynchronous-irregular state, corresponding to activity in the cortex of awake animals (Destexhe, 2009). In addition, this model utilizes a recently developed LFP proxy (Mazzoni et al., 2015), allowing us to analyze a population signal using our spindle detection algorithm. SNR values calculated from these data were uniformly below 0 dB, confirming the robustness of our approach in uniquely detecting spindle frequency activity through a known ground truth dataset.

CNN for sleep spindle detection

Request a detailed protocol

We developed a CNN to detect spindles during sleep. The model is motivated by the successful implementation of convolutional networks for waveform detection with earthquakes and gravitational waves in high-noise settings (George and Huerta, 2018; Perol et al., 2018). If trained properly, it has the ability to detect clearly formed spindles ranging from low to high amplitudes (Figure 1d) and provides a great opportunity to study the spatial and temporal analysis of spindle activities across the cortex. We implemented an architecture similar to the one proposed by George and Huerta, 2018, with small modifications to the input and convolutional layers to take into account the basic features of the spindle rhythm in cortex (e.g. average duration). The CNN architecture is also slightly tailored to different sampling rates in each recording modality. As in previous work, the convolutional layer is designed to start by extracting local features, gradually extracting longer-timescale features by decreasing the feature space. Using this strategy, the CNN model can efficiently learn to detect the specific waveform characteristics of the sleep spindle in different types of recordings.

We verified model quality using ECoG recordings by minimizing the difference between predicted and training labels marked by the SNR approach. In addition, we verified that the proposed CNN model is not sensitive to the slight change in the number of layers (e.g. 4, 5, and 6 convolutional layers) and hyperparameters such as learning rate, maximum number of epochs used for training, and pooling parameters by conducting a comprehensive sensitivity analysis (Supplementary file 3). To perform this sensitivity analysis, for each CNN architecture, we made a grid search over the potential range of hyperparameters, measuring the similarity of model output by Cronbach’s alpha. Similarity across hyperparameters within 10–50% of those used in our analysis was greater than 0.96, indicating high reproducibility under moderate parameter variability. We selected one of the best architectures and a combination of hyperparameters that we tested in the grid search (Figure 1c). We then used the same architecture and hyperparameters across all subjects and recording datasets. We trained a separate CNN model for each subject on a portion of the available recording and then applied the trained model to detect spindles across the entire recording. Our CNN model is a one-dimensional (1D) model (applied always independently to individual electrodes in the recording) with five convolutional layers (with 32, 64, 128, 192, and 256 filters) and four fully connected layers (with sizes 128, 64, 32, and 2). Each convolutional layer is followed by a maxpool and rectified linear unit layers, and the output of the fifth convolutional layer is gradually flattened into 2D vectors using the fully connected layers followed by rectified linear unit layers. Our classifier has an additional softmax layer at the end which returns the probability of a spindle in addition to the predicted label.

We trained a separate CNN model for each subject over a subset of spindle and non-spindle windows selected from the sleep recordings. To deal with the classification problem of a highly imbalanced training set, we randomly selected a subset of non-spindle windows (up to twice the number of spindles) and then trained the model over the new dataset. In our training process, we noticed that a subset of approximately 1500 windows of spindle and non-spindles can provide enough data for training the CNN model. After training the CNN model, we implemented the model over the entire sleep recording. The CNN model takes a sliding window of sleep recording (500 ms which is bandpass filtered at 1–100 Hz after removal of line noise and harmonics) as an input and predicts its label (spindle or non-spindle). The sliding window starts at the beginning of the recording and moves 100 ms in each step. To find the start and end time of a spindle, we first combined all overlapping spindle windows. We then included neighbor windows if there is any spindle within 100 ms of the combined windows to account for potentially mislabeled windows. The start of a spindle is finally set to the beginning of the first window, and the end of the spindle is set to the end of the last window. To study the spatial extent of spindles, we then classify detected events as local (1–2 recording sites), regional (3–10 sites), or multi-area (more than 10 sites) in each window. We note that classifying events in this way allows correctly characterizing an event that starts as a local spindle and then evolves to a multi-area event, by counting the times where the event was local separately from the times where the event was distributed across many sites.

PSD estimate

Request a detailed protocol

To verify performance of the CNN, SNR, and AT approaches, we compared PSD estimates of spindle and non-spindle activities (Welch’s method; Figure 1b, Figure 1—figure supplements 3b and 10). In both cases, we first removed line noise artifacts. We then computed PSD over windows of 0.5 s with no overlap and average spectra over detected events. Matched non-spindle PSDs were estimated over a large number of randomly selected non-spindle windows. The increase in the power during the natural frequency range of sleep (~9–18 Hz) in spindle versus non-spindles activities demonstrates the ability of both the CNN model and SNR algorithm to correctly identify spindle activities.

Time-shifted averaging control

Request a detailed protocol

As an additional control analysis, we computed averages over detected spindles, with activity shifted to centrally align the largest oscillation peak in the detected time window. To compute this average, we first needed to correct for the time offset between different spindles. To do this, we shifted detected spindles to the largest positive value within the detected window, corresponding to the positive potential of an individual spindle oscillation cycle, and then took the average over all time-shifted windows. The average of time-shifted signals is computed over spindle windows detected by the CNN approach, as well as matched randomly selected non-spindle windows. Importantly, while the time-shifted average clearly exhibits 11–15 Hz oscillatory structure when computed over spindle events detected by the CNN, this need not be the case, as demonstrated by application of the same approach to matched non-spindle events (Figure 1—figure supplement 2). The peaks observed in the center of the signal averaged over non-spindle windows are due to the alignment procedure. Naturally, this peak exhibited a decay consistent with the autocorrelation time present in the signal; importantly, however, it shows no oscillatory structure consistent with spindle activity in the non-spindle windows. This result demonstrates that our CNN model can correctly distinguish between spindle and non-spindle events.

We also systematically studied the sensitivity of the CNN model as a function of the SNR threshold used for building the training set. To do this, we computed the time-shifted average over spindle events detected by the CNN model at different levels of the SNR threshold (Figure 1—figure supplement 11). Clear, well-formed 11–15 Hz oscillatory structure is observed in the time-shifted averages above 0 dB threshold, verifying the quality of detected spindles by the corresponding CNN models. However, the 11–15 Hz oscillatory structure starts to disappear below 0 dB because an SNR threshold below 0 dB introduces errors into the training sets by mislabeling noise signals as spindles. On the other hand, similar oscillatory shapes of time-shifted average above 0 dB confirms the ability of the CNN model to perform robustly while trained over different sets of clearly formed spindles.

Comparison with AT approach

Request a detailed protocol

The AT approach has been used extensively in the literature to automatically detect spindles during sleep (Gais et al., 2002; Nir et al., 2011). In this approach, a spindle is detected when the amplitude of the bandpass signal stays above a threshold for a limited period of time (e.g. at least 500 ms; cf. Figure S5 in Nir et al., 2011). To implement this approach, we first bandpass filter the signal at the frequency of 11–15 Hz and then compute the signal envelope using Hilbert transformation over a sliding window of 0.5 s. The sliding window starts at the beginning of the recording and moves 100 ms in each step. The start and end time of each spindle is computed similarly as with the CNN model, where we combine overlapping spindle windows and neighbor windows within 100 ms. A spindle is detected whenever the signal envelope stays above the predetermined threshold for at most 3 s.

To determine the most appropriate threshold for comparison to the CNN and SNR approaches, we first computed the distribution of electrode-level RMS amplitude that results in approximately 2 spindles per minute and then set the overall threshold to its average across all electrodes. The overall threshold is computed independently for each subject to account for the differences across subjects as well as across different electrodes. The quality and extent of detected spindles by the AT approach was then compared with the CNN and SNR (Figure 2c, Figure 1—figure supplements 1 and 3, 4, 10, and 12, Figure 2—figure supplements 1 and 2, and Figure 3—figure supplement 1). The CNN model has a relatively amplitude-invariant nature in comparison with the AT approach, which is highly sensitive to a predefined cutoff AT. The AT approaches may only select spindles with the largest-amplitude events, or could miss ones that temporarily dip below the threshold, while our approach has the ability to find well-formed spindles that are both large and small in amplitude (cf. first and third EEG spindles of Figure 1d which were not detected by AT and Figure 1—figure supplement 4). Consistent with our expectation, the AT approach detects spindles of higher amplitude than the CNN approach (Figure 1—figure supplement 3b) with the exception of the EEG dataset, where lower SNR may obscure this effect.

Simulated data control – signal amplitude

Request a detailed protocol

We simulated 60 min of recording containing on average 3 spindles per minute. The spindles were simulated using

(1) f(t) = Aei(ωt+θ0)

where A is the oscillation amplitude, ω is the oscillation angular frequency,t[1,Nt] is sample number, θ0 is the initial polar angle. Oscillation amplitude (A) was set to a constant value, and the oscillation angular frequency (ω) and initial polar angle (θ0) were randomly selected from, respectively, 11–15 Hz spindle frequency range and [0, 2π]. We also added two types of noise to the signal including white noise with constant power spectrum, and Brownian noise with 1/f2 power spectrum.

We utilized these surrogate data, which have a clear PSD peak in the spindle frequency range (Figure 1—figure supplement 1a), to study how sensitive the SNR approach is to the change in spindle and noise amplitude as opposed to the AT techniques. We first applied the SNR algorithm and AT approach to the simulated signal to detect spindle activities. For the SNR approach, we used either the 99th percentile of the SNR distribution, or 0 dB if greater (which represents parity between power in the spindle passband and the rest of the signal spectrum) as the threshold. For the AT approach, we used three standard deviations of instantaneous amplitude. We then repeated the entire analysis once after we doubled the noise amplitude, and once after dividing the spindle amplitude in half. Figure 1—figure supplement 1b contains an example of spindle activity detected by both approaches in the original signal. Interestingly, in the higher-noise setting, as well as with lower-amplitude spindles, the SNR approach was still able to detect the spindle activity while the AT approach failed to detect the spindles (Figure 1—figure supplement 1c and d). These results demonstrate the superior and robust performance of the SNR algorithm in face of changes in signal and noise amplitude.

Simulated data control – varying noise

Request a detailed protocol

We simulated 30 min recording of a 10 by 10 array of electrodes containing on average 2 spindles per minute. The spindles were simulated using Equation 1 where oscillation amplitude (A) was set such that it follows the standard spindle waning and waxing pattern, and the oscillation angular frequency (ω) and initial polar angle (θ0) were randomly selected from 11 to 15 Hz spindle frequency range and [0, 2π], respectively.

We added different types of noise to the signal to verify that the CNN model is relatively general to the type of noise, provided there is enough training data and high-quality marked events. To do this, we chose noise with different type of power spectrum including (1) white noise with constant power spectrum; (2) noise with 1/f power spectrum; (3) pink noise with 1/f power spectrum; and lastly (4) Brownian noise with 1/f2 power spectrum. We also studied change in the performance of the CNN and AT approaches under biological forms of noise such as REM theta oscillation and non-biological artifacts. We simulated theta oscillations with similar characteristics to the spindle oscillation with angular frequency (ω) and duration randomly selected from 4 to 8 Hz and 400–1000 ms. For non-biological artifacts, we first randomly chose a subset of artifacts detected as spindles by the AT approach in the iEEG recording. We then used the fast Fourier transform (FFT) to convert these artifacts into frequency domain. We next used the signal amplitude and randomly selected phases from [0, 2π], to generate a new set of artifacts and used the inverse FFT to convert the signal back to time domain. We visually inspected the simulated artifacts and verified the signals by comparing the PSD of the simulated artifacts with the original artifacts (Figure 1—figure supplement 6a).

For each type of noise, we first used the SNR algorithm to generate a subset of high-quality spindles for training the CNN model. After training the CNN model, we implemented the CNN model and AT approach to detect spindle activities. Lastly, we compared the detected activity by CNN, SNR, and AT with the actual spindles (Supplementary file 1). In all cases, the CNN was able to efficiently learn to detect the specific waveform characteristics distinguishing the sleep spindle rhythm in these recordings (e.g. Figure 1—figure supplement 5) showing that we can expect this approach to generalize well across recordings with different types of noise.

Moreover, we further studied the effect of artifacts on performance of the CNN and AT approach by systematically increasing the number of artifacts per minute in the surrogate data. In particular, the AT approach seems to be very sensitive to recording artifacts. The CNN model performs robustly in face of increases in the number of artifacts, while the performance of the AT gradually decreases as the number of artifacts per minute increases (Figure 1—figure supplement 6b). This result further verifies that our approach is not sensitive to different types of artifact in the recording as opposed to the AT approach.

Rotating wave direction

Request a detailed protocol

To estimate the degree of rotational activity in the multi-area spindles, we compute

(2) αTPF(t) = (1/n) n = 1N(eixn,teiθn) ;
(3) αTFP(t) = (1/n)n = 1N(e-ixn,teiθn);

where αTPF(t) and αTFP(t) are, respectively, positive and negative rotation direction at time t,xt(n,t) is the phase angle of nth electrode at time t, N is the total number of electrodes, and θ is the electrodes’ polar angle with respect to the sagittal plane along the midline of the brain. This metric allows us to quantify the strength of the rotational pattern of activity in TPF and TFP on the array of electrodes during spindle activity (Figure 4). The proposed metrics are validated using simulated data (Figure 4—figure supplement 1 and Videos 2 and 3) in the following section.

Video 2
Simulated TPF waves are well detected by our computational approach.

An example of surrogate data, with simulated rotating spindles in the TPF direction. Z-score of bandpass filtered (here 9–18 Hz) signals are plotted in falsecolor in a lateral view of the scalp electroencephalogram (EEG) (where frontal, temporal, and parietal lobes are, respectively, located on the right-hand side, the bottom center and top center).

Video 3
Simulated TFP waves are well detected by our computational approach.

An example surrogate data, with simulated rotating spindles in the TFP direction.

Simulated data control – rotating wave

Request a detailed protocol

We simulated multi-area spindles with rotational wave organization across the 10–20 system of EEG recordings. We simulated rotating waves using

(4) f(t,θ) = Aei(ωt-γθ)+ση(t)

where A is the oscillation amplitude, ω is the oscillation angular frequency, γ is the polar wavenumber, θ is the polar angle with respect to the wave center, t[1,Nt] is sample number, and η(t) is a real-valued Gaussian white noise term (compare with Equation 12 in Muller et al., 2016). Oscillation amplitude (A) and polar wave number (γ) were set to unity, without loss of generality, and oscillation angular frequency, ω, is selected randomly from 11 to 15 Hz spindle frequency range, and lastly, the noise term ση(t) was set to zero for the sake of simplicity. We set θ with respect to the polar angle of electrodes in the 10–20 system and the sagittal plane along the midline of the brain (compare with Videos 2 and 3). As expected, the average positive direction estimated over 1000 simulated multi-area spindles was almost 0.9, while the average over the negative direction was almost 0.1, which is close to the case with no synchronized activity (Figure 4—figure supplement 1a, Video 2). We verified the direction by reversing the traveling wave in our simulation and achieving the opposite result (Figure 4—figure supplement 1b, Video 3). The simulation results confirm the accuracy of our measure for estimating rotational direction in multi-area spindles.

CNN visualization and interpretation

Request a detailed protocol

To better understand how the CNN model detects spindle oscillation, we studied the filter and saliency maps of the CNN models. To do this, we first simulate six spindle/non-spindle signals (500 ms), including (1) white noise and spindle; (2) white noise and ‘half spindle’ signal; (3) white noise and combination of two spindles of different amplitude; (4) white noise and theta oscillation; (5) pure white noise; and (6) white noise and Brownian noise. We then visualized the filters and saliency maps across these simulated signals as an input to a CNN model trained on the EEG dataset.

To better visualize the filters, we plotted the output features of the CNN layers. The feature maps visualize CNN filters applied into input signals. In Figure 1—figure supplement 7, we plot an example feature map from the last convolutional layer of the CNN model across the six simulated signals. This feature map can accurately detect the maximum amplitude within each cycle using the maximum activation of the CNN model across all oscillations. Interestingly, we observe a relatively similar pattern of activation across the remaining filters, indicating that the timing and relative height of the maximum amplitudes represent the type of activity pattern in the input signal (spindle versus non-spindle). Using these features, the CNN model can learn to reliably detect the specific waveform characteristics of the sleep spindle in different types of recordings.

In addition, we studied gradient attribution maps to identify part of the signal that are most important for classification in the CNN model. To do this, we plot the gradient of the predicted spindle/non-spindle class with respect to simulated signals (Figure 1—figure supplement 8). The area of the signal with the highest modulation in amplitude has the greatest impact on the classification. For example, in the ‘half spindle’ signal, only the half of the signal containing the spindle oscillation is of importance in the classification. In the signal with spindles of different amplitude, the spindle with the highest amplitude is relatively more important than the other half of the signal. The pattern of activations and gradient map across the simulated signals provides insights into the underlying mechanism by which the CNN model efficiently distinguishes between different types of oscillation.

CNN choice of architecture and hyperparameter setting

Request a detailed protocol

The CNN architecture should be tailored with respect to the duration of rhythmic activity, type of oscillation, and sampling rates of recording modality. The duration of the rhythmic activity and sampling rate determine the length of the sliding window for the CNN model. For example, in our NHP EEG recording with the sampling rate of 1000 Hz, the sliding window is set to 0.5 s which is the minimum duration of spindle activities observed during sleep and contains 500 data points. We next specify the filter size with respect to the length of the sliding window and types of rhythmic activity. The CNN layer is designed to start by extracting local features, gradually extracting longer-timescale features by decreasing the feature space. Filter sizes covering approximately one oscillation cycle (70–120 ms) are effective in detecting spindle activity. To understand this further, we simulate 10 recordings with 2 spindles per minute and add different types of noise and artifacts (Supplementary file 1). Using these surrogate recordings for which we have the exact timing of spindles, we demonstrate that longer filter size is ineffective at detecting spindles. Specifically, we gradually increase the filter size (Figure 1—figure supplement 9) and compute the performance of the CNN model. As expected, the CNN performance drops as we increase the filter size, verifying this mechanism. This result further validates the generality of the CNN approach for detecting neural rhythms, while also getting at the mechanism. We believe that a similar mechanism can be implemented for specifying the filter size for other neural and biological rhythms. The current CNN architecture works perfectly with slight changes in the sliding window (duration and sampling rate) and type of oscillation, but it requires modification otherwise.

Electrode localization

Request a detailed protocol

For the purpose of electrode localization in the iEEG recordings, we developed an image processing pipeline which involves electrode contact localization, brain tissue segmentation, and atlas fitting. Semi-automatic contact localization was performed in 3D Slicer using the SEEG Assistant (SEEGA) module (Narizzano et al., 2017). The entry and target points of each electrode were manually defined on the post-operative CT image. The entry/target labels were provided to the SEEGA algorithm, which automatically segmented the electrode contacts. To obtain brain location information for each contact, brain tissue segmentation and atlas fitting was carried out. To enable the use of anatomical priors during tissue segmentation, the pre-operative T1w MRI was non-linearly registered to the MNI152 2009c Nonlinear Symmetric template (https://www.bic.mni.mcgill.ca/ServicesAtlases/ICBM152NLin2009) using NiftyReg (Modat et al., 2010). An anatomical mask was generated by applying the inverse transform to the T1w image using the antsApplyTransforms algorithm from Advanced Normalization Tools 2.2.0 (ANTS; http://stnava.github.io/ANTs; Cook, 2022). Segmentation of gray matter, white matter, and cerebrospinal fluid was performed using the Atropos algorithm from ANTS (Avants et al., 2011b), which implements k-means classification (k=3). The resulting posteriors were merged into a 4D volume using the fslmerge algorithm from FMRIB Software Library v6.0 (FSL; https://fsl.fmrib.ox.ac.uk/fsl/fslwiki). The CerebrA atlas (Manera et al., 2020) was used to obtain anatomical labels for each electrode contact. Normalization to template space (MNI152NLin2009cAsym) was performed using the non-linear SyN (Avants et al., 2011a) symmetric diffeomorphic image registration algorithm from ANTS, using both the brain masks of the pre-operative T1w and template space. Using the inverse of the non-linear transform, the CerebraA atlas labels were warped to the pre-operative T1w MRI space. The atlas labels were then dilated using the fslmaths algorithm from FSL. The final T1w brain tissue/atlas segmentation was mapped to the contacts to provide location information for each contact (tissue probability and brain anatomical region). This custom processing pipeline has been made available on GitHub (https://github.com/akhanf/clinical-atlasreg; Khan, 2020).

Code availability

Request a detailed protocol

Our custom MATLAB (MathWorks) implementations of all computational analyses, along with the analysis scripts used for this study are available as an open-access release on GitHub (https://github.com/mullerlab/spindlecnn, swh:1:rev:0c503d103e4a0cf041e43903a896bb25b0c66b9b, Mofrad, 2022).

Data availability

Source data for non human primate electrocorticographic (ECoG) recording and human EEG recordings were made freely available through following links: http://neurotycho.org/sleep-task, https://osf.io/chav7. Our custom MATLAB (MathWorks) implementations of all computational analyses, along with the analysis scripts used for this study will be made available as an open-access release on GitHub (http://github.com/mullerlab/spindlecnn, copy archived at swh:1:rev:0c503d103e4a0cf041e43903a896bb25b0c66b9b).

References

  1. Book
    1. Destexhe A
    2. Sejnowski TJ
    (2001)
    Thalamocortical Assemblies
    Oxford, UK: OUP.
    1. Gais S
    2. Mölle M
    3. Helms K
    4. Born J
    (2002)
    Learning-dependent increases in sleep spindle density
    The Journal of Neuroscience 22:6830–6834.
  2. Book
    1. Richards MA.
    (2005)
    Fundamentals of radar signal processing
    Tata McGraw-Hill Education.
    1. Silber MH
    2. Ancoli-Israel S
    3. Bonnet MH
    4. Chokroverty S
    5. Grigg-Damberger MM
    6. Hirshkowitz M
    7. Kapen S
    8. Keenan SA
    9. Kryger MH
    10. Penzel T
    11. Pressman MR
    12. Iber C
    (2007)
    The visual scoring of sleep in adults
    Journal of Clinical Sleep Medicine 3:121–131.

Decision letter

  1. Demba Ba
    Reviewing Editor; Harvard University, United States
  2. Timothy E Behrens
    Senior Editor; University of Oxford, United Kingdom
  3. Carmen Varela
    Reviewer; Florida Atlantic University, United States
  4. Demba Ba
    Reviewer; Harvard University, United States

In the interests of transparency, eLife publishes the most substantive revision requests and the accompanying author responses.

Decision letter after peer review:

[Editors’ note: the authors submitted for reconsideration following the decision after peer review. What follows is the decision letter after the first round of review.]

Thank you for submitting the paper "Waveform detection by deep learning reveals multi-area spindles that are selectively modulated by memory load" for consideration by eLife. Your article has been reviewed by 2 peer reviewers, and the evaluation has been overseen by a Reviewing Editor and a Senior Editor. The following individuals involved in review of your submission have agreed to reveal their identity: Carmen Varela (Reviewer #1).

Comments to the Authors:

We are sorry to say that, after consultation with the reviewers, we have decided that this work will not be considered further for publication by eLife.

Overall, as a group, the reviewers found the application of CNNs to the detection of spindles novel and interesting. More generally, the reviewers thought that a method to detect rhythmic behavior from noisy data could have an important impact on neuroscience research. At the same time, the reviewer expressed some reservations, and felt that the manuscript requires additional work to assess the utility of the CNNs as an addition to sleep biologists' toolkits. Should the authors decide to submit a revised version of the manuscript, we will do our best to have it assessed by the reviewing editor and the reviewers who provided the comments below.

The following represent three main critiques/suggestions from the reviews.

1. The reviewers found the comparisons to existing methods for spindle detection too simplistic and lacking. More specifically, the authors compare their method mainly to the AT method. The reviewers found the lack of details of the authors' implementation of the AT method, and the lack of thorough comparison to spindle-detection algorithms more sophisticated than the AT method, such as the SNR method, surprising. The reviewers suggest that the authors describe their implementation of the AT method in more detail (e.g. threshold selection etc…). The reviewers also suggest that the authors perform thorough comparisons of the CNN to methods/algorithms more sophisticated than the AT method, both in simulations and on the data used in the manuscript. Evidence that the CNN detects substantially more spindles than these methods would motivate its adoption by biologists.

2. The reviewers felt that the analyses do not support the authors' interpretations and their claims of new scientific findings. First, differences in spatial sampling of neural activity by EEG, EcOG and iEEG, together with the fact that volume conduction may contribute to synchronous activity, limits the interpretation of the authors result. Second, a line of work by Dehghani, Cash, Halgren and colleagues investigating the spatial extent of spindles suggest a lack of synchrony: e.g. (1) doi: 10.1152/jn.00198.2010, (2) doi: 10.1002/hbm.21183, (3) doi: 10.1016/j.clinph.2010.06.018. The reviewers suggest that the authors temper their claims significantly, and put them in the context of the above literature.

3. The reviewers thought the manuscript could benefit from an expanded discussion of the motivation for utilizing the CNN for spindle detection, in particular of how, if at all, they tailored the architecture to the spindle-detection setting. The description of the architecture (layers etc…) provided in the manuscript seems to mirror the description of that developed for gravitational wave detection. More generally, the reviewers thought that a discussion of how to tailor the architecture to the detection of brain rhythms other than spindles would benefit the neuroscience community. For instance, variables such as the length of the rhythmic pattern of interest ought to impact architectural choices.

4. The reviewing editor asks that, where applicable, the authors modify the manuscript to address the comments from the detailed reviews shared below.

Reviewer #1:

Mofrad et al. trained a convolutional neural network (CNN) to improve the detection of multi-area spindles and study their regulation by working memory load. They used the CNN algorithm on three different sleep datasets (scalp EEG from humans that had performed a working memory task, intracranial human EEG, and ECoG recorded from macaques). They find that their CNN method identifies more regional and multi-area sleep spindles compared to an amplitude-based spindle detection method. Importantly, they found that the rate of occurrence of regional and multi-site spindles was higher in the sleep after human subjects performed a high-load visual working memory task, compared to after the subjects performed a low-load visual working memory task.

The results from this study advance the methods used to detect sleep events. As discussed by the authors, most algorithms to detect sleep oscillations in EEG and LFP data are based on amplitude thresholds; this approach is inadequate when studying regional and global spindles, in which the amplitude may not always remain above threshold in all recording sites. The authors overcome this limitation with a ML approach that detects significantly higher numbers of regional and multi-site spindles. The results presented here demonstrate how ML detection algorithms can offer novel insights on the function of spindles; similar approaches could be helpful with other sleep oscillations. Specifically, the results presented by Mofrad et al. suggest that, compared to amplitude-based detection methods, CNNs improve the detection of spindles that occur temporally correlated across cortical regions. In addition, the authors used the CNN spindle detection method on human sleep data after subjects performed a working memory task. The results are consistent with the hypothesis that cross-regional spindle synchronization is regulated by working memory load prior to sleep, providing insight into how specific cognitive processes influence sleep spindles.

An additional strength of this study is the demonstration that the CNN method yields similar results in ECoG, EEG, and iEEG, and in data from macaques and from humans, suggesting that the detection method is applicable across species and across electrophysiological recording methods.

A few points should be considered when interpreting the results. The definition of local, regional, and multi-site spindles used in this manuscript is based on the absolute number of electrode sites in which a spindle was detected. Since the ECoG, EEG, and iEEG have different spatial sampling, the definition could overestimate regional and multi-site spindles in iEEG recordings, particularly in the subjects that had large number of electrode contacts in one brain region. This could contribute to the much higher ratio of global spindles observed in the iEEG data compared to ECoG and EEG. Nonetheless, the analyses that support a correlation between widespread spindles and memory load were based on 64-channel scalp EEG and are not expected to be biased by differences in electrode spatial sampling.

Lastly, the results are interpreted in the context of the systems memory consolidation hypothesis, which proposes that new memories become integrated into neocortical networks in a process that is facilitated by sleep oscillations. The results from the sleep EEG analyses after subjects performed a low- or high- load visual working memory task are indeed consistent with the idea that multi-site spindles contribute to the consolidation of memories formed before the subjects went to sleep. Likewise, the results are also consistent with the synaptic homeostasis hypothesis, which states that sleep oscillations are important in scaling synaptic plasticity of networks that may have been particularly active during wakefulness. The results of this study are therefore congruent with the possibility of multi-site spindles coordinating the homeostatic regulation of synaptic strength across multiple cortical regions in relation to the cognitive load experienced by the subjects during wakefulness prior to sleep.

Recommendations for the authors

The work presented in this manuscript is highly relevant to advance the methods used to detect sleep oscillations, and to understand the functional significance of sleep spindle coordination across cortical areas. The manuscript text and figures are clear and well-organized. Below are suggestions that may help further improve the quality of this work.

The distinction of three spindle types (local, regional, multi-area) based on the co-occurrence across recording sites should be discussed with regard to the different spatial sampling of EEG, ECoG and, specially, iEEG electrodes. For example, spindles detected in multiple recording sites in subjects such as B and C which have multiple contacts in one brain region (Suppl. Table 1), might be incorrectly labelled as regional or multi-area. Although sampling biases are not expected to affect the results regarding memory, it will be helpful for readers to explain how these potential biases were addressed in the analyses of iEEG data. Could the much higher ratio of multi-area spindles in iEEG data (Figure 2c, right panel) be explained by electrode sampling bias? Along the same lines, it may be more appropriate to refer to 'multi-area' spindles as 'multi-site' spindles if they do not necessarily imply occurrence in distinct cortical areas.

It would also be helpful to expand the justification for the choice of CNN algorithms; the text refers to their previous use detecting earthquakes or gravitational waves but is not immediately clear why noise in these applications would be similar to noise in sleep signals.

Line 35: remove 'stage 2': the references provided are mostly from rodents, in which separate NREM sleep stages are less defined; and spindles are not necessarily exclusive of N2 even in humans.

Lines 61,62: Long-range connections are one potential link to temporarily coordinate spindles across cortical areas. Another important set of connections that could coordinate spindles are the thalamocortical projections, mainly those thalamic pathways with widespread projections to superficial layers of cortex (references such as Clasca et al., 2012, Eur J Neurosci 35(10):1524-32. doi: 10.1111/j.1460-9568.2012.08033.x.).

Lines 102-103: please specify how descriptive statistics are reported; are the values reported in parenthesis the mean and standard deviations? Clarify in the methods section how duration was calculated. The methods state that 'The CNN model takes a window of sleep recording (500 ms which is bandpass…)' (Line 303), is that a 500ms moving window? With what step size? What determines the start and end of a spindle in the CNN and AT methods?

Line 161: include p value for local spindles (as done in the figure legend).

Given the unique dataset and methods, and the focus of this work on the locality and globality of spindles, the authors may want to report on potential changes in the density of local spindles specifically over visual and frontal cortical areas. Is it possible that increases in local spindle density are not significant when all regions are considered together but may be significant in certain regions engaged by the visual working memory task? In that sense, it is interesting that the SNR approach detects a (smaller) increase in local spindles with high memory load (Supp. Figure 4a). A more precise analysis of local spindles over relevant cortical areas will not diminish the author's main result on multi-area spindles, but it will provide additional cues on the role of spindle spatial synchronization in sleep and will enhance the value of this interesting work.

Lines 241-242 say that 'the quality of sleep was studied by the degree of spatial synchronization'; it will be helpful to clarify this statement because if the degree of spatial synchronization was used to select the sleep data used for this work, the dataset may include more synchronized activity compared to other datasets. This does not nullify the results, but clarification will be important to ensure replicability. Likewise, the methods used for sleep detection should be described in more detail, was sleep detected in the same way in all datasets?

Line 250: only visual inspection was used to detect stage 2 in the EEG data? clarify if only NREM sleep was used in all datasets used for analysis? (same for the iEEG recordings)

Not clear what is added by the paragraph between Lines 283-288 unless more details are provided. If the model simulates spiking activity in the awake state, is this helpful as a ground truth control for sleep LFP? What type of spiking model? What network architecture?

Paragraph starting in line 290. Was the CNN model trained and optimized for each dataset?

Lines 296-297: 'best' based on what?

Line 350 states that the overall threshold for the AT method was based on the average root mean squared. Was the AT approach developed and applied independently for each data set (EEG, ECoG, iEEG)? It is not clear from the text if the different signal amplitudes in the datasets were considered for AT detection (were the data and RMS normalized?).

Figure Supp. 5: correct typo in the figure legends (spindle)

Reviewer #2:

The primary goal of this paper is to provide evidence that sleep rhythms are more spatially extended than previously known, using a deep learning convolutional neural network (CNN) model specifically designed to characterized rhythmic activity that has not previously been applied to sleep data.

Strengths

1. The authors establish that the CNN method proposed can detect spatially extended sleep rhythms and that difference occur when measuring spindles after a low vs high load memory tasks in multiple data sets.

2. The CNN method presented may provide a powerful technique to characterize a range of brain rhythms and difference across tasks or patient populations.

3. The results and figures are clearly presented.

4. The methods are sound, with the exception that more detail on the method and how it uniquely accounts for rhythmic activity is warranted and should be discussed in the Results section of the paper, rather than in the methods.

Weaknesses

1. The results of the paper do not establish that this CNN method is better than prior methods at detecting spatially extend sleep rhythms, or that the methods is better able to distinguish sleep spindles after a low vs high load memory task. As such, while a new method for detecting spindles is clearly presented, there do not appear to be any new scientific findings in this paper. The authors compare the CNN with only the AT methods for the main result of the paper (Figure 3, Supplementary Figure 3), and not the SNR method. Why not compare directly to the SNR also to see if CNN is actually better? The SNR method is used to generate the data set that the CNN is trained on, as such is it possible for the CNN to do better? Would the AT method be able to pick up more multi-area spindles with a lower threshold?

The CNN and SNR methods are directly compared only for the low vs high memory load task in Supplementary Figure 4. A visual comparison of the low vs high memory load results from the CNN (Figure 3c) and SNR methods (Supplementary Figure 4 top) suggests that the SNR method is equally able to distinguish these conditions. Overall, the advantage of the CNN is not clear.

2. There are several high-level strong claims in the paper that are not directly investigated or supported by the evidence in this paper. For example, "These results thus provide specific neural mechanisms by which memories can be stored in distributed neocortical networks during sleep". "Taken together, these results provide substantial evidence of a specific role for spindles in linking neuron groups distributed widely across cortex during memory consolidation". "The key missing piece is to understand how spindles can guide specific long-range excitatory connections to strengthen during sleep-dependent memory consolidation. We hypothesized that widespread, multi-area spindles might provide this mechansism". At best, the results in this paper provide supportive evidence that spindles could do these things by they do not investigate or establish causality in any way.

3. It is stated that in the ECoG and iEEG data, the AT method detected a subset of spindles that are significantly higher-amplitude than those detected by the CNN methods, using a one-sided Wilcoxon sign rank test. Does this mean that CNN does not detect some of the high amplitude spindles? Is this advantageous? There is something confusing about the way this is stated. A Figure of the distribution in number and amplitude of spindles detected with the 3 methods (CCN, AT, SRN) would be useful.

4. The 3 different recording methods sample activity across different spatial scales, and depth electrodes (iEEG) are sampling vastly different areas (i.e. deeper sources) than ECoG and EEG. As such, it is difficult to relate findings related to "simultaneous spindle detection" in local, regional, multi-area electrodes across these three different measures. A primary concern is that the finding that there are more multi-area spindles (e.g. Figure 2b for iEEG – similar results for EEG and ECoG are not quantified) could be due to volume spread of the spindle source. Is there a way to rule this out? There are ways to minimize the influence of volume conduction with EEG and ECoG source analysis, however, to my knowledge, these methods currently don't exist for iEEG.

Recommendations for the authors:

In Figure 1d, there appear to be other spindle in each of these example traces. Were these not picked up by the algorithm, or simple not highlighted in red? Clarification would be helpful.

Why are the randomly sampled red dashed line in Supplementary Figure 1 not flat? Is the 1Hz filter somehow biasing the amplitude at the center?

It would be helpful to see the results shown in Figure 2b for iEEG for ECoG and EEG data as well.

There is are claims about SNR and CNN being independent of spindle amplitude. Clarification of how the SNR power calculations are independent of amplitude would be useful.

The methods state "We first tested CNN models with different architectures and selected one of the best architectures across sleep recording data sets". What does "one of the best" mean? Quantification would be helpful. Overall, clarification of advantages of this CNN method in identifying rhythmic activity, other than training on rhythmic activity (?), would be helpful.

[Editors’ note: further revisions were suggested prior to acceptance, as described below.]

Thank you for resubmitting your work entitled "Waveform detection by deep learning reveals multi-area spindles that are selectively modulated by memory load" for further consideration by eLife. Your revised article has been evaluated by Timothy Behrens (Senior Editor) and a Reviewing Editor.

The manuscript has been improved but there are some remaining issues that need to be addressed, as outlined below:

We thank the authors for carefully considering the feedback they received during the first round of reviews, and for performing additional analyses and modifying their manuscript with the feedback in mind. We also thank the authors for patiently waiting for these reviews. The reviewers found the new version of the manuscript a substantial improvement compared to the first submission. The reviewers feel that additional changes and clarifications can put the authors' contribution in its proper context, and broaden the reach of their contribution in the community. The reviewers think the authors ought to

1. Make their comparisons of the CNN and AT methods in the presence of noise more realistic than currently. Experimentalists would care, for instance, about robustness of the CNN, compared to the AT method, to biological forms of noise (e.g. confounding oscillations such as a brief period of REM theta synched across channels, does the CNN offer an advantage to the AT in that case?). Moreover, the authors seem to have made the comparisons with a fixed noise level/SNR (not to confuse with the 'SNR' method). Does the choice of noise amount have a biological basis? The authors will find text in the reviews unpacking this comment

2. Clarify apparent inconsistencies in certain statistics reported (e.g. spindle rates), as well as conduct statistical tests for some of their data more appropriate than ones currently used. The authors will find text in the reviews unpacking this comment

3. Tone down claims of generality of the CNN, and provide clear explanations for what makes the choice of architecture suitable to the current application to spindle detection. The reviewing editor finds this the most important piece of feedback from the reviews. As written, the reviewing editor feels that the manuscript may promote a culture of using deep learning in neuroscience without understanding how deep learning works. The authors have a unique opportunity of not only presenting an application of deep learning that leads to new scientific findings/hypotheses but also of doing so in a manner that promotes a culture of looking inside the black box and trying to understand it. The authors will find text in the reviews unpacking this comment

Reviewer #1:

The manuscript has improved substantially and many of my concerns have been addressed. The deep-learning spindle detection method is more fully reported and the new controls with simulated data (varying noise and amplitude) are an improved approach to compare the CNN and AT methods (Figure S1, new Supplementary Table 1). The new analyses investigating the rotating spindle waves provide a framework for understanding the differences with prior reports of low synchrony.

However, there are several important concerns that should be addressed.

The new comparisons between CNN and AT models using surrogate data with systematically varied noise are useful. However, what was the rationale to compare the specific types of noise presented in Supplementary Table 1? Both the CNN and the AT have similar sensitivities and specificities for all noises, which may suggest that the different noise conditions do not constitute substantially different challenges for the detectors (also suggested by the example in Supplementary Figure 1, in which the spindle has a high signal to noise ratio with all types of noise). I think the readers will be more convinced of the value of the CNN method it can offer an advantage under a 'noise' condition in which the traditional AT methods are likely to struggle. For example, a more interesting source of noise may be the presence of non-spindle oscillations due to brief state changes (e.g., REM or α), or noise artifacts (e.g., examples in Supplementary Figure 12). The sensitivity of traditional methods is likely to go down in examples like these, how does the CNN compare?

Line 92: "co-occurrence"; please clarify the time window or time overlap used to determine co-occurrence. Did any amount of time overlap between spindles count as co-occurrence? Likewise, in Line 150: "simultaneously detected spindles", line 151: "based on co-occurrence". I think it's worth emphasizing in this paragraph that 'co-occurrence' and 'simultaneous' detection do not imply 0-lag synchrony (as discussed in the 'rotating waves' sections).

Lines 102 and 104: "low and high visual memory task" should be "low and high-load…" (also in the Figure 3 legend, and in lines 261-262, 263: 'high-load' visual memory tasks).

There are some inconsistencies in the reported rates (spindles/min) across analyses and figures: Suppl. Figure 4a indicates average spindles/min around 4-5/min in all data types, including EEG data used in the visual memory task. However, the average from the distributions shown in Figure 3b seems much lower, even when combining all spindles across each memory condition, e.g. for the H-VM is about 1/min in local and regional and about 0.5/min for multi-area. In other words, could you clarify how the averages of the distributions in Figure 3b add up to 4 spindles/min in Figure 4a? In addition, in Figure 3b, within the low-VM condition the rate of multi-area spindles is substantially lower (< 0.5/min) than local and regional (about 1/min on average); and within high-VM the rate also appears much lower (<1/min) for multi-area compared to the distributions for local and regional spindles (up to ~2/min). The text indicates p-values for what seems to be a comparison of relative rates (between H-VM and L-VM), which is interesting but is this result due to a significantly lower multi-area spindle rate with low memory? Based on the figure it looks like multi-area spindles occur at lower rates regardless of memory load? It will be helpful to provide statistical comparisons of spindle rates (among all spindle types) within each condition (L-VM, H-VM) so that the relative changes with memory can be interpreted in the context of the absolute spindle rates in each memory condition. Based on the distributions it seems that non-parametric tests would be more appropriate than t-tests.

Suppl. Figure 4: missing (a),(b) labels. Supplementary Figure 4a is not described in the text?

Line 725: 12? sites

Line 154: it'd be useful to reference the Supplementary Table 2 (with the list of the cortical regions).

Lines 170, 171: I believe the verb should be in past tense.

Line 179: The interpretations regarding the association between the visual memory load and subsequent sleep spindles is limited to memory consolidation processes. However, memory performance after sleep was not studied in this work, therefore "the impact on memory consolidation" cannot directly be assessed. Sentences such as these (lines 179-180): "If this is the case, what is the impact of the distribution on sleep-dependent memory consolidation? To answer this question…" could be rephrased to state the key question that the data can address. Other sentences in the manuscript are more accurate and valuable in this respect, e.g., Lines 289-290 "(2) this spatial extent can be modulated by the specific memory conditions prior to sleep (Figure 3)". Indeed, memory consolidation is one of several mechanisms that determine how wakefulness influences sleep oscillations. The authors correctly cover literature on the association between memory and spindles "An increase in spindle density after memory tasks and its relationship with memory consolidation is well established (Clemens et al., 2005; Dang-Vu et al., 2008; Gais et al., 2002; Schabus et al., 2007, 2004)", but the impact of the authors' findings would be enhanced by discussing other hypotheses in the sleep field that are also consistent with the presented results; for example, the high-load working memory condition may increase firing rates and entrain downscaling processes during subsequent sleep (Tononi and Cirelli, 2014; Crunelli et al., 2018; Klinzing et al.,2019).

Paragraph starting 215: is this only with the EEG data?

Figure 4a: missing (a),(b) labels for the plots. Indicate the x,y value of the removed outlier point in parenthesis (in 'a' and 'c') since it's not in the plot but still used in quantifications.

Line 552: "We simulated 60-minutes recording containing" should it be "60-minutes of recording…"?

Reviewer #2:

The authors suggest that the lack of algorithms for reliably identifying spindles in neural recordings has led to the under reporting/underestimation of the spatial extent to which spindles occur in the brain. The authors propose an approach based on CNNs that they claim can detect spindles more reliably than existing ones, lead to new insights as to how cross-region spindles may contribute to the integration of 'information' across brain areas.

Strengths

1. The applicability of the proposed methods to neural data from multiple modalities, i.e. EEG, EcOG and iEEG.

2. The framework the authors lay out for constructing high-quality data sets to train the CNNs.

3. The combination of computational methods and new suggested insights into how spindles can facilitate the integration of 'information' across brain areas.

Weaknesses

1. The authors could improve the explanations as to why the CNN seems to do better than legacy methods such as the AT. The explanation surrounding the CNN's ability to detect spindles of different amplitudes does not seem satisfactory enough.

2. The authors claim the applicability of the CNN to data from different modalities, and its generality, as a strength. Given the black-box nature of CNNs, and the lack of an attempt in the manuscript to explain the CNN, its success/failure modes, which aspects of the data it focuses on to detect spindles (e.g. saliency maps), these claims do not feel justified to the reviewer and may contribute to the proliferation of black-box CNNs in neuroscience

Recommendations for the authors:

The reviewing editor found the manuscript a much improved version of the initial submission.

1. The reviewing editor found that the analyses currently in the manuscript do not support the authors' claims of generality of the CNN (e.g. lines 376-377), and that the language surrounding these claims may contribute to supporting an already-widespread tendency to utilize black-box neural networks for analyzing neural data.

a) One feedback from the first round of reviews had to do with the lack of details surrounding the CNN. The reviewing editor appreciates the authors' attempt to improve this. The authors seem to want to emphasize the generality of the architecture. Such claims of generality do not give insight to the reader on how to pick an architecture for detecting oscillatory patterns in a different context (e.g. ripples). The success of the architecture for gravitational wave detection, its use in the current manuscript, do not give license to use it w/o changes in any application.

A practitioner would appreciate guidance on architecture design. Given the length of pattern of interest and a sampling rate, a user would want to know how to pick filter sizes. For instance, for a patter of length 0.5 seconds, a 256 Hz sampling rate, the effective filter (explained below) associated with an architecture ought to have size on order of 100 samples (0.5*256 = 128 samples), not 10 or 1000. The reviewing editor thinks this manuscript could have a much stronger impact if the authors took such questions into considerations. Have authors visualized the filters they learn? Have they considered generating saliency maps to determine which parts of inputs the network focuses on to detect spindles? At present, the emphasis on the generality of the CNN, w/o considerations for what about the authors' specific context makes the choice of architecture suitable (other than the fact that it works for gravitational-way detection) feels a bit disappointing.

More details on guidance on how to design architecture/why current architecture works: For any conv net, the choice of filter size at each layers, together with number of layers relates to size of features one would like to detect. Each layer a conv. Composition of conv equals a conv with a filter size roughly proportional to the sum of filter sizes at each layers. 7x7 filters not uncommon in image proc. Cascading 3-4 such layers (ignoring nonlinearities) gives a filter with an effective 30x30 size.

Concrete suggestions: (a) adding text to the manuscript explaining what what about the authors' specific context makes the choice of architecture suitable (other than the fact that it works for gravitational-way detection), (b) visualizing filters learned by the architecture, (c) consider generating saliency maps.

(b) The reviewing editor feels that the authors can improve the section detailing the training of the CNN. The editor suggests the authors consider using a table to summarize different aspects (number of layers, filter sizes etc….). The editor also suggests that, very early on, the authors mention the subject-dependence of the training.

2. Misc questions

(a) In supplementary figure 4b, the scatter plot for EEG (green) suggests a similar max PSD for the CNN and AT. I would expect a similar rate of spindle detection. Supplementary figure 4a suggests otherwise. Can the authors explain?

(b) In Figure 2c, EcOG the fact that the CNN detects fewer local and regional spindles per minute than the AT method requires explanation. Does this figure refer to correctly detected spindles? Should the editor interpret the lower ratio as the AT method having more false alarms than the CNN?

https://doi.org/10.7554/eLife.75769.sa1

Author response

[Editors’ note: the authors resubmitted a revised version of the paper for consideration. What follows is the authors’ response to the first round of review.]

Overall, as a group, the reviewers found the application of CNNs to the detection of spindles novel and interesting. More generally, the reviewers thought that a method to detect rhythmic behavior from noisy data could have an important impact on neuroscience research. At the same time, the reviewer expressed some reservations, and felt that the manuscript requires additional work to assess the utility of the CNNs as an addition to sleep biologists' toolkits. Should the authors decide to submit a revised version of the manuscript, we will do our best to have it assessed by the reviewing editor and the reviewers who provided the comments below.

We would like to thank the Editor and both Referees for their thoughtful consideration of the merits and advances in our manuscript. We appreciate that both Referees recognized the novelty and importance of the scientific results; however, they also provided important critiques and comparisons to strengthen this work. We have taken their comments to heart, and in the revised manuscript, we have worked to address them in full.

In summary, we have worked to more fully document the deep-learning waveform detection method we introduce in this work, including more detailed comparisons to previous methods. We have added a new section with simulations using surrogate data and systematically varying noise to demonstrate that our deep learning approach is robust to different types of noise on the recording electrode (new Figure S1). These results indicate that our CNN approach can achieve a very high performance when trained under varying noise and recording conditions (new Table S2). In addition, in our revised manuscript we have gone back to the data from the visual memory task and discovered a novel finding: rotating spindle waves significantly increase following high-load visual memory tasks (new Figure 4 and Video S1). These new results specifically tie the increase in multi-area spindles we reported in the original submission to a neural circuit mechanism for rotating waves in consolidating memories across distributed networks in cortex (see Figure 2, Muller et al., eLife 5, 2016).

We are very grateful to the Editor and Referees for the points raised in the first round of review, and we believe our work to address them has substantially strengthened the manuscript.

The following represent three main critiques/suggestions from the reviews.

1. The reviewers found the comparisons to existing methods for spindle detection too simplistic and lacking. More specifically, the authors compare their method mainly to the AT method. The reviewers found the lack of details of the authors' implementation of the AT method, and the lack of thorough comparison to spindle-detection algorithms more sophisticated than the AT method, such as the SNR method, surprising. The reviewers suggest that the authors describe their implementation of the AT method in more detail (e.g. threshold selection etc…). The reviewers also suggest that the authors perform thorough comparisons of the CNN to methods/algorithms more sophisticated than the AT method, both in simulations and on the data used in the manuscript. Evidence that the CNN detects substantially more spindles than these methods would motivate its adoption by biologists.

We appreciate the opportunity to discuss this point. In this work, we focused on the standard implementation of the amplitude-thresholding approach implemented by Mölle et al. (J Neurosci 22, 2002). As detailed in a comprehensive analysis by Warby et al. (Nature Methods 11, 2014), who performed an exhaustive comparison of six commonly utilized sleep spindle detection algorithms (Bódizs et al., J Neurosci. Methods 178, 2009; Ferrarelli et al., AM. J Psychiatry 164, 2007; Mölle et al., J Neurosci 22, 2002; Martin et al., Neurobiol. Aging 34, 2013; Wamsley et al., Biol. Psychiatry 71, 2012; Wendt et al., IEEE Eng. Med. Biol. Soc. 2012, 2012), these approaches are methodologically similar, relying on bandpass filtering (or wavelets) followed by a thresholding procedure. In this work, we aimed to focus on the signal processing principle common to these approaches: across choices of specific frequency filtering and thresholding procedures, problems can arise from (1) intrusion from signals outside the frequency band of interest (due to bandpass filtering, cf. Figure S12) and (2) missing clearly formed spindle waveforms that fall under the defined threshold. For this reason, we believe that our manuscript presents not only the most pertinent comparison between our CNN approach and the AT method, but also identifies the underlying reason for the new results uncovered by our algorithm. With this in mind, we have clarified in our revised manuscript the general signal processing framework underlying this work: when identifying spindles based on their amplitude characteristics, rather than on waveform characteristics (as with our deep learning approach), the limitations on detecting events in high-noise situations will be general across methods.

We also appreciate the opportunity to clarify our two-step approach for training the CNN algorithm to detect spindles. Importantly, we introduced the SNR algorithm in a previous manuscript (Muller et al., eLife 5, 2016) as a conservative approach to detect spindles while maintaining a low false-positive rate. This method, which is based on the constant false-alarm rate (CFAR) method in radar (Levanon, N., Wiley, New York, 1988; Minkler, G. and Minkler, J., Magellan, 1990), detects many “true” spindles while minimizing false detections; however, its aim is not to detect all spindles. For this reason, while the SNR approach is excellent for generating a high-quality training dataset and for providing a second check on results from the CNN model, we know that it cannot provide a comprehensive detection method approaching all of the spindles in a recording. In the initial submission, we thus introduced the CNN approach, which detects many more well-formed and high-quality spindles than the AT approach (Figures 2, S3, and S10) and used the CNN approach to reveal multi-area spindles that are selectively modulated by memory load (Figure 3).

We apologize that the implementation of the AT was not clear in the original manuscript. In the revised version, we provide details of implementation including steps taken to compute amplitude using signal envelope over a sliding window of time, setting the threshold based on the existing work in the literature (approximately 2 spindles per minute), and specifying spindle start and end time from the amplitude crossings. We also note that the CNN and AT approach were implemented in as similar a manner as possible to facilitate comparison.

Our revised manuscript provides specific evidence that the CNN detects substantially more spindles than AT. On a set of surrogate data generated to test the robustness of the CNN to different types of noise, the CNN model was able to detect at least 98% of spindle activities, with a specificity above 99%. In contrast, even though the surrogate data does not have any artifact such as line noise, electrical noise, and movement artifacts to introduce signal distortion, the AT achieved lower performance (specificity around 84%-87%). The difference between these methods is even more pronounced in the actual recording, where our comparison reveals that multi-area spindles systematically occur 1.5 (ECoG) to 10 (iEEG) times more often with the CNN than with the AT. Taking these results together, we believe we have provided substantial evidence that the CNN model we have introduced can accurately detect spindles across a range of sleep recording types and provide better performance than the commonly used AT technique.

We appreciate the Referees bringing up these three important points, which we have addressed in our revised manuscript: (1) explaining more clearly how our method compares to other approaches for sleep spindle detection, such as the SNR approach (lines 110-117), (2) clarifying our implementation of the AT approach (Methods – Comparison with amplitude threshold approach), and (3) providing specific evidence that the CNN detects spindles more effectively than the AT approach using surrogate data (Lines 82-86 in the main text, new supplementary section “Methods – Simulated data control – varying noise” and new Figure S1 and Table S1).

We hope these additions fully address the Referees’ concerns.

2. The reviewers felt that the analyses do not support the authors' interpretations and their claims of new scientific findings. First, differences in spatial sampling of neural activity by EEG, EcOG and iEEG, together with the fact that volume conduction may contribute to synchronous activity, limits the interpretation of the authors result. Second, a line of work by Dehghani, Cash, Halgren and colleagues investigating the spatial extent of spindles suggest a lack of synchrony: e.g. (1) doi: 10.1152/jn.00198.2010, (2) doi: 10.1002/hbm.21183, (3) doi: 10.1016/j.clinph.2010.06.018. The reviewers suggest that the authors temper their claims significantly, and put them in the context of the above literature.

We thank the Referees for bringing up these important points, which we have considered very seriously in the process of revision. In our revised manuscript, we have added a new analysis of the spatiotemporal structure in the sleep spindles we detect. In addition to providing novel insight into the lack of synchrony in sleep spindles, this analysis addresses the technical questions raised in this point.

To explain in more detail, we have now added an analysis of spatiotemporal dynamics in the sleep spindles, extending methods developed in our previous work (Muller et al., eLife 5, 2016). For this analysis, we focused on the EEG dataset, with the aim of comparing spatiotemporal dynamics following low- and high-load visual memory tasks. While EEG certainly has reduced spatial resolution compared with the intracranial ECoG and iEEG also studied here, we find that we can robustly detect rotating waves recorded at the scalp (new Figure 4 and Video S1). These waves travel within approximately the same speed as detected in our previous work, which is consistent with the axonal conduction speed of long-range fibers in cortex. Importantly, the observation of clear rotating waves in the scalp EEG directly addresses concerns about volume conduction in the multi-area spindles observed here: the spindles we analyze exhibit tens of milliseconds of offset during individual cycles, and this offset is inconsistent with the effect of volume conduction, which must necessarily be instantaneous. For this reason, it would be difficult for volume conduction effects to be a major confound for our present results on multi-area spindles.

These points open up to a broader and important discussion on synchrony in spindles. We have previously collaborated closely with Dehghani, Cash, and Halgren, and we appreciate the opportunity to discuss the results from analysis of MEG and EEG, which found that correlation measures across sites during spindles was low. Recent work from Birgit Frauscher’s lab studying iEEG depth electrodes has also found similar results (Frauscher et al., NeuroImage 105, 2015), notably showing that sites across cortex exhibit broad increases in 11-15 Hz power during spindles detected at the scalp, but exhibit low correlation across electrode sites. In fact, because traveling waves introduce systematic phase offsets across sites, our work can provide an underlying mechanism for these results, accounting for the broad increases of spindle power across cortex (due to multi-area spindles) with low measures of synchrony (due to phase offsets introduced by the traveling wave). Further, with this in mind, it is critical to distinguish measures of synchrony (as in the work of Dehghani, Cash, and Halgren) from the extent of spindle occurrence (as addressed in our work). We have added citations and additional discussion in the main text to highlight and address these points (lines 273-284).

Going further, we can ask about the functional role of these waves. In our previous work, we introduced a specific circuit mechanism for how rotating waves during spindles could facilitate building connections between distributed networks in cortex (cf. Figure 2 in Muller et al., eLife 5, 2016). This mechanism addresses an important point about the connection between sleep spindles and circuit plasticity: if cortical spindles were perfectly synchronized, spikes emitted during one cycle of the spindle oscillation would arrive at their post-synaptic targets with the time delay due to axonal conduction (tens of milliseconds in cortex) (Author response image 1A, left), leading always to a pairing within the window for long-term depression (LTD) that would progressively weaken long-range connections (Author response image 1A, right). If, however, spindles are self-organized into large-scale wave-like activity patterns, with phase speeds matching those of the underlying fiber networks and stereotyped, precisely repeating trajectories (Author response image 1B, left), then excitatory postsynaptic potentials (EPSPs) caused by spikes traveling along pyramidal axons to distant regions in the cortex could align with the local burst of population activity (Author response image 1B, right), creating the conditions necessary for both synaptic strengthening and weakening to occur.

Author response image 1
Schematic of spindles and axonal delays.

(A) Spikes emitted from region A will arrive at B with a temporal delay of 20 milliseconds (left). If spindle oscillations were perfectly synchronized across the cortex, EPSPs from region A would occur after the spikes in region B, within the window for long-term depression (right). (B) In contrast, if spindles are spatiotemporally organized with stereotyped trajectories (left), then EPSPs from region A would align with population spiking in region B, allowing for synaptic strengthening to occur. Adapted from Muller et al., eLife 5, 2016.

This mechanism for building distributed circuits in cortex opens the possibility that pairings between specific sites during spindles, mediated by spatiotemporal organization into traveling waves, can be modulated by memory load. In our revised manuscript, we have compared rotating waves following low and high visual memory tasks in the EEG dataset. Surprisingly, we find that rotating waves in the temporal -> frontal -> parietal (TFP) direction are significantly increased during high visual memory tasks (new Figure 4). This result, that rotating waves dynamically change under different memory conditions, provides an additional corroboration that our results are independent of confounds from volume conduction, as the only difference between the two EEG recordings from the same individual and recording setup was memory load. Further, these new results provide intriguing evidence that spatiotemporal dynamics in spindles may adapt to specific memory content during sleep-dependent consolidation. In this case, the significant increase in TFP waves may implicate a set of top-down circuits implicated by the specific memory task; in upcoming work, we aim to understand more fully how the spatiotemporal structure of sleep rhythms changes with memory content during sleep.

We hope that these additional details address concerns about the connection between the observations on sleep spindles in the present manuscript and mechanisms of human memory consolidation. We have added additional details and discussion on this point in the main text (lines 62-71 and 215-271).

Finally, while we consider several very different recording modalities in this work, it is important to emphasize that we do not draw comparisons across different electrode types. For example, we do not compare the rate of multi-area spindles in ECoG with those in iEEG. Rather, we focus on the result which clearly comes out consistently from our analysis: AT-type techniques systematically underestimate the number of multi-area spindles across all types of recordings. The main result of our paper, which is that the CNN model reveals multi-area spindles occur much more often than previously thought, is thus not affected by differences in spatial sampling of the electrodes (Figure 2c and S6b). We apologize that the explanation was not clear in the original manuscript and we have added additional clarification on this point (lines 95-102 and 296-300). We hope these additional clarifications fully address the Referees’ concerns on this point.

3. The reviewers thought the manuscript could benefit from an expanded discussion of the motivation for utilizing the CNN for spindle detection, in particular of how, if at all, they tailored the architecture to the spindle-detection setting. The description of the architecture (layers etc…) provided in the manuscript seems to mirror the description of that developed for gravitational wave detection. More generally, the reviewers thought that a discussion of how to tailor the architecture to the detection of brain rhythms other than spindles would benefit the neuroscience community. For instance, variables such as the length of the rhythmic pattern of interest ought to impact architectural choices.

We thank the Editor and Referees for bringing up this important point. Our work indeed is motivated by the utilization of CNN models in high-noise settings for waveform detection such as earthquakes (Perol et al., 2018) and gravitational waves (George and Huerta, 2018). A range of CNN architectures was proposed in these studies, with 1D to 2D channels as input layers, and convolutional layers ranging from 3 to 8 convolutional layers. We implement an architecture similar to the one proposed by George and Huerta (2018) with small modifications to the input and convolutional layers to take into account the basic features of the spindle rhythm in cortex (e.g. average duration). The CNN architecture is also slightly tailored to different sampling rates in each recording modality. As in previous work, the convolutional layer is designed to start by extracting local features, gradually extracting longer-timescale features by decreasing the feature space. Using this strategy, the CNN model can efficiently learn to detect the specific waveform characteristics of the sleep spindle in different types of recordings. In addition, we verified that the proposed CNN model is not sensitive to the slight change in the number of layers (e.g. 4, 5, and 6 convolutional layers) and hyperparameters such as learning rate, maximum number of epochs used for training, and pooling parameters. Details of these analyses are added in lines 445-466 and 475-488 in the revised manuscript.

We do agree that our two-step combination of the SNR algorithm (which generates a high-quality training dataset) and CNN model (which detects a more complete set of clear, well-formed spindles ranging from large- to small-amplitude events) can be applied more generally to detect brain rhythms other than spindles. We appreciate the Editor and Referees highlighting this potential contribution of our work, and we have added this point to the concluding paragraphs of the revised manuscript (lines 292-300).

Reviewer #1:

Mofrad et al. trained a convolutional neural network (CNN) to improve the detection of multi-area spindles and study their regulation by working memory load. They used the CNN algorithm on three different sleep datasets (scalp EEG from humans that had performed a working memory task, intracranial human EEG, and ECoG recorded from macaques). They find that their CNN method identifies more regional and multi-area sleep spindles compared to an amplitude-based spindle detection method. Importantly, they found that the rate of occurrence of regional and multi-site spindles was higher in the sleep after human subjects performed a high-load visual working memory task, compared to after the subjects performed a low-load visual working memory task.

The results from this study advance the methods used to detect sleep events. As discussed by the authors, most algorithms to detect sleep oscillations in EEG and LFP data are based on amplitude thresholds; this approach is inadequate when studying regional and global spindles, in which the amplitude may not always remain above threshold in all recording sites. The authors overcome this limitation with a ML approach that detects significantly higher numbers of regional and multi-site spindles. The results presented here demonstrate how ML detection algorithms can offer novel insights on the function of spindles; similar approaches could be helpful with other sleep oscillations. Specifically, the results presented by Mofrad et al. suggest that, compared to amplitude-based detection methods, CNNs improve the detection of spindles that occur temporally correlated across cortical regions. In addition, the authors used the CNN spindle detection method on human sleep data after subjects performed a working memory task. The results are consistent with the hypothesis that cross-regional spindle synchronization is regulated by working memory load prior to sleep, providing insight into how specific cognitive processes influence sleep spindles.

An additional strength of this study is the demonstration that the CNN method yields similar results in ECoG, EEG, and iEEG, and in data from macaques and from humans, suggesting that the detection method is applicable across species and across electrophysiological recording methods.

A few points should be considered when interpreting the results. The definition of local, regional, and multi-site spindles used in this manuscript is based on the absolute number of electrode sites in which a spindle was detected. Since the ECoG, EEG, and iEEG have different spatial sampling, the definition could overestimate regional and multi-site spindles in iEEG recordings, particularly in the subjects that had large number of electrode contacts in one brain region. This could contribute to the much higher ratio of global spindles observed in the iEEG data compared to ECoG and EEG. Nonetheless, the analyses that support a correlation between widespread spindles and memory load were based on 64-channel scalp EEG and are not expected to be biased by differences in electrode spatial sampling.

Lastly, the results are interpreted in the context of the systems memory consolidation hypothesis, which proposes that new memories become integrated into neocortical networks in a process that is facilitated by sleep oscillations. The results from the sleep EEG analyses after subjects performed a low- or high- load visual working memory task are indeed consistent with the idea that multi-site spindles contribute to the consolidation of memories formed before the subjects went to sleep. Likewise, the results are also consistent with the synaptic homeostasis hypothesis, which states that sleep oscillations are important in scaling synaptic plasticity of networks that may have been particularly active during wakefulness. The results of this study are therefore congruent with the possibility of multi-site spindles coordinating the homeostatic regulation of synaptic strength across multiple cortical regions in relation to the cognitive load experienced by the subjects during wakefulness prior to sleep.

We thank the Reviewer for the thoughtful and thorough review, and for highlighting the novelty and importance of the scientific results presented in this work. In our revised manuscript, we have included additional details in order to more clearly document the methods and results presented here. In particular, we have addressed concerns about the difference in spatial sampling across the recordings, added new analysis of the spatiotemporal structure in the sleep spindles we detect, and provided a clear explanation of the connection between our observations on sleep spindles in the present manuscript and mechanisms of human memory consolidation. We hope this additional information and analysis have expanded and substantially strengthened the manuscript.

Recommendations for the authors

The work presented in this manuscript is highly relevant to advance the methods used to detect sleep oscillations, and to understand the functional significance of sleep spindle coordination across cortical areas. The manuscript text and figures are clear and well-organized. Below are suggestions that may help further improve the quality of this work.

We thank the Reviewer for the positive comments on our submission, including highlighting the importance of developing an accurate, automatic sleep spindle detection algorithm. We also appreciate their helpful comments to improve the paper. In this revised manuscript, we have introduced updates to address all comments in full.

The distinction of three spindle types (local, regional, multi-area) based on the co-occurrence across recording sites should be discussed with regard to the different spatial sampling of EEG, ECoG and, specially, iEEG electrodes. For example, spindles detected in multiple recording sites in subjects such as B and C which have multiple contacts in one brain region (Suppl. Table 1), might be incorrectly labelled as regional or multi-area. Although sampling biases are not expected to affect the results regarding memory, it will be helpful for readers to explain how these potential biases were addressed in the analyses of iEEG data. Could the much higher ratio of multi-area spindles in iEEG data (Figure 2c, right panel) be explained by electrode sampling bias? Along the same lines, it may be more appropriate to refer to 'multi-area' spindles as 'multi-site' spindles if they do not necessarily imply occurrence in distinct cortical areas.

We appreciate the opportunity to discuss this important point. While we consider several very different recording modalities in this work, it is important to emphasize that we do not draw comparisons across different electrode types. For example, we do not compare the rate of multi-area spindles in ECoG with those in iEEG. Rather, we focus on the result which clearly comes out consistently from our analysis: AT-type techniques systematically underestimate the number of multi-area spindles across all types of recordings. The main result of our paper, which is that the CNN model reveals multi-area spindles occur much more often than previously thought, is thus not affected by differences in spatial sampling of the electrodes (Figure 2c and S6b). We apologize that the explanation was not clear in the original manuscript and we have added additional clarification on this point on lines 95-102 and 296-300 in the main text.

In our original submission, we took the multi-area point very seriously, and by identifying the cortical regions for each iEEG contact, we quantified the number of unique cortical regions participating in each spindle (Figure 2b). This analysis demonstrated that spindles classified as “multi-site” by our CNN algorithm do in fact occur across multiple cortical areas (60% of recorded regions, bar plot, Figure 2b), corroborating the “multi-area” designation in our analysis. In our revised manuscript, we have realized the importance of this critical point, and we have introduced updates to highlight it more clearly in the main text (lines 153-159 and 320-321).

In addition, we have expanded on this analysis with new supplementary figures to further support this point. Our new Figure S6a demonstrates that the increase in the percentage of parcellated cortical regions participating in a spindle is significant at the individual level in all five iEEG subjects analyzed in this work. Our new Figure S7 shows these increases remain significant if we consider spindle participation at the level of cortical lobes (frontal, temporal, parietal, occipital) and at the level of cortical systems (executive, limbic, visual, auditory, somatosensory). Both of these analyses provide further evidence to support that spindles detected by the CNN model are distributed widely across multiple brain areas, lobes, and systems, providing insight into the role spindles play in memory consolidation. Finally, at all points results from the CNN model are compared with the AT approach, which tends to detect only local spindles, consistent with our proposition that thresholding-type algorithms may over-emphasize local events (lines 170-173).

It would also be helpful to expand the justification for the choice of CNN algorithms; the text refers to their previous use detecting earthquakes or gravitational waves but is not immediately clear why noise in these applications would be similar to noise in sleep signals.

We thank the reviewer for raising this point, which we have addressed in the main text. Previous works on detecting earthquakes and gravitational waves have demonstrated the potential and efficacy of CNN models for waveform detection in very high-noise settings. Importantly, these methods are relatively general to the type of noise in each recording, provided there is enough training data and high-quality marked events. In this case, the CNN can efficiently learn to detect the specific waveform characteristics distinguishing the sleep spindle rhythm in these recordings. We have added additional clarification on this point on lines 82-86 in the main text.

In addition, to add further support to this point, we have added a new supplementary figure and table in the revised manuscript. By training the CNN model on surrogate data with systematically varying noise properties, we demonstrate that this approach is robust to different types of noise on the recording electrode (Methods – Simulated data control – varying noise and new Table S1 and Figure S1), showing that we can expect this approach to generalize well across recordings with different types of noise. This analysis further supports our validations in the original submission, where we showed that detected spindles exhibit a clear power increase in the expected range (Figure 1b), detected spindles are well-formed in individual examples (Figure 1d), and averaging across detected spindles without bandpass filtering still results in the expected waveform (Figure S3). These additional analyses provide further evidence that the AT has lower detection power even in the surrogate data with no artifact such as line noise, electrical noise, and movement artifacts that can introduce signal distortion in experimentally recorded signals.

Taking these results together, we believe we have provided substantial evidence that the CNN model we have introduced can accurately detect spindles across a range of sleep recording types. We have included text to more clearly highlight the utility and potential applications of this resource in future work (lines 296-300).

Line 35: remove 'stage 2': the references provided are mostly from rodents, in which separate NREM sleep stages are less defined; and spindles are not necessarily exclusive of N2 even in humans.

Thank you for raising this point. We have removed stage 2 (line 39) in the revised manuscript.

Lines 61,62: Long-range connections are one potential link to temporarily coordinate spindles across cortical areas. Another important set of connections that could coordinate spindles are the thalamocortical projections, mainly those thalamic pathways with widespread projections to superficial layers of cortex (references such as Clasca et al., 2012, Eur J Neurosci 35(10):1524-32. doi: 10.1111/j.1460-9568.2012.08033.x.).

We thank the Referee for bringing this important point to our attention. We have added the thalamocortical connections as a potential pathway for coordination of spindles in the revised manuscript (lines 246-254).

Lines 102-103: please specify how descriptive statistics are reported; are the values reported in parenthesis the mean and standard deviations? Clarify in the methods section how duration was calculated.

We thank the Reviewer for this and apologize that it was not clear in the original submission: the descriptive statistics here are average ± SEM (line 124). We have added detailed explanations to clarify the calculation of spindle duration in the supplementary section “Methods Convolutional neural network (CNN) for sleep spindle detection” (lines 480-488). We have also clarified the calculation of spindle duration, which is included in our response to the next comment.

The methods state that 'The CNN model takes a window of sleep recording (500 ms which is bandpass…)' (Line 303), is that a 500ms moving window? With what step size? What determines the start and end of a spindle in the CNN and AT methods?

Thank you for bringing up this important point. We have added a detailed explanation of the entire process in the supplementary section “Methods – Convolutional neural network (CNN) for sleep spindle detection” (lines 480-488) and “Methods – Comparison with amplitude-threshold approach” (lines 531-537).

Briefly, the CNN model takes a sliding window of sleep recording (500 ms, signal bandpassed 1-100 Hz) as an input and determines its label (spindle or non-spindle). The sliding window starts at the beginning of the recording and moves 100 ms in each step. To find the start and end time of a spindle, we first combined all overlapping spindle windows. We then included neighbor windows if there is any spindle within 100 ms of the combined windows to account for potentially mislabeled windows. The start of a spindle is finally set to the beginning of the first window, and the end of the spindle is set to the end of the last window.

To create a clear comparison, we implement precisely the same procedure for the AT approach. To do this, we compute the signal envelope over a sliding window of 500 ms with 100 ms step size. We then compare signal envelope for each window to the AT threshold. Finally, we combine windows as with the CNN approach to get the start and end time of each spindle.

Line 161: include p value for local spindles (as done in the figure legend).

We appreciate this suggestion, and we have added the local spindle p-value to the main text

(lines 198-199).

Given the unique dataset and methods, and the focus of this work on the locality and globality of spindles, the authors may want to report on potential changes in the density of local spindles specifically over visual and frontal cortical areas. Is it possible that increases in local spindle density are not significant when all regions are considered together but may be significant in certain regions engaged by the visual working memory task? In that sense, it is interesting that the SNR approach detects a (smaller) increase in local spindles with high memory load (Supp. Figure 4a). A more precise analysis of local spindles over relevant cortical areas will not diminish the author's main result on multi-area spindles, but it will provide additional cues on the role of spindle spatial synchronization in sleep and will enhance the value of this interesting work.

We thank the Referee for suggesting this new analysis which provides additional insight on the role of spatial synchronization of spindles during sleep. Following this suggestion, we studied the change in density of spindles in frontal, occipital and parietal lobes (new Figure S9) in low and high visual memory conditions. To do this, in each cortical lobe, we computed the percentage of electrode sites with spindles within the detected windows by the CNN model. Interestingly, we observe a significant increase in the percentage of electrode sites in high versus low visual memory conditions across all cortical lobes. On average the occipital lobe with more than 20% electrode-site participation has the highest and the frontal lobe with about 10% has the lowest participation (lines 207-213).

Lines 241-242 say that 'the quality of sleep was studied by the degree of spatial synchronization'; it will be helpful to clarify this statement because if the degree of spatial synchronization was used to select the sleep data used for this work, the dataset may include more synchronized activity compared to other datasets. This does not nullify the results, but clarification will be important to ensure replicability. Likewise, the methods used for sleep detection should be described in more detail, was sleep detected in the same way in all datasets?

Line 250: only visual inspection was used to detect stage 2 in the EEG data? clarify if only NREM sleep was used in all datasets used for analysis? (same for the iEEG recordings)

We thank the Referee for noting this important point on selecting sleep epochs in the recordings. In our analysis, we studied the full recordings annotated as sleep without excluding REM states. As mentioned in the Methods section, sleep state was assessed in the recordings using a method best adapted to each recording type. For example, in the EEG recordings, sleep state was manually annotated by an expert for stage 2 NREM sleep. In the iEEG recordings, sleep annotations were provided by hospital staff who observed times when a patient was sleeping and noted in their chart. Finally, in the ECoG dataset, NHP subjects were placed in a quiet and dark room with eyes covered for up to 90 minutes. The sleep state in this case was determined using the degree of spatial synchronization and power increase in the 1-4 Hz δ frequency band.

We appreciate the Reviewer’s comment on this point, and in developing this analysis, we carefully considered the differences in determining sleep state across the datasets analyzed here. Because in the ECoG data spatial synchronization was only assessed in the 1-4 Hz frequency band, which is known to be consistently more synchronized during sleep (Destexhe et al., J Neurosci 19, 1999; Dang-Vu et al., Neuroimage 28 2005; Chauvette et al., J Neurosci 31 2011; Murphy et al., PNAS 106.5 2009), we believe that these datasets are not more synchronized than other sleep recordings. Further, we find consistent results across the datasets we analyzed, which varied in the way they detected the sleep state. With this in mind, we believe our analysis provides considerable evidence to indicate that this potential concern does not affect our results.

We fully agree that more details would be helpful to document the recordings, and we appreciate the Reviewer’s suggestion along these lines. In our revised manuscript, we have included several updates to the supplementary section “Methods – Recordings” to detail the behavioral and electrophysiological aspects of these datasets (lines 372-373, 383-386 and 393-394). We hope this additional information also addresses the Reviewer’s more general point about reproducibility, which is an important commitment in our work.

Not clear what is added by the paragraph between Lines 283-288 unless more details are provided. If the model simulates spiking activity in the awake state, is this helpful as a ground truth control for sleep LFP? What type of spiking model? What network architecture?

We thank the Reviewer for noting that more details are necessary to clarify the spiking network control simulation. With this analysis, we sought to provide additional evidence that our approach can robustly detect sleep rhythms while also – importantly – providing no detections when sleep-like rhythms are not present. To do this, we used data produced by a spiking neural network model we have currently developed to study the spontaneous background activity of awake cortical circuits (Davis, Benigno, et al., Nature Communications, 2021 [link]). This model, composed of several million neurons with biologically realistic synaptic connectivity (several thousand synapses per cell), creates realistic ongoing activity patterns consistent with the well-studied asynchronous-irregular state. In addition, we have incorporated a recently developed LFP proxy (Mazzoni et al., PLoS Comput Biol, 2015) in the model, allowing us to analyze a population signal using our spindle detection algorithm. Starting with the first step in this process, we first apply the SNR algorithm to detect well-formed spindles for training data. As expected, however, the SNR approach does not detect any spindles in the simulated data.

Because the simulation lacks the mechanisms for generating large-scale thalamocortical and corticocortical rhythms, this provides a ground-truth test that our approach does not detect spurious spindles in neural population signals. The results from this analysis provide additional confirmation, in a setting where we know rhythms such as spindles do not exist, that our approach is not only robust to various types of noise but also sensitive to spindle rhythms embedded in population signals. We have added additional details about this model in the supplementary section “Methods – Signal-to-noise ratio (SNR) measure for sleep spindle detection” (lines 430-435).

Paragraph starting in line 290. Was the CNN model trained and optimized for each dataset?

Lines 296-297: 'best' based on what?

This is an excellent point, and we thank the Reviewer for bringing this up. In the revised manuscript, we have included additional details on the process of training and optimization of the CNN model. We implement an architecture similar to the one proposed by George and Huerta (2018) with small modifications to the input and convolutional layers to take into account the basic features of the spindle rhythm in cortex (e.g. average duration). The CNN architecture is also slightly tailored to different sampling rates in each recording modality. As in previous work, the convolutional layer is designed to start by extracting local features, gradually extracting longer-timescale features by decreasing the feature space and increasing channels. Using this strategy, the CNN model can efficiently learn to detect the specific waveform characteristics distinguishing the sleep spindle rhythm in different types of recordings. In addition, we verified that the proposed CNN model is not sensitive to the slight change in the number of layers (e.g. 4, 5, and 6 convolutional layers) and hyperparameters such as learning rate, maximum number of epochs used for training, and pooling parameters using similarity measure across the predicted values. To perform this sensitivity analysis, for each CNN architecture, we made a grid search over the potential range of hyperparameters, measuring the similarity of model output by Cronbach's α. Similarity across hyperparameters within 10-50% of those used in our analysis was greater than 0.99, indicating high reproducibility under moderate parameter variability. We then used the same architecture and hyperparameters across all subjects and recording datasets. We trained a separate CNN model for each subject on a portion of the available recording and then applied the trained model to detect spindles across the entire recording.

We thank the Reviewer for pointing out the importance of including more details on this point. We have added additional material in lines 445-466 and 475-488 of the revised manuscript to clarify the optimization, parameter tuning, training, and implementation of the CNN.

Line 350 states that the overall threshold for the AT method was based on the average root mean squared. Was the AT approach developed and applied independently for each data set (EEG, ECoG, iEEG)? It is not clear from the text if the different signal amplitudes in the datasets were considered for AT detection (were the data and RMS normalized?).

This is an important point for the clarity of the manuscript, and we appreciate the Reviewer for bringing this up. As with the CNN model, we applied the AT algorithm independently for each subject to account for the difference across subjects and recordings. We have updated the Methods on line 541-543 to clarify this point.

Figure Supp. 5: correct typo in the figure legends (spindle)

Thanks for catching this typo. We have corrected the error in the revised manuscript (Figure S10).

Reviewer #2:

The primary goal of this paper is to provide evidence that sleep rhythms are more spatially extended than previously known, using a deep learning convolutional neural network (CNN) model specifically designed to characterized rhythmic activity that has not previously been applied to sleep data.

Strengths

1. The authors establish that the CNN method proposed can detect spatially extended sleep rhythms and that difference occur when measuring spindles after a low vs high load memory tasks in multiple data sets.

2. The CNN method presented may provide a powerful technique to characterize a range of brain rhythms and difference across tasks or patient populations.

3. The results and figures are clearly presented.

4. The methods are sound, with the exception that more detail on the method and how it uniquely accounts for rhythmic activity is warranted and should be discussed in the Results section of the paper, rather than in the methods.

We thank the Reviewer for these positive comments on the methods and results presented in this work. In our revised manuscript, we have included additional detail in order to more clearly document the methods in this work. In particular, we have added more specific characterization of how the CNN model specifically and accurately detects sleep spindle activity in the main text and supplement (lines 82-86, and 95-102 and, Figure S1 and Table S1).

Weaknesses

1. The results of the paper do not establish that this CNN method is better than prior methods at detecting spatially extend sleep rhythms, or that the methods is better able to distinguish sleep spindles after a low vs high load memory task. As such, while a new method for detecting spindles is clearly presented, there do not appear to be any new scientific findings in this paper.

We appreciate the Referee's point, which we considered carefully in preparing our revised submission. The main focus of our work is to compare our CNN approach to standard amplitude-thresholding approaches. The hypothesis underlying this comparison is that amplitude-based approaches, in general, will miss many spindles that exhibit transient dips in power, or those that are very clearly formed but small in amplitude (see Figure S5). While the simplest implementation of this algorithm (Mölle et al., J Neurosci 22, 2002) can be improved by imposing additional conditions on the AT procedure (e.g. Hagler et al., J Neurosci 38, 2018 and Warby et al., Nature Methods 11, 2014), it remains difficult to cover all cases by successively adding additional conditions. Training a CNN model to detect spindles, however, has the advantage of developing these conditions automatically, without requiring hand-tuning by the analyst.

We thank the reviewer for bringing up this point about clearly comparing the CNN method to prior approaches, and we have now highlighted this more clearly in the revised manuscript. In addition, in the original submission we directly compared the changes in sleep spindles following high- versus low-load visual memory tasks (Figure 3b and S8b), and we found that the CNN was able to detect a significant increase, while the amplitude-thresholding approach found no change. We believe this result clearly demonstrates the potential for our CNN approach to find new scientific results in analyses of sleep rhythms, and we have now highlighted this more clearly in the revised text.

Finally, in our revised manuscript, we have now gone back to the recordings from the visual memory task and discovered a novel finding about spatiotemporal dynamics during multi-area spindles: rotating spindle waves significantly increase following high-load visual memory tasks, which we had previously observed to be associated with increased multi-area spindles (new Figure 4 and Video S1). These new results specifically tie the increase in multi-area spindles we reported in the original submission to a neural circuit mechanism for rotating waves in consolidating memories across distributed networks in cortex (see Figure 2, Muller et al., eLife 5, 2016). We have highlighted these new results in our responses to the main critiques, and also revised the manuscript to reflect these points (lines 62-71, 104-107 and 215-271 and new Figure 5).

In summary, we hope that these additions clearly and sufficiently address the Referee’s original concerns about our manuscript. We believe that the new computational approach presented in this work is clearly more effective at detecting multi-area spindles, which are not observed using the previous approach, and at finding new scientific results, such as the selective increase in multi-area spindles and rotating waves following challenging memory tasks.

The authors compare the CNN with only the AT methods for the main result of the paper (Figure 3, Supplementary Figure 3), and not the SNR method. Why not compare directly to the SNR also to see if CNN is actually better? The SNR method is used to generate the data set that the CNN is trained on, as such is it possible for the CNN to do better? Would the AT method be able to pick up more multi-area spindles with a lower threshold?

We thank the Reviewer for bringing up this important point. We introduced the SNR algorithm ourselves in a previous manuscript (Muller et al., eLife 5, 2016) as a conservative approach to detect spindles while maintaining a low false-positive rate. This method, which is based on the constant false-alarm rate (CFAR) method in radar (Levanon, N., John Wiley and Sons, New York, 1988; Minkler, G. and Minkler, J., Magellan, Baltimore, 1990), detects many “true” spindles while minimizing false detections; however, its aim is not to detect all spindles. For this reason, while the SNR approach is excellent for generating a high-quality training dataset (as used here for the CNN model, Figure 1) and for providing a second check on results from the CNN model (as in Figure S10), we know that it cannot provide a comprehensive detection method approaching detection of all spindles in a recording. For this reason, we have adapted the deep learning waveform detection approaches to create the method we introduce in this work. The advantage of the CNN approach we introduce is that it can detect many more spindles at high quality.

Unfortunately, because of potential artifacts associated with the AT approach, decreasing the amplitude threshold in that method will not only detect more spindles, but also more mis-classified activity patterns (i.e. false positive detections). For example, we can start with an amplitude threshold that produces 2 spindle detections per minute (Gaïs et al., J Neurosci, 2002) and then reduce this threshold by half. In Author response image 2, we provide several examples of clearly non-spindle activity detected using this reduced threshold.

Author response image 2
Impact of AT threshold on spindle quality.

(a) Plotted are the average time-shifted over spindle detected by the AT algorithm using 2 spindles per minute threshold (black line), additional spindles detected by the AT algorithm after decreasing the threshold to its half (blue line), and lastly a subset of randomly matched non-spindle windows (red line).

As expected by decreasing the threshold, the average over detected spindle activities starts to drift away from the expected 11-15 Hz oscillatory structure. (b) Examples of non-spindle activity (blue line) detected by the AT approach when we decrease the threshold producing 2 spindles per minute (solid green line) by half (dashed green line). The red line is the signal envelope.

In contrast to reducing the threshold in the AT approach, however, our CNN method is able to detect high-quality spindles that co-occur across the cortex much more often than previously thought. In our original submission, we validate this approach through analyzing the power spectrum of detected events (Figure 1b), through a time-aligned averaging approach (Supplementary Figure S3), and finally through comparison to the subset of spindles detected by the AT approach (Figure S5). At the same time, we appreciate that we may not have made the importance and impact of the CNN and SNR approaches clear in the original submission. In our revised submission, we have added additional text clarifying the SNR algorithm we have introduced previously, its use for generating a high-quality training dataset for the CNN (lines 110-117 and 292-300 in the main text), and its use in validating the results from the CNN. We appreciate that these points may not have been clear in the original submission, and we hope these new additions fully address the Referee’s concerns.

The CNN and SNR methods are directly compared only for the low vs high memory load task in Supplementary Figure 4. A visual comparison of the low vs high memory load results from the CNN (Figure 3c) and SNR methods (Supplementary Figure 4 top) suggests that the SNR method is equally able to distinguish these conditions. Overall, the advantage of the CNN is not clear.

We thank the Referee for raising this important point, which may not have been clear in the original manuscript. As detailed in the previous response, the main disadvantage of the SNR approach is that it is overly conservative: it will exclude too many spindles at the expense of keeping false positives low. Thus, while the SNR approach is useful for quickly identifying a set of spindles with low false-positive rate, it is not adequate for studying the full population of spindles or for answering questions about changes in distribution in general.

At the same time, however, the fact that the subset of spindles detected by the SNR approach exhibits the same increase under high versus low visual memory load as observed by the CNN approach provides an important corroboration of this result. These two approaches provide clearly converging evidence that distributed spindles appear following high visual memory tasks, a change that is not detected by the AT approach. For this reason, we believe that we have not only validated the performance of the CNN approach in contrast to amplitude-based approaches, but also clearly demonstrated that this approach is able to find qualitatively new results providing insight into the process of human memory consolidation.

We again thank the Referee for raising this point, which we have tried to clarify in the main text (lines 202-207). We believe that this additional explanation more clearly introduces the advantages of the CNN approach and addresses the Referee’s point in full.

2. There are several high-level strong claims in the paper that are not directly investigated or supported by the evidence in this paper. For example, "These results thus provide specific neural mechanisms by which memories can be stored in distributed neocortical networks during sleep". "Taken together, these results provide substantial evidence of a specific role for spindles in linking neuron groups distributed widely across cortex during memory consolidation". "The key missing piece is to understand how spindles can guide specific long-range excitatory connections to strengthen during sleep-dependent memory consolidation. We hypothesized that widespread, multi-area spindles might provide this mechansism". At best, the results in this paper provide supportive evidence that spindles could do these things by they do not investigate or establish causality in any way.

We thank the Referee for raising this point, with which we fully agree, and we apologize for any ambiguity in the original submission. This manuscript was originally meant as a follow up to a previous article (Muller et al., eLife 5, 2016), in which we reported that spindles were often organized as waves rotating across the cortex. In the previous manuscript (and in additional modeling work), we studied the phase speed distributions of the traveling spindle waves, which fell in the range of 2-5 m/s. As these speeds fall uniquely within the conduction velocity of the long-range fibers in cortex, we proceeded to study the long-range fibers as a potential propagation substrate and introduced a neural circuit mechanism for how these waves can enable the consolidation of memories across sets of neurons distributed widely across cortex.

We appreciate the Referee’s point and again apologize that this important connection was not clear in the original manuscript. We have taken the Referee’s concern to heart, and in the revised manuscript, we have connected our analysis of multi-area spindles to our previous results on spindle traveling waves (new Figure 4). We specifically have gone back to the data from the visual memory task and discovered a novel finding: rotating spindle waves significantly increase following high visual memory tasks, which we had previously observed to be associated with increased multi-area spindles. These new results specifically tie the increase in multi-area spindles we initially reported in the original submission to a neural circuit mechanism for rotating waves in consolidating memories across distributed networks in cortex (see Figure 2, Muller et al., eLife 5, 2016). We added new analysis in line 215-271 in the main text and the details of the computation in the new sections, “Methods – Rotating wave direction” and “Methods – Simulated data control – rotating wave”, in the supplement.

3. It is stated that in the ECoG and iEEG data, the AT method detected a subset of spindles that are significantly higher-amplitude than those detected by the CNN methods, using a one-sided Wilcoxon sign rank test. Does this mean that CNN does not detect some of the high amplitude spindles? Is this advantageous? There is something confusing about the way this is stated. A Figure of the distribution in number and amplitude of spindles detected with the 3 methods (CCN, AT, SRN) would be useful.

We appreciate this point, and in the revised submission we have this in the text to make the point more clear. While the AT approach detects only the highest amplitude spindles in the dataset (new Figure S4a), the CNN approach is able to detect some lower amplitude (but still clearly formed) spindles in addition. Thus, as expected, this result is reflected in the fact that the set of spindles detected by the AT approach (which preferentially detects the highest-amplitude events) has higher average amplitude than those detected by the CNN (which detects a broader set of spindles). Specific examples of this can be observed in the EEG recordings in Figure 1c, where the first and second spindles were not detected by AT and in Figure S5, which provides examples where the CNN detects clear spindles but the AT does not.

Moreover, to address the Referee's concern regarding the number of spindles, we perform a new analysis, in which we computed the number of spindles per minute for each session across all recordings. Interestingly, the CNN model tends to detect on average 4-5 spindles per minute (almost twice of what was reported in the literature) as opposed to the AT which resulted in 2-3 spindles per minute except in the NHP ECoG recording (new Figure S4b). The result further demonstrates that if trained properly, the CNN model has the ability to detect hidden spindles that are being unnoticed and provides a great opportunity to study the spatial and temporal analysis of spindle activities across the cortex.

We thank the Referee again for raising this important point, which we have attempted to clarify in the revised submission. We have specifically focused on clarifying why spindles detected with the AT method are expected to have higher amplitude overall (lines 138-147) and how the CNN method, which detects spindles missed by the AT approach, improves our understanding of sleep spindles and changes following visual memory tasks (lines 202-207 and 286-292). We believe these additional points fully address the Referee’s concern on this point.

4. The 3 different recording methods sample activity across different spatial scales, and depth electrodes (iEEG) are sampling vastly different areas (i.e. deeper sources) than ECoG and EEG. As such, it is difficult to relate findings related to "simultaneous spindle detection" in local, regional, multi-area electrodes across these three different measures. A primary concern is that the finding that there are more multi-area spindles (e.g. Figure 2b for iEEG – similar results for EEG and ECoG are not quantified) could be due to volume spread of the spindle source. Is there a way to rule this out? There are ways to minimize the influence of volume conduction with EEG and ECoG source analysis, however, to my knowledge, these methods currently don't exist for iEEG.

We thank the Referee for raising this critical point. We agree that it is important to carefully consider the differences in spatial sampling by the ECoG, EEG, and iEEG electrodes studied in this work. For this specific point, it is important to note that iEEG depth electrodes record cortical activity with the highest spatial resolution possible of intracranial recording techniques (Mukamel and Fried, Ann Rev Psych 63, 2012), while also quickly becoming standard in the field of neurosurgery because of their increased safety and improved patient outcomes compared to subdural grid arrays (Tandon et al., JAMA Neurol 76, 2019). Because iEEG electrodes have the highest spatial resolution of the recording techniques considered here, with ECoG (Dubey and Ray, J Neurosci 39, 2019) and EEG (Babiloni et al., Clin Neurophysiol 112, 2001) representing increasingly lower spatial resolution, it would be difficult for the higher number of multi-area spindles observed in iEEG to result from increased volume spread of the spindle source, as the influence of volume conduction is minimized in this recording technique. Rather, if increased volume spread were the underlying cause for this finding, we would expect the pattern to be the opposite, with more multi-area spindles in the EEG data (where volume conduction effects are highest). We thank the Referee for raising this important point, which we have clarified in the revised text (lines 167-169).

Recommendations for the authors:

In Figure 1d, there appear to be other spindle in each of these example traces. Were these not picked up by the algorithm, or simple not highlighted in red? Clarification would be helpful.

We thank the Referee for pointing this out and apologize for any confusion. In the original figure, we only highlighted the spindles in the central window (at the middle of the plot). In the revised Figure 1d (line 304), we have now highlighted all detected spindles, in order to demonstrate more clearly the events detected by our CNN approach.

Why are the randomly sampled red dashed line in Supplementary Figure 1 not flat? Is the 1Hz filter somehow biasing the amplitude at the center?

We thank the Referee for bringing this point up. In Figure S3, we verify that spindles detected by the CNN approach exhibit high-quality waveforms and rule out potential artifacts from bandpass filtering. To do this, we took each signal highpassed at 1 Hz, re-aligned to the largest oscillation peak in the time window, and then averaged across detected events. With this procedure, the average across detected spindles exhibits clear 11-15 Hz oscillations, even without bandpass filtering. This calculation thus provides strong evidence that the CNN approach detects high-quality spindle events.

We then repeated this calculation on a matched number of randomly selected non-spindle windows, which are those plotted as red dashed lines in Figure S3. Because we align signals to the central peak in the window, we expect there to be a positive peak at this point. Naturally, this peak will exhibit a decay consistent with the autocorrelation time present in the 1 Hz filtered signal. Importantly, however, this control condition exhibits no 11-15 Hz oscillations (i.e. additional peaks beyond the central one), demonstrating that the CNN method correctly identifies non-spindle windows, as well.

We again thank the Referee for raising this point. In the revised submission, we have included a few sentences clarifying this point in the main text (lines 136-138) and in the supplementary section “Methods – Time-shifted averaging control” (lines 509-514).

It would be helpful to see the results shown in Figure 2b for iEEG for ECoG and EEG data as well.

This is an important point, and we appreciate the Referee bringing this up. For the analysis in Figure 2b, we developed an atlas-based computational approach to placements of iEEG electrodes in collaboration with a neuroimaging laboratory. This approach included white and gray matter segmentation, registration, and cortical segmentation of iEEG electrodes. Unfortunately, we could not perform similar parcellation in EEG and ECoG recordings but we were able to further confirm that multi-area spindles are in fact distributed widely across the brain in the EEG dataset. To do this, we studied cortical lobe participation (new Figure S9) and using electrode coordinates we analyzed the change in rotational pattern of multi-area spindles (new Figure 4) in the EEG.

There is are claims about SNR and CNN being independent of spindle amplitude. Clarification of how the SNR power calculations are independent of amplitude would be useful.

Thank you for bringing up this concern. We introduced the SNR algorithm in a previous manuscript (Muller et al., eLife 5, 2016) as a conservative approach to detect spindles while maintaining a low false-positive rate. This method is based on the constant false-alarm rate (CFAR) method in radar (Levanon, N., Wiley, New York, 1988; Minkler, G. and Minkler, J., Magellan, 1990) which is used for narrow-band rhythmic activity detection in a high-noise situation with the goal of reducing false-positive detection. The CFAR is an adaptive process for target detection which adjusts the detection threshold accordingly (Weinberg, Graham Victor, IET Radar, Sonar and Navigation 7.2, 2013) to account for the change in the noise power/amplitude. Inspired by this technique, the SNR algorithm detects spindle activities by measuring the ratio of power within the frequency band of interest (here, 9-18 Hz) to power in the rest of the spectrum (1-100 Hz bandpass, with band-stop at 9-18 Hz) which make this technique approximately amplitude-invariant. While conducting our analysis, we observed a lot of examples which verified our claim of SNR being approximately amplitude invariant. An example is presented in Figure 1d in the main manuscript. In this example, the spindles detected in the EEG recording contain high and low amplitude spindles. All of these spindles were detected by both the CNN model and SNR approach, while the AT approach fails to detect the first (low-amplitude activity) and last spindles (temporary drop in amplitude) (Author response image 3) (Lines 549-550).

Author response image 3
AT sensitivity to amplitude.

The AT algorithm fails to detect clearly formed spindles detected by the CNN model in Figure 1d EEG recording because of low amplitude or temporary drop of amplitude below the threshold.

To further validate this claim, we performed this new analysis in which we used surrogate data with generated signals containing spindles and additive noise (of several types, including white and brownian noise). We then utilized the surrogate data to study how sensitive the SNR approach is to the change in spindle and noise amplitude as opposed to the AT techniques (lines 552-575). We first implemented both approaches on the surrogate data and detected spindle activities. We then repeated the entire analysis, once after doubling the noise amplitude and once after dividing the spindle amplitude in half. Lastly, using an example spindle detected by both approaches in the original setting, we showed how robust the SNR algorithm is in face of change in the signal/noise amplitude (new Figure S2). We added the details of the computation in Lines 110-117 and the new supplementary section, “Methods – Simulated data control signal amplitude”.

The methods state "We first tested CNN models with different architectures and selected one of the best architectures across sleep recording data sets". What does "one of the best" mean? Quantification would be helpful. Overall, clarification of advantages of this CNN method in identifying rhythmic activity, other than training on rhythmic activity (?), would be helpful.

We thank the Referee for pointing this out and apologize that the explanation was not clear in the original manuscript. Starting from the model architectures developed in previous work, we implement an architecture similar to the one proposed by (George and Huerta, 2018) with small modifications to the input and convolutional layers to take into account the basic features of the spindle rhythm in cortex (e.g. average duration). The CNN architecture is also slightly tailored to different sampling rates in each recording modality. We verified model quality using ECoG recordings by minimizing the difference between predicted and training labels marked by the SNR approach. In addition, we confirmed that the proposed CNN model is not sensitive to the slight change in the number of layers (e.g. 4, 5, and 6 convolutional layers) and hyperparameters such as learning rate, maximum number of epochs used for training, and pooling parameters using similarity measure across the predicted values. To perform this sensitivity analysis, for each CNN architecture, we made a grid search over the potential range of hyperparameters, measuring the similarity of model output by Cronbach's α. Similarity across hyperparameters within 10-50% of those used in our analysis was greater than 0.99, indicating high reproducibility under moderate parameter variability. We then used the same architecture and hyperparameters across all subjects and recording datasets. We trained a separate CNN model for each subject on a portion of the available recording and then applied the trained model to detect spindles across the entire recording (Lines 445-466).

The main advantage of our CNN method is its ability to identify spindles based on waveform characteristics, rather than their amplitude or power (which causes limitations on detecting low-amplitude events in high-noise situations) (Figure 1 and S4). Furthermore, the CNN method can be generalized well across recordings with different types of noise, provided there is enough training data and high-quality marked events. Previous works on detecting earthquakes and gravitational waves have demonstrated the potential and efficacy of CNN models for waveform detection in very high-noise settings. We further validate this claim by training the CNN model on surrogate data with systematically varying noise properties. We demonstrate that this approach is robust to different types of noise on the recording electrode (new Table S1 and Figure S1). Taking these points together, we believe that our two-step combination of the SNR algorithm (which generates a high-quality training dataset) and CNN model (which detects a more complete set ranging from clearly formed large- to small-amplitude activities) can be applied more generally to detect brain rhythms.

We thank the Reviewer for pointing out the importance of including more details on this point. We have added additional material in lines 82-86 and 292-302 in the main text and the supplementary section, “Methods – Convolutional neural network (CNN) for sleep spindle detection”, and “Methods – Simulated data control – varying noise” of the revised manuscript to clarify the optimization, parameter tuning, training, and implementation of the CNN.

[Editors’ note: what follows is the authors’ response to the second round of review.]

We thank the authors for carefully considering the feedback they received during the first round of reviews, and for performing additional analyses and modifying their manuscript with the feedback in mind. We also thank the authors for patiently waiting for these reviews. The reviewers found the new version of the manuscript a substantial improvement compared to the first submission. The reviewers feel that additional changes and clarifications can put the authors' contribution in its proper context, and broaden the reach of their contribution in the community. The reviewers think the authors ought to

We would like to thank the Editor and the Referee again for their valuable and thoughtful comments, which have substantially strengthened this paper. We appreciate that both the Editor and Reviewers find the revised manuscript to be substantially improved. In this new revision, we have performed additional analyses and provided more details on our approach. We hope the additional material provided further strengthens our manuscript and addresses all remaining critiques and comments in full.

1. Make their comparisons of the CNN and AT methods in the presence of noise more realistic than currently. Experimentalists would care, for instance, about robustness of the CNN, compared to the AT method, to biological forms of noise (e.g. confounding oscillations such as a brief period of REM theta synched across channels, does the CNN offer an advantage to the AT in that case?). Moreover, the authors seem to have made the comparisons with a fixed noise level/SNR (not to confuse with the 'SNR' method). Does the choice of noise amount have a biological basis? The authors will find text in the reviews unpacking this comment

We thank the Editor and Reviewers for raising this important point and suggesting a new analysis to provide additional evidence on the advantages of using the CNN model than the traditional AT approach. The logic behind the noise sensitivity analysis was indeed to demonstrate improved performance of the CNN model under different noise types and magnitudes. Following the Editor and Reviewer's suggestion, we studied the change in the performance of the CNN and AT approaches under biological forms of noise, such as theta oscillations, and non-biological artifacts, such as line noise, electrical noise, and movement artifacts. On the new set of surrogate data containing simulated theta oscillations as confounds, the CNN model was able to detect at least 97% of spindle activities, with a specificity above 99%, while the AT achieved lower performance (specificity around 72%). In particular, the AT approach seems to be very sensitive to recording artifacts. Our new analysis demonstrates the CNN model performs robustly in face of increases in the number of artifacts, while the performance of the AT substantially decreases as the number of artifacts per minute increases(new Figure 1—figure supplement 6). These new analyses further confirm that the CNN approach introduced here can accurately detect spindles across a range of sleep recording types and provide better performance than the commonly used AT technique.

We appreciate the Editor and Reviewers raising this important point, which we have addressed in our revised manuscript by providing additional evidence that the CNN detects spindles more effectively than the AT approach using surrogate data (lines 166-170 in the main text, updated supplementary section “Simulated data control – varying noise” and new Figure 1—figure supplement 6 and updated File S1). We hope these additions fully address the Referees’ concerns.

2. Clarify apparent inconsistencies in certain statistics reported (e.g. spindle rates), as well as conduct statistical tests for some of their data more appropriate than ones currently used. The authors will find text in the reviews unpacking this comment

These are important points for the clarity of the manuscript, and we appreciate the Editor and Reviewers bringing them up. We apologize that the descriptions were not clear in the original manuscript. The key terminological distinction, which has not been clearly established in the literature, is between spindle rates reported at the single-electrode level (as in Gais et al., J Neurosci, 2002) and at the level of multiple electrodes, where spindles occurring simultaneously across multiple electrodes can be recognized as the same event. To clarify this distinction, we now define a “single-electrode spindle rate” (Figure 1—figure supplement 3) and compare this both to previous work (Gais et al. 2002, Fogel, S. M., et al. 2007 and Purcell, S. M., et al. 2017) and the “multi-electrode rate” (Figure 3b) computed over the entire array of electrodes. Because the multi-electrode spindle rate considers spindles occurring simultaneously on multiple electrodes as a single event and divides these events into one of three groups (local, regional, or multi-area), this measure will generally be lower than the single-electrode rate. We have now added a specific discussion in the manuscript on this point, along with further details, to clearly distinguish between these two measures and to highlight the results of our comparison (lines 225-230).

Furthermore, to provide additional insight on the increase in multi-area spindles following high-load visual memory tasks, we compare multi-electrode spindle rates between high- and low- load visual memory conditions. Comparing multi-electrode spindle rates demonstrates that, though multi-area spindles are indeed more rare events involving coordination across multiple brain areas, these events uniquely display a rather substantial and significant modulation across pre-sleep memory engagement (lines 231-234). We also note that our analyses earlier in this work indicate that the EEG datasets may contain fewer multi-area spindles (due to their decreased signal-to-noise ratio) than the iEEG (lines 612-615). Future work involving pre-sleep memory tasks during iEEG may have an opportunity to resolve even larger changes in these sleep events recorded at very high spatial and temporal resolution (lines 329-331).

Lastly, we appreciate the suggestion to use a non-parametric test by the Editor and Reviewer to assess statistical significance. While we initially took care in applying the t-test to these data, for example by verifying normality of the difference between paired observations with a Kolmogorov-Smirnov test before implementing the paired-sample t-test, we agree that a non-parametric test may be more suitable in this case. Following the Editor and Reviewer's suggestion, we switched to the paired-sample Wilcoxon signed-rank test to evaluate the statistical significance between low- and high-load conditions, with results unchanged from before (Figure 3b, Figure 3—figure supplement 1 and lines 231-232). To maintain consistency throughout the manuscript, we have now updated the statistical tests for analyzing rotating waves (Figure 4) to the paired-sample Wilcoxon signed-rank, with similar results as previously (lines 260-261, 263-264 and 266-267).

3. Tone down claims of generality of the CNN, and provide clear explanations for what makes the choice of architecture suitable to the current application to spindle detection. The reviewing editor finds this the most important piece of feedback from the reviews. As written, the reviewing editor feels that the manuscript may promote a culture of using deep learning in neuroscience without understanding how deep learning works. The authors have a unique opportunity of not only presenting an application of deep learning that leads to new scientific findings/hypotheses but also of doing so in a manner that promotes a culture of looking inside the black box and trying to understand it. The authors will find text in the reviews unpacking this comment

We appreciate the Reviewing Editor raising this important general point. Our goal in introducing the CNN technique is to provide a flexible and robust method to identify sleep spindles in noisy electrophysiological recordings, a problem where solutions have often relied on data manually scored by neurologists (expensive and subjective) or on amplitude-based automated techniques that miss many spindle events (as demonstrated here). Our goal with this technique, however, is by no means to encourage blind application of deep learning to this problem; rather, for each main result in the analysis we have utilized our two-step approach to create a validated and well-controlled process. To be specific, throughout the manuscript we (1) use the CNN method to detect a comprehensive set of spindles in the recordings and (2) validate the result obtained on the CNN-detected spindles on the subset of events detected by the more-conservative SNR method. In this way, in addition to providing a useful method to train the CNN on high-quality events that do not require manual detection (lines 133-135), we believe this two-step process represents a methodological advance that can make the most of the benefits of the deep learning approach (flexible waveform detection in difficult, high-noise settings) while also providing systematic validation of results obtained by the CNN on a high-quality subset (lines 86-96 and 103-105). In this way, we believe this work points towards a more general idea for application of deep learning techniques for neural data, allowing us to answer an important question in the sleep field: how do we make population-level statements about a set of neural events detected under uncertainty, without relying on arbitrary thresholds?

With this in mind, we certainly agree with the Reviewing Editor that application without understanding is a concern when introducing deep learning methods for neural data. This concern motivated our considerations throughout this work, and it is a concern that should remain central for applications of deep learning approaches in neuroscience. To this end, we have added a section in the manuscript specifically addressing this concern and highlighting the ways in which our two-step approach provides a flexible and rigorous solution to detecting specific neural events in electrophysiological recordings (lines 349-357). We have also added additional analyses to understand the choice of CNN architecture and hyperparameters, and features most important for spindle detection by the CNN model, including the filters and saliency maps for the trained CNN (new Figure 1—figure supplement 7 and 8), filter size of convolutional layers (new Figure 1—figure supplement 9), along with new supplementary sections “CNN choice of architecture and hyperparameter setting” and “CNN Visualization and interpretation” and additional text to discuss this point (lines 170-173). Finally, we have also added a section in the text carefully noting potential caveats in applications of deep learning for analysis of neural recordings and emphasizing the importance of well-controlled and interpretable analyses in this and future work (lines 357-372).

We believe that highlighting the necessity for well-controlled analyses and interpretability of these algorithms can address the concern on the motivation for deep learning in neuroscience. We believe it is possible to integrate deep learning methods into tightly controlled analysis workflows for electrophysiological and neuroimaging data, but agree that without proper controls, potential caveats of these methods would remain unaddressed. We hope the additional text and analyses fully address the concern of the Reviewing Editor here.

Reviewer #1:

The manuscript has improved substantially and many of my concerns have been addressed. The deep-learning spindle detection method is more fully reported and the new controls with simulated data (varying noise and amplitude) are an improved approach to compare the CNN and AT methods (Figure S1, new Supplementary Table 1). The new analyses investigating the rotating spindle waves provide a framework for understanding the differences with prior reports of low synchrony.

We thank the Reviewer for the thoughtful comments and concerns and appreciate acknowledging our work to address them in the first round of revision. We take your new comments to heart and have worked to address them all in this new revision.

However, there are several important concerns that should be addressed.

The new comparisons between CNN and AT models using surrogate data with systematically varied noise are useful. However, what was the rationale to compare the specific types of noise presented in Supplementary Table 1? Both the CNN and the AT have similar sensitivities and specificities for all noises, which may suggest that the different noise conditions do not constitute substantially different challenges for the detectors (also suggested by the example in Supplementary Figure 1, in which the spindle has a high signal to noise ratio with all types of noise). I think the readers will be more convinced of the value of the CNN method it can offer an advantage under a 'noise' condition in which the traditional AT methods are likely to struggle. For example, a more interesting source of noise may be the presence of non-spindle oscillations due to brief state changes (e.g., REM or α), or noise artifacts (e.g., examples in Supplementary Figure 12). The sensitivity of traditional methods is likely to go down in examples like these, how does the CNN compare?

We thank the Reviewer for raising this important point and suggesting a new analysis to provide additional evidence on the advantages of using the CNN model than the traditional AT approach. The logic behind this analysis was indeed to demonstrate improved performance of the CNN model under different noise types and magnitudes. Following this suggestion, we studied the change in the performance of the CNN and AT approaches under biological forms of noise, such as theta oscillations, and non-biological artifacts, such as line noise, electrical noise, and movement artifacts. We simulated theta oscillation using the approach proposed in the previous revision. We used Equation 1 in the paper with the oscillation angular frequency randomly selected from the 4-8 Hz frequency range to simulate theta (lines 652-656). To simulate non-biological artifacts, we first randomly chose a subset of artifacts detected as spindles by the AT approach in the iEEG recording. We then used the Fast Fourier Transform (FFT) to convert these artifacts into frequency domain. We next used the signal amplitude and randomly selected phases from [0, 2π], to generate a new set of artifacts and used the Inverse FFT to convert the signal back to time domain (lines 656-662). We visually inspected the simulated artifacts and verified the signals by comparing the PSD of the simulated artifacts with the original artifacts (new Figure 1—figure supplement 6a). We finally ran the CNN model and AT over both simulated datasets. Similar to previous cases, CNN model tends to have a higher performance than AT approach. However, the performance of AT drops significantly once we added noise artifact (Updated File S1).

We then studied the effect of artifacts on performance of the CNN and AT approach by systematically increasing the number of artifacts per minute in the surrogate data. In particular, the AT approach seems to be very sensitive to recording artifacts. The CNN model performs robustly in face of increases in the number of artifacts, while the performance of the AT gradually decreases as the number of artifacts per minute increases (new Figure 1—figure supplement 6b). This result further verifies that our approach is not sensitive to different types of artifact in the recording, as opposed to the AT approach (lines 672-678).

Taking these results together, we believe that, in contrast to the AT approach, which tends to be sensitive to recording artifacts, the CNN model performs robustly across a diverse range of artifacts and noise. These new analyses further confirm that the CNN approach introduced here can accurately detect spindles across a range of sleep recording types and provide better performance than the commonly used AT technique. We have added new figures (Figure 1—figure supplement 6) and analysis in supplementary section “Simulated data control varying noise” and lines 166-170 in the revised manuscript to further demonstrate that our deep learning approach is robust to different types of noise on the recording electrode.

Line 92: "co-occurrence"; please clarify the time window or time overlap used to determine co-occurrence. Did any amount of time overlap between spindles count as co-occurrence? Likewise, in Line 150: "simultaneously detected spindles", line 151: "based on co-occurrence". I think it's worth emphasizing in this paragraph that 'co-occurrence' and 'simultaneous' detection do not imply 0-lag synchrony (as discussed in the 'rotating waves' sections).

We thank the Reviewer for raising this point, and we appreciate this opportunity to make this point clear. Indeed, the two concepts of co-occurrence and zero-lag synchrony are distinct features of spindle events that are often not clearly distinguished in the literature. Here, by “co-occurrence” we indicate spindles that are detected simultaneously in different electrodes in the same 500-ms detection window. As mentioned by the Reviewer, this does not necessarily indicate the spindles also exhibit zero-lag synchrony, and we indeed find in our phase analyses that spindles systematically exhibit phase offsets across electrodes. In the revised submission, we have included a few sentences clarifying this point in the main text (lines 107, 119-121, 123-125 and 176).

Lines 102 and 104: "low and high visual memory task" should be "low and high-load…" (also in the Figure 3 legend, and in lines 261-262, 263: 'high-load' visual memory tasks).

Thank you for pointing this out. We have updated the revised manuscript accordingly.

There are some inconsistencies in the reported rates (spindles/min) across analyses and figures: Suppl. Figure 4a indicates average spindles/min around 4-5/min in all data types, including EEG data used in the visual memory task. However, the average from the distributions shown in Figure 3b seems much lower, even when combining all spindles across each memory condition, e.g. for the H-VM is about 1/min in local and regional and about 0.5/min for multi-area. In other words, could you clarify how the averages of the distributions in Figure 3b add up to 4 spindles/min in Figure 4a?

These are important points for the clarity of the manuscript, and we appreciate the Reviewer bringing them up. We apologize that the descriptions were not clear in the original manuscript. As noted for Main Critique 2, the key distinction here is between spindle rates reported at the single-electrode level (as in Gais et al., J Neurosci, 2002) and at the level of multiple electrodes, where spindles occurring simultaneously across multiple electrodes can be recognized as the same event. To clarify this distinction, we now define an “single-electrode spindle rate” (Figure 1—figure supplement 3) and compare this both to previous work (Gais et al. 2002, Fogel, S. M., et al. 2007 and Purcell, S. M., et al. 2017) and the multi-electrode spindle rate (Figure 3b) computed over the entire array of electrodes. Because the multi-electrode spindle rate considers spindles occurring simultaneously on multiple electrodes as a single event and divides these events into one of three groups (local, regional, or multi-area), this measure will generally be lower than the single-electrode rate. We have now added a specific discussion in the manuscript on this point, along with further details, to clearly distinguish between these two measures and to highlight the results of our comparison (lines 225-230).

In addition, in Figure 3b, within the low-VM condition the rate of multi-area spindles is substantially lower (< 0.5/min) than local and regional (about 1/min on average); and within high-VM the rate also appears much lower (<1/min) for multi-area compared to the distributions for local and regional spindles (up to ~2/min). The text indicates p-values for what seems to be a comparison of relative rates (between H-VM and L-VM), which is interesting but is this result due to a significantly lower multi-area spindle rate with low memory? Based on the figure it looks like multi-area spindles occur at lower rates regardless of memory load? It will be helpful to provide statistical comparisons of spindle rates (among all spindle types) within each condition (L-VM, H-VM) so that the relative changes with memory can be interpreted in the context of the absolute spindle rates in each memory condition. Based on the distributions it seems that non-parametric tests would be more appropriate than t-tests.

We thank the Reviewer for bringing up these important points, which we believe are an opportunity to provide additional insight on the increase in multi-area spindles following high-load visual memory tasks. Importantly, we note that our comparison was always between the absolute rate of spindles in the L-VM and H-VM conditions. For example, the detection rate of multi-area spindles in the L-VM condition is 0.38 ± 0.02 detections/minute (average across subjects ± SEM). In H-VM, however, this rate increases to 0.62 ± 0.08 detections/minute, almost a doubling of the rate. By contrast, the rate of local spindles in the L-VM condition is 0.77 ± 0.10 detections/minute, with a similar rate of 0.79 ± 0.11 detections/minute in H-VM. This comparison of absolute multi-electrode spindle rates demonstrates that, though multi-area spindles are indeed more rare events, involving coordination across multiple brain areas, these events uniquely display a significant and rather substantial modulation across pre-sleep memory engagement. We appreciate the opportunity to clarify this point, as the comparison here was not sufficiently clear in the previous version. We do note that the rates estimated in this work include partially overlapping windows; however, because our comparison in this work is focused on the difference between L-VM and H-VM, this does not bear relevance to the differences reported here (as verified in additional subsequent analyses with varying window offset). Finally, we also note that our analyses earlier in this work indicate that the EEG datasets may contain fewer multi-area spindles (due to their decreased signal-to-noise ratio) than the iEEG (lines 612-615). Future work involving pre-sleep memory tasks during iEEG may have an opportunity to resolve even larger changes in these sleep events recorded at very high spatial and temporal resolution (lines 329-331). We again thank the reviewer for pointing out the importance of stating the absolute spindle rates, which we have now included in lines 232-234 and 544-549 of the text.

We also appreciate the suggestion to use a non-parametric test by the Reviewer to assess statistical significance. While we initially took care in applying the t-test to these data, for example by verifying normality of the difference between paired observations with a Kolmogorov-Smirnov test before implementing the paired-sample t-test, we agree that a non-parametric test may be more suitable in this case. Following the Reviewer's suggestion, we switched to the paired-sample Wilcoxon signed-rank test to evaluate the statistical significance between low- and high-load conditions, with results unchanged from before (Figure 3b, Figure 3—figure supplement 1 and lines 231-232). To maintain consistency throughout the manuscript, we have now updated the statistical tests for analyzing rotating waves (Figure 4) to the paired-sample Wilcoxon signed-rank, with similar results as previously (lines 260-261, 263-264 and 266-267).

Suppl. Figure 4: missing (a),(b) labels.

Thanks for catching the missing labels. We have added the labels in the revised manuscript.

Supplementary Figure 4a is not described in the text?

We thank the Reviewer for noting this point and apologize that this figure (Supplementary Figure 4a, now updated to Figure 1—figure supplement 3a) was not described in the manuscript. We have added a reference and detailed description for Figure 1—figure supplement 3a in the main text (lines 156-157 and 345-346)

Line 725: 12? sites

Thanks for catching this typo. We have corrected the error in the revised manuscript (Figure 1—figure supplement 4).

Line 154: it'd be useful to reference the Supplementary Table 2 (with the list of the cortical regions).

We appreciate this suggestion, and we have added a reference to File S2 (previously called Table S2) in the revised manuscript (line 181).

Lines 170, 171: I believe the verb should be in past tense.

We apologize for the inconsistent verb tenses. We have corrected the verb tenses throughout the revised manuscript.

Line 179: The interpretations regarding the association between the visual memory load and subsequent sleep spindles is limited to memory consolidation processes. However, memory performance after sleep was not studied in this work, therefore "the impact on memory consolidation" cannot directly be assessed. Sentences such as these (lines 179-180): "If this is the case, what is the impact of the distribution on sleep-dependent memory consolidation? To answer this question…" could be rephrased to state the key question that the data can address. Other sentences in the manuscript are more accurate and valuable in this respect, e.g., Lines 289-290 "(2) this spatial extent can be modulated by the specific memory conditions prior to sleep (Figure 3)". Indeed, memory consolidation is one of several mechanisms that determine how wakefulness influences sleep oscillations. The authors correctly cover literature on the association between memory and spindles "An increase in spindle density after memory tasks and its relationship with memory consolidation is well established (Clemens et al., 2005; Dang-Vu et al., 2008; Gais et al., 2002; Schabus et al., 2007, 2004)", but the impact of the authors' findings would be enhanced by discussing other hypotheses in the sleep field that are also consistent with the presented results; for example, the high-load working memory condition may increase firing rates and entrain downscaling processes during subsequent sleep (Tononi and Cirelli, 2014; Crunelli et al., 2018; Klinzing et al.,2019).

We thank the Reviewer for bringing up this important point. In our revised manuscript, we have updated the language to more carefully focus on the impact of pre-sleep memory tasks on the distribution of spindles in cortex (lines 72, 209 and 273-275). We also appreciate the Reviewer’s point that these results could be consistent with the process of synaptic downscaling. We have included a more detailed treatment of this point in the Discussion (lines 289-295), along with specific suggestions for how future work could address these hypotheses by studying sleep recordings (lines 317-318). We again thank the Reviewer for this suggestion and hope the additions to the manuscript fully address this point.

Paragraph starting 215: is this only with the EEG data?

Yes, this analysis is only performed on the EEG data which had the unique feature of testing sleep after tasks with varying memory loads. In this analysis, we are focusing on the comparison of spatiotemporal dynamics in low-load and high-load memory tasks. We have now noted this more clearly in the revised text (lines 121 and 253).

Figure 4a: missing (a),(b) labels for the plots. Indicate the x,y value of the removed outlier point in parenthesis (in 'a' and 'c') since it's not in the plot but still used in quantifications.

Thanks for catching the missing labels and the suggestion. We have added the outliers and labels in the revised manuscript (Figure 4a, b and c).

Line 552: "We simulated 60-minutes recording containing" should it be "60-minutes of recording…"?

Thanks for catching this typo. We have corrected it in the revised manuscript (line 617).

Reviewer #2:

The authors suggest that the lack of algorithms for reliably identifying spindles in neural recordings has led to the under reporting/underestimation of the spatial extent to which spindles occur in the brain. The authors propose an approach based on CNNs that they claim can detect spindles more reliably than existing ones, lead to new insights as to how cross-region spindles may contribute to the integration of 'information' across brain areas.

Strengths

1. The applicability of the proposed methods to neural data from multiple modalities, i.e. EEG, EcOG and iEEG.

2. The framework the authors lay out for constructing high-quality data sets to train the CNNs.

3. The combination of computational methods and new suggested insights into how spindles can facilitate the integration of 'information' across brain areas.

We thank the Reviewer for highlighting the novelty and strength of our submission and appreciate the opportunity to discuss these points.

Weaknesses

1. The authors could improve the explanations as to why the CNN seems to do better than legacy methods such as the AT. The explanation surrounding the CNN's ability to detect spindles of different amplitudes does not seem satisfactory enough.

We appreciate the opportunity to further discuss these important points. One of the main advantages of the CNN model is its ability to detect clearly-formed spindles ranging from lower to high amplitudes. Thus, while AT approaches tend to detect only the highest-amplitude spindle events (as in Figure 1—figure supplement 4), the CNN detects a broader set of spindles. The second advantage of the CNN model is its ability to generalize well across recordings with different types of noise, provided there is sufficient training data and high-quality marked events. The results from our previous analysis indicate that our CNN approach can achieve a very high performance when trained under varying noise and recording conditions (Figure 1—figure supplement 5 and File S2). In this revision, we also study change in the performance of the CNN and AT approaches under biological forms of noise such as the REM theta oscillation and non-biological artifacts. Under these conditions, the CNN approach clearly outperforms the AT, due in large part to the linear decrease of AT performance with increase in the frequency of artifacts (lines 672-678, and Figure 1—figure supplement 6b). Taking these results together, we believe the evidence in this work demonstrates the CNN model detects a broader range of spindles and performs robustly across a diverse range of artifacts and noise as opposed to the AT approach. We have added additional explanation in lines 166-170 and supplementary section “Simulated data control – varying noise” in the revised manuscript to fully address the Referee’s concern on this point.

2. The authors claim the applicability of the CNN to data from different modalities, and its generality, as a strength. Given the black-box nature of CNNs, and the lack of an attempt in the manuscript to explain the CNN, its success/failure modes, which aspects of the data it focuses on to detect spindles (e.g. saliency maps), these claims do not feel justified to the reviewer and may contribute to the proliferation of black-box CNNs in neuroscience

We thank the Referee for bringing up this important point, which we have considered very seriously in the process of revision. Our goal in introducing the CNN technique is to provide a flexible and robust method to identify sleep spindles in noisy electrophysiological recordings, a problem where solutions have often relied on data manually scored by neurologists (expensive and subjective) or on amplitude-based automated techniques that miss many spindle events (as demonstrated here). Our goal with this technique, however, is by no means to encourage blind application of deep learning to this problem; rather, for each main result in the analysis we have utilized our two-step approach to create a validated and well-controlled process.

With this in mind, we certainly agree with the Reviewer that application without understanding is a concern when introducing deep learning methods for neural data. This concern motivated our considerations throughout this work, and it is a concern that should remain central for applications of deep learning approaches in neuroscience. To this end, we have added a section in the manuscript specifically addressing this concern and highlighting the ways in which our two-step approach provides a flexible and rigorous solution to detecting specific neural events in electrophysiological recordings (lines 349-372). We have also added additional analyses to understand the choice of CNN architecture and hyperparameters, and features most important for spindle detection by the CNN model, including the filters and saliency maps for the trained CNN (new Figure 1—figure supplement 7 and 8), filter size of convolutional layers (new Figure 1—figure supplement 9), along with new supplementary sections “CNN choice of architecture and hyperparameter setting” and “CNN Visualization and interpretation” and additional text to discuss this point (lines 170-173).

We believe that highlighting the necessity for well-controlled analyses and interpretability of these algorithms can address the Reviewer’s concern on the motivation for deep learning in neuroscience. We believe it is possible to integrate deep learning methods into tightly controlled analysis workflows for electrophysiological and neuroimaging data, but agree that without proper controls, potential caveats of these methods would remain unaddressed. We hope the additional text and analyses fully address the concern of the Referee here.

Recommendations for the authors:

The reviewing editor found the manuscript a much improved version of the initial submission.

1. The reviewing editor found that the analyses currently in the manuscript do not support the authors' claims of generality of the CNN (e.g. lines 376-377), and that the language surrounding these claims may contribute to supporting an already-widespread tendency to utilize black-box neural networks for analyzing neural data.

a) One feedback from the first round of reviews had to do with the lack of details surrounding the CNN. The reviewing editor appreciates the authors' attempt to improve this. The authors seem to want to emphasize the generality of the architecture. Such claims of generality do not give insight to the reader on how to pick an architecture for detecting oscillatory patterns in a different context (e.g. ripples). The success of the architecture for gravitational wave detection, its use in the current manuscript, do not give license to use it w/o changes in any application.

A practitioner would appreciate guidance on architecture design. Given the length of pattern of interest and a sampling rate, a user would want to know how to pick filter sizes. For instance, for a patter of length 0.5 seconds, a 256 Hz sampling rate, the effective filter (explained below) associated with an architecture ought to have size on order of 100 samples (0.5*256 = 128 samples), not 10 or 1000. The reviewing editor thinks this manuscript could have a much stronger impact if the authors took such questions into considerations. Have authors visualized the filters they learn? Have they considered generating saliency maps to determine which parts of inputs the network focuses on to detect spindles? At present, the emphasis on the generality of the CNN, w/o considerations for what about the authors' specific context makes the choice of architecture suitable (other than the fact that it works for gravitational-way detection) feels a bit disappointing.

More details on guidance on how to design architecture/why current architecture works: For any conv net, the choice of filter size at each layers, together with number of layers relates to size of features one would like to detect. Each layer a conv. Composition of conv equals a conv with a filter size roughly proportional to the sum of filter sizes at each layers. 7x7 filters not uncommon in image proc. Cascading 3-4 such layers (ignoring nonlinearities) gives a filter with an effective 30x30 size.

Concrete suggestions: (a) adding text to the manuscript explaining what what about the authors' specific context makes the choice of architecture suitable (other than the fact that it works for gravitational-way detection),

We thank the Reviewer for noting that more details are necessary to clarify the choice of CNN architecture and we apologize that our explanation was not clear in the previous revision. The CNN architecture should be tailored with respect to the duration of rhythmic activity, type of oscillation and sampling rates of recording modality. The duration of the rhythmic activity and sampling rate determine the length of the sliding window for the CNN model. For example, in our NHP EEG recording with the sampling rate of 1000 Hz, the sliding window is set to 0.5 sec which is the minimum duration of spindle activities observed during sleep and contains 500 data points. We then specify the filter size with respect to the length of the sliding window and types of rhythmic activity. The CNN layer is designed to start by extracting local features, gradually extracting longer-timescale features by decreasing the feature space. Filter sizes covering up to approximately one oscillation cycle (70-120 msec) are effective in detecting spindle activity. To understand this further, we simulate 10 recordings with 2 spindles per minute and add different types of noise including the theta oscillation and artifact. Using these surrogate recordings for which we have the exact timing of spindles, we demonstrate that longer filter size is ineffective at detecting spindles. Specifically, we gradually increase the filter size (new Figure 1—figure supplement 9) and compute the performance of the CNN model. As expected, the CNN performance drops as we increase the filter size, in accordance with this underlying mechanism. This result further validates the generality of the CNN approach for detecting neural rhythms, while also getting at the mechanism. We believe that a similar mechanism can be implemented for specifying the filter size for other neural and biological rhythms. The current CNN architecture works well with slight changes in the sliding window (duration and sampling rate) and type of oscillation, but it requires modification otherwise.

We have added a new supplementary section “CNN choice of architecture and hyperparameter setting” and figure (Figure 1—figure supplement 9) to explain the underlying mechanism of setting filter size in the CNN model. We hope these additional clarifications fully address the Referees’ concerns on this point. We are also providing a guideline on how to set the parameters of the CNN model in our toolbox for different detection settings. The first GitHub version of the toolbox is attached for reference.

b) visualizing filters learned by the architecture, c) consider generating saliency maps.

We thank the Reviewer for these suggestions. To understand this point further, we studied the filter and saliency maps of the CNN models. To do this, we first simulate six spindle/non-spindle signals and visualize the feature and gradient maps corresponding to these input signals. The feature map can accurately detect the maximum amplitude within each cycle using the maximum activation of the CNN model across all oscillations (new Figure 1—figure supplement 7). Using these features, which specify the timing and relative height of the detected maximum amplitudes, the CNN model can learn to reliably detect the specific waveform characteristics of the sleep spindle in different types of recordings. In addition, we studied gradient attribution maps to identify part of the signal that are most important for classification in the CNN model (new Figure 1—figure supplement 8). The area of the signal with the highest modulation in amplitude has the greatest impact on the classification. The pattern of activations and gradient map across the simulated signals provides insights into the underlying mechanism by which the CNN model efficiently distinguishes between different types of oscillation. We thank the Reviewer for pointing out the importance of including more details on how the CNN model works. We have added a few sentences in the main text (lines 170-173), a new supplementary section, “CNN Visualization and interpretation” and figures (Figure 1—figure supplement 7 and 8) in the revised manuscript.

b) The reviewing editor feels that the authors can improve the section detailing the training of the CNN. The editor suggests the authors consider using a table to summarize different aspects (number of layers, filter sizes etc….). The editor also suggests that, very early on, the authors mention the subject-dependence of the training.

We appreciate the Reviewer’s comment on the training aspect of the CNN model. As requested in the previous round of revision, we made a grid search over the potential range of hyperparameters and measured the detection similarity using Cronbach's α. New File S3 contains the details of our sensitivity analysis in the updated manuscript. We also performed further analysis on the choice of CNN architecture and filter size in the new supplementary section, “CNN choice of architecture and hyperparameter setting”. The result from our sensitivity analysis confirms that the proposed CNN architecture has high reproducibility under moderate parameter variability.

Finally, following the Reviewer’s suggestion, we added a few sentences early on in the main manuscript (lines 100-104 and 132-133) explaining that the CNN model is subject-dependent. We appreciate your comments to improve the paper and we have added additional clarification in the revised manuscripts to answer your concerns.

2. Misc questions

a) In supplementary figure 4b, the scatter plot for EEG (green) suggests a similar max PSD for the CNN and AT. I would expect a similar rate of spindle detection. Supplementary figure 4a suggests otherwise. Can the authors explain?

We thank the Reviewer for noting this point. While the AT approach consistently detects spindles of higher amplitude than the CNN in the iEEG and ECoG recordings, this effect is indeed not observed in the EEG recordings. We believe this may be due to lower overall signal-to-noise ratio in the EEG; however, it is difficult to determine the precise cause underlying this change in these two datasets. We have noted this point in the text and suggested possible analyses in simultaneous EEG-iEEG recordings (which are often taken by our clinical colleagues at LHSC) for future work (lines 317-318 and 612-615).

b) In Figure 2c, EcOG the fact that the CNN detects fewer local and regional spindles per minute than the AT method requires explanation. Does this figure refer to correctly detected spindles? Should the editor interpret the lower ratio as the AT method having more false alarms than the CNN?

We thank the Reviewer for noting this point. For one subject in the ECoG recordings analyzed here, the distribution of signal amplitudes in the 11-15 Hz frequency band varies so widely that a fixed threshold in the AT technique no longer distinguishes well between spindle and non-spindle activity at all electrodes. The variation in amplitude across electrodes results in incorrect detections in electrodes with lower amplitudes and missed detections in electrodes with higher amplitudes. For this reason, the AT technique in this subject caused more detections of local and regional spindles. Importantly, this point further demonstrates the potential problems with the AT approach, where artifacts or recording noise can significantly affect results on spindle distribution or occurrence. We appreciate the Reviewer’s comment and have now updated the manuscript to explain this point (lines 196-199).

https://doi.org/10.7554/eLife.75769.sa2

Article and author information

Author details

  1. Maryam H Mofrad

    1. Department of Mathematics, Western University, London, Canada
    2. Brain and Mind Institute, Western University, London, Canada
    Contribution
    Conceptualization, Data curation, Formal analysis, Methodology, Software, Validation, Visualization, Writing - original draft, Writing - review and editing
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-0094-6694
  2. Greydon Gilmore

    1. Brain and Mind Institute, Western University, London, Canada
    2. Department of Biomedical Engineering, Western University, London, Canada
    Contribution
    Data curation, Formal analysis, Resources, Software, Validation, Visualization, Writing - review and editing
    Competing interests
    No competing interests declared
  3. Dominik Koller

    Advanced Concepts Team, European Space Agency, Noordwijk, Netherlands
    Contribution
    Methodology, Visualization, Writing - review and editing
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-7449-9516
  4. Seyed M Mirsattari

    1. Department of Clinical Neurological Sciences, Schulich School of Medicine and Dentistry, Western University, London, Canada
    2. Department of Medical Imaging, Schulich School of Medicine and Dentistry, Western University, London, Canada
    3. Department of Medical Biophysics, Schulich School of Medicine and Dentistry, Western University, London, Canada
    4. Department of Psychology, Western University, London, Canada
    Contribution
    Data curation, Investigation, Resources, Supervision, Validation, Writing - review and editing
    Competing interests
    No competing interests declared
  5. Jorge G Burneo

    1. Department of Clinical Neurological Sciences, Schulich School of Medicine and Dentistry, Western University, London, Canada
    2. Department of Epidemiology and Biostatistics, Schulich School of Medicine and Dentistry, Western University, London, Canada
    Contribution
    Data curation, Investigation, Resources, Supervision, Validation, Writing - review and editing
    Competing interests
    No competing interests declared
  6. David A Steven

    1. Department of Clinical Neurological Sciences, Schulich School of Medicine and Dentistry, Western University, London, Canada
    2. Department of Epidemiology and Biostatistics, Schulich School of Medicine and Dentistry, Western University, London, Canada
    Contribution
    Data curation, Investigation, Resources, Supervision, Validation, Writing - review and editing
    Competing interests
    No competing interests declared
  7. Ali R Khan

    1. Brain and Mind Institute, Western University, London, Canada
    2. Department of Biomedical Engineering, Western University, London, Canada
    3. Department of Medical Biophysics, Schulich School of Medicine and Dentistry, Western University, London, Canada
    Contribution
    Data curation, Formal analysis, Methodology, Resources, Software, Supervision, Validation, Writing - review and editing
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-0760-8647
  8. Ana Suller Marti

    1. Brain and Mind Institute, Western University, London, Canada
    2. Department of Clinical Neurological Sciences, Schulich School of Medicine and Dentistry, Western University, London, Canada
    Contribution
    Data curation, Formal analysis, Methodology, Resources, Software, Supervision, Validation, Writing - review and editing
    Competing interests
    No competing interests declared
  9. Lyle Muller

    1. Department of Mathematics, Western University, London, Canada
    2. Brain and Mind Institute, Western University, London, Canada
    Contribution
    Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Resources, Software, Supervision, Validation, Writing - original draft, Writing - review and editing
    For correspondence
    lmuller2@uwo.ca
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-5165-9890

Funding

Canada First Research Excellence Fund (BrainsCAN at Western University)

  • Lyle Muller

National Science Foundation (NeuroNex Grant No. 2015276)

  • Lyle Muller

Fields Institute for Research in Mathematical Sciences (Postdoctoral Fellowship)

  • Maryam H Mofrad

Compute Canada (Resource Allocation Competitions (RAC))

  • Lyle Muller

Kyoto University (SPIRITS 2020)

  • Lyle Muller

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Acknowledgements

The authors would like to thank the entire team of clinical neurologists and neurosurgeons at London Health Sciences Centre (LHSC, London, Ontario) for their collaboration in this work. The authors would also like to thank the clinical patients for their participation in the research. The authors additionally thank Ingrid Johnsrude, Julio Martinez-Trujillo, and Laura Batterink for helpful comments and discussions on this manuscript. This work was supported by the Canadian Institute for Health Research and NSF (NeuroNex Grant No. 2015276) and BrainsCAN at Western University through the Canada First Research Excellence Fund (CFREF).

Ethics

We analyze a clinical dataset of intracranial electroencephalography (iEEG) recordings from 5 epileptic patients in the Epilepsy Monitoring Unit (EMU) at London Health Sciences Centre (LHSC). Patients were implanted using depth electrodes for the sole purpose of surgical evaluation. Informed consent and consent to publish were collected from the patients in accordance with local Research Ethics Board (REB) guidelines (Western University HSREB #106516 and Lawson REB #114420).

Senior Editor

  1. Timothy E Behrens, University of Oxford, United Kingdom

Reviewing Editor

  1. Demba Ba, Harvard University, United States

Reviewers

  1. Carmen Varela, Florida Atlantic University, United States
  2. Demba Ba, Harvard University, United States

Publication history

  1. Preprint posted: May 17, 2021 (view preprint)
  2. Received: November 23, 2021
  3. Accepted: May 27, 2022
  4. Version of Record published: June 29, 2022 (version 1)
  5. Version of Record updated: August 4, 2022 (version 2)

Copyright

© 2022, Mofrad et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 744
    Page views
  • 145
    Downloads
  • 0
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Maryam H Mofrad
  2. Greydon Gilmore
  3. Dominik Koller
  4. Seyed M Mirsattari
  5. Jorge G Burneo
  6. David A Steven
  7. Ali R Khan
  8. Ana Suller Marti
  9. Lyle Muller
(2022)
Waveform detection by deep learning reveals multi-area spindles that are selectively modulated by memory load
eLife 11:e75769.
https://doi.org/10.7554/eLife.75769

Further reading

    1. Computational and Systems Biology
    Zhuang Liu et al.
    Research Article

    MicroRNAs (miR), as important epigenetic control factors, reportedly regulate wound repair. However, our insufficient knowledge of clinically relevant miRs hinders their potential therapeutic use. For this, we performed paired small RNA and long RNA sequencing and integrative omics analysis in human tissue samples, including matched skin and acute wounds collected at each healing stage and chronic non-healing venous ulcers (VU). On the basis of the findings, we developed a compendium (https://www.xulandenlab.com/humanwounds-mirna-mrna), which will be an open, comprehensive resource to broadly aid wound healing research. With this first clinical, wound-centric resource of miRs and mRNAs, we identified 17 pathologically relevant miRs that exhibited abnormal VU expression and displayed their targets enriched explicitly in the VU gene signature. Intermeshing regulatory networks controlled by these miRs revealed their high cooperativity in contributing to chronic wound pathology characterized by persistent inflammation and proliferative phase initiation failure. Furthermore, we demonstrated that miR-34a, miR-424, and miR-516, upregulated in VU, cooperatively suppressed keratinocyte migration and growth while promoting inflammatory response. By combining miR expression patterns with their specific target gene expression context, we identified miRs highly relevant to VU pathology. Our study opens the possibility of developing innovative wound treatment that targets pathologically relevant cooperating miRs to attain higher therapeutic efficacy and specificity.

    1. Computational and Systems Biology
    2. Neuroscience
    Emmanuelle Bioud et al.
    Research Article

    To decide whether a course of action is worth pursuing, individuals typically weigh its expected costs and benefits. Optimal decision-making relies upon accurate effort cost anticipation, which is generally assumed to be performed independently from goal valuation. In two experiments (n = 46), we challenged this independence principle of standard decision theory. We presented participants with a series of treadmill routes randomly associated to monetary rewards and collected both ‘accept’ versus ‘decline’ decisions and subjective estimates of energetic cost. Behavioural results show that higher monetary prospects led participants to provide higher cost estimates, although reward was independent from effort in our design. Among candidate cognitive explanations, they support a model in which prospective cost assessment is biased by the output of an automatic computation adjusting effort expenditure to goal value. This decision bias might lead people to abandon the pursuit of valuable goals that are in fact not so costly to achieve.