1. Neuroscience
Download icon

Uncovering temporal structure in hippocampal output patterns

  1. Kourosh Maboudi
  2. Etienne Ackermann
  3. Laurel Watkins de Jong
  4. Brad E Pfeiffer
  5. David Foster
  6. Kamran Diba  Is a corresponding author
  7. Caleb Kemere  Is a corresponding author
  1. University of Michigan, United States
  2. University of Wisconsin-Milwaukee, United States
  3. Rice University, United States
  4. University of Texas Southwestern, United States
  5. University of California, Berkeley, United States
Research Article
  • Cited 0
  • Views 2,182
  • Annotations
Cite as: eLife 2018;7:e34467 doi: 10.7554/eLife.34467

Abstract

Place cell activity of hippocampal pyramidal cells has been described as the cognitive substrate of spatial memory. Replay is observed during hippocampal sharp-wave-ripple-associated population burst events (PBEs) and is critical for consolidation and recall-guided behaviors. PBE activity has historically been analyzed as a phenomenon subordinate to the place code. Here, we use hidden Markov models to study PBEs observed in rats during exploration of both linear mazes and open fields. We demonstrate that estimated models are consistent with a spatial map of the environment, and can even decode animals’ positions during behavior. Moreover, we demonstrate the model can be used to identify hippocampal replay without recourse to the place code, using only PBE model congruence. These results suggest that downstream regions may rely on PBEs to provide a substrate for memory. Additionally, by forming models independent of animal behavior, we lay the groundwork for studies of non-spatial memory.

https://doi.org/10.7554/eLife.34467.001

Introduction

Large populations of neurons fire in tandem during hippocampal sharp-waves and their accompanying CA1 layer ripple oscillations (Buzsáki, 1986). By now, multiple studies have shown that during many sharp-wave ripple-associated PBEs, hippocampal ‘place cells’ (O'Keefe and O’Keefe, 1976) fire in compressed sequences that reflect the firing order determined by the sequential locations of their individual place fields (Diba and Buzsáki, 2007; Foster and Wilson, 2006; Lee and Wilson, 2002; Nádasdy et al., 1999). While the firing patterns during active exploration are considered to represent the brain’s global positioning system and provide a substrate for spatial and episodic memory, instead it is the synchronized activity during PBEs that is most likely to affect cortical activity beyond the hippocampus (Buzsáki, 1989; Carr et al., 2011; Diekelmann and Born, 2010; Siapas and Wilson, 1998). Likewise, widespread activity modulation is seen throughout the brain following these sharp-wave ripple population bursts (Logothetis et al., 2012).

The literature on PBEs has largely focused on developing templates of firing patterns during active behavior and evaluating the extent to which these templates’ patterns are reprised during subsequent PBE. But what if the fundamental mode of the hippocampus is not the re-expression of place fields, but rather the PBE sequences during SWR? PBE sequences are enhanced during exploration of novel environments (Cheng and Frank, 2008; Foster and Wilson, 2006), they presage learning-related changes in place fields (Dupret et al., 2010), and appear to be critical to task learning (Ego-Stengel and Wilson, 2010; Girardeau et al., 2009; Jadhav et al., 2012). Here, we examine the information provided by CA1 and CA3 pyramidal neurons, the output nodes of the hippocampus, through the looking glass of PBE firing patterns.

We developed a technique to build models of PBE sequences strictly outside of active exploration and independent of place fields and demonstrate that this nevertheless allows us to uncover spatial maps. Furthermore, these models can be used to detect congruent events that are consistent with replay but without any explicit place cell template. Our technique therefore provides new possibilities for evaluating hippocampal output patterns in single-trial and other fast learning paradigms, where a reliable sequential template pattern is not readily available. Overall, our work suggests that a sequence-first approach can provide an alternative view of hippocampal activity that may shed new light on how memories are formed, stored, and recalled.

Results

Awake population burst events

We began by analyzing the activity of large numbers of individual neurons in areas CA1 and CA3 of the dorsal hippocampus as rats navigated linear mazes for water reward (linear track: n=3 rats, m=18 sessions; previously used by [Diba and Buzsáki, 2007]). Using pooled multiunit activity, we detected PBEs during which many neurons were simultaneously active. The majority of these events occurred when animals paused running (speed <5 cm/s, corresponding to 54.0%±20.1% sd of events) to obtain reward, groom, or survey their surroundings (Buzsáki et al., 1983), and were accompanied by SWR complexes, distinguished by a burst of oscillatory activity in the 150–250 Hz band of the CA1 LFP. Because we are interested in understandinginternally generated activity during PBEs, we included only these periods without active behavior, ensuring that theta sequences would not bias our results. While we identified active behavior using a speed criterion, we found similar results when we instead used a theta-state detection approach (not shown). We did not add any other restrictions on behavior, LFP, or the participation of place cells. We found that inactive PBEs occupied an average of 1.8% of the periods during which animals were on the linear track (16.9±15.1 s of 832.6±390.5 s). In comparison, classical Bayesian approaches to understand PBE activity require the 34.8% of time animals are running (speed >10 cm/s) on the track (254.4±106.6 s of832.6±390.5 s) to build models of place fields.

Learning hidden Markov models from PBE data

Activity during PBEs is widely understood to be internally-generated in the hippocampal-entorhinal formation, and likely to affect neuronal firing in downstream regions (Buzsáki, 1989; Chrobak and Buzsáki, 1996; Logothetis et al., 2012; Yamamoto and Tonegawa, 2017). Given the prevalence of PBEs during an animal’s early experience, we hypothesized that the neural activity during these events would be sufficient to train a machine learning model of sequential patterns—a hidden Markov model—and that thismodel would capture the relevant spatial information encoded in the hippocampus independent of exploration itself.

Hidden Markov models have been very fruitfully used to understand sequentially structured data in a variety of contexts. A hidden Markov model captures information about data in two ways. First, it clusters observations into groups (‘states’) with shared patterns. In our case, this corresponds to finding time bins in which the same sets of neurons are co-active. This is equivalent to reducing the dimension of the ensemble observations into a discretized latent space or manifold. Second, it models the dynamics of state transitions. This model is Markovian because it is assumed that the probability to transition to the next state only depends on the current state. Critically, these operations of clustering and sequence modeling are jointly optimized, allowing the structure of ensemble firing corresponding to each of the final states to combine information over many observations. Given the role of the hippocampus in memory, in our HMMs, the unobserved latent variable presumably corresponds to the temporal evolution of a memory trace that is represented by co-active ensembles of CA1 and CA3 neurons. The full model will correspond to the structure which connects all the memory traces activated during PBEs.

The parameters of our model that are fit to data include the observation model (the cluster descriptions, or predicted activity of each excitatory neuron within the CA1/CA3 ensemble for a given state), the state transition model (the probability that the CA1/CA3 ensemble will transition from a start state to a destination state in the next time bin), and the initial state distribution (the probability for sequences to start in each given state). In prior work using HMMs to model neural activity, a variety of statistical distributions have been used to characterize ensemble firing during a specific state (the observation model, (Chen and Wilson, 2017; Chen et al., 2012; Chen et al., 2014; Deppisch et al., 1994; Kemere et al., 2008; Radons et al., 1994). We opted for the Poisson distribution to minimize the number of parameters per state and per neuron (see Materials and methods). We used the standard iterative EM algorithm (Rabiner, 1989) to learn the parameters of an HMM from binned PBE data (20 ms bins). Figure 1 depicts the resultant state transition matrix and observation model for an example linear-track session.

Figure 1 with 1 supplement see all
A hidden Markov model of ensemble activity during PBEs.

A hidden Markov model of ensemble activity during PBEs. (a) Examples of three PBEs and a run epoch. (b) Spikes during seven example PBEs (top) and their associated (30 state HMM-decoded) latent space distributions (bottom). The place cells are ordered by their place fields on the track, whereas the non-place cells are unordered. The latent states are ordered according to the peak densities of the lsPFs (lsPFs, see Materials and methods). (c) The transition matrix models the dynamics of the unobserved internally-generated state. The sparsity and banded-diagonal shape are suggestive of sequential dynamics. (d) The observation model of our HMM is a set of Poisson probability distributions (one for each neuron) for each hidden state. Looking across columns (states), the mean firing rate is typically elevated for only a few of the neurons and individual neurons have elevated firing rates for only a few states.

https://doi.org/10.7554/eLife.34467.002

Using separate training- and test-datasets (cross-validation) mitigates over-fitting to training data, but it is still possible for the cross-validated goodness-of-fit to increase with training without any underlying dynamics, e.g., if groups of neurons tend to activate in a correlated fashion. Does the model we have learned reflect underlying sequential structure of memory traces beyond pairwise co-firing? To answer this question, we cross-validated the model against both real ‘test’ data and against surrogate ‘test’ data derived from shuffling each PBE in two ways: one in which the binned spiking activity was circularly permuted across time for each neuron independently of the other neurons (‘temporal shuffle’, which removes co-activation), and one in which the order of the binned data was scrambled coherently across all neurons (‘time-swap’, which maintains co-activation). Note that the second shuffle preserves pairwise correlations while removing the order of any sequential patterns that might be present. Using five-fold cross-validation, we compared learned models against both actual and surrogate test data and found that the model likelihood was significantly greater for real data (vs. temporal shuffle, p<0.001, vs. time-swap, p<0.001, n=18 sessions, Wilcoxon signed-rank test, Figure 1—figure supplement 1).

What do the learned model parameters tell us about PBEs?

To begin to understand what structure we learn from PBE activity, we compared our HMMs (trained on real data) against models trained on multiple different surrogate datasets (Figure 2a,b). These surrogate datasets were obtained from actual data following: (1) temporal shuffles and (2) time-swaps, as above, and (3) by producing a surrogate PBE from independent Poisson simulations according to each unit’s mean firing rate within the original PBEs. First, we investigated the sparsity of the transition matrices using the Gini coefficient (see Materials and methods and Figure 2—figure supplement 1). A higher Gini coefficient corresponds to higher sparsity. Strikingly, the actual data yielded models in which the state transition matrix was sparser than in each of the surrogate counterparts (p<0.001, Figure 2c), reflecting that each state transitions only to a few other states. Thus, intricate yet reliable details are captured by the HMMs. Next, we quantified the sparsity of the observation model. We found that actual data yielded mean firing rates which were highly sparse (Figure 2d), indicating that individual neurons were likely to be active during only a small fraction of the states. Using a graph search algorithm (see Materials and methods), we simulated paths through state space generated by these transition matrices, and found that this increased sparsity accompanied longer trajectories (Figure 2—figure supplement 3) through the state space of the model. Thus, the state transition matrices we learn are suggestive of dynamics in which each sparse state is preceded and followed by only a few other, in turn, sparse states, providing long sequential paths through state space-consistent with spatial relationships in the environment in which the animal was behaving, but generated from PBEs. The increased sparsity of the observation model and transition matrix in the example session was representative of a significant increase over all remaining sessions (p<0.05, n=18 sessions, Wilcoxon signed-rank tests, Figure 2e,f).

Figure 2 with 3 supplements see all
A hidden Markov model of ensemble activity during population burst events.

Models of PBE activity are sparse. We trained HMMs on neural activity during PBEs (in 20 ms bins), as well as on surrogate transformations of those PBEs. (a) (top) The transition matrices for the actual and surrogate PBE models with states ordered to maximize the transition probability from state i to state i+1. (bottom) Undirected connectivity graphs corresponding to the transition matrices. The nodes correspond to states (progressing clockwise, starting at the top). The weights of the edges are proportional to the transition probabilities between the nodes (states). The transition probabilities from state i to every other state excepti+1 are shown in the interior of the graph, whereas for clarity, transition probabilities from state i to itself, as well as to neighboring state i+1 are shown between the inner and outer rings of nodes (the nodes on the inner and outer rings represent the same states). (b) The observation matrices for actual and surrogate PBE models show the mean firing rate for neurons in each state. For visualization, neurons are ordered by their firing rates. (c) We quantified the sparsity of transitions from one state to all other states using the Gini coefficient of rows of the transition matrix for the example session in (a). Actual data yielded sparser transition matrices than shuffles. (d) The observation models—each neuron’s expected activity for each state—learned from actual data for the example session are significantly sparser than those learned after shuffling. This implies that as the hippocampus evolves through the learned latent space, each neuron is activeduring only a few states. (e) Summary of transition matrix sparsity and f. Observation model sparsity with corresponding shuffle data pooled over all sessions/animals. (***: p<0.001, *: p<0.05; single session comparisons: n=250realizations, Welch’s t-test; aggregated comparisons - n=18 sessions, Wilcoxon signed-rank test).

https://doi.org/10.7554/eLife.34467.004

These observations indicate that PBEs inform an HMM about extant spatial relationships within the environment. So, next we asked how the firing patterns of neurons during actual behavior project into the learned latent spaces. To observe the evolution of the latent states during behavior, we used our model to determine the most likely sequence of latent states corresponding to decode the neural activity observed in 100 ms bins during epochs that displayed strong theta oscillations (exclusive of PBEs) when rats were running (speed >10 cm/s; see Materials and methods). If the learned model was distinct from ensemble patterns during behavior, we might expect the resulting state space probability distributions at each point in time to be randomly spread among multiple states. Instead, we found distributions that resembled sequential trajectories through the latent space (Figure 3a) in parallel with the physical trajectories made by the animal along the track, further demonstrating that the latent state dynamics learned from PBEs corresponds to an internalized model of physical space.

Figure 3 with 1 supplement see all
Latent states capture positional code.

Latent states capture positional code. (a) Using the model parameters estimated from PBEs, we decoded latent state probabilities from neural activity during periods when the animal was running. An example shows the trajectory of the decoded latent state probabilities during six runs across the track. (b) Mapping latent state probabilities to associated animal positions yields latent-state place fields (lsPFs) which describe the probability of each state for positions along the track. (c) Shuffling the position associations yields uninformative state mappings. (d) For an example session, position decoding during run periods through the latent space gives significantly better accuracy than decoding using the shuffled tuning curves. The dotted line shows the animal’s position during intervening non run periods. (e) The distribution of position decoding accuracy over all sessions (n=18) was significantly greater than chance. (p<0.001).

https://doi.org/10.7554/eLife.34467.008

To better understand the relationship between the latent space and physical space, we used the latent state trajectories decoded during running to form an estimate of the likelihood of each state as a function of location on the track (see Materials and methods). These lsPF ‘lsPFs’ (lsPF, Figure 3b) in many ways resembled neuronal place fields and similarly tiled the extent of the track. This spatial localization went away when we re-estimated the lsPF with shuffled positions (Figure 3c). To quantify how informative the latent states were about position, we used the lsPF to map decoded state sequences to position during running periods (Figure 3d). In our example session, decoding through the latent space resulted in a median accuracy of 5 cm, significantly greater than the 47 cm obtained from shuffled lsPF (p<0.001, Wilcoxon signed-rank test, Figure 3d). When we evaluated decoding error over our full set of sessions, we observed a similar result (p<0.001, Wilcoxon signed-rank test, Figure 3e, Figure 3—figure supplement 1). As our method required discretizing the state space, a potential caveat is that the number of latent states is a relevant parameter, which we arbitrarily chose to be 30. However, latent-state place fields were informative of position over a wide range of values of this parameter (Figure 3—figure supplement 1). Note that decoding into the latent space and then mapping to position resulted in slightly higher error than simply performing Bayesian decoding on the neural activity during behavior. This suggests that the latent space we learn from PBEs may not capture all the information about space that is present in hippocampal activity during behavior, though this may also reflect the limited number of PBEs from which we can learn.

HMM-congruent PBEs capture sequence replay

We and others have previously described how the pattern of place cell firing during many PBEs recapitulates the order in which they are active when animals run on the track (Figure 4a). We employed the versatile and widely-used Bayesian decoding method to ascribe a replay score to sequential patterns during PBEs. Briefly, for each PBE, we used place-field maps to estimate a spatial trajectory (an a posteriori distribution of positions) in 20 ms bins. We generated surrogate data via a column-cycle shuffle (i.e., a circular shift across positions for each time bin [Davidson et al., 2009]) of the a posteriori distributions during PBEs. The real and surrogate trajectories were scored (see Materials and methods), and we defined replay events as those for which the score of the actual trajectory was larger than a threshold fraction of the null distribution generated by the surrogate scores. Using this approach, we found that 57% of PBE (1064 of 1883) were identified as replay beyond a threshold of 99% (median across datasets 54.2%, interquartile range=32.861.0%, Figure 4—figure supplement 1). Thus, as has been reported many times (Davidson et al., 2009; Diba and Buzsáki, 2007; Foster and Wilson, 2006; Karlsson and Frank, 2009), only a fraction of PBEs (but many more than expected by chance) represent statistically significant replay. Given that we use all PBEs for model learning and our models capture the structure of the environment and the patterns expressed by place cells during exploration, we were interested in understanding whether we could also use our latent-space models to find these replay events. Indeed, for many events when we decode trajectories through state space, they resemble the sequential patterns observed when we decode position using Bayesian techniques and the place cell map (Figure 4b, left). However, given previous evidence for replay of environments not recently experienced (Gupta et al., 2010; Karlsson and Frank, 2009), we hypothesized that some PBEs might contain ensemble neural activity which is unstructured and thus unrelated to the learned model, and that these would correspond to the ‘non-replay’ events found using traditional methods.

Figure 4 with 1 supplement see all
Replay Events Can Be Detected Via HMM Congruence.

(a) Example PBEs decoded to position using Bayesian decoding. (b) (left) Same examples decoded to the latent space using the learned HMM. (right) Examples decoded after shuffling the transition matrix, and (middle) the sequence likelihood using actual and shuffled models. (c) Effect of significance threshold on the fraction of events identified as replay using Bayesian decoding and model congruent events using the HMM approach. (d) Comparing Bayesian and model-congruence approaches for all PBEs recorded, we find statistically significant agreement in event identification (60.9% agreement, n=1883 events from 18 sessions, p<0.001, Fisher’s exact test two sided).

https://doi.org/10.7554/eLife.34467.010

To assess how well the pattern of ensemble activity during individual PBEs related to the overall state-space model learned from PBE activity (‘congruence’), we developed a statistical approach for identifying the subset of strongly structured PBEs. Specifically, rather than comparing real and surrogate PBEs, we compared the goodness-of-fit for each event to a null distribution generated via a computationally-efficient manipulation of the transition matrix of the model (Figure 4b); we row-wise shuffled the non-diagonal elements of the transition matrix to assess whether an individual PBEs is a more ordered sequence through state space than would be expected by chance. Maintaining the diagonal avoids identifying as different from chance sequences which consist of few repeated states, marked by transitions between state i and itself. As described above, the fraction of events identified as replay using Bayesian decoding is strongly tied to how the null-distribution is generated (i.e., what shuffle is used), some secondary criteria (e.g., number of active cells, unit cluster quality, peak firing rate, trajectory ‘jumps’, etc.), and the value of the significance threshold arbitrarily chosen to be 90%, 95%, or 99% of shuffles in different reports. When we combined across datasets, we found that our transition matrix shuffle yielded a null distribution for which a 99% confidence interval identified slightly fewer PBEs as significant than the column-cycle shuffle did for Bayesian decoding (Figure 4c). To make a principled comparison of Bayesian- and HMM-based replay detection schemes, we fixed the Bayesian-based significance threshold at 99% but selected the significance threshold for the HMM-congruence null distribution so that the fraction of replay events detected would be the same between the two schemes. Following this approach, we found that model-congruent/incongruent PBEs largely overlapped with the replay/non-replay events detected using Bayesian decoding of the place cell map (Figure 4d). Thus, using only the neural activity during PBEs, without access to any place cell activity, we are remarkably able to detect the sequential patterns typically described as ‘replay’ based only on their consistency with the structure of other PBE activity.

There were, however, also differences between the Bayesian and HMM-congruent approaches, including events that reached significance in one but not the other formalism. We wanted to understand where and why these approaches differed in identifying significant sequences. When we examined individual PBEs, we found sequences for which both Bayesian and model-congruence replay detection approaches appeared to malfunction (Figure 5a). This was not a failure of the choice of significance threshold, as for both techniques we found what appeared to be false-negatives (patterns which looked like replay but were not significant) as well as false-positives (patterns which looked noisy but were identified as replay). Thus, in order to quantitatively compare the two approaches, we asked eight humans to visually examine all the PBEs in our database. They were instructed to label as replay PBEs in which the animal’s Bayesian decoded position translated sequentially without big jumps (Silva et al., 2015; see Materials and methods).

Figure 5 with 1 supplement see all
Comparing HMM congruence and Bayesian decoding in replay detection.

(a) Eight examples from one session show that Bayesian decoding and HMM model-congruence can differ in labeling of significant replay events. For each event, spike rasters (ordered by the location of each neuron’s place field) and the Bayesian decoded trajectory are shown. ‘+' (‘-') label corresponds to significant (insignificant) events. (left) Both methods can fail to label events that appear to be sequential as replay and (right) label events replay that appear non-sequential. (b) We recruited human scorers to visually inspect Bayesian decoded spike trains and identify putative sequential replay events. Using their identifications as labels, we can define an ROC curve for both Bayesian and HMM model-congruence which shows how detection performance changes as the significance threshold is varied. (inset) Human scorers identify 24% of PBEs as replay. Setting thresholds to match this value results in agreement of 70% between Bayesian and HMM model-congruence. (c) Using the same thresholds, we find 70% agreement between algorithmic and human replay identification. (All comparison matrices, p<0.001, Fisher’s exact test two-tailed.).

https://doi.org/10.7554/eLife.34467.012

We marked each event as a ‘true’ community replay if it was identified by a majority of scorers (six individuals scored n=1883 events, two individuals scored a subset of n=1423 events, individual scores are shown in Figure 5—figure supplement 1). We calculated an ROC curve which compared the rate of true positive and false positive detections as the significance thresholds for Bayesian and model-congruence approaches were varied (Figure 5b). A perfect detector would have an AUC of unity. We did not find a significant difference between the AUC of Bayesian decoding and model-congruence (p=0.14, bootstrap, see Materials and methods). If we select thresholds such that our algorithms yield a similar fraction of significant vs. total events as the 24% denoted by our human scorers, we find that both Bayesian and model-congruence yield agreement of 70% labeled events with each other and with human scorers (Figure 5c).

Thus, congruence with an HMM trained only on PBEs appears to work as reliably as Bayesian decoding in detecting sequential reactivation of linear track behaviors. However, when we examined individual sessions, we noticed that performance was quite variable. Given that our models are learned only from PBEs, we reasoned that the statistics or structure of the PBEs within each session might yield models which vary in quality depending on the number of recorded units, the number of PBEs detected, and their self-consistency across events. We created a model quality metric by comparing cross-validated learning statistics to models which were learned from shuffled events (see Materials and methods). We found that the performance of model-congruence detection was tied to model quality (R2=0.17, F=2.9, n=18 sessions, Figure 5—figure supplement 1). Model quality, in turn, was highly correlated with the number of PBEs during the session (R2=0.96, F=392.6, n=18 sessions, Figure 5—figure supplement 1). Not surprisingly, the performance of Bayesian decoding relative to human scorers was independent of the quality of the HMM, or the number of PBEs, as the place field model is learned from ensemble neural activity during running. Thus, we find an intriguing contrast—when there is an abundance of PBEs (indicating novelty, learning, hippocampus-dependent planning, etc. [Buzsáki, 2015]), even in the absence of repeated experience, replay detection based on PBE activity is highly effective. Conversely, when there are few PBEs (i.e., scenarios in which PBEs are uncorrelated with cognitive function), but an abundance of repeated behavioral trials, Bayesian decoding of these limited events proves more effective.

Modeling internally generated activity during open field behavior

The linear track environment represents a highly-constrained behavior. We therefore asked whether the HMM approach could generalize to more complex environments and behavioral tasks. (Pfeiffer and Foster, 2013, Pfeiffer and Foster, 2015) had previously recorded activity of CA1 neurons in rats as they explored in a 2 m ×2 m open field arena for liquid reward. Briefly, animals were trained to discover which one of 36 liquid reward wells would be the ‘home’ well on a given day. Then, they then were required to alternate between searching for a randomly rewarded well and returning to the home well. Using the place cell map in this task and Bayesian decoding, many PBEs were decoded to trajectories through two-dimensional space that were predictive of behavior and shaped by reward. Using this same dataset, we trained a HMMs on neural activity during PBEs in the open field. Here, we used the same PBE detected previously (Pfeiffer and Foster, 2013; Pfeiffer and Foster, 2015) which occupied an average of 2.53±0.42% of the periods during which animals were behaving (77.91±21.16 s out of 3064.86±540.26 s). Given the large number of units available in this dataset and the increased behavioral variability in the open field environment compared to the linear track, we chose to estimate HMMs with 50 latent states. The transition matrix and observation model from a sample session are shown in Figure 6a,b. Despite the complex and varied trajectories displayed by animals, the HMM captured sequential dynamics in PBE activity, as in the 1D case, when we compared learned models against both actual and surrogate test data, we found that the model likelihood was significantly greater for real data (p<0.001, Wilcoxon signed-rank test).

Figure 6 with 4 supplements see all
Modeling PBEs in open field.

Modeling PBEs in open field. (a) The transition matrix estimated from activity detected during PBEs in an example session in the open field. (b) The corresponding observation model (203 neurons) shows sparsity similar to the linear track. (c) Example latent state place fields show spatially-limited elevated activity in two dimensions. (d) For an example session, position decoding through the latent space gives significantly better accuracy than decoding using the shuffled latent state place fields. (e) Comparing the sparsity of the transition matrices (mean Gini coefficient of the departure probabilities) between the linear track and open field reveals that, as expected, over the sessions we observed, the open field is significantly less sparse (p<0.001), since the environment is less constrained. (f) In contrast, there is not a significant difference between the sparsity of the observation model (mean Gini coefficient of the rows) between the linear track and the open field. Note that the linear track models are sparser than in Figure 2 due to using 50 states rather than 30 to match the open field.

https://doi.org/10.7554/eLife.34467.014

In the case of the linear track, we linked sparsity of the transition matrix to the sequential nature of behaviors in that environment. An unconstrained, two-dimensional environment permits a much richer repertoire of behavioral trajectories. However, behavior is still constrained by the structure of space—arbitrary teleportation from one location to another is impossible. We found that learning from PBEs in the open field yielded transition matrices (Figure 6a) that were significantly sparser thanmodels learned from shuffled data (p<0.05, Wilcoxon signed-rank test, n=8 sessions, Figure 5—figure supplement 1). However, consistent with increased freedom of potential behaviors, when we compared the sparsity of models learned from open field acpPBEs with 50-state models learned from PBEs in linear tracks, the open field transition matrices were less sparse (p<0.001, Mann–Whitney U test comparing 8 and 18 sessions, Figure 4—figure supplement 1). Likewise, when we examined the observation model for the open field, we found that the activity across states for individual neurons was significantly more sparse than in models learned from shuffled data (p<0.05, Wilcoxon signed-rank test, n=8 sessions, Figure 6—figure supplement 1). The sparsity of linear track and open field observation models were not significantly different (p=0.44, Mann–Whitney U test).

Do the latent states learned from PBEs capture spatial information in a 2D environment? We used the PBE-trained model to decode run data, as in the linear track case. We found that the latent states corresponded with specific locations in the open field, as we expected (Figure 6c). Moreover, we were able to decode animals’ movements with significantly greater than chance accuracy by converting decoded latent states to positions using the lsPF (p<0.001, Figure 6d). Finally, we examined model-congruency for PBEs detected in the open field. Previously, it was reported that 27.3% (815 of 2980, n=8 sessions) were identified as ‘trajectory events’ (Pfeiffer and Foster, 2015). We chose a significance threshold to match this fraction (Figure 6—figure supplement 3) and found that there was significant overlap between the events detected through Bayesian and model-congruence techniques (p<0.01, Fisher’s exact test). These events overlapped significantly with replay events detected using traditional Bayesian decoding (Figure 6—figure supplement 3). Thus, an HMM of the activity during population bursts captures the structure of neural activity in two dimensional environments during complex tasks and can be used to decode events consistent with trajectories through that environment.

Extra-spatial information

As described earlier, while we observed a similar fraction of events to be similar by HMM-congruence and Bayesian decoding, there was not an exact event-to-event correspondence. An intriguing potential explanation is that the latent space represented in PBE sequential firing and captured by the HMM is richer than simply the spatial structure of the present environment. In most hippocampal ensemble recording experiments, maze or open field tasks are structured to intentionally map memory elements to spatial behavior, and thus this potential richness is difficult to test. We used two sample datasets to explore the potential of the HMM to capture extra-spatial richness in the PBE sequences.

First, we considered the possibility that in the awake behaving animal, PBE activity might be sequential reactivation of environments other than the one being explored (‘remote replay’). We reasoned that we could enhance the model’s representation of remote environments by filtering out local replay from the training data. We evaluated how the model-quality of our HMM changed as progressively more sequences labeled as replay by Bayesian decoding were removed from the training data. In the linear track sessions we considered, we found that refining the training data resulted in models that lowered in quality at different rates as the threshold for Bayesian replay was decreased (Figure 7). Most, but not all, models dropped precipitously in quality: >50% when we removed events detected as Bayesian replay at a 95% threshold, as would be expected if the HMM represented only the local environment. In many outlier sessions in which model quality decreased more slowly, the initial (baseline)model quality was low. Intriguingly, however, in at least one outlier session where model quality decreased slowly with refinement (blue line,Figure 7a), the initial model quality was still high, and we further noted that position decoding using lsPF yielded relatively high error (blue dot, Figure 7b). Thus, we wondered whether this and similar sessions might have contained non-local or extra-spatial PBEs that were captured by the HMM.

Extra-spatial structure.

Examples of remote replay events identified with HMM-congruence. We trained and evaluated HMMs on the events that were not Bayesian significant (residual events) to identify potential extra-spatial structure. (a) The normalized session quality drops as local-replay events above the Bayesian significance threshold are removed from the data. Each trace corresponds to one of the 18 linear track sessions, with the stroke width and the stroke intensity proportional to the baseline (all-events) session quality. The blue line identifies a session in which model quality drops more slowly, indicating the potential presence of extra-spatial information. The reduction in session quality for a W maze experiment with known extra-spatial information is even slower (green). When, instead, Bayesian-significant remote events are removed, rapid reduction in session quality is again revealed (red). (b) The lsPF-based median decoding errors are shown as a function of baseline session quality for all 18 linear track sessions. The blue dot indicates the outlier session from panel (a) with potential extra-spatial information: this session shows high decoding error combined with high session quality. Session quality of the W maze session is also indicated on the x-axis (decoding error is not directly comparable). (c–n) Two example HMM-congruent but not Bayesian-significant events from the W maze session are depicted to highlight the fact that congruence can correspond to remote replay. (c) Spikes during ripple with local place cells highlighted (top panel) and the corresponding latent state probabilities (bottom panel) decoded using the HMM show sequential structure (grayscale intensity corresponds to probability). (d) In this event, the Bayesian score relative to the shuffle distribution (top panel) indicates that the event is not-significant, whereas the HMM score relative to shuffles indicates (bottom panel) the ripple event is HMM-congruent. (e) Estimates of position using local place fields show jumpy, multi-modal a posteriori distributions over space in 1D (top left panel) and 2D (top right panel; distribution modes and time is denoted in color). Bayesian decoding using the remote environment place fields (bottom panel) indicates that the sample event is a remote replay. Note that in a typical experiment, only the local place fields would be available. (f–h) Same as (c–e), but for a different ripple event.

https://doi.org/10.7554/eLife.34467.019

In order to validate the concept of model-training refinement, we considered a dataset in which multiple environments were explored on the same day and remote replay was previously observed (Karlsson et al., 2015). These data consisted of a series of short exploratory sessions in which an animal first explored a novel maze (E1) and then was placed in a familiar one (E2). We identified awake PBEs during the familiar E2 session and used them to train an HMM. When we refined this model by removing Bayesian-significant local replay events from the training data, we found that the model quality decreased comparatively slowly (Figure 7a, green line), indicating that the HMM was capturing more than the local spatial structure. In contrast, when we used place fields from E1 to identify Bayesian-significant remote replay events and removed these from the training data, we found that the model quality decreased rapidly as with the general linear track cases (Figure 7a, red line). When we examined individual events in detail in this data, we found many examples in which HMM-significant, Bayesian non-significant PBEs decoded to extended state sequences which turned out to correspond to reactivation of the remote track (two are shown in Figure 7c–l). If we imagine that in this experiment data were only recorded during exploration of the familiar environment, classical Bayesian decoding would treat these events as noise, as shown in the bottom half of the two examples. In contrast, our HMM-based analysis finds these events to be significant, as shown in the top half of the two examples. Thus, by combining classical Bayesian decoding and HMM-congruence, we are able to identify a signature of when a HMM trained on PBEs captures sequential structure distinct from that dictated by the local environment. Additionally, in these cases, we show that specific non-local reactivation events can be identified.

Finally, we considered the potential of our methodology for uncovering temporal patterns in PBE activity under scenarios where complex behavior does not permit identification of well-defined place-fields or in the absence of behavior, such as during sleep. As we have emphasized, a remarkable aspect of learning HMMs from PBE activity is that the model can be built entirely without behavioral data, so can our model capture significant sequential information outside of immobility periods during quiet waking? To demonstrate this potential, we examined HMMs trained on PBEs in sleep following the learning phase of an object-location memory task when animals explored three objects in an open field (see Material and methods). Previous studies have demonstrated that subsequent recall of this memory is hippocampus-dependent, and requires consolidation in post-task sleep (Prince et al., 2014; Inostroza et al., 2013). However, while this task involves spatial exploration of objects in an arena, whether the subsequent post-task sleep contains sequential structure and whether object memory is contained in this code has remained elusive (Larkin et al., 2014). In order to assess the presence of sequential structure in the PBEs, we first used cross validation to generate a distribution of sequence HMM-congruence scores. For each set of test PBEs, we also generated surrogates by shuffling time bins across events (pooled time-swap). Using our HMM-congruence score which explicitly tests for sequences through state space, the large difference between actual and shuffled score distributions indicates evidence for significant sequential structure in the PBEs (p<0.001, Mann–Whitney U test, Figure 8). While more work is needed to evaluate the mnemonic relevance of these HMM-congruent sequences, these data support the notion that the HMM can uncover sequential activity in sleep away from the task environment. This approach further demonstrates the utility of the HMM approach as an initial analysis of a novel dataset, or as a way of comparing the sequential content encoded in PBEs during different periods.

Temporal structure during a sleep period following object-location memory task.

Using cross validation, we calculate the HMM-congruence score (which ranges from 0 to 1) for test PBEs. For each event, we also calculate the score of a surrogate chosen using a pooled time-swap shuffle across all test events. The distribution of scores of actual events is significantly higher than that of the surrogate data (p<0.001, Mann–Whitney U test).

https://doi.org/10.7554/eLife.34467.020

Discussion

Increasing lines of evidence point to the importance of hippocampal ensemble activity during PBEs in guiding on-going behavior and active learning. Despite being the strongest output patterns of the hippocampus, however, this activity has been assumed to be interpretable only in the context of other theta-associated place cell activity expressed during behavior. Our findings demonstrate that over the course of a behavioral session, ensemble activity during PBEs alone is sufficient to form a model which captures the spatial relationships within an environment. This suggests that areas downstream of the hippocampus might be able to make use solely of PBE activity to form models of external space. In an extreme view, place cell activity might merely subserve the internal mechanisms in the hippocampus which generate PBE sequences. To the extent that animals might wish to use the spatial code obtained from PBEs to identify their current location, we show that this can be done after translating ensemble activity into the latent states of the model. Do the PBEs contain ‘full information’ about the environment? Bayesian decoding of location from place cell activity results in lower error than location estimates generated using the latent states and lsPF. This suggests that the manifold defined by the HMM may not capture all the dimensions of information represented during exploration. However, it is possible that with more PBE data, we would learn a more refined state space. Thus, the difference between the latent space represented during behavior and within PBEs may be an interesting focus of future study.

When we examined the transition matrices we learned from PBEs, we found that they were marked by significant sparsity. This sparsity results from the sequential patterns generated during PBEs. Latent variable models have previously been used to analyze the structure of hippocampal place cell activity (Chen et al., 2012; 2014; Dabaghian et al., 2014). In these studies, the learned transition matrices were mapped to undirected graphs which could be analyzed using topological measures. It is intriguing that similar structure is apparent in PBE activity. For example, we observed that transition matrices learned from PBEs associated with linear track behavior were significantly sparser than those learned from the open field, which we hypothesize is a consequence of the greater freedom of behavior in the latter (a topological difference). Whether hippocampal PBE activity must always be sequential, i.e., evolve through a sparsely-connected latent space, is an open and interesting question, as are differences between the latent state space dynamics learned during PBEs and those learned from place cell activity.

Graded, non-binary replay detection

Remarkably, evaluating the congruence or likelihood of test data against our HMM provided a highly novel method to detect events that are consistent with replay, without a need to access the ‘play’ itself. In the process of evaluating the potential of HMMs for detecting replay, we developed an approach to compare different replay-detection strategies. Our results highlight how the data do not readily admit to a strict separation between ‘replay’ and ‘non-replay’ events. While it is possible that with additional shuffles or other restrictions (Silva et al., 2015), automated performance might be rendered closer to human-labeling, even human scorers had variation in their opinions. This calls into doubt judgments of memory-related functions which build on a binary distinction between replay and non-replay sequences. Model congruence, either as a raw statistical likelihood or weighted against a shuffle distribution, seems to be a very reasonable metric to associate with individual PBEs. Moreover, evaluating congruence with an HMM does not require access to repeated behavioral sequences, which may be infeasible under widely-used single- or few-trial learning paradigms or when the events involve replay of a remote internalized environment. Given these benefits, along with computational efficiency, we would suggest that future analyses of the downstream impact of hippocampal reactivation regress effects against this measure rather than assuming a binary distinction.

Learning, Model Congruence and Replay Quality

Not surprisingly, the rate of PBEs had a large effect on our ability to measure model congruence. Interestingly, it has been noted that the density of PBE is higher during early exposure to a novel environment (Cheng and Frank, 2011; Frank et al., 2004; Kemere et al., 2013; Kudrimoti et al., 1999). This might suggest that for the animal, PBE activity could be an important source for generating models of the world when the animal is actively learning about the environment. If as hypothesized, replay is a form of rehearsal signal generated by the hippocampus to train neocortical modules (McClelland et al., 1995; Buzsáki, 1989), then indeed the brain’s internal machinery may also be evaluating whether a given sequential PBE pattern is congruent and consistent with previously observed PBEs. In later sessions, as animals have been repeatedly exposed to the same environments, downstream regions will have already witnessed many PBEs from which to estimate the structure of the world. Overall, our approach provides a novel viewpoint from the perspective of hippocampal PBEs. An interesting future line of inquiry would be to assess the extent to which a model built on PBEs during first experience of a novel environment is slower or faster to converge to the final spatial map than models built on theta-associated place activity.

Application to Extra-spatial behaviors

We have analyzed data gathered in experiments in which rats carried out simple spatial navigation tasks. Thus, to some extent it is not surprising that when we decoded ensemble activity during behavior we found that spatial positions the animal is exploring are strongly associated with the latent states.

We anticipate that our approach for calculating lsPF would be equally useful in tasks in which the hippocampal map is organized around time (Eichenbaum, 2014; Rodriguez and Levy, 2001) or other continuous variables (e.g. sound frequency [Aronov et al., 2017]). Our two proof-of-concept analyses, however, suggest that it should be possible to use HMMs to infer the presence of extra-spatial sequential reactivation in PBEs. For example, we showed that there is significant sequential structure during sleep after an animal explores novel objects in an environment. We anticipate that careful experimental design and further algorithmic development would allow for the conjunctive coding of object identity and spatial locations to be detected in the latent states we learn from PBEs, with model-congruence providing a tool to study sequential hippocampal reactivation in these types of tasks.

Conjunctive, non-spatial information might be one source of the apparent variability that results in many PBEs not being detected as replay using traditional Bayesian decoding. Another proposed source of this variability is reactivation of other environments. Our second proof-of-concept analysis suggests that HMMs learned from PBEs can, in fact, capture the spatial structure of environments beyond the one the animal is currently exploring. It appears that it should be possible to use only the PBEs and information about the place-cell map of the local environment to refine the training set for remote replay activity and learn the structure of a remote environment. While we used Bayesian decoding to detect putative local replays, we anticipate related approaches might use an HMM or other approaches to model local place cell activity.

Future possibilities

It has been previously observed that the rate of hippocampal reactivations in PBEs during awake behavior is much higher than during sleep (Grosmark and Buzsáki, 2016; Karlsson and Frank, 2008), but the reasons for this are not well understood. One hypothesis is that many sleep PBEs contain the reactivation of contexts other than those measured during a behavioral experiment. Another hypothesis is that sleep activity involves remodeling of dynamic network architectures (Buhry et al., 2011; Tononi and Cirelli, 2014). Our approach has the potential to illuminate some sources of variability during sleep. While we have given preliminary evidence that information about a remote context can be present in PBEs along with the local context, further work is required to understand how our model’s ability to capture this structure scales with the number of different contexts. With sufficient data, our HMM approach should be able to learn disjoint sets of latent states (or ‘sub-models’) which would capture these separate contexts and allow us to test this possibility. Alternatively, sleep PBEs could yield models which represent a known behavioral context but are markedly different (e.g., less sparse) than those learned from awake PBEs. This might support the network remodeling function of sleep. In the latter case, we might imagine that only a small subset of sleep PBEs—corresponding to learning-related replay—would be congruent with a model learned from awake PBE data.

Conclusions

We have demonstrated a new analytical framework for studying hippocampal ensemble activity which enables primacy of PBEs in model formation. We use an unsupervised learning technique commonly used in the machine learning field to study sequential patterns, the hidden Markov model. This contrasts with existing approaches in which the model—estimated place fields for the ensemble—is formed using the theta-associated place cell activity. We find that our PBE-first approach results in a model which still captures the spatial structure of the behavioral tasks we studied. Additionally, we demonstrate that we can use model-congruence as a tool for assessing whether individual PBEs contain hippocampal replay. Finally, we present proofs-of-concept that this analytical approach can detect the presence of sequential reactivation in experimental scenarios in which existing approaches are insufficient. Thus, the use of unsupervised learning of latent variable models—specifically HMMs and statistical congruence as a marker of individual events—bears much promise for expanding our ability to understand how PBEs enable the cognitive functions of the hippocampus.

Materials and methods

Experiment paradigm/Neural data recording

We analyzed neural activity recorded from the hippocampus of rats during periods in which they performed behavioral tasks in different paradigms. First, we considered data from animals running back and forth in a linear track 150 or 200 cm long. As previously reported using these same data (Diba and Buzsáki, 2007), we recorded neural activity using chronically-implanted silicon probes to acquire the activity of hippocampal CA1/CA3 neurons. From these experiments, we chose sessions during which we observed at least 20 place cells during active place-field exploration, and at least 30 PBEs (see below). Place cells were identified as pyramidal cells which had (i) a minimum peak firing rate of 2 Hz, (ii) a maximum mean firing rate of 5 Hz, and (iii) a peak-to-mean firing rate ratio of at least 3, all estimated exclusively during periods of run (as defined before, that is, when the animal was running >10 cm/s). This selection yielded n=18 session with 41–203 neurons (36–186 pyramidal cells). All procedures were approved by the Institutional Animal Care and Use Committee of Rutgers University and followed US National Institutes of Health animal use guidelines (protocol 90–042).

A second dataset used tetrodes to record a large number (101–242) of putative pyramidal neurons in area CA1 during two sessions each in four rats. Briefly, as was previously reported using these data (Pfeiffer and Foster, 2013, Pfeiffer and Foster, 2015), rats explored an arena in which there were 36 reward sites. In each session, one site was designated as ‘home’. During a session, rats would repeatedly alternate between retrieving a random reward site in one of the remaining 35 locations and retrieving a reward at the home location. All procedures were approved by the Johns Hopkins University Animal Care and Use Committee and followed US National Institutes of Health animal use guidelines (protocols RA08M138, RA11M16, and RA14M48).

In order to investigate remote replay, we used data from an experiment in which this phenomenon has been previously reported (Karlsson and Frank, 2009). Briefly, rats were implanted with multi-electrode microdrives with tetrodes targeting CA1 and CA3. They were trained to carry out a continuous-alternation task in an initially novel ‘w’-shaped maze (E2) for liquid reward for multiple daily run sessions interspersed by rest-periods in an enclosed box. After they learned the task, they were introduced to a novel w-maze (E1) in a different orientation in which they had two run sessions followed by a run in the now-familiar E2. For our proof-of-concept analysis (Figure 7), we used data from the second day of the novel maze (i.e., third and fourth exposures) in animal ‘Bon’.

Finally, we recorded neural activity during an object-location memory task using a 32-channel silicon probes (Buzsaki32, Neuronexus, MI) equipped with light fibers lowered to area CA1 of the dorsal hippocampus. The animal was previously infused with AAV-CamKIIa-ArchT-GFP for the purpose of another experiment. Putative pyramidal cells and interneurons were distinguished based on their spike waveforms and spike auto-correlograms. On the day before the recordings, the animal was repeatedly exposed to an empty test chamber on four successive six minute blocks, interleaved by three minute rest periods in the home cage. On the recording day, the first of these six-minute blocks was again the empty test chamber, but on the remaining blocks, the animal was exposed to a fixed configuration of three different novel objects placed in the northeast, center and southeast corners of the box. These blocks were again interleaved with three minute rest periods in the home cage. The test chamber was a 60 × 60 cm2 box with a local cue (8.5 in. × 11 in. sheet printout) placed on one test wall. Following the last acquisition exposure, the animal was returned to its home cage for a four hour extended sleep period. The subsequent day, one of the objects in the box was displaced and the animal was reintroduced into the box to test for interactions with the displaced versus non-displaced objects. All procedures were approved by the Institutional Animal Care and Use Committee of the University of Wisconsin-Milwaukee and followed US National Institutes of Health animal use guidelines (protocol 13–14 #28)

Population burst events

To identify PBEs in the linear track data, a SDF was calculated by counting the total number of spikes across all recorded single and multi-units in non-overlapping 1 ms time bins. The SDF was then smoothed using a Gaussian kernel (20 ms standard deviation, 60 ms half-width). Candidate events were identified as time windows with a peak SDF of at least three standard deviations above the mean calculated over all the session. The boundaries of each event were set to time points of crossing the mean, preceding and following the peak. Events during which animals were moving (average movement speed of >5 cm/s) were excluded from all further analyses to prevent possible theta sequences from biasing our results. For analysis, we then binned each PBE into 20 ms (non-sliding) time bins. Spikes from putative interneurons (mean firing rate when moving >10 Hz) were excluded, as were events with duration less than four time bins or with fewer than four active pyramidal cells. For the open field data, we used the previously reported criteria (Pfeiffer and Foster, 2013) for identifying PBEs prior to binning (10 ms standard deviation kernel, minimum of 10% of units active, duration between 50 ms and 2000 ms).

Hidden markov model of PBE activity

We trained HMMshe complete sequence on the PBEs. In an HMM, an unobserved discrete latent state qt evolves through time according to a first order Markov process. The temporal evolution of the latent state is described by the M×M matrix A, whose elements {aij} signify the probability after each time bin of transitioning from state i to state j, aij=Pr(qt+1=j|qt=i). The number of states, M, is a specified hyperparameter. We found that our results were insensitive to the value of M through a wide range of values from 20 to 100 (Figure 3—figure supplement 1). During each time bin of an event, the identity of the latent state influences what is observed via a state-dependent probability distribution. We modeled the N-dimensional vector of binned spiking from our ensemble of N neurons at time t, Ot, as a Poisson process. Specifically, for each state, i, we model neuron n as independently firing according to a Poisson process with rate λni.

Pr(Ot|qt=i)=n=1N Pr(ont|qt=i)n=1N (λni)onexp(λni)

where ont is the number of spikes observed from neuron n at time t. The final parameter which specifies our model is the probability. distribution of the initial state for a given event, πi=Pr(q1=i). Thus, our model is specified by parameters θ={ A,Λ,π }, where Λ={λni} is an N×M matrix and π={πi} is an N-dimensional vector.

To learn model parameters, we follow the well-known iterative EM procedure (Rabiner, 1989), treating each training PBE as an observation sequence. In order to regularize the model, we impose a minimum firing rate for each neuron of 0.001 (0.05 Hz) during the M-step of EM. For a given PBE (i.e., observation sequence) with K bins, we use the ‘forward-backward algorithm’ (Rabiner, 1989) to calculate the probability distribution of the latent state for each time bin, Pr(qt|O1,,Ot,,OK). For a particular time bin, t, in a given sequence, the forward-backward algorithm allows information from all observation bins, previous and subsequent, to affect this state probability distribution (as well the observation bin at time t). The forward-backward algorithm also efficiently calculates the ‘score’, or likelihood of the complete sequence, Pr(O1,,OK). All HMMs learned in this work used five-fold cross validation, that is, the PBEs were divided into five randomly selected fifths (‘folds’), and then each fold was evaluated as a test set, with the model trained using the remaining four folds. We define the model likelihood of an HMM as the product of the scores of each event using this five-fold cross validation. To initially evaluate model learning, we compared model likelihoods calculated using real and shuffled test data. Models which have learned to properly represent the data should show significant increases. To quantify the presence of PBE sequences in a model we used a model quality metric as described below.

Ordering states for visualization

For visualization, we wanted to order the states to maximize the super diagonal of the transition matrix. We used a greedy approach which typically yields this solution. We started by assigning the first index to the state with the highest initial probability and added states based on the most probable state transitions. The undirected connectivity graphs were then generated from this transition matrix, averaging the strength of reciprocal connections, aij and aji.

Surrogate datasets and shuffle methods

In order to analyze the HMMs we learned, we compared them against different types of surrogate datasets obtained by shuffling the neural activity during PBEs in distinct ways. (1) Temporal shuffle: within each event, the binned spiking activity was circularly permuted across time for each unit, independently of other units. This goal of this shuffle is to disrupt unit co-activation, while maintaining the temporal dynamics for each unit. (2) Time-swap shuffle: within each event, the order of the binned columns of neural activity was randomly permuted across time, coherently across units. The goal of this shuffle is to change the temporal dynamics of ensemble activity, while maintaining unit co-activation. (3) Poisson surrogate ‘shuffle’: we estimated each unit’s mean firing rate across all PBEs, and then produced surrogate PBEs from independent Poisson simulations according to each unit’s mean firing rate. (4) Pooled time-swap shuffle: the order of the binned columns of neural activity was randomly permuted across all pooled events, coherently across units. This shuffle has been previously used in Bayesian replay detection (Davidson et al., 2009).

Calculating sparsity and connectivity of the model parameters

Sparsity of the transitions from individual states (departure sparsity) was measured by calculating the Gini coefficient of corresponding rows of the transition matrix (Hurley and Rickard, 2009). The Gini coefficient is a measure of how variable the values of this probability distribution are, with equality across states corresponding to a coefficient of zero (minimal sparsity), and a singular distribution with a probability-one transition to a single other state corresponding to a coefficient of one (maximal sparsity). The sparsity of the full transition matrix was calculated by averaging the Gini coefficient across rows. For analyses of PBE models from linear tracks, we computed the mean sparsity across states for each of the 250 surrogate datasets, and these means were used to generate the box plots of Figure 2c. Note that for the actual data, we generate a distribution by randomly initializing the model 250 times and calculating the mean sparsity over all initializations. For analyses of models learned from PBEs in open fields (and the linear track comparison with 50 states), we created 50 surrogates/random initializations (Figure 6—figure supplement 1). To compare across sessions, we calculated the mean sparsity by averaging over all 250 surrogate datasets to obtain a single mean sparsity per session, so that n=18 per-session means were used to create the box-plots of Figure 2e.

Firing rates can be highly variable for different units. Thus, when evaluating the sparsity of the observation matrix, we measured the extent to which individual units were specifically active in a few states by calculating the Gini coefficients of the rows of the observation matrix. As with transitions, we calculated mean sparsity across units for each surrogate dataset (e.g., linear track, Figure 2d; open field, Figure 6—figure supplement 2), and we then averaged over all surrogate datasets to obtain aper-session average, used in Figure 2f.

Model connectivity and sequences

To measure the degree of sequential connectivity within the graph corresponding to the transition matrix—with nodes and edges representing the states and transitions, respectively—we developed an algorithm for measuring the length of the longest path that can be taken through the graph. This method is analogous to the ‘depth-first search’ algorithm for traversing the graph’s tree structure without backtracking. First, we made an adjacency matrix for a corresponding unweighted directed graph by binarizing the transition matrix using a threshold of 0.2 on the transition probabilities. Starting from each node, we then found the longest path that ended at either a previously visited node or a terminal node (a node without any outgoing edges). To compare models trained on actual versus surrogate datasets, we adjusted the thresholds to match the average degree (defined as the average number of edges per node) between the models, thus ruling out possible effects due to differences in the number of graph edges. We carried out this analysis on the same set of models that were generated for analyzing sparsity. To compare across sessions, we calculated the median maximum path length for each session (n=18) and used the per-session medians to generate box plots of Figure 2—figure supplement 3c.

Latent state place fields

To calculate the latent state place fields, we first identified bouts of running by identifying periods when animals were running (speed >10 cm/s). We then binned the spiking during each of these bouts in 100 ms bins. Using the forward-backward algorithm (Rabiner, 1989) and the HMM model parameters learned from PBEs, we decoded each bout into a sequence of latent state probability distributions, Pr(qt|Ot). Using the track positions corresponding to each time bin, we then found the average state distribution for each position bin, xp, and normalized to yield a distribution for each state, Pr(xp|qt=i).

Decoding position from latent state sequences

We used the lsPF to decode the animal’s position after determining the probability of different latent state trajectories during bouts of running. With five-fold cross validation, we estimated lsPF in a training dataset, then used the HMM model to decode latent state trajectory distributions from ensemble neural activity in the test data. The product of lsPF and decoded latent state distribution at time t is the joint distribution Pr(xp,qt|Ot). We decode position as the mean of the marginal distribution Pr(xp|Ot).

Bayesian replay detection

We followed a frequently used Bayesian decoding approach to detect replay in our 1D data (Kloosterman, 2012). For each 20 ms time bin t within a PBE, given a vector comprised of spike counts from N units, Ot=(o1to2toNt) in that bin, the posterior probability distribution over the binned track positions was calculated using Bayes’ rule:

Pr(xp|Ot)=Pr(Ot|xp)Pr(xp)q=1P Pr(Ot|xq)Pr(xq),

where xp is the center of p-th linearized position bin (of P total bins). We assumed Poisson firing statistics, thus the prior probability, Pr(Ot|xp,), for the firing of each unit n is equal to

Pr(Ot|xp)=n=1N Pr(ont|xp,)n=1N (τλnp)ontexp(τλnp)

where τ is the duration of time bin (100 ms during estimation, 20 ms during decoding), and λnp characterizes the mean firing rate of the n-th unit in the p-th position bin. We assumed a uniform prior distribution Pr(xp) over the position bins.

For each PBE, the estimated posterior probability distribution was used to detect replay as follows. Many (35,000) lines with different slopes and intercepts were sampled randomly following the approach in (Kloosterman, 2012). The Bayesian replay score for a given event was the maximum score obtained from all candidate lines, where the score for a particular line was defined as the mean probability mass under the line, within a bandwidth (of 3 cm). For time bins during which the sampled line fell outside of the extent of the track, the median probability mass of the corresponding time bin was used, and for time bins during which no spikes were observed, we used the median probability mass across all on-track time bins. To evaluate the significance of this score, for each event we generated 5000 surrogates of the posterior probability distribution by cycling the columns (i.e., for each time bin, circularly permuting the distribution over positions by a random amount) and calculated the replay score for each surrogate. The Monte Carlo p-value for each event was obtained from the number of shuffled events with replay scores higher than the raw data. The threshold for significance was varied as described in the text. For the open field, we used previously reported criteria (Pfeiffer and Foster, 2013) to identify replay events from PBEs.

Replay detection via PBE model congruence

To identify replay as model congruence, for each PBEs, we used the forward-backward algorithm to calculate the sequence likelihood Pr(O1,,OK), as defined earlier. Using five-fold cross validation, the parameters of a HMM were learned from training PBE. The sequence score was then calculated for each event in the test data. To evaluate the significance of this score, for each event we generated 5000 surrogate scores using a computationally-efficient scheme. Specifically, for each surrogate, we randomly shuffle the rows of the transition matrix, excepting the diagonal. By maintaining the diagonal (i.e., transitions that begin and end in the same state) and leaving the observation model unchanged, this shuffle specifically selects against PBEs in which the latent states do not evolve in temporal sequences. The Monte Carlo p-value for each event was calculated as the fraction of shuffled events with HMM sequence scores higher than the raw data. The threshold for significance was varied as described in the text. Note that while we describe this as HMM-congruence, we have maintained the diagonal of the transition matrix, which specifically selects against PBEs which might be model-congruent by maintaining a single state over many time bins. In reality there are other dimensions of the HMM that we could assess congruence against, for example the observation model, the initial state distribution, or combinations of these and the transition matrix. In comparing against Bayesian decoding, our current definition seemed most appropriate for sequence detection, but we can imagine future studies expanding on our approach.

Human scoring and detection comparison

We organized a group of human scorers to visually evaluate whether individual PBEs should be described as replay. More specifically, scorers were only presented with Bayesian decoded probability distributions such as those in Figure 4a, but without access to the spike raster or any additional information. The scorers included six graduate students (including one of the authors) and two undergraduates, all of whom were generally familiar with the concept of hippocampal replay. We built an automatic presentation system which would display each event in random order, and record one of six possible scores: ‘excellent’ (highly sequential with no jumps and covering most of the track), ‘good’ (highly sequential with few or no jumps), ‘flat’ (decoded position stayed mostly in the same place, i.e. no temporal dynamics), ‘uncertain’ (some semblance of structure, but not enough to fall into any of the previous categories) or ‘noise’ (no apparent structure, or nonsensical trajectories such as teleportation). An event was then designated as replay if it was labeled as ‘excellent’ or ‘good’ by a majority of scorers (ties were labeled as non-replay).

To calculate an ROC curve for replay detection algorithms, we used our shuffle statistics for each event to create a vector which related the significance threshold (e.g., 99%) to the label supplied by the algorithm (i.e., significant replay or not). Then, as a function of threshold, the sensitivity (fraction of true positives identified) and selectivity (fraction of true negatives identified) were averaged over events to yield an ROC curve. To evaluate whether the AUC differed between Bayesian and model-congruence techniques we used a bootstrap approach. To generate a null hypothesis, we combined the event/threshold vectors from both groups, and then sampled two random groups (A and B) with replacement from the pooled data. The AUC for these two random groups of events were measured, and a distribution for the difference between the randomly chosen AUC was calculated. The two-sided p-value we report is the fraction of differences in random AUC which are more extreme than the actual difference.

HMM model quality across sessions

In order to understand the extent to which an HMM trained on PBEs from a given session contained sequentially-structured temporal dynamics, we calculated the ‘session quality’ (equivalently model quality) as follows. Again using five-fold cross validation, we learn an HMM on the training subset of PBEs, and score (using the forward-backward algorithm, as before), the remaining subset of test PBEs. Then, we also score a pooled time-swap surrogate of the test PBE and we repeat this pooled time-swap scoring n=2500 times. Finally, we obtain a z score for each PBE by comparing the score from the actual test PBE, to the distribution of pooled time-swap scores of the corresponding PBE. The session quality is then defined as the average of these z scores, over all events in a session. This measure of session quality was then used to detect the presence of putative remote replay events or other extra-spatial structure in PBEs, since a high session quality after removing local Bayesian significant events is highly suggestive of remaining (unexplained) sequential structure.

Software and data analysis

Data analyses were performed using Matlabr and Python. Jupyter notebooks (using Python) are available at https://github.com/kemerelab/UncoveringTemporalStructureHippocampus (copy archived at https://github.com/elifesciences-publications/UncoveringTemporalStructureHippocampus), where most of the results presented here are reproduced. We have also developed and open-sourced a Python package (namely nelpy) to support the analyses of electrophysiology data with HMMs, which is available from https://github.com/nelpy (Ackermann et al., 2018; copies archived at https://github.com/elifesciences-publications/nelpyhttps://github.com/elifesciences-publications/tutorialshttps://github.com/elifesciences-publications/example-analyseshttps://github.com/elifesciences-publications/example-data and https://github.com/elifesciences-publications/test-data).

References

  1. 1
    Nelpy: Neuroelectrophysiology Object Model and Data Analysis in Python
    1. E Ackermann
    2. J Chu
    3. S Dutta
    4. C Kemere
    (2018)
    Nelpy: Neuroelectrophysiology Object Model and Data Analysis in Python.
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  19. 19
  20. 20
  21. 21
  22. 22
  23. 23
  24. 24
  25. 25
  26. 26
  27. 27
  28. 28
  29. 29
    Comparing measures of sparsity
    1. N Hurley
    2. S Rickard
    (2009)
    IEEE Transactions on Information Theory 55:4723–4741.
    https://doi.org/10.1109/TIT.2009.2027527
  30. 30
  31. 31
  32. 32
  33. 33
  34. 34
  35. 35
  36. 36
  37. 37
    Analysis of hippocampal memory replay using neural population decoding
    1. F Kloosterman
    (2012)
    In: T Fellin, M Halassa, editors. Neuronal Network Analysis: Concepts and Experimental Approaches. Humana Press. pp. 259–282.
  38. 38
  39. 39
  40. 40
  41. 41
  42. 42
  43. 43
  44. 44
  45. 45
  46. 46
  47. 47
  48. 48
  49. 49
  50. 50
  51. 51
  52. 52
  53. 53
  54. 54

Decision letter

  1. Laura Colgin
    Reviewing Editor; The University of Texas at Austin, Center for Learning and Memory, United States

In the interests of transparency, eLife includes the editorial decision letter and accompanying author responses. A lightly edited version of the letter sent to the authors after peer review is shown, indicating the most substantive concerns; minor comments are not usually included.

[Editors' note: the authors’ plan for revisions was approved and the authors made a formal revised submission.]

Thank you for sending your article entitled "Uncovering temporal structure in hippocampal output patterns" for peer review at eLife. Your article is being evaluated by three peer reviewers, one of whom is a member of our Board of Reviewing Editors and the evaluation is being overseen by a Reviewing Editor and Michael Frank as the Senior Editor.

Given the list of essential revisions, including new experiments, the editors and reviewers invite you to respond within the next two weeks with an action plan and timetable for the completion of the additional work. We plan to share your responses with the reviewers and then issue a binding recommendation.

The enthusiasm for the new technique presented in the paper was high. However, the reviewers recommend a clear demonstration that this new technique can be used to discover something new. In other words, the reviewers feel that the paper would be more exciting to a broad readership if results were obtained that would not have been possible with standard analyses already used in the field. The reviewers are confident that the recommended revisions could be done with the authors' existing data sets (i.e. without requiring collection of new data, which would take a long time). Please find the reviewers' specific comments below:

Reviewer #1:

This is a potentially very interesting paper that uses hidden Markov models to uncover structure in hippocampal place cell ensembles during sharp wave-related states. The advantage of this method, compared to existing methods, is that place cell sequences can be identified without the use of a place cell training data set from active running behaviors. This could allow for the identification of sequences that represent non-spatial memories or sequences from single trials. However, the paper is written in a way that may not be readily understandable by a general readership. My specific comments are as follows:

1) I am afraid that the HMM approach is not explained in a way that is entirely clear to a general audience. For example, the authors write, "the unobserved latent variable represents the temporal evolution of a memory trace which is coherent across the CA1 ensemble" (subsection “Learning hidden Markov models from PBE data”). What does this mean exactly? Similarly, the meaning of "the probability that the CA1 ensemble will transition from a start state to a destination state in the next time bin" is unclear. It seems like a state basically corresponds to a representation of a location, but this is not really explicitly explained. I did not understand the cartoon and its explanation in Figure 1A. Does this represent sequences? Do the colors correspond to the place cells shown? In Figure 2, I did not understand what was meant by "nodes on the same radius correspond to the same state". Also, in Figure 2, I did not understand the difference between panels C and E, and between panels D and F. The authors should probably better explain the measures from graph theory since they will probably be unknown to many readers. In subsection “Ordering states for visualization”, the authors refer to the state with the highest initial probability, but I did not understand what this meant exactly. The readers use HMM jargon like "the forward-backward algorithm" that may need to be briefly explained.

2) It seems concerning that none of the data from rat 3 seems to show a difference between the actual data and the surrogate data. (Figure 1—figure supplement 1, Figure 2—figure supplement 1).

3) It would be useful to have more information about the human scorers. Who were they? How were they trained? The authors write (subsection “Human scoring and detection comparison”) that an event was designated as replay if it was labeled as "excellent" or "good" by a majority of scorers. Do they mean just more than half of the scorers?

Reviewer #2:

Maboudi et al., propose the use of Hidden Markov Models to represent multi-neuron activity patterns in the hippocampal subfield CA1. The model is trained from activity in Population Bursts Events (Likely to a large extent coinciding with sharp wave ripple events). It is shown that HMM can successfully capture at least part of the statistical structure of the data, and while they are trained on spontaneous activity, they can recover the spatial responses observed during active behavior. The method has several attractive features, most importantly, that it detects patterns in an unsupervised fashion. In the literature, spontaneous activity in the hippocampus has been typically analyzed in the context of "replay" and making use of Bayesian decoders trained on the spatial responses of the neurons. While this approach has been quite successful, it also restricts analysis to patterns that are representative of the behavioral situation (spatial environment etc.) that has been used to train the decoder itself. Whatever information is carried by the activity, but not covered by the decoder, goes undetected. Unsupervised methods hold the promise of uncovering structure in a template free way.

I am quite sympathetic with the approach, but I think that it needs a better demonstration of its success and usefulness. I am fairly confident this can be done based on the data the authors use here, and to a large extent the same methods. My main points:

1) The used surrogate data methods (essentially column-wise and row-wise shuffling), is the intuitive one, and probably correct. We see that the sparseness of the model trained from real data is higher than for the shuffled data. However, the effects (Figure 2C-F) seem to be very small. One is then left wondering about what the model is actually doing. For example, one could imagine this pattern of results may emerge in the extreme sparsity case, when each state only represents activation of one neuron. In that case, shuffling would clearly leave the sparsity of the observation matrix unchanged. Why the sparsity of the transition matrix doesn't change with shuffling is more puzzling and should be investigated

2) The PBE derived model recovers the spatial response properties of the ensemble quite well. I am a bit puzzled about how this reflects the known statistics of hippocampal activity though. We know that on a given track (e.g. Figure 3), about 30% of CA1 neurons fire, while during SWRs most neurons activate. Then, how is it possible that all states derived by PBE are activated on the track? wouldn't we expect only a subset to do so?

3) I am a bit puzzled by the congruency measure. We see in Figure 4 that the model with the shuffled transition matrix yields lower likelihood scores, however in the examples shown in Figure 4B we see that the sequential structure is by and large preserved even in the shuffled model. This seems to suggest that the HMM optimization (forward/backward algorithm) is dominated by the observation probabilities, and the transition matrix is less important. But then one should ask what of the essential structure of the data is the transition matrix capturing.

4) I understand that organizing a "human scorers team" was probably a huge effort. Yet I am not sure that this is the best comparison for the analytical procedure, given its strongly subjective nature.

5) A more general question: The bottom line of the paper is that the outcome of PBE/HMM parallels the outcome of Bayesian decoding. Yet what would be most interesting is what you may get in addition to what you get from Bayesian methods. In particular, focusing on the events that are not detected by Bayesian decoding but are detected by PBE/HMM may be interesting, and give more detail on their time distribution, structure etc.

Reviewer #3:

The main contribution of this paper is the Introduction of a standard analysis framework (hidden Markov models – HMMs) to the study of place cells in the hippocampus during population burst events (PBEs) when the animal is not moving and reply events are believed to be happening. In the standard analysis techniques, behavioral data (e.g. location) is used during navigation to characterize tuning properties of individual neurons and then activity sequences during PBE events is viewed through this tuning to determine if there is replay activity consistent with the measured tuning properties. In the proposal of this manuscript, the HMM analysis technique can be used only on the PBE events and the resulting latent states appear to discover the same localized spatial tuning as what has been found during active navigation. These result show that a completely unsupervised method (i.e. NOT using behavioral location data) can be used to find structure and assess questions about replay, demonstrating that the structure of activity during PBEs is strong enough to discover in a data-driven way.

In my opinion, the technique is promising and has potential to uncover very interesting information coding principles and test hypotheses. As a methods paper, I think it is a strong contribution to the literature. For eLife, I am unclear whether it is a good fit because I find the current draft lacking in the articulation of the scientific discovery that the technique enables. What have we learned about the system by doing the analysis in the paper? While I find the analysis interesting, I'm having trouble understanding what we've learned that has significant impact on our understanding. Similarly, for the future, the manuscript points toward some novel analysis methods that this approach would allow but never quite makes a compelling case for what we would learn from it. This is a good paper and should be published, and I am open to be convinced that it should be published in eLife. But, I think the manuscript needs to make a more compelling case for what we can learn from the data being analyzed in this way that we cannot learn otherwise.

- Some of the language in the Introduction regarding the motivation for the study is vague and unclear. What does it mean for the "fundamental mode" of the hippocampus to be "PBE sequences"? This seems like a statement trying to make a big claim that their study could support, but it's unclear and doesn't quite make the connection for me.

- In the Results section, I think you could be a little more clear on the details like 1) what is the latent state representing (position), what is the prior, and what is the mapping to Λ (e.g., the tuning curve from state to firing rate). These come up in the Materials and methods but giving a little hint here would help make things more clear.

- I find some of the figure captions too abbreviated to actually understand what's going on in the figures. For example, in Figure 1A, I don't really understand the plot on the bottom left.

- In subsection “What do the learned model parameters tell us about PBEs?”, the latent states are used to estimate a likelihood for position on the track. The comparison is made with the shuffled data, but there is no comparison to the decoding using the standard technique (i.e., place cell tuning curves). I would expect the standard decoding to do better (since it's developed with a supervised learning approach), but it would be good to show the comparison and discuss.

- In subsection “What do the learned model parameters tell us about PBEs?” where the manuscript discusses assessing the patterns of activity for congruence, I think I am confused about the approach. The method is described as comparing against a single null model. As far as I understand, these results would depend entirely on the specific form of the null model that is the baseline for comparison. Why is this an acceptable approach? It seems so dependent on that arbitrary decision that I don't understand how the results are interpretable as a generalized consistency metric.

- There were some punctuation and wording issues that affected clarity slightly. Subsection “HMM-congruent PBEs capture sequence replay”: "between model quality or the number of PBEs and Bayesian Decoding". Subsection “Modeling internally generated activity during open 1eld behaviour”: "Can an HMM trained on PBEs indeed".

[Editors' note: further revisions were requested prior to acceptance, as described below.]

Thank you for resubmitting your work entitled "Uncovering temporal structure in hippocampal output patterns" for further consideration at eLife. Your revised article has been favorably evaluated by Michael Frank (Senior Editor) and a Reviewing Editor.

The manuscript has been greatly improved but there are some remaining issues that need to be addressed before acceptance, as outlined below.

In particular, the authors' analyses of events that were non-significant with the Bayesian approach and demonstration that many of them correspond to replayed representations of a different, previously explored environment (i.e., new Figure 7) satisfactorily address the major concerns raised during the first round of review.

However, the description of Figure 7 is not entirely clear. For example, the x-axes in panels D and I are labeled "time", but the color bar presumably corresponding to space is also shown below the x-axis. What is this supposed to indicate? Also, panels E-G and J-L need a more thorough explanation. The figure legend states that panel e (bottom) shows decoded positions using E2, which is the local environment, yet the top of panel e presumably corresponds to the remote environment (i.e. E1). This is confusing. The descriptions of Figure 7A-B in the Results section and the figure legend are also not clear. The text states that the model quality decreased rapidly when remote replay events were removed, and that model quality decreased slowly when local replay events were removed. But, the figure appears to show the opposite. The light blue line (local removed) decreases more rapidly than the green line (remote removed). Also, are the top two slowest high quality decay sessions indicated in panel B the ones that were used to find remote replay events? It isn't clear what panel B is supposed to show- it should be explained clearly in the text and in the figure legend.

There is also insufficient detail provided about Figure 8. How do these figures relate to the object-location memory task? Couldn't these sleep sequences simply have been detected from exploration of the 60 x 60 enclosure regardless of object-location memory? And wouldn't they also have been potentially detected with Bayesian decoding? The authors claim in subsection “Extra-spatial Information” that the HMM can uncover sequential activity that is not readily detectable with the Bayesian approach, but it is unclear how Figure 8 shows this. In any case, Figure 8 is not really necessary as long as Figure 7 is explained clearly.

A few minor issues also remain to be clarified. In subsection “Awake population burst events”, the authors state that similar results were obtained when a theta detection approach was used, but it is unclear where this approach is shown or described. Also, it is unclear to what "these" refers. Also, in Figure 1B, the "pyramidal cells (non place)" label should be changed to "putative pyramidal cells (non place)". Also, the difference between the mean departure Gini and mean Λ row Gini in Figure 6E-F should be explained more clearly for a general readership. Finally, as was raised in the initial review, the authors should explicitly state in the first two paragraphs of the Materials and methods section that the Diba and Buzsaki and Pfeiffer and Foster datasets were recorded previously and used in other publications. Otherwise, readers may misunderstand and assume that methods were adopted from these papers, but that similar data were collected again for the present study.

https://doi.org/10.7554/eLife.34467.025

Author response

The enthusiasm for the new technique presented in the paper was high. However, the reviewers recommend a clear demonstration that this new technique can be used to discover something new. In other words, the reviewers feel that the paper would be more exciting to a broad readership if results were obtained that would not have been possible with standard analyses already used in the field. The reviewers are confident that the recommended revisions could be done with the authors' existing data sets (i.e. without requiring collection of new data, which would take a long time).

In addition to revising the text to better explain the methodology to a lay reader, we have performed a series of new analyses on our existing data as well as two additional datasets that highlight the novelty and power of our approach. Inspired by the second reviewer’s suggestions, we removed events that were detected as significant using Bayesian replay and retrained the hidden Markov model (HMM) on the remaining (non-significant) events. We note, as suggested by the reviewer, that this approach could not even be attempted with a Bayesian decoding approach. Interestingly, we found that in some sessions the HMM still captured a large of amount of the structure in the remaining population burst events (PBEs) (new Figure 7A). We conjectured that these events may correspond to replay of a remote environment. By analyzing a dataset where the recording spanned behavior of an animal in two separate environments, we confirmed that indeed many HMM-congruent events that are found as non-significant with the Bayesian approach correspond to replay of the remote environment. As a final test, we applied the HMM to PBEs in sleep following the exploration phase of a hippocampus-dependent object-location task (Figure 8) and found that we could detect a significant number of HMM-congruent events in post-task sleep, without reference to behavior. We believe these new results significantly elevate the impact of our report and we thank the reviewers for their suggestions in our acknowledgements.

Reviewer #1:

This is a potentially very interesting paper that uses hidden Markov models to uncover structure in hippocampal place cell ensembles during sharp wave-related states. The advantage of this method, compared to existing methods, is that place cell sequences can be identified without the use of a place cell training data set from active running behaviors. This could allow for the identification of sequences that represent non-spatial memories or sequences from single trials. However, the paper is written in a way that may not be readily understandable by a general readership. My specific comments are as follows:

1) I am afraid that the HMM approach is not explained in a way that is entirely clear to a general audience. For example, the authors write, "the unobserved latent variable represents the temporal evolution of a memory trace which is coherent across the CA1 ensemble" (subsection “Learning hidden Markov models from PBE data”). What does this mean exactly? Similarly, the meaning of "the probability that the CA1 ensemble will transition from a start state to a destination state in the next time bin" is unclear. It seems like a state basically corresponds to a representation of a location, but this is not really explicitly explained.

We have now revised the manuscript with an eye towards further clarity and well-explained terminology. With regards to these specific examples, we now clarify that a HMM is a model of ensemble firing (coactive neurons corresponding to latent states) and how these ensembles change over time. While we do find that states correspond to location, that is only because location is the organizing principle of the activity in the task under consideration. In a different task, states might very well correspond to time, frequency space, or some more complex arbitrary manifold.

The specific lines mentioned by the reviewer have been eliminated or revised and the beginning of the Results section now provides a better introduction to the HMM methodology.

"Hidden Markov models have been very fruitfully used to understand sequentially structured data in a variety of contexts. A hidden Markov model captures information about data in two ways. […] Given the role of the hippocampus in memory, in our HMM, the unobserved latent variable presumably corresponds to the temporal evolution of a memory trace that is represented by co-active ensembles of CA1 and CA3 neurons. The full model will correspond to the structure which connects all the memory traces activated during PBEs."

I did not understand the cartoon and its explanation in Figure 1A. Does this represent sequences? Do the colors correspond to the place cells shown?

The cartoon illustration has been removed from Figure 1 and has been replaced by a latent-state-space-view that hopefully helps readers understand the salient parts of the HMM approach better than the cartoon did.

In Figure 2, I did not understand what was meant by "nodes on the same radius correspond to the same state".

We have now clarified that the inner and outer rings represent the same nodes – this makes it easier to visualize the self-transitions (starting and ending in the same state) and the transitions to the next node. In the caption, we now state:

"The nodes correspond to states (progressing clockwise, starting at the top). The weights of the edges are proportional to the transition probabilities between the nodes (states). The transition probabilities from state i to every other state except i+1 are shown in the interior of the graph, whereas for clarity transition probabilities from state i to itself, as well as to neighboring state i+1 are shown between the inner and outer rings of nodes (the nodes on the inner and outer rings represent the same states)."

Also, in Figure 2, I did not understand the difference between panels C and E, and between panels D and F.

We have revised the captions of the figure to clarify that panels C and E are the transition matrices (state dynamics) for the example sessions and all sessions, respectively, and D and F are the observation models (firing rate / state) for the example session and all sessions, respectively.

The authors should probably better explain the measures from graph theory since they will probably be unknown to many readers.

In the Materials and methods section, we now expand on the Gini coefficient when it is introduced:

"Sparsity of the transitions from individual states (departure sparsity) was measured by calculating the Gini coefficient of corresponding rows of the transition matrix (Hurley and Rickard, 2009). The Gini coefficient is a measure of how variable the values of this probability distribution are, with equality across states corresponding to a coefficient of zero (minimal sparsity), and a singular distribution with a probability-one transition to a single other state corresponding to a coefficient of one (maximal sparsity). The sparsity of the full transition matrix was calculated by averaging the Gini coefficient across rows."

We also replaced the term “measures from graph theory” in the main text and also referred to Materials and methods section for a more detailed explanation of the search algorithm, “Using a graph search algorithm (see Materials and methods section), we simulated paths through state space generated by […]”

In subsection “Ordering states for visualization”, the authors refer to the state with the highest initial probability, but I did not understand what this meant exactly.

The Initial State probability is one of the parameter estimated during model learning, along with the Observation Model and State Transition Probabilities. It describes the probability that a given state will be the beginning of a sequence. We now explicitly state:

“The parameters of our model that are fit to data include the observation model (the cluster descriptions or predicted activity of each excitatory neuron within the CA1/CA3 ensemble for a given state), the state transition model (the probability that the CA1/CA3 ensemble will transition from a start state to a destination state in the next time bin), and the initial state distribution (the probability for sequences to start in each given state). In prior work using HMMs to model neural activity, a variety of statistical distributions have been used to characterize ensemble firing during a specific state (the observation model.”

The readers use HMM jargon like "the forward-backward algorithm" that may need to be briefly explained.

We have revised the text throughout to replace jargon with more readily understandable explanations. e.g.

“To observe the evolution of the latent states during behavior, we used our model to determine the most likely sequence of latent states corresponding to the neural activity observed in 100 ms bins during epochs that displayed strong theta oscillations (exclusive of PBEs) when rats were running (speed > 10 cm/s; see Materials and methods section).”

And in the Materials and methods section, we have expanded the description to say:

"For a given PBE (i.e. observation sequence) with K bins, we use the “forward-backward algorithm” (Rabiner, 1989) to calculate both the probability distribution of the latent state for each time bin, Pr(qt|O1,…,Ot,…,OK). For a particular time bin, t, in a given sequence, the forward-backward algorithm allows information from all observation bins, previous and subsequent, to affect this state probability distribution (as well the observation bin at time t). The forward-backward algorithm also can efficiently calculate, and the “score”, or likelihood of the complete sequence, Pr(O1,…,OK)."

2) It seems concerning that none of the data from rat 3 seems to show a difference between the actual data and the surrogate data. (Figure 1—figure supplement 1, Figure 2—figure supplement 1).

The models from rat 3 were indeed generally of lower quality largely because of a lower number of PBEs (Figure 5—figure supplement 1C). However, we would point out that the sessions from this animal still yielded transition matrices that were sparser than shuffles (significance stars in Figure 2—figure supplement 1).

3) It would be useful to have more information about the human scorers. Who were they? How were they trained? The authors write (subsection “Human scoring and detection comparison”) that an event was designated as replay if it was labeled as "excellent" or "good" by a majority of scorers. Do they mean just more than half of the scorers?

We now describe the scorers:

"The scorers included six graduate students (including one of the authors) and two undergraduates, all of whom were generally familiar with the concept of hippocampal replay.

And indeed, majority by definition means more than half the scorers:

"An event was then designated as replay if it was thus labeled as “excellent” or “good” by a majority of scorers (ties were labeled as non-replay)."

Reviewer #2:

[…] 1) The used surrogate data methods (essentially column-wise and row-wise shuffling), is the intuitive one, and probably correct. We see that the sparseness of the model trained from real data is higher than for the shuffled data. However, the effects (Figure 2C-F) seem to be very small. One is then left wondering about what the model is actually doing. For example, one could imagine this pattern of results may emerge in the extreme sparsity case, when each state only represents activation of one neuron. In that case, shuffling would clearly leave the sparsity of the observation matrix unchanged. Why the sparsity of the transition matrix doesn't change with shuffling is more puzzling, and should be investigated.

We recognize that no single measure captures the entire structure of spatial representations and each shuffle alters this structure in unique ways that preserves some of this structure, as the reviewer has pointed out. For example, because we used a very conservative per-event shuffle in these surrogates, much of the sparsity of the original events was retained. Additionally, the fact that our measure is between 0 and 1 tends to limit the range of the values of sparsity that are found. Thus, differences can appear small even when they are significant. The large variability of session sparsity values across sessions, can further moderate consistent differences. This is why we show the per-session differences in the supplemental figure. Finally, when there is little structure in either the surrogate or actual data, the EM algorithm will tend to converge to solutions in which the observation matrix is degenerate. In these instances when the state likelihoods approach zero, the sparsity measure loses meaning. This reflects the fact that for limited sizes of training data, EM will tend towards solutions which overfit individual observations rather than yielding a transition matrix that would be fully connected. Despite these limitations we show that for the majority of sessions in our dataset the sparsity measure provides a useful measure of the structure of the transition and observation matrices.

2) The PBE derived model recovers the spatial response properties of the ensemble quite well. I am a bit puzzled about how this reflects the known statistics of hippocampal activity though. We know that on a given track (e.g. Figure 3), about 30% of CA1 neurons fire, while during SWRs most neurons activate. Then, how is it possible that all states derived by PBE are activated on the track? wouldn't we expect only a subset to do so?

The reviewer may be thinking of SWRs during deep sleep, whereas here we are reporting on SWRs during quiet waking on the track, during which, in fact, on a subset of neurons are active during SWRs. It is worth noting (and possibly future study) that we observed a bias (~1.5 times) for neurons with place fields vs nonplace field neurons to be activated during PBEs. Nevertheless, a major benefit of an HMM model with unsupervised learning is that with sufficient training data the learned model is often insensitive to variability that non-maze-active neurons might represent. Nevertheless, inspired by the reviewer’s comments, we have performed new analyses where we investigate events which do not decode well to trajectories in the animal’s current environment (Figure 7). Remarkably, we find that some of these events decode to trajectories in a previously-experienced remote environment.

3) I am a bit puzzled by the congruency measure. We see in Figure 4 that the model with the shuffled transition matrix yields lower likelihood scores, however in the examples shown in Figure 4B we see that the sequential structure is by and large preserved even in the shuffled model. This seems to suggest that the HMM optimization (forward/backward algorithm) is dominated by the observation probabilities, and the transition matrix is less important. But then one should ask what of the essential structure of the data is the transition matrix capturing

The reviewer is correct that the transition matrix shuffle that we have implemented by design retains much of the structure of the original data that is captured by the model. It is worth noting that we shuffle only across rows of the transition matrix, but not columns, and we exclude elements on the diagonal Our aim with this shuffle is strictly to provide a test comparison against the sequential component of the original model. To make this point better and avoid confusion, we now clarify in the text:

“Note that while we describe this as HMM-congruence, we have maintained the diagonal of the transition matrix, which specifically selects against PBEs which might be model-congruent by maintaining a single state over many time bins. In reality there are other dimensions of the HMM that we could assess congruence against, for example the observation model, the initial state distribution, or combinations of these and the transition matrix. In comparing against Bayesian decoding, our current definition seemed most appropriate for sequence detection, but we can imagine future studies expanding on our approach.”

4) I understand that organizing a "human scorers team" was probably a huge effort. Yet I am not sure that this is the best comparison for the analytical procedure, given its strongly subjective nature.

While some subjectivity is inherent in human scoring of replay, we argue here that methods and thresholds used in different studies also inherently involve subjective criterion. This inherent subjectivity is precisely what we want to reveal for readers who may have the sense that patterns which exceed a 95% threshold against a shuffle distribution are very different from those at 94.5%. We do not make the claim that human scorers provide “the best comparison” for replay detection. The point of showing results from human scorers is to contradict the sense in the literature that a binary distinction between replay sequences and non-replay can be easily identified in the a posteriori likelihood of Bayesian decoding.

5) A more general question: The bottom line of the paper is that the outcome of PBE/HMM parallels the outcome of Bayesian decoding. Yet what would be most interesting is what you may get in addition to what you get from Bayesian methods. In particular, focusing on the events that are not detected by Bayesian decoding but are detected by PBE/HMM may be interesting, and give more detail on their time distribution, structure etc.

We appreciate this suggestion of the reviewer, as it led us to add an additional section in the revised paper. Specifically, we investigate how our model quality measure changes as we remove Bayesian decoding significant sequences from the training data. We note that such an analysis is not conceivable using the Bayesian methodology. For many of our data sets, the model quality drops very quickly, which suggests to us that the PBEs in these data are largely encoding information about the local environment. However, in other data sets, including in sessions during which remote replay had previously been reported, we find that the model quality changes more gradually when Bayesian replay events are dropped. Remarkably, when we investigate specific HMM-congruent events that fail to pass the threshold for Bayesian replay in these data, we find that they correspond to remote replay of the previously-experienced environment. Thus, we now suggest that combining information from the Bayesian-decoding approach and the HMM-congruence measure in this way would be a way to specifically look for extra-spatial or remote replay.

Reviewer #3:

[…] In my opinion, the technique is promising and has potential to uncover very interesting information coding principles and test hypotheses. As a methods paper, I think it is a strong contribution to the literature. For eLife, I am unclear whether it is a good fit because I find the current draft lacking in the articulation of the scientific discovery that the technique enables. What have we learned about the system by doing the analysis in the paper? While I find the analysis interesting, I'm having trouble understanding what we've learned that has significant impact on our understanding. Similarly, for the future, the manuscript points toward some novel analysis methods that this approach would allow but never quite makes a compelling case for what we would learn from it. This is a good paper and should be published, and I am open to be convinced that it should be published in eLife. But, I think the manuscript needs to make a more compelling case for what we can learn from the data being analyzed in this way that we cannot learn otherwise.

We have revised the manuscript and performed a number of new analyses to highlight the novel benefits of our approach. First, we would argue that no previous analysis has revealed that PBEs contain enough information to form a complete map of the environment. Inspired by the second reviewer, we have also now shown how we can combine Bayesian decoding and HMM-congruence to create a signature for remote replay. While remote replay has been previously reported, no prior approach would allow for remote replays to be identified without behavioral data. Finally, we also now note how the distribution of HMM-congruence can be used to indicate the presence of consistent, ordered sequential activity in PBE data. Bayesian decoding maps between behavior tracking, neural activity during behavior, and neural activity during PBEs. By performing unsupervised learning on the PBEs, we reveal that we can infer important useful structure without these other data, and when we have them, can use them to further illuminate. We believe this is a powerful and useful technique that may have significant, widespread potential and will be of interest to many eLife readers.

- Some of the language in the Introduction regarding the motivation for the study is vague and unclear. What does it mean for the "fundamental mode" of the hippocampus to be "PBE sequences"? This seems like a statement trying to make a big claim that their study could support, but it's unclear and doesn't quite make the connection for me.

We have revised the manuscript throughout to make it more precise and attempted to remove any unsupported claims. The motivation of our study was to figure out if we could infer the structure of hippocampal activity only from PBEs. We believe that we have presented substantial evidence that this is possible. We agree that calling PBEs a "fundamental mode" would be self-aggrandizing. However, ours is a question rather than a claim and is meant to provoke further consideration; if a reader believes that "place cell activity during exploration is a fundamental mode of the hippocampus", we hope to provoke them to ask whether it is possible that PBEs might also be and to consider what evidence would demonstrate this.

- In the Results section, I think you could be a little more clear on the details like (1) what is the latent state representing (position), what is the prior, and what is the mapping to Λ (e.g., the tuning curve from state to firing rate). These come up in the Materials and methods but giving a little hint here would help make things more clear.

The text and Materials and methods section have been revised for improved clarity. We note, however, that the HMM is agnostic as to whether the latent state represents position or another construct (e.g. time or memory). In the Results section we now provide additional description:

“Hidden Markov models have been very fruitfully used to understand sequentially structured data in a variety of contexts. A hidden Markov model captures information about data in two ways. […] In prior work using HMMS to model neural activity, a variety of statistical distributions have been used to characterize ensemble firing during a specific state (the observation model.”

- I find some of the figure captions too abbreviated to actually understand what's going on in the figures. For example, in Figure 1A, I don't really understand the plot on the bottom left.

The cartoon illustration has been removed from Figure 1 and has been replaced by a latent-state-space-view that hopefully helps readers understand the salient parts of the HMM approach better than the cartoon did.

- In subsection “What do the learned model parameters tell us about PBEs?”, the latent states are used to estimate a likelihood for position on the track. The comparison is made with the shuffled data, but there is no comparison to the decoding using the standard technique (i.e., place cell tuning curves). I would expect the standard decoding to do better (since it's developed with a supervised learning approach), but it would be good to show the comparison and discuss.

We have added median Bayesian decoding error to the first panel of the Figure 3—figure supplement 1. At the end of that section, we have added the text:

"Note that decoding into the latent space and then mapping to position resulted in slightly higher error than simply performing Bayesian decoding on the neural activity during behavior. This suggests that the latent space we learn from PBEs may not capture all the information about space that is present in hippocampal activity during behavior, though this may reflect the limited number of PBEs from which we can learn."

- In subsection “What do the learned model parameters tell us about PBEs?” where the manuscript discusses assessing the patterns of activity for congruence, I think I am confused about the approach. The method is described as comparing against a single null model. As far as I understand, these results would depend entirely on the specific form of the null model that is the baseline for comparison. Why is this an acceptable approach? It seems so dependent on that arbitrary decision that I don't understand how the results are interpretable as a generalized consistency metric.

The reviewer makes a very insightful remark. We have added additional text to the Materials and methods section to clarify:

"Note that while we describe this as HMM-congruence, we have maintained the diagonal of the transition matrix, which specifically selects against PBEs which might be model-congruent by representing a single state for many time bins. In reality there are other dimensions of the HMM that we could assess congruence against, for example the observation model, the initial state distribution, or combinations of these and the transition matrix. In comparing against Bayesian decoding, our current definition seemed most appropriate, but we can imagine future studies expanding on our approach."

- There were some punctuation and wording issues that affected clarity slightly. Subsection “HMM-congruent PBEs capture sequence replay”: "between model quality or the number of PBEs and Bayesian Decoding". Subsection “Modeling internally generated activity during open 1eld behaviour”: "Can an HMM trained on PBEs indeed"

The first sentence has been rephrased:

"Not surprisingly, the performance of Bayesian decoding relative to human scorers was independent of the quality of the HMM model or the number of PBEs as the place field model is learned from ensemble neural activity during running."

And the second sentence has been rephrased:

"Do the latent states learned from Can an HMM trained on PBEs indeed capture spatial information about in a 2D environment?"

[Editors' note: further revisions were requested prior to acceptance, as described below.]

The manuscript has been greatly improved but there are some remaining issues that need to be addressed before acceptance, as outlined below.

In particular, the authors' analyses of events that were non-significant with the Bayesian approach and demonstration that many of them correspond to replayed representations of a different, previously explored environment (i.e., new Figure 7) satisfactorily address the major concerns raised during the first round of review.

However, the description of Figure 7 is not entirely clear. For example, the x-axes in panels D and I are labeled "time", but the color bar presumably corresponding to space is also shown below the x-axis. What is this supposed to indicate?

Color in panel (c) and the bottom of (e) corresponds to time (this copies the original Karlsson and Frank scheme). To make things clearer, we’ve split panel (e) into (e)/(f) and updated the caption for panel (f) to indicate that the color is the same as in (c). This allows us to remove the color bar from the middle of (d) which was understandably confusing.

Also, panels E-G and J-L need a more thorough explanation. The figure legend states that panel e (bottom) shows decoded positions using E2, which is the local environment, yet the top of panel e presumably corresponds to the remote environment (i.e., E1). This is confusing.

Alongside the lettering changes, we’ve updated the figure caption to better explain this figure:

“c–n. Two example HMM-congruent but not Bayesian-significant events from the W maze session are depicted to highlight the fact that congruence can correspond to remote replay. c. Spikes during ripple with local place cells highlighted. d. In this event, the Bayesian estimates of position using linearized local place fields have jumpy, multi-modal a posteriori distributions over space in 1D and 2D (2D depicts distribution modes and time is denoted in color). e. The replay score relative to the shuffle distribution indicates that this event is not Bayesian significant. e. Latent state probabilities decoded using the HMM trained on PBEs show sequential structure (grayscale intensity corresponds to probability). f. The distribution of HMM-score shuffles indicate the ripple event is HMM-congruent. h. Bayesian decoding using the remote environment indicate that the sample event is a remote replay. Note that in a typical experiment, only the local place fields would be available. i–n. Same as c–h, but for a different ripple event.”

The descriptions of Figure 7A-B in the Results section and the figure legend are also not clear. The text states that the model quality decreased rapidly when remote replay events were removed, and that model quality decreased slowly when local replay events were removed. But, the figure appears to show the opposite. The light blue line (local removed) decreases more rapidly than the green line (remote removed).

We apologize for this error. Indeed, as understood by the editors, the caption and text were correct, but the legend text on the figure itself mis-identified the lines and which events were being removed. This has now been corrected in the figure.

Also, are the top two slowest high quality decay sessions indicated in panel B the ones that were used to find remote replay events? It isn't clear what panel B is supposed to show- it should be explained clearly in the text and in the figure legend.

The flow of text was indeed confusing. We’ve removed the red dot from the figure, and specifically discussed the blue line and blue dot in the figure caption and the text. From the text:

“Captions: The blue line identifies a session in which model quality drops more slowly, indicating the potential presence of extra-spatial information. The reduction in session quality for a W maze experiment with known extra-spatial information is even slower (green) […] The blue dot indicates the outlier session from panel a with potential extra-spatial information: this session shows high decoding error combined with high session quality.

Text: Intriguingly, however, in at least one outlier session where model quality decreased slowly with refinement (blue line, Figure 7A), the initial model quality was still high, and we further noted that position decoding using lsPFs yielded relatively high error (blue dot, Figure 7B). Thus, we wondered whether this and similar sessions might have contained non-local or extra-spatial PBEs that were captured by the HMM.”

There is also insufficient detail provided about Figure 8. How do these figures relate to the object-location memory task? Couldn't these sleep sequences simply have been detected from exploration of the 60 x 60 enclosure regardless of object-location memory? And wouldn't they also have been potentially detected with Bayesian decoding? The authors claim in subsection “Extra-spatial Information” that the HMM can uncover sequential activity that is not readily detectable with the Bayesian approach, but it is unclear how Figure 8 shows this. In any case, Figure 8 is not really necessary as long as Figure 7 is explained clearly.

We agree with this assessment and have revised the text to better avoid unjustified claims. We’ve removed the phrase “not readily detectable with the Bayesian approach”. eloThe motivation for this analysis is also better explained—to examine the potential for HMM is uncovering sequence structure during sleep (rather than only waking as elsewhere in our manuscript), without reference to behavior. We’ve streamlined the text to emphasize model quality assessment and removed the second panel from the figure. If the editor prefers, we are also willing to remove this figure entirely.

A few minor issues also remain to be clarified. In subsection “Awake population burst events”, the authors state that similar results were obtained when a theta detection approach was used, but it is unclear where this approach is shown or described.

For the sake of space, we did not show these results. We’ve updated the text to state this.

“While we identified active behavior using a speed criterion, we found similar results when we instead used a theta-state detection approach (not shown).”

Also, in subsection “Awake population burst events”, it is unclear to what "these" refers.

We changed the text to state:

“We found that inactive PBEs occupied an average of 1.8% of the periods during which animals were on the linear track.”

Also, in Figure 1B, the "pyramidal cells (non place)" label should be changed to "putative pyramidal cells (non place)".

This has been updated.

Also, the difference between the mean departure Gini and mean Λ row Gini in Figure 6E-F should be explained more clearly for a general readership.

We modified the earlier text to better explain the intuition behind Gini coefficients:

“First, we investigated the sparsity of the transition matrices using the Gini coefficient (see Materials and methods and Figure 2—figure supplement 1). A higher Gini coefficient corresponds to higher sparsity. Strikingly, the actual data yielded models in which the state transition matrix was sparser than in each of the surrogate counterparts (p < 0.001, Figure 2C), reflecting that each state transitions only to a few other states. Thus, intricate yet reliable details are captured by the HMM. Next, we quantified the sparsity of the observation model. We found that actual data yielded mean firing rates which were highly sparse (Figure 2D), indicating that individual neurons were likely to be active during only a small fraction of the states.”

And, we changed the labels in Figure 6 to match Figure 2, and updated the caption to say:

“Comparing the sparsity of the transition matrices (mean Gini coefficient of the departure probabilities) between the linear track and open field reveals that, as expected, over the sessions we observed, the open field is significantly less sparse (p<0.001), since the environment is less constrained. f. In contrast, there is not a significant difference between the sparsity of the observation model (mean Gini coefficient of the rows) between the linear track and the open field.”

Finally, as was raised in the initial review, the authors should explicitly state in the first two paragraphs of the Materials and methods section that the Diba and Buzsaki and Pfeiffer and Foster datasets were recorded previously and used in other publications. Otherwise, readers may misunderstand and assume that methods were adopted from these papers, but that similar data were collected again for the present study.

We now say “As previously reported using these same data” in reference to Diba and Buzsaki, and “Briefly, as was previously reported using this dataset” in reference to Pfeiffer and Foster.

https://doi.org/10.7554/eLife.34467.026

Article and author information

Author details

  1. Kourosh Maboudi

    1. Departmentof Anesthesiology, University of Michigan, Ann Arbor, United States
    2. Department of Psychology, University of Wisconsin-Milwaukee, Milwaukee, United States
    Contribution
    Conceptualization, Data curation, Software, Methodology, Writing—original draft, Writing—review and editing
    Contributed equally with
    Etienne Ackermann
    Competing interests
    No competing interests declared
    ORCID icon 0000-0001-9675-4031
  2. Etienne Ackermann

    Department of Electrical and Computer Engineering, Rice University, Houston, United States
    Contribution
    Conceptualization, Resources, Data curation, Software, Visualization, Methodology, Writing—original draft, Writing—review and editing
    Contributed equally with
    Kourosh Maboudi
    Competing interests
    No competing interests declared
  3. Laurel Watkins de Jong

    1. Departmentof Anesthesiology, University of Michigan, Ann Arbor, United States
    2. Department of Psychology, University of Wisconsin-Milwaukee, Milwaukee, United States
    Contribution
    Gathered experimental data, Data curation, Data curation, Writing—review and editing
    Competing interests
    No competing interests declared
  4. Brad E Pfeiffer

    Department of Neuroscience, University of Texas Southwestern, Dallas, United States
    Contribution
    Gathered experimental data, Data curation, Data curation, Writing—review and editing
    Competing interests
    No competing interests declared
  5. David Foster

    1. Department of Psychology, University of California, Berkeley, Berkeley, United States
    2. Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, United States
    Contribution
    Data curation, Writing—review and editing
    Competing interests
    No competing interests declared
  6. Kamran Diba

    1. Departmentof Anesthesiology, University of Michigan, Ann Arbor, United States
    2. Department of Psychology, University of Wisconsin-Milwaukee, Milwaukee, United States
    Contribution
    Conceptualization, Gathered experimental data, Data curation, Supervision, Methodology, Writing—original draft, Writing—review and editing
    Contributed equally with
    Caleb Kemere
    For correspondence
    kdiba@umich.edu
    Competing interests
    No competing interests declared
    ORCID icon 0000-0001-5128-4478
  7. Caleb Kemere

    Department of Electrical and Computer Engineering, Rice University, Houston, United States
    Contribution
    Conceptualization, Software, Supervision, Methodology, Writing—original draft, Writing—review and editing
    Contributed equally with
    Kamran Diba
    For correspondence
    caleb.kemere@rice.edu
    Competing interests
    No competing interests declared
    ORCID icon 0000-0003-2054-0234

Funding

National Science Foundation (IOS-1550994)

  • Etienne Ackermann
  • Caleb Kemere

Human Frontier Science Program (RGY0088)

  • Etienne Ackermann
  • Caleb Kemere

Ken Kennedy Institute (ERIT)

  • Caleb Kemere

National Institute of Mental Health (R01MH109170)

  • Kourosh Maboudi
  • Kamran Diba

National Institute of Mental Health (R01MH085823)

  • Brad E Pfeiffer
  • David Foster

Alfred P. Sloan Foundation

  • Brad E Pfeiffer
  • David Foster

Brain and Behavior Research Foundation (NARSAD Young Investigator Grant)

  • Brad E Pfeiffer
  • David Foster

McKnight Endowment Fund for Neuroscience

  • David Foster

National Science Foundation (CBET-1351692)

  • Etienne Ackermann

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Ethics

Animal experimentation: As reported previously, all procedures were approved by the Johns Hopkins University, Rutgers University, and University of California, San Francisco Animal Care and Use Committees and followed US National Institutes of Health animal use guidelines (protocol 90-042).

Reviewing Editor

  1. Laura Colgin, Reviewing Editor, The University of Texas at Austin, Center for Learning and Memory, United States

Publication history

  1. Received: December 20, 2017
  2. Accepted: May 14, 2018
  3. Accepted Manuscript published: June 5, 2018 (version 1)
  4. Version of Record published: June 21, 2018 (version 2)

Copyright

© 2018, Maboudi et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 2,182
    Page views
  • 424
    Downloads
  • 0
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Download citations (links to download the citations from this article in formats compatible with various reference manager tools)

Open citations (links to open the citations from this article in various online reference manager services)

Further reading

    1. Neuroscience
    Jiefeng Jiang et al.
    Research Article
    1. Neuroscience
    2. Structural Biology and Molecular Biophysics
    Doreen Matthies et al.
    Research Article