Abstract
Representational drift refers to the dynamic nature of neural representations in the brain despite the behavior being seemingly stable. Although drift has been observed in many different brain regions, the mechanisms underlying it are not known. Since intrinsic neural excitability is suggested to play a key role in regulating memory allocation, fluctuations of excitability could bias the reactivation of previously stored memory ensembles and therefore act as a motor for drift. Here, we propose a rate-based plastic recurrent neural network with slow fluctuations of intrinsic excitability. We first show that subsequent reactivations of a neural ensemble can lead to drift of this ensemble. The model predicts that drift is induced by co-activation of previously active neurons along with neurons with high excitability which leads to remodeling of the recurrent weights. Consistent with previous experimental works, the drifting ensemble is informative about its temporal history. Crucially, we show that the gradual nature of the drift is necessary for decoding temporal information from the activity of the ensemble. Finally, we show that the memory is preserved and can be decoded by an output neuron having plastic synapses with the main region.
Introduction
In various brain regions, the neural code tends to be dynamic although behavioral outputs remain stable. Representational drift refers to the dynamic nature of internal representations as they have been observed in sensory cortical areas [1–3] or the hippocampus [4, 5] despite stable behavior. It has even been suggested that pyramidal neurons from the CA1 and CA3 regions form dynamic rather than static memory engrams [5, 6], namely that the set of neurons encoding specific memories varies across days. In the amygdala, retraining of a fear memory task induces a turnover of the memory engram [7]. Additionally, plasticity mechanisms have been proposed to compensate for drift and to provide a stable read-out of the neural code [8], suggesting that information is maintained. Altogether, this line of evidence suggest that drift might be a general mechanism with dynamical representations observed in various brain regions.
However, the mechanisms underlying the emergence of drift and its relevance for the neural computation are not known. Drift is often thought to arise from variability of internal states [2], neurogenesis [1, 9] or synaptic turnover [10] combined with noise [11, 12]. On the other hand, excitability might also play a role in memory allocation [13–16], so that neurons having high excitability are preferentially allocated to memory ensembles [16–18]. Moreover, excitability is known to fluctuate over timescales from hours to days, in the amygdala [18], the hippocampus [17, 19] or the cortex [20, 21]. Subsequent reactivations of a neural ensemble at different time points could therefore be biased by excitability [22], which varies at similar timescales than drift [23]. Altogether, this evidence suggest that fluctuations of excitability could act as a cellular mechanism for drift [14].
In this short communication, we aimed at proposing how excitability could indeed induce a drift of neural ensembles at the mechanistic level. We simulated a recurrent neural network [24] equipped with intrinsic neural excitability and Hebbian learning. As a proof of principle, we first show that slow fluctuations of excitability can induce neural ensembles to drift in the network. We then explore the functional implications of such drift. Consistent with previous works [23, 25–27], we show that neural activity of the drifting ensemble is informative about the temporal structure of the memory. This suggest that fluctuations of excitability can be useful for time-stamping memories (i.e. for making the neural ensemble informative about the time at which it was form). Finally, we confirmed that the content of the memory itself can be steadily maintained using a read-out neuron and local plasticity rule, consistently with previous computational works [8]. The goal of this study is to show one possible mechanistic implementation of how excitability can drive drift.
Results
Many studies have shown that memories are encoded in sparse neural ensembles that are activated during learning and many of the same cells are reactivated during recall, underlying a stable neural representation [14, 28, 29]. After learning, subsequent reactivations of the ensemble can happen spontaneously during replay, retraining or during a memory recall task (e.g. following presentation of a cue [28, 30]). Here, we directly tested the hypothesis that slow fluctuations of excitability can change the structure of a newly-formed neural ensemble, through subsequent reactivations of this ensemble. To that end, we designed a rate-based, recurrent neural network, equipped with intrinsic neural excitability (Methods). We considered that the recurrent weights are all-to-all and plastic, following a Hebbian rule (Methods). The network was then stimulated following a 4-day protocol: the first day corresponds to the initial encoding of a memory and the other days correspond to spontaneous or cue-induced reactivations of the neural ensemble (Methods). Finally, we considered that excitability of each neuron can vary on a day long timescale: each day, a different subset of neurons has increased excitability (Fig. 1a, Methods).
Fluctuations of intrinsic excitability induce drifting of neural ensembles
While stimulated the naive network on the first day, we observed the formation of a neural ensemble: some neurons gradually increase their firing rate (Fig. 1b and c, neurons 10 to 20, time steps 1000 to 3000) during the stimulation. We observed that these neurons are highly recurrently connected (Fig. 1d, leftmost matrix) suggesting that they form an assembly. This assembly is composed of neurons that have a high excitability (Fig. 1a, neurons 10 to 20 have increase excitability) at the time of the stimulation. We then show that further stimulations of the network induce a remodeling of the weights. During the second stimulation for instance (Fig. 1b and c, time steps 4000 to 6000), neurons from the previous assembly (10 to 20) are reactivated along with neurons having high excitability at the time of the second stimulation (Fig. 1a, neurons 20 to 30). Moreover, across several days, recurrent weights from previous assemblies tend to decrease while others increase (Fig. 1d). Indeed, neurons from the original assembly (Fig. 1c, red traces) tend to be replaced by other neurons, either from the latest assembly or from the pool of neurons having high excitability. This is translated at the synaptic level, where weights from previous assemblies tend to decay and be replaced by new ones. Overall, each new stimulation updates the ensemble according to the current distribution of excitability, inducing a drift towards neurons with high excitability. Finally, in our model, the drift rate does not depend on the size of the original ensemble (Fig. S2, Methods).
Activity of the drifting ensemble is informative about the temporal structure of the past experience
After showing that fluctuations of excitability can induce a drift among neural ensembles, we tested whether the drifting ensemble could contain temporal information about its past experiences, as suggested in previous works [25]. Inspired by these works, we asked whether it was possible to decode relevant temporal information from the patterns of activity of the neural ensemble. We first observed that the correlation between patterns of activity after just after encoding across days decreases (Fig. 2a, Methods), indicating that after each day, the newly formed ensemble resembles less the original one. Because the patterns of activity differ across days, they should be informative about the absolute day from which they were recorded. To test this hypothesis, we designed a day decoder (Fig. 2b, Methods), following the work of Rubin et al., 2015 [25]. This decoder aims at inferring the reactivation day of a given activity pattern by comparing the activity of this pattern during training and the activity just after memory encoding without increase in excitability (Fig. 2b, Methods). We found that the day decoder perfectly outputs the reactivation day as compared to using shuffled data (Fig. 2c, blue and orange bars).
After showing that the patterns of activity are informative about the reactivation day, we took a step further by asking to whether the activity of the neurons is also informative about the order in which the memory evolved. To that end, we used an ordinal time decoder (Methods, as in Rubin et al., 2015 [25]) that uses the correlations between activity patterns for pairs of successive days, and for each possible permutation of days p (Fig. 2d, Methods). The sum of these correlations S(p) differs from each permutation p and we assumed that the neurons are informative about the order at which the reactivations of the ensemble happened if the permutation maximising S(p) corresponds to the real permutation preal (Fig. 2e, Methods). We found that S(preal) was indeed statistically higher than S(p) for the other permutations E (Fig. 2f, Student’s t-test, Methods). However, this was only true when the amplitude of the fluctuations of excitability E was in to a certain range. Indeed, when the amplitude of the fluctuations is null, i.e. when excitability is not increased (E = 0), the ensemble does not drift (Fig. S1a). In this case, the patterns of activity are not informative about the order of reactivations. On the other hand, if the excitability amplitude is too high (E = 3), each new ensemble is fully determined by the distribution of excitability, regardless of any previously formed ensemble (Fig. S1c). In this regime, the patterns of activity are not informative about the order of the reactivations either. In the intermediate regime (E = 1.5), the decoder is able to correctly infer the order at which the reactivations happened, better than using the shuffled data (Fig. 2f, Fig. S1b).
Finally, we sought to test whether the results are independent on the specific architecture of the model. To that end, we defined a change of excitability as a change in the slope of the activation function, rather than of the threshold (Fig. S5, Methods). We also used sparse recurrent synaptic weights instead of the original all-to-all connectivity matrix (Fig. S4, Methods). In both cases, we observed a similar drifting behavior and were able to decode the temporal order in which the memory evolved.
A read-out neuron can track the drifting ensemble
So far, we showed that the drifting ensemble contains information about its history, namely about the days and the order at which the subsequent reactivations of the memory happened. However, we have not shown that we could use the neural ensemble to actually decode the memory itself, in addition to its temporal structure. To that end, we introduced a decoding output neuron connected to the recurrent neural network, with plastic weights following a Hebbian rule (Methods). As shown by Rule et al., 2022 [8], the goal was to make sure that the output neuron can track the ensemble even if it is drifting. This can be down by constantly decreasing weights from neurons that are no longer in the ensemble and increasing those associated with neurons joining the ensemble (Fig. 3a). We found that the output neuron could steadily decode the memory (i.e. it has a higher firing than in the case where the output weights are randomly shuffled; Fig. 3b, blue trace for the real output and white, orange and red traces for the shuffled weights). This is due to the fact that weights are plastic under Hebbian learning, as shown by Rule et al. 2022 [8]. We confirmed that this was induced by a change in the output weights across time (Fig. 3c). In particular, the weights from neurons that no longer belong to the ensemble are decreased while weights from newly recruited neurons are increased, so that the center of mass of the weights distribution drifts across time (Fig. 3d). Finally, we found that the quality of the read-out decreases with the rate of the drift (Fig. S3, Methods).
Two memories drift independently
Finally, we tested whether the network is able to encode two different memories and whether excitability could make two ensembles drift. On each day, we stimulated a random half of the neurons (context A) and the other half (context B) sequentially (Methods). We found that, day after day, the two ensembles show a similar drift than when only one ensemble was formed (Fig. S6). In particular, the correlation between the patterns activity on the first day and the other days decay in a similar way (Fig. S7.a). For both contexts, the temporal order of the reactivations can be decoded for a certain range of excitability amplitude (Fig. S7.b). Finally, we found that using two output decoders allowed us to decode both memories independently. The output weights associated to both ensembles are remodeled to follow the drifting ensembles, but are not affected by the reactivation of the other ensemble (Fig. S7.c). Indeed, both neurons are able to “track” the reactivation of their associated ensemble while not responding to the other ensemble (Fig. S7.d).
Discussion
Overall, our model suggests a potential cellular mechanisms for the emergence of drift that can serve a computational purpose by “time-stamping” memories while still being able to decode the memory across time. Although the high performance of the day decoder was expected, the performance of the ordinal time decoder is not trivial. Indeed, the patterns of activity of each day are informative about the distribution of excitability and therefore about the day at which the reactivation happened. However, the ability for the neural ensemble to encode the order of past reactivations requires drift to be gradual (i.e. requires consecutive patterns of activity to remain correlated across days). Indeed, if the amplitude of excitability is too low (E = 0) or too high (E = 3), it is not possible to decode the order at which the successive reactivations happened. This result is consistent with the previous works showing gradual change in neural representations, that allows for decoding temporal information of the ensemble [25]. Moreover, such gradual drifts could support complex cognitive mechanisms like mental time-travel during memory recall [25].
In our model, drift is induced by co-activation of the previously formed ensemble and neurons with high excitability at the time of the reactivation. The pool of neurons having high excitability can therefore “time-stamps” memory ensembles by biasing allocation of these ensembles [23, 25, 26]. We suggest that such time-stamping mechanism could also help link memories that are temporally close and dissociate those which are spaced by longer time [3, 14, 31]. Indeed, the pool of neurons with high excitability varies across time so that any new memory ensemble is allocated to neurons which are shared with other ensembles formed around the same time. This mechanism could be complementary to the learning-induced increase in excitability observed in amygdala [18], hippocampal CA1 [17] and dentate gyrus [32].
Finally, we intended to model drift in the firing rates, as opposed to a drift in the turning curve, of the neurons. Recent studies suggest that drifts in the mean firing rate and tuning curve arise from two different mechanisms [33, 34]. Experience drives a drift in neurons turning curve while the passage of time drives a drift in neurons firing rate. In this sense, our study is consistent with these findings by providing a possible mechanism for a drift in the mean firing rates of the neurons driven a dynamical excitability. Our work suggests that drift can depend on any experience having an impact on excitability dynamics such as exercise as previously shown experimentally [9, 35] but also neurogenesis [9, 31, 36], sleep [37] or increase in dopamine level [38]. Overall, our work is a proof of principle which highlights the importance of considering excitability when studying drift, although further work would be needed to test this link experimentally.
Methods
Recurrent neural network with excitability
Our rate-based model consists of a single region of N neurons (with firing rate ri, 1≤ i≤ N). All-to-all recurrent connections W are plastic and follow a Hebbian rule given by:
where i and j correspond to the pre- and post-synaptic neuron respectively. τW and τdecay are the learning and the decay time constants of the weights, respectively. A hard bound of [0, c] was applied to these weights. We also introduced a global inhibition term dependent on the activity of the neurons:
Where I0, I1 and I2 are positive constants. All neurons receive the same input, Δ(t), during stimulation of the network (Fig. 1c, black bars). Finally, excitability is modeled as a time-varying threshold εi of the input-output function of each neuron i. The rate dynamics of a neuron i is given by:
where τr is the decay time of the rates and ReLU is the rectified linear activation function. We considered that a neurons is active when its firing rate reaches the active threshold θ.
In Fig. S4, we applied a random binary mask to the recurrent weights in order to set 50% of the synapses at 0. A new mask was randomly sampled for each simulation.
In Fig. S5, we modeled excitability as a change of the slope of the activation function (ReLU) instead of a change of the threshold as previously used (Fig. S5.a):
Protocol
We designed a 4-day protocol, corresponding to the initial encoding of a memory (1st day) and subsequent random or cue-induced reactivations of the ensemble [28, 30] (2nd, 3rd and 4th day). Each stimulation consists of Nrep repetitions of interval T spaced by a inter-repetition delay IR. Δ(t) takes the value δ during these repetitions and is set to 0 otherwise. The stimulation is repeated four times, modeling four days of reactivation, spaced by an inter-day delay ID. Excitability εi of each neuron i is sampled from a chi-squared distribution of parameter 1. In Fig. S5, excitability εi is sampled from the square of a normal distribution of mean 0.4 and standard deviation 0.2. Neurons 10 to 20, 20 to 30, 30 to 40 and 40 to 50 then receive an increase of excitability of amplitude E, respectively on days 1, 2, 3 and 4 (Fig. 1a). A different random seed is used for each repetitions of the simulations. When two memories were modeled (Fig. S6,S7), we stimulated a random half of the neurons (context A) and the other half (context B) successively (Fig. S6.a), every day.
Decoders
For each day d, we recorded the activity pattern Vd, which is a vector composed of the firing rate of the neurons at the end of the last repetition of stimulation. To test the decoder, we also stimulated the network while setting the excitability at baseline (E = 0), and recorded the resulted pattern of activity for each day d. We then designed two types of decoders, inspired by previous works [25]: (1) a day decoder which infers the day at which each stimulation happened and (2) an ordinal time decoder which infers the order at which the reactivations occurred. For both decoders, the shuffled data was obtained by randomly shuffling the day label of each neuron. When two memories were modeled (Fig. S6,S7), the patterns of activity were taken at the end of the stimulations by context A and B, and the decoders were used independently on each memory.
1. The day decoder aims at inferring the day at which a specific patterned of activity occurred. To that end, we computed the Pearson correlation between the pattern with no excitability of the day d and the patterns of all days d′ from the first simulation Vd′. Then, the decoder outputs the day dinf that maximises the correlation:
The error was defined as the difference between the inferred and the real day dinf − d.
2. The ordinal time decoder aims at inferring the order at which the reactivations happened from the patterns of activity Vd of every days d. To that end, we computed the pairwise correlations of each consecutive days, for the 4! possible permutations of days p. The real permutation is called preal = (1, 2, 3, 4) and corresponds to the real order of reactivations: day 1→ day 2→ day 3→ day 4. The sum of these correlations over the 3 pairs of consecutive days is expressed as:
We then compared the distribution of these quantities for each permutation p to that of the real permutation S(preal) (Fig. 2). The patterns of activity are informative about the order of reactivations if S(preal) corresponds to the maximal value of S(p). To compare S(preal) with the distribution S(p), we performed a Student’s t-test, where the t-value is defined as :
where μ and σ corresponds to the mean and standard deviation of the distribution S(p), respectively.
The drift rate Δ (Fig. S2 and Fig. S3) was computed as:
Memory read-out
To test if the network is able to decode the memory at any time point, we introduced a read-out neuron with plastic synapses to neurons from the recurrent network, inspired by previous computational works [8]. The weights of these synapses are named and follow the Hebbian rule defined as:
where and corresponds to the learning time and decay time constant, respectively. h(W out) is a homeostatic term defined as which decreases to 0 throughout learning. h takes the value 1 before learning and 0 when the sum of the weights reaches the value 1. y is the firing rate of the output neuron defined y as:
The read-out quality index Q (Fig. S3) was defined as:
where yd corresponds to the value of y taken at the end of the last repetition of day d, and equivalent with shuffled outputs weights. indicates the average over Nshuffle = 10 simulations.
In Fig. S6,S7, two output decoders yk, k ∈ {1, 2 }, with corresponding weights are defined as:
and follow the Hebbian rule defined as:
Then, we aimed at allocating y1 and y2 to the first and the second ensemble (context A and B), respectively. To that end, we used supervised learning on the first day by adding a current βk to the output neurons which is positive when the corresponding context is on:
The shuffled traces were obtained by randomly shuffling the output weights Wout or for each ensemble k.
Table of parameters
The following parameters have been used for the simulations. When unspecified, the defaults values were used. All except N are in arbitrary unit. Fig. S5 corresponds to the change from a threshold-based to a slope-based excitability. Fig. S6,S7 corresponds to the stimulation of two ensembles. Fig. S4 corresponds to the sparsity simulation.
Supplementary
References
- 1.Dynamic Reorganization of Neuronal Activity Patterns in Parietal CortexCell 170:986–999https://doi.org/10.1016/j.cell.2017.07.021
- 2.Contribution of behavioural variability to representational drifteLife 11https://doi.org/10.7554/eLife.77907
- 3.Representational drift: Emerging theories for continual learning and experimental future directionsCurrent Opinion in Neurobiology 76https://doi.org/10.1016/j.conb.2022.102609
- 4.Long-term dynamics of CA1 hippocampal place codesNature Neuroscience 16:264–266https://doi.org/10.1038/nn.3329
- 5.Parallel emergence of stable and dynamic memory engrams in the hippocampusNature 558:292–296https://doi.org/10.1038/s41586-018-0191-2
- 6.Continuous attractors for dynamic memorieseLife 10https://doi.org/10.7554/eLife.69499
- 7.Turnover of fear engram cells by repeated experienceCurrent Biology 31https://doi.org/10.1016/j.cub.2021.10.004
- 8.Self-healing codes: How stable neural populations can track continually reconfiguring neural representationsProceedings of the National Academy of Sciences 119https://doi.org/10.1073/pnas.2106692119
- 9.Exercise increases information content and affects long-term stability of hippocampal place codesCell Reports 41https://doi.org/10.1016/j.celrep.2022.111695
- 10.Impermanence of dendritic spines in live adult CA1 hippocampusNature 523:592–596https://doi.org/10.1038/nature14467
- 11.Drifting assemblies for persistent memory: Neuron transitions and unsupervised compensationProceedings of the National Academy of Sciences 118https://doi.org/10.1073/pnas.2023832118
- 12.Manz, P. & Memmesheimer, R.-M. Purely STDP-based assembly dynamics: stability, learning, overlaps, drift and aging Mar. 2023. doi:10.1101/2022.06.20.496825.https://doi.org/10.1101/2022.06.20.496825
- 13.CREB regulates excitability and the allocation of memory to subsets of neurons in the amygdalaNature Neuroscience 12:1438–1443https://doi.org/10.1038/nn.2405
- 14.The brain in motion: How ensemble fluidity drives memoryupdating and flexibilityeLife 9https://doi.org/10.7554/eLife.63550
- 15.Synaptic tagging during memory allocationNature Reviews Neuroscience 15:157–169https://doi.org/10.1038/nrn3667
- 16.Molecular and Cellular Approaches to Memory Allocation in Neural CircuitsScience 326:391–395https://doi.org/10.1126/science.1174519
- 17.A shared neural ensemble links distinct contextual memories encoded close in timeNature 534:115–118https://doi.org/10.1038/nature17955
- 18.Competition between engrams influences fear memory formation and recallScience 353:383–387https://doi.org/10.1126/science.aaf0594
- 19.Diversity in neural firing dynamics supports both rigid and learned hippocampal sequencesScience 351:1440–1443https://doi.org/10.1126/science.aad1935
- 20.Human Cortical Excitability Increases with Time AwakeCerebral Cortex 23:1–7https://doi.org/10.1093/cercor/bhs014
- 21.NREM sleep in the rodent neocortex and hippocampus reflects excitable dynamicsNature Communications 10https://doi.org/10.1038/s41467-019-10327-5
- 22.Mau, W. et al. Ensemble remodeling supports memory-updating June 2022. doi:10.1101/2022.06.02.494530.https://doi.org/10.1101/2022.06.02.494530
- 23.The Same Hippocampal CA1 Population Simultaneously Codes Temporal Information over Multiple TimescalesCurrent Biology 28:1499–1508https://doi.org/10.1016/j.cub.2018.03.051
- 24.Intrinsic neural excitability induces time-dependent overlap of memory engramsbioRxiv https://doi.org/10.1101/2022.08.27.505441
- 25.Hippocampal ensemble dynamics timestamp events in long-term memoryeLife 4https://doi.org/10.7554/eLife.12247
- 26.Variance and invariance of neuronal longterm representationsPhilosophical Transactions of the Royal Society B: Biological Sciences 372https://doi.org/10.1098/rstb.2016.0161
- 27.Memory: Ironing Out a Wrinkle in TimeCurrent Biology 28:R599–R601https://doi.org/10.1016/j.cub.2018.03.053
- 28.Memory engrams: Recalling the past and imagining the futureScience 367https://doi.org/10.1126/science.aaw4325
- 29.What is memory? The present state of the engramBMC Biology 14https://doi.org/10.1186/s12915-016-0261-6
- 30.Off-line replay maintains declarative memories in a model of hippocampalneocortical interactionsNature Neuroscience 7:286–294https://doi.org/10.1038/nn1202
- 31.Potential role for adult neurogenesis in the encoding of time in new memoriesNature Neuroscience 9:723–727https://doi.org/10.1038/nn1707
- 32.Engram Cell Excitability State Determines the Efficacy of Memory RetrievalNeuron 101:274–284https://doi.org/10.1016/j.neuron.2018.11.029
- 33.Time and experience differentially affect distinct aspects of hippocampal representational driftNeuron https://doi.org/10.1016/j.neuron.2023.05.005
- 34.Active experience, not time, determines within-day representational drift in dorsal CA1Neuron 111:2348–2356https://doi.org/10.1016/j.neuron.2023.05.014
- 35.Exercise accelerates place cell representational driftCurrent Biology 33:R96–R97https://doi.org/10.1016/j.cub.2022.12.033
- 36.Adult neurogenesis acts as a neural regularizerProceedings of the National Academy of Sciences 119https://doi.org/10.1073/pnas.2206704119
- 37.Sleep regulation of the distribution of cortical firing ratesCurrent Opinion in Neurobiology. Neurobiology of Sleep 44:34–42https://doi.org/10.1016/j.conb.2017.02.013
- 38.A locus coeruleus-dorsal CA1 dopaminergic circuit modulates memory linkingNeuron https://doi.org/10.1016/j.neuron.2022.08.001
Article and author information
Author information
Version history
- Preprint posted:
- Sent for peer review:
- Reviewed Preprint version 1:
- Reviewed Preprint version 2:
- Version of Record published:
Copyright
© 2023, Delamare et al.
This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.
Metrics
- views
- 1,899
- downloads
- 141
- citations
- 6
Views, downloads and citations are aggregated across all versions of this paper published by eLife.