Network model and sequence retrieval. a. Schematic of network connectivity after learning with a plasticity rule that combines temporally symmetric and asymmetric com-ponents. The network stores a sequence of patterns that activate non-overlapping sets of neurons (colored according to the pattern that activates them). Note connections both within each set, and from one set to the next. b. Correlation of each stored pattern with network activity following initialization to the first pattern. Retrieval speed is fixed by the balance of symmetry/asymmetry at the synapse. c. Relative retrieval speed as a function of temporal symmetry (z), showing linear relationship. Solid line: 1 − z, the speed computed from MFT (see Methods). Black dots: Network simulations. d. Connectivity of a network with two types of neurons, asymmetric (left) and symmetric (right). Note that the connections from left neurons project to neurons active in the next pattern in the sequence, while connections from right neurons project to neurons active in the same pattern as the pre-synaptic neu-ron. The two types of neurons can be driven differentially by external inputs ( and, respectively) e. Solid lines: correlations as in (a) for two distinct pairs of input strengths (in the range [-1,0] for and), demonstrating two different retrieval speeds. Dashed lines: correlations with noisy time-dependent heterogeneous input added to the network (see Methods). In the simulations shown on the center and right panels, N = 80,000, c = 0.005, τ = 10ms, P = 16, A = 2, θ = 0, and σ = 0.1. For simplicity, we depict only 3 of the 16 stored patterns in the left schematics.

Retrieval properties depend on external inputs. a. Retrieval quality, defined as the peak correlation mP of the final pattern in the sequence, as a function of external inputs to asymmetric population, and to symmetric population. The white line bounds the region of successful retrieval. Below this line (black region), retrieval is not possible, regardless of initial condition (see Methods). b. Retrieval speed, measured by averaging the inverse time between consecutive pattern correlation peaks (see Methods). c. Solid lines: firing rates of three randomly selected neurons during retrieval for parameters corresponding to the circle (left) and diamond (right) in panels (a-b), which are the same parameters used in Fig. 1e. Note the approximate (but not exact) temporal scaling by a factor ∼ 3 between these two sets of external inputs. Dashed lines: firing rates in response to the same noisy inputs as in Fig. 1e. d. Activity of 125 units (from a total of 80,000), sorted by peak firing rate time, for parameters corresponding to the circle (left) and diamond (right) in panels (a-b). All other parameters are as in Fig. 1b.

Retrieval properties in networks with nonlinear learning rules. a. Correlations between stored patterns and network states in asymmetric (left) and symmetric (right populations, for three different external input combinations (denoted by the inset symbol, see right panel). b. Retrieval speed as a function of parameters describing external inputs, similarly as in Fig. 2. White dots indicate the region in which stable persistent activity of the first pattern is present. Hatched diagonal lines indicate the region in which incomplete sequential activity terminates in stable persistent activity. All parameters are as in Fig. 2, except A = 20 and σ = 0.05. The parameters of the learning rule are xf = 1.5, xg = 1.5, qf = 0.8, and qg = 0.933.

Transition from persistent activity (‘preparatory’ period) to sequence retrieval (‘execution’ period) mediated by external input a. Inputs provided to the asymmetric (black) and symmetric population (orange) consist of a “preparatory period” input lasting 200ms, followed by an “execution period” input that is fixed for the rest of the interval. During a 200ms preparatory period, a brief input is presented to the symmetric population for the first 10ms, which drives the network to a state which is strongly correlated with the first pattern in a sequence. This input is removed after 10ms, but the network remains in a persistent activity state corresponding the the first pattern, because a strong negative input is presented to the asymmetric population throughout the entire 200ms, which prevents the network from retrieving the sequence. At the end of this period, the input to the symmetric population is decreased, while the asymmetric population is increased, which leads to retrieval of the sequence (‘execution period’). Sequence retrieval can happen at different speeds, depending on the inputs to the asymmetric and symmetric populations. b. Correlations with stored patterns in the sequence in each population, in each input scenario. Note correlations in the slow retrieval case are temporally scaled by a factor ∼ 2.5 compared to the fast retrieval case. c. Example single unit firing rates in each population. Note that for some neurons firing rates do not follow a simple temporal rescaling - for instance the purple neuron in the symmetric population is active at around t = 0.45 in the slow retrieval case, but is not active in the fast retrieval case. All parameters are as in Fig. 3, except θ = 0.07 and σ = 0.05.

Retrieval of sequences in networks with heterogeneous learning rules described by a continuum of degrees of symmetry. a. Firing rate dynamics of five representative neurons during retrieval for each external input configuration (see inset symbols in panel c). b. Correlation of network activity with each stored pattern during retrieval for each external input configuration. c. Retrieval speed as described in Fig. 2. All parameters are as in Fig. 3 except A = 10.

Desired target speeds can be reached by adjusting external inputs using a reward-based learning rule. The black and grey lines denote the trajectories for two learning trials targeting different speeds. External inputs start at − 0.2 (marked with a circle) and terminate at values implementing desired target speeds of 0.8 and 0.3 (marked with crosses). All parameters are as in Fig. 2.

Retrieval in a network of excitatory and inhibitory spiking neurons, in which excitatory neurons are subdivided into asymmetric and symmetric populations. a. Fast retrieval when inputs are biased to asymmetric neurons. Top: Correlation of network activity with each stored pattern. Middle: Voltage traces of three representative neurons. Bottom: The bottom two panels show raster plots of excitatory and inhibitory neurons, sorted by the latency of their peak firing rate. b. Slow retrieval when inputs are biased to symmetric neurons. Note that only external inputs differ in (a) and (b). See Methods for parameters.