A plastic recurrent network can generate sequences under unstructured inputs.

A. Schematic of the model network with 1,200 excitatory and 240 inhibitory AdEx neurons, where E-to-E and I-to-E connections are plastic. B. Example EPSPs initiated by connections with different Weff on an isolated neuron. The effective weight of a connection (Weff) depends on the synaptic weight (W ), the postsynaptic resting potential (V0), and the firing threshold (Vth). C. The network develops a lognormal E-to-E weight distribution in the steady-state. D. Different presynaptic neurons have similarly-distributed output connections. Solid line indicates the average and shade represents the standard deviation. Pooled over all E neurons in the network (n = 1, 200). E. Top: When a randomly chosen “source” neuron is forced to spike, its “followers” activate reliably and sequentially. Neurons are sorted by their spike time in example trial. Blue and red ticks indicate excitatory and inhibitory followers. Bottom: the source neuron was forced to spike in 1,000 consecutive trials and increases in firing rate were calculated to identify reliable followers. F. The jitter of the followers is positively correlated with the median delay. Same followers as E.

Turnover of strong connections in the plastic network.

A. Evolution of weight distribution (line and shading; mean ±0020std) and example single connections (crosses) at multiple time points. Colored crosses indicate strong connections (Weff > 2.5). The distribution reaches a steady state, but single connections continue to fluctuate in strength. B. Ratio of strong E-to-E connections over time. C. Average strength of multiple groups of E-to-E connections randomly picked from the tail of the distribution at different time points (colored labels), after the weight distribution has reached a steady state. D. Decay of strong E-to-E (blue) and I-to-E connections (red). Dots indicate the ratio of strong connections that remain strong after a given interval (abscissa). Lines indicate exponential fits. Gray represents E-to-E connections when neurons are forced to fire Poisson spike trains. τee = 1, 719 s, τie = 741 s, τpoisson = 679 s, n = 10 networks. E. Decay of strong E-to-E (Weff > 2.5, same as the average of blue points in D, up to 2,000 s) and weak E-to-E connections (0.5 < Weff ≤2.5, n = 10 networks). Dashed lines indicate exponential fits with baseline.

Motif analysis of strong connections.

A. Definitions of the four triplet motifs: linear chains, divergence motifs, convergence motifs, and fan-in/out motifs. B. Frequency of triplet motifs of strong connections over time, relative to the number of all possible sites to form a motif. The dashed line corresponds to the data points in C. C. Motif frequency in our model network (colored) compared to the estimated motif frequency in a random network with the same ratio of strong E-to-E connections (black) (n = 10 networks; mean ± std). ***: P < 0.001, paired sample t-test. D. Decay of the fan-in/out motif (colored) compared to a null model (gray) with shuffled connections. Lines indicate exponential fits (τfan = 907 s, τshuffle = 562 s). E. Percentage of motifs among followers relative to the percentage among all neurons in the network (n = 74 sequences; mean ± std). Pie charts: percentages among followers (linear chains: 69.95%; divergence: 13.42%; convergence: 8.05%; fan-in/out: 8.58%, summing up to 100%) and in the full network (linear chains: 65.94%; divergence: 16.08%; convergence: 13.28%; fan-in/out: 4.70%, summing up to 100%). *: P < 0.05, ***: P < 0.001, single sample t-test.

Turnover of followers and sequences.

A. Excitatory followers of an example source neuron at two time points. Blue circles indicate common followers at both time points (Remaining). Hollow circles indicate cells classifying as followers earlier but not later (Lost). Orange circles indicate those classifying as followers later but not earlier (New). B. Mean number of new and lost followers over 50 sequences in 100 s intervals. C. Decay of the probability that a follower continues to classify as a follower after a given interval (pooled over 2,931 followers from 50 sequences, mean ± std). Dashed line indicates exponential fit with baseline. D. Ratio of remaining followers after 1,000 s as a function of their responding probability (pooled over 24,311 followers from 50 sequences at 8 time points, mean ± std). E. Same as D but as a function of number of synaptic jumps from the source. F. Distributions of the responding probability for new, remaining, and lost followers in 100 s intervals. G. Distribution of the followers’ lifetime. Pooled from followers detected every 100 s from 50 different sequences. H. Mean responding probability for different groups of followers, grouped by by their lifetime. Pooled over 2,338 followers from 50 sequences.

Structured inputs lead to structured connectivity and stable turnover.

A. Schematic of training protocol with spatially structured inputs. Correlated inputs were sequentially presented to non-overlapping groups (which determined the assemblies) of excitatory neurons (color-coded) for 100 s (gray shading). Only one assembly received inputs at a time. Other excitatory and inhibitory neurons received uncorrelated inputs. The targeted group cycled throughout the simulation. B. Ratio of strong E-to-E connections where pre- and postsynaptic neurons are in the same assembly (within) or in different assemblies (between), (n = 10; mean ± std). C. Ratio of strong connections within and between assemblies at steady state (t = 25, 000 s in B). Most of the strong connections are formed within the correlated assemblies (diagonal). D. Top: Distribution of responding probability (left) and median delay (right) of followers in the same assembly with the source neuron (blue, n = 155) or in other assemblies (gray, n = 97). Bottom: Two example sequences triggered at two source neurons in different assemblies. Purple: source neurons. Blue: Followers within an assembly. Gray: Followers in other assemblies. E. Decay rate of strong E-E connections in the network with and without correlated inputs. Blue: E-to-E connections within assemblies. Gray: Decay in the network with uncorrelated inputs (same as Fig. 2D, blue). Pink: Same as blue but turnover of a connection was not counted if it was replaced by a new connection in the same assembly. Lines indicate exponential fits (τwithin = 5, 259 s, τuncorr = 1, 721 s, τcomp = 17, 761 s). F. Ratio of strong connections, in networks first trained with uncorrelated inputs and then correlated inputs (n = 10; mean ± std). G. Same as F but networks were first trained with one correlated input pattern and then switched to a different correlated input pattern. Top: within and between connections relative to first pattern. Middle: same relative to second pattern. Bottom: Example sequences from the same source at two time points. Orange and hollow circles are new and lost followers relative to the first time point.

Temporally structure inputs lead to sequentially organized followers.

A. Schematic of training protocol with temporally structured inputs. Correlated inputs were sequentially presented to non-overlapping groups (which determined the assemblies) of excitatory neurons (color-coded) for 10 ms (gray shading). Only one assembly received inputs at a time. Inhibitory neurons received uncorrelated inputs. The targeted group cycled throughout the simulation. B. Ratio of strong E-to-E connections within the same assembly (blue), from one assembly to the next assembly (orange), and all others (black) (n = 10; mean ± std). C. Ratio of strong connections within and between assemblies at steady state (t = 16, 000 s in B). D. Left: Average count of followers in different assemblies. Assembly number is relative to the source neuron, starting from 1. Note most followers fall in the immediate next assembly. Right: example sequence extending from assembly 1 to 5, with followers vertically sorted by the assembly number relative to the source neuron, as indicated by different colors and numbers. E. Decay rate of strong E-to-E connections. Blue: to a neuron in the same or next assembly. Pink: same as blue but turnover is counted only if the connection was not replaced by a new one. Black: Decay in the network with uncorrelated inputs (same as Fig. 2D). Lines indicate exponential fits (τee = 4, 738 s, τuncorr = 1, 721 s, τcomp = 28, 551 s). F. Distributions of responding probability (left) and median delay (right) of followers in the same assembly as the source neuron (blue), in the next assembly (orange), or in other assemblies (gray) (n = 116 sequences, nwithin = 57, nnext = 107, nother = 326). G. Ratio of strong connections within assemblies, from one assembly to the next, and anywhere else, in a network first trained with unstructured inputs and then with temporally structured inputs (n = 10; mean±std). H. Same as G but network was first trained with one pattern and then with a different one (n = 10; mean±std).

Parameters for the single neuron model

Parameters in the network model

Parameters for the plasticity rules

Parameters of the spatially structured inputs (Fig. 5)

Parameters of the temporally structured inputs (Fig. 6)

Hub neurons

Synaptic plasticity mechanisms in the model neural network.

follow a Hebbian pair-based eSTDP rule and I-to-E connections follow a pair-based iSTDP rule. Synaptic normalization preserves the total sum of incoming and outgoing weights (E-to-E and I-to-E). Intrinsic adjusts the firing threshold of excitatory neurons if the firing rate is higher or lower than the target firing rate. A. Schematic of the model network with 1,200 excitatory and 240 inhibitory AdEx neurons, where E-to-E and I-to-E connections are plastic. B. The STDP rules that changes the E-to-E (eSTDP) and I-to-E (iSTDP) synaptic weights. C. Synaptic normalization preserves the total sum of incoming and outgoing weights (E-to-E and I-to-E). D. Intrinsic plasticity adjusts the firing threshold of excitatory neurons if the firing rate is higher or lower than the target firing rate.

Statistics of sequence generation.

A. Sequence generation plotted against source neurons’ maximal outgoing effective weight. (Gray bars) Histogram of the maximal outgoing Weff of the 1,200 excitatory neurons in a model network. (Blue line) The ratio of source neurons that had at least one excitatory follower, given the maximal outgoing Weff, calculated within the n = 155 source neurons that we chose. (Red line) The ratio of source neurons that had at least one inhibitory follower, given the maximal outgoing Weff, calculated within the n = 155 source neurons that we chose. B. Distribution of the number of excitatory followers and inhibitory followers of the n = 155 source neurons that we tested. C. Distribution of the number of followers that spiked after the forced spike of the same source neuron. Pooled from 1,000 consecutive trials.

More examples of sequences generated in the same network as in Fig. 1.

Blue curves indicate median delays of excitatory followers. Red curves indicate median delays of inhibitory followers. Cyan and pink shades indicate jitters.

Sequence properties.

A. Normalized entropy of sequence in the model network (black) and the null model with shuffled sequences (red). Data pooled from all source neurons with at least one follower (n = 135). Each sequence was shuffled 10 times. B. Rank correlation as a function of spike rank in the model network (black) and the null model with shuffled sequences (red). Same data as D. C. Delay of excitatory followers after source spike. Solid line indicates the median delay and shading indicates the jitter (standard deviation across all trials). D. Same as C but for inhibitory followers.

Two dimensional density plot of followers’ responding probability, median delay, and jitter.

A. Density plot of responding probability and median delay of followers. Median delay is negatively correlated to responding probability. Pearson correlation r = −0.507. B. Density plot of responding probability and jitter of followers. Jitter is negatively correlated to responding probability. Pearson correlation r = −0.246. C. Density plot of median delay and jitter of followers. Jitter is positively correlated to median delay. (Blue) Excitatory followers; (Red) Inhibitory followers. Darker color indicates higher probability density. Pearson correlation r = 0.343. Pooled over 7,769 excitatory followers and 2,796 inhibitory followers from 135 sequences.

Feedback inhibition from inhibitory followers.

A. Source neuron triggers feedback inhibition in the model. Blue: source neuron membrane potential after a spike. Pooled over n = 15 neurons. Black: membrane potential of isolated neurons. Pooled over n = 1, 200 neurons. Solid lines indicate means and shadings the standard deviations. B. Quantification of feedback inhibition (A). The average hyperpolarized membrane potential is quantified as the mean of the membrane potential within 50 ms to 100 ms after the spike.

The turnover of I-to-E connections.

A. Average strength of multiple groups of I-to-E connections picked from the tail of the Weff distribution at different time points, after the weight distribution has reached a steady state. B. The decay rate of strong I-to-E weights (Weff > 2.5) and weak I-to-E connections (0.5 < Weff < 2.5). Dots indicate ratios calculated from simulations with 10 instantiations of the network. Dashed lines indicate exponential fits with different time constants and baselines for each group.

A model network without presynaptic normalization and intrinsic plasticity generates hub neurons and short sequences.

A. The comparison between the motif frequency at t = 6, 000 s in the alternative model network with hub neurons and the estimated ratio in a random control network given the p0 value at that time point. Calculated from 10 trials. Error bars represent standard deviation. ***: P < 0.001. B. Hub neurons. The synaptic strength is proportional to the firing rate of neurons. Neurons with a high firing rate have a much greater impact on the network dynamics. Left: presynaptic neurons in E-to-E connections. Right: Postsynaptic neurons in E-to-E connections. C. Short sequences as a result of hub neurons. Left: excitatory followers. Right: inhibitory followers. D. Explosive dynamics in which a great number of followers can be generated within a time window as short as 30 ms. Left: excitatory neurons. Right: inhibitory neurons.

The decay of three-neuron motifs in our model network (color-coded) and in the null model (gray) in which the connections were shuffled.

A. Linear chains. The decay of linear chains in our model network was comparable to the null model, consistent with Fig. 3D in which the ratio of linear chains was comparable to the null model. B. divergence motifs. The decay of divergence motifs in our model network was much faster than the null model, consistent with Fig. 3D in which the ratio of divergence motif was much lower than the null model. C. Convergence motifs. The decay of convergence motifs in our model network was much faster than the null model, consistent with Fig. 3D in which the ratio of convergence motifs was much lower than the null model.

A. The remaining ratio of followers after a 1,000 s interval decreases with the median delay of followers. Solid line indicates the average and shade represents the standard deviation. B. The strong synaptic jump is negatively correlated to the responding probability. Pearson’s correlation r = − 0.386, pooled over 2,508 followers at t = 8, 000 s in Fig. 4. C. The strong synaptic jump is positively correlated to the median delay. Pearson’s correlation r = 0.494, same followers as B.

E-to-E distribution reached a steady state after training with structured inputs.

A. The network develops a stable distribution which can be fit as lognormal when trained with spatially structured inputs. Left: t = 10, 000 s, right: t = 25, 000 s, orange curve shows the fit, R2 = 0.809 for the left and R2 = 0.821 for the right. B. The network develops a stable distribution which can be fit as lognormal when trained with temporally structured inputs. Left: t = 8, 000 s, right: t = 16, 000 s, orange curve shows the fit, R2 = 0.669 for the left and R2 = 0.669 for the right.

Alternative plasticity models can also support sequence-generating networks.

A. Steady-state distribution of E-to-E synaptic weight under different combinations of plasticity rules (counterpart of Fig. 1C). Inset: Zoomed-in view of the distribution from 1 to 10 nS/mV. B. Ratio of strong E-to-E connections (Weff > 2.5) over time (counterpart of Fig. 2B solid line). C. Histogram of the maximal outgoing Weff across the 1,200 E neurons in the steady-state model network (counterpart of Fig. S2A, gray shade). The black dashed line indicates the minimal Weff required to initiate a sequence (Weff = 2.5). D. Traces of population firing rate of E neurons over time, smoothed using a third-order Savitzky-Golay filter with a 50-second time window. The y-axis is clipped at 3 Hz for visualization. E. Distribution of firing thresholds of E neurons in the steady-state network when intrinsic plasticity is active. Black arrow on the left indicates the initial firing threshold, which was identical for all excitatory neurons.