Learning recurrent dynamics in spiking networks

  1. Christopher M Kim  Is a corresponding author
  2. Carson C Chow  Is a corresponding author
  1. National Institutes of Health, United States
7 figures and 1 additional file

Figures

Figure 1 with 2 supplements
Synaptic drive and spiking rate of neurons in a recurrent network can learn complex patterns.

(a) Schematic of network training. Blue square represents the external stimulus that elicits the desired response. Black curves represent target output for each neuron. Red arrows represent …

https://doi.org/10.7554/eLife.37124.002
Figure 1—figure supplement 1
Learning arbitrarily complex target patterns in a network of rate-based neurons.

The network dynamics obey τx˙i=xi+j=1NWijrj+Ii where rj=tanh(xj). The synaptic current xi to every neuron in the network was trained to follow complex periodic functions f(t)=Asin(2π(tT0)/T1)sin(2π(tT0)/T2) where the initial phase T0 and frequencies T1,T2 were …

https://doi.org/10.7554/eLife.37124.003
Figure 1—figure supplement 2
Training a network that has no initial connections.

The coupling strength of the initial recurrent connectivity is zero, and, prior to training, no synaptic or spiking activity appears beyond the first few hundred milliseconds. (a) Training synaptic …

https://doi.org/10.7554/eLife.37124.004
Learning multiple target patterns.

(a) The synaptic drive of neurons learns two different target outputs. Blue stimulus evokes the first set of target outputs (red) and the green stimulus evokes the second set of target outputs …

https://doi.org/10.7554/eLife.37124.005
Figure 3 with 3 supplements
Quasi-static and heterogeneous patterns can be learned.

Example target patterns include complex periodic functions (product of sines with random frequencies), chaotic rate units (obtained from a random network of rate units), and OU noise (obtained by …

https://doi.org/10.7554/eLife.37124.006
Figure 3—figure supplement 1
Learning target patterns with low-population spiking rate.

The synaptic drive of networks consisting of 500 neurons were trained to learn complex periodic functions f(t)=Asin(2π(tT0)/T1)sin(2π(tT0)/T2) where the initial phase T0 and frequencies T1,T2 were selected randomly from [500 ms, 1000 …

https://doi.org/10.7554/eLife.37124.007
Figure 3—figure supplement 2
Learning recurrent dynamics with leaky integrate-and-fire and Izhikevich neuron models.

Synaptic drive of a network of spiking neurons were trained to follow 1000 ms long targets f(t)=Asin(2π(tT0)/T1)sin(2π(tT0)/T2) where T0,T1 and T2 were selected uniformly from the interval [500 ms, 1000 ms]. (a) Network consisted of N=200

https://doi.org/10.7554/eLife.37124.008
Figure 3—figure supplement 3
Synaptic drive of a network of neurons is trained to learn an identical sine wave while external noise generated independently from OU process is injected to individual neurons.

The same external noise (gray curves) is applied repeatedly during and after training. (a)-(b) The amplitude of external noise is varied from (a) low, (b) medium to (c) high. The target sine wave is …

https://doi.org/10.7554/eLife.37124.009
Learning innate activity in a network of excitatory and inhibitory neurons that respects Dale’s Law.

(a) Synaptic drive of sample neurons starting at random initial conditions in response to external stimulus prior to training. (b) Spike raster of sample neurons evoked by the same stimulus over …

https://doi.org/10.7554/eLife.37124.010
Generating in vivo spiking activity in a subnetwork of a recurrent network.

(a) Network schematic showing cortical (black) and auxiliary (white) neuron models trained to follow the spiking rate patterns of cortical neurons and target patterns derived from OU noise, …

https://doi.org/10.7554/eLife.37124.011
Sampling and tracking errors.

Synaptic drive was trained to learn 1 s long trajectories generated from OU noise with decay time τc. (a) Performance of networks of size N=500 as a function of synaptic decay time τs and target decay …

https://doi.org/10.7554/eLife.37124.012
Capacity as a function of network size.

(a) Performance of trained networks as a function of target length T for networks of size N=500 and 1000. Target patterns were generated from OU noise with decay time τc=100 ms. (b) Networks of fixed sizes …

https://doi.org/10.7554/eLife.37124.013

Additional files

Download links