Figures and data in Evolving interpretable plasticity for spiking networks

Figures
Tables
Additional files

12 figures, 3 tables and 1 additional file

Figures

Figure 1

Download asset Open asset

Artificial evolution of synaptic plasticity rules in spiking neuronal networks.

(A) Sketch of cortical microcircuits consisting of pyramidal cells (orange) and inhibitory interneurons (blue). Stimulation elicits action potentials in pre- and postsynaptic cells, which, in turn, influence synaptic plasticity. (B) Synaptic plasticity leads to a weight change ( $Δ w$ ) between the two cells, here measured by the change in the amplitude of post-synaptic potentials. The change in synaptic weight can be expressed by a function $f$ that in addition to spike timings ( $t_{pre}, t_{post}$ ) can take into account additional local quantities, such as the concentration of neuromodulators (ρ, green dots in A) or postsynaptic membrane potentials. (C) For a specific experimental setup, an evolutionary algorithm searches for individuals representing functions $f$ that maximize the corresponding fitness function $ℱ$ . An offspring is generated by modifying the genome of a parent individual. Several runs of the evolutionary algorithm can discover phenomenologically different solutions ( $f_{0}, f_{1}, f_{2}$ ) with comparable fitness. (D) An offspring is generated from a single parent via mutation. Mutations of the genome can, for example, exchange mathematical operators, resulting in a different function $f$ .

Figure 2

Download asset Open asset

Representation and mutation of mathematical expressions in Cartesian genetic programming.

(A) The genotype of an individual is a two-dimensional Cartesian graph (top). In this example, the graph contains three input nodes ( $0 - 2$ ), six internal nodes ( $3 - 8$ ) and a single output node (9). In each node, the genes of a specific genotype are shown, encoding the operator used to compute the node’s output and its inputs. Each operator gene maps to a specific mathematical function (bottom). Special values ( $- 1, - 2$ ) represent input and output nodes. For example, node four uses the operator 1, the multiplication operation '*', and receives input from nodes 0 and 2. This node’s output is hence given by $x_{0} * x_{2}$ . The number of input genes per node is determined by the operator with the maximal arity (here two). Fixed genes that cannot be mutated are highlighted in red. ∅ denotes non-coding genes. (B) The computational graph (phenotype) generated by the genotype in A. Input nodes ( $x_{0}, x_{1}, x_{2}$ ) represent the arguments of the function $f$ . Each output node selects one of the other nodes as a return value of the computational graph, thus defining a function from input $𝒙$ to output $𝒚 = 𝒇 (𝒙)$ . Here, the output node selects node four as a return value. Some nodes defined in the genotype are not used by a particular realization of the computational graph (in light gray, e.g., node 6). Mutations that affect such nodes have no effect on the phenotype and are therefore considered ‘silent’. (C) Mutations in the genome either lead to a change in graph connectivity (top, green arrow) or alter the operators used by an internal node (bottom, green node). Here, both mutations affect the phenotype and are hence not silent.

Figure 3

Download asset Open asset

Cartesian genetic programming evolves various efficient reward-driven learning rules.

(A) Network sketch. Multiple input neurons with Poisson activity project to a single output unit. Pre- and postsynaptic activity generate an eligibility trace in each synapse. Comparison between the output activity and the target activity generates a reward signal. $\bar{R}$ , and ${\bar{R}}^{+}$ , ${\bar{R}}^{-}$ represent the expected reward, the expected positive and the expected negative reward, respectively. Depending on the hyperparameter settings either the former or the latter two are provided to the plasticity rule. (B) Raster plot of the activity of input neurons (small black dots) and output neuron (large golden dots). Gray (white) background indicate patterns for which the output should be active (inactive). Top indicates correct classifications (+) and incorrect classifications (-). We show 10 trials at the beginning (left) and the end of training (right) using the evolved plasticity rule: $Δ w_{j} = η (R - 1) E_{j}^{r}$ . (C) Fitness of best individual per generation as a function of the generation index for multiple example runs of the evolutionary algorithm with different initial conditions but identical hyperparameters. Labels show the expression $f$ at the end of the respective run for three runs resulting in well-performing plasticity rules. Gray lines represent runs with functionally identical solutions or low final fitness. (D) Fitness of a selected subset of evolved learning rules on the 10 experiments used during the evolutionary search (blue) and additional 80 fitness evaluations, each on 10 new experiments consisting of sets of frozen noise patterns and associated class labels not used during the evolutionary search (orange). Horizontal boxes represent mean, error bars indicate one standard deviation over fitness values. Gray line indicates mean fitness of LR0 for visual reference. Black stars indicate significance ( $p < 10^{- 16}$ ) with respect to LR0 according to Welch’s T-tests (Welch, 1947). See main text for the full expressions for all learning rules.

Figure 4

Download asset Open asset

Cartesian genetic programming evolves efficient error-driven learning rules.

(A) Network sketch. Multiple input neurons with Poisson activity project to two neurons. One of the neurons (the teacher) generates a target for the other (the student). The membrane potentials of teacher and student as well as the filtered pre-synaptic spike trains are provided to the plasticity rule that determines the weight update. (B) Root mean squared error between the teacher and student membrane potential over the course of learning using the evolved plasticity rule: $Δ w_{j} (t) = η [v (t) - u (t)] {\bar{s}}_{j} (t)$ . (C) Synaptic weights over the course of learning corresponding to panel B. Horizontal dashed lines represent target weights, that is, the fixed synaptic weights onto the teacher. (D) Fitness of the best individual per generation as a function of the generation index for multiple runs of the evolutionary algorithm with different initial conditions. Labels represent the rule at the end of the respective run. Colored markers represent fitness of each plasticity rule averaged over 15 validation tasks not used during the evolutionary search; error bars indicate one standard deviation.

Figure 5

Download asset Open asset

Cartesian genetic programming evolves diverse correlation-driven learning rules.

(A) Network sketch. Multiple inputs project to a single output neuron. The current synaptic weight w_j and the eligibility trace $E_{j}^{c}$ are provided to the plasticity rule that determines the weight update. (B) Membrane potential $u$ of the output neuron over the course of learning using Equation 17. Gray boxes indicate presentation of the frozen-noise pattern. (C) Fitness (Equation 13) of the best individual per generation as a function of the generation index for multiple runs of the evolutionary algorithm with different initial conditions. Blue and red curves correspond to the two representative plasticity rules selected for detailed analysis. Blue and red markers represent fitness of the two representative rules and the orange marker the fitness of the homeostatic STDP rule (Equation 17; Masquelier, 2018), respectively, on 20 validation tasks not used during the evolutionary search. Error bars indicate one standard deviation over tasks. (**D, E**): Learning rules evolved by two runs of CGP (D: LR1, Equation 19; E: LR2, Equation 20). (F): Homeostatic STDP rule Equation 17 suggested by Masquelier, 2018. Top panels: STDP kernels $Δ w_{j}$ as a function of spike timing differences $Δ t_{j}$ for three different weights w_j. Bottom panels: homeostatic mechanisms for those weights. The colors are specific to the respective learning rules (blue for LR1, red for LR2), with different shades representing the different weights w_j. The learning rate is $η = 0.01$ .

Appendix 1—figure 1

Download asset Open asset

Fitness of best individual per generation as a function of the generation index for multiple runs of the evolutionary algorithm with different initial conditions for hyperparameter set 0.

Appendix 1—figure 2

Download asset Open asset

Appendix 1—figure 3

Download asset Open asset

Appendix 1—figure 4

Download asset Open asset

Appendix 1—figure 5

Download asset Open asset

Causal and homeostatic terms of LR-LR6 over trials.

$c^{+}, c^{-}$ represent causal terms (prefactors of eligibility trace), $h^{+}, h^{-}$ represent homeostatic terms, for positive and negative rewards, respectively.

Appendix 1—figure 6

Download asset Open asset

Cumulative reward of LR-LR5 over trials.

Solid line represent mean, shaded regions indicate plus/minus one standard deviation over 80 experiments. Cumulative reward of LR0 shown in all panels for comparison. Gray line indicates maximal performance (maximal reward received in each trial).

Appendix 1—figure 7

Download asset Open asset

Evolution of membrane potential for two evolved learning rules.

Membrane potential $u$ of the output neuron over the course of learning using the two evolved learning rules LR1 (top row, Equation 19) and LR2 (bottom row, Equation 20) (compare Figure 5B). Gray boxes indicate presentation of the frozen-noise pattern.

Tables

Appendix 1—table 1

Description of the network model used in the reward-driven learning task (4.5).

A model summary
Populations		2
Topology		—
Connectivity		Feedforward with fixed connection probability
Neuron model		Leaky integrate-and-fire (LIF) with exponential post-synaptic currents
Plasticity		Reward-driven
Measurements		Spikes
B populations
Name	Elements	Size
Input	Spike generators with pre-defined spike trains (see 4.5)	$N$
Output	LIF neuron	1
C connectivity
Source	Target	Pattern
Input	Output	Fixed pairwise connection probability $p$ ; synaptic delay $d$ ; random initial weights from $𝒩 (0, σ_{w}^{2})$
D neuron model
Type		LIF neuron with exponential post-synaptic currents
Subthreshold dynamics		$\frac{d u (t)}{d t} = - \frac{u (t) - E_{L}}{τ_{m}} + \frac{I_{s} (t)}{C_{m}}$ if not refractory
		$u (t) = u_{r}$ else $I_{s} (t) = \sum_{i, k} w_{k} e^{- (t - t_{i}^{k}) / τ_{s}} Θ (t - t_{i}^{k})$ , $k$ : neuron index, $i$ : spike index
Spiking		Stochastic spike generation via inhomogeneous Poisson process with intensity $ϕ (u) = ρ e^{(u - u_{th}) / Δ u}$ ; reset of $u$ to $u_{r}$ after spike emission and refractory period of $τ_{r}$
E synapse model
Plasticity		Reward-driven with episodic update (Equation 2, Equation 3)
Other		Each synapse stores an eligibility trace (Equation 22)
F simulation parameters
Populations		$N = 50$
Connectivity		$p = 0.8, σ_{w} = 10^{3} pA$
Neuron model		$\begin{array}{ll} ρ = 0.01 Hz, Δ u = 0.2 mV, E_{L} = - 70 mV, u_{r} = - 70 mV, u_{th} = - 55 \\ mV, τ_{m} = 10 m s, C_{m} = 250 pF, τ_{r} = 2 m s, τ_{s} = 2 m s \end{array}$
Synapse model		$η = 10, τ_{M} = 500 ms, d = 1 ms$
Input		$M = 30, r = 6 Hz, T = 500 ms, n_{training} = 500, n_{exp} = 10$
Other		$h = 0.01 ms, R \in {- 1, 1}, m_{r} = 100$
G CGP parameters
Population		$μ = 1, p_{mutation} = 0.035$
Genome		$n_{inputs} = {3, 4}, n_{outputs} = 1, n_{rows} = 1, n_{columns} = {12, 24}, l_{max} = {12, 24}$
Primitives		Add, Sub, Mul, Div, Const(1.0), Const(0.5)
EA		$λ = 4, n_{breeding} = 4, n_{tournament} = 1, reorder = {true, false}$
Other		$m a x g e n e r a t i o n s = 1000, m i n i m a l f i t n e s s = 500$

Appendix 1—table 2

Description of the network model used in the error-driven learning task (4.6).

A model summary
Populations		3
Topology		—
Connectivity		Feedforward with all-to-all connections
Neuron model		Leaky integrate-and-fire (LIF) with exponential post-synaptic currents
Plasticity		Error-driven
Measurements		Spikes, membrane potentials
B populations
Name	Elements	Size
Input	Spike generators with pre-defined spike trains (see 4.6)	$N$
Teacher	LIF neuron	1
Student	LIF neuron	1
C connectivity
Source	Target	Pattern
Input	Teacher	All-to-all; synaptic delay $d$ ; random weights $w \sim 𝒰 [w_{min}, w_{max}]$ ; weights randomly shifted by $w_{shift}$ on each trial
Input	Student	All-to-all; synaptic delay $d$ ; fixed initial weights w₀
D neuron model
Type		LIF neuron with exponential post-synaptic currents
Subthreshold dynamics		$\frac{d u (t)}{d t} = - \frac{u (t) - E_{L}}{τ_{m}} + \frac{I_{s} (t)}{C_{m}}$ $I_{s} (t) = \sum_{i, k} J_{k} e^{- (t - t_{i}^{k}) / τ_{s}} Θ (t - t_{i}^{k})$ $k$ : neuron index, $i$ : spike index
Spiking		Stochastic spike generation via inhomogeneous Poisson process with intensity $ϕ (u) = ρ e^{(u - u_{th}) / Δ u}$ ; no reset after spike emission
E synapse model
Plasticity		Error-driven with continuous update (Equation 7, Equation 9)
F simulation parameters
Populations		$N = 5$
Connectivity		$w_{min} = - 20, w_{max} = 20, w_{shift} \sim {- 15, 15}, w_{0} = 5$
Neuron model		$ρ = 0.2 Hz, Δ u = 1.0 mV, E_{L} = - 70 mV, u_{th} = - 55 mV, τ_{m} = 10 ms, C_{m} = 250 pF, τ_{s} = 2 ms$
Synapse model		$η = 1.7, d = 1 ms, τ_{I} = 100.0 ms$
Input		$r_{min} = 150 Hz, r_{max} = 850 Hz, T = 10, 000 ms, n_{exp} = 15$
Other		$h = 0.01 ms, δ t = 5 ms$
G CGP parameters
Population		$μ = 4, p_{mutation} = 0.045$
Genome		$n_{inputs} = 3, n_{outputs} = 1, n_{rows} = 1, n_{columns} = 12, l_{max} = 12$
Primitives		Add, Sub, Mul, Div, Const(1.0)
EA		$λ = 4, n_{breeding} = 4, n_{tournament} = 1$
Other		$m a x g e n e r a t i o n s = 1000, m i n i m a l f i t n e s s = 0.0$

Appendix 1—table 3

: Description of the network model used in the correlation-driven learning task (4.7).

A model summary
Populations		2
Topology		—
Connectivity		Feedforward with fixed connection probability
Neuron model		Leaky integrate-and-fire (LIF) with exponential post-synaptic currents
Plasticity		Reward-driven
Measurements		Spikes
B populations
Name	Elements	Size
Input	Spike generators with pre-defined spike trains (see 4.5)	$N$
Output	LIF neuron	1
C connectivity
Source	Target	Pattern
Input	Output	Fixed pairwise connection probability $p$ ; synaptic delay $d$ ; random initial weights from $𝒩 (0, σ_{w}^{2})$
D neuron model
Type		LIF neuron with exponential post-synaptic currents
Subthreshold dynamics		$\frac{d u (t)}{d t} = - \frac{u (t) - E_{L}}{τ_{m}} + \frac{I_{s} (t)}{C_{m}}$ if not refractory
$u (t) = u_{r}$ else $I_{s} (t) = \sum_{i, k} w_{k} e^{- (t - t_{i}^{k}) / τ_{s}} Θ (t - t_{i}^{k})$ , $k$ : neuron index, $i$ : spike index
Spiking		Stochastic spike generation via inhomogeneous Poisson process with intensity $ϕ (u) = ρ e^{(u - u_{th}) / Δ u}$ ; reset of $u$ to $u_{r}$ after spike emission and refractory period of $τ_{r}$
E synapse model
Plasticity		Reward-driven with episodic update (Equation 2, Equation 3)
Other		Each synapse stores an eligibility trace (Equation 22)
F simulation parameters
Populations		$N = 50$
Connectivity		$p = 0.8, σ_{w} = 10^{3} pA$
Neuron model		$\begin{array}{ll} ρ = 0.01 Hz, Δ u = 0.2 mV, E_{L} = - 70 mV, u_{r} = - 70 mV, u_{th} = - 55 mV, \\ τ_{m} = 10 m s, C_{m} = 250 pF, τ_{r} = 2 m s, τ_{s} = 2 m s \end{array}$
Synapse model		$η = 10, τ_{M} = 500 m s, d = 1 m s$
Input		$M = 30, r = 6 Hz, T = 500 ms, n_{training} = 500, n_{exp} = 10$
Other		$h = 0.01 ms, R \in {- 1, 1}, m_{r} = 100$
G CGP parameters
Population		$μ = 8, p_{mutation} = 0.05$
Genome		$n_{inputs} = 2, n_{outputs} = 1, n_{rows} = 1, n_{columns} = 5, l_{max} = 5$
Primitives		Add, Sub, Mul, Div, Pow, Const(1.0)
EA		$λ = 8, n_{breeding} = 8, n_{tournament} = 1$
Other		$m a x g e n e r a t i o n s = 2000, m i n i m a l f i t n e s s = 10.0$

Additional files

Transparent reporting form: https://cdn.elifesciences.org/articles/66273/elife-66273-transrepform-v1.docx
Download elife-66273-transrepform-v1.docx

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Article PDF

Open citations (links to open the citations from this article in various online reference manager services)

Mendeley

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

Jakob Jordan
Maximilian Schmidt
Walter Senn
Mihai A Petrovici

(2021)

Evolving interpretable plasticity for spiking networks

eLife 10:e66273.

https://doi.org/10.7554/eLife.66273

Figures

Artificial evolution of synaptic plasticity rules in spiking neuronal networks.

Representation and mutation of mathematical expressions in Cartesian genetic programming.

Cartesian genetic programming evolves various efficient reward-driven learning rules.

Cartesian genetic programming evolves efficient error-driven learning rules.

Cartesian genetic programming evolves diverse correlation-driven learning rules.

Fitness of best individual per generation as a function of the generation index for multiple runs of the evolutionary algorithm with different initial conditions for hyperparameter set 0.

Fitness of best individual per generation as a function of the generation index for multiple runs of the evolutionary algorithm with different initial conditions for hyperparameter set 1.

Fitness of best individual per generation as a function of the generation index for multiple runs of the evolutionary algorithm with different initial conditions for hyperparameter set 2.

Fitness of best individual per generation as a function of the generation index for multiple runs of the evolutionary algorithm with different initial conditions for hyperparameter set 3.

Causal and homeostatic terms of LR-LR6 over trials.

Cumulative reward of LR-LR5 over trials.

Evolution of membrane potential for two evolved learning rules.

Tables

Description of the network model used in the reward-driven learning task (4.5).

Description of the network model used in the error-driven learning task (4.6).

: Description of the network model used in the correlation-driven learning task (4.7).

Additional files

Transparent reporting form

Download links

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

Be the first to read new articles from eLife

Share this article

Cite this article

Artificial evolution of synaptic plasticity rules in spiking neuronal networks.

Representation and mutation of mathematical expressions in Cartesian genetic programming.

Cartesian genetic programming evolves various efficient reward-driven learning rules.

Cartesian genetic programming evolves efficient error-driven learning rules.

Cartesian genetic programming evolves diverse correlation-driven learning rules.

Fitness of best individual per generation as a function of the generation index for multiple runs of the evolutionary algorithm with different initial conditions for hyperparameter set 0.

Fitness of best individual per generation as a function of the generation index for multiple runs of the evolutionary algorithm with different initial conditions for hyperparameter set 1.

Fitness of best individual per generation as a function of the generation index for multiple runs of the evolutionary algorithm with different initial conditions for hyperparameter set 2.

Fitness of best individual per generation as a function of the generation index for multiple runs of the evolutionary algorithm with different initial conditions for hyperparameter set 3.

Causal and homeostatic terms of LR-LR6 over trials.

Cumulative reward of LR-LR5 over trials.

Evolution of membrane potential for two evolved learning rules.

Description of the network model used in the reward-driven learning task (4.5).

Description of the network model used in the error-driven learning task (4.6).

: Description of the network model used in the correlation-driven learning task (4.7).

Transparent reporting form

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)