Structural and dynamical properties of the efficient E-I spiking network.

(A) (i) Encoding of a target signal representing the evolution of a stimulus feature (top) with one E (middle) and one I spiking neuron (bottom). The target signal x(t) integrates the input signal s(t). The readout of the E neuron tracks the target signal and the readout of the I neuron tracks the readout of the E neuron. Neurons spike to bring the readout of their activity closer to their respective target. Each spike causes a jump of the readout, with the sign and the amplitude of the jump being determined by neuron’s tuning parameters. (ii) Schematic of the matrix of tuning parameters. Every neuron is selective to all stimulus features (columns of the matrix), and all neurons participate in encoding of every feature (rows).

(B) (i) Schematic of the network with E (red) and I (blue) cell type. E neurons are driven by the stimulus features while I neurons are driven by the activity of E neurons. E and I neurons are connected through recurrent connectivity matrices. (ii) Schematic of E (red) and I (blue) synaptic interactions. Arrows represent the direction of the tuning vector of each neuron. Only neurons with similar tuning are connected and the connection strength is proportional to the tuning similarity.

(C) (i) Schematic of similarity of tuning vectors (tuning similarity) in a 2-dimensional space of stimulus features. (ii) Synaptic strength as a function of tuning similarity.

(D) Coding and dynamics in a simulation trial. Top three rows show the signal (black), the E estimate (red) and the I estimate (blue) for each of the three stimulus features. Below are the spike trains. In the bottom row, we show the average instantaneous firing rate (in Hz).

(E) Top: Example of the target signal (black) and the E estimate in three simulation trials (colors) for one stimulus feature. Bottom: Distribution (across time) of the time-dependent bias of estimates in E and I cell type.

(F) Left: Distribution of time-averaged firing rates in E (top) and I neurons (bottom). Black traces are fits with log-normal distribution. Right: Distribution of coefficients of variation of interspike intervals for E and I neurons.

(G) Distribution (across neurons) of time-averaged synaptic inputs to E (left) and I neurons (right). In E neurons, the mean of distributions of inhibitory and of net synaptic inputs are very close.

(H) Sum of synaptic inputs over time in a single E (top) and I neuron (bottom) in a simulation trial.

(I) Distribution (across neurons) of Pearson’s correlation coefficients measuring the correlation of synaptic inputs AEy and AIy (as defined in Methods, Eq. (43)) in single E (red) and I (blue) neurons. All statistical results (E-F, H-I) were computed on 10 simulation trials of 10 second duration. For model parameters, see Table 1.

Table of default model parameters for the efficient E-I network

Parameters above the double horizontal line are the minimal set of parameters needed to simulate model equations (Eqs. 29a-29h in Methods). Parameters below the double horizontal line are biophysical parameters, derived from the same model equations and from model parameters listed above the horizontal line. Parameters N E, M, τ and were chosen for their biological plausibility and computational simplicity. Parameters N I, , σ, ratio of mean E-I to I-I synaptic connectivity and β are parameters that maximize network efficiency (see the section “Criterion for determining model parameters” in Methods). The metabolic constant β and the noise strength σ are interpreted as global network parameters and are for this reason assumed to be the same across the E and I population, e.g., βE = βI = β and σE = σI = σ (see Eq. 3). The connection probability of pxy = 0.5 is the consequence of rectification of the connectivity (see Eq. 24 in Methods).

Table of parameter ranges for Monte-Carlo sampling.

Minimum and maximum of the uniform distributions from which we randomly drew parameters during Monte-Carlo random sampling.

Table of best four parameter settings from Monte-Carlo sampling.

Best four parameter settings out of 10000 tested settings. The performance was evaluated using trial- and time-averaged loss. Each parameter setting was evaluated on 20 trials, with each trial using an independent realization of tuning parameters, noise in the non-specific current and initial conditions for the integration of the membrane potentials.

Monte-Carlo joint random sampling on six model parameters.

(A) Distribution of the trial-averaged loss, with weighting gL=0.7, from 10000 random simulations and using 20 simulation trials of duration of 1 second for each parameter configuration. The red cross marks the average loss of the parameter setting in Table 1. Inset: The average loss of the parameter setting in Table 1 (red cross) and of the first- and second-best parameter settings from the random search.

(B) Distribution of the average loss across 20 simulation trials for the parameter setting in Table 1 (red) and for the first four ranked points according to the trial-averaged loss in A. Stars indicate a significant two-tailed t-test against the distribution in red (*** indicate p < 0.001).

(C) Same as in A, for different values of weighting of the error with the cost gL. Parameters for all plots are in Table 1

Mechanism of lateral excitation/inhibition in the efficient spiking network.

(A) Left: Schematic of the E-I network and of the stimulation and measurement in a perturbation experiment. Right: Schematic of the propagation of the neural activity between E and I neurons with similar tuning.

(B) Trial and neuron-averaged deviation of the firing rate from the baseline, for the population of I (top) and E (bottom) neurons with similar (magenta) and different tuning (gray) to the target neuron. The stimulation strength corresponded to an increase in the firing rate of the stimulated neuron by 28.0 Hz.

(C) Scatter plot of the tuning similarity vs. effective connectivity to the target neuron. Red line marks zero effective connectivity and magenta line is the least-squares line. Stimulation strength was ap = 1.

(D) Correlation of membrane potentials vs. the tuning similarity in E (top) and I cell type (bottom), for the efficient E-I network (left), for the network where each E neuron receives independent instead of shared stimulus features (middle), and for the network with unstructured connectivity (right). In the model with unstructured connectivity, elements of each connectivity matrix were randomly shuffled. We quantified voltage correlation using the (zero-lag) Pearson’s correlation coefficient, denoted as , for each pair of neurons.

(E) Average cross-correlogram (CCG) of spike timing with strongly similar (orange), weakly similar (green) and different tuning (black). Statistical results (B-E) were computed on 100 simulation trials. The duration of the trial in D-E was 1 second. Parameters for all plots are in Table 1.

Effects of connectivity structure on coding efficiency, neural dynamics and lateral inhibition.

(A) Left: Root mean squared error (RMSE) in networks with structured and randomly shuffled recurrent connectivity. Random shuffling consisted of a random permutation of the elements within each of the three (E-I, I-I, I-E) connectivity matrices. Right: Distribution of decoding weights after training the decoder on neural activity from the structured network (green), and a sample from uniform distribution as typically used in the optimal network. (B) Metabolic cost in structured and shuffled networks with matched average balance. The average balance of the shuffled network was matched with the one of the structured network by changing the following parameters: and by decreasing the amplitude of the OU stimulus by factor of 0.88. (C) Standard deviation of the membrane potential (in mV) for networks with structured and unstructured connectivity. Distributions are across neurons. (D) Average firing rate of E (top) and I neurons (bottom) in networks with structured and unstructured connectivity. (E) Same as in D, showing the coefficient of variation of spiking activity in a network responding to a constant stimulus. (F) Same as in D, showing the average net synaptic input, a measure of average imbalance. (G) Same as in D, showing the time-dependent correlation of synaptic inputs, a measure of instantaneous balance. (H) Voltage correlation in E-E (top) and I-I neuronal pairs (bottom) for the four cases of unstructured connectivity (colored dots) and the equivalent result in the structured network (grey dots). We show the results for pairs with similar tuning. (I) Scatter plot of effective connectivity versus tuning similarity to the photostimulated E neuron in shuffled networks. The title of each plot indicates the connectivity matrix that has been shuffled. The magenta line is the least-squares regression line and the photostimulation is at threshold (ap = 1.0). Results were computed using 200 (A-G) and 100 (H-I) simulation trials of 1 second duration. Parameters for all plots are in Table 1.

Relation of time constants of single-neuron and population readout set an adaptation or a facilitation current.

The population readout that evolves on a faster (slower) time scale than the single neuron readout determines a spike-triggered adaptation (facilitation) in its own cell type.

Adaptation, network coding efficiency and excitation-inhibition balance.

(A) The encoding error (left), metabolic cost (middle) and average loss (right) as a function of single neuron time constants and , in units of ms. These parameters set the sign, the strength, as well as the time constant of the feedback current in E and I neurons. Best performance (lowest average loss) is obtained in the top right quadrant, where the feedback current is spike-triggered adaptation in both E and I neurons. The performance measures are computed as a weighted sum of the respective measures across the E and I populations with equal weighting for E and I. All measures are plotted on the scale of the natural logarithm for better visibility. (B) Top: Log-log plot of the RMSE of the E (red) and the I (blue) estimates as a function of the time constant of the single neuron readout of E neurons, , in the regime with spike-triggered adaptation. Feedback current in I neurons is set to 0. Bottom: Same as on top, as a function of while the feedback current in E neurons is set to 0. (C) Same as in B, showing the average loss. (D) Same as in B, showing the firing rate. (E) Firing rate in E (left) and I neurons (right), as a function of time constants and . (F) Same as in E, showing the coefficient of variation. (G) Same as E, showing the average net synaptic input, a measure of average imbalance. (H) Same as E, showing the average net synaptic input, a measure of instantaneous balance. All statistical results were computed on 100 simulation trials of 1 second duration. For other parameters, see Table 1.

State-dependent coding and dynamics are controlled by non-specific currents.

(A) Spike trains of the efficient E-I network in one simulation trial, with different values of the metabolic constant β. The network received identical stimulus across trials. (B) Top: RMSE of E (red) and I (blue) estimates as a function of the metabolic constant. Bottom: Normalized average metabolic cost and average loss as a function of the metabolic constant. Black arrow indicates the minimum loss and therefore the optimal metabolic constant. (C) Average firing rate (top) and the coefficient of variation of the spiking activity (bottom), as a function of the metabolic constant. Black arrow marks the metabolic constant leading to optimal network efficiency in B.(D) Average imbalance (top) and instantaneous balance (bottom) balance as a function of the metabolic constant. (E) Same as in A, for different values of the noise strength σ. (F) Same as in B, as a function of the noise strength. The noise is a Gaussian random process, independent over time and across neurons. (G) Same as C, as a function of the noise strength. (H) Top: Same as in D, as a function of the noise strength. (I) The encoding error measured as RMSE (left), the metabolic cost (middle) and the average loss (right) as a function of the metabolic constant β and the noise strength σ. Metabolic constant and noise strength that are optimal for the single parameter search (in B and F) are marked with a red cross in the figure on the right. For plots in B-D and F-I, we computed and averaged results over 100 simulation trials with 1 second duration. For other parameters, see Table 1.

Optimal ratios of E-I neuron numbers and of mean I-I to E-I efficacy.

(A) Schematic of the effect of changing the number of I neurons on firing rates of I neurons. As encoding of the stimulus is distributed among more I neurons, the number of spikes per I neuron decreases.

(B) Average firing rate as a function of the ratio of the number of E to I neurons. Black arrow marks the optimal ratio.

(C) Average net synaptic input in E neurons (top) and in I neurons (bottom).

(D) Top: Encoding error (RMSE) of the E (red) and I (blue) estimates, as a function of the ratio of E-I neuron numbers. Bottom: Same as on top, showing the cost and the average loss. Black arrow shows the minimum of the loss, indicating the optimal parameter.

(E) Top: Optimal ratio of the number of E to I neurons as a function of the weighting of the average loss of E and I cell type (using the weighting of the error and cost of 0.7 and 0.3, respectively). Bottom: Same as on top, measured as a function of the weighting of the error and the cost when computing the loss. (The weighting of the losses of E and I neurons is 0.5.) Black triangles mark weightings that we typically used.

(F) Schematic of the readout of the spiking activity of E (red) and I population (blue) with equal amplitude of decoding weights (left) and with stronger decoding weight in I neuron (right). Stronger decoding weight in I neurons results in a stronger effect of spikes on the readout, leading to less spikes by the I population.

(G-H) Same as in D and B, as a function of the ratio of mean I-I to E-I efficacy.

(I) Average imbalance (top) and instantaneous balance (bottom) balance, as a function of the ratio of mean I-I to E-I efficacy.

(J) The encoding error (RMSE; left) the metabolic cost (middle) and the average loss (right) as a function of the ratio of E-I neuron numbers and the ratio of mean I-I to E-I connectivity. The optimal ratios obtained with single parameter search (in D and G) are marked with a red cross. All statistical results were computed on 100 simulation trials of 1 second duration. For other parameters, see Table 1.

Dependence of efficient coding and neural dynamics on stimulus parameters and comparison of E-I versus one cell type model architecture.

(A) Top: Root mean squared error (RMSE) of E estimates (red) and I estimates (blue), as a function of the time constant (in ms) of stimulus features. The time constant τs is the same for all stimulus features. Bottom: Same as on top, showing the metabolic cost (MC) of E and I cell type.

(B) Left: Mean squared error between the targets and their estimates for every stimulus feature (marked as dimensions), as a function of time constants of OU stimuli in E population (top) and in I population (bottom). In the first dimension, the stimulus feature has a time constant fixed at 10 ms, while the second and third feature increase their time constants from left to right. The time constant of the third stimulus feature (x-axis on the bottom) is the double of the time constant of the second stimulus feature (x-axis on top). Right: Same as on the left, showing the RMSE that was averaged across stimulus features (top), and the metabolic cost (bottom) in E (red) and I (blue) populations.

(C) Top: Same as in A top, measured as a function of the number of stimulus features M. Bottom: Normalized cost and the average loss as a function of the number of stimulus features. Black arrow marks the minimum loss and the optimal parameter M.

(D) Top: Optimal number of encoded variables (stimulus features) as a function of weighting of the losses of E and I population. The weighting of the error with the cost is 0.7. Bottom: Same as on top, as a function of the weighting of the error with the cost and with equal weighting of losses of E and I populations.

(E) Tuning curves of 10 example E (left) and I neurons (right). We computed tuning curves using M =3 stimulus features that were constant over time. We varied the amplitude of the first stimulus feature s1, while two other stimulus features were kept fixed.

(F) Distribution of the selectivity index across E (red) and I neurons (blue).

(G) Root mean squared error (left) and metabolic cost (right) in E and I populations in the E-I model and in the 1CT model. The distribution is across 100 simulation trials.

(H) Left: Average loss in the E population of the E-I model and of the 1CT model. The distribution is across 100 simulation trials. Right: Average loss in the E population of the E-I models and in the 1CT model as a function of the weighting gL, averaged across trials.

(I) Firing rate in the 1CT model as a function of the metabolic constant. All statistical results were computed on 100 simulation trials of 1 second duration. For other parameters of the E-I model see Table 1, and for the 1CT model see Supplementary Table S2

Table of optimal model parameters for the efficient E-I network without non-specific synaptic currents.

As in Table 1, for the E-I model without non-specific currents. The model is defined in Eq. 25.

Table of default model parameters for the efficient network with one cell type.

The parameters N,M,τ and were chosen identical to the E-I network (see Table 1 in the main text). Parameters σ1 β1 were determined as values that maximize network efficiency (see section “Performance measures” in the main text).

Efficient spiking model with one cell type and the encoding bias of the E-I network.

(A) Schematic of efficient coding with a single spiking neuron. In this toy example, the neuron has a positive decoding weight and responds to a single stimulus feature s(t) (top). The target signal x(t) (bottom, black) integrates the stimulus feature from top. The neuron spikes to keep the readout of its activity (magenta) close to the target. (B) Schematic of an efficient 1CT model. The target signal is computed from the stimulus feature. The spiking network estimates the targets by generating population readouts of its spiking activity. (C) Schematic of excitatory (red) and inhibitory (blue) synaptic interactions in the 1CT model. Neurons with similar selectivity inhibit each other (blue), while neurons with different selectivity excite each other (red). The same neuron is sending excitatory and inhibitory synaptic outputs, which is not consistent with Dale’s law. (D) Strength of recurrent synapses as a function of pair-wise tuning similarity. (E) Simulation of the model with 1CT. Top three rows show the target (black), and the estimate (magenta) in each of the 3 input dimensions. (F) Top left: The target (black) and the estimate (red) of the E population in the first signal dimension (in response to the first stimulus feature s1(t)). The estimate is averaged across 100 trials, with trials varying about the initial conditions and the noise in the membrane potential. Bottom left: Same as on top left, for the I population. Right: Time-dependent bias of estimates in E (top) and I (bottom) population in each of the three stimulus dimensions. (G) Left: Root mean squared error (RMSE) as a function of the metabolic constant β1. Right: Normalized metabolic cost (green) and normalized average loss (black) as a function of the metabolic constant β1. The black arrow denotes the minimum of the loss and thus the optimal parameter β1. (H) Same as in G, measured as a function of the noise strength σ1. Results in F,G and H were computed in 100 simulation trials of duration of 1 second. For other parameters, see Table 1 (E-I model) and Table S2 (1CT model). This figure is related to Fig. 1 in the main paper.

Tuning similarity and its relation to lateral excitation/inhibition.

(A) Pair-wise tuning similarity for all pairs of E neurons. Tuning similarity is measured as cosine similarity of decoding vectors between the target neuron and every other E neuron. (B) Histogram of tuning similarity across all E-E pairs shown in A.(C) Tuning similarity to a single, randomly selected target neuron. Tuning similarity to a target neuron corresponds to a vector from the tuning similarity matrix in A. We sorted the tuning similarity to target from the smallest to the biggest value. Neurons with negative similarity are grouped as neurons with different tuning, while neurons with positive tuning similarity are grouped as neurons with similar tuning. (D) Histogram of tuning similarity of E neurons to the target neuron shown in C. With distribution of tuning parameters that is symmetric around zero as used in our study, any choice of the target neuron gives approximately the same number of neurons with similar and different selectivity. (E) Top: Trial and neuron-averaged deviation of the instantaneous firing rate from the baseline in presence of weak feedforward stimulus. We show the ± standard error of the mean (SEM) of neurons with similar (orange) and different tuning (gray) to the target neuron. The photostimulation intensity is at threshold (ap = 1.0). The feedforward stimulus was received by all E neurons and it induced, together with the external current, the mean firing rates of 7.3 Hz and 13.5 Hz in E and I neurons, respectively. Bottom: Scatter plot of the tuning similarity versus effective connectivity. Magenta line marks the least-squares line. (F) Same as in E, for the network with partial (fine-grained) removal of connectivity structure. Partial removal of connectivity structure is achieved by shuffling the synaptic weights among pairs of neurons with similar tuning . For model parameters, see Table 1. This figure is related to Fig. 3 in the main paper.

Effect of removal of connectivity structure and of jittering of synaptic weights.

(A) Distribution of decoding weights after training a linear decoder on neural activity generated by the network without connectivity structure. (B) RMSE in E (top) and I neurons (bottom) in networks with partial removal of connectivity structure. Partial removal of connectivity structure is achieved by limiting the permutation of synaptic connectivity to neuronal pairs with similar tuning, e.g. to neuronal pairs for which the following is true: . (C) Same as in B, showing the average metabolic cost on spiking. (D) Same as in B, showing the average net synaptic input, a measure of the average E-I balance. (E) Same as in B, showing the correlation of synaptic inputs, a measure of instantaneous balance. (F) Average deviation of the instantaneous firing rate from the baseline for the population of I (top) and E (bottom) neurons in networks with fully removed structure in E-I (left), I-E (middle) and in all connectivity matrices (right). We show the mean ± SEM for neurons with similar (ochre) and different (green) tuning to the target neuron. The mean traces of the network with structured connectivity are shown for comparison, with magenta and gray for similar and different tuning, respectively. (G) Top: The RMSE (top) in E and I cell type, as a function of the strength of perturbation of the synaptic connectivity by random jittering. Bottom: Same as on top, showing the normalized metabolic cost (green) and average loss (black). (H) Target signals, E estimates and I estimates in three input dimensions (three top rows), spike trains (fourth row) and the instantaneous estimate of the firing rate of E and I populations (bottom) in a simulation trial, with significant jitter of recurrent connectivity (jittering strength of 0.5, see Methods). In spite of a relatively strong jittering, the network shows excellent encoding of the target signal. All statistical results were computed in 100 simulation trials of duration of 1 second. Other parameters are in Table 1. This figure is related to Fig. 4 in the main paper.

Dependence of optimal parameters on weighting of the encoding error and the metabolic cost and analysis of mean ratio of I-I to E-I connectivity by varying the number of E neurons.

(A) Optimal set of time constants of E and I neurons for different weightings between the error and the cost when computing the loss. Optimal time constants show little dependency on this weighting. (B) Top: Optimal metabolic constant as a function of the weighting of the average loss of E and I cell type. Bottom: Same as on top, as a function of the weighting between the error and the cost. Black triangles mark weightings that are typically used to estimate optimal model efficiency. (C) Same as in B, as a function of noise strength. (D) Same as in B, as a function of the optimal ratio of I-I to E-I connectivity. This analysis was performed by varying the number of I neurons while the number of E neurons stays fixed. (E) Top: Encoding error (RMSE) of the E (red) and I (blue) estimates as a function of mean I-I to E-I connectivity. The ratio was varied by changing the number of E neurons and keeping the number of I neurons fixed at a value specified in Table 1. Bottom: Same as on top, showing the normalized cost and average loss. (F) Same as in E, showing, the average firing rate (top), and average coefficient of variation (bottom) in E and I cell type. (G) Same as in E, showing the average imbalance and instantaneous balance of synaptic currents in E and I neurons. (H) Same as in D, for the optimal ratio measured by varying the number of E neurons. All results were computed in 100 trials of duration of 1 second for each trial. For other parameters, see Table 1. This figure is related to Fig. 7 in the main paper.

Effect of stimulus properties on efficient neural coding and dynamics.

(A) Average firing rate (top), and average coefficient of variation (bottom) in E and I cell type, as a function of the time constant of the stimulus features τs. All stimulus features have the same time constant. (B) Average imbalance (top) and instantaneous balance (bottom) as a function of the time constant of stimuli τs. (C) Top: RMSE of E (red) and I (blue) estimates as a function of the time constant of the targets τx. All targets have the same time constant. Middle: Metabolic cost in the E and I population. Bottom: Average loss in the E and I population. Black arrow indicates the minimum loss and therefore the optimal time constant. (D-E) Same as in A-B, as a function of the time constant of the targets τx. (F-G) Same as in A-B, as a function of the number of encoded variables M.

All results were computed in 100 trials of duration of 1 second. For parameters, see Table 1. This figure is related to Fig. 8 in the main paper.