Figures and data
Structural and dynamical properties of the efficient E-I spiking network.
(A) Encoding of a target signal representing the evolution of a stimulus feature (top) with one E (middle) and one I spiking neuron (bottom). The target signal x(t) integrates the input signal s(t). The readout of the E neuron tracks the target signal and the readout of the I neuron tracks the readout of the E neuron. Neurons spike to bring the readout of their activity closer to their respective target. Each spike causes a jump of the readout, with the sign and the amplitude of the jump being determined by neuron’s tuning parameters.
(B) Schematic of the matrix of tuning parameters. Every neuron is selective to all stimulus features (columns of the matrix), and all neurons participate in encoding of every feature (rows).
(C) Schematic of the network with E (red) and I (blue) cell type. E neurons are driven by the stimulus features while I neurons are driven by the activity of E neurons. E and I neurons are connected through recurrent connectivity matrices.
(D) Schematic of E (red) and I (blue) synaptic interactions. Arrows represent the direction of the tuning vector of each neuron. Only neurons with similar tuning are connected.
(E) Schematic of similarity of tuning vectors (tuning similarity) in a 2-dimensional space of stimulus features.
(F) Synaptic strength as a function of tuning similarity.
(G) Coding and dynamics in a simulation trial. Top three rows show the signal (black), the E estimate (red) and the I estimate (blue) in each of the three stimulus dimensions. Below are the spike trains. In the bottom row, we show the average instantaneous firing rate (in Hz).
(H) Top: Example of the target signal (black) and the E estimate in 3 simulation trials (colors) in one signal dimension. Bottom: Distribution (across time) of the time-dependent bias of estimates in E and I cell type.
(I) Left: Distribution of time-averaged firing rates in E (top) and I neurons (bottom). Black traces are fits with log-normal distribution. Right: Distribution of coefficients of variation of interspike intervals for E and I neurons.
(J) Distribution (across neurons) of time-averaged synaptic inputs to E (left) and I neurons (right). In E neurons, the distribution of inhibitory and of net synaptic inputs overlap.
(K) Sum of synaptic inputs over time in a single E (top) and I neuron (bottom) in a simulation trial.
(L) Distribution (across neurons) of Pearson’s correlation coefficients measuring the correlation of synaptic inputs in single E (red) and I (blue) neurons. For model parameters, see Table 1.
Table of default model parameters for the efficient E-I network
Parameters above the double horizontal line are the minimal set of parameters needed to simulate model equations (Eqs. 30a-30h in Methods). Parameters below the double horizontal line are biophysical parameters, derived from the same model equations and from model parameters listed above the horizontal line. Parameters NE, M, τ and 
Mechanism of lateral excitation/inhibition in the efficient spiking network.
(A) Left: Schematic of the E-I network and of the stimulation and measurement in a perturbation experiment. Right: Schematic of the propagation of the neural activity between E and I neurons with similar tuning.
(B) Trial and neuron-averaged deviation of the firing rate from the baseline, for the population of I (top) and E (bottom) neurons with similar (magenta) and different tuning (gray) to the target neuron. The stimulation strength corresponded to an increase in the firing rate of the stimulated neuron by 28.0 Hz.
(C) Scatter plot of the tuning similarity vs. effective connectivity to the target neuron. Red line marks zero effective connectivity and magenta line is the least-squares line. Stimulation strength was ap = 1.
(D) Top: Firing rate of the photostimulated neuron as a function of the photostimulation strength. Middle: Effective connectivity with I neurons with similar and different tuning to the target neuron. Bottom: Effective connectivity with E neurons.
(E) Effective connectivity with I (top) and E neurons (bottom) while varying the length of the stimulation window. The window for measuring the effective connectivity was always 50 ms longer than the stimulation window.
(F) Correlation of membrane potentials vs. the tuning similarity in E (top) and I cell type (bottom), for the efficient E-I network (left), for the network where each E neuron receives independent instead of shared stimulus features (middle), and for the network with unstructured connectivity (right). In the model with unstructured connectivity, elements of each connectivity matrix were randomly shuffled. We quantified voltage correlation using the (zero-lag) Pearson’s correlation coefficient, denoted as 
(G) Average cross-correlogram (CCG) of spike timing with strongly similar (orange), weakly similar (green) and different tuning (black).
(H) Distribution of noise correlations across neuronal pairs. The correlation coefficient was measured in bins of 30 ms.
Effects of connectivity structure on coding efficiency, neural dynamics and lateral inhibition.
(A) Relative error of networks with unstructured (shuffled) recurrent connectivity. The relative error is the RMSE of the unstructured network, relative to the RMSE of the structured network (dashed line). From left to right, we show the relative error for the unstructured E-I, I-I, I-E and all connectivities. (B Same as in A, showing the metabolic cost (MC) of unstructured networks relative to the metabolic cost of the structured network.
(C) Target signal (black), E estimate (red) and I estimate (blue) in one particular input dimension, for networks with unstructured connectivity.
(D) Standard deviation of the membrane potential (in mV) for networks with unstructured connectivity. Distributions are across neurons. The black vertical line marks the average SD of the structured network.
(E) Average firing rate of E neurons (top) and I neurons (bottom), for different cases of unstructured networks. Dashed lines show the same measures for the structured case.
(F) Same as in E, showing the average net synaptic input.
(G) Same as in E, showing the time-dependent correlation of synaptic inputs.
(H) Voltage correlation in E-E (top) and I-I neuronal pairs (bottom) for the four cases of unstructured connectivity (colored dots) and the equivalent result in the structured network (grey dots). We show the results for pairs with similar tuning.
(I) Scatter plot of effective connectivity in I (top) and E neurons (bottom) versus tuning similarity to the stimulated (“target”) E neuron, for networks with unstructured connectivity. The magenta line is the least-squares regression line. The strength of the photostimulation is at threshold (ap = 1.0). Other parameters for all plots are in Table 1.
Relation of time constants of single-neuron and population readout set an adaptation or a facilitation current.
The population readout that evolves on a faster (slower) time scale than the single neuron readout determines a spike-triggered adaptation (facilitation) in its own cell type.
Adaptation, network coding efficiency and excitation-inhibition balance.
(A) The encoding error (left), metabolic cost (middle) and average loss (right) as a function of single neuron time constants 
(B) Top: Log-log plot of the RMSE of the E (red) and the I (blue) estimates as a function of the time constant of the single neuron readout of E neurons,
(C) Firing rate in E (left) and I neurons (right), as a function of 
(D) Same as in (C), showing the coefficient of variation.
(E) Average net synaptic input in E neurons (left) and in I neurons (right) as a function of 
(F) Correlation coefficient of synaptic inputs to E (left) and I neurons (right) as a function of 
State-dependent coding and dynamics are controlled by non-specific currents.
(A) Spike trains of the efficient E-I network in one simulation trial, with different values of the metabolic constant β. The network received identical stimulus across trials.
(B) Top: RMSE of E (red) and I (blue) estimates as a function of the metabolic constant. Bottom: Normalized average metabolic cost and average loss as a function of the metabolic constant. Black arrow indicates the minimum loss and therefore the optimal metabolic constant.
(C) Average firing rate (top) and the coefficient of variation of the spiking activity (bottom), as a function of the metabolic constant. Black arrow marks the metabolic constant leading to optimal network efficiency in B.
(D) Average imbalance (top) and instantaneous balance (bottom) balance as a function of the metabolic constant.
(E) Same as in A, but for different values of the noise intensity σ.
(F) Same as in B, as a function of the noise intensity. The noise is a Gaussian random process, independent over time and across neurons.
(G) Same as C, as a function of the noise intensity.
(H) Top: Same as in D, as a function of the noise intensity. For plots in B-D and F-H, we computed and averaged results over 100 simulation trials with 1 second of simulation time. For other parameters, see Table 1.
Optimal ratios of E-I neuron numbers and of mean I-I to E-I efficacy.
(A) Schematic of the effect of changing the number of I neurons on firing rates of I neurons. As encoding of the stimulus is distributed among more I neurons, the number of spikes per I neuron decreases.
(B) Average firing rate as a function of the ratio of the number of E to I neurons. Black arrow marks the optimal ratio.
(C) Average net synaptic currents in E neurons (top) and in I neurons (bottom).
(D) Top: Encoding error (RMSE) of the E (red) and I (blue) estimates, as a function of the ratio of E-I neuron numbers. Bottom: Same as on top, showing the cost and the average loss. Black arrow shows the minimum of the loss, indicating the optimal parameter.
(E) Top: Optimal ratio of the number of E to I neurons as a function of the weighting of the average loss of E and I cell type (using the weighting of the error and cost of 0.7 and 0.3, respectively). Bottom: Same as on top, measured as a function of the weighting of the error and the cost when computing the loss. (The weighting of the losses of E and I neurons is 0.5.) Black triangles mark weightings that we typically used.
(F) Schematic of the readout of the spiking activity of an E neuron (red) and an I neuron (blue) with equal amplitude of decoding weight (left) and with stronger decoding weight in the I neuron (right). Stronger decoding weight in the I neuron results in a stronger effect of spikes of the I neuron on the readout, leading to less spikes by the I neuron.
(G) Same as in (D), as a function of the ratio of mean I-I to E-I efficacy.
(H) Same as in B, as a function of the ratio of mean I-I to E-I efficacy.
(I) Average imbalance (top) and instantaneous balance (bottom) balance, as a function of the ratio of mean I-I to E-I efficacy. For other parameters, see Table 1.
Dependence of efficient coding and neural dynamics on stimulus parameters and advantages of E-I versus one cell type model architecture.
(A) Top: Root mean squared error (RMSE) of E estimates (red) and I estimates (blue), as a function of the time constant of stimulus features. Bottom: Same as on top, showing the metabolic cost (MC) of E and I cell type. The time constant τs is the same for all stimulus features.
(B) Top: Same as in A top, measured as a function of the number of stimulus features M. Bottom: Normalized cost and the average loss as a function of the number of input features. Black arrow marks the minimum loss and the optimal parameter M.
(C) Root mean squared error (top) and metabolic cost (bottom) in E and I populations in the E-I model and in the 1CT model. The distribution is across simulation trials.
(D) Average loss in the E-I and 1CT models with weighting gL = 0.7 for the error (and 0.3 for the cost).
(E) Firing rate in the 1CT model as a function of the metabolic constant. For other parameters of the E-I model see Table 1, and for the 1CT model see Supplementary Table S1.
Table of default model parameters for the efficient network with one cell type.
The parameters N, M, τ and σw are chosen identical to the E-I network (see Table 1 in the main text). Parameters σ1 and β1 are determined as values that maximize network efficiency (see section “Performance measures” in the main text).
Efficient spiking model with one cell type.
(A) Schematic of efficient coding with a single spiking neuron with positive weight. The target signal (bottom, black) integrates the input signal (top). The neuron spikes to keep the readout of its activity (magenta) close to the target signal.
(B) Schematic of the efficient 1CT model. Target signal x(t) is computed from stimulus features s(t). The network generates the estimate of the target signal with the population readouts of the spiking activity.
(C) Schematic of excitatory (red) and inhibitory (blue) synaptic interactions in 1CT model. Neurons with similar selectivity inhibit each other (blue), while neurons with different selectivity excite each other (red). The same neuron is sending excitatory and inhibitory synaptic outputs.
(D) Strength of recurrent synapses as a function of the tuning similarity.
(E) Simulation of the network with 1CT. Top three rows show the signal (black), and the estimate (magenta) in each of the 3 input dimensions.
(F) Left: Root mean squared error (RMSE) as a function of the metabolic constant β1. Right: Normalized metabolic cost (green) and normalized average loss (black) as a function of the metabolic constant β1. The black arrow denotes the minimum of the loss and thus the optimal parameter β1.
(G) Same as in F, measured as a function of the noise intensity σ1.
(H) Average loss as a function of the weighting of the encoding error and the metabolic cost, gL, in the E-I model (black) and in the 1CT model (magenta). For plots F-H, results were computed in 100 simulation trials of duration of 1 second of simulated time. For other parameters, see Table 1 (E-I model) and Table S1 (1CT model).
Tuning similarity and its relation to lateral excitation/inhibition.
(A) Pair-wise tuning similarity for all pairs of E neurons. Tuning similarity between pairs of neurons is measured as the similarity of normalized tuning vectors.
(B) Histogram of tuning similarity across all E-E pairs shown in A.
(C) Tuning similarity to a single, randomly selected target neuron. Tuning similarity to a single neuron corresponds to a vector from the tuning similarity matrix in A. We sorted the tuning similarity to a single neuron from smallest to biggest value. Neurons with negative similarity are grouped as neurons with different tuning, while neurons with positive tuning similarity are grouped as neurons with similar tuning.
(D) Histogram of tuning similarity of E neurons to the target neuron shown in C. With distribution of tuning parameters symmetric around zero as used in our study, any choice of target neuron gives approximately the same number of neurons with similar and different selectivity.
(E) Top: Trial and neuron-averaged deviation of the instantaneous firing rate from the baseline firing rate, for the population of I (top) and E (bottom) neurons with similar tuning (magenta) and different tuning (gray). The baseline firing rates were 6.8 Hz and 12.7 Hz in the E and I cell types, respectively. The stimulation intensity is ap = 0.4. Figure shows the mean ± standard error of the mean (SEM), with SEM capturing the variance across neurons and across trials. Bottom: Scatter plot of the tuning similarity versus effective connectivity in I (top) and E neurons (bottom). Tuning similarity and effective connectivity are measured with respect to the (same) target neuron. Red line marks zero effective connectivity and magenta line marks the least-squares line.
(F) Same as in E, for stimulation intensity of ap = 0.8.
(G) Same as in E, in presence of weak feedforward stimulus, showing the activity of neurons with similar tuning (orange) and different tuning (gray) to the stimulated neuron. We used the stimulation intensity at threshold (ap = 1.0). The feedforward stimulus was received by all E neurons and it induced, together with the external current, the mean firing rates of 7.3 Hz and 13.5 Hz in E and I neurons, respectively. For model parameters, see Table 1. This figure is related to the Fig. 2 in the main paper.
Effect of complete and partial removal of connectivity structure and of minimal perturbation of synaptic weights.
(A) Average coefficient of variation in networks with fully unstructured connectivity. The dashed line marks the same measure in a structured network.
(B) Mean firing rate in E (top) and I neurons (bottom) in networks with partial removal of connectivity structure in recurrent connectivity. Partial removal of connectivity structure is achieved by limiting the permutation of synaptic connectivity to neuronal pairs with similar tuning.
(C) Same as in B, showing the coefficient of variation of spiking activity.
(D) Same as in B, showing the average net synaptic current, neural correlate of the average E-I balance.
(E) Same as in B, showing the correlation coefficient of synaptic currents, neural correlate of the instantaneous E-I balance.
(F) Encoding error in networks with partially unstructured recurrent connectivity, relative to the encoding error of the structured network (dashed line). From left to right: we perturb synaptic weights in E-I, I-I, I-E and in all three recurrent connectivities at once.
(G) Same as in F, showing the metabolic cost on spiking in E and I populations, relative to the metabolic cost in the structured network (dashed line).
(H) The RMSE (top) and the normalized metabolic cost (green) and average loss (black) average firing rate (bottom) in E and I cell type, as a function of the strength of perturbation of the synaptic connectivity.
(I) Average firing rate (top) and the coefficient of variation (bottom) as a function of the strength of random perturbation of all recurrent connectivities.
(J) Target signals, E estimates and I estimates in three input dimensions (three top rows), spike trains (fourth row) and the instantaneous estimate of the firing rate of E and I populations (bottom) in a single simulation trial, with significant perturbation of recurrent connectivity (perturbation strength of 0.5, see Methods). In spite of a relatively strong perturbation, the network shows excellent encoding of the target signal. Other parameters are in Table 1. This figure is related to the Fig. 3 in the main paper.
Lateral excitation/inhibition in models with full and partial removal of connectivity structure.
(A) Average deviation of the instantaneous firing rate from the baseline for the population of I (top) and E (bottom) neurons in networks with fully removed structure in E-I (left), I-E (middle) and in all connectivity matrices (right). We show the mean ± SEM for neurons with similar (ochre) and different (green) tuning to the stimulated neuron. The mean traces of the network with structured connectivity is shown for comparison, magenta and gray for similar and different tuning, respectively.
(B) Same as in A, for partial (fine-grained) removal of connectivity structure. Partial removal of connectivity structure is achieved by limiting the permutation of synaptic weights among neurons with similar tuning. Such manipulation maintains the like-like connectivity structure, but removes any structure beyond the like-like.
(C) Scatter plot of tuning similarity versus effective connectivity for networks with partial removal of connectivity structure. In such networks, the specificity of effective connectivity with respect to tuning similarity is largely preserved, in particular in E neurons. For all results, we iterated simulations in 200 trials, where we varied randomly the membrane potential noise and initial conditions of the membrane potentials in each trial, while tuning and synaptic parameters were kept fixed. In all cases, we used stimulation intensity at threshold (ap = 1.0). For model parameters, see Table 1. This figure is related to the Fig. 3 in the main paper.
Dependence of coding efficiency and neural dynamics on the ratio of mean I-I to E-I connectivity, computed by changing the mean E-I connectivity.
(A) Top: Encoding error (RMSE) of the E (red) and I (blue) estimates. Bottom: Normalized metabolic cost and average loss.
(B) Average firing rate (top), and average coefficient of variation (bottom) in E and I cell type.
(C) Average imbalance and instantaneous balance of synaptic currents in E and I neurons.
(D) Top: Optimal ratio of mean I-I to E-I connectivity as a function of the weighting of the average loss of E and I cell type. Bottom: Same as on top, as a function of the weighting between the error and the cost. Black triangles mark weightings that are typically used to estimate optimal efficiency. For other parameters, see Table 1. This figure is related to the Fig. 6 in the main paper.
Effect of stimulus properties on efficient neural coding and dynamics.
(A) Average firing rate (top), and average coefficient of variation (bottom) in E and I cell type, as a function of the time constant of the stimulus τs.
(B) Average imbalance (top) and instantaneous balance (bottom) as a function of the time constant of the stimulus τs.
(C-D) Same as in A-B, as a function of the number of encoded variables. For parameters, see Table 1. This figure is related to the Fig. 7 in the main paper.
