We study the properties of a spiking neural network in which the dynamics and structure of the network are analytically derived starting from first principles of efficient coding of sensory stimuli. The model relies on a number of assumptions, described next.

The network responds to ${\displaystyle M}$ time-varying features of a sensory stimulus, ${\displaystyle \overrightarrow{s}(t)=[s_1(t),\dots ,s_M(t){]}^{\mathrm{\top }}}$ (e.g. for a visual stimulus, contrast, orientation, etc.) received as inputs from an earlier sensory area. We model each feature ${\displaystyle s_k(t)}$ as an independent Ornstein–Uhlenbeck (OU) processes (see Materials and methods). The network’s objective is to compute a leaky integration of sensory features; the target representations of the network, ${\displaystyle {\displaystyle \overrightarrow{x}(t)}}$, is defined as

with ${\displaystyle \tau }$ a characteristic integration time-scale (Figure 1A(i)). We assumed leaky integration of sensory features for consistency with previous theoretical models (Barrett et al., 2016; Chalk et al., 2016; Gutierrez and Denève, 2019). This assumption stems from the finding that, in many cases, integration of sensory evidence by neurons is well described by an exponential kernel (Scott et al., 2017; Danskin et al., 2023). Additionally, a leaky integration of neural activity with an exponential kernel implemented in models of neural activity readout often explains well the perceptual discrimination results (Gold and Shadlen, 2001; Usher and McClelland, 2001; Chong et al., 2020). This suggests that the assumption of leaky integration of sensory evidence, although possibly simplistic, captures relevant aspects of neural computations.

Figure 1 with 1 supplement see all

Download asset Open asset

The network is composed of two neural populations of excitatory (E) and inhibitory (I) neurons, defined by their postsynaptic action which respects Dale’s law. For each population, ${\displaystyle y\in \{E,I\}}$, we define a population readout of each feature, ${\displaystyle {\overrightarrow{\hat{x}}}^y(t)}$, as a filtered weighted sum of spiking activity of neurons in the population,

where ${\displaystyle f_i^y(t)}$ is the spike train of neuron ${\displaystyle i}$ of type ${\displaystyle y}$ and ${\displaystyle {\overrightarrow{w}}_i^y=[w_{1i}^y,\dots ,w_{Mi}^y{]}^{\mathrm{\top }}}$ is the vector of decoding weights of the neuron for features ${\displaystyle k=1,...,M}$ (Figure 1A(ii)). We assume that every neuron encodes multiple (${\displaystyle M>1}$) stimulus features and that the encoding of every stimulus is distributed among neurons. As a result of the optimization, the decoding weights of the neurons are equivalent to the neuron’s stimulus tuning parameters (see Materials and methods and Brendel et al., 2020). We sampled tuning parameters uniformly from a ${\displaystyle M}$-dimensional hypersphere with unit radius, giving tuning vectors with unit length to all neurons (see Materials and methods). To control the amount of inhibition in the network, we then multiplied the tuning vectors of I neurons with a factor ${\displaystyle d>1}$, homogeneously across all I neurons. Normalization of decoding vectors preserves the heterogeneity of decoding weights across neurons, which may benefit coding efficiency (Zeldenrust et al., 2021).

Following previous work (Boerlin et al., 2013; Barrett et al., 2016; Chalk et al., 2016), we impose that E and I neurons have distinct normative objectives and we define specific loss functions relative to each neuron type. To implement at the same time, as requested by efficient coding, the constraints of faithful stimulus representation with limited computational resources (Tavoni et al., 2019), we define the loss functions of the population ${\displaystyle y\in \{E,I\}}$ as a weighted sum of a time-dependent encoding error and time-dependent metabolic cost:

We refer to ${\displaystyle \beta }$, the parameter controlling the relative importance of the metabolic cost over the encoding error, as the metabolic constant of the network. We hypothesize that population readouts of E neurons, ${\displaystyle {\overrightarrow{\hat{x}}}^E(t)}$, track the target representations, ${\displaystyle \overrightarrow{x}(t)}$, and the population readouts of I neurons, ${\displaystyle {\overrightarrow{\hat{x}}}^I(t)}$, track the population readouts of E neurons, ${\displaystyle {\overrightarrow{\hat{x}}}^E(t)}$, by minimizing the squared error between these quantities (Koren and Panzeri, 2022) (see also (Boerlin et al., 2013; Denève et al., 2017) for related approaches). Furthermore, we hypothesize the metabolic cost to be proportional to the instantaneous estimate of network’s firing frequency. We thus define the variables of loss functions in Equation 3 as

where ${\displaystyle r_i^y,y\in \{E,I\}}$, is the low-pass filtered spike train of neuron ${\displaystyle i}$ (single neuron readout) with time constant ${\displaystyle {\tau }_r^y}$, proportional to the instantaneous firing rate of the neuron: ${\displaystyle z_i^y(t)=({\tau }_r^y{)}^{-1}{r}_i^y(t)}$. We then impose the following condition for spiking: a neuron emits a spike at time ${\displaystyle t}$ only if this decreases the loss function of its population (Equation 3) in the immediate future. The condition for spiking also includes a noise term (Materials and methods) accounting for sources of stochasticity in spike generation (Faisal et al., 2008) which include the effect of non-specific inputs from the rest of the brain.

We derived the dynamics and network structure of a spiking network that instantiates efficient coding (Figure 1B, see Materials and methods). The derived dynamics of the subthreshold membrane potential ${\displaystyle {\displaystyle V_i^E(t)}}$ and ${\displaystyle {\displaystyle V_i^I(t)}}$ obey the equations of the generalized leaky integrate and fire (gLIF) neuron

where ${\displaystyle I_i^{\text{syn},y}}$, ${\displaystyle I_i^{\text{ad},y}}$, and ${\displaystyle I_i^{\text{ext},y}}$ are synaptic current, spike-triggered adaptation current and non-specific external current, respectively, ${\displaystyle R_m}$ is the membrane resistance and ${\displaystyle V_{\text{rest}}^y}$ is the resting potential. This dynamics is complemented with a fire-and-reset rule: when the membrane potential reaches the firing threshold ${\displaystyle {\vartheta }^y}$, a spike is fired and ${\displaystyle V_i^y(t)}$ is set to the reset potential ${\displaystyle V_i^{\text{reset},y}}$. The analytical solution in Equation 5 holds for any number of neurons (with at least 1 neuron in each population) and predicts an optimal spike pattern to encode the presented external stimulus. Following previous work (Boerlin et al., 2013) in which physical units were assigned to derived mathematical expressions to interpret them as biophysical variables, we express computational variables (target stimuli in Equation 1, population readouts in Equation 2 and the metabolic constant in Equation 3) with physical units in such a way that all terms of the biophysical model (Equation 5) have realistic physical units.

The synaptic currents in E neurons, ${\displaystyle I_i^{\text{syn},E}}$, consist of feedforward currents, obtained as stimulus features ${\displaystyle \overrightarrow{s}(t)}$ weighted by the tuning weights of the neuron, and of recurrent inhibitory currents (Figure 1B). Synaptic currents in I neurons, ${\displaystyle I_i^{\text{syn},I}}$, consist of recurrent excitatory and inhibitory currents. Note that there are no recurrent connections between E neurons, a consequence of our assumption of no across-feature interaction in the leaky integration of stimulus features (Equation 1). This assumption is likely to be simplistic even for early sensory cortices (Emanuel et al., 2021). However, in other studies, we found that many properties of efficient networks implementing leaky integration hold also when input features are linearly mixed during integration (Koren and Panzeri, 2022; Koren et al., 2023).

The optimization of the loss function yielded structured recurrent connectivity (Figure 1B(ii)–C). Synaptic strength between two neurons is proportional to their tuning similarity, forming like-to-like connectivity, if the tuning similarity is positive; otherwise the synaptic weight is set to zero (Figure 1C(ii)) to ensure that Dale’s law is respected. A connectivity structure in which the synaptic weight is proportional to pairwise tuning similarity is consistent with some empirical observations in visual cortex (Znamenskiy et al., 2024) and has been suggested by previous models (Boerlin et al., 2013; Sadeh and Clopath, 2020). Such connectivity organization is also suggested by across-neuron influence measured with optogenetic perturbations of visual cortex (Chettih and Harvey, 2019; Oldenburg et al., 2024). While such connectivity structure is the result of optimization, the rectification of the connectivity that enforces Dale’s law does not emerge from imposing efficient coding, but from constraining the space of solutions to biologically plausible networks. Rectification also sets the overall connection probability to 0.5, which is consistent with empirically observed connection probability from pyramidal (E) neurons to parvalbumin-positive (I) neurons (Pala and Petersen, 2015; Campagnola et al., 2022), but likely overestimates the connection probability from parvalbumin-positive neurons to pyramidal neurons, which tends to be lower (Campagnola et al., 2022). (For a study of how efficient coding would be implemented if the above Dale’s law constraint were removed and each neuron were free to have either an inhibitory or excitatory effect depending on the postsynaptic target, see Appendix 1 and Figure 1—figure supplement 1A–E).

The spike-triggered adaptation current of neuron ${\displaystyle i}$ in population ${\displaystyle y}$, ${\displaystyle I_i^{\text{ad},y}}$, is proportional to its low-pass filtered spike train. This current realizes spike-frequency adaptation or facilitation depending on the difference between the time constants of population and single neuron readout (see Results subsection ‘Weak or no spike-triggered adaptation optimizes network efficiency’).

Finally, non-specific external currents ${\displaystyle I_i^{\text{ext},y}(t)}$ have a constant mean that depends on the parameter ${\displaystyle \beta }$, and fluctuations that arise from the noise with strength ${\displaystyle \sigma }$ in the condition for spiking. The relative weight of the metabolic cost over the encoding error, ${\displaystyle \beta }$, controls how the network responds to feedforward stimuli, by modulating the mean of the non-specific synaptic currents incoming to all neurons. Together with the noise strength ${\displaystyle \sigma }$, these two parameters set the non-specific synaptic currents to single neurons that are homogeneous across the network and akin to the background synaptic input discussed in Destexhe et al., 2003. By allowing a large part of the distance between the resting potential and the threshold to be taken by the non-specific current, we found a biologically plausible set of optimally efficient model parameters (Table 1) including the firing threshold at about 20 mV from the resting potential, which is within the experimental ballpark (Constantinople and Bruno, 2013), and average synaptic strengths of 0.75 mV (E-I and I-E synapses) and 2.25 mV (I-I synapses), which are consistent with measurements in sensory cortex (Campagnola et al., 2022). An optimal network without non-specific currents can be derived (see Materials and methods, Equation 25), but its parameters are not consistent with biology (see Appendix 2). The non-specific currents can be interpreted as synaptic currents that are modulated by larger-scale variables, such as brain states (see subsection ‘Non-specific currents regulate network coding properties’).

To summarize, the analytical derivation of an optimally efficient network includes gLIF neurons (Burkitt, 2006; Jolivet et al., 2008; Gerstner et al., 2014; Schwalger et al., 2017; Harkin et al., 2021), a distributed code with linear mixed selectivity to the input stimuli (Chang and Tsao, 2017; Kaufman et al., 2014), spike-triggered adaptation, structured synaptic connectivity, and a non-specific external current akin to background synaptic input.

The equations for the E-I network of gLIF neurons in Equation 5 optimize the loss functions at any given time and for any set of parameters. In particular, the network equations have the same analytical form for any positive value of the metabolic constant ${\displaystyle \beta }$. To find a set of parameters that optimizes the overall performance, we minimized the loss function averaged over time and trials. We then optimized the parameters by setting the metabolic constant ${\displaystyle \beta }$ such that the encoding error weights 70% and the metabolic error weights 30% of the average loss, and by choosing all other parameters such as to minimize numerically the average loss (see Materials and methods). The numerical optimization was performed by simulating a model of 400 E and 100 I units, a network size relevant for computations within one layer of a cortical microcolumn (Lefort et al., 2009). The set of model parameters that optimized network efficiency is detailed in Table 1. Unless otherwise stated, we will use the optimal parameters of Table 1 in all simulations and only vary parameters detailed in the figure axes.

With optimally efficient parameters, population readouts closely tracked the target signals (Figure 1D, M=3, ${\displaystyle R^2=[0.95,0.97]}$ for E and I neurons, respectively). When stimulated by our three-dimensional time-varying feedforward input, the optimal E-I network provided a precise estimator of target signals (Figure 1E, top). The average estimation bias (${\displaystyle B^E}$ and ${\displaystyle B^I}$, see Materials and methods) of the network minimizing the encoding error was close to zero (${\displaystyle B^E}$ = 0.02 and ${\displaystyle B^I}$ = 0.03) while the bias of the network minimizing the average loss (and optimizing efficiency) was slightly larger and negative (${\displaystyle B^E}$= –0.15 and ${\displaystyle B^I}$= −0.34), but still small compared to the stimulus amplitude (Figure 1E, bottom, Figure 1—figure supplement 1F). Time- and trial-averaged encoding error (RMSE) and metabolic cost (MC, see Materials and methods) were comparable in magnitude (${\displaystyle RMSE=[3.5,2.4]}$, ${\displaystyle MC=[4.4,2.8]}$ for E and I), but with smaller error and lower cost in I, leading to a better performance in I (average loss of 2.5) compared to E neurons (average loss of 3.7). We report both the encoding error and the metabolic cost throughout the paper, so that readers can evaluate how these performance measures may generalize when weighting differently the error and the metabolic cost.

Next, we examined the emergent dynamical properties of an optimally efficient E-I network. I neurons had higher average firing rates compared to E neurons, consistently with observations in cortex (Neske et al., 2015). The distribution of firing rates was well described by a log-normal distribution (Figure 1F, left), consistent with distributions of cortical firing observed empirically (Buzsáki and Mizuseki, 2014). Neurons fired irregularly, with mean coefficient of variation (CV) slightly smaller than 1 (Figure 1F, right; CV = [0.97, 0.95] for E and I neurons, respectively), compatible with cortical firing (Softky and Koch, 1993). We assessed E-I balance in single neurons through two complementary measures. First, we calculated the average (global) balance of E-I currents by taking the time-average of the net sum of synaptic inputs (shortened to net synaptic input, see Ahmadian and Miller, 2021). Second, we computed the instantaneous (Okun and Lampl, 2008; also termed detailed in Vogels et al., 2011) E-I balance as the Pearson correlation (${\displaystyle \rho }$) over time of E and I currents received by each neuron (see Materials and methods).

We observed an excess inhibition in both E and I neurons, with a negative net synaptic input in both E and I cells (Figure 1H), indicating an inhibition-dominated network according to the criterion of average balance (Ahmadian and Miller, 2021). In E neurons, net synaptic current is the sum of the feedforward current and recurrent inhibition and the mean of the net current is close to the mean of the inhibitory current, because feedforward inputs have vanishing mean. Furthermore, we found a moderate instantaneous balance (Xue et al., 2014), stronger in I compared to E cell type (Figure 1G, I, ${\displaystyle \rho =[0.44,0.25]}$, for I and E neurons, respectively), similar to levels measured empirically in rat visual cortex (Tan et al., 2013).

We determined optimal model parameters by optimizing one parameter at a time. To independently validate the so obtained optimal parameter set (reported in Table 1), we varied all six model parameters explored in the paper with Monte-Carlo random joint sampling (10,000 random samples), uniformly within a biologically plausible parameter range for each parameter (Table 2). We did not find any parameter configuration with lower average loss than the setting in Table 1 (Figure 2A–B) when using the weighting of the encoding error with metabolic cost between ${\displaystyle 0.4<g_L<0.81}$ (Figure 2C). The three parameter settings that came the closest to our configuration on Table 1 had stronger noise but also stronger metabolic constant than our configuration (Table 3). The second, third and fourth configurations had longer time constants of both E and I single neurons. Ratios of E-I neuron numbers and of I-I to E-I connectivity in the second, third, and fourth best configuration were either jointly increased or decreased with respect to our optimal configuration. This suggests that joint covariations in parameters may influence the network’s optimality. While our finite Monte-Carlo random sampling does not fully prove the global optimality of the configuration in Table 1, it shows that it is highly efficient.

Figure 2

Download asset Open asset

We next explored coding properties emerging from recurrent synaptic interactions between E and I populations in the optimally efficient networks.

An approach that has recently provided empirical insight into local recurrent interactions is measuring effective connectivity with cellular resolution. Recent effective connectivity experiments photostimulated single E neurons in primary visual cortex and measured its effect on neighbouring neurons, finding that the photostimulation of an E neuron led to a decrease in firing rate of similarly tuned close-by neurons (Chettih and Harvey, 2019). This effective lateral inhibition (Lochmann et al., 2012) between E neurons with similar tuning to the stimulus implements competition between neurons for the representation of stimulus features (Chettih and Harvey, 2019). Since our model instantiates efficient coding by design and because we removed connections between neurons with different selectivity, we expected that our network implements lateral inhibition and would thus give comparable effective connectivity results in simulated photostimulation experiments.

To test this prediction, we simulated photostimulation experiments in our optimally efficient network. We first performed experiments in the absence of the feedforward input to ensure all effects are only due to the recurrent processing. We stimulated a randomly selected single target E neuron and measured the change in the instantaneous firing rate from the baseline firing rate, ${\displaystyle \mathrm{\Delta }{z}_i(t)}$, in all the other I and E neurons (Figure 3A, left). The photostimulation was modeled as an application of a constant depolarising current with a strength parameter, ${\displaystyle a_p}$, proportional to the distance between the resting potential and the firing threshold (${\displaystyle a_p=0}$ means no stimulation, while ${\displaystyle a_p=1}$ indicates photostimulation at the firing threshold). We quantified the effect of the simulated photostimulation of a target E neuron on other E and I neurons, distinguishing neurons with either similar or different tuning with respect to the target neuron (Figure 3A, right; Figure 3—figure supplement 1A–D).

Figure 3 with 1 supplement see all

Download asset Open asset

The photostimulation of the target E neuron increased the instantaneous firing rate of similarly-tuned I neurons and reduced that of other similarly-tuned E neurons (Figure 3B). We quantified the effective connectivity as the difference between the time-averaged firing rate of the recorded cell in presence or absence of the photostimulation of the targeted cell, measured during perturbation and up to 50 ms after. We found positive effective connectivity on I and negative effective connectivity on E neurons with similar tuning to the target neuron, with a positive correlation between tuning similarity and effective connectivity on I neurons and a negative correlation on E neurons (Figure 3C). We confirmed these effects of photostimulation in presence of a weak feedforward input (Figure 3—figure supplement 1E), similar to the experiments of Chettih and Harvey, 2019 in which photostimulation was applied during the presentation of visual stimuli with weak contrast. Thus, the optimal network replicates the preponderance of negative effective connectivity between E neurons and the dependence of its strength on tuning similarity found in Chettih and Harvey, 2019.

In summary, lateral excitation of I neurons and lateral inhibition of E neurons with similar tuning is an emerging coding property of the efficient E-I network, which recapitulates competition between neurons with similar stimulus tuning found in visual cortex (Chettih and Harvey, 2019; Oldenburg et al., 2024). An intuition of why this competition implements efficient coding is that the E neuron that fires first activates I neurons with similar tuning. In turn, these I neurons inhibit all similarly tuned E neurons (Figure 3A, right), preventing them to generate redundant spikes to encode the sensory information that has already been encoded by the first spike. Suppression of redundant spiking reduces metabolic cost without reducing encoded information (Boerlin et al., 2013; Koren and Denève, 2017).

While perturbing the activity of E neurons in our model qualitatively reproduces empirically observed lateral inhibition among E neurons (Chettih and Harvey, 2019; Oldenburg et al., 2024), these experiments have also reported positive effective connectivity between E neurons with very similar stimulus tuning. Our intuition is that our simple model cannot reproduce this finding because it lacks E-E connectivity.

To explore further the consequences of E-I interactions for stimulus encoding, we next investigated the dynamics of lateral inhibition in the optimal network driven by the feedforward sensory input but without perturbing the neural activity. Previous work has established that efficient spiking neurons may present strong correlations in the membrane potentials, but only weak correlations in the spiking output, because redundant spikes are prevented by lateral inhibition (Boerlin et al., 2013; Deneve and Machens, 2016b). We investigated voltage correlations in pairs of neurons within our network as a function of their tuning similarity. Because the feedforward inputs are shared across E neurons and weighted by their tuning parameters, they cause strong positive voltage correlations between E-E neuronal pairs with very similar tuning and strong negative correlations between pairs with very different (opposite) tuning (Figure 3D, top-left). Voltage correlations between E-E pairs vanished regardless of tuning similarity when we made the feedforward inputs independent across neurons (Figure 3D, top-middle), showing that the dependence of voltage correlations on tuning similarity occurs because of shared feedforward inputs. In contrast to E neurons, I neurons do not receive feedforward inputs and are driven only by similarly tuned E neurons (Figure 3A, right). This causes positive voltage correlations in I-I neuronal pairs with similar tuning and vanishing correlations in neurons with different tuning (Figure 3D, bottom-left). Such dependence of voltage correlations on tuning similarity disappears when removing the structure from the E-I synaptic connectivity (Figure 3D, bottom-right).

In contrast to voltage correlations, and as expected by previous studies (Boerlin et al., 2013; Deneve and Machens, 2016b), the coordination of spike timing of pairs of E neurons (measured with cross-correlograms or CCGs) was very weak (Figure 3E). For I-I and E-I neuronal pairs, the peaks of CCGs were stronger than those observed in E-E pairs, but they were present only at very short lags (lags < 1 ms). This confirms that recurrent interactions of the efficient E-I network wipe away the effect of membrane potential correlations at the spiking output level, and shows information processing with millisecond precision in these networks (Boerlin et al., 2013; Deneve and Machens, 2016b; Koren and Denève, 2017).

The analytical solution of the optimally efficient E-I network predicts that recurrent synaptic weights are proportional to the tuning similarity between neurons. We next investigated the role of such connectivity structure by comparing the behavior of an efficient network with an unstructured E-I network, similar to the type studied in previous works (Brunel, 2000; Renart et al., 2010; Mazzoni et al., 2008). We removed the connectivity structure by randomly permuting synaptic weights across neuronal pairs (see Materials and methods). Such shuffling destroys the relationship between tuning similarity and synaptic strength (as shown in Figure 1C(ii)) while it preserves Dale’s law and the overall distribution of connectivity weights.

We found that shuffling the connectivity structure significantly altered the efficiency of the network (Figure 4A, B), neural dynamics (Figure 4C, D, F–H) and lateral inhibition (Figure 4I). In particular, structured networks differ from unstructured ones by showing better encoding performance (Figure 4A), lower metabolic cost (Figure 4B), weaker variance of the membrane potential over time (Figure 4C), lower firing rates (Figure 4D) and weaker average (Figure 4F) and instantaneous balance (Figure 4G) of synaptic inputs. However, we found only a small difference in the variability of spiking between structured and unstructured networks (Figure 4E). While these results are difficult to test experimentally due to the difficulty of manipulating synaptic connectivity structures in vivo, they highlight the importance of the connectivity structure for cortical computations.

Figure 4 with 1 supplement see all

Download asset Open asset

We also compared structured and unstructured networks about their relation between pairwise voltage correlations and tuning similarity, by randomizing connections within a single connectivity type (E-I, I-I or I-E) or within all these three connectivity types at once (‘all’). We found the structure of E-I connectivity to be crucial for the linear relation between voltage correlations and tuning similarity in pairs of I neurons (Figure 4H, magenta).

Finally, we analyzed how the structure in recurrent connectivity influences lateral inhibition that we observed in efficient networks. We found that the dependence of lateral inhibition on tuning similarity vanishes when the connectivity structure is fully removed (Figure 4I, ‘all’ on the right plot), thus showing that connectivity structure is necessary for lateral inhibition. While networks with unstructured E-I and I-E connectivity still show inhibition in E neurons upon single neuron photostimulation (because of the net inhibitory effect of recurrent connectivity; Figure 4—figure supplement 1F), this inhibition was largely unspecific to tuning similarity (Figure 4I, ‘E-I’ and ‘I-E’). Unstructured connectivity decreased the correlation between tuning similarity and effective connectivity from ${\displaystyle r=[0.31,-0.54]}$ in I and E neurons in a structured network to ${\displaystyle r=[0.02,-0.13]}$ and ${\displaystyle r=[0.57,0.11]}$ in networks with unstructured E-I and I-E connectivity, respectively. Removing the structure in I-I connectivity, in contrast, increased the correlation between effective connectivity and tuning similarity in E neurons (${\displaystyle r=[0.30,-0.65]}$, Figure 4I, second from the left), showing that lateral inhibition takes place irrespectively of the I-I connectivity structure.

Previous empirical (Znamenskiy et al., 2024) and theoretical work has established the necessity of strong E-I-E synaptic connectivity for lateral inhibition (Sadeh and Clopath, 2020; Mackwood et al., 2021). To refine this understanding, we asked what is the minimal connectivity structure necessary to qualitatively replicate empirically observed lateral inhibition. We did so by considering a simpler connectivity rule than the one obtained from first principles. We assumed neurons to be connected (with random synaptic efficacy) if their tuning vectors are similar (${\displaystyle J_{ij}^{xy}>0}$ if ${\displaystyle {\varphi }_{ij}^{xy}>0}$) and unconnected otherwise (${\displaystyle J_{ij}^{xy}=0}$ if ${\displaystyle {\varphi }_{ij}^{xy}\le 0}$), relaxing the precise proportionality relationship between tuning similarity and synaptic weights (as on Figure 1C(ii)). We found that networks with such simpler connectivity respond to activity perturbation in a qualitatively similar way as the optimal network (Figure 3—figure supplement 1F) and still replicate experimentally observed activity profiles in Chettih and Harvey, 2019.

While optimally structured connectivity predicted by efficient coding is biologically plausible, it may be difficult to realise it exactly on a synapse-by-synapse basis in biological networks. Following Calaim et al., 2022, we verified the robustness of the model to small deviations from the optimal synaptic weights by adding a random jitter, proportional to the synaptic strength, to all synaptic connections (see Materials and methods). The encoding performance and neural activity were barely affected by weak and moderate levels of such perturbation (Figure 4—figure supplement 1G, H), demonstrating that the network is robust against random jittering of the optimal synaptic weights.

In summary, we found that some aspects of recurrent connectivity structure, such as the like-to-like organization, are crucial to achieve efficient coding. Instead, for other aspects there is considerable flexibility; the proportionality between tuning similarity and synaptic weights is not crucial for efficiency and small random jitter of optimal weights has only minor effects. Structured E-I and I-E, but not I-I connectivity, is necessary for implementing experimentally observed pattern of lateral inhibition whose strength is modulated by tuning similarity.

We next investigated the role of within-neuron feedback triggered by each spike, ${\displaystyle I_i^{\text{ad},y}}$, that emerges from the optimally efficient solution (Equation 5). A previous study (Gutierrez and Denève, 2019) showed that spike-triggered adaptation, together with structured connectivity, redistributes the activity from highly excitable neurons to less excitable neurons, leaving the population readout invariant. Here, we address model efficiency in presence of adapting or facilitating feedback as well as differential effects of adaptation in E and I neurons.

The spike-triggered within-neuron feedback ${\displaystyle I_i^{\text{ad},y}}$ has a time constant equal to that of the single neuron readout ${\displaystyle {\tau }_r^E}$ (E neurons) and ${\displaystyle {\tau }_r^I}$ (I neurons). The strength of the current is proportional to the difference in inverse time constants of single neuron and population readouts, ${\displaystyle 1/\tau -1/{\tau }_r^y}$. This spike-triggered current is negative, giving spike-triggered adaptation (Mensi et al., 2012), if the single-neuron readout has longer time constant than the population readout (${\displaystyle {\tau }_r^y>\tau }$), or positive, giving spike-triggered facilitation, if the opposite is true (${\displaystyle {\tau }_r^y<\tau }$; Table 4). We expected that network efficiency would benefit from spike-triggered adaptation, because accurate encoding requires fast temporal dynamics of the population readouts, to capture fast fluctuations in the target signal, while we expect a slower dynamics in the readout of single neuron’s firing frequency, ${\displaystyle {\displaystyle r_i^y(t)}}$, a process that could be related to homeostatic regulation of single neuron’s firing rate (Abbott and Nelson, 2000; Turrigiano and Nelson, 2004). In our optimal E-I network we indeed found that optimal coding efficiency is achieved in absence of within-neuron feedback or with weak adaptation in both cell types (Figure 5A). The optimal set of time constants ${\displaystyle [{\tau }_r^E,{\tau }_r^I]}$ only weakly depended on the weighting of the encoding error with the metabolic cost ${\displaystyle g_L}$ (Figure 7—figure supplement 1A). We note that adaptation in E neurons promotes efficient coding because it enforces every spike to be error-correcting, while a spike-triggered facilitation in E neurons would lead to additional spikes that might be redundant and reduce network efficiency. Contrary to previously proposed models of adaptation in LIF neurons (Brette and Gerstner, 2005; Schwalger and Lindner, 2013), the strength and the time constant of adaptation in our model are not independent, but they both depend on ${\displaystyle {\displaystyle {\tau }_r^y}}$, with larger ${\displaystyle {\displaystyle {\tau }_r^y}}$ yielding both longer and stronger adaptation.

Figure 5

Download asset Open asset

To gain insights on the differential effect of adaptation in E vs I neurons, we set the adaptation in one cell type to 0 and varied the strength of adaptation in the other cell type by varying the time constant of the single neuron readout. With adaptation in E neurons (and no adaptation in I), we observed a slow increase of the encoding error in E neurons, while the encoding error increased faster with adaptation in I neurons (Figure 5B). Similarly, network efficiency increased slowly with adaptation in E and faster with adaptation in I neurons (Figure 5C), thus showing that adaptation in E neurons decreases less the performance compared to the adaptation in I neurons. With increasing adaptation in E neurons, the firing rate in E neurons decreased (Figure 5D), leading to E estimates with smaller amplitude. Because E estimates are target signals for I neurons and because weaker E signals imply weaker drive to I neurons, average loss of the I population decreased by increasing adaptation in E neurons (Figure 5C top, blue trace).

Firing rates and variability of spiking were sensitive to the strength of adaptation. As expected, adaptation in E neurons caused a decrease in the firing levels in both cell types (Figure 5D, E). In contrast, adaptation in I neurons decreased the firing rate in I neurons, but increased the firing rate in E neurons, due to a decrease in the level of inhibition. Furthermore, adaptation decreased the variability of spiking, in particular in the cell type with strong adaptation (Figure 5F), a well-known effect of spike-triggered adaptation in single neurons (Schwalger and Lindner, 2013).

In regimes with adaptation, time constants of single neuron readout ${\displaystyle {\tau }_r^y}$ influenced the average balance (Figure 5G) as well as the instantaneous balance (Figure 5H) in E and I cell type. To gain a better understanding of the relationship between adaptation, E-I interactions and network optimality, we measured the instantaneous and time-averaged E-I balance while varying the adaptation parameters and studied their relation with the loss. By increasing adaptation in E neurons, the average imbalance got weaker in E neurons (Figure 5G, left), but stronger in I neurons (Figure 5G, right). Regimes with precise average balance in both cell types were suboptimal (compare Figure 5A, right and G), while regimes with precise instantaneous balance were highly efficient (compare Figure 5A, right and H).

To test how well the average balance and the instantaneous balance of synaptic inputs predict network efficiency, we concatenated the column-vectors of the measured average loss and of the average imbalance in each cell type and computed the Pearson correlation between these quantities. The correlation between the average imbalance and the average loss was weak in the E cell type (${\displaystyle r=0.16}$) and close to zero in the I cell type (${\displaystyle r=0.02}$), suggesting almost no relation between efficiency and average imbalance. In contrast, the average loss was negatively correlated with the instantaneous balance in both E (${\displaystyle r=-0.35}$) and in I cell type (${\displaystyle r=-0.45}$), showing that instantaneous balance of synaptic inputs is positively correlated with network efficiency. When measured for varying levels of spike-triggered adaptation, unlike the average balance of synaptic inputs, the instantaneous balance is thus mildly predictive of network efficiency.

In sum, our results show that the optimally efficient solution does not include within-neuron feedback, while a model with weak and short-lasting spike-triggered adaptation is slightly suboptimal, although still highly efficient. Our results predict that information coding would be more efficient with adaptation than with facilitation. Assuming that our I neurons describe parvalbumin-positive interneurons, our results suggest that the weaker adaptation in I compared to E neurons, reported empirically (Pala and Petersen, 2015), may be beneficial for the network’s encoding efficiency.

Spike-triggered adaptation in our model captures adaptive processes in single neurons that occur on time scales lasting from a couple of milliseconds to tens of milliseconds after each spike. However, spiking in biological neurons triggers adaptation on multiple time scales, including much slower time scales on the order of seconds or tens of seconds (Pozzorini et al., 2013). Our model does not capture adaptive processes on these longer time scales (but see Gutierrez and Denève, 2019).

In our derivation of the optimal network, we obtained a non-specific external current (in the following, non-specific current) ${\displaystyle {\displaystyle I_i^{\text{ext},y}(t)}}$. Non-specific current captures all synaptic currents that are unrelated and unspecific to the stimulus features. This non-specific term collates effects of synaptic currents from neurons untuned to the stimulus (Levy et al., 2020; Zylberberg, 2018), as well as synaptic currents from other brain areas. It can be conceptualized as the background synaptic activity that provides a large fraction of all synaptic inputs to both E and I neurons in cortical networks (Destexhe and Paré, 1999), and which may modulate feedforward-driven responses by controlling the distance between the membrane potential and the firing threshold (Destexhe et al., 2003). Likewise, in our model, the non-specific current does not directly convey information about the feedforward input features, but influences the network dynamics.

Non-specific current comprises mean and fluctuations (see Materials and methods). The mean is proportional to the metabolic constant ${\displaystyle \beta }$ and its fluctuations reflect the noise that we included in the condition for spiking. Since ${\displaystyle \beta }$ governs the trade-off between encoding error and metabolic cost (Equation 3), higher values of ${\displaystyle \beta }$ imply that more importance is assigned to the metabolic efficiency than to coding accuracy, yielding a reduction in firing rates. In the expression for the non-specific current, we found that the mean of the current is negatively proportional to the metabolic constant ${\displaystyle \beta }$ (see Materials and methods). Because the non-specific current is typically depolarizing, this means that increasing ${\displaystyle \beta }$ yields a weaker non-specific current and increases the distance between the mean membrane potential and the firing threshold. Thus, an increase of the metabolic constant is expected to make the network less responsive to the feedforward signal.

We found the metabolic constant ${\displaystyle \beta }$ to significantly influence the spiking dynamics (Figure 6A). The optimal efficiency was achieved for non-zero levels of the metabolic constant (Figure 6B), with the mean of the non-specific current spanning more than half of the distance between the resting potential and the threshold (Table 1). Stronger weighting of the loss of I compared to E neurons and stronger weighting of the error compared to the cost yielded weaker optimal metabolic constant (Figure 7—figure supplement 1B). Metabolic constant modulated the firing rate as expected, with the firing rate in E and I neurons decreasing with the increasing of the metabolic constant (Figure 6C, top). It also modulated the variability of spiking, as increasing the metabolic constant decreased the variability of spiking in both cell types (Figure 6C, bottom). Furthermore, it modulated the average balance and the instantaneous balance in opposite ways: larger values of ${\displaystyle \beta }$ led to regimes that had stronger average balance, but weaker instantaneous balance (Figure 6D). We note that, even with suboptimal values of the metabolic constant, the neural dynamics remained within biologically relevant ranges.

Figure 6

Download asset Open asset

The fluctuation part of the non-specific current, modulated by the noise strength ${\displaystyle \sigma }$ that we added in the definition of spiking rule for biological plausibility (see Materials and methods), strongly affected the neural dynamics as well (Figure 6E). The optimal performance was achieved with non-vanishing noise levels (Figure 6F), similarly to previous work showing that the noise prevents excessive network synchronization that would harm performance (Chalk et al., 2016; Koren and Denève, 2017; Timcheck et al., 2022). The optimal noise strength depended on the weighting of the error with the cost, with strong weighting of the error predicting stronger noise (Figure 7—figure supplement 1C).

The average firing rate of both cell types, as well as the variability of spiking in E neurons, increased with noise strength (Figure 6G), and some level of noise in the non-specific inputs was necessary to establish the optimal level of spiking variability. Nevertheless, we measured significant levels of spiking variability already in the absence of noise, with a coefficient of variation of about 0.8 in E and 0.9 in I neurons (Figure 6G, bottom). This indicates that the recurrent network dynamics generates substantial variability even in absence of an external source of noise. The average and instantaneous balance of synaptic currents exhibited a non-linear behavior as a function of noise strength (Figure 6H). Due to decorrelation of membrane potentials by the noise, instantaneous balance in I neurons decreased with increasing noise strength (Figure 6H, bottom).

Next, we investigated the joint impact of the metabolic constant and the noise strength on network optimality. We expect these two parameters to be related, because larger noise strength requires stronger metabolic constant to prevent the activity of the network to be dominated by noise. We thus performed a two-dimensional parameter search (Figure 6I). As expected, the optima of the metabolic constant and the noise strength were positively correlated. A weaker noise required lower metabolic constant, and-vice-versa. While achieving maximal efficiency at non-zero levels of the metabolic cost and noise (see Figure 6I) might seem counterintuitive, we speculate that such setting is optimal because some noise in the non-specific current prevents over-synchronization and over-regularity of firing that would harm efficiency, similarly to what was shown in previous works (Chalk et al., 2016; Koren and Denève, 2017; Timcheck et al., 2022). In the presence of noise, a non-zero metabolic constant is needed to suppress inefficient spikes purely induced by noise that do not contribute to coding and increase the error. This gives rise to a form of stochastic resonance, where an optimal level of noise is helpful to detect the signal coming from the feedforward currents.

In summary, non-specific external currents derived in our optimal solution have a major effect on coding efficiency and on neural dynamics. In qualitative agreement with empirical measurements (Destexhe and Paré, 1999; Destexhe et al., 2003), our model predicts that more than half of the average distance between the resting potential and firing threshold is accounted for by non-specific synaptic currents. Similarly to previous theoretical work (Chalk et al., 2016; Koren and Denève, 2017), we find that some level of external noise, in the form of a random fluctuation of the non-specific synaptic current, is beneficial for network efficiency. This remains a prediction for experiments.

Next, we investigated how coding efficiency and neural dynamics depend on the ratio of the number of E and I neurons (${\displaystyle {\displaystyle N^E:N^I}}$ or E-I ratio) and on the relative synaptic strengths between E-I and I-I connections.

Efficiency objectives (Equation 3) are based on population, rather than single-neuron activity. Our efficient E-I network thus realizes a computation of the target representation that is distributed across multiple neurons (Figure 7A). Following previous reports (Barrett et al., 2016), we predict that, if the number of neurons within the population decreases, neurons have to fire more spikes to achieve an optimal population readout because the task of tracking the target signal is distributed among fewer neurons. To test this prediction, we varied the number of I neurons while keeping the number of E neurons constant. As predicted, a decrease of the number of I neurons (and thus an increase in the ratio of the number of E to I neurons) caused a linear increase in the firing rate of I neurons, while the firing rate of E neurons stayed constant (Figure 7B, top). However, the variability of spiking and the average synaptic inputs remained relatively constant in both cell types as we varied the E-I ratio (Figure 7B, bottom, C), indicating a compensation for the change in the ratio of E-I neuron numbers through adjustment in the firing rates. These results are consistent with the observation in neuronal cultures of a linear change in the rate of postsynaptic events but unchanged postsynaptic current in either E and I neurons for variations in the E-I neuron number ratio (Sukenik et al., 2021).

Figure 7 with 1 supplement see all

Download asset Open asset

The ratio of the number of E to I neurons had a significant influence on coding efficiency. We found a unique minimum of the encoding error of each cell type, while the metabolic cost increased linearly with the ratio of the number of E and I neurons (Figure 7D). Using the usual weighting ${\displaystyle g_L=0.7}$, we found the optimal ratio of E to I neuron numbers to be in range observed experimentally in cortical circuits (Figure 7D, bottom, black arrow, ${\displaystyle {\displaystyle N^E:N^I=3.75:1}}$; Markram et al., 2004). The optimal ratio depended on the weighting of the error with the cost, decreasing when increasing the cost of firing (Figure 7E, bottom). Also the encoding error (RMSE) alone, without considering the metabolic cost, predicted optimal ratio of the number of E to I neurons within a plausible physiological range, ${\displaystyle {\displaystyle N^E:N^I=[3.75:1,5.25:1]}}$, with stronger weightings of the encoding error by I neurons predicting higher ratios (Figure 7E, top).

Next, we investigated the impact of the strength of E and I synaptic efficacy (EPSPs and IPSPs). As evident from the expression for the population readouts (Equation 2), the magnitude of tuning parameters (which are also decoding weights) determines the amplitude of jumps of the population readout caused by spikes (Figure 7F). The larger these weights are, the larger is the impact of spikes on the population signals.

E and I synaptic efficacies depend on the tuning parameters. We parametrized the distribution of tuning parameters as uniform distributions centered at zero, but allowed the spread of distributions in E and I neurons (${\displaystyle {\sigma }_w^E}$ and ${\displaystyle {\sigma }_w^I}$) to vary across E and I cell type (Materials and methods). In the optimally efficient network, as found analytically (Materials and methods section ‘Dynamic equations for the membrane potentials’), the E-I connectivity is the transpose of the of the I-E connectivity, which implies that these connectivities are exactly balanced and have the same mean. We also showed analytically that by parametrizing tuning parameters with uniform distributions, the scaling of synaptic connectivity of E-I (equal to I-E) and I-I connectivity is controlled by the variance of tuning parameters of the pre and postsynaptic population as follows: ${\displaystyle {\displaystyle ⟨J^{xy}⟩\propto {\sigma }_w^x{\sigma }_w^y}}$. Using these insights, we were able to analytically evaluate the mean E-I and I-I synaptic efficacy (see Materials and methods section ‘Parametrization of synaptic connectivity’).

We next searched for the optimal ratio of the mean I-I to E-I efficacy as the parameter that maximizes network efficiency. Network efficiency was maximized when such ratio was about 3–1 (Figure 7G). Our results suggest the optimal E-I and I-E synaptic efficacy, averaged across neuronal pairs, of 0.75 mV, and the optimal I-I efficacy of 2.25 mV, values that are consistent with empirical measurements in the primary sensory cortex (Cossell et al., 2015; Pala and Petersen, 2015; Campagnola et al., 2022). The optimal ratio of mean I-I to E-I connectivity decreased when the error was weighted more with respect to the metabolic cost (Figure 7—figure supplement 1D).

Similarly to the ratio of E-I neuron numbers, a change in the ratio of mean E-I to I-E synaptic efficacy was compensated for by a change in firing rates, with stronger I-I synapses leading to a decrease in the firing rate of I neurons (Figure 7H, top). Conversely, weakening the E-I (and I-E) synapses resulted in an increase in the firing rate in E neurons (Figure 7—figure supplement 1E, F). This is easily understood by considering that weakening the E-I and I-E synapses activates less strongly the lateral inhibition in E neurons (Figure 3) and thus leads to an increase in the firing rate of E neurons. We also found that single neuron variability remained almost unchanged when varying the ratio of mean I-I to E-I efficacy (Figure 7H, bottom) and the optimal ratio yielded optimal levels of average and instantaneous balance of synaptic inputs, as found previously (Figure 7I). The instantaneous balance monotonically decreased with increasing ratio of I-I to E-I efficacy (Figure 7I, bottom, Figure 7—figure supplement 1G).

Further, we tested the co-dependency of network optimality on the above two ratios with a 2-dimensional parameter search. We expected a positive correlation of network performance as a function of these two parameters, because both of them regulate the level of instantaneous E-I balance in the network. We found that the lower ratio of E-I neuron numbers indeed predicts a lower ratio of the mean I-I to E-I connectivity (Figure 7J). This is because fewer E neurons bring less excitation in the network, thus requiring less inhibition to achieve optimal levels of instantaneous balance. The co-dependency of the two parameters in affecting network optimality might be informative as to why E-I neuron number ratios may vary across species (for example, it is reported to be 2:1 in human cortex [Fang et al., 2022] and 4:1 in mouse cortex). Our model predicts that lower E-I neuron number ratios require weaker mean I-I to E-I connectivity.

In summary, our analysis suggests that optimal coding efficiency is achieved with more E neurons than I neurons and with mean I-I synaptic efficacy stronger than the E-I and I-E efficacy, and that these two parameters are positively correlated. Optimal ratios of E to I neurons and of connection strengths are broadly consistent with empirical measurements of these parameters in biological networks. The optimal network has less I than E neurons, but the impact of spikes of I neurons on the population readout is stronger, also suggesting that spikes of I neurons convey more information.

We further investigated how the network’s behavior depends on the timescales of the input stimulus features. We manipulated the stimulus timescales by changing the time constants of ${\displaystyle M=3}$ OU processes. The network efficiently encoded stimulus features when their time constants varied between 1 and 200 ms, with stable encoding error, metabolic cost (Figure 8A) and neural dynamics (Figure 8—figure supplement 1A, B). To examine if the network can efficiently encode also stimuli that evolve on different timescales, we tested its performance in response to ${\displaystyle M=3}$ input variables, each with a different timescale. We kept the timescale of the first variable constant at ${\displaystyle {\tau }_1^s=10}$ ms, while we varied the time constants of the other two keeping the time constant of the third twice as long as that of the second. We found excellent performance of the network in response to such stimuli that was stable across timescales (Figure 8B). The prediction that the network can encode information effectively over a wide range of time scales can be tested experimentally, by measuring the sensory information encoded by the activity of a set of neurons while varying the sensory stimulus timescales over a wide range.

Figure 8 with 1 supplement see all

Download asset Open asset

We next examined network performance while varying the timescale of targets ${\displaystyle {\tau }_x}$ (see Equation 1). Because we assumed that the target time constants equal the membrane time constant of E and I neurons (${\displaystyle {\tau }_x={\tau }^E={\tau }^I=\tau }$), it is not surprising that the best performance was achieved when these time constants were similar (Figure 8—figure supplement 1C). Firing rates, firing variability and the average and instantaneous balance did not change appreciably with this time constant (Figure 8—figure supplement 1D–E).

Next, we tested how the network’s behavior changed when we varied the number of stimulus features ${\displaystyle M}$. Because all other parameters were optimized using ${\displaystyle M=3}$, the encoding error of E (RMSE^E) and I neurons (RMSE^I) achieved a minimum around this value (Figure 8C, top). The metabolic cost increased monotonically with ${\displaystyle M}$ (Figure 8C, bottom). The number of features that optimized network efficiency (and minimized the average loss) depended on ${\displaystyle g_L}$, with stronger penalty of firing yielding a smaller optimal number of features. Increasing ${\displaystyle M}$ beyond the optimal number resulted in a gentle monotonic increase in firing rates for both E and I neurons, and it increased the average E-I balance and weakened the instantaneous balance (Figure 8—figure supplement 1F, G).

We next characterized the tuning and the stimulus selectivity of E and I neurons. E neurons receive a feedforward current, which is expected to make them stimulus-selective, while I neurons receive synaptic inputs from E neurons through dense E-I connectivity. We measured stimulus tuning by computing tuning curves for each neuron in response to M=3 constant stimulus features (see Materials and methods). Similarly to previous work (Barrett et al., 2016), tuning curves of both E and I neurons were strongly heterogeneous (Figure 8E). We tested if the selectivity differs across E and I cell types. We computed a selectivity index for each neuron as the stimulus-response gain (average change in the firing rate in response to a small change in the stimulus divided by the stimulus change size, see Materials and methods), and found that E and I neurons had similar mean stimulus selectivity (${\displaystyle p=0.418}$, two-tailed t-test; Figure 8F). Thus, I neurons, despite not receiving direct feedforward inputs and acquiring stimulus selectivity only through structured E-I connections, are tuned to the input stimuli as strongly as the E neurons.

Neurons in the brain are either excitatory or inhibitory. To understand how differentiating E and I neurons benefits efficient coding, we compared the properties of our efficient E-I network with an efficient network with a single cell type (1CT). The 1CT model can be seen as a simplification of the E-I model (see Appendix 1) and has been derived and analyzed in previous studies (Bourdoukan et al., 2012; Boerlin et al., 2013; Koren and Denève, 2017; Gutierrez and Denève, 2019; Alemi et al., 2018; Brendel et al., 2020). We compared the average encoding error (RMSE), the average metabolic cost (MC), and the average loss (see Appendix 3) of the E-I model against the one cell type (1CT) model. Compared to the 1CT model, the E-I model exhibited a higher encoding error and metabolic cost in the E population, but a lower encoding error and metabolic cost in the I population (Figure 8G). The 1CT model can perform similar computations as the E-I network. Instead of an E neuron directly providing lateral inhibition to its neighbor (Figure 1—figure supplement 1A–C), it goes through an interneuron in the E-I model (Figure 1A(i) and B). Because the E population of the E-I model and the 1CT model perform a similar computation, we compared the efficiency of the E population of the E-I model with the 1CT model. We found that the 1CT model is slightly more efficient than the E population of the E-I model, consistently for different weightings of the error with the cost (Figure 8H).

We further compared the robustness of firing rates to changes in the metabolic constant of the two models. Consistently with previous studies (Koren and Denève, 2017; Rullán Buxó and Pillow, 2020), firing rates in the 1CT model were highly sensitive to variations in the metabolic constant (Figure 8I, note the logarithmic scale on the y-axis), with a superexponential growth of the firing rate with the inverse of the metabolic constant in regimes with metabolic cost lower than optimal. This is in contrast to the E-I model, whose firing rates exhibited lower sensitivity to the metabolic constant, and never exceeded physiological limits (Figure 7C, top). Because our E-I model does not incorporate a saturating input-output function that constrains the range of firing as in Kadmon et al., 2020, the ability of the E-I model to maintain firing rates within biologically plausible limits emerges as a highly desirable dynamic property. One reason for higher stability of our E-I model compared to the 1CT model is that the delay of the lateral inhibition in the E-I model is twice that of the 1CT model (because in the E-I model, the lateral inhibition travels through an additional synapse). A second reason is that the recurrent connectivity of the 1CT model has exactly the same amount of average excitation and inhibition, while the E-I model is inhibition-dominated, which makes the E-I model more stable.

In summary, although the optimal E-I model is slightly less efficient than the optimal 1CT model, it does not enter into states of physiologically unrealistic firing rates when the metabolic constant is lower than the optimal one.

In the following, we present a detailed derivation of the E-I spiking network implementing the efficient coding principle. The analytical derivation is based on previous works on efficient coding with spikes (Boerlin et al., 2013; Koren and Denève, 2017), and in particular on our recent work (Koren and Panzeri, 2022). While these previous works analytically derived feedforward and recurrent transmembrane currents in leaky integrate-and fire neuron models, they did not contain any synaptic currents unrelated to feedforward and recurrent processing. Non-specific synaptic currents were suggested to be important for an accurate description of coding and dynamics in cortical networks (Destexhe et al., 2003). In the model derivation that follows, we also derived non-specific external current from efficiency objectives.

Moreover, we here revisited the derivation of physical units in efficient spiking networks. We built on a previous work Boerlin et al., 2013 that assigned physical units to mathematical expressions that correspond to membrane potentials, firing thresholds, etc. Here, we instead assigned physical units to the computational variables such as the target signals and the population readouts, and then derived units of the membrane potentials and firing thresholds.

With this model, we aim to describe neural dynamics and computation in early sensory cortices such as the primary visual cortex in rodents, even though many principles of the model developed here could be relevant throughout the brain.

We consider two types of neurons, excitatory neurons ${\displaystyle E}$ and inhibitory neurons ${\displaystyle I}$. We denote as ${\displaystyle N^E}$ and ${\displaystyle {\displaystyle N^I}}$ the number of ${\displaystyle E}$-cells and ${\displaystyle I}$-cells, respectively. The spike train of neuron ${\displaystyle i}$ of type ${\displaystyle y\in \{E,I\}}$, ${\displaystyle i=1,2,...,N^y}$, is defined as a sum of Dirac delta functions,

where ${\displaystyle t_i^{y,\alpha }}$ is the time of the ${\displaystyle \alpha }$-th spike of that neuron, defined as a time point at which the membrane potential of neuron ${\displaystyle i}$ crosses the firing threshold.

We define the readout of the spiking activity of neuron ${\displaystyle i}$ of type ${\displaystyle y}$ (in the following, ‘ingle neuron readout’) as a leaky integration of its spike train,

with ${\displaystyle {\lambda }_r^y}$ denoting the inverse time constant. This way, the quantity ${\displaystyle z_i^y(t)={\lambda }_r^y{r}_i^y(t)}$ represents an estimate of the instantaneous firing rate of neuron ${\displaystyle i}$.

We denote as ${\displaystyle \overrightarrow{s}(t):=[s_1(t),\dots ,s_M(t){]}^{\mathrm{\top }}}$ the set of ${\displaystyle M}$ dynamical features of the external stimulus (in the following, stimulus features) which are transmitted to the network through a feedforward sensory pathway. The stimulus features have the unit of the square root of millivolt, (mV)^1/2. The target signal is then obtained through a leaky integration of the feedforward variable, ${\displaystyle \overrightarrow{s}(t)}$ (Bourdoukan et al., 2012), with inverse time constant ${\displaystyle \lambda }$, as

with ${\displaystyle \overrightarrow{x}(t):=[x_1(t),\dots ,x_M(t){]}^{\mathrm{\top }}}$ the vector of ${\displaystyle M}$ target signals. Furthermore, we define a linear population readout of the spiking activity of E and I neurons

with ${\displaystyle {\overrightarrow{\hat{x}}}^y(t):=[{\hat{x}}_1^y(t),\dots ,{\hat{x}}_M^y(t){]}^{\mathrm{\top }}}$ the vector of estimates of cell type ${\displaystyle y}$ and ${\displaystyle {\overrightarrow{w}}_i^y}$ in units of (mV)^1/2. Here, each neuron ${\displaystyle i}$ of type ${\displaystyle y}$ is associated with a vector ${\displaystyle {\overrightarrow{w}}_i^y:=[w_{1i}^y,\dots ,w_{Mi}^y{]}^{\mathrm{\top }}}$ of ${\displaystyle M}$ tuning parameters representing the decoding weight of neuron ${\displaystyle i}$ with respect to the ${\displaystyle M}$ population readouts in Equation 9. These decoding vectors can be combined in the ${\displaystyle M\times N^y}$ matrix ${\displaystyle W^y=[w_{ki}^y]}$. The rows of this matrix define the patterns of decoding weights ${\displaystyle {\overrightarrow{\tilde{w}}}_k^y:=[w_{k1}^y,\dots ,w_{k{N}^y}^y{]}^{\mathrm{\top }}}$ for each signal dimension ${\displaystyle k=1,\dots ,M}$.

We assume that the activity of a population ${\displaystyle y\in \{E,I\}}$ is set so as to minimize a time-dependent encoding error and a time-dependent metabolic cost:

with ${\displaystyle {\displaystyle {\beta }^y>0}}$ in units of mV the Lagrange multiplier which controls the weight of the metabolic cost relative to the encoding error. The time-dependent encoding error is defined as the squared distance between the targets and their estimates, and the role of estimates is assigned to the population readouts ${\displaystyle {\displaystyle {\overrightarrow{\hat{x}}}^y(t)}}$. In E neurons, the targets are defined as the target signals ${\displaystyle \overrightarrow{x}(t)}$, and their estimators are the population readouts of the spiking activity of E neurons, ${\displaystyle {\displaystyle {\overrightarrow{\hat{x}}}^E(t)}}$. In I neurons, the targets are defined as the population readouts of E neurons ${\displaystyle {\displaystyle {\overrightarrow{\hat{x}}}^E(t)}}$ and their estimators are the population readouts of I neurons ${\displaystyle {\displaystyle {\overrightarrow{\hat{x}}}^I(t)}}$. Furthermore, the time-dependent metabolic cost is proportional to the squared estimate of the instantaneous firing rate, summed across neurons from the same population. Following these assumptions, we define the variables of loss functions in Equation 10 as

We use a quadratic metabolic cost because it promotes the distribution of spiking across neurons (Boerlin et al., 2013). In particular, the loss function of I neurons, ${\displaystyle {\mathrm{L}}^I(t)}$ implies the relevance of the approximation: ${\displaystyle {\displaystyle {\overrightarrow{\hat{x}}}^E(t)\approx {\overrightarrow{\hat{x}}}^I(t)}}$ (see ${\displaystyle {ϵ}^I}$ in the Equation 11), which will be used in what follows.

We minimize the loss function by positing that neuron ${\displaystyle i}$ of type ${\displaystyle y\in \{E,I\}}$ emits a spike as soon as its spike decreases the loss function of its population ${\displaystyle y}$ in the immediate future (Koren and Panzeri, 2022). We also define ${\displaystyle t^{-}}$ and ${\displaystyle t^{+}}$ as the left- and right-sided limits of a spike time ${\displaystyle t=t_i^{y,\alpha }}$, respectively. Thus, at the spike time, the following jump condition must hold:

with ${\displaystyle {\displaystyle {\xi }_i^y}}$ in units of mV. Here, the arguments ${\displaystyle t^{-}}$ and ${\displaystyle t^{+}}$ denote the left- and right-sided limits of the respected functions at time ${\displaystyle t}$. Furthermore, we added a noise term on the right-hand side of the Equation 12 in order to consider the stochastic nature of spike generation in biological networks (Faisal et al., 2008). A convenient choice for the noise ${\displaystyle {\displaystyle {\xi }_i^y(t)}}$ is the Ornstein-Uhlenbeck process obeying

where ${\displaystyle {\eta }_i^y}$ is a Gaussian white noise with auto-covariance function ${\displaystyle ⟨{\eta }_i(t){\eta }_j(t^{\mathrm{\prime }})⟩={\delta }_{ij}\delta (t-t^{\mathrm{\prime }})}$. The process ${\displaystyle {\xi }_i^y(t)}$ has zero mean and auto-covariance function ${\displaystyle ⟨{\xi }_i(t){\xi }_j(t^{\mathrm{\prime }})⟩=({\sigma }_{\xi }^y{)}^2{\delta }_{ij}{e}^{-\lambda |t-t^{\mathrm{\prime }}|}}$, with ${\displaystyle ({\sigma }_{\xi }^y{)}^2}$ the variance of the noise.

By applying the condition for spiking in Equation 12 using ${\displaystyle y=E}$ and ${\displaystyle y=I}$, respectively, we get

According to the definitions in Equation 7 and Equation 9, if neuron i fires a spike at time ${\displaystyle {\displaystyle t=t_i^{y,\alpha }}}$, it causes a jump of its own filtered spike train (but not of other neurons ${\displaystyle j\ne i}$), as well as of the population readout of the population it belongs to. Therefore, when neuron ${\displaystyle i}$ fires a spike, we have for a given neuron ${\displaystyle j}$ and a given population readout ${\displaystyle k}$:

By inserting Equation 15a, Equation 15b in Equation 12, we find that neuron ${\displaystyle i}$ of type ${\displaystyle y}$ should fire a spike if the following condition holds:

with ${\displaystyle \Vert {\overrightarrow{w}}_i^y{\Vert }^2:=\sum _{k=1}^M(w_{ki}^y{)}^2}$ the squared length of the tuning vector of neuron ${\displaystyle i}$ of type ${\displaystyle y}$. These equations tell us when the neuron ${\displaystyle i}$ of type ${\displaystyle E}$ and ${\displaystyle I}$, respectively, emits a spike, and are similar to the ones derived in previous works (Koren and Panzeri, 2022; Boerlin et al., 2013). In addition to what has been found in these previous works, we here also find that each term on the left- and right-hand side in the Equation 16a has the physical units of millivolts.

We note that the expression derived from the minimization of the loss function of E neurons in the top row of Equation 16a is independent of the activity of I neurons, and would thus lead to the E population being unconnected with the I population. In order to derive a recurrently connected E-I network, the activity of E neurons must depend on the activity of I neurons. We impose this property by using the approximation of estimates that holds under the assumption of efficient coding in I neurons (see ${\displaystyle {\displaystyle {ϵ}^I}}$ in the Equation 11), ${\displaystyle {\displaystyle {\overrightarrow{\hat{x}}}^I(t)\approx {\overrightarrow{\hat{x}}}^E(t)}}$. This yields the following conditions:

We now define new variables ${\displaystyle u_i^y(t)}$ and ${\displaystyle {\displaystyle {\theta }_i^y}}$ as proportional to the left- and the right-hand side of these expressions,

The variables ${\displaystyle u_i^y(t)}$ and ${\displaystyle {\theta }_i^y}$ are interpreted as the membrane potential and the firing threshold of neuron ${\displaystyle i}$ of cell type ${\displaystyle y\in \{E,I\}}$.

In this section, we develop the exact dynamic equations of the membrane potentials ${\displaystyle {\dot{u}}_i^y(t)}$ for ${\displaystyle y\in \{E,I\}}$ according to the efficient coding assumption. We rewrite Equation 9 in vector notation as

with ${\displaystyle {\overrightarrow{f}}^y(t):=[f_1^y(t),\dots ,f_{N^y}^y(t){]}^{\mathrm{\top }}}$ the vector of spike trains for ${\displaystyle {\displaystyle N^y}}$ neurons of cell type ${\displaystyle y\in \{E,I\}}$.

In the case of E neurons, the time-derivative of the membrane potential ${\displaystyle {\displaystyle {\dot{u}}_i^E(t)}}$ in Equation 17, is obtained as

By inserting the dynamic equations of the target signal ${\displaystyle \overrightarrow{\dot{x}}(t)}$, its estimate ${\displaystyle {\overrightarrow{\dot{\hat{x}}}}^I(t)}$ (Equation 18) and of the single neuron readout ${\displaystyle {\dot{r}}_i^E(t)}$ (Equation 7 in the case ${\displaystyle y=E}$), we get

where in the last line we used the definition of ${\displaystyle u_i^E(t)}$ from the Equation 17.

In the case of I neurons, the time derivative of the membrane potential ${\displaystyle {\dot{u}}_i^I(t)}$ in Equation 17 is

By inserting the dynamic equations of the population readouts of E neurons ${\displaystyle {\overrightarrow{\dot{\hat{x}}}}^E(t)}$ and of the I neurons ${\displaystyle {\overrightarrow{\dot{\hat{x}}}}^I(t)}$ (Equation 18) and of the single neuron readout ${\displaystyle {\dot{r}}_i^I(t)}$ (Equation 7 in the case ${\displaystyle y=I}$), we get

where in the last line we used the definition of ${\displaystyle u_i^I(t)}$ from Equation 17.