Abstract
Pyramidal cells and interneurons expressing parvalbumin (PV), somatostatin (SST), and vasoactive intestinal peptide (VIP) show celltypespecific connectivity patterns leading to a canonical microcircuit across cortex. Experiments recording from this circuit often report counterintuitive and seemingly contradictory findings. For example, the response of SST cells in mouse V1 to topdown behavioral modulation can change its sign when the visual input changes, a phenomenon that we call response reversal. We developed a theoretical framework to explain these seemingly contradictory effects as emerging phenomena in circuits with two key features: interactions between multiple neural populations and a nonlinear neuronal inputoutput relationship. Furthermore, we built a cortical circuit model which reproduces counterintuitive dynamics observed in mouse V1. Our analytical calculations pinpoint connection properties critical to response reversal, and predict additional novel types of complex dynamics that could be tested in future experiments.
Introduction
Three major nonoverlapping classes of interneurons expressing parvalbumin, somatostatin and vasoactive intestinal peptide (henceforth denoted PV, SST and VIP respectively) make up more than 80% of GABAergic cells of mouse cortex (Rudy et al., 2011). These neurons show celltypespecific connectivity among themselves and with excitatory (E) neurons (Pfeffer et al., 2013; Jiang et al., 2015) forming a canonical microcircuit in the cortex. This microcircuit motif, initially proposed theoretically (Wang et al., 2004), has been the subject of numerous recent experimental studies using optogenetic tools applied to behaving mice (Lee et al., 2012; Saleem et al., 2013; Kepecs and Fishell, 2014; Hawrylycz et al., 2016) as well as computational studies (Lee and Mihalas, 2015; Lee and Mihalas, 2017; Lee et al., 2017; Yang et al., 2016; Yang and Wang, 2017). However, we still do not fully understand the mechanisms that underlie the behavior of this microcircuit which are often complex and counterintuitive.
A notable observation was that pyramidal neurons and VIP interneurons concomitantly increase their activities in the primary visual cortex V1 during locomotion in comparison with immobility (Niell and Stryker, 2010), even in the complete absence of visual input (Keller et al., 2012). Moreover, optogenetically activating (respectively inactivating) VIP interneurons mimics (respectively eliminates) the effect of running (Fu et al., 2014). Since VIP cells primarily target SST cells, a natural explanation for this phenomenon is disinhibition (Wang et al., 2004; Lee et al., 2013): activation of VIP cells suppresses SST cells, therefore neurons targeted by the SST population are disinhibited, enhancing the overall activity of excitatory neurons. However, recent experiments show that the network behavior might be more complex. Namely, in darkness the activation of VIP cells results in an average decrease of SST population activity (Fu et al., 2014), whereas in the presence of visual stimulation the response of SST cells is reversed and its firing rate increases during locomotion compared to immobility (Pakan et al., 2016). These findings, which have been further confirmed in a recent preprint (Dipoppa et al., 2017), appear to challenge the disinhibition hypothesis, suggesting that the nature of the interaction between VIP and SST could be stimulus dependent.
These experimental results raise two questions: First, the external activation of a population that directly inhibits a second population can trigger a positive response of the latter. What is the mechanism behind this apparently paradoxical behavior? Second, the same topdown modulation can trigger both a positive response and a negative response of certain populations of the circuit depending on the sensory input. Under which conditions can we expect one response or the other?
In this study, we model cortical activity and provide a comprehensive answers to these two questions. We show that these counterintuitive phenomena rely on two basic features of cortical networks: (i) the presence of multiple populations of interneurons and (ii) nonlinear responses to input. Finally, we use our model to predict complex behaviors that have not yet been experimentally tested. Beyond the mechanistic explanation for the observed behavior in mice V1, our work provides a very general and powerful framework to explain the dynamics of neural networks with multiple interneuron types, their contextdependent interactions, and the emsergence of counterintuitive effects that may occur across different cortical structures and animals.
Results
We simulate microcircuit activity using a four population firing rate model. The average rate of each population is given by a nonlinear function of its input that we refer to as the fI curve (Abbott and Chance, 2005). The fI curve is such that when the input is low (below threshold), cells are little responsive to changes in external input. Instead for high input (above threshold) small changes in the input can drive substantial changes in the response (Miller and Troyer, 2002) (see Figure 1b). This nonlinearity has been analyzed experimentally and theoretically (Murphy and Miller, 2003; Phillips and Hasenstaub, 2016) and as we will show later, it is a key feature of the model.
Populations are connected according to the microcircuit scheme in Figure 1a which contains the connections reported in both Jiang et al., 2015 and Pfeffer et al. (2013). We also consider three sources of input: (i) topdown modulation that targets VIP cells (ii) local recurrent input and (iii) constant background input set so that the populations have some fixed baseline activity (see Materials and methods for details).
Response to topdown modulation depends on baseline activity
To illustrate possible complex behaviors displayed by the network, we first focused on the circuit responses to topdown modulation. The simulation results from our model allow us to identify two qualitatively different scenarios depending on the baseline activity of the network (the baseline activity is the activity before the onset of topdown modulation and we control it by changing the constant background input, see Materials and methods for details). On the one hand, when the baseline activity is low, topdown modulation will result in a decrease of the rate of the SST population and an increase of the rates of the other populations (E, PV and VIP) (see Figure 1c). On the other hand, when baseline activity is high, the rate of all populations increases with topdown modulation (see Figure 1d). These simulations reveal that population responses to topdown modulation depend in a complex way on the initial state of the network.
The striking behavior exhibited by the SST population can be explained heuristically by analyzing the response of the different populations to external excitatory input targeting VIP cells. When the topdown modulation starts, the rate of the VIP population increases. By calculating the time derivatives of the rates right after the onset of the topdown modulation (see Materials and methods) one can see that this effect always results in a transient reduction of SST activity and therefore a reduction of inhibition to VIP, PV and E cells. When baseline activity is low the E population is below threshold and this change in net input has a small effect in the output. In that situation, all populations quickly reach a stationary state. However, when the baseline activity is high, the E population is above threshold and a small change in input from SST cells has a big effect on the rate of the E population. If the recurrent excitation in the microcircuit is strong enough, it can reverse the initial response of the SST population making it increase its activity to a higher rate than the baseline.
Circuit behavior explained by response matrix
In order to formally characterize the steady state response of a population to external input we introduce the response matrix $M$. The intuition behind the response matrix is that if we change the input to population $j$ (where $j=E,P,S,V$ for excitatory, PV, SST and VIP populations respectively) by a small amount $\delta {I}_{j}$, then the change in rate of the population $i$ will be $\delta {r}_{i}=\delta {I}_{j}{M}_{ij}$. If ${M}_{ij}$ is positive (negative), an increase of the external excitation to $j$ will result in an increase (decrease) of the rate of population $i$ (see Materials and methods and Table 3 for details). In contrast to the connectivity matrix, which takes into account only the direct path from population $j$ to $i$, the response matrix contains information about all the possible ways in which population $j$ can affect population $i$, namely through indirect connections $j$$h$$i$. Due to the complexity of these indirect pathways, for different values of the connectivity matrix (but preserving the excitatory/inhibitory structure) ${M}_{ij}$ can be positive or negative irrespective of whether the connection from $j$ to $i$ is inhibitory or excitatory. Furthermore, due to the nonlinearities in the fI curve, the response depends on the baseline rate of each of the populations and, as shown before, it can reverse its sign.
As an example, we analyze in detail the response of the SST population to external input to VIP cells. As we show in the Materials and methods section, this term of the response matrix is given by:
where ${w}_{ij}$ are the absolute values of the connection weights and therefore are positive by definition and for the system to be stable $C$ has to be positive (see Materials and methods for details). The terms ${d}_{i}$ are proportional to the inverse of the first derivative of the fI curves and are always positive. In particular, ${d}_{E}$ becomes arbitrarily large when the input is very low and tends monotonically to a positive constant ${d}_{E}^{\mathrm{\infty}}$ for high input. Therefore, if ${w}_{EE}\le {d}_{E}^{\mathrm{\infty}}$ then ${M}_{SV}$ will always be negative. However, for ${w}_{EE}>{d}_{E}^{\mathrm{\infty}}$ the behavior is much richer: if input is high then ${d}_{E}$ will be close to its minimum ${d}_{E}^{\mathrm{\infty}}$ and ${w}_{EE}>{d}_{E}$ allowing for ${M}_{SV}$ to be positive (provided that the product ${w}_{EP}{w}_{PE}$ is small enough). Instead if the input is low, ${d}_{E}$ will become very large and ${M}_{SV}$ will be negative.
It is remarkable that this change in the interaction between VIP and SST populations depends on the activation level of E: modifying the state of one population has a impact in the interactions between other populations. The heuristic explanation is that if the recurrent excitation is strong enough and the E population is already strongly excited (above threshold), a small decrease in the inhibition from SST to the E population can boost its activity and therefore strongly drive the whole microcircuit. If instead, the E population is in a low activation state the change in inhibition will have a weak effect that will not be able to reverse the response of SST.
This observation provides an explanation to the reversal of the response of SST to VIP activation when the baseline activity is changed: as we show in Figure 2a and c for low baseline activity, ${M}_{SV}$ is negative and the presence of an external excitatory current targeting VIP cells will result in a negative response of SST cells and positive response of E, PV and VIP cells, conforming to the disinhibitory hypothesis. On the other hand, for high baseline activity (panels 2b and 2d), the response of the SST population to input to VIP cells becomes positive leading to the response reversal regime.
A similar analysis can be conducted for all terms in $M$. For example, another case of response reversal in this circuit is that of ${M}_{EE}$ which can have different signs for different baseline activity levels, meaning that the excitatory population can have a negative response to excitatory input to itself. Intuitively, if an external excitatory current targets the E population, its rate will increase transiently and thus the excitation that SST and VIP receive will also increase. If this effect is stronger in SST than in VIP the rate of the VIP population will decrease and therefore the inhibition that SST receives will decrease as well resulting in stronger inhibition to E cells. Note that for this to happen both SST and VIP have to be in the high activity baseline (i.e. ${d}_{S}$, ${d}_{V}$ have to be small) and ${w}_{SV}$, ${w}_{VS}$ have to be strong. The explicit expression of ${M}_{EE}$ (see Table 3) reveals that if the SSTVIPSST loop is not strong enough or if ${d}_{S}$, ${d}_{V}$ are large ${M}_{EE}$ will always be positive.
Random network model
Experimental recordings showed a great diversity across neural responses even when recording from the same class of cells (Pyramidal, SST, PV or VIP) (Pakan et al., 2016). Although this diversity can have many origins, such as intrinsic heterogeneity in the cells within the same class, we proposed that random connectivity alone is sufficient to explain it. To do so we develop an extension of our model where each population is composed of multiple identical randomly connected rate units and where the probability that one connection exists from one unit to another depends on the populations of the presynaptic and postsynaptic units according to data extracted from Jiang et al. (2015); Pfeffer et al. (2013) (see Materials and methods for details).
For each unit, we measure the rate modulation (rate during topdown modulation minus baseline activity) for the different baselines. If the rate modulation is positive it means that the neuron is more active in the presence of the modulatory current and vice versa. In Figure 3, we show scatter plots of the rate modulation under the low baseline condition versus the rate modulation under the high baseline condition for each unit. These simulations reveal that the behavior of individual neurons can be quite variable while the population average still corresponds to the behavior of the populationbased model. Since all units of each population are identical, variability in the response has to be due to the heterogeneity in the connectivity. This variability can result in cells within the same population having responses with opposite sign, as has been observed to be the case in mouse V1 (Reimer et al., 2014; Pakan et al., 2016) and A1 (Kuchibhotla et al., 2017). In addition, variability might also have further implications for gating of signals, since variability in inhibitory cells has been proposed to modulate the response gain of neural circuits (Mejias and Longtin, 2014).
Model of mouse V1 accounts for experimental measurements
Our framework allows us to easily understand the counterintuitive behavior of V1 during locomotion. In the experiments mice with their head fixed face a screen where different visual stimuli are presented and can run freely on a treadmill (Fu et al., 2014; Pakan et al., 2016). Different visual stimuli result in different baseline activities in V1 and topdown modulation is triggered when the mice start running.
To model visual input we use external currents. In the case of sizevarying gratings, this input has two sources: thalamic input that targets excitatory cells and cortical input that targets SST cells. In order to reproduce the surround suppression effect (Ozeki et al., 2009; Adesnik et al., 2012), excitatory cells have a small receptive field and therefore receive center input and SST cells have a large receptive field and receive surround input (see Materials and methods for details).
Figure 4b shows the response reversal phenomenon when a weak visual stimulus is presented. Before the visual stimulation, the SST has higher activity for immobility than for locomotion, by contrast, when the visual stimulus is presented, the activity of the SST population is higher for locomotion. In Figure 4c, we show the experimental data from Pakan et al. (2016) for three different experimental conditions (darkness, gray screen and grating) and in Figure 4d our simulations of V1 under the same conditions. Figure 4—figure supplement 1 shows the experimental data from the preprint (Dipoppa et al., 2017) for gratings of different sizes alongside with the behavior of our model.
Our simulations of this V1 circuit model reproduce the phenomena described in the literature: in the presence of visual stimulation, the activities of all populations, including SST, increase during locomotion (Pakan et al., 2016). In darkness, the activities of excitatory, PV and VIP populations increase during locomotion while the activity of SST decreases as reported in Fu et al. (2014) and in the preprint (Dipoppa et al., 2017). In Pakan et al. (2016), the response of SST to locomotion in darkness is weakly positive but this result is not statistically significant while the other two are.
To show that our results do not rely on a fine tuning of the connectivity parameters or even on certain details of the microcircuit structure, we have run the model with several connectivity matrices and perturbations of them (Figure 4—figure supplement 2) and we find that different connectivity parameters can reproduce the same circuit behavior as has been shown before in other systems (Marder et al., 2015). We have also considered other microcircuit structures to account for the differences between studies ([Pfeffer et al., 2013] reports projections from PV to VIP and (Jiang et al., 2015) from PV to SST) and we also consider thalamic input to PV (Figure 4—figure supplement 3). In all these cases, the results were consistent with our original findings showing that the phenomenon and the analysis are robust and not a peculiarity of one specific circuit.
Discussion
We have developed a theoretical model of cortical circuit with multiple interneuron types that accounts for newly identified complex interactions between cell types. The model has been used to reproduce and explain two counterintuitive phenomena observed in mouse cortex. First, in certain cases the activation of VIP cells results in an overall positive response of the SST population (Pakan et al., 2016). Second, the sign of the SST population response to excitation of VIP cells depends on the baseline activity of the circuit (Fu et al., 2014). Two features of the system lead to this behavior: the presence of multiple interneuron populations and the nonlinearity of fI curves.
We explained heuristically the response reversal by closely looking at transient dynamics of the circuit. One experimentally testable prediction of our analysis is that, as Figure 1d and our calculations of the transient behavior show, in the response reversal regime, the overall SST population response to topdown modulation should initially decrease and later increase until reaching a higher rate than the baseline.
Based on our model, we introduced the response matrix $M$, which is a comprehensive framework to understand counterintuitive steady state responses. It provides explicit information about the contribution of each individual connection. For example by looking at the elements in ${M}_{SV}$ (see Table 3), one can readily see that if the recurrent excitation between pyramidal cells is not large enough, ${M}_{SV}$ can only be negative and therefore response reversal of SST would not happen. This statement can be easily tested by repeating the experiments while suppressing the activation of the E population. As we discussed before, another example is that if both SST and VIP populations have high baseline activities and if the SSTVIPSST loop is strong enough, ${M}_{EE}$ can be negative, that is the excitatory population can have a negative response to excitatory input (see Table 3 for the explicit expression of ${M}_{EE}$). If the connections between the SST and the VIP populations are removed (or weakened) or if their baseline activities are sufficiently lowered ${M}_{EE}$ will always be positive. This constitutes another interesting prediction that can be experimentally tested.
Our calculations also revealed sign correlations between entries of $M$, for example ${M}_{SV}$ and ${M}_{SS}$ have opposite signs for any connectivity matrix (given the microcircuit) and for any baseline activity. This predicts that in the regime where SST activity has a positive response to excitatory input targeting VIP, SST has to have a negative response to external input targeting SST. In addition, our results are in line with experimental studies that show that VIP interneurons play an important role in cortical activity modulation (Mesik et al., 2015; Ibrahim et al., 2016; Jackson et al., 2016).
Our approach constitutes a general conceptual framework in which previous work regarding complex cortical interactions can be better understood (Tsodyks et al., 1997; Ozeki et al., 2009; LitwinKumar et al., 2016). The analysis of the response matrix shows that for the given microcircuit structure all terms of the matrix can be positive or negative. This is not the case in EI networks (networks with one excitatory (E) population and only one inhibitory (I) population) (Tsodyks et al., 1997; Ozeki et al., 2009). In that case ${M}_{EE}$ and ${M}_{IE}$ are always positive, ${M}_{EI}$ is always negative and only ${M}_{II}$ can have both signs (see Materials and methods). In this sense, having more than one inhibitory population results in a much more versatile network. Another important point that can be derived from our calculations is the relationship between response reversal and inhibition stabilized networks (ISN) (Ozeki et al., 2009). Looking at the terms of the response matrix for an EI network, we can see that the condition to have response reversal and the condition to be an ISN is the same: ${W}_{EE}$ has to be larger than ${d}_{E}^{\mathrm{\infty}}$. When analysing networks with more than one inhibitory population the relationship is not necessarily bidirectional any more. In the network that we analyzed, we found that in the high baseline activity the network is in the ISN regime and ${M}_{SV}$ is positive (as observed in [LitwinKumar et al., 2016), whereas in the low baseline activity the network is not in the ISN regime and ${M}_{SV}$ is negative, so in this case there is a clear relationship between being an ISN and exhibiting response reversal. However, the condition for other cases of response reversal such as ${M}_{EE}$ do not involve ${W}_{EE}$ and therefore do not require the network to be an ISN.
Finally, this study provides a parsimonious yet powerful explanation to striking observations of interneuronal circuits in V1 (Fu et al., 2014; Pakan et al., 2016; Lee et al., 2017) without requiring the assumption of topdown excitatory inputs explicitly targeting SST or PV neurons. Both our computational neural network model and the approach presented here (the response matrix analysis) go beyond circuit dynamics in mice V1 and can be easily applied to other species and cortical areas. By extending previous works (Tsodyks et al., 1997; Ozeki et al., 2009), it naturally explains the response reversal observed in cat visual cortex (Ozeki et al., 2009). It could also be applied to explain similar phenomena observed in mouse primary auditory cortex (Seybold et al., 2015; Kuchibhotla et al., 2017). In particular, in Kuchibhotla et al. (2017), the authors find that locomotion reduces the activity of excitatory cells. Assuming that the main modulation in the circuit is mediated by VIP cells this observation implies that ${M}_{EV}<0$ which is the case when the connections ${W}_{EP}$ and ${W}_{P}S$ are strong enough. In mouse somatosensory cortex, activating VIP neurons results in an intuitive decrease in SST activity, instead of a response reversal (Lee et al., 2013). As our results suggest, this qualitative difference between V1 and somatosensory cortex may be explained by the quantitative difference between their circuit architectures: in a recent study the authors showed that cell densities of different types of interneurons differ substantially across cortical areas resulting in counterintuitive impacts on circuit responses (Kim et al., 2017). These responses can be readily understood using the response matrix.
In this work, we mainly focused on steadystate responses. However, neural responses in many cortical areas, including primary auditory cortex, are largely transient and dynamical (Wehr and Zador, 2003). In addition, synaptic connections to and from interneurons are often subject to shortterm plasticity (Reyes et al., 1998). Understanding transient dynamics in nonlinear, multitype interneuronal circuits would be an important topic for future research.
We have shown that similar to the now wellknown paradoxical effect that the presence of a single inhibitory neuron type can cause (Tsodyks et al., 1997; Ozeki et al., 2009), the presence of multiple types of interneurons has an even stronger impact on the activity of neural circuits. We have also exposed the effect of nonlinearity of the fI curve. Our analysis suggests that in a circuit with multiple populations, the most interesting circuit behavior is found when spontaneous baseline activity is close to threshold since in that regime responses will change the most with small changes in population rates. These two features significantly broaden the richness of the dynamics of cortical circuits and enhance their usefulness for cognitive and behavioral computations. We conclude that computational models and mathematical analysis are critical to fully understand the dynamics of neural circuits underlying behavior (Gjorgjieva et al., 2016), especially when several types of interneurons are involved as intuition alone may be misleading and provide erroneous predictions on such circuits.
Materials and methods
Firingratebased population model
Request a detailed protocolThe state of the system is characterized by the rates ${r}_{i}$. To model the average rate of each population we use a function of the input ${V}_{i}$ as the one introduced in Abbott and Chance (2005)
where ${V}_{th}=50$ mV and ${V}_{r}=60$ mV are the threshold and reset potentials respectively, $\tau $ is the membrane time constant and $v=1$ mV. ${V}_{i}$ is the average input to each of the populations and is given by
where ${V}_{l}=70$ mV is the reversal potential and ${g}_{l}^{i}$ is the membrane conductance. $W$ is the connectivity matrix and therefore ${\sum}_{j}{W}_{ij}{r}_{j}$ is the recurrent local input. ${I}_{i}$ is the external input current and ${I}_{bkg}^{i}$ is a constant current that is tuned to obtain the desired baseline activity and we find the specific values by solving the system ${r}_{i}=f({V}_{l}+(\sum _{j}{W}_{ij}{r}_{j}+{I}_{i}+{I}_{bkg}^{i})/{g}_{l}^{i})$. For example, for the baseline activity steadystate the background currents needed to obtain the desired rates (1, 10, 3 and 2 Hz for pyramidal, PV, SST and VIP, respectively) are 114.7, 233.6, 94.3 and 89.9 pA. The rate dynamics are given by
where ${\tau}_{r}=2$ ms (Gerstner, 2000). Since the parameters of the fI curve are population dependent (see Table 2), different populations will have different rates for the same input. The nonlinearity of the fI curve has very important consequences. Namely, for low input $f({V}_{i})$ is almost flat, and therefore changes in the input will have almost no effect on the rate. By contrast, for strong input $f({V}_{i})$ tends asymptotically to a straight line with slope $\frac{1}{{\tau}_{i}({V}_{th}{V}_{r})}$ and changes in the input will elicit a large change in the rate. As we will show later, this feature is key to reproduce the response reversal observed in the experiments.
The connectivity matrix $W$ used in the simulations is generated by rejection sampling, that is by generating random matrices that have the microcircuit structure given in Figure 1a and selecting the ones that produce the desired responses. The simulations of Figures 1 and 2 were done with the connectivity matrix given in Table 1.
Behavioral state is modeled with a constant topdown modulatory current of 10 pA that targets VIP cells. The constant background inputs ${I}_{bkg}^{i}$ are set so that in the absence of the topdown modulatory current, the E, PV, SST and VIP populations will have spontaneous average rates of 1, 10, 3 and 2 Hz, respectively, for the low baseline activity scenario and 30, 50, 30 and 20 Hz for the high baseline activity.
Time derivatives of the rates after the onset of modulation
Request a detailed protocolIn this section, we calculate analytically the changes in rate right after the onset of the modulatory current. The intuition behind these calculations is that the initial change in activity of a population is driven by the fastest path from the external input to the neurons in that population.
We assume that the system is at a fixed point (therefore $\frac{d{r}_{i}}{dt}=0$ for all populations) and that at time $t=0$ an excitatory topdown modulatory current targets the VIP population. Taking into account that the time derivatives of the rates are given by Equation (3) and since $f(V)$ is monotonously increasing and the modulatory current ${I}_{V}>0$, then $\frac{d{r}_{V}}{dt}(0)$ will be positive and all other derivatives will still be 0. In order to estimate the behavior of the initial slope of $\frac{d{r}_{i}}{dt}$, we calculate the second derivatives at $t=0$:
where in the last step we used the fact that $\frac{d{r}_{i}(0)}{dt}=0$ except for VIP. Since $\frac{df}{d{V}_{i}}$, ${g}_{l}^{i}$ and $\frac{d{r}_{V}}{dt}$ are positive, the sign of $\frac{{d}^{2}{r}_{i}}{d{t}^{2}}$ will depend on the sign of ${W}_{iV}$. In particular, for SST we obtain
meaning that in all regimes the initial (transient) response of the SST population to topdown modulation targeting VIP cells will be negative.
Response matrix and response reversal
Request a detailed protocolIn order to characterize the response of a population to external excitatory input to the network we calculate how its rate will change for a small change in external input. We focus on stationary states ${r}_{i}=f({V}_{i})$. If we apply a small perturbation to the external input $\delta {I}_{i}$, the network will reach a new stationary state
where ${f}^{\prime}({V}_{i})$ is the derivative of $f$ with respect to $V$ and
Since ${r}_{i}=f({V}_{i})$, when we linearize $f$ around $V$ and ignore terms of order $\delta {V}^{2}$ and higher we obtain the following selfconsistent equation
We define the entries of response matrix as the derivative ${M}_{ij}=\frac{\partial {r}_{i}}{\partial {I}_{j}}$, which can be obtained from the limit $\delta {I}_{j}\to 0$ in the system of equations given by (Equation 8) and in matrix form can be written as
where $D$ is a diagonal matrix with entries ${D}_{ii}={g}_{l,i}/{f}^{\prime}({V}_{i})$. As it was explained in the results section, the nonlinear behavior of the terms ${D}_{ii}$ is essential to explain the response reversal regime. ${D}_{ii}$ becomes arbitrarily large as ${V}_{i}\to \mathrm{\infty}$ and decreases monotonically to ${d}_{i}^{\mathrm{\infty}}={\tau}_{i}({V}_{th}{V}_{r})/{g}_{l}^{i}$ when ${V}_{i}\to \mathrm{\infty}$.
In Table 3, we give the explicit formulas to all the entries of the response matrix in terms of the entries of the connectivity matrix $W$ and $D$ (we denote $w=W$, ${d}_{i}={D}_{ii}$ and $C=\text{det}{(DW)}^{1}$). Note that, because of the complex interactions in the network, the sign of ${M}_{ij}$ is never determined exclusively by that of ${W}_{ij}$.
Random network model
Request a detailed protocolWe consider a network with 800 E units, 100 PV units, 50 SST units and 50 VIP units. Each unit within a population has the same fI curve with the parameters in Table 2. The probabilities ${p}_{ij}$ of a connection from each unit in population $j$ to each unit in population $i$ are estimated from data (Pfeffer et al., 2013; Jiang et al., 2015) and are given in Table 4.
The strengths of the connections are rescaled so that the average input of a unit in population $i$ from all units in population $j$ is ${W}_{ij}$ as given in Table 1. More specifically, each unit in population $i$ will receive in average ${m}_{ij}={p}_{ij}{N}_{j}$ projections from population $j$ (where ${N}_{j}$ is the number of units in population $j$) and therefore the weight of these connections will be ${W}_{ij}/{m}_{ij}$.
Topdown modulatory current and background input is identical to all units within the same population and has the same value as in the population based model.
Mouse V1 model
Request a detailed protocolIn the simulations of V1 activity, we use the connectivity matrix given in Table 5.
We model visual input with an external excitatory current that targets E and SST cells. In the experiments in Pakan et al. (2016) and in the preprint (Dipoppa et al., 2017) the authors consider three levels of visual stimulation which are: darkness, gray screen and grating. To model darkness condition, we assume a total absence of visual stimulation (therefore ${I}_{E}=0$ pA, ${I}_{S}=0$ pA). For gray screen, we use a small input current to the excitatory population (${I}_{E}=50$ pA, ${I}_{S}=0$ pA). Finally to model different grating diameters the value of the input is a sigmoid function of the grating diameter $\theta $:
where ${b}_{E}=2$, ${b}_{S}=6$, ${a}_{E}=100$ pA, ${a}_{S}=20$ pA. With this parameters E cells receive center input (input saturates for diameters $\sim 20$ deg) and SST cells receive surround input (input to SST saturates for diameters of $\sim 60$ deg) (Dipoppa et al., 2017).
To demonstrate that our results do hold for a wide range of connectivity matrices and do not have to be fine tuned, we simulate several different connectivity matrices that produce the same qualitative behavior. We also make perturbations of these matrices by multiplying each entry by a random variable uniformly distributed in the interval $[0.9,1.1]$. This amounts to randomly modifying each connection within ±10% of its original value (see Figure 4—figure supplement 2).
In the alternative models of Figure 4—figure supplement 3 where visual stimulus input also targets PV cells, we use ${I}_{P}=0$ pA for darkness, ${I}_{P}=10$ pA for gray screen and ${b}_{P}=2$, ${a}_{P}=20$ pA for gratings.
Response matrix for an EI network
Request a detailed protocolFor the sake of completeness, here we analyze the response matrix for a fully connected EI network (Tsodyks et al., 1997, Ozeki et al., 2009) . The connectivity matrix is
and therefore the response matrix is
where $C={(({d}_{E}{w}_{EE})({w}_{II}+{d}_{I})+{w}_{EI}{w}_{IE})}^{1}$. Note that the only term that can change sign is ${M}_{II}$ so the only population that can exhibit response reversal is the $I$ population. Furthermore, note that the condition for having response reversal (${w}_{EE}>{d}_{E}^{\mathrm{\infty}}$) is the same that defines the ISN regime, so this two properties are equivalent in the EI network.
References

1
Drivers and modulators from pushpull and balanced synaptic inputProgress in Brain Research 149:147–155.https://doi.org/10.1016/S00796123(05)490111
 2
 3
 4
 5
 6
 7
 8

9
VIP+ interneurons control neocortical activity across brain statesJournal of Neurophysiology 115:3008–3017.https://doi.org/10.1152/jn.01124.2015
 10
 11
 12
 13

14
Parallel processing by cortical inhibition enables contextdependent behaviorNature Neuroscience 20:62–71.https://doi.org/10.1038/nn.4436
 15

16
A disinhibitory circuit mediates motor integration in the somatosensory cortexNature Neuroscience 16:1662–1670.https://doi.org/10.1038/nn.3544
 17

18
A computational analysis of the function of three inhibitory cell types in contextual visual processingFrontiers in Computational Neuroscience 11:28.https://doi.org/10.3389/fncom.2017.00028

19
Visual processing mode switching regulated by VIP cellsScientific Reports 7:1843.https://doi.org/10.1038/s41598017018300

20
Inhibitory stabilization and visual coding in cortical circuits with multiple interneuron subtypesJournal of Neurophysiology 115:1399–1409.https://doi.org/10.1152/jn.00732.2015

21
Robust circuit rhythms in small circuits arise from variable circuit components and mechanismsCurrent Opinion in Neurobiology 31:156–163.https://doi.org/10.1016/j.conb.2014.10.012

22
Differential effects of excitatory and inhibitory heterogeneity on the gain and asynchronous state of sparse cortical networksFrontiers in Computational Neuroscience 8:107.https://doi.org/10.3389/fncom.2014.00107
 23

24
Neural noise can explain expansive, powerlaw nonlinearities in neural response functionsJournal of Neurophysiology 87:653–659.https://doi.org/10.1152/jn.00425.2001

25
Multiplicative gain changes are induced by excitation or inhibition aloneJournal of Neuroscience 23:10040–10051.
 26
 27
 28
 29
 30
 31

32
Targetcellspecific facilitation and depression in neocortical circuitsNature Neuroscience 1:279–285.https://doi.org/10.1038/1092

33
Three groups of interneurons account for nearly 100% of neocortical GABAergic neuronsDevelopmental Neurobiology 71:45–61.https://doi.org/10.1002/dneu.20853

34
Integration of visual motion and locomotion in mouse visual cortexNature Neuroscience 16:1864–1869.https://doi.org/10.1038/nn.3567
 35

36
Paradoxical effects of external modulation of inhibitory interneuronsJournal of Neuroscience 17:4382–4388.
 37
 38

39
A dendritic disinhibitory circuit mechanism for pathwayspecific gatingNature Communications 7:12815.https://doi.org/10.1038/ncomms12815

40
A disinhibitory motif and flexible information routing in the brainCurrent Opinion in Neurobiology In press.
Decision letter

Peter LathamReviewing Editor; University College London, United Kingdom

Timothy E BehrensSenior Editor; University of Oxford, United Kingdom
In the interests of transparency, eLife publishes the most substantive revision requests and the accompanying author responses.
Thank you for submitting your article "Paradoxical response reversal of topdown modulation in cortical circuits with three interneuron types" for consideration by eLife. Your article has been favorably evaluated by Timothy Behrens (Senior Editor) and three reviewers, one of whom is a member of our Board of Reviewing Editors. The reviewers have opted to remain anonymous.
The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.
Summary:
This manuscript presents some important insights into the diverse, counterintuitive behaviors of circuits with interacting inhibitory neuron populations. The authors show that, in a circuit with three types of interneuron, the functional sign of interactions can change depending on the exact activity level of the different cell types in the network – a population that inhibits another in one regime may suppress it in another regime. The essential features to enable this are: 1) more than one type of interneuron, 2) with diverse thresholds/nonlinearities. They relate this result to the experimental literature through citations, and directly compare their model results to data figures from other authors. In the discussion, they give testable predictions.
Essential revisions:
1) In the standard mode (van Vreeswijk and Sompolinsky, Neural Computation 1998), connectivity is high, and so the diagonal terms, d_{i}, are large. In this regime, there is no response reversal. It's an open question exactly how high connectivity is; it certainly isn't infinity, which is what physicists would like it to be, and the effective strength of the connectivity drops as the firing rate drops. However, when firing rates drop, fluctuations become important, and firing rate models become less believable. We're not asking the authors to do full network simulations (although we would suggest that it would be an interesting avenue for future research). However, they should at least comment on this. Even better would be a back of the envelope calculation showing that the connection strengths between populations are in the right range.
2) The authors do a good job citing the relevant literature. However they avoid framing their work in the context of inhibitory stabilized networks (ISNs). ISNs have very strong recurrent excitation that needs to be stabilized by recurrent inhibition (Ozeki et al.), and show the signature of a complex transient before settling in the equilibrium state – reminiscent of the author's Figure 1D. As remarked in the manuscript the sign flip of M_{SV} requires w_{EE} to be sufficiently large. Is the network an ISN? Figure 2 of LitwinKumar et al. (2016) extends ISNs to circuits with multiple interneurons subtypes, and they show that if the total inhibition received by E cells reduces under VIP stimulation then the network is an ISN. What regime is the author's model in? Is this a useful label for their network?
3) In Figure 2D, M_{EE} is negative (0.35), and if we understand things correctly, it's always negative in the high gain regime. Thus, in the high baseline activity state, an input to the pyramidal neuron population will result in a decrease of pyramidal neuron firing rates. This is at odds with most (all?) data sets. The authors remark on this feature in the last paragraph of the subsection “Circuit behavior explained by response matrix”, but do not address the plausibility of this prediction. Do the authors think this result is a problem for their model? More generally, with new parameters can the authors explain the sign flip in M_{SV} without a sign flip in M_{EE} or are these tethered together somehow?
4) Dipoppa et al. 2016 is important for justifying the model and the authors cite it frequently (and even republish some of its figures). But this paper has not been peerreviewed (it's a Biorxiv report), giving it the same veridical status as a personal communication or SFN abstract. It's not appropriate for a peerreviewed manuscript to depend on data that has not been reviewed. In addition, we're not sure how this will affect Dipoppa et al.'s attempts to get their work published. eLife is peer reviewed, and many journals won't let you republish work that's already published in a peer reviewed journal. In this manuscript, the authors actually take figures out of the other group's nonreviewed preprint and publish them in their own paper.
It seems to us that Dipoppa et al. is not absolutely essential; Figure 4E could be dropped without affecting the paper much. If the authors do want to include it, they should do two things. First, they should make it crystal clear that Dipoppa et al. is not peerreviewed, every single time the citation is made. They can leave no doubt in the readers' minds that data is not yet part of the scientific literature. Second, they should get permission from Dipoppa et al. before publishing their data. We're guessing eLife requires this, but even if it doesn't, it's not worth irritating one's colleagues for something that is not essential to one's story.
https://doi.org/10.7554/eLife.29742.sa1Author response
Essential revisions:
1) In the standard mode (van Vreeswijk and Sompolinsky, Neural Computation 1998), connectivity is high, and so the diagonal terms, d_{i}, are large. In this regime, there is no response reversal. It's an open question exactly how high connectivity is; it certainly isn't infinity, which is what physicists would like it to be, and the effective strength of the connectivity drops as the firing rate drops. However, when firing rates drop, fluctuations become important, and firing rate models become less believable. We're not asking the authors to do full network simulations (although we would suggest that it would be an interesting avenue for future research). However, they should at least comment on this. Even better would be a back of the envelope calculation showing that the connection strengths between populations are in the right range.
The network in our model is not a balanced network in the sense of [van Vreeswijk and Sompolinsky, 98]. In fact, it is dominated by inhibition (i.e. the sum of all the entries of the connectivity matrix is negative).
In the section “Random network model” (Figure 3) we build a network where each population has multiple units and the connections between units are random. In that case the weights are set so that, in average, the input to each unit of population i from population j is the same as in the population based model. This means that the scaling of the weights in our model is 1/m (where m is the expected number of connections from population j to each unit of population i) and not 1/sqrt(m) as in [van Vreeswijk and Sompolinsky, 98].
We have added a sentence in the Materials and methods section “Random network model” (first paragraph) to make this point clearer.
2) The authors do a good job citing the relevant literature. However they avoid framing their work in the context of inhibitory stabilized networks (ISNs). ISNs have very strong recurrent excitation that needs to be stabilized by recurrent inhibition (Ozeki et al.), and show the signature of a complex transient before settling in the equilibrium state – reminiscent of the author's Figure 1D. As remarked in the manuscript the sign flip of M_{SV} requires w_{EE} to be sufficiently large. Is the network an ISN? Figure 2 of LitwinKumar et al. (2016) extends ISNs to circuits with multiple interneurons subtypes, and they show that if the total inhibition received by E cells reduces under VIP stimulation then the network is an ISN. What regime is the author's model in? Is this a useful label for their network?
For EI networks, the only term of the response matrix that can flip its sign is M_{II} (as analyzed in [Tsodyks et al., 97, Ozeki et al., 09]). In order to have M_{II} < 0 the network has to be an ISN, so in EI networks having response reversal and being ISN are equivalent.
For a network with multiple interneuron types, the equivalence no long holds. The condition to realize M_{SV} > 0 is W_{EE} > d^{∞}_{E,}therefore the network has to be an ISN. However, the sign of other entries of the response matrix does not depend on whether W_{EE} is larger or smaller than d^{∞}_{E}, meaning that in general response reversal is not related to ISNs.
In order to clarify this point we have extended the paragraph of the Discussion where we mention ISNs (fifth paragraph). We have also added a short Materials and methods section analyzing the response matrix for EI networks.
3) In Figure 2D, M_{EE} is negative (0.35), and if we understand things correctly, it's always negative in the high gain regime. Thus, in the high baseline activity state, an input to the pyramidal neuron population will result in a decrease of pyramidal neuron firing rates. This is at odds with most (all?) data sets. The authors remark on this feature in the last paragraph of the subsection “Circuit behavior explained by response matrix”, but do not address the plausibility of this prediction. Do the authors think this result is a problem for their model? More generally, with new parameters can the authors explain the sign flip in M_{SV} without a sign flip in M_{EE} or are these tethered together somehow?
The negative value of M_{EE} in the high baseline scenario is not a feature of the model, but of the particular matrix that we showed. In fact, it is easy to find other matrices for which M_{EE} is always positive.
In order to avoid confusions we have changed the matrix that we used for figures 1 and 2 so that in the current version M_{EE} is always positive.
4) Dipoppa et al. 2016 is important for justifying the model and the authors cite it frequently (and even republish some of its figures). But this paper has not been peerreviewed (it's a Biorxiv report), giving it the same veridical status as a personal communication or SFN abstract. It's not appropriate for a peerreviewed manuscript to depend on data that has not been reviewed. In addition, we're not sure how this will affect Dipoppa et al.'s attempts to get their work published. eLife is peer reviewed, and many journals won't let you republish work that's already published in a peer reviewed journal. In this manuscript, the authors actually take figures out of the other group's nonreviewed preprint and publish them in their own paper.
It seems to us that Dipoppa et al. is not absolutely essential; Figure 4E could be dropped without affecting the paper much. If the authors do want to include it, they should do two things. First, they should make it crystal clear that Dipoppa et al. is not peerreviewed, every single time the citation is made. They can leave no doubt in the readers' minds that data is not yet part of the scientific literature. Second, they should get permission from Dipoppa et al. before publishing their data. We're guessing eLife requires this, but even if it doesn't, it's not worth irritating one's colleagues for something that is not essential to one's story.
We would like to thank the reviewers for this important remark. Following their advice, we have removed Figure 4E from the main text and we have included it as a supplement to Figure 4 (Figure 4—figure supplement 1). Furthermore, we have explicitly mentioned that [Dipoppa et al., 16] is a preprint whenever we cited it in the text.
We have also the explicit permission of Dipoppa and his collaborators to present their data in our manuscript.
https://doi.org/10.7554/eLife.29742.sa2Article and author information
Author details
Funding
Office of Naval Research (N000141712041)
 XiaoJing Wang
Science and Technology Commission of Shanghai Municipality (14JC1404900)
 XiaoJing Wang
NIH Blueprint for Neuroscience Research (R01MH062349)
 XiaoJing Wang
Science and Technology Commission of Shanghai Municipality (15JC1400104)
 XiaoJing Wang
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Acknowledgements
This work was supported by the NIH grant R01MH062349, the ONR grant N000141712041, STCSM grants 14JC1404900 and 15JC1400104.
Senior Editor
 Timothy E Behrens, University of Oxford, United Kingdom
Reviewing Editor
 Peter Latham, University College London, United Kingdom
Publication history
 Received: June 19, 2017
 Accepted: December 2, 2017
 Accepted Manuscript published: December 19, 2017 (version 1)
 Version of Record published: January 22, 2018 (version 2)
 Version of Record updated: August 2, 2018 (version 3)
 Version of Record updated: March 13, 2020 (version 4)
Copyright
© 2017, Garcia del Molino et al.
This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.
Metrics

 2,238
 Page views

 454
 Downloads

 12
 Citations
Article citation count generated by polling the highest count across the following sources: Crossref, Scopus, PubMed Central.