Firing rate homeostasis via gain adaptation (cartoon examples).

(A) Gain adaptation after a shift in environmental statistics, without changes in tuning curve shapes. The bottom row shows the stimulus distributions in the first (left column) and the second(middle and right columns) environments. The top row shows the tuning curves of the same four neurons (main plots) and their mean firing rates (insets) when adapted to the first (left) and the second (right) environments, as well as immediately after the transition, but before adaptation to the new stimulus statistics (middle). Immediately after the transition the mean firing rates differ from their values in the first environment (middle and left insets). Neurons adapt to the new environment solely by independently adjusting their gains, i.e., by scaling their tuning curves up or down (correspondingly, the tuning curves of the same color in the middle and right columns — which belong to the same neuron — are scaled versions of each other and have identical shape), to a degree appropriate for returning their stimulus averaged firing rates to the level before the environmental shift (right and left insets). (B) Gain adaptation after a change in tuning curve shapes, due, e.g., to a change in the system’s computational goals. The plots have the same format as in (A). In this case, the stimulus distribution (bottom row) does not change between the two contexts, but the tuning curves have different shapes in the second context (top row, second and third columns vs. the first). Again, the average firing rates change immediately after the transition (middle vs. left insets), but the neurons adapt their gains to the right degree to return their rates to the level before the context changed (right inset).

The assumptions and the generative model underlying the theoretical framework.

The environment gives rise to a stimulus distribution P from which the stimulus s is drawn. Conceptually, the brain performs some computation on s, the result of which are represented by neurons via their normalised tuning curves, Ωa(s), which we thus refer to as representational curves. These are multiplied by adaptive gains, ga, to yields the actual tuning curves ha(s). The single-trial neural responses are noisy emissions based on ha(s). We assume the only function of gain modulation is to mitigate the eventual corruption of computational results by neural noise, subject to metabolic energy constraints.

The effect of changing the gain on tuning curves and different components of the objective function.

(A) As the gain, g, of a neuron is increased, the shape of its tuning curve remains the same, but all firing rates are scaled up. The inset shows the gain-independent representational curve, which we define as the tuning curve normalised to have a peak value of 1. (This cartoon shows a one-dimensional example, but our theory applies to general tuning curve shapes and joint configurations of population tuning curves, on high-dimensional stimulus spaces.) (B) Cartoon representation of the efficient coding objective function. Optimal neural gains maximise an objective function, ℒ, which is the weighted difference between mutual information, I, capturing coding fidelity, and metabolic cost, E, given by average population firing rate.

The structure of pre-modulated average population responses in the simulated environments.

We simulated a one-parameter family of environments parameterised by ν ∈ [0, 1]. In the environments on the two extremes, for each unit, the gain-independent average normalised responses ωa(ν = 0) and ωa(ν = 1) are randomly and independently sampled from a Beta(1, 6) distribution, with density shown on the left and right panels. For intermediate environments (i.e., for 0 < ν < 1), ωa(ν) is then obtained by linearly interpolating these values (middle panel). The middle panel shows K = 10000 such interpolations, with every 500th interpolation (ordered by initial value) coloured black.

Optimal gains under uncorrelated power-law noise lead to homeostasis of rates.

(A) Illustration of power-law noise. The trial-to-trial variance of spike counts, as a function of the trial-averaged spike count, is shown for different values of the noise scaling parameter α. (B) Distribution of average firing rates before and after a discrete environmental shift from ν = 0 to ν = 1 for the case α = 1/2. Top: the histogram of firing rates adapted to the original environment (ν = 0) before the shift, as determined by the optimal gains Eq. (5) in this environment, . Middle: the histogram of firing rates immediately following the environmental shift to ν = 1, but before gain adaptation; these are proportional to . Bottom: the histogram of firing rates after adaptation to the new environment, proportional to . We have normalised firing rates such that the pre-adaptation firing rate distribution has mean 1. (C) Deviation from firing rate homeostasis for different values of α as a function of environmental shift. For each environment (parametrised by ν), we compute the optimal gains and find the adapted firing rate under these gains. The average relative deviation of the (post-adaptation) firing rates from their value before the environmental shift, , is plotted as a function of ν.

Firing rate homeostasis under Poissonian noise with uniform noise correlations.

(A) Illustration of an effective noise correlation matrix with uniform correlation coefficients; the (green) off-diagonal elements have the same value, p < 1, while the (black) diagonal elements are 1. Panels B and C have the same format as in Fig. 5.

Firing rate homeostasis under signal-aligned noise.

(A) Eigenspectra of noise covariance matrix for the aligned noise model for various values of γ; in this model, the n-th eigenvalue of the average noise correlation matrix, W, is proportional to 1/nγ. Panels B and C have the same format as in Fig. 5.

Optimal gain adaptation in a clustered population accounts for the broad distribution of cortical single-neuron firing rates.

(A) A cartoon description of the clustered neural population model. Top: The single neurons in the population belong to different functional groups or clusters (shown by different colors). Bottom: The normalised tuning curves of the single neurons belonging to the same cluster (thin lines of the same color) are very similar, and close to their cluster’s stereotype normalised tuning curve (solid lines) which can be very different across clusters. (B) Each panel shows the distribution of log firing rates of the active single neurons for different choices of cluster size k and effective dimensionality, D, of the within-cluster tuning curve variations. The distributions shown are for the active neurons with non-zero firing rate, with their percentage of the total population given in each panel (purple text). For all simulations we fixed the cluster firing rate to k × 5 Hz (corresponding to µ = 10, if we assume a coding interval of 0.1 s), such that the mean rate of single neurons is 5 Hz on average.

Accuracy of homeostatic approximation to optimal gains.

(A) The relative error, Eq. (15), of different approximate solutions for the optimal gains, averaged across environments (the standard deviations of these values across environments were negligible relative to the average), as a function of the noise scaling parameter α. To give a sense of the scale of variation of the optimal gains across environments, the black line shows the relative change in the optimal gains between the most extreme environments (a measure of effect size), . (B) Relative improvement in the objective for the homeostatic approximation ghom as measured by Eq. (16). Panels C and E (panels F and D) are the same as panel A (panel B), but for the uniformly correlated noise and the aligned noise models, respectively.

Schema of the generative and recognition models underlying a DDC.

According to the internal generative model, sensory inputs, s, are caused by latent causes, z, that occur in the environment according to a prior distribution π(z). A conditional distribution f (s | z), the so-called likelihood function, describes how the latent causes generate or give rise to the sensory inputs s. The task of the brain is to invert this generative process by inferring the latent causes based on the current sensory input, which is done by computing the posterior distribution Π(z|s).

Components of the model for stimulus-specific adaptation in V1.

(A) The distribution of stimulus orientations used in the experiments (Benucci et al., 2013), for an adaptor probability of 30% is shown in blue. We assume V1’s internal model use continuous prior distributions and thus we used instead a mixture of the uniform distribution with a smooth Gaussian, with standard deviation σπ, centered at the adaptor orientation (red curve). (B) The baseline tuning curves, i.e., the tuning curves adapted to the uniform orientation distribution. The blue curve was obtained by averaging the experimentally measured tuning curves adapted to the uniform orientation distribution, after centering those curves at 0°. The model’s baseline tuning curves (red) are given by the convolution of the Gaussian likelihood function, with width σf, of the postulated internal generative model, and the Gaussian DDC kernel, with width σϕ, and thus are themselves Gaussian with width . We fit this quantity to the experimentally observed baseline curves as described in Sec. 4.5.

A homeostatic DDC model accounts for the observations of stimulus-specific adaptation in V1 Benucci et al. (2013).

Panels A-C (left column) and E-F (right column) correspond to the experimental results and the predictions of our model, respectively. (A, D) The tuning curves for the adapted (red) and unadapted (blue) populations, averaged across experimental conditions. (B, E) The trial-averaged firing rates of the adapted (red) and unadapted (blue) populations exposed to the non-uniform stimulus ensemble. The trial-averaged population responses to the uniform stimulus ensemble were normalised to 1. (C, F) The repulsion of preferred orientations, obtained from the average tuning curves in panel A, as a function of the deviation of the neurons’ preferred orientation from the adaptor orientation.

Table summarising empirical findings from four papers which estimate a scaled power-law relationship relationship between mean and variance of cortical spike count responses.

To find the values in the last column we adjust the reported to account for different coding intervals and cluster sizes used in each experiment (see Eq. (31)), and assume βµ = 10.

Empirically estimated values of the noise scaling parameter β and the base noise-level σ2, after adjusting for coding interval duration and cluster size using the formula Eq. (31).

The dashed lines show the values for Poissonian noise (σ2 = 1, β = 1).

Distribution of average firing rates before and after a discrete environmental shift for the uncorrelated power-law noise subfamily.

Each panel has the same format as in Fig. 5B, but for different values of the noise scaling parameter α. The case α = 0.5 is shown in Fig. 5B.

Distribution of average firing rates before and after a discrete environmental shift for the Poissonian noise subfamily with uniform noise correlations.

Each panel has the same format as in Fig. 6B, but for different values of the noise correlation coefficient p. The case p = 0.2 is shown in Fig. 6B.

Distribution of average firing rates before and after a discrete environmental shift for the aligned Poissonian noise subfamily.

Each panel has the same format as in Fig. 7B, but for different values of the noise spectrum decay parameter γ. The case γ = 1.5 is shown in Fig. 7B.

Accuracy of homeostatic approximation to optimal gains.

The plots in the panels A, C, and E show the relative improvement in the efficient coding objective for the zeroth-order approximation for the optimal gains g(0), Eq. (11), for the three noise sub-families; see the caption of Fig. 9 for a description of panels B, D, and F there, respectively. Plots in panels B, D, and F here similarly show the relative improvement in the efficient coding objective for the first-order approximation g(1), Eq. (106), for the three noise sub-families.

Systematic exploration of how the firing rate distribution of individual neurons varies as the statistical properties of cluster responses change.

The panels here have the same description as those in Fig. 8, but show the single-neuron firing rate distributions for different choices of cluster size k and effective tuning curve dimension, D. (Note that the upper-right panels corresponding to cases in which D < k are left empty; this is because in such cases the covariance matrix is singular, and therefore the quadratic program does not have a unique solution. Such cases are also less relevant biologically given realistic estimates for the size of the similarly tuned neurons vs. the effective dimensionality of tuning curves, noting that the nominal dimension of the tuning curve function space is infinite.)