To measure how peripheral motion changes the response of ganglion cells, we presented a stimulus to simulate image movement in the retina due to two different conditions: a moving object in a static scene or global motion as would be caused by movement of the eye or a large object. We chose a set of brief stimuli that could produce a wide variation in excitation – from extremely weak to very strong – depending on a cell’s location, to determine the effect of peripheral excitation, and how that effect varies with the cell’s central excitation. The stimulus consisted of a central square object with a constant luminance in front of a checkerboard peripheral pattern. The object covered the classical linear receptive field center plus part of the surround of most cells (Figure 1A, top panel). To present the same central stimulus in the presence and absence of strong peripheral stimulation, either the object alone (object shift) or the whole stimulus (global shift) was shifted abruptly by one peripheral square, 50 µm in length, ~1 degree of visual field in the salamander. We then varied the central luminance level to measure the effect of the strong peripheral stimulus in encoding the luminance of the central stimulus. When the object moved alone, many cells showed transient activity, which depended on the location of the cell relative to the object border (Figure 1B, left column). As expected, those cells near the center of the object showed the weakest activity because they experienced little change in light intensity, and overall, the response was insensitive to the central luminance value. In the global shift condition, however, brief strong firing events occurred during both right and left shifts for most cells including those in the center of the object (Figure 1B, right column). To assess the effect of the peripheral shift on the ability of a cell to distinguish different central stimuli, we computed the slope of a line fit to the firing rate as a function of the log of central intensity, and found this slope to be much greater during the global shift (Figure 1C). We examined which cells were most strongly affected by the periphery by comparing responses to all constant luminance objects under both peripheral conditions. In doing so, we found that the cells with the weakest response to the object condition showed the greatest enhancement of sensitivity from peripheral motion, with 39 out of 76 cells at least doubling the slope of their firing rate vs. the log of central luminance (Figure 1D). This increased activity during the global shift condition could not be attributed to the linear receptive field of the cell overlapping the object border, as steps to the right and left would have linear contributions of opposite signs, even if such stimuli might be within the spatial region occupied by the classical receptive field surround (Demb et al., 2001; Borghuis et al., 2013). Accordingly, the effect of the global shift was mostly insensitive to the phase of the checkers in the periphery (see Materials and methods). Thus, peripheral stimulation enhanced the sensitivity to weak central stimuli during abrupt global image motion.

Figure 1

Download asset Open asset

To analyze the dynamics of the effects of peripheral stimuli on the processing of central input, we decoupled the central and peripheral inputs by presenting a stimulus with a continuously flickering light intensity in the center, combined with brief peripheral motion. Although this stimulus differs from a global image shift, which simultaneously changes central and peripheral regions as in (Figure 1), the independent control of central and peripheral stimuli allowed us to assess the dynamics of how the periphery changes the encoding of a central stimulus. To create a local naturalistic input, the central object intensity flickered with a temporal power spectrum resembling natural scenes, which was inversely proportional to the frequency – called ‘pink noise’ (Simoncelli and Olshausen, 2001) (Figure 2A, right panel inset). The periphery was a checkered pattern that was either still, producing no temporal input, or reversed every 0.5 s, producing strong synchronized peripheral stimulation. For most recorded cells, shortly after peripheral stimulation there was an excitatory effect – indicated by an increase in firing (Krüger and Fischer, 1973) – followed by strong inhibition as might underlie a previously reported component of saccadic suppression (Roska and Werblin, 2003), and then a slower recovery to the baseline state (Figure 2B and Figure 2—figure supplement 1A). This effect also occurred with both left and right shifts of the checkers (Figure 2B, inset), indicating that this effect was primarily not caused by the classical (linear) receptive field surround, which would have produced opposite effects for the two background phases.

Figure 2 with 1 supplement see all

Download asset Open asset

The presence of excitatory input, however, does not reveal how peripheral input changes the neural code for central input. For example, peripheral excitation could potentially saturate the cell, and thus mask central input. We therefore measured the sensitivity to the central region using a linear-nonlinear (LN) model (see Materials and methods), consisting of a linear temporal filter representing the average feature preferred by the cell followed by a static nonlinearity capturing the cell’s average threshold and sensitivity, defined as the average slope of the nonlinearity (Chichilnisky, 2001; Baccus and Meister, 2002). In this case, although the LN model does not capture all of the nonlinear dynamics of the receptive field center (Gollisch and Meister, 2010) (some of which is captured in a model below), the nonlinearity can be used as a statistical measure of sensitivity to the central stimulus (Kastner and Baccus, 2011; Baccus and Meister, 2002; Rieke, 2001; Zaghloul et al., 2005) at different times relative to the peripheral shift.

To observe changes in the neural code associated with peripheral excitation, we computed two LN models: one when the periphery was still and the second one under the shift condition using only spikes from a 50 ms gating window centered on the peak of excitation (Figure 2C). Even though the firing rate greatly increased during 50–100 ms after the peripheral shift, the temporal filter changed little when compared to the still condition, indicating that the cell continued to convey the same average feature about the visual stimulus during the 50 ms high firing rate interval (Figure 2C). We defined the sensitivity to the central stimulus as the average slope of the nonlinearity (Kastner and Baccus, 2011; Baccus and Meister, 2002) and found that the sensitivity was the greatest 50–100 ms after a peripheral shift, during the high firing rate window. This indicates that the increase in firing rate after the peripheral shift is not due to a response independent of the center stimulus, as such an effect would have shifted the nonlinearity vertically, leaving the sensitivity to the center unchanged (Figure 2C, dashed line). Instead, we find that an abrupt peripheral shift dynamically gates the response of a cell, enhancing the cell’s sensitivity to its preferred visual feature near the receptive field center.

However, the amount of information conveyed is influenced not only by sensitivity but also by noise, and thus an increase in sensitivity does not necessarily imply an increase in information transmission. Therefore, to confirm that the increased activity and sensitivity were accompanied by an increase in transmitted information, we estimated the mutual information between the stimulus sequence in the object region and the cell’s response as measured by the spike count at different times relative to the peripheral shift. This quantity is $I\left(R;C|p\right)$, the mutual information between the response, R, and the central intensity, C, given a particular peripheral stimulus $p\in P$, where p is the time relative to the peripheral shift (see Materials and methods).

We found that just after a peripheral shift, information about the central stimulus sharply increased as compared to when the periphery was still, indicating that a signal from the periphery increases information transmission from the central region (Figure 2D, Figure 2—figure supplement 1B and C). After this increase, information transmission then decreased (100–300 ms after the global shift) and then recovered to the baseline state. All cell types showed a sharp increase of information during the gating window, with the peak time depending on the cell type (Figure 2E). The leftward shift of the nonlinearity (Figure 2C) increased the slope of the nonlinearity and information transmission because the threshold of the nonlinearity in the baseline condition was positioned to the right of the mean stimulus, which is the case for ganglion cells of diverse species including mammalian and primate retina (Chichilnisky, 2001; Keat et al., 2001).

In this analysis, by considering $I\left(R;C|p\right)$ we are taking the more traditional point of view that cells encode signals in the center of their receptive fields and are modified by other (peripheral) signals. However, one could argue that ganglion cells are actually encoding the periphery and being modified by the center. The total information between the response and both central and peripheral stimuli, $I(R;P,C)$ can be separated into two components by the chain rule for mutual information (Cover and Thomas, 1991) (see Materials and methods).

However, on average, for all cell types studied, the information that the cell carried about the peripheral stimulus $I\left(R;P\right)$ was substantially smaller than the information the cell carried about the center given the peripheral stimulus ($I\left(R;C|P\right)$), averaging 27% of $I\left(R;C|P\right)$ (Figure 2F). These results support the view that peripheral input gates information transmission from the central region.

Although it may seem puzzling that little information is conveyed about the periphery even though there is a large timed increase in the average firing rate, this is because the large peak at ~100 ms represents only the average response to the stimulus. On any given trial, the decision to fire is primarily controlled by the central input and in many cases, no spikes occurred when the periphery moved. Nonetheless, the brain could use a population of cells to identify the gating window as a time when activity increases synchronously throughout the retina. Our analysis using the number of spikes in a time bin neglects more complex encoding due to latency (Gollisch and Meister, 2008) and firing patterns in the population (Schnitzer and Meister, 2003). Yet, this analysis suggests that the brain could extract this increased information simply by counting spikes, without the need for a more complex decoding scheme across time bins or using the population.

We then compared the amount of information transmission with the theoretical maximum given the cell’s stochastic firing properties. We assessed the theoretical maximum by computing the sigmoidal nonlinearity that maximized information about the stimulus, given a cell’s maximum firing rate and measured spiking noise as defined by its spike-count distribution for a given average response, $P\left(r|⟨r⟩\right)$. The average firing rate however was not constrained at a particular value, in contrast to previous studies, meaning that a leftward shift of the nonlinearity with the same maximum would generate a higher average firing rate (Pitkow and Meister, 2012) (see Materials and methods). After a shift, the information conveyed was 0.47 ± 0.04, (n = 18 cells) of the maximal value, compared with 0.24 ± 0.05 before the shift. Thus, after the shift, the neural code used more of the capacity of the cell given its noise properties and the spike count code. However, this came at the increased cost of energy in terms of a higher firing rate (6.9 ± 1.7 Hz after the shift, and 1.9 ± 0.3 before the shift). Previous results indicate that the high threshold of ganglion cells allows them to conserve spikes at the expense of maximal information transmission (Pitkow and Meister, 2012). Our analysis indicates that after a peripheral shift, the neural code shifts away from energy conservation and towards high-throughput information transmission.

To examine which properties of the neural code changed between the shift and still conditions, we presented a Gaussian white noise sequence with a fixed mean and different contrasts, defined as the standard deviation divided by the mean. This allowed us to compute and compare separate LN models for each contrast. We observed that both excitation and inhibition from a peripheral shift depended on the central contrast, with a much stronger increase in firing observed at low contrast (Figure 3A). This result is consistent with the observation that cells with the weakest response to a moving square had the strongest effect from peripheral stimuli (Figure 1). We then fitted LN models as stated above at different contrasts during 50 ms time windows corresponding to gating, suppression, and recovery (Figure 3B–C). As the contrast in the central region decreased, the filter in some cells became slower and more monophasic as previously reported (Baccus and Meister, 2002). However, although at low contrast, the gating window’s firing rate exceeded the recovery window’s firing rate by more than 20-fold, the filters were markedly similar. Thus, peripheral stimulation affected the sensitivity, but had a minimal effect on the average local features preferred by the ganglion cells.

Figure 3 with 1 supplement see all

Download asset Open asset

Changes in sensitivity are also known to be caused by adaptation to the local contrast. We therefore tested how peripheral stimulation influenced adaptation to the central contrast by measuring changes in adaptation as a function of time since the peripheral shift. For an ideally adapting cell, the sensitivity would scale in inverse proportion to the contrast (Figure 3—figure supplement 1A–B). Previous results, however, have shown that ganglion cells adapt less than this ideal amount, in particular at low contrasts (Ozuysal, 2012). We found that in the gating window, responses to different contrasts were much more similar to each other, indicating a greater level of adaptation to the central contrast (Figure 3C). This effect arose because at low contrast, the slope of the nonlinearities changed more than at high contrast, similar to the stronger effects of gating seen with cells that responded more weakly to a shifting object (Figure 1D). At 3% contrast, the slope changed by 5.5 ± 1.1 Hz per contrast unit (one s.d. of the nonlinearity input was 0.03 contrast units, or 3% ) and at 24% contrast the slope changed by 0.20 ± 0.06 Hz per contrast unit (one s.d. was 0.24 contrast units, or 24% ) (Figure 3—figure supplement 1C). As a result of these effects at different contrasts, during the gating window nonlinearities reached a more similar height across contrasts. Accordingly, when normalized by the stimulus standard deviation at each contrast, nonlinearities also had a more similar shape across contrasts in the gating window. We computed an index of adaptation that takes the value of 1 for an ideally adapting cell and 0 for a non-adapting cell (see Materials and methods and Figure 3—figure supplement 1A–B), and found that most cells increased their adaptation to contrast during the gating window, such that all contrasts were represented with more similar responses than during the recovery window (Figure 3D). This indicates that an increase in adaptation underlies the increase in information during the gating window, such that near the receptive field center, both weak and strong signals are conveyed.

Cells exhibiting contrast adaptation – by changing their sensitivity when the contrast changes – will increase information about fluctuations in intensity, but potentially lose information about the contrast itself (Fairhall et al., 2001). Because many cells showed increased adaptation after a peripheral shift, we tested whether the cells encoded different properties of the stimulus – the sequence of light intensities and the contrast – at different times relative to a peripheral shift. We designed a flickering stimulus that had a relatively small number of conditions to facilitate the estimation of information about the stimulus sequence and/or the contrast. The center followed a binary white noise M-sequence, at four possible contrasts $\sigma$, where the instantaneous intensity value was $\mu +\sigma \cdot m$ and $\mu$ is the mean intensity, fixed throughout the experiment and $m=\pm 1$ are the instantaneous values of the M - sequence (see Materials and methods). All combinations of binary sequences (up to four frames, lasting 400 ms, $m^{\left(4\right)}\in M^{\left(4\right)}$, contrasts and times relative to peripheral excitation were presented an equal number of times (see Materials and methods and Figure 4A). Figure 4B shows raster plots for one-cell ordered according to contrast and mixing the responses to all M - sequences. We estimated the mutual information between the response and two stimulus parameters at different times relative to the peripheral shift – the light intensity sequence in the previous four frames ($M^{\left(4\right)}$), $I\left(R;M^{\left(4\right)}|p\right)$, and the center’s contrast $(\sum )$, $I\left(R;\sum |p\right)$, where $\sigma \in \sum$ (Figure 4C). We found that the responses coming from the same cell at different times carry different types of information. When computing $I\left(R;M^{\left(4\right)}|p\right)$, the analysis was conducted as if the brain was decoding the stimulus sequence without knowing the contrast. The results were similar, however, if we considered that the brain might decode the contrast, and use this knowledge to better decode the stimulus sequence (eq (10) and Figure 4—figure supplement 1). Whereas a static neural code would typically face the choice between adaptation and preserving the adapting statistic, a dynamic neural code avoids this tradeoff by rapidly switching between complementary representations of the same stimulus.

Figure 4 with 1 supplement see all

Download asset Open asset

To identify the minimum components required to produce the dynamic neural code we observed, we began with the known structure of excitatory input to a ganglion cell consisting of a central pathway comprising a linear filter, a static nonlinearity and an adaptive stage. This central pathway model was analogous to an accurate model of contrast adaptation (Ozuysal, 2012), where the rectified nonlinearity and adaptation likely occur at the bipolar cell synaptic terminal, although here we used a simplified adaptive stage. Nonlinear peripheral input is delivered only in the inner retina, as horizontal cells do not respond to fine gratings (Baccus et al., 2008). Therefore, the only remaining choice in the model structure was the level at which to combine the peripheral input. Because a peripheral shift caused the overall nonlinearity of the LN model to shift laterally, rather than vertically with respect to the central pathway’s linear input (Figures 2 and 3), the peripheral pathway delivers input prior to the nonlinearity, corresponding to input to the bipolar cell terminal (Figure 5A). The peripheral pathway can be represented by small rectified subunits that cause the response to be insensitive to the peripheral pattern (Victor, 1979; Olveczky et al., 2003). Rather than explicitly simulate spatiotemporal dynamics of the periphery, we modeled the net effect of the abrupt peripheral stimulus via a timed biphasic signal. The model effectively replicated the data for Gaussian stimuli at different contrasts (Figure 5C, D), as well as for constant luminance stimuli (Figure 5—figure supplement 1).

Figure 5 with 1 supplement see all

Download asset Open asset

A key aspect of the model is the order in which the signals are combined. Because the peripheral input is delivered prior to the threshold and adaptive stage, it is summed with the unadapted measure of the central input. This causes the peripheral input to have a larger effect on weak central stimuli than on strong ones as experimentally observed (Figures 1 and 3). Furthermore, weak stimuli only cross threshold when peripheral input is applied (Figure 5B), allowing the adaptive stage to encode and adapt to signals of all strengths, including those from a central stimulus with a constant intensity (Figure 5—figure supplement 1B). Thus, when peripheral excitation is present (Figure 5B, Figure 5—figure supplement 1A), the model applies a lower threshold relative to the central input, conveying more information about the full range of contrast environments, but less information about the contrast level. When peripheral input is absent or inhibitory, the model applies a high threshold with respect to the central input (Figure 5B and Figure 5—figure supplement 1A), conveying primarily higher contrasts and encoding information about the contrast level.

An important conclusion we derived from the model is that a single amplitude of peripheral input (equivalent to a fixed threshold shift with respect to the central input) replicates the results at all central contrasts. Thus even though the effects of peripheral input differ with the central contrast, this is because of the differing downstream effects of adaptation; the peripheral signal itself delivered prior to the threshold does not depend on the central contrast.

We then measured the scale of excitation and compared it to the scale of transient peripheral inhibition, which is thought to play a role in suppressing the effects of eye movements (Olveczky et al., 2003; Roska and Werblin, 2003). To measure the distance over which lateral excitation acts, we designed a stimulus to measure peripheral gating of sensitivity to a central object as a function of distance from the peripheral stimulus. The stimulus had three different components. First, the periphery was composed of 50 μm checkers that covered the whole screen. Second, on top of the periphery, a mean intensity gray mask covered the peripheral checkers over the central region; the size of the mask, L, was varied. Third, the central object consisting of a pink noise flickering sequence was then added on top of the central gray region and was presented as a fenestrated checkerboard pattern of fixed size (Figure 6A–B). By decreasing the mask size, more peripheral checkers were present, and the distance between peripheral checkers to any measured cell was decreased. At the smallest mask sizes, peripheral checkers were intercalated with the central region object, and occupied all space not covered by the central object (Figure 6B, bottom). The excitatory influence of gating acted at distances of up to 1 mm (Figure 6C). This further confirms that the effect was distinct from the linear receptive field, which would not be activated by a fine checkerboard at this distance. The observed changes in sensitivity indicate that excitatory and inhibitory influences acted over a similar scale, equivalent to a radius of ~20 deg. of visual angle.

Figure 6 with 1 supplement see all

Download asset Open asset

Our results indicate that transient peripheral input increases information transmission about the visual feature in the receptive field center. However, for natural images and an abrupt global shift in the image as from motion of a large object, the eye or head, the transient effect of gating would be combined with transient changes in image statistics created by the image shift. We therefore compared the timing of our gating results (Figure 2) to the dynamics of the expected information transmission for natural scenes during an abrupt shift in the image. The statistics over the receptive field center depend both on the image and the sequence of eye movements, and have strong correlations over time. Consequently, attaining representative statistics of the distributions of both light intensity and responses over multiple time points requires many trials. Owing to the difficulty in sampling the high dimensional distribution of the stimulus and responses over multiple time points (see Materials and methods), such an experiment would be prohibitively long. We therefore addressed this question using a spatiotemporal version of our model of gating with simulated eye movements and a large number of natural images.

To estimate the timing of information transmission under natural images, we combined natural images with simulated global shifts of the image (Figure 7A-i). The spatiotemporal model used for fast Off-type ganglion cells had the same structure as the reduced model (Figure 5), but we replaced the initial linear filter with the spatiotemporal receptive field measured from a fast Off-type bipolar cell (Baccus et al., 2008). Because peripheral gating is delivered prior to the threshold in the model, it is likely that it represents an input presynaptic to the bipolar cell terminal. Furthermore, because strong adaptation to contrast is thought to arise in the presynaptic terminal, the model is effectively of the synaptic release from bipolar cells, although the actual density of bipolar cells was not modeled. Thus, although further nonlinearities exist in the inner retina that would make the responses of ganglion cells to natural scenes more complex, we expect this model will be informative as to the relative timing of bipolar cell release and peripheral gating. Fast Off-type bipolar cells are roughly linear at a constant mean intensity for a stimulus that flickers (Figure 7—figure supplement 1) or jitters as in fixational eye movements, and previous models indicate that these cells may convey the primary input to fast Off-type ganglion cells (Baccus et al., 2008). Because the model was used to estimate the information transmitted after a global image shift, we measured the noise in bipolar cells at different contrasts. The signal-to-noise ratio (SNR) increased with contrast, which was incorporated into the model ( Figure 7A-iii, see Materials and methods).

This model does not capture all nonlinearities of the bipolar cell response, including luminance adaptation, a slightly saturating nonlinearity for high contrast stimuli, and weak contrast adaptation. Furthermore, because we did not include luminance adaptation, the model effectively assumes that during the fixation period, adaptation has reached a steady state, and that the global shift is too brief to cause substantial luminance adaptation. The main goal of the model, however, was to gain insight into the dynamics of information transmission under sudden image shifts.

The input to the model consisted of a set of natural images combined with fixational drift eye movements interrupted by a sudden image shift of 6 degrees (Figure 7A), sufficient to exceed most local image correlations (Figure 7—figure supplement 1A). Because natural images have strong correlations, and because bipolar cell receptive fields are biphasic both in space and time, fixational drift created a relatively small change in the filtered stimulus (Figure 7A-ii). However, for a brief window of time after the shift, the light intensity seen by the bipolar cell was less correlated with its previous values (Figure 7B, D and F). During this window, a wider range of possible filtered stimuli occurred, but still with most values remaining close to zero and under the fixed threshold (Figure 7A-ii, B). Based on the noise measurements in bipolar cells, because of the higher variance after the shift, the SNR of the bipolar cell membrane potential would be expected to increase. Note that these simulations do not include the effects of moving objects in the environment, which would cause the stimulus to more closely resemble experiments in Figures 2–4, where central and peripheral inputs are uncorrelated.

Figure 7 with 2 supplements see all

Download asset Open asset

We first estimated the mutual information between the set of linearly filtered stimuli, $G=\left\{g\left(t\right)\right\}$, and the model’s response, $R=\left\{r\left(t\right)\right\}$, as a function of time from the shift (Figure 7C, dashed line). Because of strong correlations in the stimulus (discussed further below), sequential measurements of the intensity in a new environment are largely redundant, and thus may not convey added information. To account for this effect, we estimated the novel information learned about the stimulus from a response given that the system has access to the previous response. This is the conditional mutual information $I\left(G_t;R_t|R_{t-\Delta },p\right)$ between the set of linear predictions $G_t$ and the responses$R_t$ at the same time t given the response at the previous time point $R_{t-\Delta }$, computed for a given delay p from the shift (see Materials and methods). As expected, because during fixational eye movements responses are highly correlated in time, each additional sample conveyed little novel information. However, after the sudden shift moved the receptive field to a new location, the conditional mutual information abruptly increased (Figure 7C, solid line). Unlike the mutual information at different times relative to the shift, $I\left(G;R|p\right)$, the conditional mutual information captures the intuition that most of the information about the new environment should occur in a short amount of time (Figure 7—figure supplement 2). Most importantly, the conditional mutual information showed faster dynamics (Figure 7C, continuous black line), with new information arriving faster than the peak of the linear prediction (Figure 7C, green line).

We then compared the timing of the increase in information from gating (Figures 2–3) with the timing of the expected increase in the conditional information following the shift (Figure 7D). We found that these greatly overlapped, meaning that the active increase in information produced by gating is timed to match the expected increase in novel information generated by the shift. Given the statistics of natural scenes and the measured noise in bipolar cells, our model indicates that gating represents a mechanism that increases information transmission at the expected peak of novel information after a global shift in the image.

Adaptation is typically considered to be a process that optimizes information transmission given the current environment, and previous studies have focused on which threshold response curve would maximize information in the current environment (Laughlin, 1981; Brenner et al., 2000). However, it is clear that information transmission is not the only objective, as the threshold of retinal ganglion cells is much higher than predicted by this ideal. Consequently, it has been proposed as an alternative factor that ganglion cells conserve spikes at the expense of maximal information transmission (Pitkow and Meister, 2012). We propose that neither view of a neural code optimized for a single current environment – either for maximal information transmission or for energy efficiency – is fully representative of natural vision. Our findings indicate that a peripheral shift causes a switch from a code that conserves energy to one of increased information transmission, with higher information transmission occurring at the expected time of higher signal-to-noise ratio and higher information content. These results suggest a general principle of neural coding – the dynamic allocation of neural activity to times most likely to contain novel information. This principle of adaptation acts to allocate resources across environments, and in fact is analogous to known principles of communication theory that act to enhance dynamic information transmission under an energy constraint.

An energy-efficient communications channel that carries signals over a range of frequencies should allocate power such that signal power plus noise power is a constant, except for frequencies where noise exceeds a value set by the available power (Shannon, 1998; Warland and Rieke, 1999). This concept of efficient power allocation is known as ‘water-filling’, as suggested by the notion of pouring power allocated to signal transmission into a basin whose depth varies according to noise but whose surface level is constant.

It is less well appreciated in neuroscience that the same water-filling principle applies to efficiently allocate power over time when the noise level dynamically varies as can occur during wireless transmission (Goldsmith and Varaiya, 1997). In this case, greater power should be allocated to times of higher SNR. Because a higher variance signal has a higher SNR in the bipolar cell (Figure 7A-iii [Ozuysal, 2012]), this principle agrees with our observation that additional spiking is allocated to times of expected high variance of the filtered stimulus (Figure 7C–D). However, because both signal and noise are changing dynamically, further studies are needed to compare dynamic changes in information transmission to an estimated optimal allocation of power over time given the changing stimuli and noise.

Our results indicated that gating occurs at the time when the expected statistics of the central input will change. This expectation may arise from the sudden movement of large objects, and we expect that peripheral gating will be important during natural saccadic eye movements. Previous results have strongly implicated transient nonlinear peripheral inhibition in suppressing the effects of fixational eye movements on object motion sensitive ganglion cells (Olveczky et al., 2003), and the similar spatial scale of peripheral nonlinear gating and nonlinear inhibition (Figure 6) is consistent with a role of gating in eye movements.

Although salamanders and other amphibians have differences in their eye movements in that they make head saccades to target prey (Manteuffel and Roth, 1993), they have similar fixational drift (Manteuffel et al., 1977) and optokinetic head movements (Manteuffel, 1984) to mammals. Accordingly, the property of object motion sensitivity related to fixational eye movements is common to both salamanders and mammals. Similarly, the basic phenomenon of peripheral retinal excitation has been observed in mammals, and we expect that effects on neural coding we observe here will be similar. The duration of the ~1 degree global image shifts we have used (Figure 1) is one stimulus frame (~30 ms), similar to the duration of a ~1 degree saccade in a number of species, for rabbit, cat and monkey, 20–50 ms (Collewijn and Zuidam, 1977; Evinger et al., 1981; Fuchs and Johns, 1967); humans, ~20 ms (Baloh et al., 1975); and fish, ~70 ms (Easter, 1975). Although larger saccadic eye movements are longer in duration, the key property we find is that the timing of the linear filter in the receptive field center is coincident with temporal filtering from the peripheral input. Thus, even if the global shift is more smooth as in the case of a larger saccade or an amphibian head saccade, we expect that the excitation from both center and periphery will still coincide.

Timing signals that indicate a changing stimulus have been observed in other systems that use active sensation, including sniffing in olfaction and whisking in the vibrissae system (Shusterman et al., 2011; Hill et al., 2011). In these cases, an efferent copy of a motor command can provide the timing signal. But because the retina lacks such an efferent copy, a signal that the stimulus is changing must be computed from the sensory input. Inhibitory amacrine cells are known to deliver signals laterally across long distances, and could increase the firing rate through synaptic disinhibition (Barlow et al., 1977). We note that a similar organization is found in the hippocampus, where a common signal is generated by oscillations in inhibitory neurons (Buzsáki, 2002). On the principle that the threshold should be lowered when greater information is expected, synchronous oscillations between brain regions may perform a similar function of allocating sensitivity to time intervals of greater information content.

The neural code embodies a choice between tradeoffs. A high threshold may be efficient in terms of energy, at the expense of the amount of information (Pitkow and Meister, 2012). A biphasic filter and a threshold may emphasize novelty in natural scenes (Srinivasan et al., 1982), but certain stimuli such as a constant luminance will be rejected. An adaptive system may improve information transmission across an entire set of stimuli, but the particular statistic that triggers adaptation may be lost. It is commonly assumed that a cell makes a single choice among these alternatives, whatever the benefits and consequences. Our results, however, show that cells can sequentially switch between complementary representations to capture the benefits of both.

Larval tiger salamander retinal ganglion cells were recorded using an array of 60 electrodes (Multichannel Systems) as described (Kastner and Baccus, 2011). Intracellular recordings from bipolar cells were performed using sharp microelectrodes as previously described (Ozuysal, 2012)

A video monitor projected stimuli at 60 Hz, and values of intensity changed at 30 Hz. The monitor was calibrated using a photodiode to ensure the linearity of the display. Stimuli had a constant mean intensity of ~10 mW/m². Contrast was defined as the standard deviation divided by the mean of the intensity values, unless otherwise noted.

To measure the difference between object and global shifts (Figure 1), the stimulus consisted of a square object 1200 µm on a side and a constant luminance of one of four logarithmically spaced values, and was presented in front of a black and white background checkerboard (50-µm squares). Either the object alone or the entire image was suddenly displaced 50 µm left and right at 1 Hz. The experiment was repeated with both phases of the background checkerboard, for a total of 16 combinations of shifts. Each combination was presented for 110 s twice in interleaved format with movements happening every 0.5 s. The first 10 s of each presentation were discarded leaving 200 trials per condition, with an equal number of left and right shifts that were analyzed independently (see Figure 1C).

LN models for Gaussian stimuli (Figure 3) consisted of the light intensity passed through a linear temporal filter, which describes the average response to a brief flash of light in a linear system, followed by a static nonlinearity, which describes the threshold and sensitivity of the cell (Baccus and Meister, 2002). To compute LN models for white noise stimuli, we first computed linear filters, F(t), which were the time-reverse of the spike-triggered average. Then, we calculated linear prediction, g(t), as the convolution of the temporal filter and the central stimulus, s(t),

A static nonlinearity, $N\left(g\right)$, was computed by averaging the value of the firing rate, $r\left(t\right)$, over bins of g(t). The filter, $F\left(t\right)$, was normalized in amplitude such that it did not amplify the stimulus, i.e. the variance of s and g were equal (Baccus and Meister, 2002). Thus, the linear filter contained only relative temporal sensitivity, and the nonlinearity represented the overall sensitivity of the transformation.

For pink noise stimuli (Figure 2), a sequence was generated with an amplitude spectrum that was inversely proportional to the frequency ($1/f$). Because the stimulus contained temporal correlations, the linear filter was computed by reverse correlation while normalizing by the autocorrelation of the stimulus (Baccus and Meister, 2002).

In our experimental designs, the full stimulus $S$ consisted of two components, the periphery, P, and a center stimulus $C$. In all experiments, the periphery was either still (with zero entropy and thus the set of responses R contained no information about it), or reversed at 1 Hz. By discretizing time in 50 ms bins we create 20 different peripheral conditions $p \in P$, each of which represents a time relative to the period of the peripheral stimulus. By the chain rule of information (Cover and Thomas, 1991)

The last term can be understood as the average information that the response carries about the central stimulus if the peripheral stimulus was known. This equation can also be written as (Cover and Thomas, 1991)

where the last term is an average over all instances of the peripheral stimulus $p \in P$.

We computed $I\left(R;C|p\right)$ (the quantity inside the in eq (8)) for each time bin p relative to the peripheral period (Figure 2D, inset) so that for each p there is no information between the cell’s response and the peripheral stimulus (because under the set of stimuli analyzed there is only one peripheral stimulus, which has zero entropy). To analyze how information varies as a function of time relative to the peripheral shift, we show $I\left(R;C|p\right)$ averaged over both phases of the periphery, each of which had a similar time course (Figure 2D, inset).

A 200 s sequence of a Gaussian pink noise (1/f amplitude spectrum) stimulus with an equivalent contrast of 10% was repeated 10–20 times. For the stability of information calculations with this number of repeats, see (Figure 2—figure supplement 1B). In the Still condition, the periphery was static and in the Shift condition the peripheral checkerboard shifted every 0.5 s. As stated above, each time relative to the peripheral period was analyzed separately, so that under each condition there was no information between the response of a cell and the periphery’s position. Although the central sequences were not the same for each time bin relative to the peripheral shift, by having 200 central sequences per periphery we limit the chance of biasing any particular periphery by associating it with more (or less) discriminable central stimuli. Center sequences were identical between the Still and the Shift conditions, and therefore the differences between $I\left(R;C|p\right)$ under still and shift for any peripheral stimulus p is only attributable to the one difference in the experimental conditions, the presence or absence of peripheral stimulation. The response, r_i, during trial i was defined as the number of spikes in a 50 ms time bin; other intervals from 12 to 160 ms yielded similar results (Figure 2—figure supplement 1B).

The mutual information, $I\left(R;C|p\right)$, was computed by taking the difference between the total response entropy, $H\left(R\right|p)$, and the conditional (noise) entropy, $H\left(R;C|p\right)$ (Cover and Thomas, 1991),

where

Entropy values were calculated from a histogram estimate of probability distributions.

The central stimulus followed a 4-bit M-sequence, with each stimulus frame having one of two values, $\mu +\Delta I$ and $\mu -\Delta I$, and a Michelson contrast $\left(\Delta I/\mu \right)$ of one of four possible values, 3, 6, 12 and 24%. Each four frame sequence occurred once in a repetition of the M-sequence, where $M^{\left(4\right)}$ is the set of all 16 possible combinations of four binary digits. The luminance in the center was updated at 10 Hz, and therefore one presentation of the M-sequence lasted $0.1\text{s}\cdot 2^4=1.6\text{s}$. The sequence was repeated for 11 trials at a given contrast and the responses for the first trial after a transition to a new contrast were discarded from the analysis. Contrasts were picked randomly without replacement and then a different order chosen once all four contrasts were tested. A stimulus was defined as the combination of the center (both sequence and contrast) and the periphery. By binning time in 100 ms there are 10 possible peripheral stimuli (time p relative to peripheral period), 16 possible sequences, $m^{\left(4\right)}$ and 4 possible contrasts (σ) for a total of 640 different stimuli. Each stimulus was measured at least 10 times.

When the central stimulus is divided into the stimulus sequence, $m^{\left(4\right)}\in M^{\left(4\right)}$ and the contrast, $\sigma \in \sum$, the total information from eq (1) can be further expanded into:

Where $I(R;\sum |p)$and $I(R;M^{\left(4\right)}|\sum , p)$ are functions of time p. The quantity $I(R;M^{\left(4\right)}|\sum , p)$ represents the information the brain could extract about the stimulus-sequence at a given time relative to the peripheral stimulus if it knew the contrast. Whereas $I(R;M^{\left(4\right)}|p)$ represents the information that the brain could extract about the stimulus sequence at a given time if it did not know the contrast. Comparisons of the time course of $I(R;M^{\left(4\right)}|p)$ and $I(R;\sum |p)$ (Figure 4C), as well as between $I(R;M^{\left(4\right)}|\sum , p)$ and $I(R;\sum |p)$ (Figure 4—figure supplement 1), indicate that the dynamics of information about sequence and contrast are different whether the brain can use contrast information to decode the sequence or not.

Spatial extent of peripheral changes in sensitivity.

To measure the spatial extent of increases and decreases in sensitivity, the central stimulus consisted of a mask in the pattern of a checkerboard, with squares 100 µm in size, that flickered with a pink noise stimulus intensity that was the same in all squares (Figure 5A). The overall size of the mask was 1.2 mm. The central stimulus was identical in all conditions. The background was a more finely scaled checkerboard, with squares 50 µm in size, and a central blank region that was varied in size from the full size of the monitor (no checkers in the periphery) to 0 µm (checkers everywhere except in central stimulus). At smaller values of the central blank region, the background was intercalated with the central region (Figure 5B, bottom). For each location of the peripheral stimulus, we computed an LN model between the center stimulus and the cell’s response and calculated the average slope of the nonlinearity for different time windows.

The model of long-range excitation consisted of a central stimulus s(t) that was passed through a linear filter F(τ), yielding the filtered central input

The filtered central input was combined with a signal, a(t), that depended on background motion. Because the goal of this model was to investigate the integration of central and peripheral signals, and not the origin of the peripheral signal as has been studied elsewhere in greater detail (Passaglia et al., 2009), we defined a(t) to be a biphasic function of time that began at the time t = 0, representing the time of the saccade. As to a plausible origin of this input, the peripheral effect occurred for a high spatial frequency checker and did not depend on the direction of the shift. Such a response could be generated by a group of rectified subunits as found in the receptive fields of polyaxonal amacrine cells (Baccus et al., 2008), but this was not explicitly implemented here. Because a(t) had two phases, positive and negative, whereas polyaxonal amacrine cells are thought only to deliver inhibition, it is expected that the positive phase would arise through disinhibition delivered through a second intervening amacrine cell that provides tonic inhibition.

The central and peripheral inputs were then passed through a threshold nonlinearity $N(.)$.

The nonlinearity was chosen with a slope of one and the threshold equal to 0.9 times the maximum amplitude of the peripheral signal a(t)·.

The output of the threshold function was then scaled by feedforward divisive adaptation, yielding the model output

Fα(t) was an exponentially decaying filter with an integral of one, and a time constant set to 10 s, although this parameter could be varied from 1 to 100 s with little effect. Smaller values of α yield more complete adaptation. However, for constant luminance experiments, equation (10) reduces to a constant independent of the luminance value when α = 0, therefore non-zero values of α are needed. Thus, the value of alpha was optimized to yield changes in adaptation as observed in the data, as well as responses to different levels of constant luminance. We used α = 0.30, but values from 0.05 to 2.00 could be used with similar results.

From previous studies, the first sharp threshold encountered in the retina is at the bipolar cell synaptic terminal (Mennerick and Matthews, 1996). Thus, the filter in the model should correspond to the spatiotemporal receptive field of a bipolar cell, which we took from our previous measurements of fast-Off bipolar cells (Baccus et al., 2008) (Figure 7A). In order to use the model to assess information transmission, we measured the noise in the membrane potential of bipolar cells as a function of contrast and found that the level of noise increases roughly linearly with contrast (Figure 7A-iii). This allowed us to choose a noise level at each time point that depended on the filtered stimulus, approximating measured bipolar cell noise.

A set of 342 images were taken from a database of natural scenes (Tkačik et al., 2011). The bipolar cell membrane potential was simulated by combining a linear receptive field pathway (Figure 7A-i) and noise (Figure 7A-iii). Because the filtering and noise of bipolar cells was measured at a fixed mean intensity, to avoid the need to incorporate luminance adaptation into the model, we normalized the mean intensity of all images. The linear receptive field center and surround were modeled independently, with each being a filter separable in space and time. Spatial linear predictions were made by convolving each image with a spatial disk of 1.0 and 2.5 degrees of visual space corresponding to center and surround. To compute the complete linear prediction of cells, images were jittered around according to a random walk with mean velocity of 0.33° per second simulating fixational eye movements and an instantaneous saccade simulated by a step of 6° in a random orientation in the image location. The temporal receptive fields for the center and surround were convolved with this image series and summed, generating the complete linear prediction for each location as a function of time; g_x,t where x denotes the location and t the time. From the 342 images, a total of 82,863 identical bipolar cells with non-overlapping centers were simulated.

Because bipolar cell noise depends on the stimulus contrast (Figure 7A-iii), we used a model of the noise whereby the instantaneous standard deviation of the noise at each point in time relative to the shift depended on the standard deviation of the linear prediction across the bipolar cell population (Figure 7A-iii, lower panel, black trace). This created the greatest noise during the gating window, and thus potentially underestimates the actual information conveyed. From the signal standard deviation at a particular time, an equivalent Gaussian contrast was found that would generate a linear prediction with the same standard deviation. With the equivalent Gaussian contrast, a level of noise was chosen such that the signal to noise ratio was the same in the simulation and in the bipolar cell’s measured membrane potential noise under repetitions of Gaussian stimulation at different contrasts (Figure 7A-iii, upper panel). Each cell in the model received independent noise which was generated from a Gaussian distribution.

The linear prediction $g\left(t\right)$ and model output $r\left(t\right)$ were binned to compute mutual information and conditional mutual information. The linear prediction $g\left(t\right)$ and the response $r\left(t\right)$ were divided into 16 unequal bins, positioned to maximize information about the total range of $g\left(t\right)$ and $r\left(t\right)$. The same bins were kept for all delays, p relative to the shift. The information that the response carries about the linear prediction at a particular delay p relative to the shift, was computed as $I\left(G_t;R_t|p\right)$ by taking the difference between the total response entropy at a given delay, $H\left(R_t|p\right)$, and the conditional (noise) entropy, $H\left(R_t|G_t,p\right)$. The conditional mutual information between the linear prediction $g\left(t\right)$ and the response $r\left(t\right)$ given the previous response $r\left(t-\Delta \right)$ at a given delay, p, was computed as

To compute a change in adaptation at different times relative to a shift, an index was computed that compared the measured change in the slope of the nonlinearity to the change expected from complete adaptation. After presenting a series of contrasts ${\sigma }_i$, we computed the nonlinearities $N(.)$ of an LN model, and from each nonlinearity extracted the slope m_i. Picking one contrast as reference, we normalized the slopes and the contrasts by those of the reference $\tilde{m}= m/m_0$ and $\tilde{\sigma }=\sigma /{\sigma }_0$ and fitted a line to $\tilde{m}$ vs. $\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$\sigma $}\right.$. The adaptive index is the slope of the fitted line, which will be one for an ideally adapting cell and zero for a nonadapting cell (Figure 3—figure supplement 1A–B).

The goal was to find the nonlinearity that maximized the mutual information $I\left(G;R\right)$, between the set of linear predictions, G and the set of spike counts, R, given the noise properties of the cell. We began with the linear prediction as a function of time $g\left(t\right)$, the spike count distribution at that time, $P\left(r\left(t\right)\right)$, computed over trials, and the average rate at that time, $⟨r\left(t\right)⟩$, computed by averaging over trials. The nonlinearity $N_0\left(g\right)$ maps $g\left(t\right)$ at a time t onto a model average firing rate $⟨r^{\prime }\left(t\right)⟩$ at that time, but to include noise that was most consistent with the observed noise we computed from the data the distribution of spike counts for a given average rate, $P\left(r\left(t\right)|⟨r⟩\right)$. This function mapped each average rate $⟨r\left(t\right)⟩$ at each time onto a distribution $P\left(r\left(t\right)\right)$. The optimized nonlinearity, $N_0\left(g\right)$, was a sigmoid parameterized by a slope, $x_1{}^{-1}$ and a midpoint, $x_0$, and was constrained to have the same minimum (zero) and maximum rate as the measured data $N_0\left(x\right)=\frac{r_{max}}{1+e^{-(x-x_0)/x_1}}$. To find for each candidate $N_0(.)$ the best estimated joint distribution $P\left(g\left(t\right),r^{\prime }\left(t\right)\right)$ between $g\left(t\right)$ and the model distribution of spike counts $r^{\prime }\left(t\right)$, we used $N_0(.)$ to map$g\left(t\right)$ onto $⟨r^{\prime }\left(t\right)⟩$, and then used the function $P\left(r\left(t\right)|⟨r⟩\right)$ to map $⟨r^{\prime }⟩$ onto $P\left(r^{\prime }\left(t\right)|g\left(t\right)\right)$ for a particular value of $g\left(t\right)$, i.e. $P\left(r^{\prime }\left(t\right)|g\left(t\right)\right)=P\left(r\left(t\right)|N_0\left(g\right)\right)$. Then, we weighted this conditional distribution by the marginal probability of the linear prediction g, P(g), which has a Gaussian distribution, to compute the full joint distribution of $g\left(t\right)$ and $r^{\prime }\left(t\right)$,

from which the mutual information was computed. Then we performed a grid search of the parameters of $N_0( )$ and found from $P\left(g\left(t\right),r^{\prime }\left(t\right)\right)$ the nonlinearity that maximized $I\left(G_t,R_t{}^{\prime }\right)$. The maximum value of $I\left(G_t,R_t{}^{\prime }\right)$ was taken as the maximum amount of information given the measured noise of the cell, and its minimum and maximum firing rate.

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

We thank D Kastner, S Ganguli, T Clandinin and L Nevin for comments on the manuscript; T Moore and D Kastner for helpful discussions. This work was supported by grants from the US National Eye Institute, Pew Charitable Trusts, McKnight Endownment Fund for Neuroscience, E. Mathilda Ziegler Foundation (SAB); and by a Walter & Idun Berry Fellowship (PDJ).

Animal experimentation: This study was performed in strict accordance with the recommendations in the Guide for the Care and Use of Laboratory Animals of the National Institutes of Health, and the Stanford institutional animal care and use committee (IACUC) protocol (11619).

© 2015, Jadzinsky et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.