A classic test for nonlinear spatial integration is to measure the response to a periodically modulated spatial grating restricted to the RGC receptive field center. Typically such measurements are made in ‘steady state’, well after grating onset. Nonlinear spatial integration causes a response at twice the temporal frequency of modulation – that is a ‘frequency-doubled’ or F2 response (Enroth-Cugell and Robson, 1966; Hochstein and Shapley, 1976). As observed previously, responses of both On and Off parasol RGCs showed strong frequency-doubled responses to contrast-reversing gratings (Petrusca et al., 2007; Crook et al., 2008; Cafaro and Rieke, 2013; Figure 1B, right gray box). Excitatory synaptic inputs and spike outputs exhibited similar F2 responses across a wide range of grating contrasts (Figure 1B, right, plots the F2 response normalized by the response to a modulated spatially uniform spot). The F2 responses are consistent with a nonlinearity in the bipolar output signals which creates spatial subunits in the On parasol receptive field (Figure 1A).

The model in Figure 1A, however, failed to predict On parasol responses to spatial structure in several other stimuli. First, the onset of a spatial grating produced very different responses in On and Off parasol cells. This was true for a periodic grating (Figure 1B, left gray box) or a flashed grating (Figure 1C). Specifically, Off parasol cells, as predicted from the model in Figure 1A, responded at the onset of a periodic grating and at both the onset and offset of a flashed grating. On parasol cells, however, responded minimally at the onset of a periodic or flashed grating and strongly at the offset of a flashed grating. Thus, surprisingly, On parasol cells do not behave according to the model in Figure 1A at the onset of a grating stimulus, but do exhibit nonlinear spatial integration for subsequent phases of a modulated grating.

Second, On parasol cells responded weakly to spatial structure in natural inputs (Figure 1D). As in previous work (Turner and Rieke, 2016; Turner et al., 2018), we probed spatial integration of natural images by comparing responses to spatially structured natural inputs with responses to spatially uniform stimuli with temporal variations chosen to match those of the natural inputs (see below). We designed these stimuli to explore responses to naturalistic spatial and temporal structure in the receptive field center; there are clearly other aspects of real visual inputs that they do not capture.

Natural movie stimuli mimicked the retinal input during free viewing by translating a natural image across the retina according to measured human eye movements (Van Der Linde et al., 2009; Figure 1D, top left). The linearity of spatial integration was tested by comparing responses to natural movies with those to a linear-equivalent disc movie. The intensity of each frame of the linear-equivalent disc movie was calculated by linearly integrating the intensity of the corresponding frame of the natural movie weighted by an estimate of the cell’s linear receptive field center. By construction, a cell that integrates inputs linearly over the receptive field center would produce identical responses to the original image and the disc. Both naturalistic and linear-equivalent stimuli were restricted to the receptive field center using an aperture with a size determined for each recorded cell from the dependence of response amplitude on spot size (Turner and Rieke, 2016; see Materials and methods). Figure 1D (bottom left) shows example images and corresponding linear-equivalent discs.

As reported previously, Off parasol RGCs responded differently to natural and linear-equivalent movies (green and black traces in Figure 1D, right). This difference indicates that Off parasol RGCs respond nonlinearly to spatial structure in natural visual inputs, as is expected from their responses to contrast-reversing and flashed gratings. On parasol RGCs, however, often showed similar responses to natural and linear-equivalent movies (Figure 1D, right; see Turner and Rieke, 2016 for quantification); thus, unlike their responses to temporally modulated gratings, On parasol RGCs integrated spatial structure within their receptive field center linearly or near linearly for many natural inputs. Spatial integration in On parasol responses varied considerably across different naturalistic stimuli, even within the same cell (Turner and Rieke, 2016, Freedland and Rieke, 2021). This further suggests that the spatial and/or temporal structure present in a natural movie can shape the spatial integration properties of On parasol cells.

The results summarized in Figure 1 demonstrate that spatial integration in On parasol RGCs exists on a stimulus-dependent spectrum from linear integration (e.g., for the onset of grating stimuli or for most natural movies) to nonlinear integration (e.g., for subsequent phases of a modulated grating). This spectrum is not explained by differences in image contrast: temporally modulated gratings elicit F2 responses across a broad range of contrasts (Figure 1B, right), including those encountered in natural scenes (see Materials and methods), and even high contrast gratings fail to elicit a response at onset. To understand how On parasol cells can behave according to the model in Figure 1A under some stimulus conditions but not others, we first identify the synaptic input properties that control spatial integration in these cells.

Interpreting responses to natural image movies like those in Figure 1D is complicated by possible history dependence – for example, the response at a given time could reflect the onset of a new image feature and/or the offset of a previous feature. To simplify the correspondence between image features and responses, we turned to flashed image patches (Figure 2). All patches were flashed starting from the same gray background, and hence differences in response can be attributed to the structure in the patch itself rather than the past history of the stimulus. To focus on patches that elicited spike responses from On parasol RGCs, we selected patches with a higher luminance in the receptive field center compared to the gray background.

Figure 2

Download asset Open asset

Figure 2—source data 1

Source data for Figure 2A–F.

https://cdn.elifesciences.org/articles/70611/elife-70611-fig2-data1-v2.xlsx

Download elife-70611-fig2-data1-v2.xlsx

For each cell, we recorded spike responses, excitatory synaptic inputs, and inhibitory synaptic inputs elicited by a set of image patches and corresponding linear-equivalent discs. Figure 2A–C shows responses of one example cell for 30 patches from a single image. Traces at the top show the time course of responses to two example patches, and the main panels plot mean (± standard error of the mean, SEM) for responses to six repeats of each individual patch. Responses were quantified by integrating the response over the time that the stimulus was presented. Consistent with the responses to time-varying naturalistic stimuli in Figure 1D, On parasol RGCs generated similar spike responses to flashed natural images and linear-equivalent discs (Figure 2A); this indicates linear or near-linear spatial integration.

Two aspects of the responses to flashed image patches suggested that inhibitory synaptic input contributes to the linearity of On parasol RGC spike responses. First, two image patches that elicited similar amplitude excitatory input could elicit quite different spike responses; for example, the black traces at the top of Figure 2A, B are from the same two image patches. The excitatory inputs for these patches were similar in amplitude (Figure 2B, top), but the example patch on the left elicited a smaller spike response than the patch on the right (Figure 2A, top). This is consistent with inhibitory input suppressing the response to the patch on the left relative to that on the right. Second, excitatory inputs elicited by many image patches exceeded responses to linear-equivalent discs, for example the black and green responses at the top of Figure 2B and the cluster of points below the unity line in the bottom panel. The difference in image and disc responses indicates nonlinear spatial integration. Spike responses of the same cell to the same image patches are much more linear, as indicated by the similarity of the black and green traces at the top of Figure 2A and the similarity of the image and disc responses across patches illustrated in the bottom panel.

Figure 2A,B shows that spatial integration of On parasol excitatory inputs differs from that of spike output – at least for this cell and image. To test the generality of this difference across cells and images, we summarized the response to each patch with a nonlinearity index (NLI). This index measures the normalized difference between responses to the image and disc

where R_image is the response to the image and R_disc the response to the disc. We averaged NLIs across all patches probed for a given cell and image – resulting in a single value for excitatory synaptic input and one for spike output for each cell. The resulting average NLIs varied considerably across different cells and images (Figure 2D; each point corresponds to a different cell), and some images systematically produced positive NLIs for both spike output and excitatory input (see also Turner and Rieke, 2016). Across cells, however, the NLI for excitatory inputs was significantly larger than that for spike output (Figure 2D; 0.25 ± 0.05 vs 0.08 ± 0.07, mean ± SEM, p < 0.01, n = 10). This is not due to large NLIs in a small subset of image patches; instead, more than half of the patches from a typical image showed nonlinear spatial integration for excitatory inputs (e.g., points below the unity line in the patch-by-patch analysis of Figure 2B) and near-linear spatial integration for spike outputs (e.g., points on or near the unity line in Figure 2A).

The difference in the sensitivity of On parasol excitatory input and spike output to spatial structure in natural images differs markedly from the situation for periodic gratings, in which both excitatory inputs and spike output exhibit clear nonlinear spatial integration (Figure 1B, right). We explore the origin of this difference below.

A logical hypothesis that follows from the results above is that natural images recruit strong inhibitory synaptic input to On parasol cells that cancels nonlinear excitatory input. The results described below support this hypothesis. We will then return to the issue of why periodic grating stimuli fail to recruit similar inhibitory synaptic input.

Figure 2C shows inhibitory inputs in response to the same image patches in the same example cell as the spikes and excitatory inputs in Figure 2A, B. Plotted points show the mean (± SEM) integrated response from six repeats of each patch. Inhibitory synaptic input exhibited strongly nonlinear spatial integration, as indicated by the points that fall below the unity line in Figure 2C. Image patches that elicit similar excitatory input can elicit very different inhibitory input; for example, the filled points and corresponding black traces in the top panels of Figure 2B, C. Further, this difference corresponds to the difference in spike response (black traces in Figure 2A). Patches that elicit highly nonlinear excitatory input can also elicit highly nonlinear inhibitory input (e.g., the filled triangles in Figure 2B, C) – that is, spatial structure in natural image patches recruits both excitatory and inhibitory input. These observations are consistent with cancelation of nonlinear excitatory input by inhibitory input and the insensitivity of the spike response to spatial structure in patches that elicit nonlinear excitatory input (e.g., the filled triangle and corresponding responses in Figure 2A).

Figure 2A–C shows results from one example cell. To test the generality of these observations, we quantified the relationship between excitatory and inhibitory input across patches from multiple images and RGCs. To effectively cancel nonlinear excitatory input, inhibitory input should overlap in time with excitatory input and have at least two additional properties. First, nonlinear inhibitory input should be recruited by the same image patches that elicit nonlinear excitatory input (as in the example patches at the top of Figure 2B, C). To test this, we compared the NLI for excitatory input with the difference between the (normalized) inhibitory input elicited by the image and disc (we did not use NLIs for inhibitory input because they tended to be either 0 or 1, obscuring any graded dependence on excitatory NLI). We divided the NLIs for excitatory input for individual patches into five discrete bins, and then measured the average difference in inhibitory input elicited by the image and linear-equivalent disc across all RGCs and patches for each excitatory NLI bin. Figure 2E plots the results; each point represents the mean (± SEM) difference in inhibitory input plotted against the mean (± SEM) excitatory NLI. Nonlinear inhibitory input increased systematically with increasing excitatory NLI. Thus, patches that elicited nonlinear excitatory input also elicited nonlinear inhibitory input.

Second, inhibitory input should be sufficiently large to effectively cancel nonlinearities in excitatory input. To test this, we measured the ratio of the amplitude of inhibitory input to excitatory input (the I/E ratio) as a function of the nonlinearity in excitatory input (Figure 2F). We again averaged inhibitory/excitatory ratios across image patches and RGCs. Figure 2F plots this average I/E ratio (mean ± SEM) against the excitatory NLI, organized into the same discrete bins as Figure 2E. The electrical driving force on excitatory synaptic input is about threefold larger than that on inhibitory synaptic input near the threshold for spike generation. Because of this difference in driving force, inhibitory inputs (measured as currents at the reversal potential for excitatory inputs) need to be at least threefold larger than excitatory inputs to have a substantial impact on spike output. Patches that elicited near-linear excitatory input corresponded to I/E ratios <1; in this case, inhibitory input is unlikely to substantially affect spike output. Patches that elicited strongly nonlinear excitatory input, however, corresponded to I/E ratios >3; for these patches inhibitory input is sufficiently large to impact spike output.

Collectively, the results of Figure 2 indicate that (1) excitatory synaptic input to On parasol cells exhibits larger spatial nonlinearities than spike outputs, and (2) inhibitory synaptic input has the required properties to compensate for nonlinearities in excitatory input and account for the linearity of the spike response. We next characterize inhibitory input in more detail and investigate why it does not linearize responses to grating stimuli as it does responses to natural images.

Inhibitory synaptic input to On parasol RGCs has two distinct components that originate from different retinal circuits (Figure 3A, left; Cafaro and Rieke, 2013; Crook et al., 2014). Feedforward inhibitory input originates from activity in On retinal pathways and, like excitatory input, is elicited by increases in light intensity (Figure 3A, top left). Crossover inhibitory input originates in Off retinal pathways and is elicited by decreases in light intensity (Figure 3A, bottom left). This means that a spatially uniform increase in light intensity increases both excitatory input and feedforward inhibitory input (Figure 3A, top right), while a spatially uniform decrease in intensity decreases excitatory input and increases crossover inhibitory input (Figure 3A, middle right).

Figure 3 with 1 supplement see all

Download asset Open asset

Figure 3—source data 1

Source data for Figure 3B–D.

https://cdn.elifesciences.org/articles/70611/elife-70611-fig3-data1-v2.xlsx

Download elife-70611-fig3-data1-v2.xlsx

Inhibitory synaptic input contributes minimally to On parasol spike responses to spatially uniform stimuli (Cafaro and Rieke, 2013). The increase in feedforward inhibitory input elicited by a spatially uniform increment is not sufficiently large to have a sizable impact on spike responses (I/E ratios are generally less than 1). And the decrease in excitatory input alone is sufficient to eliminate spiking for a spatially uniform decrement, and hence crossover inhibitory input makes little contribution. However, different regions of spatially structured stimuli can elicit different types of synaptic input – for example bright regions of a stimulus can cause an increase in excitatory input while dark regions can cause an increase in crossover inhibitory input (Figure 3A, bottom right). This can cause the (potentially large) crossover inhibitory input to overlap in time with excitatory input and hence impact spike output (Cafaro and Rieke, 2013).

Different natural image patches elicited inhibitory input that varied widely in amplitude and time course. The linearity of spatial integration similarly varied widely across patches; in some cases, responses to the image patch and the linear-equivalent disc were similar, while in others they were very different. These observations suggest that the contributions of feedforward and crossover inhibitory input differed across different patches. Testing this suggestion required estimating the contributions of each component of inhibitory input to the response to a given image patch. To do this, we used a principal components-based approach to cluster inhibitory responses elicited by different image patches (see Materials and methods); clustering used only the time course of the responses to the flashed image patches, and did not use any information about the patch itself or responses to the linear-equivalent discs. We set the target number of clusters to three as this was the minimum needed to separate responses with clearly different time courses. The goal of this analysis was to organize the responses according to their time course, and the discrete clusters were a matter of convenience only and do not reflect discreteness in the actual responses.

The three clusters defined by this approach had consistently different spatial integration properties. Figure 3B (left) compares inhibitory synaptic inputs elicited by flashed image patches and corresponding linear-equivalent discs for each response in each cluster for an example RGC. Figure 3B (right) shows the time course of average image and disc responses in each cluster. Responses in the first cluster had a time course consistent with feedforward inhibitory input (see Figure 3A, top right) and showed linear spatial integration – that is responses to image patches and linear-equivalent discs were near identical (Figure 3B, top right). Responses in the third cluster had a time course consistent with crossover inhibitory input and exhibited strongly nonlinear spatial integration – with much larger responses to images than to the corresponding linear-equivalent discs (Figure 3B, bottom right). Responses in the second cluster were smaller in amplitude and had more variable time courses and spatial integration properties than the others.

As a control, we applied the clustering procedure used for images to responses to increment and decrement spots. We retained the cluster definitions determined for the images – that is responses to increment and decrement spots were assigned to the clusters that were defined from the time course of responses to flashed image patches. High contrast increment spots, which elicit feedforward inhibitory input, led to responses in the first cluster, intermediate contrast increments led to responses in the second cluster, and decrement spots, which elicit crossover inhibitory input, led to responses in the third cluster (Figure 3—figure supplement 1). Hence, the clustering approach separates responses associated with known increases and decreases in light intensity: cluster 1 is dominated by feedforward inhibitory input, cluster 3 is dominated by by crossover inhibitory input, and cluster 2 is a mixture of the two.

We next compared the properties of the image patches corresponding to each inhibitory cluster. We started with how the difference between the inhibitory response to the image and disc – that is, the magnitude of spatially nonlinear inhibitory input – depended on the mean intensity of the image patch. As expected from the stimulus dependence of feedforward and crossover inhibitory input (e.g., Figure 3A), image patches eliciting responses in the first cluster were bright compared to the image mean (closed circles in Figure 3C); flashing these patches results in a net increase in light intensity in the receptive field center. Image patches in the third cluster had a mean intensity similar to the image mean (open circles in Figure 3C), and hence flashing these patches resulted in little change in light intensity in the receptive field center. Image patches in cluster 2 were again intermediate (gray circles in Figure 3C). The distinctions among clusters held across cells and images (Figure 3C, bottom; each point represents measurements of responses to a single image patch from a single RGC). We did not probe image patches with large negative intensity because we focused on patches that elicited an increase in spike rate (as in Figure 2). What is clear from this analysis, however, is that patches with little change in mean intensity – that is those corresponding to cluster 3 – can elicit large and spatially nonlinear crossover inhibitory input.

We next identified the excitatory synaptic inputs corresponding to the first and third inhibitory clusters – that is, those that we could associate with feedforward and crossover inhibitory input. Figure 3D (top) plots the excitatory inputs elicited by linear-equivalent discs against those elicited by the corresponding image patch for the same cell and patches as Figure 3B, C; symbols correspond to the inhibitory clusters defined in Figure 3B. The excitatory inputs differed systematically for different inhibitory clusters. Excitatory responses to patches corresponding to the first cluster were large and exhibited minimal spatial nonlinearity (Figure 3D, closed circles). Excitatory responses corresponding to the third cluster were relatively small and exhibited strong spatial nonlinearities (Figure 3D, open circles). Excitatory responses corresponding to the second cluster had intermediate properties (Figure 3D, gray circles).

To extend this comparison to multiple RGCs and images, we compared the mean NLI for excitatory inputs corresponding to inhibitory clusters 1 and 3. This collapses the excitatory inputs for each cell into two numbers: the mean NLI for patches corresponding to inhibitory cluster 1 and the mean NLI for patches corresponding to inhibitory cluster 3. We then plotted these against each other (Figure 3D, bottom; open circles are individual cells, closed circle is mean ± SEM). Across cells, the NLIs for excitatory responses corresponding to cluster 3 were larger than those for patches corresponding to cluster 1.

In summary, different natural image patches elicited inhibitory responses with different contributions of feedforward and crossover inhibitory input, and correspondingly different spatial integration properties. In particular, crossover inhibitory input showed strongly nonlinear responses to spatial structure, and was elicited by image patches that also elicited strongly nonlinear excitatory inputs. These observations suggest that crossover inhibitory input cancels nonlinear excitatory input and linearizes responses to natural inputs.

What functional properties of inhibitory input are required for effective cancellation of nonlinear excitatory input? It is difficult to selectively manipulate inhibitory input experimentally to answer this question due to off-target effects of pharmacological agents. Instead, we used subunit receptive field models with parallel excitatory and inhibitory paths (Figure 4A; see Materials and methods); these are an extension of the bipolar cell subunit receptive field models we used previously to account for Off parasol responses to natural images (Turner and Rieke, 2016). These models allowed us to investigate how sensitivity to spatial structure in the input depended on specific properties of the inhibitory circuitry.

Figure 4 with 1 supplement see all

Download asset Open asset

The first stage of the model filters spatial input through two regularly spaced grids of subunits – with separate subunits for excitatory and inhibitory paths. The receptive field size of each subunit was set to be consistent with responses to contrast-reversing gratings (Turner and Rieke, 2016). The filtered signal in each subunit was passed through a nonlinearity determined by the contrast–response function for either excitatory or inhibitory synaptic input (Figure 4—figure supplement 1). Inhibitory subunits had ‘U’-shaped nonlinearities, reflecting both feedforward and crossover components. Excitatory (blue) and inhibitory (red) subunit outputs were weighted by a Gaussian profile representing the receptive field center and then summed, with a threefold larger weighting of excitatory inputs reflecting the larger driving force near spike threshold. Spike responses (black) were predicted by thresholding this summed signal to eliminate negative responses. These models focus exclusively on spatial integration and do not consider time.

Figure 4A shows modeled excitatory, inhibitory, and spike responses for a collection of natural image patches from a single image. This ‘base’ model captured the measured cancelation of nonlinear excitatory input by inhibitory input and the linearity of the spike response (Figure 4A, right). In particular, the model captured nonlinear spatial integration in excitatory inputs (indicated by the points falling below the unity line in Figure 4A, left – see gray box) and the linear spatial integration in the spike output for these same image patches (Figure 4A, right). We summarized spatial integration for a given image by averaging the NLI (Equation 1) across patches for both excitatory input and spike output. We repeated this analysis for each image and averaged the resulting NLIs. Consistent with experiment (Figure 2D), the NLI for excitatory input was larger than that for spike output (Figure 4E, closed circle). We did not attempt to make quantitative predictions of responses to specific patches because of uncertainties in model details such as the subunit positions in a recorded cell.

We next probed how three manipulations of inhibitory input affected model responses. First, we replaced the measured inhibitory subunit nonlinearity with a linear contrast–response function. This forces inhibitory input to integrate linearly across space. Inhibitory input in this case was unable to cancel nonlinear excitatory input (see gray box in Figure 4B), and the NLIs for excitatory input and spike output were similar (Figure 4E, open square). This result highlights the necessity of nonlinear spatial integration if inhibitory input is to cancel nonlinearities in excitatory input. Second, we eliminated feedforward inhibitory input by setting the positive contrast region of the contrast–response function for inhibitory input to zero (Figure 4C). This had minimal effect on the model spike response (compare Figure 4A, right and Figure 4C) and on the difference in the NLIs for excitatory input and spike output (Figure 4E, closed square). Hence, crossover inhibitory input alone was sufficient to cancel nonlinear excitatory input, and feedforward inhibitory input contributed little to shaping model responses (see also Cafaro and Rieke, 2013). Third, we doubled the size of the inhibitory subunits without changing the size of the excitatory subunits (Figure 4D). This substantially decreased the ability of inhibitory input to cancel nonlinear excitatory input (Figure 4D, gray square and Figure 4E, closed and open circles). This analysis suggests that linearity of the spike response requires that nonlinear subunits for excitatory and inhibitory circuits have similar sizes.

In summary, the modeling of Figure 4 indicates that the insensitivity of the On parasol spike response to spatial structure in natural inputs depends on nonlinear crossover inhibitory input that has a similar spatial scale as nonlinear excitatory input and exhibits strong nonlinear spatial integration. Importantly for the analyses below, this means that Off retinal circuits are the main source of the inhibitory input that regulates spatial integration.

The experiments described thus far (1) indicate that the linearity of On parasol responses to spatial structure in natural images originates from recruitment of inhibitory input, and (2) identify key properties of inhibitory input that make it effective in linearizing spike responses. This description, however, does not explain why On parasol cells respond to spatial structure in some stimuli but not others (see Figure 1). The experiments and analyses described in the next two sections indicate that the periodic time course of classic grating stimuli substantially impacts synaptic integration by rapidly modulating the adaptational state of the cone photoreceptors. This time-dependent adaptation contributes strongly to responses to contrast-reversing gratings, such as those in Figure 1B and many other studies.

Figure 1C introduced On parasol responses to a flashed grating. Here, we analyze these responses in more detail since they can be compared directly to the flashed images in Figure 2. The onset of a flashed grating elicited a minimal spike response, while the grating offset elicited a clear response (Figure 5A, black). To collect results across cells, we compared the number of spikes elicited at grating onset with that at grating offset (Figure 5B, left). The offset response exceeded the onset response in each recorded cell (gray points; onset response 0 ± 5 spikes, offset response 57 ± 7 spikes, mean ± SEM, n = 7, p < 0.001). To explore the origin of this difference, we compared excitatory and inhibitory inputs produced at grating onset and offset (Figure 5A). Both the relative magnitude and the timing of excitatory and inhibitory inputs differed for grating onset and offset. To quantify the difference in amplitude, we computed the I/E ratio based on the integrated inhibitory and excitatory currents at grating onset and offset. The I/E ratio was about twofold larger at grating onset than offset (Figure 5B, middle; p < 0.01). To quantify the difference in timing, we compared the time at which the excitatory and inhibitory currents reached a half-maximal value (T_1/2). Excitatory input was delayed relative to inhibitory input at grating onset but not offset (Figure 5B, right; onset p < 0.01, offset p = 0.9). Both of these factors should contribute to the larger spike response at grating offset.

Figure 5 with 1 supplement see all

Download asset Open asset

Figure 5—source data 1

Source data for Figure 5B–D.

https://cdn.elifesciences.org/articles/70611/elife-70611-fig5-data1-v2.xlsx

Download elife-70611-fig5-data1-v2.xlsx

Like the difference in responses at the onset and offset of a flashed grating, responses to contrast-reversing gratings depended on time since grating onset (see Figure 1B). Figure 5C, D quantifies this dependence by integrating responses (spikes, inhibitory input, and excitatory input) during each half-cycle of the grating. The spike response during the first half-cycle of a contrast-reversing grating was substantially smaller than that during subsequent half-cycles (Figure 5C, left and bottom right). Similarly, the ratio of inhibitory input to excitatory input was substantially larger during the first half-cycle of the grating than during subsequent half-cycles (Figure 5C, left and top right). These features held across cells (Figure 5D, left, plots mean ± SEM, n = 13). Off parasol cells exhibited the opposite behavior, with the smallest I/E ratios and largest spike responses during the first half-cycle of a grating (Figure 5—figure supplement 1). This asymmetry between On and Off parasol responses is particularly interesting since Off circuits dominate inhibitory input to On parasol RGCs (Figure 3). We will pursue this On/Off difference in the next section.

The kinetics of excitatory and inhibitory synaptic inputs in response to a modulated grating also differed. Excitatory and inhibitory input increased with similar time courses during the first half-cycle of the grating, but for subsequent half-cycles increases in excitatory input led increases in inhibitory input (Figure 5D, top right, shows excitatory and inhibitory synaptic currents from the same cell as Figure 5C). The firing rate was highest in the time window in which excitatory inputs had increased but inhibitory input was still small (e.g., compare black and blue traces in Figure 5C, left). We quantified this time shift across cells by measuring the phase difference between sinusoidal fits to excitatory and inhibitory currents (Figure 5D, bottom right). The measured phase differences correspond to time shifts of 10–15 ms.

The analyses summarized in Figure 5 highlight the dependence of nonlinear spatial integration on the stimulus time course. Nonlinear spatial integration is weak or absent at the onset of a grating because inhibitory input is large compared to excitatory input and the two overlap in time. Nonlinear spatial integration increases for later cycles of a periodic grating stimulus, due to a decrease in the I/E ratio and a delay of inhibitory input relative to excitatory input.

Nonlinear spatial integration in excitatory synaptic input is typically generated by rectifying nonlinearities at the bipolar output synapse (Demb et al., 1999; Demb et al., 2001; Borghuis et al., 2013; but see Schreyer and Gollisch, 2021). The results summarized in Figures 2—4 indicate that the impact of this spatial nonlinearity on spike output can be regulated by inhibitory synaptic input. The results in the previous section identify stimulus dynamics as a key regulator of the excitatory/inhibitory balance and hence the strength of nonlinear spatial integration. This section explores the origin of this dependence on stimulus dynamics.

The time course of the light intensity encountered by a single receptive-field subunit differs substantially between the onset and offset of a flashed grating and likewise between the first and subsequent half-cycles of a modulated grating. Prior to the onset of either stimulus type, subunits encounter the mean stimulus intensity, while at later time points they undergo larger dark–light or light–dark transitions. Several time-dependent nonlinear mechanisms could cause this difference in stimulus time course to contribute to the responses illustrated in Figure 5. One such mechanism is adaptation in the cone photoreceptors, which we explore below.

Measured cone photocurrents to sinusoidal stimuli showed three features that could potentially shape RGC responses (Figure 6A): (1) the amplitude of the cone response was smaller on the first half-cycle of the grating than on subsequent cycles; (2) cone responses were asymmetric, with larger responses to the dark phase of the grating (decrements) than to bright phase (increments); and, (3) the kinetics of cone responses to dark-to-light transitions were sped relative to responses to light-to-dark transitions. These properties are apparent in both the cone current (Figure 6A, top) and voltage (Figure 6A, bottom) responses. These properties of the cone responses are created by rapid adaptation in the phototransduction process (Angueyra et al., 2022), due to a light-dependent increase in PDE activity (Nikonov et al., 2000), which causes the gain of the cone light response to be modulated during each cycle of the sinusoid. The cone responses are well captured by an empirically derived biophysical model of cone phototransduction (see Figure 6—figure supplement 1, Materials and methods and Angueyra et al., 2022 for details). This model allows us to predict and manipulate cone responses to contrast-reversing gratings.

Figure 6 with 2 supplements see all

Download asset Open asset

Different aspects of light stimuli control excitatory and inhibitory synaptic inputs to On parasol cells: light increments dominate excitatory inputs, whereas light decrements dominate inhibitory inputs (Figure 3A). This suggests that the asymmetry in the amplitude and timing of increment and decrement responses in the cones could control the relative magnitude and timing of excitatory and inhibitory synaptic inputs to an On parasol RGC and hence how these inputs are integrated to control spike output. The difference in the responses of On and Off parasol cells to the onset of a grating (Figures 1B, C, 5C and Figure 5—figure supplement 1) supports this suggestion.

To test for a role of cone adaptation in controlling spatial integration, we modified the subunit model from Figure 4 to take cone signals as input (see Materials and methods). Hence, we simulated responses of an array of cones to contrast-reversing gratings, and then used these simulated cone responses as input to models with excitatory and inhibitory subunits, including measured contrast–response functions. We used this approach to predict responses to gratings for models incorporating both adapting and nonadapting (i.e., linear) cones (Figure 6B; Figure 6—figure supplement 2; see Materials and methods for model details). The linear cone model was implemented by filtering the stimulus with a linear filter fit to the cone response to a brief, low-contrast flash. Note that the measured RGC contrast–response functions should capture time-independent (i.e., static) nonlinearities in the cone responses, particularly the asymmetry between the amplitude of increment and decrement responses; these contrast–response functions, however, will not capture time-dependent nonlinearities in the cone responses. Hence, this modeling specifically predicts how the time-dependent nonlinearities introduced by adaptation in the cones impact RGC responses.

Models with adapting cones predicted a 70–80% larger I/E ratio on the first grating cycle compared to the steady-state ratio (Figure 6C, top, thin line). This time dependence of the I/E ratio was very similar to that observed in RGC synaptic inputs (closed circles) and was absent in predictions of models with linear cones (thick line). Models with adapting cones also captured the temporal delay between excitatory and inhibitory input (thin lines in Figure 6B; dashed line in Figure 6C, bottom); this delay was again absent for models with linear cones (thick lines in Figure 6B). Excitatory input is dominated by cones undergoing a dark-to-light transition, whereas inhibitory input is dominated by cones undergoing a light-to-dark transition. The timing difference in the model originates from the temporal asymmetry in the responses of cones encountering increases and decreases in light level (Figure 6A, right). This difference in cone kinetics is a property of adaptation and is absent in linear models of the cone responses and in the RGC models with linear cone inputs. The light-to-dark vs dark-to-light difference in the kinetics of the cone responses is absent in the first half-cycle of a periodic grating, because all cones begin from the same mean intensity, but present in subsequent cycles when different cones start from different adaptational states.

The modeling summarized in Figure 6 supports the following picture for the time dependence of responses to periodic grating stimuli. Prior to grating onset, all cones are in a uniform adaptational state. The onset of the grating produces an increase in both excitatory and inhibitory synaptic input to an On parasol cell, and these occur with near-identical kinetics (Figures 5A, C, 6B). Rapid cone adaptation (time constant <100 ms) limits the amplitude of the increase in excitatory input at grating onset (compare thick and thin lines in Figure 6B). The uniform adaptational state of the cones, however, is altered by the grating. Hence, for subsequent cycles of the grating, cones encountering an increase in light intensity start from a higher signaling gain and respond more quickly than cones encountering a decrease in light intensity. This increases the amplitude of excitatory input relative to inhibitory input, and creates a time window in which excitatory input increases before inhibitory input. Without adaptation (i.e., with linear cones), this shift in amplitude and timing of excitatory input relative to inhibitory input does not occur.

To directly test the hypothesis that cone adaptation shapes On parasol RGC responses to contrast-reversing gratings, we used the cone model to design grating-like stimuli that minimized the impact of adaptation on cone responses. In the case of a sinusoidal stimulus like a contrast-reversing grating, this means identifying a stimulus time course for which the cone model predicts a sinusoidal response. To achieve this, we transformed a grating stimulus to minimize the difference between the response of the adapting cone model to the transformed stimulus and the response of a linear cone model to the original stimulus (Figure 7A). We refer to this procedure as the ‘light-adaptation clamp’ (see Materials and methods). Since a linear cone responds sinusoidally to a sinusoidal input, the transformed stimulus we seek is one for which the adapting cone model predicts a sinusoidal response. Cone responses to the modified grating stimuli are predicted to exhibit less initial history dependence and symmetrical responses to increments and decrements (in both dynamics and amplitude).

Figure 7

Download asset Open asset

Figure 7—source data 1

Source data for Figure 7B–D.

https://cdn.elifesciences.org/articles/70611/elife-70611-fig7-data1-v2.xlsx

Download elife-70611-fig7-data1-v2.xlsx

We tested the stimuli constructed as illustrated in Figure 7A in recordings from voltage-clamped cones (Figure 7B, top). Across recorded cones, the transformed stimuli altered cone responses as predicted by the model (Figure 7B). First, the kinetics of the response to the dark-to-light transition was slowed (closed arrowhead in Figure 7B, top). Second, the difference in amplitude of the response to the first vs second cycle was reduced (Figure 7B, bottom left). Third, the increment/decrement asymmetry was reduced (open arrowhead in Figure 7B, top; Figure 7B, bottom right). Hence the light-adaptation clamp procedure successfully identified stimuli that minimized the impact of adaptation on cone responses.

The light-adaptation clamp procedure provided a tool to test the impact of adaptation in cone phototransduction on responses of On parasol RGCs to periodic gratings. Specifically, if cone adaptation contributes to the small spike response of On parasol RGCs during the onset of a contrast-reversing grating (Figure 5C, D), then On parasol responses to the modified stimuli should show less time dependence. This was indeed the case (Figure 7C, thick traces are responses to the transformed stimuli and thin traces are responses to the original grating). First, the decrease in I/E ratio observed for the original grating was largely absent for the transformed grating (Figure 7D, top left). The time dependence of the I/E ratios for original and transformed stimuli agreed well with predictions from the models of Figure 6B with linear or nonlinear cone inputs (Figure 7D, top left). Second, the suppression of the spike response at the onset of the original grating stimulus was absent for the transformed stimulus (Figure 7D, bottom left).

As illustrated in Figure 6A, adaptation also causes cones to respond faster to dark-to-light vs light-to-dark transitions. Correspondingly, minimizing cone adaptation changed the relative timing of excitatory and inhibitory inputs to On parasol cells (Figure 7D, middle and right). Excitatory inputs in response to the transformed stimuli were substantially delayed compared to excitatory inputs in response to the original stimuli (Figure 7D, middle). This shift in timing of excitatory input caused it to overlap more with inhibitory input, and the substantial phase shift between excitatory and inhibitory input observed for conventional gratings was almost entirely eliminated by the transformed gratings (Figure 7D, right). The difference in timing of excitatory and inhibitory inputs for both grating types was well predicted by the models in Figure 6B based on linear or nonlinear cones (dashed line and open circles in Figure 7D, right). These results indicate that the temporal differences in cones exposed to increments and decrements (Figure 6A) plays a central role in determining the kinetics of excitatory and inhibitory input and hence the effectiveness with which inhibitory input cancels excitatory input.

The combination of modeling in Figure 6 and direct manipulation of the cone responses in Figure 7 indicate that cone adaptation can explain the majority of the observed time dependence of the On parasol responses to contrast-reversing gratings. The decrease in I/E ratio after grating onset and the speeding of excitatory input relative to inhibitory input account for the strength of the steady-state responses to contrast-reversing gratings, and both of these depend on adaptation in the cones. Downstream time-dependent nonlinearities (e.g., depression at the bipolar output synapse; Ke et al., 2014) could also contribute, but such contributions appear small compared to cone adaptation (see Discussion). These results add an unexpected element to classic subunit models such as those in Figure 1A: the impact of nonlinearities at the bipolar output synapse on RGC spike responses is controlled by how the RGC integrates excitatory and inhibitory input, and this E/I balance is in turn shaped by adaptation in the cone photoreceptors.

Natural inputs vary over time due to eye movements and object motion within an image. Why then do On parasol RGCs respond weakly to spatial structure in time-varying natural inputs (Figure 1D) despite their strong responses to periodic grating stimuli? To answer this question, we used the cone/subunit model described above to predict how cone adaptation impacts the balance of excitatory and inhibitory inputs for On parasol cell responses to time-varying natural inputs.

Stimuli were created by sampling a static natural image with a simulated eye movement trajectory. These stimuli are identical to the movies used in Figure 1 except for the use of simulated rather than real eye movements. Eye movements were simulated by a diffusional process to account for fixational eye movements, interrupted by occasional large and discrete changes in position to account for saccades (see Materials and methods for details). The resulting spatiotemporal pattern of inputs was converted to signals in the modeled cone photoreceptor array (Figure 8A, bottom, shows modeled photocurrent responses of a single cone). This was repeated for adapting and linear cone models (adapting cone models followed Figure 6—figure supplement 1; the linear cone model consisted of a linear filter convolved with the stimulus). Adaptation substantially attenuated the amplitude of modeled cone responses compared to responses of cones without adaptation (Figure 8A, bottom).

Figure 8

Download asset Open asset

We used the modeled adapting and nonadapting cone signals as input to subunit models for excitatory and inhibitory synaptic inputs as in Figure 6B. We used these models to predict I/E ratios in every 30-ms time bin (Figure 8B). These predicted I/E ratios are sensitive to changes both in the relative timing and in the amplitude of excitatory and inhibitory input. I/E ratios varied widely across time bins, but were systematically larger for models with adapting cones compared to models with nonadapting cones (Figure 8B). To test for a similar effect across images and trajectories, we constructed histograms of I/E ratios for models with adapting and nonadapting cones; Figure 8C compares histograms from many images, each with an independent eye movement trajectory. Cone adaptation substantially increased the number of patches in which inhibitory input was sufficiently large to attenuate excitatory input (I/E ratios exceeding 3).

This analysis suggests that cone adaptation shapes responses to contrast-reversing gratings and time-varying natural inputs quite differently. This difference can be explained by the different luminance trajectories encountered by cones during these stimuli, and the corresponding differences in the time course of cone adaptation. In the case of a sinusoidally modulated grating, an increase in intensity at a given spatial location always occurs after a decrease, and likewise decreases occur after increases. Hence, a cone encountering an increase in intensity starts from a state in which signaling gain is high (due to the preceding decrease in intensity and relief of adaptation), and a cone encountering a increase in intensity starts from a low gain state (due to the preceding decrease in intensity and engagement of adaptation). This both speeds and increases the amplitude of responses of cones signaling intensity increases relative to those signaling decreases (see Figure 6A) – which in the case of an On parasol cell means shifting signaling in favor of excitatory synaptic inputs over inhibitory synaptic inputs (see Figure 6B).

Naturalistic inputs typically do not show such predictable behavior: intensity increases are not systematically preceded by low intensity regions of a scene. Instead, on average, cones encountering decreasing of increasing intensity start from the average image intensity – much like the first half-cycle of a modulated grating or the onset of a flashed grating. In all such cases, cone adaptation had a strong impact on the I/E ratio and spike response (e.g., linear vs adapting cones in Figures 6 and 8 and control vs transformed stimuli in Figure 7). Specifically, adaptation in the cones shifted the I/E balance in favor of inhibitory input, and this in turn decreased the sensitivity of On parasol RGC responses to spatial structure.

On and Off ganglion cells of the same type are classically assumed to respond to opposite polarity but otherwise identical stimulus features. Asymmetries between On and Off cells challenge this simple picture (Chichilnisky and Kalmar, 2002; Zaghloul et al., 2003; Sagdullaev and McCall, 2005; Murphy and Rieke, 2006; Nichols et al., 2013; Turner and Rieke, 2016; Ravi et al., 2018). The experiments and analysis below suggest that asymmetries in spatial integration (as in Figure 1) cause joint activation of On and Off parasol cells to encode image features not encoded by either cell type alone. This analysis and the associated conclusions are restricted to activation of the receptive field center.

We used subunit models to identify image patches predicted to coactivate On and Off parasol cells. We first sampled patches of natural inputs through a grid of subunits as in Figure 4. We then constructed two RGC models (see Materials and methods and Turner and Rieke, 2016): (1) in a spatially linear model, subunit outputs were weighted by a Gaussian receptive field with a size approximating the parasol receptive field center and then linearly summed; (2) in a spatially nonlinear model, subunit outputs were rectified, weighted by a Gaussian and summed. Unlike the model of Figure 4, these models do not incorporate separate excitatory and inhibitory paths, but they do capture the linear vs nonlinear difference in how On and Off parasol RGCs integrate inputs across space.

We used responses to many image patches to define constant–response contours for each of these models in a space spanned by the local luminance and spatial contrast of the sampled patches (patch statistics were measured within the receptive field center, see white circle in Figure 9B). These contours identify the set of image properties (i.e., local luminance and spatial contrast) that resulted in similar responses of a given RGC model and hence that cannot be distinguished based on the model response.

Figure 9 with 2 supplements see all

Download asset Open asset

As expected, response contours for spatially linear models did not depend on spatial contrast (i.e., the contours are near horizontal; Figure 9A, left). The contours for the spatially nonlinear models, however, did depend on spatial contrast because of nonlinear spatial integration (Figure 9A, right). Interestingly, the contours for Off parasol cells showed a stronger dependence on spatial contrast than those for On parasol cells. This On/Off difference is not due to an asymmetry in the models, as it was absent for spatial gaussian noise in which the distribution of bright and dark pixels was symmetrical. The difference instead is created by the relative abundance of dark pixels in natural images (Ratliff et al., 2010; Cooper and Norcia, 2015), and correspondingly could be recapitulated by spatial noise with a skewed pixel distribution (not shown).

Our results suggest that On parasol RGCs exhibit near-linear spatial integration (i.e., top left in Figure 9A) while Off parasol RGCs show nonlinear integration (bottom right). In this case, image patches with positive local luminance and high spatial contrast should activate both On and Off parasol cells; such coactivation would be absent if both cells exhibited linear spatial integration. These patches are located at the bright side of edges in an image (red points in Figure 9B).

The modeled response contours of On and nonlinear Off parasol cells have different dependencies on changes in mean luminance and contrast for image patches in which the cells are coactive: On parasol model responses are affected only by changes in luminance, while Off parasol model responses depend on both luminance and spatial contrast. This means that the joint responses of On and Off cells should permit discrimination of image patches with small differences in local luminance and/or spatial contrast (Figure 9—figure supplement 1); such discrimination is not possible from responses of either cell type considered alone. Furthermore, the linear On/nonlinear Off combination provides better discrimination than a nonlinear On/linear Off combination (Figure 9—figure supplement 1) due to the stronger Off responses to spatial structure (Figure 9A). Consistent with this modeling, image patches with similar mean luminance within the receptive field center that elicited near-identical measured spike responses from On parasol cells could elicit quite different responses from Off parasol cells (Figure 9—figure supplement 2).

This analysis indicates that joint activity of On and Off parasol cells may encode stimulus features that would be ambiguous if both cell types integrated linearly over space. Note that our analysis here focuses entirely on stimuli restricted to the receptive field center. The improved coding of specific stimulus aspects comes with a cost of sensitivity to other stimulus features. For example, with nonlinear spatial integration, Off parasol cells do not uniquely encode decreases in luminance. We return to this issue in Discussion.

Artificial stimuli, by design, often activate specific circuit mechanisms while avoiding activation of other mechanisms. But complex stimuli, including natural inputs, recruit multiple circuit mechanisms that act in concert to control circuit outputs. Interactions among these coactive mechanisms pose a fundamental challenge to understanding the operation of neural circuits.

Here, we show that the manner in which On parasol RGCs respond to spatial structure in their inputs is shaped by a combination of rapid adaptive nonlinearities in cone phototransduction (time constant <100 ms) together with rectifying synaptic nonlinearities in the circuits generating excitatory and inhibitory inputs — for example nonlinearities at the bipolar output synapse (Demb et al., 1999; Demb et al., 2001; Borghuis et al., 2013). Thus, for contrast-reversing gratings, cones encountering increases and decreases in light intensity start from different adaptation states and consequently generate responses with different amplitudes and kinetics. These differences in cone inputs to On and Off circuits, coupled with rectification in the circuits generating excitatory and inhibitory input to On parasol cells, favor excitatory input over inhibitory input. Thus, gratings elicit a relatively small ratio of inhibitory input to excitatory input, and a phase shift such that excitatory input increases prior to inhibitory input. These properties contribute to strong responses to the spatial structure of the grating.

Natural inputs lack the periodicity of contrast-reversing gratings; consequently, inhibitory input is both large and temporally in phase with excitatory input, and hence is well poised to decrease sensitivity to fine spatial structure. The opposite is true for Off parasol cells – where the periodic changes in cone adaptation during grating stimuli favor inhibitory input over excitatory input. This provides a clear example, with clear functional consequences (see below), of how several nonlinear mechanisms acting in concert control circuit outputs in unexpected ways.

Changes in rectification at bipolar output synapses can also shape the time course of RGC responses to periodic stimuli (Ke et al., 2014). In mouse On sustained RGCs, steady light can produce a sustained depolarization of cone bipolar cells, leading to synaptic depression; transient hyperpolarizing stimuli can relieve this depression and enable subsequent depolarizing stimuli to produce large excitatory input (Ke et al., 2014). Parasol RGC responses, however, do not appear to be similarly shaped by synaptic depression in our experiments. The onset of a spatially uniform light step from a constant background elicited large excitatory input and strong spike responses in both On and Off parasol cells (Figure 4—figure supplement 1; see also Turner and Rieke, 2016), indicating that at the mean luminance levels used here, bipolar cells upstream of parasol RGCs have vesicles available to produce strong synaptic output. Thus, synaptic depression in upstream bipolar cells is not likely to play a major role in the asymmetry seen at the onset of a periodic grating stimulus (e.g., Figure 1).

The differences in cone signaling that control the linearity or nonlinearity of On parasol spatial integration are subtle. Yet these relatively small changes in input qualitatively change the computational properties of the circuit. This is one of several examples in which control of computation relies on fine tuning of common circuit elements rather than recruitment of distinct circuits or other more exotic mechanisms (see also Grimes et al., 2014). The sensitivity of neural circuits to subtle changes in input or in operating point will be important in constructing models for neural computation, including artificial neural networks that attempt to match the efficiency of real neural circuits in specific tasks.

Cortical neurons typically receive high rates of excitatory and inhibitory inputs which largely cancel – a mode referred to as balanced excitation and inhibition (reviewed by Isaacson and Scanziani, 2011). The discovery of E/I balance has been an important breakthrough in our understanding of how cortical circuits work, and disruption of E/I balance can underlie the aberrant neural signaling characteristic of several brain disorders (reviewed by Sutula and Dudek, 2007; Nelson and Valakh, 2015).

Signaling in circuits exhibiting E/I balance often relies on differences in timing of excitatory vs inhibitory inputs (reviewed by Isaacson and Scanziani, 2011). For example, in auditory (Wehr and Zador, 2003; Wu et al., 2008), somatosensory (Pouille and Scanziani, 2001; Swadlow, 2002; Wilent and Contreras, 2005), and visual cortices (Liu et al., 2010), the onset of a sensory stimulus creates a brief time window in which excitatory input exceeds inhibitory input; action potentials are preferentially generated in this window. Several circuit features can contribute to these timing differences and their impact on spike output, including delays in inhibitory signals relative to excitatory signals due to routing through extra synapses (Pouille and Scanziani, 2001; Gabernet et al., 2005) and the location of excitatory inputs on the dendrites and inhibitory inputs on or close to the soma of a target neuron (Stokes and Isaacson, 2010).

Here, we show that the relative timing of excitatory and inhibitory input is highly stimulus dependent. As described above, adaptation in the cones during periodic stimuli like contrast-reversing gratings creates kinetic differences in cones encountering increases and decreases in light intensity. Cones encountering these different polarity stimuli dominate input to the circuits generating excitatory and inhibitory input to On parasol RGCs, and the difference in kinetics provides a time window in which excitatory input substantially exceeds inhibitory input. For nonperiodic stimuli, however, the adaptational state of the cones does not correlate strongly with the change in intensity encountered and cones responding to increases and decreases in light level respond with similar kinetics. Hence, spatial structure in such stimuli creates near-simultaneous changes in excitatory and inhibitory input and little or no spike response.

The circuit features giving rise to this stimulus-dependent control of E/I balance are common but also distinctive. The distinct circuit features – notably the sensitivity of the circuits controlling excitatory and inhibitory inputs to different aspects of the circuit input (here increases and decreases in light level) – may help identify other circuits in which a similar control could be at work.

On and Off RGCs of the same type are classically assumed to respond to similar stimulus features with opposite polarity. Multiple findings show that this picture is too simple, and in fact that corresponding On and Off RGCs can exhibit asymmetries in addition to the polarity of their responses. This includes differences in receptive field size (Balasubramanian and Sterling, 2009), contrast sensitivity (Zaghloul et al., 2003), rectification (Turner and Rieke, 2016), and balance of excitatory and inhibitory inputs (Murphy and Rieke, 2006). Some of these asymmetries – for example the smaller receptive fields of Off RGCs – have been suggested to be important in population coding of natural inputs (Balasubramanian and Sterling, 2009; Ratliff et al., 2010; Pandarinath et al., 2010; Karklin and Simoncelli, 2011). On/Off asymmetries also differ for different RGC types (Ravi et al., 2018), a factor that will need to be considered in efficient coding arguments.

On and Off parasol RGCs both exhibit nonlinear spatial integration in response to contrast-reversing gratings (Petrusca et al., 2007; Crook et al., 2008). Off parasol RGCs similarly respond nonlinearly to spatial structure in natural images, but unexpectedly On parasol RGCs respond near linearly. The difference in sensitivity to spatial structure in natural images suggests that certain features – specifically image patches that have similar positive luminance and differences in spatial contrast – may be distinguishable based on the joint activity of pairs of On and Off cells even when the responses of the individual cells are ambiguous. Determining whether this increased sensitivity to specific image features, at the expense of sensitivity to others, is advantageous will require considering how the entire RGC population encodes natural inputs. Nonetheless, the symmetry of On and Off parasol RGC responses to contrast-reversing gratings and asymmetry of their responses to natural inputs emphasizes the complexity of the relationship between the statistics of input stimuli and circuit nonlinearities that control important functional aspects of neural responses.

Experimental procedures followed those described previously (Turner and Rieke, 2016; Turner et al., 2018). In brief, macaque retinas (Macaca fascicularis, Macaca nemistrina, and Macaca mulatta) were obtained through the Tissue Distribution Program of the Regional Primate Research Center at the University of Washington. Pieces of retina attached to the pigment epithelium were stored in ∼32–34°C oxygenated (95% O₂/5% CO₂) Ames medium (Sigma, St Louis, MO) and dark adapted for >1 hr. Pieces of retina were then isolated from the pigment epithelium under infrared illumination and flattened onto poly-L-lysine slides. Once under the microscope, tissue was perfused with oxygenated Ames medium at a rate of ∼8 ml/min.

Parasol RGCs were targeted for electrical recordings based on their characteristic soma size and response to a light step. Spike responses were measured in the cell-attached or extracellular configurations using electrodes filled with Ames medium. Synaptic inputs were measured in the whole-cell voltage-clamp configuration using electrodes (2–3 MΩ) filled with a solution containing (in mM): 105 Cs methanesulfonate, 10 Tetraethylammonium chloride (TEA-Cl, a potassium channel blocker), 20 4-(2-hydroxyethyl)-1-piperazineethanesulfonic acid (HEPES buffer), 10 ethylene glycol-bis(β-aminoethyl ether)-N,N,N′,N′-tetraacetic acid (EGTA, a calcium chelator), 2 QX-314, 5 Mg-ATP, 0.5 Tris-GTP (∼280 mOsm; pH ∼7.3 with CsOH). To isolate excitatory or inhibitory synaptic input, cells were held at the estimated reversal potential for inhibitory or excitatory input of ∼−60 and ∼+10 mV. These voltages were adjusted for each cell to maximize isolation (see Cafaro and Rieke, 2013). Voltages have been corrected for a ~−8.5 mV liquid junction potential.

Retina sensitivity was checked before collecting data based on the contrast sensitivity of On parasol cells. All data collected came from retinas in which a 5% contrast spot modulated at 4 Hz produced at least a 20 spikes/s modulation in the On parasol spike rate. 60–70% of the preparations met this criterion.

Stimuli were presented and data acquired using custom written stimulation and acquisition software packages Stage (stage-vss.github.io) and Symphony (symphony-das.github.io). Labwide acquisition packages can be found at https://github.com/Rieke-Lab/riekelab-package (Yu, 2021a copy archived at swh:1:rev:abdd32f596f57613cb470e4b5328e7d6f678ce5e) and protocols used in this study can be found at https://github.com/Rieke-Lab/turner-package (Yu, 2021b copy archived at swh:1:rev:e09ed136af28ceea83df29d84b6cf661d7361fb3). Details of stimulus presentation followed previous work (Turner and Rieke, 2016; Turner et al., 2018). Unless otherwise noted, mean light levels produced 4000 isomerizations (R*)/M or L-cone/s, 1000 R*/S-cone/s, and 9000 R*/rod/s; this corresponds to 3x10^-12 sugar-cube inches/rod/s (https://en.wikipedia.org/wiki/Denis_Baylor). Stimuli were restricted to a circular aperture with a diameter equal to twice the standard of a Gaussian fit to responses to the dependence of response on spot size (see Turner and Rieke, 2016); this process was repeated for each recorded cell. Spatial contrast of image patches was estimated by dividing the standard deviation of the pixel values in the receptive field center by the mean value. The average spatial contrast of the image patches probed was 0.3, and many patches had spatial contrasts exceeding 0.5.

Images were moved across the retina to simulate eye movements. For Figure 1, eye movements were taken directly from those measured in human observers viewing the corresponding image (Van Der Linde et al., 2009), resampled to our 60 Hz monitor refresh rate (see Turner and Rieke, 2016). For the modeling in Figure 9, we simulated eye movements using a random-walk process, with independent steps in x and y. Steps in the random walk occurred every ms, were discrete, had a size equal to the cone spacing in the model. This diffusive process was interrupted every 300 ms by a saccade-like jump in position. The discrete jumps in position consisted of random displacements in x and y drawn from a uniform distribution between −100 and +100 cone spacings.

Inhibitory synaptic inputs elicited by natural image patches are clustered in Figure 3. Input to the clustering algorithm consisted only of the time course of responses to all image patches sampled in recording from a given cell. The first three principal components of these combined responses provided a space for the clustering. Each individual response was projected along these three principal components, and the results were clustered using Matlab’s kmeans algorithm.

Subunit models were used to explore spatial integration. The goal of these models was to explore the impact of specific circuit features on predicted sensitivity to spatial structure rather than explain the responses of a specific cell to specific images.

These models consisted of regular grids of subunits for excitatory and inhibitory pathways. Spatial inputs were filtered through the receptive field of each subunit, the filtered signals were (optionally) passed through a rectifying nonlinearity, subunit outputs were summed to generate the RGC input, and this summed signal was thresholded at 0 to generate a predicted spike output. Subunit nonlinearities were set equal to the average excitatory and inhibitory contrast–response functions (see Figure 4—figure supplement 1), and outputs of excitatory subunits were given a threefold larger weight than outputs of inhibitory subunits to reflect the larger driving force associated with excitatory inputs near spike threshold. Subunit receptive field sizes and spacing were set to 1/5th the RGC receptive field center size for excitatory subunits, and inhibitory subunits were 50% larger. These reflect the subunit sizes indicated by responses to contrast-reversing gratings (Turner and Rieke, 2016).

Several properties of the inhibitory subunits were altered from this ‘base’ model for the analyses in Figure 4. Inhibitory subunit size and spacing were increased, the positive contrast part of the contrast–response function for inhibitory subunits was set to 0, or the subunit nonlinearity for inhibitory subunits was removed altogether, rendering the subunits linear.

A front-end model of cone phototransduction was added to the model for the analyses in Figures 6—8. This model consists of a set of differential equations that capture measured cone responses to a broad range of stimuli (Angueyra et al., 2022; Figure 6—figure supplement 1). All parameters of the cone model were set by previous measurements. Spatiotemporal inputs were passed through this front-end model, and the predicted cone responses provided input to subunit models constructed as above.

For the analysis of Figure 9, parasol models were simplified by removing the inhibitory pathway and making the excitatory pathway linear (for On parasol RGCs) or nonlinear (for Off parasol RGCs). These models captured the essential distinction between linear and nonlinear spatial integration.

Figure 7 uses stimuli designed to minimize nonlinearities in the cone responses. We used two models of the cone responses to identify these stimuli: (1) a full model of the cone responses that captures responses to a broad range of stimuli, including those invoking adaptation; (2) a linear model. The linear model was determined by the response of the full model to a brief, low-contrast flash (i.e., a flash within the linear range of the full model behavior). The stimulus for the full model was a transformed version of the original stimulus, while the original stimulus (untransformed) provided input to the linear model. We then sought a stimulus transformation that minimized the difference between the outputs of the two models. For sinusoidal stimuli, such as the contrast-reversing gratings we used in Figure 7, this is particularly simple: the response of the linear model to these stimuli is also sinusoidal, and hence our procedure identifies a stimulus to the full model that creates a sinusoidal output. We refer to this as a ‘light-adaptation clamp’ because the procedure aims to generate cone responses that lack adaptation.

We identified the appropriate stimulus transformation using a gradient-descent approach. We discretized the stimulus into time bins and then perturbed the stimulus at these discrete times. We retained perturbations that decreased the mean-squared difference between the two models’ responses (see Figure 7A) using Matlab’s fminsearch algorithm. We iterated this process while decreasing the size of the time bins until achieving a stable minimum of the mean-square difference.

Data, where appropriate, are plotted as mean ± SEM. Reported p values are from Wilcoxon rank sum tests.

The funders had no role in study design, data collection, and interpretation, or the decision to submit the work for publication.

We thank Mark Cafaro and Shellee Cunnington for excellent technical support. Tissue was provided by the Tissue Distribution Program at the Washington National Primate Research Center (WaNPRC), and we are grateful for assistance from the WaNPRC staff, especially Chris English. Qiang Chen, EJ Chichilnisky, Greg Field, Will Grimes, Mike Manookin, and Greg Schwartz provided invaluable discussions and feedback on an early draft of the manuscript. This work was supported by NIH grants F31-EY026288 (MHT) and EY028542 (FR).

Experiments were performed on primate retina obtained through the Tissue Distribution Program of the University of Washington's Regional Primate Research Center. Recordings were made from retinas from Macaca fascicularis, Macaca nemestrina, and Macaca mulatta of both sexes, aged 2–20 years. All use of primate tissue was in accordance with the University of Washington Institutional Animal Care and Use Committee (protocol 4140-01).

1. Received: May 22, 2021
2. Preprint posted: June 30, 2021 (view preprint)
3. Accepted: March 13, 2022
4. Accepted Manuscript published: March 14, 2022 (version 1)
5. Version of Record published: March 25, 2022 (version 2)

© 2022, Yu et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.