1. Neuroscience

Curvature-processing domains in primate V4

  1. Rendong Tang
  2. Qianling Song
  3. Ying Li
  4. Rui Zhang
  5. Xingya Cai
  6. Haidong D Lu  Is a corresponding author
  1. State Key Laboratory of Cognitive Neuroscience and Learning, IDG/MGovern Institute for Brain Research, Beijing Normal University, China
https://doi.org/10.7554/eLife.57502
Share this article
Cite this article
  1. Rendong Tang
  2. Qianling Song
  3. Ying Li
  4. Rui Zhang
  5. Xingya Cai
  6. Haidong D Lu
(2020)
Curvature-processing domains in primate V4
eLife 9:e57502.
https://doi.org/10.7554/eLife.57502
  2. Download BibTeX
  3. Download .RIS

Abstract

Neurons in primate V4 exhibit various types of selectivity for contour shapes, including curves, angles, and simple shapes. How are these neurons organized in V4 remains unclear. Using intrinsic signal optical imaging and two-photon calcium imaging, we observed submillimeter functional domains in V4 that contained neurons preferring curved contours over rectilinear ones. These curvature domains had similar sizes and response amplitudes as orientation domains but tended to separate from these regions. Within the curvature domains, neurons that preferred circles or curve orientations clustered further into finer scale subdomains. Nevertheless, individual neurons also had a wide range of contour selectivity, and neighboring neurons exhibited a substantial diversity in shape tuning besides their common shape preferences. In strong contrast to V4, V1 and V2 did not have such contour-shape-related domains. These findings highlight the importance and complexity of curvature processing in visual object recognition and the key functional role of V4 in this process.

Introduction

Shape extraction is crucial for object recognition and is a major function of the primate visual cortex. This process has been reported to occur in the ventral visual pathway (V1-V2-V4-IT) in a hierarchical manner (Connor et al., 2007). The mid-level area V4 plays an important role in this process (Roe et al., 2012; Pasupathy et al., 2020). Many V4 neurons selectively respond to curvatures (Pasupathy and Connor, 1999) and complex shape features (Desimone and Schein, 1987; Gallant et al., 1993; Kobatake and Tanaka, 1994). However, the distribution of these shape-selective neurons in V4 remains unclear.

In the V1-V2-V4-IT pathway, orientation maps have been reported in V1, V2, and V4. Functional maps for faces (Wang et al., 1996; Tsao et al., 2006) and various shape features (Fujita et al., 1992; Tsunoda et al., 2001; Sato et al., 2009) have been reported in IT. This processing hierarchy appears to lack functional structures for shape features of intermediate complexity, including curvatures and angles. A previous fMRI study reported several patches of the ventral cortex that prefer curved features with one being located in V4 (Yue et al., 2014). However, given the limited spatial resolution of fMRI signals, these patches were relatively large (at the cm-scale). It remains unclear whether there are submillimeter functional columns for curvatures that are similar to those for edge orientations.

Results

Curvature domains imaged using intrinsic signal optical imaging (ISOI)

Through ISOI on anaesthetized macaques, we imaged cortical hemodynamic signals for various contour features. The stimulus set included simple contour shapes, including circles and triangles, as well as parts of these shapes, including curves, angles, and short straight lines. As shown in Figure 1C, the contours were presented on a full screen and drifted along one of four or eight directions. Each stimulus condition was presented for 3.5 s and repeated 25–50 times in a random sequence. The light reflectance was imaged through an optical chamber covering parts of V1, V2, and V4 (Figure 1A and B). The exposed V4 region corresponded to the visual field of 0°−10° eccentricity in a lower quadrant. We obtained support vector machine (SVM) maps that compared cortical images collected during two stimulus conditions (Figure 1D–L). In the circle vs. triangle map (Figure 1D), the black and white patches corresponded to regions that preferred circles and triangles, respectively. This V4 pattern significantly differed from the orientation (Figure 1E) and color patterns (Figure 1F) obtained with established methods (Tanigawa et al., 2010; Li et al., 2013). In contrast to V4, we did not observe circle vs. triangle patterns in V1 and V2, or in the cortex anterior to the superior temporal sulcus (Figure 1D). We also examined single-condition maps (stimulus vs. blank); however, the responses were dominated by feature-non-specific activation and the shape-selective response patterns were not apparent (Figure 1—figure supplement 1).

Figure 1 with 2 supplements see all
Download asset Open asset
Intrinsic signal optical imaging (ISOI) of curvature domains in area V4.

(A) A schematic representation of the macaque brain showing the imaging region (green circle) and sulci locations. lu, lunate sulcus; st, superior temporal sulcus; and io, inferior occipital sulcus. (B) In vivo image of the blood vessel pattern in the 16 mm diameter imaging region in Case 1, which included parts of V1, V2, and V4. The exposed V4 region was between the st and lu sulci. A, anterior; M, medial. (C) An illustration of the full-screen stimulus pattern used for ISOI imaging. Circle diameter: 2.5°. Drifting speed: 4°/s. (D–L) Functional maps (SVM maps) from Case 1. The icons shown at the top represent the stimulus conditions being compared. In the ISOI maps, dark and white regions were preferentially activated by the stimulus icons shown on the left and right, respectively. For the maps shown in G, H, J, and K, data from different stimulus orientations were pooled for lines, curves, and angles. Other maps were obtained with the exact stimuli as shown on the maps. (D) The circle vs. triangle map shows clear patches in V4 (dark regions preferred circles and white regions preferred triangles), which were absent in V1 and V2. The dotted line represents the border between V1 and V2. (E) The orientation preference map shows the 45° (dark) and 135° (white) orientation domains in V2 and V4. The lack of orientation domains in V1 could have been resulted from the low SF of the stimulus gratings (0.25 c/degrees). (F) The color preference map shows the color domains in V1, V2, and V4. (G) The circle vs. straight line map shows a preference pattern similar to that in the circle vs. triangle map (D). (H) The curve vs. straight line map shows patterns similar to those in D and G. The curve was a half-circle. (I) The curve-orientation map shows smaller and weaker patches preferring different curve orientations. (J) The triangle vs. straight line map shows weaker and larger patches than those in the circle vs. straight line map in G. The dark domains preferred triangles over lines and occupied the same regions as the white patches in D. (K) The angle vs. straight line map shows a very weak pattern similar to that in J. (L) There was no clear pattern in the angle-orientation map.

Similar V4 patterns were observed in the circle vs. triangle (Figure 1D), circle vs. line (Figure 1G), and curve vs. line maps (Figure 1H). Although the strength of the dark patches in the latter two maps appeared weaker, they had similar general layouts and locations as the first one (also see Figure 1—figure supplement 2O–R for enlarged views). This similarity suggests that the key contour feature responsible for these dark patches was the curviness. Therefore, we have henceforth referred to these patches as ‘curvature domains’. These curvature domains contained smaller substructures. For example, a map comparing differently oriented curves showed smaller and weaker patches within the curvature domains (Figure 1I, Figure 1—figure supplement 2). Note that this curve orientation (also called direction) is different from the grating orientation (0–180°) and has a range of 0–360°.

Compared with the curvature domains, triangle-activated cortical regions appeared significantly weaker. In the triangle vs. line (Figure 1J) and angle vs. line maps (Figure 1K), the dark patches (angle-preferring) were apparently more diffused and weaker than those in the circle vs. line or curve vs. line maps (Figure 1G and H). This was also observed in the circle vs. triangle map (Figure 1D), which had fewer and weaker white patches (triangle-preferring) than dark patches (circle-preferring) (also see Figure 1—figure supplement 2G). Moreover, the orientation subdomains observed in the curve-orientation map (Figure 1I) were not present in the angle-orientation map (Figure 1L).

Imaging results of seven hemispheres (cases) of six macaque monkeys were consistent (Figure 2B, Figure 2—figure supplement 1B). We observed curvature domains in the entire exposed V4 surface representing both foveal and peripheral visual fields, but peripheral V4 tend to prefer larger circles (Figure 2—figure supplement 2A–D). Moreover, similar patterns can be repeatedly imaged from the same chamber over months (Figure 2—figure supplement 3). Modification of the experimental conditions (e.g. monocular left- or right-eye viewing conditions or binocular conditions, see Figure 1—figure supplement 2M and N) and noncritical stimulus parameters (e.g. contour brightness, filled disks, and regular vs. random contour arrangements) did not alter the main pattern features (Figure 2—figure supplement 2E–I).

Figure 2 with 3 supplements see all
Download asset Open asset
Comparison of curvature and orientation maps.

(A–D) Maps from Cases 2–4 are shown in rows 1–3. (A) The blood vessel patterns of the imaging regions. (B) The circle vs. triangle maps obtained using the same stimuli as those for Figure 1D. (C) Orientation preference maps showing the 45° vs. 135° orientation patterns in V1, V2, and V4. The dotted lines represent the borders between V1 and V2. Case 4 did not have V2 exposed on the surface. (D) The spatial relationship between the curvature and orientation domains (0°, 45°, 90°, 135°) in each case. (E) The curvature and orientation domains had similar domain sizes. (F) The curvature and orientation domains had similar response amplitudes. (G) The size of the overlap regions between the curvature and orientation domains was smaller than that of the random prediction (p=0.030; paired t-test, n = 7), which indicates a tendency of separation of these two types of domains.

The curvature domains had an average diameter of 518 ± 25 μm (mean ± s.e.m), which was similar to that of the orientation domains (479 ± 16 μm, p=0.074, paired t-test, n = 7; Figure 2E). The curvature domains occupied 12.1 ± 1.7% of the V4 surface, smaller than the coverage of the orientation domains (22.3 ± 3.7%, p=0.025, paired t-test, n = 7). The intrinsic optical signal amplitude (percentage change), which was measured from the raw subtraction maps, was numerically, but not significantly, larger for the circle vs. triangle maps than the orientation maps (curvature domains: 0.036%, orientation domains: 0.029%, p=0.15, paired t-test, n = 7, Figure 2F).

Both the curvature and orientation domains represent shape features; therefore, it is logical to expect that they might overlap. However, we observed little overlap between the curvature and orientation domains (Figure 2D). The overlap size was significantly smaller than that expected in a random distribution (p=0.03, paired t-test, n = 7, Figure 2G), which indicates an avoidance tendency between both domains. Moreover, the curvature domains did not overlap with the color domains (p=0.004, paired t-test, n = 7, Figure 2—figure supplement 1F). Contrastingly, the triangle-preferring white patches in the circle vs. triangle maps tended to overlap with the orientation (p=0.003, paired t-test, n = 7; Figure 2—figure supplement 1F) but not the color domains (p=0.17, paired t-test, n = 7; Figure 2—figure supplement 1F).

Two-photon calcium imaging of the responses to contour shapes

ISOI used a metabolism-based hemodynamic signal; therefore, the actual neuronal responses and neuron constitution within these domains were unclear. Moreover, ISOI could not provide information regarding the variations in different cortical layers. We then used two-photon calcium imaging to address these questions. Based on the ISOI functional maps, we injected AAV1/9-GCaMP6s virus into multiple locations in two chambers (Cases 3 and 4 in Figure 2A, Figure 3—figure supplement 1). The injections targeted either the centers of the curvature domains or regions outside these domains. We imaged the monkeys under similar anesthesia conditions 1.5 months after the injections. Figure 3C shows an image from Case 3 (red frame in Figure 3A and B, also see Figure 3—video 1) at a 210 μm depth. The center of this image frame contained a curvature domain (Figure 3B). Mapped with both manual and computer-controlled visual stimuli, the population receptive field (RF) of this region was ~4° in size and approximately 7° from the fovea (Figure 4—figure supplement 1A).

Figure 3 with 2 supplements see all
Download asset Open asset
Two-photon calcium imaging of neuronal shape responses.

(A) An image of the cortical region obtained from Case 3 in which AAV-GCaMP6s was injected for calcium imaging. The red and green frames indicate two regions examined using two-photon imaging. (B) Circle vs. triangle map of the region shown in A. (C) Neurons in the red-frame region shown in A and B that were imaged using a 16× objective at a 210 μm depth from the cortical surface. Two neurons marked in red are examined in detail in panel F. Scale bar applies to C–E. (D and E) Single-condition response maps (ΔF) for the circle (D), and triangle (E) stimuli. Each map was obtained after averaging 15 repeats and pooling of 4 orientations. (F) Responses (ΔF/F) of 3 neurons to 14 typical contour stimuli (showed at the bottom, red: curvature stimuli; green: rectilinear stimuli). The responses to circle and triangle were the best-orientation ones. Cells 1 and 2 were selected from the curvature domain shown in C. Although both neurons preferred circle stimuli, they exhibited different response levels to other stimuli types. Cell 3 was selected from a rectilinear region (marked in Figure 4F) and was strongly activated by the triangle stimulus and one of its line segment. The gray and blue lines represent individual trials and the average, respectively. The stimulus duration (2 s) is labeled at the bottom of the last row.

We tested 19 different curved and rectilinear contours, presented at different orientations, which resulted 73 unique stimuli (Figure 6A, also see Materials and methods). Each stimulus contained a single contour element similar to those used in the ISOI experiments (Figure 3—video 1). The stimulus moved across the population RF along a direction randomly chosen. Consistent with the ISOI results (Figure 3B), we observed significant activation of the neurons by circle stimuli (Figure 3D). The whole frame showed a scaled-down response to triangles (Figure 3E). The results demonstrate that different stimuli types mainly affected the population response amplitudes rather than activating different cells within the curvature domain.

We determined the preferences of curved and rectilinear stimuli for each pixel in the two-photon image (Figure 4B). This imaged region showed a preference for curved (red) over rectilinear (green) stimuli. Consistent preference patterns were observed for this two-photon map (Figure 4B) and the ISOI map from the same region (Figure 4A). Similar preference patterns were observed at deeper layers for this (Figure 4B and C) and all five locations imaged at multiple depths (Figure 4D–I, Figure 3—figure supplement 1). Therefore, the curvature domains form a columnar structure within at least the top V4 layers.

Figure 4 with 1 supplement see all
Download asset Open asset
Consistency between ISOI and two-photon imaging results.

(A) An enlarged view of a curvature domain in an ISOI circle-triangle map in Case 3 (the red-framed region in Figure 3B). (B) A curvature (red) vs. rectilinear (green) preference map for the same image used in Figure 3C. Each pixel is color coded for its preferred responses to the curvature (red-framed icons in Figure 3F) or rectilinear stimuli (green-framed icons in Figure 3F). The brightness of the pixels is proportional to the fluorescence strength and shape preference (see Materials and methods). (C) Preference maps for the same location shown in B but obtained from a deeper layer (350 μm from the surface). Contour-type preferences are similar as shallower depths (B). (D–F) As in A-C, but for another location in Case 3 (green-framed regions in Figure 3A and B). This region contained a subregion preferring curved stimuli (top right) and a subregion preferring rectilinear stimuli (lower left). Two-photon preference maps show consistent contour-type preferences with the ISOI map, as well as consistency between different depths. Cell 3 with an arrow was also used in Figure 3F. (G–I) As in above two sites, a third example obtained from Case 4, which contained a small curvature domain. Consistency between ISOI imaging and two-photon imaging, as well as consistency between different depths of two-photon images are evident. (J) Comparison of the ISOI and two-photon responses. The Y-axis represents differences in the fluorescent responses (optimal orientation) to circles and triangles in all neurons. The X-axis represents differences in hemodynamic responses for corresponding neuron pixels in the ISOI circle vs. triangle maps. There was a significant correlation between the two types of responses (Pearson r = 0.42, p=9.45×10−83, n = 1923). (K) Similar to J, there was a significant correlation between the fluorescent responses (optimal orientation) to curves vs. lines and ISOI responses to circles vs. triangles (Pearson r = 0.43, p=9.35×10−88, n = 1923). (L) The RF sizes of all neurons measured (Y-axis) had a negative correlation with the circle-triangle preferential responses of the corresponding pixels in the ISOI imaging (X-axis, same as the X-axis values in J and K). The dotted vertical line represents the threshold chosen (2SD) in determining whether a pixel is inside the curvature domains (the right side of the line) or outside (the left side of the line). (M) The average RF size was smaller for neurons inside the curvature domains (1.95 ± 0.032°, n = 583) than those outside (2.37 ± 0.038°, n = 835, p=1.15×10−14, t-test). Error bar: s.e.m.

We also performed PCA analysis on the response patterns of all the neurons. This data-driven analysis also revealed a major difference in curved and rectilinear responses, and its association with the neurons’ locations that either inside or outside the curvature domains (Figure 5).

Figure 5
Download asset Open asset
PCA results show different response patterns for neurons inside and outside the curvature domains.

(A) Percentages of the response variance accounted for by the top 10 principle components obtained from the PCA analysis on neurons’ response matrices (n = 1556). Stimulus orientations are sorted according to the responses before the PCA analysis. (B) PC1 shows a negative response relationship between cirlces and rectilinear lines, comfirming the ISOI findings. The color of each stimulus represents the sign and strength of the population response to that stimulus correlated to this principle component. PC1 explained 32.5% of the response variance. (C) As in B, for PC2, which shows positive contribution from many stimuli except for the circles, PC2 explained 23.7% of the variance. (D) All neurons (n = 1556) plotted according to their PC1 and PC2 coordinates. Although had a large overlap, curvature neurons (red) tended to separate from the rectilinear neurons (green). Neurons in gray were those located outside of these two types of domains. (E) An example two-photon site in which neurons were labeled according to their PC1 coordinates. Neurons with positive PC1 coordinates (red) were mostly found inside the curvature domain on the top right cornor (with white outline). Neurons with negative PC1 coordinates (yellow and green) were mostly located outside the curvature domain.

Although neurons inside the domains exhibited a common preference for curve stimuli, individual neurons showed diverse response properties. Figure 3F shows the responses of 3 example neurons to 14 typical stimuli. Two neurons were selected from the curvature domain showed in Figure 3C (red markers). Both neurons were strongly activated by circle stimuli; however, cell 2 was also activated by multiple other stimuli. Cell 3 were selected from a region highly activated by rectilinear stimuli (Figure 4F arrow) and it showed strong responses to the triangle as well as a 45°-orientated line that resembled an edge of the triangle. Thus, at cellular level, contour-tuning differences exist not only between cells belonging to different functional domains but also between cells within a functional domain. This is further explored in Figure 6.

Figure 6 with 2 supplements see all
Download asset Open asset
Microarchitectures inside the curvature domains.

(A) Response matrix of an example ‘curve-orientation-preferring neuron’, which showed strong preference for a half circle oriented at 90° and an oval contained a similar curve fragment. (B) Orientation tuning curve for curves (half circles). Green dots represent response values to the curves at eight different orientations. Blue line represents fitting curve for the responses. Red circle represents the neurons’ average response level (a single value) for the circle stimuli. Pale-blue and red lines represent ± s.e.m of the responses. (C, D) As in A and B, but for a ‘dual-preference neuron’. This neuron showed significant orientation tuning to curves, along with a maximal response for the circle. (E, F) As in A-D, but for a ‘circle-preferring neuron’. This neuron responded best to the circles, but did not exhibit curve-orientation tuning. There was no fitting curve for the neuron shown in F due to its weak responses. (G) Neurons tended to cluster according to their preferences for circles or curves. Three maps are from three different two-photon imaged locations, and neurons are color coded as shown in the icon: red: circle-preferring neurons; green: curve-orientation-preferring neurons; yellow: dual-preference neurons. Neurons in gray did not pass the circle or curve-orientation preference tests. The neurons in the dotted ovals are further examined in Figure 7. (H) Neurons showed curve-orientation tuning (including curve-orientation-preferring and dual-preference neurons) tended to cluster according to their preferred orientations. The color of the neurons represents their preferred orientation (0–360°). (I) The percentages of the three neuron types inside and outside the curvature domains. The color code is the same as that in G. (J) The average cell-to-cell distances for circle-preferring neurons (red) and neurons showing curve-orientation tuning with similar preferred orientations (differences < 45°, green and yellow) were shorter than the overall average distance (dotted line) (p<0.001, t-test). Error bar: s.e.m. (K) Population averaged fluorescent responses to different contours. For each contour, the response to the optimal orientation was used. The circle-preferring neurons (red) showed gradual increase of response with the length or completeness of the circle, while the curve-orientation-preferring neurons (green) did not. Error bar: s.e.m. (L) As in K, but used orientation-averaged responses.

We identified 1923 neurons in 10 two-photon imaged locations (Figure 3—figure supplement 1); among them, 788 neurons were in the curvature domains while 1135 neurons were outside of these domains. Despite certain variability due to alignment error, a comparison of the neurons’ calcium responses and their corresponding pixel responses in ISOI maps revealed a general consistency between the two measurements (Figure 4J and K). Figure 4J and K also shows that more neurons had a greater response to curved stimuli than to rectilinear ones (more neurons had a positive Y axis value in both figures). Overall, the neurons with a stronger response to curved stimuli tended to have smaller RFs (Figure 4L and M, Figure 4—figure supplement 1B-E) and greater surround suppression (Figure 4—figure supplement 1F and G), which is consistent with previous findings on the relationship between end-stopping properties and curvature selectivity (Hubel and Wiesel, 1965; Dobbins et al., 1987; Ponce et al., 2017).

Microarchitectures of the curvature domains

Curvature domains imaged with curves and circles in ISOI were generally similar (e.g. Figure 1G and H). However, individual neurons inside the curvature domain often showed different responses to these two stimuli. Figure 6A,C and E shows response matrices of three example neurons selected from the curvature domains. Each neuron showed a wide response spectrum to the 73 contour stimuli. To analyze different responses to curves and circles, we labeled neurons that exhibited significant orientation tuning to curves as ‘curve-orientation-preferring neurons’ (e.g. Figure 6A and B) and those with stronger responses to circles than to curves as ‘circle-preferring neurons’ (e.g. Figure 6E and F). Some neurons that met both criteria were labeled ‘dual-preference neurons’ (e.g. Figure 6C and D). Note that after dual-preference neurons were isolated into a separate group, the earlier two groups no longer contained dual-preference neurons. PCA analysis of response patterns for neurons inside curvature domain also showed differences between circle-preferring neurons and curve-orientation-preferring neurons (Figure 6—figure supplement 1). Among the 788 neurons imaged inside the curvature domains, circle-preferring comprised 25.6% (202), dual-preference comprised 14.4% (113), and curve-orientation-preferring neurons comprised 19.2% (151), with the rest not passing the statistical tests. The corresponding proportions of these three groups in the 1135 neurons outside the curvature domains were 7.3% (83), 1.8% (20), and 11.7% (133), respectively (Figure 6I). As expected, the summed proportion of these three types of neurons was larger inside the curvature domains (59.1%) than the outside (20.8%). The ratio of circle-preferring to curve-orientation-preferring neurons also appeared to be larger inside (1.35) than outside (0.62) the curvature domains.

When neurons were labeled according to their preferences, there was a tendency of spatial clustering according to their types (Figure 6G). Moreover, the curve-orientation-preferring neurons further clustered based on their preferred orientations (Figure 6H), which is consistent with the ISOI results (Figure 1I, Figure 1—figure supplement 2A–D). The average distances for within-group neuron pairs were shorter than the average distance between two randomly selected neurons (Figure 6J). Periodicity analysis shows the size of curve-orientation subdomain was around 360 μm (Figure 6—figure supplement 2).

Circle-preferring neurons also showed some interesting features. Figure 6K plots average responses to different contours for circle-preferring neurons (red) and curve-orientation neurons (green). For each stimulus, neurons’ maximum orientation-responses were averaged. For circle-preferring neurons, their responses increased approximately linearly with the length of the circle fragment. This was not the case for curve-orientation neurons. Also, their responses to closed shapes seemed to be proportional to the general similarity between the shape and the circle. As a comparison, Figure 6L plotted their orientation-averaged responses. The two curves are similar to those in Figure 6K, except that the green curve becomes lower on the left side, reflecting the modulation of curve orientation on the responses of the curve-orientation-preferring neurons.

Diversity of contour tuning for neurons in the curvature domains

So far, we have shown the common response properties for neurons within the curvature domains and their subdomains. To have a full picture of the neuronal constituents within these functional domains, we examined the diversity of neuronal tunings inside the curvature domains.

In Figure 7, we show single-neuron response matrices from three example two-photon sites, including a circle-preferring site (A), a curvature-orientation-preferring site (B) and a rectilinear-preferring site (C). The location of these sites can be found in Figure 6G. For each site, 10 example neurons are shown. For the circle-preferring neurons (A), they prefer circles much better than the half circles. Their responses patterns also exhibited certain diversity, including some neurons responded modestly to the 90°-angle stimuli (cell 8–10) while others did not. They could respond well (cells 1, 2, 3, 5, 7, 9) or modestly (cells 4, 6, 8, 10) to hexagons. For cells in curve-orientation-preferring site (Figure 7B), response diversity appeared even larger. Different neurons showed strong selectivity to the trefoil shapes (cells 1, 4), or particular 90° angles (cell 3). Different degrees of circle-preference were also apparent among these neighboring neurons.

Figure 7
Download asset Open asset
Single-neuron responses in three types of cortical sites.

(A) Response matrices of 10 neighboring neurons in a circle-preferring two--photon site (also shown in Figure 6G). Each matrix is normalized to its maximal and minimal responses. (B) As in A, but for a curve-orientation-preferring site. (C) As in A and B, but for a rectilinear site.

The rectilinear site shown in Figure 7C was also located in an orientation domain that preferring the 45° orientation. Many cells responded well to the triangles and the 45°-oriented lines. Some cells also responded to the 45°-orientated ovals and flat curves, likely due to their general orientation appearance. However, neighboring neurons could respond strongly to one of these stimuli (e.g. cell 1 to oval, cells 2 and 3 to triangle, cell 4 to line) but much weaker to others. Some neurons (e.g. cell 9) exhibited a wider range of preferred shapes than their neighbors.

Thus, for neighboring neurons, although their general shape preferences were similar, which constitute the basis of the ISOI signals, their contour response patterns still vary from cell to cell, and some of the differences were substantial. The degrees of such diversity also seem to be related to the types of these sites in V4.

Shape-related responses of neurons in curvature domains, rectilinear domains, and those outside of these two were summarized in Figure 8. In the orientation-sorted average response matrices, curvature neurons showed much higher response amplitudes, and larger differences between curved and rectilinear responses, than the other two types of neurons. Nevertheless, these neurons also responded to rectilinear stimuli, for example the triangles, at about half of the response amplitude. Interestingly, neurons in rectilinear domains did not show prominent differences in their responses to rectilinear and curved stimuli. This is also consistent with the weak rectilinear domains in ISOI imaging (Figure 1D). Figure 8D further illustrates the differences among three regions, in which the size and color of the shapes are proportional to the response magnitudes of the first rows in the response matrices in A–C.

Figure 8
Download asset Open asset
A summary of neurons’ response patterns in different V4 regions.

(A–C) Average response matrices (without normalization) from neurons in the curvature domains (A), rectilinear domains (B) and those outside of these two types of domains (C). Responses are orientation-sorted before averaging. (D) An illustration of the contour-preferences for neurons in the three regions, calculated based on the optimal-orientation responses in A–C (the top rows). The size and color of the shapes represent the strength of the corresponding responses. For clarity, not all stimulus icons were drawn.

Discussion

Using two complementary imaging methods, we found a novel functional structure in V4, but not in V1 or V2, for curved contour processing. The size of these domains (518 µm) was similar to that of the orientation domains in this area; moreover, they tended to occupy regions outside the orientation domains. Within these domains, neurons preferring different curve features (e.g. curve orientation) further clustered into smaller subdomains. Interestingly, subdomains were also found in color- (Li et al., 2014; Ghose and Ts'o, 2017; Liu et al., 2020), orientation- (Ghose and Ts'o, 1997; Tanigawa et al., 2010), and direction- (Li et al., 2013) specific regions in V4. These nested functional architectures are similar to that of category domains in IT, where large category domains (animate vs. inanimate) contain subcategory domains (e.g. for face and body parts) (Liu et al., 2013; Grill-Spector and Weiner, 2014). Curvature domains are probably vertical columns. However, two-photon imaging revealed considerable complexity underlying these functional organizations. First, individual neurons usually responded to a wide range of stimuli with different levels of response amplitudes. Thus, curvature domains are not ‘curvature-only’ domains. Second, neurons inside the curvature domains and those outside do not completely differ in their stimulus coverage, although their optimal stimuli are usually different. Third, for neighboring neurons within a functional domain, although they showed common shape preferences, many of them exhibited different individual preferences, some of which were substantial. Thus, the functional maps we observed are best understood as a certain orderness of the complexity of shape responses in V4 population.

Consistent with previous findings (Pasupathy and Connor, 1999; Pasupathy and Connor, 2001; Pasupathy and Connor, 2002; Yue et al., 2014), we confirmed that V4 plays a key functional role in the representation and processing of curvature information. A majority of V4 neurons have been reported to exhibit complex multi-dimensional response patterns to shape stimuli (Gallant et al., 1993). Recent studies in V4 (Bashivan et al., 2019) and IT (Ponce et al., 2019) have shown that individual neurons in these areas can be best activated by non-natural synthetic patterns that are much more complex than those traditionally tested. Thus, classifying neurons into functionally separate groups based on their contour responses is often difficult. This is especially true when the testing stimulus set is limited (as in the present work). However, intrinsic functional structures in an area are suggestive of particular feature dimensions been emphasized in this area. We observed that V4 contains clear maps for curvature processing, which suggests that curvature is a unique feature dimension in this area, not only in single neurons but also as a populational feature. Further, both curved and rectilinear contours formed functional structures (curvature and orientation domains) with each functional structure containing nested subdomains for further detailed tuning (Figure 6G and H). This functional organization could underlie the parallel processing of curved and straight contours (Ito, 2012) and the different underlying pooling mechanisms (Nandy et al., 2013).

In human fMRI studies, V4 is more activated by circular stimulus than by curves (Dumoulin and Hess, 2007) or parallel patterns (Wilkinson et al., 2000). Using two-photon imaging, we observed many circle-preferring neurons (Figure 6I). However, these circle-preferring neurons also showed good responses to other non-circle stimuli (Figure 6E). Many curve-orientation-preferring neurons also responded best to circles (Figure 6C). This indicates that these circle-preferring neurons do not constitute a functionally distinct subdivision of V4.

Early single-cell recordings have suggested that V4 neurons may cluster based on their color or orientation preferences (e.g. Tanaka et al., 1986). Subsequent optical imaging studies confirmed these functional structures (Ghose and Ts'o, 1997; Tanigawa et al., 2010; Li et al., 2014). For shape-responsive neurons, clustering according to their selectivity to complex patterns has also been suggested (Tanaka et al., 1986; Gallant et al., 1993; Gallant et al., 1996), but studies have also shown that neurons selective to complex shape features are mixed with those selective to simple features such as orientations (Kobatake and Tanaka, 1994). Our results confirmed that, although neurons tended to cluster according to their common response preferences, substantial diversity in contour tuning still exist among these clustered neurons. This heterogeneity may make these functional organizations less detectable in electrophysiological studies, for example, compared to the orientation columns in V1.

In a recent two-photon calcium imaging study posted on bioRxiv, Jiang et al., 2019 imaged cellular responses in parafoveal V4 representations (~1° eccentricity from the fovea) to a large set of natural and artificial stimuli. Their findings suggested a separate coding for orientation, curve and angle. However, due to the spatial limitation of the two-photon technique, domain features and distributions are not available. With ISOI over a large imaging window (16–20 mm diameter), we observed the distribution of the curvature domains and their spatial relationships with other known functional maps (Figure 2D, Figure 2—figure supplement 1). We found that curvature domains existed in the entire exposed V4 surface (0–10° eccentricity from the fovea). Further, the ISOI maps of the curvature and orientation domains were comparable (Figure 2F), which suggests that they had a comparable degree of clustering.

In another study also submitted to eLife, Hu et al., 2020 investigated curvature organization in area V4 with ISOI. Despite a different stimulus protocol they used, they observed curvature domains similar to what we observed. The major ISOI findings are similar between these two studies. With the power of two-photon imaging, we not only confirmed these ISOI findings but also revealed neuronal properties underlying these domains, including cellular-level preferences and the tuning diversity of neurons within these functional domains. Thus, Hu et al., 2020 and this study both revealed the overall features of curvature domains in V4, while Jiang et al., 2019 and this study both revealed cellular-level features of the functional domains. Only this study directly linked functional domain properties with their underlining neural responses.

Curve-preferring neurons have been reported in V1, V2, and V4 (Hegdé and Van Essen, 2007; Ponce et al., 2017). However, we did not observe functional structures for curves in V1 and V2, which is consistent with a previous study (Yue et al., 2014). Curvature domains emerge in V4 (Yue et al., 2014; this study) and remain present in IT (Yue et al., 2014; Srihasam et al., 2014). Therefore, V4 contains both basic orientation (similar to V1 and V2) and curvature maps (similar to IT), which indicates its intermediate-stage role in the shape processing hierarchy. There is a progressive change in multiple feature dimensions along this hierarchy, for example receptive field size, response linearity, and degree of modulation by top-down controls. Functional architectures for shape processing form an independent dimension, which could contribute to a better understanding of the neural mechanisms for shape processing. Moreover, the consistency between functional architectures and shape processing levels highlights the importance of functional organization in the brain. This study provides another measurement type (domain-level activation) for studies on shape processing in addition to the cellular- and voxel-level measurements. Interesting topics, such as the functional importance of clustering, the plasticity of such clustering and its contribution to cognition, can be further explored.

We have shown that curvature neurons had smaller RFs than those of non-curvature neurons (Figure 4L and M). We further analyzed and showed that curvature neurons had stronger surround suppression (Figure 4—figure supplement 1F and G). Thus, consistent with previous findings, curvature neurons did exhibit end-stopping properties. However, curvature neurons showed additional curve-specific preferences, including curve-orientation tuning, preference for curves over similarly-sized angle stimuli, etc. For circle-preferring neurons, they prefer circles (larger) than the half-circles (smaller). These additional features suggest that curvature neurons were not simple end-stopping neurons, although they exhibited end-stopping properties.

Curvature domains are also unlikely the suppression-domains (S-domains) in V4 revealed by comparing activations to small and large patches of gratings (Ghose and Ts'o, 1997). In our study, curvature domains were revealed in difference maps, including: circles vs. triangles (Figure 1D), circles vs. bars (Figure 1G), curves vs. bars (Figure 1H), and curves vs. angles (Figure 1—figure supplement 2L). The two stimuli in each pair had similar sizes, and thus had similar surround suppression effects. Their contrast pattern in the difference maps could not be due to differences in suppression. The stimuli on the two sides also had balanced orientations, plus their maps had little overlap with the orientation maps (Figure 2D). The only consistent contrasts in these pairs of stimuli was the curvature feature. Thus, the curvature domains we found are unlikely related to suppression and orientation domains.

Our two-photon results have limitations in terms of the stimulus set, anesthetized preparation, and calcium signals. We only tested 19 simple contour shapes, mostly with four orientations. Considering the complexity of single-cell response patterns we observed, the best-driven stimuli we identified may not be the optimal ones for the neurons. Although a more in-depth analysis of the current data set can still improve our estimation of the optimal stimuli, new stimuli, or experimental approaches seem to be necessary. A recently reported deep-learning-network approach successfully found optimal stimuli for V4 neurons (Bashivan et al., 2019). The optimal stimuli they found were usually complex patterns and unlike the contour shapes previously tested. However, these optimal stimulus patterns also contained specific curvature features similar to the optimal ones obtained with a parametric stimulus set like ours. Thus, it is likely that the best-driven contour features we obtained should also be represented in the actual optimal stimuli for the neurons we measured. Our ISOI tests also provided a practical way in designing and narrowing down the stimulus set for the subsequent targeted two-photon imaging.

The primary new finding of this study is the functional architecture for curvature processing in V4. The curvature domain likely shares similar formational rules and functional significances with other functional architectures previously discovered (Mountcastle, 1997; Chklovskii and Koulakov, 2004). The significance of this particular structure is based on several observations described in Results: First, the curvature domains were found in V4 but not in V1 or V2, although curvature neurons were also found in the latter two areas (Hegdé and Van Essen, 2007; Ponce et al., 2017). Second, the domain strength was higher for V4 curvature domains than those for angles. These two factors are consistent with the notion that V4 plays an important role in curvature processing. Third, features of these domains are helpful in understanding the underlying neural processes. For example, the relative separation of the curvature and rectilinear domains suggests a parallel processing of these two contour features. The existence of the circle and curve-orientation sub-domains within the curvature domains suggests that there are different processes for curvatures. Finally, the existence of curvature domains also provides an efficient way to study curvature neurons in V4, for example, by targeted recording, imaging, or tracer labeling.

Materials and methods

Key resources table
Reagent type
(species) or resource		DesignationSource or referenceIdentifiersAdditional information
Strain, strain background (Macaque, male)Macaca mulattaSuzhou Xishan Zhongke animal Company, Ltd
Hubei Topgene Biotechnology Co.,Ltd		http://xsdw.bioon.com.cn/ http://topgenebio.com/
Strain, strain background (Macaque, male)Macaca fascicularisBeijing Inst. of Xieerxin Bology Resourcehttp://www.xexbio.com/
Recombinant DNA reagentAAV1.Syn.GCaMP6S.WPRE.SV40Addgenev25497
Recombinant DNA reagentAAV9.Syn.GCaMP6S.WPRE.SV40AddgeneCS1282
Software, algorithmMATALAB R2017bMathWorkshttps://www.mathworks.com
Software, algorithmCodes for ISOI data analysisThis paperhttps://osf.io/qydj5/
Software, algorithmCodes for 2P data analysisThis paperhttps://osf.io/qydj5/

A total of seven hemispheres (i.e. cases) from six adult male macaque monkeys (five Macaca mulatta, one Macaca fascicularis) were imaged. ISOI were performed on all seven cases, two-photon imaging were performed on two cases (Case 3 and Case 4). These monkeys also participated in other studies. All procedures were performed in accordance with the National Institutes of Health Guidelines and were approved by the Institutional Animal Care and Use Committee of the Beijing Normal University (protocol number: IACUC(BNU)-NKCNL2016-06).

Surgery procedures

Request a detailed protocol

Animals were sedated with ketamine (10 mg/kg) or Zoletil (tiletamine HCl and zolazepam HCl, 4 mg/kg) and transferred to the lab. They were artificially ventilated on a stereotaxic and anesthetized with isoflurane (1.5–2.5%) during surgery. A 22–24 mm (diameter) circular craniotomy and durotomy were performed (center location, 30–37 mm from midline, 15–24 mm from posterior bone ridge) to expose visual areas V1, V2, and V4 (illustrated in Figure 1A). ISOI was performed right after the surgery (see below). Then, one of two types of protocols was chosen: For cases only used for ISOI, we implanted an optical chamber and recovered the animal. For cases used for two-photon calcium imaging, we injected virus into the visual cortex, placed back the bone and sealed the craniotomy (described below). After 1.5 months, we reopen the craniotomy and implanted an optical chamber for the following weekly based two-photon imaging experiments. The same type of chronic chamber was used for ISOI and two-photon imaging (Li et al., 2017). The inside diameter of the chamber was 13–16 mm, thickness of the glass is 0.18 mm. Case 2 was only imaged once and then used for other studies thus no chamber was implanted.

Intrinsic signal optical imaging (ISOI)

Request a detailed protocol

ISOI were performed either right after the surgery (without a chamber), or on a weekly-base after a chamber was implanted. Imaging usually lasted for 8 hr. Right before the imaging, anesthesia was switched from isoflurane to a mixture of propofol (induction 2 mg/kg, maintenance 2 mg/kg/hr) and sufentanil (induction 0.15 μg/kg, maintenance: 0.15 μg/kg/hr) (in four cases), or to Zoletil (tiletamine HCl and zolazepam HCl, induction 4 mg/kg, maintenance 1.25 mg/kg/hr, in three cases). The monkeys were immobilized with vecuronium bromide (induction 0.25 mg/kg, maintenance 0.05 mg/kg/hr) to prevent eye movements. Anesthetic depth was assessed continuously via monitoring the electrocardiogram, end-tidal CO2, blood oximetry, and body temperature. Eyes were dilated (atropine sulfate, 0.5 mg/ml) and fit with contact lenses of appropriate curvatures to focus on a stimulus screen 57 cm from the eyes.

Images of reflectance change (intrinsic hemodynamic signals) corresponding to local cortical activity were acquired (Imager 3001, Optical Imaging Inc) with 632 nm illumination. Frame size was 540 × 654 pixels, representing either 15.5 × 19 mm or 18 × 22 mm of imaged area, depending on the lenses chosen. For each trial, imaging started 0.5 s before the stimulus onset and collected at 4 Hz frame rate. Each visual stimulus was presented for 3.5 s. The total imaging time for each trial was 4 s, during which 16 frames were imaged. Interstimulus intervals were at least 6 s. Stimulus conditions were displayed in a randomized order.

Virus injection

Request a detailed protocol

Virus injection and two-photon imaging procedures were similar to those described in Li et al., 2017. In two cases (Case 3 and Case 4), we injected 500 nL AAV9.Syn.GCaMP6S.WPRE.SV40 (CS1282, titer 3.34e13 GC/ml, Addgene), or AAV1.Syn.GCaMP6S.WPRE.SV40 (v25497, titer 2.5e13 GC/ml, Addgene) into 10–15 cortical locations at a depth of 500 μm. After virus injection, cortex was covered with a piece of artificial dura. And the remaining dura was covered and glued to the top of the artificial dura with medical adhesive (Beijing Compont medical devices Co. Ltd). The original bone was placed back, secured with titanium lugs and bone wax in the gap. The scalp was sutured. A second surgery was performed 1.5 months later, in which the old craniotomy was reopened and an optical chamber was implanted for the following ISOI and two-photon imaging.

Two-photon imaging

Request a detailed protocol

Two-photon calcium imaging was performed on a weekly-base after the chamber implanting. Animal anesthesia and preparation were the same as those in the ISOI experiments. Two-photon microscope was a Brucker Ultima IV Extended Reach (Bruker Nano Inc). Laser was generated with a Chameleon Ultra II (Coherent Inc). The excitation wavelength of the laser was set at 980 nm. Scanning frame rate was 1.3 Hz in a galvo scanning mode. Under a 16X objective (0.8 N.A., Nikon), 515 × 512 pixel images were collected, representing a 830 × 830 μm cortical surface. Imaging was continuous and the beginning of each stimulus presentation was synchronized with the beginning of a frame scanning.

As the microscope was vertical, we rotated the stereotaxic and the animal for ~45° so that the chamber plane was perpendicular to the laser beam. We imaged 10 cortical locations in the two virus-injected chambers (Figure 3—figure supplement 1A and E). Images were normally collected at a depth between 210 and 350 μm from the cortical surface. In six locations, we imaged at two depths, a depth around 230 μm and a depth around 310 μm (Figure 3—figure supplement 1D and H). During the imaging session, slow drifts of cortex in the imaging window was observed. The drift was normally less than 50 μm in the X-Y plane and less than 150 μm along the Z-axis in a course of 6–8 hr. We closely monitored the cell features during the imaging and adjusted position of the focal plane accordingly. Drifts in the X-Y plane were further corrected in offline data analysis (described below). In addition, population receptive field locations were re-plotted every 1–1.5 hr in order to check the stableness of the eye positions. In half of the experiments, some eye drift (1–2°) was detected during the imaging session. If drift was larger than 1°, we repeated the last stimulus run at the new receptive field position. If the drift was smaller than 1°, we adjusted the stimulus location for the following stimuli and continued the imaging. Replication of two-photon imaging of the same neuron population were tried but was not systematically tested. A general replication of the population-level responses was qualitatively observed.

Visual stimuli for ISOI

Request a detailed protocol

Visual stimuli were generated using ViSaGe (Cambridge Research Systems Ltd.) and presented on a CRT monitor positioned 57 cm from the eyes. The stimulus screen was gamma corrected and worked at 100 Hz refreshing rate. We compared contour preference maps obtained with monocular and binocular conditions in Case 1 (Figure 1—figure supplement 2M and N), as well as two monocular conditions in Case 4 and did not find obvious differences. Since ISOI signal is normally stronger in binocular condition, all but one case (Case 5) were imaged in binocular conditions. For orientation and color preference maps, full-screen drifting sinewave gratings were used. The mean luminance was 28.9 cd/m2. Gratings of two SF (0.25 and 1 c/deg, except for Case 6 in which only 0.25 c/deg were tested) and two (45° and 135°, for Cases 5 and 6) or four orientations (0°, 45°, 90°, and 135°, for other five cases) were tested. Gratings were drifting at 4°/s along a random direction perpendicular to its orientation and were presented in a random order. The initial phases of the gratings were also randomly selected.

For contour shape preference maps (e.g. Figure 1D–L), drifting contour patterns were used. Each contour element was ~2.5° and placed in a 3 × 3° grid (Figure 1C). The white contour lines was 0.2° in width and had a luminance of 111.2 cd/m2. The dark background had a luminance of 20.6 cd/m2. For each trial, a contour pattern was presented and drifted at one out of four or eight directions for 3.5 s at a speed of 4°/s. Interstimulus intervals were 6 s, during which a gray screen (20.6 cd/m2) was presented. The initial phase (i.e. relative position) of the pattern was random. Each stimulus was repeated for 25–50 times. The essential stimuli (e.g. circle, triangle, gratings) was obtained in all seven cases (Figure 2—figure supplement 1), new stimuli were added in later cases and are described in the Results whenever is necessary.

Visual stimuli for two-photon imaging

Request a detailed protocol

Visual stimuli used for two-photon imaging were generated in a similar way as those in the ISOI imaging and presented on a 21-inch LCD monitor (Dell E1913Sf) positioned 57 cm from the eyes. The stimulus screen was gamma corrected and worked at 60 Hz refreshing rate. All stimuli were bright stimulus (80.8 cd/m2) on a gray (13.2 cd/m2) background and presented monocularly to the left eye.

Population RF location was first mapped manually with a 2–3° circular patch of square-wave gratings, during which cortical fluorescent response was monitored. Then the RF were systematically mapped at the potential region with a single 2–3° patch of stimulus presented on a grid of 5 × 5 locations. Each position was tested with a square-wave grating, a random dot patch, and a circle, each moved at one out of four directions (45°, 135°, 225°, 315°) at 4°/s speed. Each stimulus was presented for 1.5 s. Interstimulus interval was 0.5 s. Population RF center (normally 5–7 degrees eccentricity) was then analyzed online based on cortical responses to these stimuli.

To obtain detailed RF size information, we presented circular patches of square-wave gratings (SF = 1 c/degrees, TF = 4 c/s, duty cycle = 0.2) of six different sizes (0.5°, 1°, 2°, 4°, 8°, 12°) at the population RF center mapped before. Each stimulus was drifted along one of eight directions, and presented for 2 s with a 3 s interval. Each stimulus was repeated for seven times. From these stimuli, a size-tuning curve was calculated, which normally peaked at patch sizes of 2°–3.5°. Population RF size was then defined as twice of this size (i.e. 4°–7°).

To test the contour-shape preferences of the neurons, we presented single contour elements in the population RF. Single contour element was the same as those used in the ISOI experiments. The size of the contour was adjusted to 40% of the population RF. Each stimulus was presented for 2 s, during which it first appeared in the center of the population RF and drifted along one of eight directions (randomly selected). After it moved to the edge of a virtual window equal to the population RF diameter, it disappeared and reappeared in the opposite side of the window, and continue its motion along the same direction (illustrated in Figure 3—video 1). The speed of the contour movement was adjusted to (half of the diameter of the population RF)/s. The interstimulus intervals were 3 s. Each stimulus was usually repeated for 10–15 times. As shown in Figures 6A, 19 different contour shapes were used, most of them was presented in four orientations, two stimuli were presented in two orientations (square, hexagon), and one stimulus (half circle) was presented in eight orientations. This made up 73 different stimulus conditions. We also included four identical circles instead of 1 to be comparable to other stimuli’s four orientations. The final stimulus matrix contained 76 stimuli.

ISOI data analysis

Request a detailed protocol

ISOI data analysis was performed using MatLab 2017 (The MathWorks, Natick, MA). We first obtained ΔR/R response to each stimulus using following formula ΔR/R=(R8-16 - R1-4)/R1-4, in whichR8-16 is the average of frames 8–16, R1-4 is the average of frames 1–4. The ΔR/R frames for each trial were then used for following analysis.

We used a pattern classifier (support vector machine, SVM) for calculating contrast maps between ΔR/R frames of two stimulus conditions. Compared with simple subtraction maps, SVM weight map achieved a higher signal-noise ratio by suppressing blood vessel noise and low-frequency noises (Xiao et al., 2008; Chen et al., 2016). The Matlab SVM program was provided by Chih-Jen Lin (LIBLINEAR: A Library for Large Linear Classification, 2008; available at https://www.csie.ntu.edu.tw/~cjlin/liblinear/).

For each SVM map (e.g. Figure 1D–L), it was first Gaussian low-pass filtered (size = 5 pixels, std = 1), then clipped at 0 ± 8 SD for display, in which SD was the standard deviation of the background pixels values (blood vessel pixels and outside-chamber pixels, determined based on the blood vessel maps like the example shown in Figure 1B). For display purposes, the original map size (540 × 654 pixels) was cropped to 540 × 540 pixels by trimming off some background regions on the two edges.

To separate domain region from non-domain regions (e.g. Figure 2D), we used 0 ± 2 SD as the threshold levels and discarded resulting regions containing less than 50 pixels. Curvature domains were identified as the dark regions in the circle vs. triangle maps. Domain coverage in Figure 2D and the neuron classification in Figures 5–8 were all based on this method. Only in estimating domain sizes (Figure 2E), we used a watershed method in order to deal with the ‘connected domain’ problem (see below). Orientation domains were identified as both the dark and white regions in the orientation maps. Color domains were identified as both the dark and white regions in the color vs. luminance maps. To compare with two-photon results (e.g. Figure 4J and K), regions in the SVM map that aligned with the two-photon imaging area were extracted and resized (bicubic interpolation) to 512 × 512 pixels for further analysis.

To estimate domain sizes (e.g. Figure 2E), neighboring domains were often connected and we separated them using a watershed algorithm. First, the image values were reversed to make the negative domains positive. Then we located the strongest response pixel, and cut out a 31 × 31 (0.9 × 0.9 mm) pixel region around this pixel. For all pixels in this square that had a value lower than 0+2SD, they were set as 0+2SD. The response pattern was fitted with a two-dimentional Gaussian. An oval was obtained from the two-dimentional Gaussian with a threshold so that the oval had half of its pixels valued larger than 0+2SD (and the other half pixel all had values of 0+2SD). This oval was taken as the representation of this curvature domain. Before search for the next domain, the pixels in the oval region in the original map was set as 0+2SD. The searching stopped until no more pixels had value larger than 0+2SD. In addition, small domains had fewer than 75 pixels were not included in the size analysis. Individual domain size was described as the diameter of an equivalent disk. Domain sizes were then averaged according to map types and cases. To measure the map strength of a functional map (e.g. Figure 2F), we calculated the differences of average pixel values in white and dark domains in the subtraction map, using the domain masks described above. For example, the map strength of a 45° vs. 135° orientation map was calculated as the subtraction between the average pixel values of 135° domains and the average pixel values of 45° domains in the 45°−135° subtraction map.

Two-photon data analysis

Request a detailed protocol

Two-photon data analysis was performed using MatLab 2017 (The MathWorks, Natick, MA). Image alignment and cell identification algorithms were modified from those used in Li et al., 2017. To correct slow cortex drifts and biological movements (likely due to heart pulsation and artificial ventilation) within the X-Y plane (normally less than 20 pixels during the whole imaging session, typically five pixels), we first obtained a frame template by averaging 1000 frames from a chosen session, then all frames of that experiment, and frames from other days of the same cortical layer, were aligned to this template using a cross-correlation method.

Fluorescent images (e.g. Figure 3C) were obtained by averaging all collected two-photon images during the imaging session, including the baseline and response images. Response images (e.g. Figure 3D and E) were obtained by subtracting the baseline image (average of the two frames before the stimulus onset) from the response image (average of the second and third frames after the stimulus onset).

Cell bodies were first identified based on their responses to each visual stimulus. We first averaged all repeats for a stimulus, then obtained a subtraction image by subtracting the baseline image (average of 2 immediate pre-stimulus frames) from the response image (average of frames 2 and 3 after the stimulus onset). This subtraction image was then filtered using two separate Gaussian filters: G1 (size = 7 pixels, std = 1.5) and G2 (size = 15 pixels, std = 7). Then the subtraction map of G1-G2 was used for cell extraction. The cells extracted in this way had smooth edges and filled cell-body. For each cell, the cell body identified from its best-responsive stimulus was used as its cell body and was used to calculate its responses in all the other stimuli.

A cell was identified if more than 30 connected pixels had values larger than mean+2.75SD, in which SD is the standard deviation of the pixel values of the whole filtered subtraction image. After cell locations were determined, we calculated fluorescence responses for cells to each stimulus using following formula: ΔF/F0=(F-F0)/F0, in which F is the average pixel value covered by the cell in the response image, and F0 is average pixel values in the baseline image. A cell was included for the following analysis if any of its stimulus responses (ΔF/F0) was larger than 0.3 for the stimuli tested.

Orientation tuning to curves

Request a detailed protocol

A cell was identified as a curve-orientation-preferred cell (e.g. Figure 6A–D) if its responses to eight curve orientations passed the Rayleigh test for circular uniformity (Fisher, 1993) at a significant level of p<0.05. Then we fitted the responses with a modified von Mises function (Mardia, 1972):

F(θ)=a0+b1×ec1×cos⁡(θ-θp)+b2×ec2×cos⁡(θ-θp-180)

where θ is the orientation of curve (0–360°); θp is the preferred orientation; a0 is the baseline offset; (b1, b2) and (c1, c2) determine the amplitude and shape of the tuning curve, respectively. Fitting parameters were obtained with a least-square nonlinear regression method (nlinfit in Matlab, Mathworks). Goodness of fit (R2) values were >0.7 for all neurons which have significant orientation tuning to curve (n = 417). Cells’ preferred orientations were then calculated from the fitting curve (Figure 6B).

Circle preference

Request a detailed protocol

A cell was determined as circle-preferred cell if it met following two criteria: 1. The cell’s response was significantly modulated by contour types (bar, 90°-angle, triangle, curve, each was represented by its optimal oriented stimulus, and circle, p<0.05, one-way ANOVA) and its maximum response was to the circle. 2. The cell’s response to the circle was significantly larger than its response to the curve of its preferred orientation (p<0.05, paired t-test).

Shape preference

Request a detailed protocol

In calculation of the preferential responses to two contours (e.g. Figure 4J and K), we first normalized the cell’s ΔF/F0 responses to the whole stimulus set to 0–1, then the differences of the responses to the two types of stimuli were calculated.

RF size and surround suppression

Request a detailed protocol

To measure cell RF sizes (e.g. Figure 4L and M), we first performed a two-way ANOVA analysis (with repetition) for the cell’s responses to square-wave gratings of 6 different sizes and eight different moving directions. If the size factor is significant (p<0.05), then we fitted a ratio of Gaussian function (Cavanaugh et al., 2002) to the direction-averaged size tuning curve:

Rx=kcLc(x)1+ksLs(x)
Lcx=(2π∫0xe-(y/wc)2dy)2
Lsx=(2π∫0xe-(y/ws)2dy)2

where x is the stimulus diameter, kc and ks are gains for the RF center and surround, Lc and Ls are the total squared responses of the center and surround, and wc and ws are the size of the center and surround (wc <ws). Fitting parameters were obtained with a least-square nonlinear regression method (nlinfit in Matlab, Mathworks). The optimal stimulus size was determined as the diameter corresponding to the peak response in the fitted function. Goodness of fit (R2) values were >0.7 for all neurons that had significant size tuning to square wave gratings (n = 1493), and 95% (1418/1493) of these neurons had a preferred stimulus size between 0.5° and 12°. If a cell’s responses were not modulated by stimulus size (i.e. size factor was not significant in the two-way ANOVA analysis), then this cell was not included in the RF size related analysis. Surround suppression index (e.g. Figure 4—figure supplement 1F) was calculated from the size tuning curve as (maximum response – minimum response)/maximum response, in which minimum response was the least response obtained for stimulus size larger than that of maximum response (maximum surround suppression).

Pixel-based contour-preference map

Request a detailed protocol

Pixel-based contour-preference maps (e.g. Figure 4B) were a combination of a gray value (G) map and a red/green hue map. For each pixel, we first calculated a shape preference index: SPI = abs(ΔFcc-ΔFbat)/max(ΔFcc,ΔFbat), where ΔFcc is the maximum fluorescence increase to curves and circles (red framed ones in Figure 3F), ΔFbat is the maximum fluorescence increase to bar, angle and triangle (green framed ones in Figure 3F). Then we calculated the gray value as: G = 200xSPIxFp/Fp_f, where Fp is the maximum of the fluorescence values in the response images to all five types of stimuli, Fp_f is the average of all pixels’ Fp in the same frame. Hue for a pixel is red if its ΔFcc is larger than ΔFbat, and is green if not.

PCA

Request a detailed protocol

PCA results were shown in Figure 5 and Figure 6—figure supplement 1. For each neuron, its response matrix was normalized to 0–1. Its responses to different orientations were sorted, so that for each contour, 0° represented its highest response and 135° for the lowest (thus no longer correspond to the stimulus icons). Only neurons tested with the full stimulus set were analyzed (n = 1556). The responses of these neurons were arranged as a 2D matrix, and the mean response to each stimulus was subtracted before the PCA analysis was performed. In Figure 5E, the color of the neurons was determined based on their coordinates on PC1.

Statistical analysis

Request a detailed protocol

All t-tests and paired t-test are two-tailed.

Data availability

Data and MATLAB code required to reproduce all figures are available at https://osf.io/qydj5/.

The following data sets were generated

References

  9. Book
    1. Fisher NI
    (1993)
    Statistical Analysis of Circular Data
    Cambridge University Press.
  27. Book
    1. Mardia KV
    (1972)
    Statistics of Directional Data
    Academic Press.

Decision letter

  1. Kristine Krug
    Reviewing Editor; University of Oxford, United Kingdom
  2. Joshua I Gold
    Senior Editor; University of Pennsylvania, United States
  3. Ed Connor
    Reviewer; Johns Hopkins University, United States
  4. Aniruddha Das
    Reviewer; Columbia University and Zuckerman Institute, United States

In the interests of transparency, eLife publishes the most substantive revision requests and the accompanying author responses.

Acceptance summary:

Curvature is a fundamental element of complex visual shapes in the world around us. This study elegantly combines cutting edge methodologies to investigate – from single brains cells to functional domains – how V4, a mid-level area of the visual cortex of primates, shows specialisation for processing and representing curvature, not seen at earlier levels of visual processing. This functional architecture complements known domains for orientation tuning and colour in V4 and significantly furthers our understanding of how primate brains recognise and discriminate visual objects.

Decision letter after peer review:

Thank you very much for submitting your article "Curvature-processing domains in primate V4" for consideration by eLife. Your article has been reviewed by three peer reviewers, and the evaluation has been overseen by a Reviewing Editor and Joshua Gold as the Senior Editor. The following individuals involved in review of your submission have agreed to reveal their identity: Ed Connor (Reviewer #1); Aniruddha Das (Reviewer #2).

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

As the editors have judged that your manuscript is of interest, but as described below that additional data analyses are required before it is published, we would like to draw your attention to changes in our revision policy that we have made in response to COVID-19 (https://elifesciences.org/articles/57162). First, because many researchers have temporarily lost access to the labs, we will give authors as much time as they need to submit revised manuscripts. We are also offering, if you choose, to post the manuscript to bioRxiv (if it is not already there) along with this decision letter and a formal designation that the manuscript is "in revision at eLife". Please let us know if you would like to pursue this option. (If your work is more suitable for medRxiv, you will need to post the preprint yourself, as the mechanisms for us to do so are still in development.)

Summary:

Lu and colleagues present potentially compelling new results on the architecture of curvature response strength and selectivity in macaque visual cortical area V4, in comparison to the lack thereof in earlier level visual areas V1 and V2. A particular strength is that the authors combine intrinsic-signal optical imaging (giving meso-scale resolution over a few mm of cortex) with 2-photon imaging (single cell resolution, over a few 100 microns; including 2-photon imaging at multiple cortical depths). The intrinsic signal data show distinct clustering of responsiveness to object fragments that constitute "intermediate" visual representations that are more complicated that the traditional single orientation columns in V1 or V2. However, the reviewers raised a number of critical issues relating to analysis, presentation and interpretation of this exciting data set and hope that these can be satisfactorily addressed in order for the study to deliver its full potential impact.

Essential revisions:

1) To include single activation condition maps for the intrinsic optical imaging and explore whether map activations could be explained by simpler, underlying response patterns.

The presented intrinsic optical imaging data comprise only difference maps, which could potentially obscure underlying results. For example, in Figure 1L, it could be that those 2 angles elicit very little localized responses, or that they elicit an identical pattern of activation, and from the difference map we don't know which is the case. It would be good to see some single condition activation maps.

That would also potentially help in investigating to what extent these responses can be explained by simple receptive field (RF) summation. For instance, there is a substantial literature in associating preferential responses to curves with end-stopped neurons: Hubel and Wiesel, in 1965 suggested that end-stopped, length tuned neurons could be useful for curvature detection and there was a nice explicit model of how that could happen by Dobbins et al. in 1987. But the manuscript makes little reference to any size tuning or surround suppression, and how that, in combination with classic orientation tuning, could create a cell that responds vigorously to curvature. This is even a greater concern given the well-established prevalence of size tuning, both electrophysiologically (1987) and with optical imaging (1997), in area V4. This has a huge impact on both novelty and interpretability; if you have a region of surround suppression (which we know exist) and it overlaps with an orientation region, this can look like a "curvature" region in that it will respond to a curve better than extended straight line or grating. But we would argue it's not a "curvature" region, since a short bar or grating without any curvature is actually the optimal stimulus. For example, can the response to triangles, on a pixel by pixel basis, be explained by simply adding or averaging the response the constituent orientations? Similarly, can the circle responses be explained by adding the responses to the half-circles?

2) To support the 2-photon results with convincing anatomical images.

One technical issue is the lack of any good underlying anatomical images to support the 2-photon data. This is concerning because all the images provided here show cells that are filled in, but living cells should appear as rings representing the cell membrane. Filled in cells are dead or compromised, and this would be a critical caveat for all the response results reported here. All of the cells in Figure 3C-F appear to be filled in. This could be because (i) these images do not have sufficient resolution to show rings, (ii) the authors did some kind of smoothing of these images (smoothing is mentioned in Materials and methods), or (iii) the cells are filled with calcium, reflecting a dead or compromised state. This concern is further exacerbated by the strange cell shapes in Figures 4G and H and 5A, D, and G. The authors should show the underlying anatomical images for those panels and explain how they defined the odd shapes shown in them. In addition to showing anatomical images, the authors should quantify how many cells are filled or ring-like in their images, and clarify whether the cells for which they report tuning include filled cells (and how many).

3) The reviewers would like to see more systematic and consistent analysis and representation across Figures 4. and 5 that allows the reader to better understand the fundamental, underlying tuning properties of neurons.

A technical concern is that Figure 4 and Figure 5 appear to present the same phenomena, but with different analyses for different images. Figure 4G labels neurons according to whether they exhibited stronger responses to circles or arcs. Figure 5 distinguishes neurons by how closely they are correlated with the average response pattern for the local curvature domain. But the examples in Figures 5B,C,E,F,H,I make it clear that the differences between correlated and not correlated again have to do with relative responses to circles vs. arcs. All of these images in Figures 4 and 5 need to be analyzed in the same ways, with an emphasis on the more explanatory analyses related to tuning, rather than unexplained correlation.

In relations to the tuning properties underlying the impressive 2-photon data, the question arises to what extent can RF summation (for example, RF summation models that include non-linearities that have been applied to V4) explain responses across the stimulus array?

See also point 4 below, in relation to this issue of the underlying tuning properties of neurons.

4) To reanalyze both datasets (intrinsic optical imaging and 2-Photon) in light of what is well-known about V4 tuning for fragments, not whole shapes, for curvature acuteness (which means responses to triangles reflect high curvature acuteness tuning) and object-relative position of contour fragments (which explains the differences between "circle" neurons and "curve" neurons, as well as the difference between triangle responses and angle responses, most of which we do not get to see). This should radically change the interpretation of Figures 1 and 2, and change the analysis emphasis of regions in the 2-photon section of the paper.

The major technical and interpretational concern is that differences between response patterns throughout the paper are presented as though the neurons were tuned for circles in some cases, arcs in others, triangles in others, etc. This ignores well-established tuning dimensions in V4 that explain response patterns like these at a more basic level. Pasupathy and Connor, 1999; 2001and 2002) and Carlson et al. (2011, Current Biology) clearly demonstrated that:

i) V4 neurons respond to contour fragments, not complete shapes,

ii) V4 neurons are strongly tuned for the object-relative position of those contour fragments,

iii) V4 neurons are tuned for curvature acuteness, with a bias toward sharp curvatures,

iv) V4 neurons are differentially tuned for convexity and concavity.

By ignoring this literature, the authors might misinterpret many of their results:

a) In Figures 1 and 2, responses to triangles are treated as representative of non-curvature regions. But, per point (iii) above, the points of those triangles will drive strong responses from the most acute curvature tuning regions. Thus, the curvature/triangle contrasts are more likely to represent organization for broad curvature vs. acute curvature (angles are geometrically just the limiting case of acute curvature) than a difference between curvature and straight lines. The same is true for line segments, whose terminations drive weaker but similar responses, and which show only weak differences from triangles and angles. The only valid contrast for finding curvature domains is curves vs. gratings, which reveals very different regions than triangles in Figure 1. This reliance on circle vs. triangle contrast has ramifications throughout the paper, because it is used to define curvature domains and thus bias the 2-photon studies away from regions of acute curvature tuning.

b) A similar problem appears in Figure 3, where the only response shown for the "rectilinear" region is a highly selective response for a triangle. Per point (ii) above, triangles provide a whole object context in which V4 neurons can exhibit their strong tuning for object-relative position of contour fragments, including high-curvature angles. This is why neuron 3 does not respond to any other rectilinear stimulus. If triangle responses like this are the basis for the green regions in Figure 3, then those regions are acute curvature regions, not non-curvature regions, and again the authors will be excluding high-curvature regions from their analyses in Figures 4 and 5.

c) Per above, Figures 4 and 5 are restricted to clusters of broader curvature tuning, strongly biasing the analyses. This is undoubtedly one reason that many example neurons respond strongly to circles. The other major problem with these figures is that they are presented as evidence that many neurons in V4 are strongly selective for circles as whole stimuli, while others are strongly selective for arcs over circles. In fact, the references cited above make clear that neurons in V4 are invariably tuned for shape fragments (i), not whole shapes, and they are strongly tuned for fragment position within shapes (ii). This is why many neurons respond most strongly to circles, because they are tuned for an arc-shaped fragment, but respond much more strongly when that fragment is in the preferred object-relative position. This is frequently true for circles, because there is also a strong correlation between curvature orientation tuning and object position tuning, since convex curves pointing up usually occur at the top of an object; curves pointing to the right occur at the right, etc. In many other cases, however, the preferred object-relative position is off angle, and when that is true the responses to circles will be low, and the responses to isolated arcs will be higher. (Responses to an entire shape with the arc at the preferred position would be higher still.) Thus, differential tuning for object-relative position explains most of the differences highlighted in Figures 4 and 5.

5) To support the 2-photon data, which surveys properties in a large number of cells, a comprehensive, more clearly structured data summary is required.

Apart from the issue raised above, the summary statements of the 2-Photon data are not particularly satisfying: the categories of orientation and curve-orientation are not very enlightening about underlying principles. Perhaps PCA analysis could be used to see the patterns of selectivity that are prevalent across the sample (perhaps after selecting the best orientation or collapsing across orientation). Another possibility would be, after rotating to a common best orientation, to superimpose all the stimuli with an opacity relating the firing rate. In the end this seems like a lost opportunity to follow up the influential Hegde and Van Essen, 2000, studies whose stimulus set parallels the one used here.

6) Incorporating any new results from the analyses above, the underlying major concepts need to be more consistently defined and applied across the paper.

Related to the technical issues raised above (points 3-5), the terminology in the manuscript is sometimes confusing. The authors propose that there is “microarchitecture” of selectivity for circles vs. curves. But then they appear to contradict themselves. For example (final paragraph of subsection “Microarchitectures of the curvature domains”) the authors talks of neurons having “preferred curve” stimuli; and in the same section, (Figure 4K) the authors state that these same neurons respond more strongly to circles than to their preferred curves. What does it mean to be selective for a “preferred curve” if the response to the “preferred” stimulus is consistently weaker than to a different stimulus?

7) Please clarify the definition of curvature domains and examine their periodicity

Another technical issue is the poor description of how curvature domains were defined and drawn. The "watershed" technique should be described in detail, and the domains should be drawn on the sections, especially the sections in Figure 5, where the domain boundaries determine the average response pattern.

Given the limited spatial sampling with 2-photon imaging, distance correlations metrics with the intrinsic optical imaging reflectance data (using the above requested single condition activity maps) would be nice to examine the periodicity of horizontal organization patterns. Given the stimulus array is that used in the 2-photon data, the absolute value of correlations is going to be different, but the key issue here is the spacing and size of "columns" and whether that depends on the type of stimulus you're looking at.

What bandpass filtering is used for intrinsic optical images (since that can "create" domains out of noise)? Can Fourier analysis be used to describe the size and spacing of domains?

8) There are also a number of places where the primary narrative of the paper seems to contradict itself. For example: starting with the Abstract, the authors make the important observation that “curvature” domains have “little overlap” with “orientation domains”. They repeat this point, e.g. in the Results; and again, in the legend of Figure 3H,: with clearly distinct “curvature” and “rectilinear” zones. On the other hand, the authors also make a point of saying that individual neurons have mixed responses – with individual neurons in “curvature” domains responding well to rectilinear contours (e.g. Figure 3P). The authors further amplify the issue of response diversity in Figure 5, with nearest-neighbor neurons showing strikingly different response. The authors should rewrite the Abstract and the relevant sections of the results to have a more consistent description of the degree of overlap and response diversity that they observe.

9) While these results are very interesting, they are of course restricted to a limited set (73?) of human-selected and human-curated stimuli. Nowhere does the analysis address the question as to whether these are close to the optimal stimuli for this region of cortex, even in the (unnatural) anesthetized state. The authors should have a more expanded discussion of these limitations, and how to extend the analysis further to get a sense of the “optimal” stimuli for this stage of cortical processing.

Furthermore, the stimulus set used parallels that used by Hedge and Van Essen, 2000, in V4, and subsequently in V1 and V2 as well. Given that we know that selectivity’s exist that cannot simply be explained by orientation filters, the paper could do a better job of emphasizing what is new and, in particular, the significance of finding functional organization. If, for example, the authors feel that the presence of functional organization for a particular parameter is especially important in understanding the computations that occur within in an area, or in the broader picture, the role that area plays in visual processing, they should state so, and why they believe this.

These issues should be discussed and laid out appropriately.

[Editors' note: further revisions were suggested prior to acceptance, as described below.]

Thank you very much for resubmitting your article "Curvature-processing domains in primate V4" for consideration by eLife. Your revised article has been reviewed by two peer reviewers, and the evaluation has been overseen by a Reviewing Editor and Joshua Gold as the Senior Editor. The following individual involved in review of your submission has agreed to reveal their identity: Aniruddha Das (Reviewer #2).

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

We would like to draw your attention to changes in our revision policy that we have made in response to COVID-19 (https://elifesciences.org/articles/57162). Specifically, we are asking editors to accept without delay manuscripts, like yours, that they judge can stand as eLife papers without additional data, even if they feel that they would make the manuscript stronger. Thus the revisions requested below only address clarity and presentation.

Summary:

Lu and colleagues present compelling new results on the architecture of curvature response strength and selectivity in macaque visual cortical area V4, in comparison to the lack thereof in earlier level visual areas V1 and V2. A particular strength is that the authors combine intrinsic-signal optical imaging (giving meso-scale resolution over a few mm of cortex) with 2-photon imaging (single cell resolution, over a few 100 microns; including 2-photon imaging at multiple cortical depths). The intrinsic signal data show distinct clustering of responsiveness to object fragments that constitute "intermediate" visual representations that are more complicated that the traditional single orientation columns in V1 or V2.

Revisions:

The authors have done an exemplary job in the revision, by addressing concerns raised and adding further analyses (clustering of single-cell-level response; PCA; more extended comparisons across classes of similar stimuli such as triangles / short oriented lines, as vs. circles / arcs; scaled to different sizes; filled / unfilled). Overall, these new analyses along with their original combination of intrinsic optical imaging and 2-photon imaging, showing the close spatial match between clusters identified separately by the two techniques make a compelling case for the specialised architecture the authors propose in the Discussion. There remain a small number of issues that would need to be addressed.

1) One disappointment is that some of the authors' responses didn't actually make into the manuscript or are "buried" in multi-panel supplementary figures. Specifically, the significance of relationship between surround suppression and curvature, and the PCA analysis (which proved interesting) are not particularly clear in the actual text of the manuscript. Also in light of some of the new analyses, the authors should probably cite some relevant literature about the existence of "sub-domain" specificity with domain-specific regions.

2) The direct correspondence between intrinsic and 2-photon imaging is certainly positive (correlation coefficients > 0), but remarkably poor (and less than one), especially given that, for many experiments, intrinsic maps are seen as veridical and generally have good correspondence with targeted electrophysiology recordings. We noticed in the response letter that all intrinsic maps were Gaussian blurred, if one does the same procedure to the 2-photon data, does the correlation improve? And, if not, what explains this result: perhaps, significant noise in one type of measurement, or more interestingly, a true lack of correspondence between haemodynamics and cellular level responses. If the later, that's a big deal given: it potentially suggests that all haemodynamic maps should be taken with "a grain of salt" with regards to reflecting neuronal response property organization. And of course, if this is true, and intrinsic imaging is that noisy, it potentially calls into question many of the intrinsic signal difference maps and results even within this manuscript. This needs to be explicitly addressed.

3) More discussion in the actual manuscript about surround suppression, and specifically the points that appear in the authors' response, would be good. This is especially important, because surround suppression has long been associated with V4 responses, and there have been claims that it is functionally organized. The authors' remarks in the response about the only things the difference maps having is common is curvature is probably the best that can be made with the current data set. But, on the other hand, given the strong differences in surround suppression and receptive field size shown in the supplementary figures, that previous reports of surround suppression and its organization in V4 might simply reflect curvature regions that were studied with inappropriate sub-optimal stimuli.

4) This is a really large data set of cellular responses, and the PCA analyses are really important, but again, it would be better for this to appear in the actual manuscript.

One reason in particular is the paper's discussion of "microorganization" or sub-domains within a curvature region. We found these discussions a little anecdotal, and we were wishing that a spatial autocorrelation analysis could be done with pairs of neurons as function of distance to reveal how big a "sub-domain" was, and in particular, how it compared with orientation columns, which of course are sub-domains within the rectiiinear domains. For example, what one could do with regard to PCA weights: PCA1 spatial autocorrelation. In a similar vein, there have been several reports of color-specific sub-domains with color-preference domains, and, especially given the classic association of V4 with color sensitivity, that literature seems relevant here in a discussion of how universal domain/sub-domain organization may be and its variation between different cortical areas.

5) Scaling of the intrinsic signal optical images. The authors' methods suggest that the SD used for the gray scale is calculated separately for each (SVM) map. Is that correct? If so, there is no way to compare the strengths of activation by different stimuli. It likely also explains why the single-condition maps (e.g. Figure 1—figure supplement 1, panels A,B,D,E,G,H) appear not only relatively featureless but also occupying a grayscale range similar to the backgrounds of the difference images. In absolute terms, the single-condition responses presumably are ~ an order of magnitude higher amplitude if they are dominated by stimulus-nonspecific haemodynamic responses? Showing the single-condition and difference images on the same absolute scale would be uninformative. But it would be useful to show all the difference images on a common scale that is not tied to the SD per image, to get a sense of the efficacies of different stimulus pairwise differences. This could be done as an additional set of panels in supplementary figures.

https://doi.org/10.7554/eLife.57502.sa1

Author response

Essential revisions:

1) To include single activation condition maps for the intrinsic optical imaging and explore whether map activations could be explained by simpler, underlying response patterns.

The presented intrinsic optical imaging data comprise only difference maps, which could potentially obscure underlying results. For example, in Figure 1L, it could be that those 2 angles elicit very little localized responses, or that they elicit an identical pattern of activation, and from the difference map we don't know which is the case. It would be good to see some single condition activation maps.

We agree with the reviewer that difference maps show the differences between two conditions, not the true response patterns. In the revised manuscript, we have included single-condition maps (new Figure 1—figure supplement 1) for comparison.

As we can see, although some activation patterns can be observed from these single-condition maps, the overall contrast of these patterns was low, and the differences between these maps were small. There were almost no differences for stimuli presented at different orientations (new Figure 1—figure supplement 1 D and E, or G and H). However, calculated from these similar single-condition response patterns, the difference maps are not similar: strong patterns can be observed in some maps (C, L), and weak (F) or no (I) patterns in others. Thus, single-condition maps here are not reliable for revealing the differences of activation patterns for different stimuli. This might be due to the diffused feature-non-specific hemodynamic responses in V4, blood vessel noise, and the reliance on blank activation. In addition, single-condition maps were more vulnerable to experimental factors like the anesthesia level, the general activation level of the preparation etc., which may vary from case to case. Thus, we choose to rely on difference maps in this study.

That would also potentially help in investigating to what extent these responses can be explained by simple receptive field (RF) summation. For instance, there is a substantial literature in associating preferential responses to curves with end-stopped neurons: Hubel and Wiesel, in 1965 suggested that end-stopped, length tuned neurons could be useful for curvature detection and there was a nice explicit model of how that could happen by Dobbins et al. in 1987. But the manuscript makes little reference to any size tuning or surround suppression, and how that, in combination with classic orientation tuning, could create a cell that responds vigorously to curvature. This is even a greater concern given the well-established prevalence of size tuning, both electrophysiologically (1987) and with optical imaging (1997), in area V4. This has a huge impact on both novelty and interpretability; if you have a region of surround suppression (which we know exist) and it overlaps with an orientation region, this can look like a "curvature" region in that it will respond to a curve better than extended straight line or grating. But we would argue it's not a "curvature" region, since a short bar or grating without any curvature is actually the optimal stimulus. For example, can the response to triangles, on a pixel by pixel basis, be explained by simply adding or averaging the response the constituent orientations? Similarly, can the circle responses be explained by adding the responses to the half-circles?

The reviewer’s question is two-fold: First, at single-cell level, whether end-stopping properties can account for the neuron’s curvature responses we observed. We have shown that curvature neurons had smaller RFs than those of non-curvature neurons (Figure 3—figure supplement 1D and E). In this revision, we further analyzed and showed that curvature neurons had stronger surround suppression (Figure 3—figure supplement 1H and I). Thus, consistent with previous findings, curvature neurons did exhibit end-stopping properties. However, curvature neurons showed additional curve-specific preferences, including curve-orientation tuning, preference for curves over similarly-sized angle stimuli, etc. For circle-preferring neurons, they prefer circles (larger) than the half-circles (smaller). These additional features suggest that curvature neurons were not simple end-stopping neurons, although they exhibited end-stopping properties.

The same question was asked at the functional domain level, i.e., whether the curvature domains could be the overlap regions of orientation domains and surround-suppression regions in V4. In this study, curvature domains were revealed in difference maps, including: circles vs. triangles (Figure 1D), circles vs. bars (Figure 1G), curves vs. bars (Figure 1H), and curves vs. angles (Figure 1—figure supplement 2L). The two stimuli in each pair had similar sizes, and thus had similar surround suppression effects. Their contrast pattern in the difference maps could not be due to differences in suppression. The stimuli on the two sides also had balanced orientations, plus their maps had little overlap with the orientation maps (Figure 2D). The only consistent contrasts in these pairs of stimuli was the curvature feature. Thus, the curvature domains we found are unlikely the overlap regions of suppression and orientation domains.

2) To support the 2-photon results with convincing anatomical images.

One technical issue is the lack of any good underlying anatomical images to support the 2-photon data. This is concerning because all the images provided here show cells that are filled in, but living cells should appear as rings representing the cell membrane. Filled in cells are dead or compromised, and this would be a critical caveat for all the response results reported here. All of the cells in Figure 3C-F appear to be filled in. This could be because (i) these images do not have sufficient resolution to show rings, (ii) the authors did some kind of smoothing of these images (smoothing is mentioned in Materials and methods), or (iii) the cells are filled with calcium, reflecting a dead or compromised state. This concern is further exacerbated by the strange cell shapes in Figures 4G and H and 5A, D, and G. The authors should show the underlying anatomical images for those panels and explain how they defined the odd shapes shown in them. In addition to showing anatomical images, the authors should quantify how many cells are filled or ring-like in their images, and clarify whether the cells for which they report tuning include filled cells (and how many).

Neurons in Figure 3C looked like filled was mainly due to the small size of the figure and the narrow clip range. In Author response image 1 we enlarged Figure 3C in panel A and also provided a larger clip-range version of the same image (panel B). It can be seen that most cell bodies appear ring-like.

Author response image 1
Download asset Open asset

GCaMP is expressed in the plasma and usually does not go into the nucleus (Tian et al., 2009; Chen et al., 2013). Thus, in ideal condition (the imaging focal plane cuts through the cell nucleus), the cells appear as rings. For neurons located slightly off the imaging focal plane, and their nucleus are not cut by the focal plane, the cells will also appear as smaller filled patches. In addition, Figure 3C was an average of all the frames over an experimental session, the preparation usually had some drift along the Z axis, the ring-like appearance will also be blurred due to this reason.

Fluorescent response images (e.g. Figure 3 D-E) were based on incremental fluorescence changes at the stimulus onset. Neurons in these images usually had responses at the center and appeared less ring-like. This fact, plus the smoothing procedure in cell identification (described in Materials and methods), made the final cell images filled shapes (e.g. Figure 5 G and H).

There were indeed some neurons having a high baseline fluorescence but weak visual responses. Since our cell identification was based on neurons’ onset responses, these neurons would not pass the significance test for further analysis.

Based on fluorescent responses, many identified neurons did not have a circular shape. This was mainly due to the shape of their fluorescence responses and/or the responses from attached fibers imaged in the same focal plane.

3) The reviewers would like to see more systematic and consistent analysis and representation across Figures 4. and 5 that allows the reader to better understand the fundamental, underlying tuning properties of neurons.

A technical concern is that Figure 4 and Figure 5 appear to present the same phenomena, but with different analyses for different images. Figure 4G labels neurons according to whether they exhibited stronger responses to circles or arcs. Figure 5 distinguishes neurons by how closely they are correlated with the average response pattern for the local curvature domain. But the examples in Figures 5B,C,E,F,H,I make it clear that the differences between correlated and not correlated again have to do with relative responses to circles vs. arcs. All of these images in Figures 4 and 5 need to be analyzed in the same ways, with an emphasis on the more explanatory analyses related to tuning, rather than unexplained correlation.

We agree that the analysis in the original Figure 4 and 5 had some overlap on the same phenomena (i.e. different neurons responded differently to circles and arcs). In the revision, we replaced the original Figure 5 with a new figure (Figure 6) showing response matrices of neighboring neurons in 3 small regions (a circle subdomain, a curve-orientation subdomain, and a rectilinear region). These raw response patterns contain richest information, from which both commonality and diversity of the responses for the neighboring neurons can be observed.

In relations to the tuning properties underlying the impressive 2-photon data, the question arises to what extent can RF summation (for example, RF summation models that include non-linearities that have been applied to V4) explain responses across the stimulus array?

Due to the scope of this study, we have not evaluated the RF summation model in explaining the responses we observed. We agree, however, that this is an important question and needs to be explored. Based on our limited stimulus set, we observed complex response patterns for different neurons. Even for the neighboring neurons, they exhibited significant differences (Figure 6). Such a complexity seems difficult to be fully explained by a RF summation model. A recently study used a neural network approach and synthesized stimulus patterns that can maximally drove V4 neurons (Bashivan et al., 2019). These “optimal” stimulus patterns revealed complex RF structures of V4 neurons and suggest that additional complexity needs to be included in V4 RF modeling.

See also point 4 below, in relation to this issue of the underlying tuning properties of neurons.

4) To reanalyze both datasets (intrinsic optical imaging and 2-Photon) in light of what is well-known about V4 tuning for fragments, not whole shapes, for curvature acuteness (which means responses to triangles reflect high curvature acuteness tuning) and object-relative position of contour fragments (which explains the differences between "circle" neurons and "curve" neurons, as well as the difference between triangle responses and angle responses, most of which we do not get to see). This should radically change the interpretation of Figures 1 and 2, and change the analysis emphasis of regions in the 2-photon section of the paper.

The major technical and interpretational concern is that differences between response patterns throughout the paper are presented as though the neurons were tuned for circles in some cases, arcs in others, triangles in others, etc. This ignores well-established tuning dimensions in V4 that explain response patterns like these at a more basic level. Pasupathy and Connor, 1999; 2001 and 2002 and Carlson et al. (2011, Current Biology) clearly demonstrated that:

i) V4 neurons respond to contour fragments, not complete shapes,

ii) V4 neurons are strongly tuned for the object-relative position of those contour fragments,

iii) V4 neurons are tuned for curvature acuteness, with a bias toward sharp curvatures,

iv) V4 neurons are differentially tuned for convexity and concavity.

By ignoring this literature, the authors might misinterpret many of their results:

a) In Figures 1 and 2, responses to triangles are treated as representative of non-curvature regions. But, per point (iii) above, the points of those triangles will drive strong responses from the most acute curvature tuning regions. Thus, the curvature/triangle contrasts are more likely to represent organization for broad curvature vs. acute curvature (angles are geometrically just the limiting case of acute curvature) than a difference between curvature and straight lines. The same is true for line segments, whose terminations drive weaker but similar responses, and which show only weak differences from triangles and angles. The only valid contrast for finding curvature domains is curves vs. gratings, which reveals very different regions than triangles in Figure 1. This reliance on circle vs. triangle contrast has ramifications throughout the paper, because it is used to define curvature domains and thus bias the 2-photon studies away from regions of acute curvature tuning.

It is potentially possible that triangles we used in Figure 1 and 2 might represent acute curvatures and not for “rectilinear contour” representations. However, we found this was not the case after we examined many comparison maps. The responses to triangles were similar to those to short lines. For example, the triangle vs. straight line map was weak (Figure 1J), and the circle vs. triangle map and circle vs. straight line map were similar (Figure 1D, G). These indicate that responses elicited by triangles were mostly due to their line component, not the sharp angle component.

Following the reviewer’s suggestion, we compared “circle vs. gratings”, and “circle vs. triangle” maps (A and B in Author response image 2). In these two maps, the curvature domains were found at the similar locations. Thus, the V4 responses to the triangles were primarily driven by their straight-line components, not by their sharp angles. The weak contrast in “triangle vs. gratings” map (Author response image 2C) also indicates the similarity of these two types of responses.

Author response image 2
Download asset Open asset

b) A similar problem appears in Figure 3, where the only response shown for the "rectilinear" region is a highly selective response for a triangle. Per point (ii) above, triangles provide a whole object context in which V4 neurons can exhibit their strong tuning for object-relative position of contour fragments, including high-curvature angles. This is why neuron 3 does not respond to any other rectilinear stimulus. If triangle responses like this are the basis for the green regions in Figure 3, then those regions are acute curvature regions, not non-curvature regions, and again the authors will be excluding high-curvature regions from their analyses in Figures 4 and 5.

We thank the reviewer for pointing this out. We re-examined cell 3 in the original Figure 3P and found that, in addition to its response to triangle, this cell did respond to line segments oriented in 45 degrees. The original figure did not include this stimulus due to the space limitation. Now we have expanded the figure and included this and other relevant stimuli (new Figure 3F). Thus, the triangle responses of this neuron were likely driven by the 45-degree line segment in the triangle, instead of its high-curvature angles. More examples neurons from the rectilinear regions were provided in a new figure (Figure 6C) in which similar responses patterns were observed.

Population-wise, neurons that responded better to bars than to curves were more likely located in the “rectilinear” regions in ISOI imaging (old Figure 3R, new Figure 4K).

Thus, consistent with the findings in ISOI results (see answers to reviewer’s question “a” above), the 2-photon results also indicate that, although triangle stimuli were used to map the rectilinear regions, neurons in these regions were mainly driven by the line segments of the rectilinear stimuli, not their sharp angles.

c) Per above, Figures 4 and 5 are restricted to clusters of broader curvature tuning, strongly biasing the analyses. This is undoubtedly one reason that many example neurons respond strongly to circles. The other major problem with these figures is that they are presented as evidence that many neurons in V4 are strongly selective for circles as whole stimuli, while others are strongly selective for arcs over circles. In fact, the references cited above make clear that neurons in V4 are invariably tuned for shape fragments (i), not whole shapes, and they are strongly tuned for fragment position within shapes (ii). This is why many neurons respond most strongly to circles, because they are tuned for an arc-shaped fragment, but respond much more strongly when that fragment is in the preferred object-relative position. This is frequently true for circles, because there is also a strong correlation between curvature orientation tuning and object position tuning, since convex curves pointing up usually occur at the top of an object; curves pointing to the right occur at the right, etc. In many other cases, however, the preferred object-relative position is off angle, and when that is true the responses to circles will be low, and the responses to isolated arcs will be higher. (Responses to an entire shape with the arc at the preferred position would be higher still.) Thus, differential tuning for object-relative position explains most of the differences highlighted in Figures 4 and 5.

The reviewer’s main concern here is that we separated curvature neurons into two sub-groups (circle-preferring and curve-orientation-preferring neurons), and why the circle responses were not explained in terms of position-related curvature responses.

We separated circle-preferring neurons from curve-orientation neurons based on two observations: First, these two sets of neurons formed separated spatial clusters on V4 surface (Figure 5 G and H) and differed in their curve orientation tunings. Second, as we can observe from the response matrices in the new Figure 6, that neurons in the circle subdomains (Figure 6A) responded best to circles, but very weakly to any of the half circles (the third, fourth columns from the right). Note that we used 8 orientations here instead of 4 in other shapes, thus likely covered all possible orientations. These neurons also appeared to have a gradual increase of responses to the completeness of the circle (from the right-most column to the middle). In addition, they all had modest responses to closed contour shapes (the four columns left to the circle). These closed shapes do not share any similar curve fragment, but have the closeness as a common feature. In particular, these neurons responded better to the offset-paired half circles (the second column on the right of the circle) than to the single half circles. Population results shown in Figure 5 K and L also confirm the above observation. These features were not easily explained by the selectivity to a curve fragment with a certain object-relative position.

Considering that these small patches of neurons were located in domains identified with ISOI and only occupied a small fraction of V4 surface, it is possible that these neurons were not well represented in the previous single-unit recordings.

5) To support the 2-photon data, which surveys properties in a large number of cells, a comprehensive, more clearly structured data summary is required.

Apart from the issue raised above, the summary statements of the 2-Photon data are not particularly satisfying: the categories of orientation and curve-orientation are not very enlightening about underlying principles. Perhaps PCA analysis could be used to see the patterns of selectivity that are prevalent across the sample (perhaps after selecting the best orientation or collapsing across orientation). Another possibility would be, after rotating to a common best orientation, to superimpose all the stimuli with an opacity relating the firing rate. In the end this seems like a lost opportunity to follow up the influential Hegde and Van Essen, 2000, studies whose stimulus set parallels the one used here.

We thank the reviewer for this suggestion. We did PCA analysis on all the neurons’ responses (n=1556). The results were shown as new Figure 4—figure supplement 1. This data-driven analysis also reveals different response patterns for neurons inside and outside the curvature domains, which further confirmed our ISOI findings. We also performed PCA analysis on neurons inside curvature domains and found that circle neurons and curve-orientation neurons could also be separated by their PC coordinates (Figure 5—figure supplement 1).

6) Incorporating any new results from the analyses above, the underlying major concepts need to be more consistently defined and applied across the paper.

Related to the technical issues raised above (points 3-5), the terminology in the manuscript is sometimes confusing. The authors propose that there is “microarchitecture” of selectivity for circles vs. curves. But then they appear to contradict themselves. For example (final paragraph of subsection “Microarchitectures of the curvature domains”) the authors talks of neurons having “preferred curve” stimuli; and in the same section, (Figure 4K) the authors state that these same neurons respond more strongly to circles than to their preferred curves. What does it mean to be selective for a “preferred curve” if the response to the “preferred” stimulus is consistently weaker than to a different stimulus?

We agree that the original categorization was not clear. In the revision, we have separated the “dual-preference neurons” from the “curve-orientation-preference neurons” so that these neurons do not have overlaps. We revised the figure (new Figure 5 G, I, J) and the relevant text accordingly.

7) Please clarify the definition of curvature domains and examine their periodicity

Another technical issue is the poor description of how curvature domains were defined and drawn. The "watershed" technique should be described in detail, and the domains should be drawn on the sections, especially the sections in Figure 5, where the domain boundaries determine the average response pattern.

In most part of this manuscript, curvature domains were obtained with a threshold (gray level lower than 0-2SD) on the circle vs. triangle map (e.g. Figure 1D). Domain coverage in Figure 2D and the neuron classification in Figure 4 and 5 (original figures) were all based on this method.

Only in estimating domain sizes (Figure 2E), we used watershed method in order to deal with the “connected domain” problem, in which two or more domains were connected and multiple peaks existed. With watershed method, each time we found the strongest domain, fill this domain with the threshold values and moved on to find the next strongest domain. This process was repeated until all domains were identified. We have added the details of watershed method in the Materials and methods section.

Given the limited spatial sampling with 2-photon imaging, distance correlations metrics with the intrinsic optical imaging reflectance data (using the above requested single condition activity maps) would be nice to examine the periodicity of horizontal organization patterns. Given the stimulus array is that used in the 2-photon data, the absolute value of correlations is going to be different, but the key issue here is the spacing and size of "columns" and whether that depends on the type of stimulus you're looking at.

What bandpass filtering is used for intrinsic optical images (since that can "create" domains out of noise)? Can Fourier analysis be used to describe the size and spacing of domains?

We thank the reviewer’s suggestion. We have analyzed domain spacing and size with a domain-averaging method. In circle-triangle maps (e.g. Figure 1D), we located the peaks of each curvature domains and averaged all these domains with their peaks aligned. We did the same analysis on orientation domains based on the orientation maps (e.g. Figure 1E). The map in Author response image 3 shows the average maps for curvature domains (n=142) and orientation domains (n=218). We also plotted the normalized gray-level profiles of these two maps in panel C for comparison. The sizes of these two domains were very similar, which is consistent with the domain size results we obtained with a fitting method (Figure 2E). The length of the blue arrow is 0.518 mm, which is the measured curvature-domain size using a fitting method in the paper. It is approximately equal to the half-height width of the profile curves. Some details also can be observed from these profiles. For example, for orientation domain, there was a suppressive white ring around the black domain (diameter 1.17 mm), indicating there were opposite domains right next to these orientation domains (domains preferring orthogonal orientations). However, such feature was absent in the averaged curvature domain, suggesting that its opposite domains (rectilinear domains) were either not prominent or not adjacent.

Author response image 3
Download asset Open asset

Above results were obtained from difference maps. As we showed in Author response image 1, the signal-noise ratio in single-condition maps were too low to be analyzed with this method. We also tried Fourier analysis but the results were not as clear as those described above.

For the bandpass filter question the reviewer asked, we only used a small low-pass Gaussian filter (diameter: 5 pixels, SD: 1 pixel) to smooth the original ISOI maps. No high-pass filter was used. This procedure is unlikely to create domains out of noises.

8) There are also a number of places where the primary narrative of the paper seems to contradict itself. For example: starting with the Abstract, the authors make the important observation that “curvature” domains have “little overlap” with “orientation domains”. They repeat this point, e.g. in the Results; and again, in the legend of Figure 3H,: with clearly distinct “curvature” and “rectilinear” zones. On the other hand, the authors also make a point of saying that individual neurons have mixed responses – with individual neurons in “curvature” domains responding well to rectilinear contours (e.g. Figure 3P). The authors further amplify the issue of response diversity in Figure 5, with nearest-neighbor neurons showing strikingly different response. The authors should rewrite the Abstract and the relevant sections of the results to have a more consistent description of the degree of overlap and response diversity that they observe.

We took the reviewer’s advice and revised the narrative in Abstract and Results to make the meaning more consistent. We have also included new analysis (Figure 5 K and L, Figure 7) and examples of individual neuron’s response patterns (Figure 6) to better describe the seemly contradicting features.

9) While these results are very interesting, they are of course restricted to a limited set (73?) of human-selected and human-curated stimuli. Nowhere does the analysis address the question as to whether these are close to the optimal stimuli for this region of cortex, even in the (unnatural) anesthetized state. The authors should have a more expanded discussion of these limitations, and how to extend the analysis further to get a sense of the “optimal” stimuli for this stage of cortical processing.

We agree that the stimulus set we used is simple and limited, and could not address the “optimal stimulus” problem. In the revised Discussion (the second last paragraph), we described these limitations and our thoughts on how to improve estimation of “optimal stimuli”.

Furthermore, the stimulus set used parallels that used by Hedge and Van Essen, 2000, in V4, and subsequently in V1 and V2 as well. Given that we know that selectivity’s exist that cannot simply be explained by orientation filters, the paper could do a better job of emphasizing what is new and, in particular, the significance of finding functional organization. If, for example, the authors feel that the presence of functional organization for a particular parameter is especially important in understanding the computations that occur within in an area, or in the broader picture, the role that area plays in visual processing, they should state so, and why they believe this.

These issues should be discussed and laid out appropriately.

We thank the reviewer’s suggestions. In the revised Discussion (the last paragraph), we discussed more on the significance of the finding of functional organization and our reasons.

[Editors' note: further revisions were suggested prior to acceptance, as described below.]

Revisions:

The authors have done an exemplary job in the revision, by addressing concerns raised and adding further analyses (clustering of single-cell-level response; PCA; more extended comparisons across classes of similar stimuli such as triangles / short oriented lines, as vs. circles / arcs; scaled to different sizes; filled / unfilled). Overall, these new analyses along with their original combination of intrinsic optical imaging and 2-photon imaging, showing the close spatial match between clusters identified separately by the two techniques make a compelling case for the specialised architecture the authors propose in the Discussion. There remain a small number of issues that would need to be addressed.

1) One disappointment is that some of the authors' responses didn't actually make into the manuscript or are "buried" in multi-panel supplementary figures. Specifically, the significance of relationship between surround suppression and curvature, and the PCA analysis (which proved interesting) are not particularly clear in the actual text of the manuscript. Also in light of some of the new analyses, the authors should probably cite some relevant literature about the existence of "sub-domain" specificity with domain-specific regions.

We added surround suppression and PCA results into the main text (new Figure 4L and M, new Figure 5). We also discussed relevant findings about “sub-domains” in different domain-specific regions and cite the relevant papers (first paragraph in Discussion).

2) The direct correspondence between intrinsic and 2-photon imaging is certainly positive (correlation coefficients > 0), but remarkably poor (and less than one), especially given that, for many experiments, intrinsic maps are seen as veridical and generally have good correspondence with targeted electrophysiology recordings. We noticed in the response letter that all intrinsic maps were Gaussian blurred, if one does the same procedure to the 2-photon data, does the correlation improve? And, if not, what explains this result: perhaps, significant noise in one type of measurement, or more interestingly, a true lack of correspondence between haemodynamics and cellular level responses. If the later, that's a big deal given: it potentially suggests that all haemodynamic maps should be taken with "a grain of salt" with regards to reflecting neuronal response property organization. And of course, if this is true, and intrinsic imaging is that noisy, it potentially calls into question many of the intrinsic signal difference maps and results even within this manuscript. This needs to be explicitly addressed.

The correlation between ISOI and 2-photon results shows a general consistency, e.g. the overall similarity between two types of maps (maps in Figure 4A-I). The relatively low correlation values (r=0.42, 0.43) in Figure 4J and K may due to the nature of the signals in these two imaging methods. Specifically, 2-photon signals contained substantial diversity in neighboring neurons’ responses to the same contours (new Figure 7), while such diversity was not reflected in the hemodynamic signals. Due to optical blurring and the nature of the hemodynamical signal, ISOI maps were much blurred than 2-photon maps. The 5x5-Gaussian-bluring was to reduce the grainy noise before the alignment. In Figure 4J and K, a cell’s 2-photon response was averaged from all pixels that the cell body covered (> 30 pixels), thus, a further 5x5 filtering would not make much differences.

Thus, we think both technical factors (nature of the signals) and intrinsic diversity (cell-cell diversity) contribute to the relatively poor correlations between the two types of maps.

3) More discussion in the actual manuscript about surround suppression, and specifically the points that appear in the authors' response, would be good. This is especially important, because surround suppression has long been associated with V4 responses, and there have been claims that it is functionally organized. The authors' remarks in the response about the only things the difference maps having is common is curvature is probably the best that can be made with the current data set. But, on the other hand, given the strong differences in surround suppression and receptive field size shown in the supplementary figures, that previous reports of surround suppression and its organization in V4 might simply reflect curvature regions that were studied with inappropriate sub-optimal stimuli.

In the revised manuscript, we added surround suppression results into the main text and discussed how these results relevant to the previous findings (the eighth paragraph in Discussion).

Many studies have shown strong surround suppression in area V4. One of them also revealed a functional structure (S regions) associated with this feature (Ghose and Ts’o 1997). Although curvature domains have common properties as those S regions (stronger surround suppression), these two types of domains are unlikely the same. First, stimuli produce S regions did not contain any curviness, while curvature domains were only obtained with curvatures. Second, surround suppression is a more fundamental property than curvature preference. Although curvature neurons showed surround suppression, not all surround-suppressive neurons are curvature neurons. In area V4, surround suppression likely also involves, for example, texture, color, motion etc, in addition to contour shapes. Thus, we believe V4 surround suppression domains are independent domains that deserves further exploring.

4) This is a really large data set of cellular responses, and the PCA analyses are really important, but again, it would be better for this to appear in the actual manuscript.

One reason in particular is the paper's discussion of "microorganization" or sub-domains within a curvature region. We found these discussions a little anecdotal, and we were wishing that a spatial autocorrelation analysis could be done with pairs of neurons as function of distance to reveal how big a "sub-domain" was, and in particular, how it compared with orientation columns, which of course are sub-domains within the rectiiinear domains. For example, what one could do with regard to PCA weights: PCA1 spatial autocorrelation. In a similar vein, there have been several reports of color-specific sub-domains with color-preference domains, and, especially given the classic association of V4 with color sensitivity, that literature seems relevant here in a discussion of how universal domain/sub-domain organization may be and its variation between different cortical areas.

We took the reviewers’ advice and analyzed the spatial periodicity of the sub-domains inside curvature domains. Specifically, we calculated the correlation of curve-orientation tuning of cell pairs inside the curvature domains, based on their responses to the 8 half-circle curves. Figure 6—figure supplement 2 plots how such correlation values vary with the cell-pair distance. It shows a trough around 360 μm and a second peak around 720 μm. Thus, the subdomain size is around 360 μm and the periodicity is around 720 μm. We had added this finding into the Results and discussed in Discussion. We did not use PCA results for such analysis since the curve-orientation was sorted before the PCA analysis.

5) Scaling of the intrinsic signal optical images. The authors' methods suggest that the SD used for the gray scale is calculated separately for each (SVM) map. Is that correct? If so, there is no way to compare the strengths of activation by different stimuli. It likely also explains why the single-condition maps (e.g. Figure 1—figure supplement 1, panels A,B,D,E,G,H) appear not only relatively featureless but also occupying a grayscale range similar to the backgrounds of the difference images. In absolute terms, the single-condition responses presumably are ~ an order of magnitude higher amplitude if they are dominated by stimulus-nonspecific haemodynamic responses? Showing the single-condition and difference images on the same absolute scale would be uninformative. But it would be useful to show all the difference images on a common scale that is not tied to the SD per image, to get a sense of the efficacies of different stimulus pairwise differences. This could be done as an additional set of panels in supplementary figures.

Different scaling methods (individual or common) were normally used for revealing different aspects of the map signals. Individual scaling is better at revealing spatial features, while common scaling can be used for signal strength comparisons.

In our manuscript, we used SVM maps, which was obtained through a non-linear process. The value of individual SVM pixel was proportional to its contribution to classification. Thus, it is not a simple representation of signal strength. Considering this, we used individual clipping to emphasize the spatial features of the domains. For analysis where comparing signal strengths were needed (e.g. Figure 2F), we used raw dR/R values instead of the SVM pixel values. We also tried a uniform clipping for Figure 1, the general appearance was similar to the individually clipped one in the manuscript.

https://doi.org/10.7554/eLife.57502.sa2

Article and author information

Author details

  1. Rendong Tang

    State Key Laboratory of Cognitive Neuroscience and Learning, IDG/MGovern Institute for Brain Research, Beijing Normal University, Beijing, China
    Contribution
    Conceptualization, Resources, Data curation, Software, Formal analysis, Supervision, Funding acquisition, Validation, Investigation, Visualization, Methodology, Writing - original draft, Project administration, Writing - review and editing
    Contributed equally with
    Qianling Song
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-3622-3383

  2. Qianling Song

    State Key Laboratory of Cognitive Neuroscience and Learning, IDG/MGovern Institute for Brain Research, Beijing Normal University, Beijing, China
    Contribution
    Conceptualization, Resources, Data curation, Software, Formal analysis, Supervision, Validation, Investigation, Methodology, Project administration
    Contributed equally with
    Rendong Tang
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-9177-7429

  3. Ying Li

    State Key Laboratory of Cognitive Neuroscience and Learning, IDG/MGovern Institute for Brain Research, Beijing Normal University, Beijing, China
    Contribution
    Data curation, Investigation
    Competing interests
    No competing interests declared

  4. Rui Zhang

    State Key Laboratory of Cognitive Neuroscience and Learning, IDG/MGovern Institute for Brain Research, Beijing Normal University, Beijing, China
    Contribution
    Data curation, Investigation
    Competing interests
    No competing interests declared

  5. Xingya Cai

    State Key Laboratory of Cognitive Neuroscience and Learning, IDG/MGovern Institute for Brain Research, Beijing Normal University, Beijing, China
    Contribution
    Data curation, Investigation
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-7829-3833

  6. Haidong D Lu

    State Key Laboratory of Cognitive Neuroscience and Learning, IDG/MGovern Institute for Brain Research, Beijing Normal University, Beijing, China
    Contribution
    Conceptualization, Resources, Data curation, Supervision, Funding acquisition, Validation, Investigation, Methodology, Writing - original draft, Project administration, Writing - review and editing
    For correspondence
    haidong@bnu.edu.cn
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-1739-9508

Funding

National Natural Science Foundation of China (31530029)

  • Haidong D Lu

National Natural Science Foundation of China (31625012)

  • Haidong D Lu

National Natural Science Foundation of China (31800870)

  • Rendong Tang

China Postdoctoral Science Foundation (2018M631373)

  • Rendong Tang

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Acknowledgements

This work was supported by the National Natural Science Foundation of China (31530029 and 31625012 to HDL and 31800870 to RT) and the China Postdoctoral Science Foundation (2018M631373 to RT). We thank Dr. Shiming Tang for valuable technical support on two-photon imaging. Lab members Jie Lu, Yan Xiao, Chen Fang, Kun Yan, Jingting Xu, Heng Ma, Jiayu Wang, Pengcheng Li, Chen Liang, and Wenhao Zhao provided technical assistance.

Ethics

Animal experimentation: All procedures were performed in accordance with the National Institutes of Health Guidelines and were approved by the Institutional Animal Care and Use Committee of the Beijing Normal University. Protocol number: IACUC(BNU)-NKCNL2016-06.

Senior Editor

  1. Joshua I Gold, University of Pennsylvania, United States

Reviewing Editor

  1. Kristine Krug, University of Oxford, United Kingdom

Reviewers

  1. Ed Connor, Johns Hopkins University, United States
  2. Aniruddha Das, Columbia University and Zuckerman Institute, United States

Publication history

  1. Received: April 2, 2020
  2. Accepted: November 18, 2020
  3. Accepted Manuscript published: November 19, 2020 (version 1)
  4. Version of Record published: December 1, 2020 (version 2)

Copyright

© 2020, Tang et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 1,533
    Page views
  • 207
    Downloads
  • 12
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Rendong Tang
  2. Qianling Song
  3. Ying Li
  4. Rui Zhang
  5. Xingya Cai
  6. Haidong D Lu
(2020)
Curvature-processing domains in primate V4
eLife 9:e57502.
https://doi.org/10.7554/eLife.57502

Categories and tags

Research organism

Further reading

    1. Neuroscience

    Curvature domains in V4 of macaque monkey

    Jia Ming Hu, Xue Mei Song ... Anna Wang Roe
    Research Article Updated

    An important aspect of visual object recognition is the ability to perceive object shape. Two basic components of complex shapes are straight and curved contours. A large body of evidence suggests a modular hierarchy for shape representation progressing from simple and complex orientation in early areas V1 and V2, to increasingly complex stages of curvature representation in V4, TEO, and TE. Here, we reinforce and extend the concept of modular representation. Using intrinsic signal optical imaging in Macaque area V4, we find sub-millimeter sized modules for curvature representation that are organized from low to high curvatures as well as domains with complex curvature preference. We propose a possible ‘curvature hypercolumn’ within V4. In combination with previous studies, we suggest that the key emergent functions at each stage of cortical processing are represented in systematic, modular maps.

    1. Neuroscience

    Cholinergic and noradrenergic axonal activity contains a behavioral-state signal that is coordinated across the dorsal cortex

    Lindsay Collins, John Francis ... David A McCormick
    Research Article Updated

    Fluctuations in brain and behavioral state are supported by broadly projecting neuromodulatory systems. In this study, we use mesoscale two-photon calcium imaging to examine spontaneous activity of cholinergic and noradrenergic axons in awake mice in order to determine the interaction between arousal/movement state transitions and neuromodulatory activity across the dorsal cortex at distances separated by up to 4 mm. We confirm that GCaMP6s activity within axonal projections of both basal forebrain cholinergic and locus coeruleus noradrenergic neurons track arousal, indexed as pupil diameter, and changes in behavioral engagement, as reflected by bouts of whisker movement and/or locomotion. The broad coordination in activity between even distant axonal segments indicates that both of these systems can communicate, in part, through a global signal, especially in relation to changes in behavioral state. In addition to this broadly coordinated activity, we also find evidence that a subpopulation of both cholinergic and noradrenergic axons may exhibit heterogeneity in activity that appears to be independent of our measures of behavioral state. By monitoring the activity of cholinergic interneurons in the cortex, we found that a subpopulation of these cells also exhibit state-dependent (arousal/movement) activity. These results demonstrate that cholinergic and noradrenergic systems provide a prominent and broadly synchronized signal related to behavioral state, and therefore may contribute to state-dependent cortical activity and excitability.

    1. Neuroscience

    Early language exposure affects neural mechanisms of semantic representations

    Xiaosha Wang, Bijun Wang, Yanchao Bi
    Research Article Updated

    One signature of the human brain is its ability to derive knowledge from language inputs, in addition to nonlinguistic sensory channels such as vision and touch. How does human language experience modulate the mechanism by which semantic knowledge is stored in the human brain? We investigated this question using a unique human model with varying amounts and qualities of early language exposure: early deaf adults who were born to hearing parents and had reduced early exposure and delayed acquisition of any natural human language (speech or sign), with early deaf adults who acquired sign language from birth as the control group that matches on nonlinguistic sensory experiences. Neural responses in a semantic judgment task with 90 written words that were familiar to both groups were measured using fMRI. The deaf group with reduced early language exposure, compared with the deaf control group, showed reduced semantic sensitivity, in both multivariate pattern (semantic structure encoding) and univariate (abstractness effect) analyses, in the left dorsal anterior temporal lobe (dATL). These results provide positive, causal evidence that language experience drives the neural semantic representation in the dATL, highlighting the roles of language in forming human neural semantic structures beyond nonverbal sensory experiences.