Figure 2A shows the time course of the responses evoked by the preferred (solid red line) and non-preferred (dashed blue line) side of border ownership, averaged over all well-isolated units that are selective for border ownership. Consistent with prior studies (Zhou et al., 2000), we observe that border ownership selectivity (difference in response to the preferred and the non-preferred side) occurs early after onset of the stimulus-evoked response. The asterisk indicates when the difference between these functions first becomes statistically significant (56.5 ms after stimulus onset; sign rank test p < 0.05 for 20 adjacent ms). When evaluating these functions in each laminar compartment we observe that these functions diverge substantially early after onset in deep cortical layers (Figure 2D; significant at 58.8 ms), but later in superficial (Figure 2B; significant at 85.8 ms) and granular layers (Figure 2C; significant at 67.3 ms). This definition of latency depends on sample size, but a subsampling analysis shows that differences in sample size between layers do not explain the shorter latency for deep layers (Figure 2—figure supplement 1). To compare the time course of border ownership modulation between layers, we defined the BOI function B for each laminar compartment as

Figure 2 with 3 supplements see all

Download asset Open asset

where R_pref(t) and R_non-pref(t) are, respectively, the evoked response functions to the preferred and the non-preferred sides of border ownership (red and blue dashed functions plotted in Figure 2A–D). B(t) is plotted for each laminar compartment in Figure 2E, confirming that border ownership modulation rises earlier in the deep layers than in the other laminar compartments. We quantified the difference by defining latencies on these functions as the crossing with a threshold defined from the null distribution obtained by shuffling the stimulus labels (see Methods). Latency was significantly shorter for deep layer units (75.8 ms, 95% CI [68.4 85.2]) than for granular layer units (94.7 ms, 95% CI [82.2 105.7], bootstrap procedure [see Methods] p = 0.006) and for superficial layer units (97.7 ms, 95% CI [78.0 103.7], bootstrap procedure p = 0.018). The same was true when well-isolated units and multiunits were pooled (deep: 78.2 ms, 95% CI [73.3 88.2], n = 210; granular: 94.0 ms, 95% CI [87.9 106.5], n = 167; superficial: 100.7 ms, 95% CI [85.1 104.6], n = 140; deep vs. granular: p = 0.007; deep vs. superficial: p = 0.009). To verify the robustness of the temporal differences between layers, we also evaluated the time course of border ownership selectivity using another method, by evaluating border ownership reliability (BOR) (Figure 2F; introduced by Zhou et al., 2000; detailed in Methods). Briefly, this metric quantifies the reliability of border ownership tuning when comparing spike counts between single trials to stimuli with opposite border ownership. Reliability values correspond to the proportion of such single trial comparisons for which the spike count is highest for the border ownership condition that is preferred across trials. We computed BOR in 100-ms sliding windows (Figure 2F; latency defined similarly as in Figure 2E, using right edge of the analysis window). Again we find that BOR rises earlier in the deep layers (89.9 ms, 95% CI [81.5 99.9]) than in the granular (105.4 ms, 95% CI [94.9 114.6]; bootstrap procedure p = 0.015) and superficial layers (109.4, 95% CI [98.9 124.9]; p = 0.006). After border ownership selectivity has been established, the average border ownership index tends to be higher in the deep compartment (Figure 2E; BOI between 200 and 500 ms after stimulus onset: mean ± standard error of the mean [SEM], 0.50 ± 0.02) than in the granular (0.43 ± 0.02) and superficial compartment (0.45 ± 0.03), but the differences do not reach statistical significance (deep vs. granular: Wilcoxon rank sum test p = 0.051; deep vs. superficial: p = 0.22). BOR saturates around 0.85 in all three compartments (Figure 2F). Together, these data indicate that border ownership selectivity does not occur first in granular layer units but instead in deep layer units.

To test whether the short latency in deep layers is specific for border ownership or a general feature of this laminar circuit, we performed two additional analyses. First, we evaluated the latency of spikes evoked by small ring stimuli in the cRF. For these responses we find, in contrast to border ownership selectivity, that the latency is shorter in the granular layer compared to the deep layers and superficial layers (Figure 3; latency defined as crossing of the functions with a threshold value a third of the way from baseline to peak). Note that these responses are derived from the same stimuli used to compute the CSD maps, but represent a different signal (spiking responses as opposed to the current sink–source patterns from local field potentials used to define the laminar compartments). Second, we evaluated the responses to the border ownership stimuli for another type of selectivity, contrast polarity. This refers to the relative luminance contrast across the edge, that is the difference between panels 1 and 3 (or between panels 2 and 4) in Figure 1A. For the contrast polarity index functions (Figure 2—figure supplement 2), we do not find an earlier rise in selectivity in the deep layers, even though they are derived from the same units as in Figure 2. Together, these data indicate that the earlier selectivity in deep layers compared to granular and superficial layers is specific for border ownership.

Figure 3

Download asset Open asset

How does the preferred side of border ownership compare between units recorded in a column of cortex? For a given edge, for example vertical, there are two possibilities for preferred side of border ownership: left or right from the edge. Border ownership for such an edge has been assumed to be represented by the activity of two oppositely tuned subpopulations (green and purple in Figure 4A, top; Craft et al., 2007). Our data show indeed that these two subpopulations exist in similar proportions (Figure 4A, top). The same is true for units that encode border ownership for horizontal edges (Figure 4A, bottom), and when we express the preferred side of border ownership not relative to the screen, but relative to the fixation point (Figure 4—figure supplement 1). This indicates that for a given edge, the two possible sides of border ownership are encoded by distinct subpopulations of neurons that are similar in size.

Figure 4 with 1 supplement see all

Download asset Open asset

These two subpopulations could be mixed in a salt-and-pepper pattern, or they could be clustered according to their preferred side of border ownership (Figure 4B). Figure 4C shows the signed BOI for all well-isolated units and multiunit clusters selective for border ownership for two example penetrations. The sign of BOI indicates which side of the border is preferred by each unit, as indicated by the cartoons above the panels. For both of these penetrations, all border ownership units recorded on the same probe prefer the same side of border ownership. In the population of all penetrations with at least four border ownership units, the proportion of units that share the same preferred side of border ownership is significantly higher than chance (well-isolated units: 74.4%, randomization test p = 0.017, 23 penetrations with 110 units; including multiunit clusters: 71.7%, p = 0.005, 47 penetrations with 288 units). Also in the subgroup of penetrations with border ownership units spanning from superficial to deep layers, we find significant clustering (Figure 4D; well-isolated units: 81.1%, randomization test p = 0.010, 6 penetrations with 30 units; including multiunit clusters: 73.7%, p = 0.003, 20 penetrations with 136 units). Together, these data indicate that border ownership preference is organized in spatial clusters that span cortical layers in a columnar fashion.

Border ownership acts on edges that are oriented. Similar to our finding that preferred border ownership is clustered, several imaging studies have shown that preferred orientation is spatially clustered in V4 in domains (e.g., Li et al., 2013; Tanigawa et al., 2010), but it is not known whether these form columns. To test whether orientation preference in V4 is shared in clusters that extend vertically across layers, we determined orientation tuning for the units in our sample from responses to luminance contrast edges centered on the cRF (independent data set from the border ownership stimuli, see Methods). Our data confirm clustering of preferred orientation and reveal that these clusters span across laminar compartments in V4 (Figure 5). Each polar plot in Figure 5 shows a penetration with at least four orientation-selective well-isolated units or multiunit clusters. Solid vectors indicate the preferred orientation of each orientation-selective unit as the resultant vector of the responses to edges of different orientations, and color indicates laminar compartment (open symbols indicate multiunit clusters). The preferred orientation for a penetration was then calculated as the resultant vector across the vectors of all orientation-selective units in that penetration, and its angle is indicated by the blue dashed line. The significance of clustering of orientation preference was assessed by comparing the magnitude of this resultant vector against a null distribution generated by randomizing the preferred orientation for each orientation-selective unit in the penetration and performing the same calculation. For each of these six penetrations, this resultant was significantly larger than expected from the null distribution (p < 0.05). This was true for 9 out of 18 penetrations with at least four orientation-selective well-isolated units distributed over all three laminar compartments (for 17 out of 34 penetrations when well-isolated units and multiunit clusters were pooled). These data suggest that orientation domains in V4 are columnar.

Figure 5

Download asset Open asset

Having found that both border ownership preference and orientation preference are organized as columnar clusters in V4, we next asked what the relation was between orientation selectivity and border ownership selectivity. We find that border ownership selectivity is significantly more common in units that are selective for orientation than in those that are not, but it is also surprisingly common in units that are not selective for orientation (Table 1, respectively, 53.3% and 36.8%, Chi square = 9.51, p = 0.002; including multiunit clusters: respectively, 45.9% and 31.3%, Chi square = 19.5, p = 0.00001).

Three example orientation-selective well-isolated units are shown in Figure 6A. These polar plots are organized according to the location of the square object relative to the cRF, for different orientations of the central edge (the edge that runs through the cRF). This is indicated by the stimulus cartoons around the plot. For opposite locations on the polar plot, the central edge has thus the same orientation, but is owned by a square positioned on opposite sides of the cRF (opposite border ownership). The BOI for each tested orientation of the central edge is shown by a red vector (filled red circles indicate statistically significant border ownership selectivity). The amplitude of this vector corresponds to |BOI|, and the polarity of the vector points toward the square location that corresponds to the preferred side of border ownership for that orientation of the central edge. For example, in case of a vertical central edge, the third unit (cyan triangle) prefers that that edge is owned by a square on the right (if it would have preferred the vertical central edge to be owned by a square on the left, the red vector that points down would have pointed up). The black line indicates the preferred orientation of a luminance contrast edge in the cRF for each unit. For the left unit in Figure 6A (red star), this preferred edge orientation matches with the orientation of the central edge for which border ownership selectivity is significant (filled red circle). This is the expected scenario: border ownership has been assumed to act on edges at the preferred orientation, and prior studies have used edges at the preferred orientation to test border ownership selectivity (Zhou et al., 2000). At the population level, the average orientation of edges with border ownership selectivity is indeed mildly biased toward the preferred orientation for edges. This is subtle but can be seen as the tendency of the data points in Figure 6C to appear close to the identity lines. Note that both plotted variables in Figure 6C are periodic and two periods are shown for each (thus each data point is plotted four times, and the identity line appears three times; the gray square outlines an area corresponding to one period for both variables). This bias is easier to see when the closest distance of the data points to the identity lines is plotted (Figure 6D). This distance is on average distance 27.6°, significantly smaller than chance (randomization test [see Materials and methods] p = 0.018; n = 68 well-isolated units tested with at least four orientations; including multiunit clusters: p = 0.017; n = 159). Orientation-tuned units thus tend to be border ownership-selective for edges with an orientation near their preferred orientation, but this relation is surprisingly subtle.

Figure 6 with 1 supplement see all

Download asset Open asset

Some units only have border ownership selectivity for orientations that are far from the identity line, such as the unit indicated with the blue square in Figure 6A (middle panel, corresponding to the same symbol in Figure 6C). This unit shows, paradoxically, significant border ownership selectivity for edge orientations that are nearly orthogonal to the preferred edge orientation. This is true for two edge orientations (filled red circles), suggesting that this misalignment is systematic. We ruled out that this is related to a difference in orientation preference for isolated edges versus edges that are part of squares by comparing orientation tuning for both data sets. Figure 6—figure supplement 1A shows that the preferred orientations for both stimuli match very well for this unit (black line vs. cyan line), and this is true for the population as well (Figure 6—figure supplement 1B).

Furthermore, we find that border ownership-selective units often show border ownership selectivity to edges that maximally differ in orientation, that is orthogonal edges. An example unit is shown in the right panel in Figure 6A (cyan triangle). This is true for 29.7 % of border ownership-selective well-isolated units (n = 182; 24.4 % including multiunits, n = 353) that were tested with sets of squares at orthogonal angles. In such cases, the preferred side of border ownership for a central edge with an orientation in between those orthogonal orientations could in theory be spatially discontiguous (Figure 6B, left panel) or contiguous (Figure 6B, middle panel) with the preferred sides of border ownership for the pair of orthogonal orientations. We find that for all 24 out of 24 well-isolated units that were selective for the border ownership of squares at orthogonal and intermediate orientations, the preferred side of border ownership for these intermediate orientations was spatially contiguous (such as the example in Figure 6B, right panel; including multiunits, this is true for 35 out of 35 units). Those units thus have a wide but spatially contiguous area of preferred sides of border ownership. For example, the unit in Figure 6B (right panel) has an area of preferred sides of border ownership that spans 120° (cyan double arrow). A span of at least this width occurs for 18.0 % of border ownership units (Figure 6E; 15.4 % including multiunit clusters).

Given that about half of the neurons responsive to edges in V2 are selective for border ownership (Zhou et al., 2000), neurons in V4 could simply inherit this property from V2 through the feedforward projection from V2 to V4. This projection is the most important source of feedforward input to V4 (Markov et al., 2011), and targets mostly layer 4 (granular layer) in V4 (Gattass et al., 1997; Rockland, 1992). Our data show that spike latency for stimuli in the cRF is shortest in the granular layer, which is similar to what has been described in other areas in macaque visual cortex (V1: e.g., Nowak et al., 1995; MT: Raiguel et al., 1999), and consistent with work by others in awake V4 (Lu et al., 2018). Note that this may not be universal across areas: in anesthetized V2 spike latency has been reported to be longer in layer 4 than in infragranular layers (Nowak et al., 1995).

If this feedforward projection would thus simply provide the earliest border ownership signals in V4, we would thus detect border ownership selectivity in the granular layer at least as early as in the other compartments, or earlier. While we do observe such a pattern for responses evoked by flashed stimuli appearing within the cRF, and also for the emergence of selectivity for contrast polarity (Figure 2—figure supplement 2), we do not observe this pattern for selectivity to border ownership. Instead, we find that deep layer neurons compute border ownership selectivity significantly earlier than neurons in the granular layer and in the superficial layers. One might argue that these data are consistent with V2 afferents preferentially targeting apical dendrites of deep layer neurons that extend in the input layer. But in that case, we would expect shorter spike latencies for deep layer neurons than for granular layer neurons irrespective of the cue or stimulus. This is, again, not what we find: spike latencies for responses evoked by small stimuli in the cRF are shorter in the granular layer than in the deep layers, and also for contrast polarity we do not observe the earliest selectivity in deep layers in V4.

Another possibility is that deep layer neurons perform this computation earliest solely by relying on an integration of feedforward signals from the V2-to-V4 projection relayed by multiple granular layer neurons. We think this is unlikely. First, this would mean that those relaying granular layer neurons do not receive sufficient contextual information to become selective for border ownership as fast as the deep layer neurons that integrate their output, despite the substantial input that granular layer neurons receive from other granular layer neurons (Xu et al., 2016). Second, while deep layer neurons certainly receive input from granular layer neurons, the densest projections from granular layer neurons typically target superficial layers (e.g., macaque V1: ; Callaway, 1998; rat barrel cortex: Lübke et al., 2000). If that projection leads to the first computation of border ownership signals, one would thus expect to observe these signals in superficial layer neurons as fast as in deep layer neurons, in contrast with our data.

We think that the earlier computation of border ownership selectivity by neurons in deep layers is more likely a consequence of their unique properties. Scanning laser photostimulation studies in primary visual cortex showed that layer 5 neurons receive significant input from nearly all cortical layers, regardless of cell type, as opposed to neurons in other layers (Briggs and Callaway, 2005; Xu et al., 2016). Deep layers of the cortex, including in primates, include tall pyramidal cells whose apical dendrites reach up to layer 1, which could thus directly sample and integrate afferent information that arrives in a wide range of layers (Binzegger et al., 2004; Callaway, 1998; Lund and Boothe, 1975; Markov et al., 2014; Zarrinpar and Callaway, 2016). Such neurons thus seem well suited to integrate information provided by feedforward input with contextual information provided through corticocortical feedback and horizontal connections (Harris and Shepherd, 2015). Corticocortical feedback in V4 terminates densely in all layers except layer 4 (Markov et al., 2011; Rockland et al., 1994). Intra-areal horizontal connections are prominent in layer 2/3 and in layer 5 (Lund et al., 1993; Yoshioka et al., 1992; Douglas and Martin, 2007). This may set these deep layer neurons up to be able to integrate the border in the cRF (light gray arrow in Figure 7) with visual context from a wide region of space (dark gray arrows in Figure 7), which is required to compute border ownership selectivity. Indeed, studies in V2 showed that border ownership selectivity does not rely on a small number of localized object features but instead occurs through integration of extraclassical stimulus features over large areas of visual space (Zhang and von der Heydt, 2010). Specialized intrinsic properties could further assist in this integration, such as the calcium spikes in apical dendrites of layer five neurons that can amplify the effects of feedback inputs (Takahashi et al., 2016). It would be interesting to test whether deep layer neurons in other areas, for example in V2, also display early border ownership signals (although obtaining penetrations orthogonal to the cortical surface in macaque V2 is challenging, given that only a small part of this area is exposed on the lateral convexity of the brain). Perhaps such contextual integration is a general computation performed by deep layer neurons in different cortical areas. In addition, as discussed below, deep layer neurons in V2 may inherit early border ownership signals from V4.

Figure 7

Download asset Open asset

Our findings are consistent with the concept of the grouping cell model (von der Heydt, 2015). This model proposes that border ownership is a modulation of feature neurons by an external grouping signal that acts through feedback. The grouping signal is supposed to represent a ‘proto-object’, a computational structure that (1) links the visual feature signals, (2) is being remapped across eye movements, and (3) serves object-selective attention. Compared to the large number of studies of feature selectivity in visual cortex, the question of the representation of object structure has prompted relatively few studies. It remains unclear where and how such grouping signals are generated (Zhu et al., 2020). It is also possible that these deep layer neurons in V4, rather than computing border ownership selectivity de novo from an external grouping signal, inherit this selectivity from higher areas through corticocortical feedback. The source of feedback, whether it provides border ownership signals or grouping signals, would need to have a spike latency that is short enough to explain the early border ownership signals in deep layers in V4. Because of shorter latencies, dorsal stream areas such as LIP may therefore be more likely sources of such feedback than inferotemporal cortex, where shape-selective signals occur after 100 ms (Brincat and Connor, 2006; Bullier, 2001; see also discussion in Zhu et al., 2020).

Our conclusion that the earliest border ownership signals in V4 are not inherited from V2 does not imply that the projection from V2 to V4 does not carry any border ownership signals. Indeed, since about half of V2 neurons are selective for border ownership (Zhou et al., 2000), this feedforward input most likely contributes to the border ownership signals in V4 later in the response. As argued in the next section, these border ownership signals in V2 may have been sculpted by border ownership-selective feedback from deep layers in V4. That V4 does not simply inherit its earliest border ownership signals from V2 is also consistent with a study in V4 on shape encoding that compared responses to identical shapes in the presence or absence of an occluder (Bushnell et al., 2011). When the preferred shapes had sharp convexities and concavities, responses were suppressed if those features were rendered incidentally by an adjacent occluder. Suppression latency was relatively short (63 ms after stimulus onset in the population, and for an example neuron as short as 46 ms). Because this is shorter than the latency of border ownership signals reported in V2 (under somewhat different stimulus consitions: ~68 ms; Zhou et al., 2000), the suppression effect is likely not due to border ownership signals that are inherited from V2. A caveat when interpreting this suppression as border ownership selectivity is that both the shape and the occluder fell in the cRF. Thus, while the suppression may be due to the change in border ownership, it may instead be due to local contour geometry at the T-junctions in the cRF. In any case, this suppression of incidental shapes in V4 could contribute to border ownership signals in V2 in the presence of occluders, through feedback.

A computational model has shown that contextual information carried by myelinated feedback afferents could indeed arrive fast enough to result in border ownership selectivity within ~20 ms after response onset (Craft et al., 2007). An alternative mechanism based solely on horizontal fibers is unlikely, given the slow conduction velocity of these fibers (Girard et al., 2001; Zhang and von der Heydt, 2010), although, as discussed by Zhaoping, 2005, it should be noted that data on horizontal fibers are scarce. That said, horizontal fibers could still provide part of the required contextual information: since these fibers are limited in length, their role could be to provide information from the portions of the stimulus closest to the cRF (reminiscent to their proposed role in surround suppression in V1 [Angelucci et al., 2017]). This would explain why the effect of near corners of squares tends to occur later than that of far corners of squares (Zhang and von der Heydt, 2010).

Once border ownership selectivity has been established in V4 deep layers, these signals may contribute to visual processing along different pathways (black arrows in Figure 7). First, V4 is the dominant source of cortical feedback to V2, and 75 % of these feedback neurons are located in deep layers in V4 (Markov et al., 2014). Early border ownership signals in deep layers of V4 may thus sculpt border ownership selectivity in V2. Second, V4 deep layers include neurons that project to the superior colliculus (Fries, 1984; Gattass et al., 2014). The superior colliculus may thus receive early border ownership signals from V4, which could contribute to planning upcoming saccades. Saccades target objects more frequently than background (Rothkopf et al., 2007). Early border ownership signals in deep layer neurons that project to the superior colliculus could serve to facilitate the rapid foveation of objects. Our data show that these signals are established in deep layers well before 100 ms after stimulus onset. They are thus computed quickly enough to allow time for saccade planning within the intersaccadic interval, which is typically on the order of 200–300 ms (Otero-Millan et al., 2008). This proposed early role for deep layer neurons in supporting saccadic decisions is consistent with recent findings showing that deep layer V4 neurons encode more information about the direction of planned eye movements than do superficial neurons (Pettine et al., 2019). Border ownership signals may thus play a role beyond visual perception, perhaps including a role in guiding rapid oculomotor behavior. Finally, deep layer neurons may convey border ownership signals to neurons in more superficial layers, which would be consistent with the canonical laminar circuit (Douglas and Martin, 2004). Our results suggest a potential strategy to study the flow of these signals within single trials. We find that border ownership preference is shared across layers in a columnar fashion. Border ownership is presumably evaluated in the brain by comparing the activity between neurons from such oppositely tuned border ownership columns. In future work, it would thus be interesting to record from such columns simultaneously, and use directionality measures such as Granger causality (e.g., Ferro et al., 2021) to study the interlaminar flow of border ownership signals within single trials.

Prior studies on border ownership only evaluated border ownership selectivity at the preferred orientation of each neuron (Hesse and Tsao, 2016; Zhang and von der Heydt, 2010; Zhou et al., 2000). In our data, there is indeed a bias toward border ownership selectivity for edges near the preferred orientation. However, we find that there is substantial scatter (Figure 6C and D). This does not seem to stem from measurement noise. First, there is a good match between preferred orientations estimated from two separate data sets: one based on recordings made with isolated edges, the other using edges that are part of squares (Figure 6—figure supplement 1), suggesting that our estimates of orientation preference are highly reliable, with relatively tight bounds. Second, for some units, border ownership selectivity, surprisingly, systematically only occurs for edges with orientations that are nearly orthogonal to the preferred orientation (Figure 6A, middle). Third, almost a third of units that are selective for border ownership to an edge with a given orientation are also border ownership-selective to an edge that is orthogonal to that orientation. This reflects systematic tuning, because the preferred sides of border ownership in such cases form a single contiguous area in retinotopic space (Figure 6B). Finally, a substantial number of units that are selective for border ownership are not selective for orientation (Table 1). Together, these data indicate that the relation between orientation and border ownership is much richer than has previously been appreciated. Encoding of orientation and of border ownership may represent separate axes of representation. This is not inconsistent with the idea that border ownership assignment represents a surface signal rather than a property tied to a border (Grossberg, 2015; Nakayama et al., 1995; Peterson and Skow, 2008).

Functional clustering is a recurring theme in primate cortical visual areas (Hubel and Livingstone, 1987), and has been reported for several modalities in V4 including orientation, hue, direction of motion, spatial frequency, and recently curvature (Hu et al., 2020; Jiang et al., 2021; Li et al., 2013; Liu et al., 2020; Lu et al., 2018; Roe et al., 2012; Tang et al., 2020; Tanigawa et al., 2010). We find here that border ownership preference is preserved across layers within a vertical penetration, indicating that this modality is also organized in a columnar fashion. Importantly though, border ownership is of a fundamentally different nature than these other modalities: the border ownership of an edge is computed based on stimulus features falling outside the cRF. Combined with our finding that early border ownership signals are computed by deep layer neurons rather than being passed on from upstream areas, this columnar organization has important implications for the functional anatomy. It indicates that there is a systematic asymmetric arrangement of the extraclassical contextual information, which, as argued above, may be provided through horizontal fibers and corticocortical feedback. For example, consider a cluster in which neurons prefer that a vertical edge belongs to an object on the left (left column in Figure 7, symbol below left column indicates preferred border ownership). The neurons in this column thus need to receive asymmetric contextual synaptic information favoring the presence of an object to the left of the edge (symbol above the left column in Figure 7). Neurons in another cluster (right column in Figure 7) will instead prefer that the same edge is part of an object on the right, and these neurons will thus necessarily receive asymmetric contextual information of the opposite polarity (symbol above right column). The clustered architecture of preferred border ownership therefore requires clusters of asymmetric contextual information in cortex, such that opposite polarities of contextual information occur in distinct and complementary clusters. The present data indicate that border ownership selectivity initially does not arrive through the feedforward pathway. They thus suggest that the substrate of these clusters consists of clustered asymmetries in the retinotopic information carried by afferents from horizontal fibers and cortical feedback.

We obtained recordings in two male rhesus macaques (Macaca mulatta), age 13 years (animal Z) and age 15 years (animal D). Both animals had no prior experimental history and were housed in separate cages in a primate room with up to six animals of the same species. This study was performed in accordance with the recommendations in the Guide for the Care and Use of Laboratory Animals of the National Institutes of Health. All procedures were approved by the Institutional Animal Care and Use Committee of the Salk Institute for Biological Studies (protocol 14-00014).

Surgical procedures have been described before (Nandy et al., 2017). In brief, in a first surgical session a titanium recording chamber was installed in a craniotomy over the prelunate gyrus, according to stereotactic coordinates derived from anatomical MRI scans from each animal (left hemisphere in animal Z, right hemisphere in animal D). In a second surgical session, the dura mater within the chamber was removed, and replaced with a silicone-based optically clear artificial dura, establishing an optical window over dorsal area V4.

At the beginning of a recording session, a sterile insert consisting of a metal ring covered with a plastic membrane on the bottom was lowered in the chamber. The membrane was perforated to allow insertion of probes. The function of the insert is to stabilize the recording site from cardiopulmonary pulsations. A linear multielectrode probe (32-channel single-shaft acute probes, 100 µm electrode pitch [ATLAS Neuroengineering, Leuven, Belgium]) was mounted on the chamber using a hydraulic microdrive on an adjustable x–y stage (MO-972A, Narashige, Japan). The probe was then lowered through the artificial dura over the prelunate gyrus, positioned orthogonally relative to the cortical surface under visual guidance (Zeiss microscope). While monitoring the voltage signals from the electrodes for multiunit activity, the probe was lowered to penetrate the cortical surface. The probe was advanced until multiunit activity was visible on the deepest ~2600 µm of the probe. Then, the probe was retracted typically by several 100 µm to ease dimpling of the cortex. Between recording sessions, probe position was varied (RF eccentricity median 4.87 degrees of visual angle (dva), interquartile range 2.51 dva). Neural signals were recorded extracellularly, filtered and saved using Intan hardware (RHD2132 amplifier chip and RHD2000 amplifier evaluation system, Intan Technologies LLC, Los Angeles, USA) controlled by a Windows computer.

Visual stimuli were presented using a LED projector, back-projected on a rear-projection screen that was positioned at a distance of 52 cm from the animal’s eyes (PROPixx, VPixx Technologies, Saint-Bruno, Canada). The MonkeyLogic software package developed in MATLAB (https://www.brown.edu/Research/monkeylogic/; https://monkeylogic.nimh.nih.gov/; Hwang et al., 2019) was used for stimulus presentation, behavioral control and recording of eye position. A photodiode was used to measure stimulus timing. Eye position was continuously monitored with an infrared eye tracking system (ISCAN model ETL-200, Woburn, MA) and eye traces were saved using MonkeyLogic. Trials were aborted if eye position deviated from the fixation point (threshold typically 1 dva radius).

At the beginning of each recording session, RF mapping data were obtained using a subspace reverse correlation approach (Nandy et al., 2017). Stimuli consisted of static modified Gabors, constructed using square-wave instead of sinusoidal gratings, and dark gray rings (80 % luminance contrast, diameter 2 dva, thickness 0.25 dva). Grating spatial frequency and phase were such that a single contrast edge was visible centrally in the Gaussian window (grating parameters: 6 orientations, 2 contrast polarities, typically 80 % luminance contrast, one half was one of seven colors or grayscale, the other half was always grayscale; window: FWHM 2 dva). The stimuli were presented every 50 or 60 ms while the animal maintained fixation. Each stimulus appeared at a random location selected from a grid sized 25 × 25 dva with 1 dva spacing centered at coordinates [7.5 dva; 7.5 dva] in the appropriate visual quadrant. During the recording session, high-gamma filtered voltage waveforms in response to these stimuli were analyzed to estimate the retinotopy of the probe position, and choose location and size for the border ownership stimuli. Detailed RFs were calculated for each unit offline after spike sorting and used to verify the proper position of the border ownership stimuli.

A CSD mapping procedure on evoked local field potentials (LFP) was used to estimate the laminar position of recorded channels (Nandy et al., 2017). Briefly, animals maintained fixation while dark gray ring stimuli were flashed (32 ms stimulus duration, 94 % luminance contrast, sized and positioned to fall within the cRF of the probe position). The CSD was calculated as the second spatial derivative of the stimulus-triggered LFP (filtered between 3.3 and 88 Hz) and visualized as spatial maps after smoothing using bicubic (2D) interpolation (Figure 1—figure supplement 1; MATLAB function interp2 with option cubic, the spatial dimension was interpolated at a resolution of 10 µm), but the laminar analysis did not critically depend on this particular method of smoothing (Figure 2—figure supplement 3). Red regions depict current sinks, blue regions depict current sources. As described in more detail in Results, we observed a consistent pattern between different penetrations, strikingly similar to the current sink–source maps reported by other laboratories in behaving macaques beyond V1 (V4: Pettine et al., 2019; V4: Lu et al., 2018; area 36: Takeuchi et al., 2011). Through histological verification Takeuchi et al., 2011 found that the prominent current sink with the shortest latency corresponded to the position of the granular layer. We therefore identified this current sink (current sink indicated by white star, between dashed and dotted white lines in Figure 1F and G; Figure 1—figure supplements 1 and 2) as the granular layer. For each unit, the positions of the electrode contacts (using the five contacts surrounding the one with the largest spike waveform) were weighed by the average peak-to-trough amplitude of the unit’s spike waveform on these contacts, to assign the unit to the depth where its spike waveform is largest. By comparing this position with the range of contacts in the granular layer, we could locate units to superficial, granular, or deep layers. Seven penetrations where the CSD map could not be interpreted were excluded from the laminar analyses, and we also required that all units included in the laminar analyses were located ≤2 mm from the most superficial channel with multiunit activity, to minimize the risk of including white matter activity. We evaluated the orthogonality of penetrations by comparing the receptive field (RF) positions for neural activity recorded on different electrode contacts on the probe. To obtain an estimate for these RF positions, for the population of neurons in the vicinity of each electrode contact, we derived multiunit activity as the amplitude of the envelope from the signal on each electrode contact, by band-pass filtering the recorded signal between 500 Hz and 5 kHz, rectifying it and then low-pass filtering it at 200 Hz (adopting the approach from Self et al., 2019). RF contours at z = 3 were then computed on the z-scored root mean square level of this analog signal, using the same procedure as for the spiking data (see Receptive field mapping). For each contour, the RF center was defined as the centroid of the region defined by the contour (using MATLAB function centroid). A line was then fitted in three dimensions through these RF centers (stacked according to the position of the electrode contact from which they were computed) from a range of electrode contacts covering 2 mm, starting with the most superficial contact that recorded multiunit activity. The variation between RF positions along the probe was then computed as D, the distance in the azimuth × elevation plane between the ends of this line, per mm depth (D is indicated for the vertical positions of RF contours shown in Figure 1H,I and Figure 1—figure supplement 1C, and the population data are described in Results).

A data set for orientation tuning was obtained using luminance contrast edges similar to those used for RF mapping, but with a circular window, sized and positioned such that the stimulus covered and was centered on the estimated aggregate cRF of the probe (12 orientations; 2 contrast polarities; 54 % luminance contrast; 200 ms stimulus duration).

From the online analysis on the RF mapping data, position, size, and color of the border ownership stimulus set were chosen. An isoluminant square was positioned on an isoluminant background, as in prior studies on border ownership (Zhou et al., 2000). There were four basic conditions (Figure 1A and C), consisting of a factorial combination of two square positions and two contrast polarities. Note that this results in two pairs of stimuli, where both scenes in a pair have identical stimulus information inside the cRF (dotted black circles in Figure 1A and C). The two luminance areas in each scene were either both grays or a combination of gray and a color, and luminance contrast was 54%, as in prior studies (Zhou et al., 2000). Square sizes were between 12 × 12 and 20 × 20 dva. At the beginning of each trial, a small light gray fixation point (0.2 × 0.2 dva, 80 % luminance contrast) was presented on a blank gray screen (with luminance set at the geometric mean of the luminances in the border ownership stimulus). After the animal maintained fixation for 400 ms, the border ownership stimulus appeared for 500 ms. Then, for a separate project, the stimulus was replaced with a stimulus in which the central edge was prolonged to cover the entire screen (with identical colors and luminances) and the other parts of the square removed, for another 1000 ms. Spikes elicited during that time window were not included in the analyses in this paper. The animal received a juice reward if it maintained fixation throughout the trial. Depending on recording time, data were obtained for different orientations or positions. Conditions were played pseudorandomly in counterbalanced blocks such that each condition was played once before repeating conditions. Typically 8–10 repetitions were obtained per condition. Often these stimuli were played at a few different orientations and/or positions, in order to increase the likelihood of proper stimulus placement for most units recorded on the probe (because the precise cRF for each unit was only available offline, after spike sorting). On some days, the trials analyzed here were randomly interdigitated with similar trials using stimuli defined for other projects. Some of these trials – not analyzed here – included a condition in which the animal had to saccade to a new fixation position if the fixation point moved.

Data were analyzed in MATLAB (MathWorks, Natick, MA). Circular statistics were computed using the MATLAB toolbox CircStat (Berens, 2009). Statistical tests for the different analyses are described in detail below. Statistical significance was defined as p < 0.05.

The data were sorted offline using SpyKING CIRCUS (Yger et al., 2018). The clusters resulting from the automatic sorting step were curated manually using the MATLAB GUI provided by the SpyKING CIRCUS software. Well-isolated units were identified based on a well-defined refractory period in the interspike-interval histogram. Multiunit clusters included in the analysis had to pass a criterion for the signal-to-noise ratio: peak-to-peak amplitude of the average waveform had to exceed five times the standard deviation of the signal 5 ms prior to the peak (similar to Kashkoush et al., 2019), after high-pass filtering the data (1-pole butterworth filter, cutoff 300 Hz, implemented using functions butter and filtfilt in MATLAB).

To determine the cRF, spikes were counted in a window [30 100] ms after each stimulus onset. The resulting mean counts per stimulus position were transformed to z-scores by first subtracting the mean of and then dividing by the standard deviation of spike counts occurring in a window of the same size preceding that stimulus position (Keliris et al., 2019). Using the stimulus positions, z-scores were transformed to a spatial map, which was smoothed with a Gaussian filter (MATLAB function imgaussfilt with σ = 1). The outline of the cRF was defined as the contour at z = 3 on this smoothed map (calculated using MATLAB function contourc).

Border ownership responses were obtained by recording evoked responses to stimuli as in Figure 1A and C (see Border ownership stimuli above). A unit’s response was evaluated for border ownership selectivity if it passed the following inclusion criteria: (1) average spike rate was ≥1 spike/s for at least one of the four conditions; (2) the evoked spike count for at least one of the four conditions was significantly different from that recorded prior to all trials across conditions (two-sided Wilcoxon rank sum test [MATLAB function ranksum] with Bonferroni correction); (3) at least six trials per condition were available; (4) the central edge of the squares in the border ownership stimulus intersected the cRF; (5) the distance between any part of the cRF contour and any part of the noncentral edges of the squares in the border ownership stimulus was ≥1 dva. Data sets obeying these inclusion criteria were candidate data sets for border ownership selectivity. The border ownership index (BOI) (Zhou et al., 2000) was then calculated for these data sets, which is defined as

where R_i represents the average spike rate in the window [50 500] ms after stimulus onset for condition i (numbering as in Figure 1A and C). Statistical significance of border ownership selectivity was evaluated using a permutation test: a null distribution was created by shuffling the border ownership stimulus labels 10,000 times, separately for each luminance contrast pair, and the p value was estimated as the fraction of |BOI_shuffled| that was at least as large as |BOI|. A data set was defined to be border ownership-selective if p < 0.05 (with Bonferroni correction if multiple orientations or positions were available for the same unit), and a unit was defined to be border ownership-selective if it had at least one border ownership-selective data set.

For the time course of evoked activity of border ownership-selective units (Figure 2A–D), each unit contributed one data set, that is the border ownership-selective data set – as defined above – for which |BOI| was maximal. For each of these, the time courses of the responses to the preferred side of border ownership (side resulting in the highest spike rate) and to the non-preferred side of border ownership (side resulting in the lowest spike rate) were calculated separately (respectively, solid red lines and dashed blue lines Figure 2A–D) as follows. Spike trains were rounded to 0.1 ms resolution and convolved with a postsynaptic kernel K(t) (Thompson et al., 1996)

where τ_g = 1 ms and τ_d = 20 ms. The resulting traces were averaged per condition, and then across both contrast polarities. These average traces were normalized for each unit’s evoked response by dividing them by the average value across conditions in the window [50 500] ms after stimulus onset. The mean ± SEM of the resulting functions across units are shown in Figure 2A–D. The latency of statistical difference between the functions to the preferred and the non-preferred functions (asterisks in Figure 2A–D; Figure 2—figure supplement 1) was defined as the first time after which the functions differed statistically (Wilcoxon sign rank test p < 0.05) for 20 adjacent milliseconds.

BOI functions (Figure 2E) were defined as the difference between the response function for the preferred side of border ownership and the function for the non-preferred side of border ownership, divided by their sum. Latency of these functions was defined as the earliest crossing of a fixed threshold that was followed by values above the threshold for 20 consecutive milliseconds. The threshold used was derived from shuffled data, by shuffling the stimulus labels for each laminar compartment (1000 shuffles) and finding the lowest value for which <1% of shuffles resulted in a defined latency. Since these values depend on sample size, the highest value across compartments was used as threshold, so that the different functions could be timed using the same threshold. Confidence intervals (95%) were calculated on these latencies using a bootstrap approach with the bias corrected and accelerated percentile method (MATLAB function bootci, 2000 bootstraps). The latencies were statistically compared between laminar compartments using a bootstrap approach similar to other studies (Self et al., 2019), by computing the difference in latency between bootstrap samples of different compartments and estimating p as the fraction of samples on which the difference was ≤0 (one-sided test).

Spike response functions to small rings in the cRF (Figure 3) were computed similarly as the response functions to border ownership stimuli (Figure 2A–D). Responses were normalized by dividing them by the peak response for each unit. Latency was defined using a threshold a third of the way from baseline to peak (0.435) using the highest baseline across compartments, and confidence intervals and statistical tests were computed in the same way as for the BOI functions.

We also evaluated the time course of border ownership selectivity using the metric border ownership reliability (BOR; Zhou et al., 2000), which reflects the trial-to-trial reliability of encoding border ownership. This metric was computed for the border ownership-selective units in each laminar compartment, in 100 ms sliding windows (1 ms steps). For each unit, 10,000 sets of four spike trains were generated, where each set contained one random spike train from each of the four conditions in Figure 1A and C. For each window position, spikes were counted for each spike train in each set. BOR for a unit at a particular window position was defined as

where j corresponds to the index of all spike train sets for the unit. A(j) and B(j) indicate whether the sign of the spike count difference between border ownership conditions for spike train set j is, respectively, the same or opposite compared to the unit’s preferred side of border ownership:

where C_j,i represents the window spike count for condition i in spike train set j, R_i is the average spike rate for condition i (for the interval [50 500] ms after stimulus onset), and sgn is the sign function. For each window position for each unit, BOR was only computed if there were at least 10 spikes across conditions. For every window position the mean was calculated across units per laminar compartment, resulting in the BOR functions shown in Figure 2F. The abscissa in Figure 2F corresponds to the position of the right edge of the window, that is it indicates the latest spike times that may have determined BOR for that window position (and thus represents a conservative estimate for the latency). Definition of latency, computation of confidence intervals, and statistical tests used to compare latencies between layers were similar to those for the BOI functions.

For a given penetration, for each orientation, the border ownership-selective data set with the highest |BOI| was selected for each unit. Then from this group of data sets the largest subgroup that shared the same edge orientation and position was retained for analysis. Each unit could thus maximally contribute one data set. The preferred side of border ownership was then determined for all these data sets (example penetrations are shown in Figure 4C). The proportion of units sharing the most common preferred side was calculated for each penetration. The average of these proportions across penetrations (P_pref) was compared with a null distribution generated by randomly assigning the preferred side of border ownership for each data set (2000 randomizations; that is, a binomial process with chance of success = 0.5). The p value was estimated as the fraction of the null distribution for which P_pref was at least as large as the actual data.

Orientation selectivity was evaluated using a Kruskal–Wallis test (MATLAB function kruskalwallis) on the evoked spike counts in the analysis window (between 30 ms and 200 after stimulus onset) from the orientation tuning data set, using orientation as the grouping variable (Pettine et al., 2019), and defined as p < 0.05. All units for which the center of the contrast edge in the orientation data set was positioned in the cRF, and the z-scored spike rate for the orientation with the highest rate was ≥3 were included. Z-scores were calculated by first subtracting the mean of and then dividing by the standard deviation of the spike rate in a window preceding the first stimulus in the trial, equal in duration to the analysis window. For each orientation-selective unit i, the response to each orientation j is summarized as ${\overrightarrow{R}}_j$ , which has a direction 2j (because orientation has a period of 180°) and magnitude equal to the spike rate. The resultant ${\overrightarrow{O}}_i$ (vectors shown as solid lines in Figure 5) for all ${\overrightarrow{R}}_j$ is then calculated. The magnitude of orientation selectivity and the preferred orientation of unit i were defined, respectively, as the magnitude and as the direction divided by 2 (because of the definition of direction of ${\overrightarrow{R}}_j$) of ${\overrightarrow{O}}_i$ .

For each penetration, the aggregate preferred orientation was defined as the direction divided by 2 of the resultant vector $\overrightarrow{O}$ of all ${\overrightarrow{O}}_i$ on that penetration (direction of $\overrightarrow{O}$ is indicated by blue dashed lines in Figure 5). Statistical significance of the aggregate preferred orientation was assessed by randomizing the directions of ${\overrightarrow{O}}_i$ (2000 randomizations), calculating the null distribution ${\overrightarrow{O}}_{shuffled}$ , and estimating the p value as the fraction of ${\overrightarrow{O}}_{shuffled}$ for which the magnitude was at least as large as that of $\overrightarrow{O}$ .

The independence between the fractions of, respectively, orientation-selective and border ownership-selective units (Table 1) was assessed using a Chi-square test (MATLAB function crosstab).

For units that were border ownership-selective to multiple orientations, the circular mean of border ownership-selective orientations (ordinate in Figure 6C) was calculated as the direction divided by 2 of the resultant vector of all vectors ${\overrightarrow{B}}_j$ that have a direction 2 j and a magnitude equal to |BOI|, for all border orientations j. The shortest distance of the data points in Figure 6C to the identity line (Figure 6D) was analyzed by comparing the mean to a null distribution generated by shuffling the values for the preferred edge orientation and the circular mean of border ownership-selective orientations, and calculating the mean of the shortest distance to the identity line (2000 shuffles). The p value was estimated as the fraction of the null distribution that was as small or smaller than the observed value.

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

This research was supported by NEI core grant for vision research P30-EY019005 to The Salk Institute for Biological Studies. We thank Dr. Edward Callaway, Dr. Anirvan Nandy, and Dr. Zachary Davis for helpful discussions. We thank Dr. Mathias LeBlanc, Dr. Sean Adams, and Ms. Catherine Williams for excellent animal care.

This study was performed in accordance with the recommendations in the Guide for the Care and Use of Laboratory Animals of the National Institutes of Health. All procedures were approved by the Institutional Animal Care and Use Committee of the Salk Institute for Biological Studies (protocol 14-00014).

© 2021, Franken and Reynolds

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.