The primate brain contains distinct areas densely populated by face-selective neurons. One of these, face-patch ML, contains neurons selective for contrast relationships between face parts. Such contrast-relationships can serve as powerful heuristics for face detection. However, it is unknown whether neurons with such selectivity actually support face-detection behavior. Here, we devised a naturalistic face-detection task and combined it with fMRI-guided pharmacological inactivation of ML to test whether ML is of critical importance for real-world face detection. We found that inactivation of ML impairs face detection. The effect was anatomically specific, as inactivation of areas outside ML did not affect face detection, and it was categorically specific, as inactivation of ML impaired face detection while sparing body and object detection. These results establish that ML function is crucial for detection of faces in natural scenes, performing a critical first step on which other face processing operations can build.https://doi.org/10.7554/eLife.18558.001
For primates, faces are among the most behaviorally salient and socially relevant visual stimuli (Leopold and Rhodes, 2010). Reflecting this importance , the primate brain contains multiple interconnected cortical areas specialized for face processing, the face-patch network (Tsao et al., 2006, 2008). The middle-lateral face patch (ML), one of the earliest nodes in the face processing hierarchy, consists of neurons highly selective for faces (Tsao et al., 2006). Half of ML neurons exhibit sensitivity for coarse contrast relationships between face regions such as vertical adjacency of a bright forehead region above a darker eye region (Ohayon et al., 2012). Such qualitative contrast signatures for faces have been shown to provide a computationally powerful means for face detection (Sinha, 2002; Viola and Jones, 2004; Gilad et al., 2009), the most fundamental computation for all subsequent face-processing (Tsao and Livingstone, 2008). The hierarchically early location of face patch ML and the functional specificity of ML neurons led us to hypothesize that ML neurons might be critical for detection of faces, but not objects, in natural scenes. Testing this hypothesis required development of a naturalistic and demanding face and object detection task, and monitoring of real-world face-detection performance during causal manipulation of face patch ML.
We first trained two macaques on a touchscreen-based object detection task. From a visual scene displayed on the touchscreen, subjects were required to select an object belonging to one of three target categories – human faces, macaque bodies (socially relevant control category represented near face patches; Popivanov et al., 2012), or shoes (newly learned object category that, like faces, exhibits high within-class similarity). The visual scene was composed of 24 objects, at one of ten visibility levels. To discourage the use of low-level features for solving the task, the face targets were randomly resized, rotated in-plane, and blended into a cluttered background at random locations (Figure 1A; see Materials and methods). Face patch ML was localized by functional magnetic resonance imaging (fMRI) using a standard face localizer. To inactivate ML, we stereotaxically targeted its fMRI-identified coordinates and infused 5 μL (8 μL in one experiment) of muscimol (5 μg/μL), a GABAA agonist. We co-infused the MR contrast agent gadolinium (Gd), the cortical spread of which closely tracks the diffusion extent of muscimol (Heiss et al., 2010; Wilke et al., 2012), as a surrogate tracer so that the extent of muscimol spread could be detected in an anatomical MR scan (Figure 1B). Overlap of muscimol diffusion with the functionally defined extent of ML was determined post-hoc by aligning the post-injection anatomical volume and previous functional scan data (see Materials and methods; Figure 1B–D).
The goal of the injections was to inactivate as much of ML as possible, with as little inactivation of outside regions as possible – with the converse goal for the off-target injections. To determine how well these goals were met, we computed the percentage overlap of injections with ML and the fraction of ML volume overlapping with the injection (Figure 1D). First, we determined the average volume of ML exceeding a threshold of t > 20 as ~58 mm3, corresponding to a ~5 mm-diameter sphere, consistent with published neurophysiological data (Aparicio et al., 2016). The average volume of the injection at a threshold of 50% peak Gd brightness was ~37 mm3, corresponding to a ~4 mm-diameter sphere. At the chosen thresholds, the theoretical maximum of volumetric ML coverage by the injection, assuming concentric spheres for both volumes, is 63.8%. We achieved 44% coverage on average. Of the total injection volume, 76% was contained within the ML volume (Figure 1D). The high overlap of target injections with ML suggests that we silenced a large fraction of highly face-selective neurons, primarily found in the center of the face patches (Aparicio et al., 2016). The corresponding numbers for off-target injections were 4% ML coverage with 10% of the injection contained within ML. It is thus unlikely that off-target injections, although overlapping slightly with ML, silenced many face-selective neurons. The subject was transferred to its home cage to perform the detection tasks between ~1 and 3 hr after muscimol injection and subsequent anatomical scan, well within the time period of maximal muscimol effects (~0.5 hr to ~4 hr) (Hikosaka and Wurtz, 1985; Lomber, 1999; Dias and Segraves, 1999; Sommer and Wurtz, 2004). In a typical experimental sequence, we gathered baseline performance data on Day 1, inactivation data on Day 2, and recovery data on Day 3 or 4.
We first compared average face-detection performance across all 11 inactivation sessions (two monkeys, three hemispheres) and their corresponding baseline sessions. Face-detection performance was reduced by ~11% during ML inactivation (pooled over all visibilities and sessions, p<0.001, paired t-test after logit transform). Separated by visibility level, these performance decrements were significant for 9 out of 10 visibility levels at p<0.05 (FDR-corrected paired t-tests after logit transform; Figure 2A). A comparison of single session performances during baseline and inactivation showed this performance reduction to be uniform across visibility levels (Figure 2B): it was well-described by a line (R2 = 0.74; p<0.001) parallel to the diagonal (slope = 0.96) with a significant negative offset (p<0.001), providing a second, visibility-independent, single-session level estimate of the performance deficit of ~15%. In Figure 2C and D, we plot the same quantities as above separated into each hemisphere and animal to demonstrate that the reduction in performance was consistent between subjects and inactivated hemispheres. At the single-hemisphere level, we observed significant differences mainly at intermediate visibility levels (p<0.05; FDR-corrected paired t-test after logit transform). Thus, inactivation of 1 out of 12 temporal lobe face patches (Tsao et al., 2008) caused a significant and sizable reduction in face-detection performance.
To determine which behavioral parameters were affected by inactivation, psychometric functions (Equation 1 in Materials and methods; Wichmann and Hill, 2001) with three parameters – shift along the abscissa (α), slope (β), and lapse rate (λ, the difference between maximum and perfect performance) - were then fit to each session’s performance data (R2 > 0.9 for face detection in all sessions). Shift (α) and lapse rate (λ) were significantly higher in the inactivated conditions (p<0.001 and p = 0.016 respectively; paired t-test) than during baseline (red disks in Figure 2E; Figure 2F). In other words, during inactivation, the psychometric function was shifted rightwards and asymptotically lowered. The slope of the function (β) remained unaltered by inactivation. As a result, the stimulus visibility required for 50% performance (the average face-detection threshold) increased from 0.37 ± 0.02 at baseline to 0.45 ± 0.03 during the inactivation conditions.
We next tested whether the behavioral impairment following inactivation was specific to faces. Figure 3A shows the cumulative (summed across sessions) performance curves for face (left), body (middle), and shoe (right) detection. ML inactivation caused a shift of the psychometric function for face detection, but neither body nor shoe detection were significantly impaired. Effects were specific to the injection both in time and space. In recovery sessions when the effect of muscimol was expected to have subsided, behavioral performance returned to baseline levels across all target categories (Figure 3B; green), and fit parameters for performance in the recovery condition were not different from baseline for faces (Figure 2E; Figure 2F; green) or the other target categories (not shown). The transience of this effect demonstrates that muscimol was a causative agent. We also injected muscimol into cortex or white matter near ML (off-target injection, see Materials and methods). The off-target injection volumes barely overlapped with ML (9.6%; Figure 1D) or with any other face patch, and consequently, we observed only a weak and non-significant reduction in detection performance (Figure 2E; Figure 2F; Figure 3B; blue).
We then compared the grand average performance deficits (across all stimulus visibilities and sessions) and threshold shifts (across all sessions) for all category/condition combinations (Figure 3C and D). We used an ANCOVA with each target category and experimental condition as an independent group (seven groups) and muscimol exposure session number as a covariate, with all fits constrained to have equal slopes on logit-transformed data to evaluate statistical significance. Only the face-inactivated condition showed significant performance deficits (F(6, 752)=17.89; p<0.0001) and threshold shifts (ANOVA; F(6, 65)=5.74; p<0.0001) that were pairwise significantly different from all of the other category/condition combinations including the body- and shoe-inactivated conditions (*p<0.05; Tukey's test). The mean face-detection performance deficit was not significantly correlated with the ML-injection overlap (Spearman’s Rho rs = 0.42; p=0.1). An ANCOVA analysis with session number as independent groups and ML-injection overlap as a covariate did not reveal a significant effect of ML-injection overlap on the observed face-detection deficit (F(1,102) = 1.35; p=0.25). These data suggest that although there are differences in the extent of ML-injection overlap between sessions, as the central portion of ML was targeted by the injection, performance deficits were largely unaffected by these small differences in injection volume and overlap. Using an ANOVA (on logit-transformed data) that did not account for the effect of repeated muscimol exposure, we observed two additional pairwise comparisons that showed cross-category significance (light gray lines in Figure 3C; **p<0.01; Tukey’s test), but which did not reach significance for the appropriate within-category comparisons. Thus ML inactivation is largely category-selective for faces, and anatomically specific for ML compared with its immediate surroundings.
Although ML inactivation did not cause statistically significant performance deficits or threshold shifts for the other object categories, it is possible that the data might contain a non-specific component, as indicated by the non-zero values of performance deficits and threshold shifts for body parts and shoes (Figure 3C and D). To gain more statistical power for testing for a non-specific effect, we repeated the ANCOVA analysis that yielded the statistically significant performance deficit in Figure 3C, but with the ‘body’ and ‘shoe’ categories grouped into a single ‘non-face’ category. This procedure also failed to yield statistically significant differences between ‘non-face’ control and inactivated conditions. Similarly, grouping the ‘body’ and ‘shoe’ categories in the ANOVA analysis of threshold failed to yield a significant difference in threshold. If a weak non-specific effect truly existed, we could not attribute this to the motivation level of the animal post injection and MRI scan, as there were no significant differences between the total number of trials the animal initiated on control or inactivation days (paired t-test, p=0.47). Alternately, although 76% of the injection volume was contained within ML, it is possible that some muscimol diffused out of ML and into neighboring body patches (Popivanov et al., 2012) in some cases, causing a smaller, non-significant effect on that object category.
Our behavioral paradigm was designed to emphasize natural face detection, thus allowing free viewing. The paradigm therefore encouraged the use of behavioral strategies that made maximal use of face-processing resources in the unaffected hemisphere. Compatible with the idea that such strategies might have been learned and used, we found muscimol-induced performance deficits to decrease with repeated muscimol exposure. This accounted for some of the variability seen in Figure 2B, and could be quantified using the ANCOVA analysis described above. We observed a highly significant effect of session number on the performance deficit magnitude (F(1,752)=15.33; p<0.0001) with a positive slope of 0.01 ± 0.002, that is, the performance deficit caused by muscimol decreased by ~1% each session (relative to baseline performance level; Figure 3E). It is also possible that over the course of the inactivation experiments, animals gradually learned to use stimulus features represented outside of ML such as low-level features of faces, despite our efforts to prevent this through scaling and in-plane rotation of targets and varying the texture backgrounds. When we performed the ANCOVA with subject identity as a covariate, we did not observe a significant effect, demonstrating that the gradual reduction of the effect of muscimol and the variability in the data (Figure 2B) did not arise from pooling data from the two subjects.
To illustrate the putative perceptual impact of ML inactivation on face detection, we derived equivalent inactivated visibility values from the psychometric curves (Figure 4A; similar to Carrasco et al., 2004). At a high visibility of 0.85 (gray dashed line, marked '1' in Figure 4A), performance in the inactivated condition was ~75% correct, and the same performance level was achieved in the baseline condition at a visibility of 0.49 (red dashed line, marked '1' in Figure 4A). Therefore, when stimulus visibility was 0.85 in the inactivated condition, the subject was behaving as if the visibility was 0.49, and so forth (Figure 4B). As expected from the increased detection threshold, equivalent inactivated visibilities for faces were significantly lower than baseline visibilities (p<0.01; paired t-test). In Figure 4C, we plot example face stimuli for two visibility pairs (filled circles in Figure 4B). Thus, the induced shift in the psychometric curve is potentially reflected as a perceptually profound reduction of equivalent face visibility.
The free-viewing nature of the task exploits natural visual search behavior to demonstrate the causal dependence of this behavior on face-patch activity. This design is similar to one used in many human visual search and object recognition studies, but, because reliable methods for eye-tracking in freely moving primates are not available, we did not track the eye position of subjects as they performed the task. Recent evidence suggests that object recognition performance in monkeys does not depend on response modality – performance measured using saccades with eye-tracking, or touch with no gaze tracking, was similar to human object recognition performance measured without eye-tracking (Rajalingham et al., 2015). Therefore, we do not expect the lack of eye-tracking to affect the main results of our study.
Lack of eye-tracking, however, introduces two main sources of variability into our data. First, we cannot clearly assess the spatial effects of unilateral inactivation. Because we inactivated ML unilaterally in our experiments, we attempted to address whether performance deficits were also lateralized by analyzing recognition performance when the target image was located in three vertically oriented bands – ipsilateral to injection site, contralateral, and in the center of the screen. This analysis reduced our overall statistical power and did not yield significant results. Previous experiments in head-fixed preparations have demonstrated that local inactivation of smaller cortical regions can cause face-processing deficits specific to the visual field they represent (e.g., Afraz et al., 2015). As face-patch ML shows a foveal bias (Janssens et al., 2014), and because the center of ML contains neurons with the strongest face selectivity (Aparicio et al., 2016), by targeting our injections to the center of ML, we may have interfered with face detection in a large foveal area that would force the subject to use alternate and slower face-detection strategies. Thus performance deficits may have been higher at target locations that correspond to retinotopic locations of the injection, and effect sizes reported here should be taken as lower bound estimates.
Second, the lack of eye-tracking limits our ability to quantify behavioral strategies that may have been used by the subject to solve the task. In particular, it adds variability to the psychometric curve and affects our estimate of the ‘chance level’ of the task because the subject may have looked away from the screen after initiating a trial. We designed our task to minimize this variability by: (1) requiring the animals to voluntarily initiate trials; (2) requiring subjects to direct their gaze to the center of the screen and perform a deliberate and accurate movement to initiate the trial; (3) requiring subjects to pay attention to the color of the central cue to determine target block identity; (4) requiring subjects to complete the task in a short time span; and (5) randomizing visibility levels within blocks and interleaving block types to prevent systematic biases. Through post-hoc analysis we determined that: (1) the animal performed similar numbers of trials on control and inactivation days suggesting no change in motivation level; (2) animals performed at a high rate on high-visibility trials on all days, suggesting no distraction at the time scale of trials. The experimental configuration ensured that the screen covered a large portion of the animal’s visual field, minimizing space available for distractions. Therefore, we believe that we have minimized the effects of extraneous sources on behavior. Additional sources of variability may remain, however, such as variations in eccentricity of the target stimulus as the subject freely views the display.
Direct cortical stimulation and suppression have been used successfully to alter face perception in humans and macaques (Allison et al., 1994; Parvizi et al., 2012; Chong et al., 2013; Jonas et al., 2014; Afraz et al., 2006; Roy et al., 2014; Afraz et al., 2015). We provide four critical advances over these past experiments: (1) We directly visualized and limited the spread of muscimol to the functionally defined coordinates of an fMRI-identified face area. Comparing on- and off-target injections, we could thus establish anatomical specificity of inactivation. (2) We used several interleaved object categories as behavioral targets. We could thus establish categorical specificity of ML inactivation. (3) We focused on face detection, the most fundamental and robust component of face processing (Sinha, 2002; Viola and Jones, 2004; Tsao and Livingstone, 2008; Gilad et al., 2009) and selected the target site of inactivation based on the known physiology of neurons (Ohayon et al., 2012). (4) We used a behavioral paradigm that was naturalistic, yet allowed for quantification of perceptual thresholds. Thus we could provide first causal evidence for the role of ML in real-life face detection. This is despite only unilaterally inactivating 1 out of 12 nodes (Tsao et al., 2006, 2008) and allowing free viewing. Receptive fields in posterior IT, the region in which ML is located, are spatially specific (Kobatake and Tanaka, 1994), and thus the two hemispheres cover different parts of the visual field (Rajimehr et al., 2014). Therefore, unilateral inactivation is expected to affect only part of the visual field (Afraz et al., 2015). That significant impairments occur during free viewing emphasizes the causal role of ML in everyday face detection. Effects were specific to faces, with little or no effects on one object category, bodies, that is represented in the direct vicinity of ML (Popivanov et al., 2012), and another one, shoes, a newly learned object category with exemplars, as in faces, exhibiting high within-class similarity. Our results thus extend our understanding of ML as a face-specific processing module containing a large number of cells tuned to facial features that are known to be useful for face detection in an important way: highlighting its causal role for real-world face-detection behavior.
All procedures conformed to the NIH Guide for Care and Use of Laboratory Animals of the National Institutes of Health. All animals were handled and all procedures and experiments conducted according to approved institutional animal care and use committee (IACUC) protocols at Weill Cornell Medicine (protocol number 2010–0029) and The Rockefeller University (protocol numbers 12585-H and 15849-H). All surgery was performed under isoflurane and a 70% nitrous oxide and 30% oxygen mixture. Every effort was made to minimize suffering.
We studied one male rhesus (Subject O) and one male cynomolgus (Subject I) macaque, 3.5–5 years in age and weighing between 4.5 and 9 kgs. An acrylic cement-based cranial implant (C&B Metabond, Parkell; Palacos LV+G, Zimmer) was anchored to the skull with ceramic screws (Thomas Recording or Rogue Research) using sterile surgical methods under standard anesthetic and postoperative protocols. A custom MRI-compatible headpost made of Ultem (SABIC) was then secured in each implant.
We acquired all MRI data on a 3T Siemens Tim Trio MRI scanner and used an AC88 gradient insert (Siemens) for functional scans. A dose of Feraheme (AMAG Pharmaceuticals) containing 8-10 mg/kg of iron was injected into a saphenous vein before each functional scan to increase functional contrast. Functional images were obtained with a custom eight-channel surface coil (Lawrence Wald, MGH Martinos Center) and a gradient-echo echo-planar imaging sequence with 54 horizontal slices, a 96 × 96 in-plane matrix, an isotropic resolution of 1 mm3, TR = 2 s, TE = 16 ms, and 2x GRAPPA acceleration. For anatomical registration and cortical surface reconstruction, we obtained images using a custom single-channel surface coil and a T1-weighted magnetization-prepared rapid gradient echo sequence (MPRAGE) with 240 sagittal slices, an in-plane matrix of 256 × 256, and isotropic resolution of 0.5 mm3 (averaged over four to six scans) under isoflurane anesthesia. After each muscimol injection, we acquired T1-weighted images using a fast MPRAGE sequence with 192 slices, an in-plane matrix of 256 × 256, and isotropic resolution of 0.5 mm3 (one or two repetitions).
Stimuli for fMRI imaging were projected on a screen placed ~35 cm in front of the subject. Stimuli consisted of grayscale pictures of rhesus macaque faces, cynomolgus macaque faces, fruits and vegetables, monkey bodies, manmade objects, and phase-scrambled versions of the fruit and vegetable pictures, presented in separate 24 s blocks. Each block of this set consisted of 15 exemplar images shown for 0.4 s, with each exemplar repeated four times in a block. Images were matched within and across all categories for total screen area, placed on a pink noise background and normalized for luminance and frequency amplitude using the SHINE toolbox (http://www.mapageweb.umontreal.ca/gosselif/shine; Willenbockel et al., 2010). The pink noise background subtended 15° visual angle, and the average foreground image subtended ~6° visual angle. A Latin square design was used to create six orders of the stimulus blocks, and the relative position of each block within a run and the identity of the immediately preceding block were balanced.
We used standard, previously published analyses to localize the face patches (Tsao et al., 2006). Briefly, we used standard preprocessing methods to correct for artifacts common to imaging head-fixed, awake macaques. We corrected for motion in three dimensions, shifts, rotations, and warping, and aligned the functional images to the high-resolution anatomical images. We used a sign-reversed hemodynamic response function to account for MION increasing contrast by decreasing the local MR signal. A general linear model was then fit, which included predictors for each stimulus type, as well as nuisance predictors for the motion parameters. To map the face patches, we used the contrast [(rhesus faces + cynomolgus faces) > all other object categories]. We constructed a model of each subject's cortical surface from the high-resolution anatomical scans using FreeSurfer (Fischl, 2012). The statistical significance values of the differential signal changes from the functional scans were projected onto the high-resolution anatomical volume and surface model for visualization.
We trained subjects on a touchscreen-based object detection task. Stimuli were presented using MATLAB software (Kofiko, provided by Dr. Shay Ohayon, MIT; https://github.com/shayo/Kofiko) on an Elo IntelliTouch 1537L 15-inch LCD touchscreen (Tyco Electronics) mounted to the subject's cage. The subject initiated the trial by touching a central disc (4.3 cm dia.), the color of which indicated the target category (red-faces; blue-body parts; green-shoes). On initiating the trial, subjects were presented with a stimulus array consisting of 24 objects, one of which belonged to the target category, presented at 36 possible random locations, balanced to ensure that each quadrant had an equal number of stimuli. The objects were alpha-blended into a cluttered background, based on a set of visual textures (Portilla and Simoncelli, 2000) pre-generated using exemplar stimuli from the three target categories. For each trial, a random 600 × 600 region of one of 32 pre-generated textures, at one of four possible rotations, was used as the background. The texture occupied a 26 × 26 cm area of the touchscreen. Subjects were free to choose viewing distance, but typically positioned themselves to minimize movement required to reach the centrally placed reward spout, resulting in a viewing distance of ~20–25 cm. At that viewing distance, the full extent of the stimulus array subtended a visual angle of ~54°–64°. Individual stimuli varied in size and subtended ~4°–7° of visual angle. Subjects had 3 s to choose the correct target object, upon which they were rewarded with a small amount of juice or water. Wrong choices, but not time-outs, resulted in an increased inter-trial delay of 0.5–1.5 s. Sessions were organized into interleaving blocks for each target category (faces, bodies, or shoes), with each block consisting of 10 stimuli at the 10 visibility levels in random order. Subjects were trained until they performed ~800–1000 trials per 2 hr session, and attained at least 75% face-detection performance at the highest visibility level.
Grayscale images of human faces were constructed from a multiracial face database (http://wiki.cnbc.cmu.edu/Face_Place, Michael Tarr). We chose 32 faces with neutral expression, uniform lighting, and viewing angle ranging from half profile to frontal views. Images included hair and in some cases, facial hair. On each face trial, we further applied a random scaling factor (0.5x – 1.5x) and random in-plane rotation (±45 deg.). The goal of these manipulations was to minimize the consistent spatial occurrence of low-level features and ensure that face stimuli possessed highly variable outlines. We used an initial set of 16 frontal face stimuli to train the subjects to criterion, and then switched to this new set just before commencing the inactivation experiments. No performance decrease was triggered by this switch, suggesting that the subjects had learned the stimulus category, and not the individual exemplars. For bodies, we used a set of 16 images consisting of various views of macaque torsos and limbs. Shoe images were downloaded from various online sources. We chose these categories to control for two effects – bodies controlled for the ethological relevance of the stimuli, and shoes controlled for the global shape similarity of objects. The target stimuli did not overlap with the stimuli used for mapping the face patches. Distractor images were drawn from a set of 178 images of airplanes, cars, chairs, flowers, fruits, manmade objects, motorbikes, familiar animal toys, technological objects, Fribbles, and Yadgits (Barry et al., 2014). The entire stimulus set was normalized for luminance and frequency amplitude using the SHINE toolbox. All stimuli were alpha-blended into the cluttered background using a transparency mask with feathered edges to achieve smooth blending. We randomized the location of the objects to one of 36 equally spaced locations on the background for every trial, and ensured that: (1) objects did not overlap, and (2) equal numbers of objects were present in each quadrant to maintain a uniform extent and density of visual search.
The behavioral task used was the result of theoretical considerations and refinement from pilot behavioral experiments. The visual texture background was introduced to both disrupt the normal coarse contrast relationships in faces, and to introduce face-like coarse contrast relationships at non-target locations. We reasoned that by doing so, pooling over a large number of ML neurons might become necessary for successful task performance. In pilot experiments without a background or with pixel-noise background, animals detected faces rapidly (<1 s) and at high performance levels for all stimulus visibility levels tested. Imposing the texture background resulted in a more typically shaped psychometric curve. Even with a textured background, when a time constraint was not imposed on behavioral response, subjects could perform the face-detection task at near-perfect levels at moderate to high levels of stimulus visibility. However, we noticed that the response time distributions were multi-peaked and had a heavy tail, indicating that the subjects may perform several ~1.5 s-long scans of the array until the target was detected. We imposed a hard time constraint of 3 s, giving subjects time for about two scans of the array. Trials that would have taken longer response times (had the time constraint not been imposed) were thus converted to timeouts or misses attributable to forced guessing, allowing us to obtain a psychometric function that better reflects rapid face detection in scenes.
After functional mapping, MR-compatible chambers were implanted over the functionally identified location of ML under anesthesia, and a craniotomy (~6 mm dia.) was performed within the chamber under sedation just before the inactivation experiments. We injected 5 μL (8 μL in one experiment) of a cocktail of 5 μg/μL muscimol (Sigma Aldrich) and 5 mM gadolinium (Magnevist) in phosphate-buffered saline. Injections were performed through a 32-gauge metal cannula (Hamilton) attached to a precision microinfusion pump (World Precision Instruments) using non-compressible fused silica tubing (Polymicro) and PEEK connectors (Labsmith), adapted from an earlier design (Noudoost and Moore, 2011). The cannula was introduced into the cortex via a custom 3D-printed grid, using MATLAB-based planning software (Ohayon and Tsao, 2012) to select the appropriate grid hole. We typically injected in 0.2 μL steps, at a rate of 0.01 μL/s, with 20–30 s between steps (average rate of 0.4 μL /min), spread over three cortical depths spaced 500 μm apart (1.6–1.8 μL per depth, deepest first). We waited 3–5 min after each set of injections before moving the cannula to a different depth or withdrawing the cannula from the cortex. The entire procedure lasted ~25 min. Immediately after the injections, the subject was transferred to the MRI scanner to obtain anatomical scans, after which the subject was transferred to its home cage where the touchscreen was already mounted. Subjects thus performed the task between ~1 and 3 hr from the end of the injection procedure. This window was well within the time period of maximal muscimol effects (~0.5 hr to ~4 hr) (Hikosaka and Wurtz, 1985; Lomber, 1999; Dias and Segraves, 1999; Sommer and Wurtz, 2004). Off-target sites included locations dorsal to the STS (n = 3; one into STS and two in gray matter 3.0 mm dorsal to STS), deeper locations comprising the white matter beneath ML (n = 3; depths 4–5.5 mm from cortical surface), and locations posterior to ML (n = 4; 2.9–3.6 mm from the center of ML at t > 20). The first sub-group of off-target injections was intended to function as sham injections, as the animal undergoes identical experimental procedures without inactivation of visual areas. We used off-target shams rather than injecting saline into ML to minimize risks of causing tissue damage and to preserve the face patches for future electrophysiological studies. We performed 11 on-target injections and 10 off-target injections in three hemispheres. To evaluate overlap between the muscimol injection and the functionally defined bounds of ML, we used AFNI to construct a binary mask volume that isolated t-values greater than 20 in the [(rhesus faces + cynomolgus faces) > all other objects] contrast. We resampled this mask volume to the resolution of the anatomical volume, and manually selected voxels corresponding to ML. In the anatomical volume, we first constructed a manual mask that included the area around the injection site. Then, we determined the maximum value of Gd brightness within this mask, and constructed a second binary mask that included all values greater than 50% of the maximum value (full-width at half maximum of the injection site). We then used AFNI to compute the volume of the injection (in voxels), and the volumetric overlap between the injection and ML (at t > 20). The injection overlap was defined as the volume of overlap divided by the volume of the injection. ML overlap was defined as the volume of overlap divided by the volume of ML.
Subjects were required to select, within 3 s, an object belonging to the target category from an array of 24 objects, blended into a cluttered background at one of 10 visibility levels (Figure 1A,B). Subjects had been trained on three target categories: human faces, macaque bodies, or shoes. Each trial had three possible outcomes – target object selected, distractor object selected, or a timeout (3s elapsed without a touch). For all analyses, we focused on the proportion of correct trials. We first compared the average performance difference between each experimental condition (inactivated, recovery or off-target) and the paired baseline condition for each category, using FDR-corrected paired t-tests to evaluate statistical significance. We then fit psychometric functions to the data from individual sessions in each experimental condition for each stimulus category. These functions were of the form (Wichmann and Hill, 2001):
where F is the Weibull function, defined as , α is the shift parameter, β is the slope parameter, λ is the lapse rate, and γ = 1/24 = 0.0417 is the chance performance level (assuming that the animal initiates trials with the intention of finishing them). A logit transform was applied to all proportional data (such as percent correct) before applying parametric statistical tests. Paired t-tests were used to evaluate whether fit parameters were different for each experimental condition from baseline. Thresholds were derived from these psychometric fits. We used ANOVAs with post-hoc Tukey's tests to compare performance deficits across experimental conditions and stimulus categories. In preliminary analyses, we noticed that performance deficits decreased as we exposed the subject to an increasing number of inactivation sessions. To control for this effect, we also performed an ANCOVA to compare effect sizes between the various groups after accounting for session number, which was modeled as a covariate. All analyses were performed in MATLAB (MathWorks, Natick, MA). We derived equivalent inactivated visibilities from the fit curves, and plotted example face stimuli blended into the background texture at the resulting alpha levels. We caution that these example stimuli render differently in print and on different monitor types (LCD or CRT).
Neurophysiological organization of the middle face patch in macaque inferior temporal cortexThe Journal of Neuroscience 36:12729–12745.https://doi.org/10.1523/JNEUROSCI.0237-16.2016
Meet the fribbles: novel stimuli for use within behavioural researchFrontiers in Psychology 5:103.https://doi.org/10.3389/fpsyg.2014.00103
Image-guided convection-enhanced delivery of muscimol to the primate brainJournal of Neurosurgery 112:790–795.https://doi.org/10.3171/2009.7.JNS09652
The advantages and limitations of permanent or reversible deactivation techniques in the assessment of neural functionJournal of Neuroscience Methods 86:109–117.https://doi.org/10.1016/S0165-0270(98)00160-5
A reliable microinjectrode system for use in behaving monkeysJournal of Neuroscience Methods 194:218–223.https://doi.org/10.1016/j.jneumeth.2010.10.009
Electrical stimulation of human fusiform face-selective regions distorts face perceptionJournal of Neuroscience 32:14915–14920.https://doi.org/10.1523/JNEUROSCI.2609-12.2012
A parametric texture model based on joint statistics of complex wavelet coefficientsInternational Journal of Computer Vision 40:49–70.https://doi.org/10.1023/A:1026553619983
Comparison of object recognition behavior in human and monkeyJournal of Neuroscience 35:12127–12136.https://doi.org/10.1523/JNEUROSCI.0573-15.2015
Retinotopy versus face selectivity in macaque visual cortexJournal of Cognitive Neuroscience 26:2691–2700.https://doi.org/10.1162/jocn_a_00672
Reversible inactivation of pSTS suppresses social gaze following in the macaque (Macaca mulatta)Social Cognitive and Affective Neuroscience 9:209–217.https://doi.org/10.1093/scan/nss123
Proceedings of the Annual Workshop on Biologically Motivated Computer Vision249–262, Qualitative representations for recognition, Proceedings of the Annual Workshop on Biologically Motivated Computer Vision.
What the brain stem tells the frontal cortex. II. role of the SC-MD-FEF pathway in corollary dischargeJournal of Neurophysiology 91:1403–1423.https://doi.org/10.1152/jn.00740.2003
Mechanisms of face perceptionAnnual Review of Neuroscience 31:411–437.https://doi.org/10.1146/annurev.neuro.30.051606.094238
Robust Real-Time face detectionInternational Journal of Computer Vision 57:137–154.https://doi.org/10.1023/B:VISI.0000013087.49260.fb
The psychometric function: I. fitting, sampling, and goodness of fitPerception & Psychophysics 63:1293–1313.https://doi.org/10.3758/BF03194544
Controlling low-level image properties: the SHINE toolboxBehavior Research Methods 42:671–684.https://doi.org/10.3758/BRM.42.3.671
Jack L GallantReviewing Editor; University of California, Berkeley, United States
In the interests of transparency, eLife includes the editorial decision letter and accompanying author responses. A lightly edited version of the letter sent to the authors after peer review is shown, indicating the most substantive concerns; minor comments are not usually included.
Thank you for submitting your article "A Causal Relationship Between Face-Patch Activity and Face-Detection Behavior" for consideration by eLife. Your article has been reviewed by two peer reviewers, and the evaluation has been overseen by a Reviewing Editor (who separately reviewed the paper) and by David Van Essen as the Senior Editor. The following individual involved in review of your submission has agreed to reveal his identity: Rufin Vogels (Reviewer #1).
The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.
The present study shows that inactivation of the macaque face selective area ML impairs the detection of faces in cluttered displays. The main finding of the study is that ML plays a causal role in face detection.
The reviewers found the work to be interesting and worthwhile, and potentially publishable in eLife after appropriate revision. However, they also had several concerns about methods and interpretation of the results that must be addressed before the paper can be considered for publication.
1) Most importantly, one reviewer and the reviewing editor were both concerned about the behavioral procedures used in the experiment, and the apparent lack of head or eye tracking during the experiment. The animals apparently freely viewed the stimulus during the 3 second response period, and no attempt was made to deal with variability in post-processing or analysis. In most vision studies the eyes are tracked closely to ensure that eye movement differences across conditions could not have accounted for the results, but apparently that was not done here. Could this have affected the results? If the eye movement data are available then the paper should be revised appropriately, incorporating those data. If not, then the authors should include a detailed rebuttal indicating why this is not a concern.
2) The behavioral data from individual hemispheres/animals should be shown so that readers can gain a better sense of how consistent the impairment is across hemispheres/animals. The overlap between the estimated injection site and the fMRI defined face patches for each individual hemisphere should also be shown.
3) Although the largest effects seem to occur for faces, inspection of Figure 3 suggests that inactivation affects detection of all three categories, although faces appear most affected. Thus, it is unclear whether the effect is truly limited to faces, or whether it a smaller impairment is also found in other categories. This issue should be addressed clearly in revision.
4) All reviewers had concerns about the location of the muscimol injections in both experimental and control conditions. This information needs to be provided in a very clear form, and the interpretation of the results of each specific injection should be made clear.
The present study has several shortcomings, however. Although I feel that these drawbacks do not affect their main result – an impairment of face detection when inactivating ML -, they make a quantitative assessment of the impairment impossible, which is a missed opportunity. First, their "naturalistic" task suffers from a major problem when one wants to quantify detection behavior: if the animal fails to respond to the display that does not mean that the animal did not detect the stimulus. The authors did not measure eye movements of the animals during the detection task and thus an unknown portion of "no go" responses may have resulted not from detection failures but simply from the animals not looking at the display or not performing the task. This causes problems when attempting to compute the proportion of correct trials: what to take as an "executed" trial?
Second, they made unilateral inactivations which will have affected mainly the contralateral visual field (see Afraz et al., 2015). Thus, whether the target stimulus was in the affected part of the visual field will depend on the fixation pattern, which may have varied from trial to trial. This will have added noise to the psychometric functions for the inactivated condition and may have underestimated the deficit to an unknown degree. The data show that ML activation has an effect, but the size of the effect cannot be estimated.
Third, the authors should show the behavioral data of the individual hemispheres/monkeys so that one can appreciate how consistent the impairment is across hemispheres/animals. Furthermore, they should show the overlap between the estimated injection site and the fMRI defined face patches for each individual hemisphere. Related to this, it appears from Figure 1D that in that monkey the injections were rather posterior and at best affected only part of ML. Is this true?
Figure 3C and D suggest an impairment after each muscimol injection, although the effect is greater for face detection after ML injection than for the other 3 conditions. Could this aspecific part of the effect result from the possibility that the animals did not work well after the injection and following the MRI scan? Control injections with saline – standard in these sort of experiments – could have answered this concern.
I am puzzled by the site of the control injections: dorsal to the STS (where? In gray matter? Non-visual!) and deeper locations in the white matter near ML (subsection “Muscimol injection”). Especially the latter is puzzling since fibers are not affected by muscimol. What was the purpose of this? Sham injections? It would have been more informative to inject in nonface-patch regions of IT.
Sadagopan and colleagues studied the behavioral effects that result from inactivation of a face selective area in the temporal lobe of monkeys. The results show that inactivating area ML results in an increased threshold for face detection. Importantly, this increased detection threshold seems to be specific for monkey faces and not other visual categories such as body parts or shoes. In addition, the authors performed muscimol injections in nearby sites that did not affect face detection, demonstrating that the effect is also site specific.
This is a carefully planned experiment that convincingly demonstrates the causal role that area ML has in the processing and detection of visual information related to faces. I have some comments that the authors could address to further improve the manuscript.
1) It is clear that ML inactivation has its strongest effect on the detection of faces. However, Figure 3A shows that ML inactivation also affects the detection of body and shoe categories. These non-selective effects did not reach statistical significance but I think that some readers will find hints of threshold increases for the shoe and body categories. I think no reasonable reader will expect 100% selectivity in any cortical inactivation study. Maybe the authors could make a more guarded interpretation in relation to category selectivity, or maybe they could comment on this.
2) Figure 3B. There seems to be no data in relation to performance obtained from off-target inactivation in the shoe and body categories. Why is this?
3) I wonder if the authors have data about the location and spread of the off-target injections. A schematic like the one in Figure 1D but for the off-target inactivation could help readers evaluate how off were the these injections from the ML area.
4) From the data on Figure 2C-D one can imagine that there were some off-target and recovery sessions that reached a statistical significant effect. Am I correct? Please mark them on those panels.
5) The analyses described in the sixth paragraph of the Results and Discussion seemed redundant to me. Yes, a displacement in detection threshold can be illustrated as a reduction in face visibility but I am not sure what it is gained by doing that.
6) The effects of muscimol decreased with each exposure. This is an interesting finding that I think is worth showing in a data figure. Please show the effect of muscimol as a function of experimental session.
[Editors' note: further revisions were requested prior to acceptance, as described below.]
Thank you for resubmitting your work entitled "A Causal Relationship Between Face-Patch Activity and Face-Detection Behavior" for further consideration at eLife. Your revised article has been favorably evaluated by David Van Essen (Senior editor), a Reviewing editor, and two reviewers.
The manuscript has been improved but there are some remaining issues that need to be addressed before a final decision can be made, as outlined below:
1) Please address the injection overlap issue identified by the first reviewer, and if possible provide a more quantitative relationship between inactivation and behavior.
Although the authors attempt to defend the use of a "naturalistic" free-viewing task to study the effect of inactivation of a face patch (located in a region with a rough retinotopy), I still feel that this study would have provided more information on the contribution of that face patch to face detection if they had use a well-controlled task (with at least proper eye movement control). Despite this shortcoming, the study shows clear face detection performance deficits, which is a valuable addition to the literature.
I have the following comment on the revised text, which needs to be addressed.
The injection overlap was defined as the volume of overlap (between ML and injection) divided by the injection volume. However, this implies that a small injection volume inside a larger ML volume will get a 100% overlap, while only part of ML was inactivated. They should also provide the proportion of overlap computed relative to the ML volume. This metric reflects how much of ML was inactivated, which is unknown now (especially given the "distorted" map shown in Figure 1), but is important for evaluation of the behavioral effects.
The authors have addressed my questions and concerns satisfactorily.
1) I think they now have a more balanced discussion on the selectivity of the muscimol inactivations.
2) They now provide data on the overlap between off-target injections and ML area for each hemisphere.
However, they do not discuss my query: "6) The effects of muscimol decreased with each exposure. This is an interesting finding that I think is worth showing in a data figure. Please show the effect of muscimol as a function of experimentalsession."
I think they missed pasting the response on the Response to Reviewers file because this effect is now shown and analyzed in the revised version of the manuscript (new panel on Figure 3E).
[Editors' note: further revisions were requested prior to acceptance, as described below.]
Thank you for resubmitting your work entitled "A Causal Relationship Between Face-Patch Activity and Face-Detection Behavior" for further consideration at eLife. Your revised article has been favorably evaluated by David Van Essen (Senior Editor), a Reviewing Editor, and two reviewers.
The reviewers were very happy with your revisions and we are ready to accept the manuscript. However, one reviewer requested a small technical change to correct some missing information. In the manuscript you currently report one degree of freedom for the ANCOVA, but there should be two degrees of freedom reported [e.g., in the following format F(df1,df2) = x]. Please provide the other degree of freedom here, or provide an explanation for why this is not required.
After we receive your revised submission we will be happy to publish this work.https://doi.org/10.7554/eLife.18558.006
- Winrich A Freiwald
- Winrich A Freiwald
- Winrich A Freiwald
- Wilbert Zarco
- Winrich A Freiwald
- Srivatsun Sadagopan
- Wilbert Zarco
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
We thank Shay Ohayon (Massachusetts Institute of Technology) for providing the behavioral control software environment, and the Planner software package for stereotactic targeting. We thank Akinori Ebihara, Rizwan Huq, Sofia Landi, Clark Fisher, Caspar Schwiedrzik, Stephen Shepherd, Pablo Polosecki, Ilaria Sani, Julia Sliwa and Margaret Fabiszak (The Rockefeller University) for assistance with imaging and training; Skye Rasmussen, Leslie Diaz, Alejandra Gonzales, Susan Hinklein, Jorge Fajardo, and Vadim Sherman (The Rockefeller University) for veterinary and technical support, Jon Dyke (Weill Cornell Medicine) for scanner support, and Lawrence L Wald for MRI coil design and support. We thank Melanie Wilke and Igor Kagan for advice on targeted muscimol injections, and Gaby Maimon and Stephen Shepherd for comments on the manuscript. We are grateful for the reviewers’ constructive input towards improving this manuscript. This manuscript is based upon work supported by the Center for Brains, Minds and Machines (CBMM), funded by NSF STC award CCF-1231216, funds from The Rockefeller University, and National Eye Institute grant R01 EY021594-01A1 to WAF. WAF is a New York Stem Cell Foundation-Robertson Investigator. SS was supported by a Leon Levy postdoctoral fellowship. WZ was supported by the Pew Fellows Program in Biomedical Sciences and a NIH-International Neuroscience Fellowship F05MH094113.
Animal experimentation: All procedures conformed to the NIH Guide for Care and Use of Laboratory Animals of the National Institutes of Health. All of the animals were handled and all procedures and experiments conducted according to the approved institutional animal care and use committee (IACUC) protocols at Weill Cornell Medicine (protocol number 2010-0029) and The Rockefeller University (protocol numbers 12585-H and 15849-H). All surgery was performed under isoflurane and in a 70% nitrous oxide and 30% oxygen mixture. Every effort was made to minimize suffering.
- Jack L Gallant, Reviewing Editor, University of California, Berkeley, United States
- Received: June 7, 2016
- Accepted: March 9, 2017
- Version of Record published: April 4, 2017 (version 1)
© 2017, Sadagopan et al.
This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.