Objects are typically seen within a rich, structured, and familiar context, such as cars on a road and chairs in a living room. Decades of behavioral work have shown that context facilitates the recognition of objects (Bar, 2004; Biederman et al., 1982; Oliva and Torralba, 2007). This contextual facilitation is crucial for everyday behavior, allowing us to recognize objects under poor viewing conditions (Figure 1), at a distance, in clutter, and in the periphery where visual resolution is low. Yet despite the pervasive influence of context on object recognition, our knowledge of the neural mechanisms of object recognition almost exclusively comes from studies in which participants view clearly visible isolated objects without context. These studies have shown that isolated object recognition results from the transformation of local, low-level features into view-invariant object representations along the ventral stream (DiCarlo et al., 2012; Liu et al., 2009; Riesenhuber and Poggio, 1999; Serre et al., 2007). Does a similar local-to-global hierarchy support context-based object recognition?

Figure 1

Download asset Open asset

One possibility is that context-based object recognition is supported by the parallel feedforward processing of local object information in the ventral stream object pathway and global scene processing in a separate scene pathway (Henderson and Hollingworth, 1999; Park et al., 2011). Output of these pathways may then be combined in downstream decision-making regions, leading to contextual benefits in object recognition. Alternatively, context-based object recognition may be supported by feedback processing, with scene context providing a prior that is integrated with ambiguous object representations in the visual cortex (Bar, 2004; Brandman and Peelen, 2017; de Lange et al., 2018). Neuroimaging studies have not been able to distinguish between these possibilities because the contextual modulation of neural activity in object-selective cortex (Brandman and Peelen, 2017; Faivre et al., 2019; Gronau et al., 2008; Rémy et al., 2014) could precede but also follow object recognition, for example reflecting post-recognition imagery (Dijkstra et al., 2018; Reddy et al., 2010).

To distinguish between these accounts, we used transcranial magnetic stimulation (TMS) to interfere with processing in right object-selective lateral occipital cortex (LOC; Grill-Spector, 2003; Malach et al., 1995) and right scene-selective occipital place area (OPA; Dilks et al., 2013; Grill-Spector, 2003) at three time points relative to stimulus onset. We additionally stimulated the early visual cortex (EVC) to investigate the causal contribution of feedback processing in this region during both isolated object recognition and context-based object recognition (Camprodon et al., 2010; Koivisto et al., 2011; Pascual-Leone and Walsh, 2001; Wokke et al., 2013). EVC stimulation was targeted around 2 cm above the inion, with the coil positioned such that TMS induced static phosphenes centrally in the visual field, where the stimuli were presented. This region corresponds primarily to V1 (Koivisto et al., 2010; Pascual-Leone and Walsh, 2001). The three regions were stimulated in separate pre-registered experiments (N=24 in each experiment; see Materials and methods).

Because TMS effects are variable across individuals, for example, due to individual differences in functional coordinates but also skull thickness and subject-specific gyral folding patterns (Opitz et al., 2013), we used a TMS-based assignment procedure to ensure the effectiveness of TMS over each of the three stimulated regions at the individual participant level (van Koningsbruggen et al., 2013). To achieve this, all 72 participants in the current study first underwent a separate TMS session in which the effectiveness of TMS over the three regions was established using object and scene recognition tasks (for the full procedure and results of this screening experiment, see Wischnewski and Peelen, 2021). Only participants who showed reduced scene recognition performance after OPA stimulation were assigned to the OPA experiment (N=24), only participants who showed reduced object recognition performance after LOC stimulation were assigned to the LOC experiment (N=24), and only participants who experienced TMS-induced phosphenes after EVC stimulation were assigned to the EVC experiment (N=24). All 72 participants satisfied at least one of these criteria such that no participants had to be excluded.

In all experiments, participants performed an unspeeded eight-alternative forced-choice object recognition task, indicating whether a briefly presented stimulus belonged to one of the eight categories (Figure 2). Participants performed this task for clearly visible isolated objects (isolated object recognition) as well as for degraded objects presented within a congruent scene context (context-based object recognition). In addition to the object recognition tasks, participants also performed a scene-alone task in which the object was cropped out and replaced with background. In this condition, participants had to guess the object category of the cropped-out object.

Figure 2

Download asset Open asset

Predictions (Figure 3a) were based on the findings of recent functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG) experiments investigating context-based object recognition (Brandman and Peelen, 2017). In those experiments, participants viewed degraded objects in scene context, degraded objects alone, and scenes alone. Behavioral results showed that the degraded objects were easy to recognize when presented in scene context (>70% correct in a nine-category task) but hard to recognize when presented alone (37% correct). fMRI results showed that the multivariate representation of the category of the degraded objects in LOC was strongly enhanced when the objects were viewed in scene context relative to when they were viewed alone. Importantly, the corresponding scenes presented alone did not evoke discriminable object category responses in LOC, providing evidence for supra-additive contextual facilitation. Interestingly, the contextual facilitation of object processing in LOC was correlated with concurrently evoked activity in scene-selective regions, suggesting an interaction between scene- and object-selective regions. MEG results showed that the information about the category of the degraded objects in scenes (derived from multivariate sensor patterns) peaked at two time points: at 160–180 ms and at 280–300 ms after stimulus onset. Crucially, only the later peak showed a significant contextual facilitation effect, with more information about the degraded objects in scenes than the degraded objects alone. Similar to the LOC results, at this time point, the scenes alone did not evoke discriminable object category responses, such that the contextual facilitation of object processing could not reflect the additive processing of scenes and objects. Taken together, these results indicate that scenes—processed in scene-selective cortex—disambiguate object representations in LOC at around 300 ms after stimulus onset.

Figure 3

Download asset Open asset

Figure 3—source data 1

Individual participant means (accuracy and RT).

https://cdn.elifesciences.org/articles/69736/elife-69736-fig3-data1-v1.xlsx

Download elife-69736-fig3-data1-v1.xlsx

The current TMS study was designed to provide causal evidence for this account. Differently from the neuroimaging studies, here we compared the recognition of degraded objects in scenes with the recognition of intact objects alone, rather than degraded objects alone. This was because the large accuracy difference between the recognition of degraded objects in scenes and degraded objects alone prevents a direct comparison of TMS effects between these conditions. Furthermore, this design allowed us to compare the causal neural mechanisms underlying object recognition based on scene context and local features, with the possibility to match the tasks in terms of recognition accuracy.

Across all TMS conditions, objects were equally recognizable when presented in isolation (without degradation; 75.8%) and when presented degraded within a scene (76.7%; main effect of Task: F(1,71)=1.22, p=0.27), showing that scene context can compensate for the loss of object visibility induced by the local degradation (Brandman and Peelen, 2017). Importantly, despite the equal performance, recognition in the two object recognition tasks was supported by different neural mechanisms in a time-specific manner (three-way interaction between Task [intact object recognition, context-based object recognition], Region [OPA, LOC, and EVC], and Time [60–100 ms, 160–200 ms, and 260–300 ms]; F(4,138)=14.37, p<0.001, η_p²=0.294). This interaction was followed up by separate analyses for each of the stimulated regions.

TMS did not significantly affect response time (RT), with no interactions involving either TMS time or TMS region: Time×Region, F(4,138)=0.163, p=0.957; Task×Region, F(2,69)=0.81, p=0.450; Time×Task, F(2,142)=0.37, p=0.689; Time×Region×Task, F(4,138)=1.153, p=0.334. There were also no significant main effects of Time (F(2,142)=2.88, p=0.060) or Region (F(2,69)=0.82, p=0.447).

Altogether, these results reveal distinct neural mechanisms underlying object recognition based on local features (isolated object recognition) and scene context (context-based object recognition). During feedforward processing, EVC and object-selective cortex supported both the recognition of objects in scenes and in isolation, while scene-selective cortex was uniquely required for context-based object recognition. Results additionally showed that feedback to EVC causally supported isolated object recognition, while feedback to object-selective cortex causally supported context-based object recognition. These results provide evidence for two routes to object recognition, each characterized by feedforward and feedback processing but involving different brain regions at different time points (Figure 4).

Figure 4

Download asset Open asset

The finding that EVC (for isolated object recognition) and LOC (for context-based object recognition) causally supported object recognition well beyond the feedforward sweep suggests that feedback processing is required for accurate object recognition. Feedback processing in EVC and LOC may be explained under a common hierarchical perceptual inference framework (Friston, 2005; Haefner et al., 2016; Lee and Mumford, 2003; Rao and Ballard, 1999), in which a global representation provides a prior that allows for disambiguating relatively more local information. For context-based object recognition, the scene (represented in OPA) would be the global element, providing a prior for processing the relatively more local shape of the object (represented in LOC). For isolated object recognition, object shape would be the global element, providing a prior for processing the relatively more local inner object features (e.g., the eyes of a squirrel; represented in EVC). Feedback based on the more global representations thus serves to disambiguate the representation of more local representations. While feedback processing was hypothesized for LOC based on previous neuroimaging findings, we did not hypothesize that feedback to EVC would be required for recognizing isolated objects. Future studies are needed to test under what conditions feedback to EVC causally contributes to object recognition (Camprodon et al., 2010; Koivisto et al., 2011; Wokke et al., 2013). In line with the reverse hierarchy theory, we expect that the specific feedback that is useful for a given task—and the brain regions involved—depend on the available information in the image together with specific task demands (Hochstein and Ahissar, 2002).

An alternative interpretation of the relatively late causal involvement of EVC in isolated object recognition, and LOC in context-based object recognition, is that these effects reflect local recurrence rather than feedback. This interpretation cannot be ruled out based on the current results alone. However, based on previous findings, we think this is unlikely, at least for LOC. In the fMRI study that used a similar stimulus set as used here (Brandman and Peelen, 2017), representations of degraded objects in LOC were facilitated (relative to degraded objects alone) by the presence of scene context, indicating input from outside of LOC considering that LOC did not represent object information from scenes presented alone. Furthermore, the corresponding MEG study showed two peaks for degraded objects in scenes, one at 160–180 ms and one at 280–300 ms. The later peak showed a significant contextual facilitation effect in the MEG study, with better decoding of degraded objects in scenes than degraded objects alone. The present finding that TMS over LOC at 260–300 ms selectively impaired context-based object recognition is fully in line with these fMRI and MEG findings, pointing to feedback processing rather than local recurrence.

Taken together with previous findings, the current results are thus best explained by an account in which information from scenes (processed in scene-selective cortex) feeds back to LOC to disambiguate object representations. This mechanism may underlie the behavioral benefits previously observed for object recognition in semantically and syntactically congruent (vs. incongruent) scene context (Biederman et al., 1982; Davenport and Potter, 2004; Munneke et al., 2013; Võ and Wolfe, 2013), as predicted by interactive accounts that propose that contextual facilitation is supported by contextual expectations (Bar, 2004; Davenport and Potter, 2004), with quickly extracted global scene ‘gist’ priming the representation of candidate objects in the visual cortex (Bar, 2004; Oliva and Torralba, 2007; Torralba, 2003). The current TMS results suggest that OPA is crucial for extracting this global scene information at around 160–200 ms after scene onset, and that this information is integrated with local object information in LOC around 100 ms later. The current results do not speak to whether OPA-LOC connectivity is direct or indirect, for example involving additional brain regions such as other scene-selective regions or the orbitofrontal cortex (Bar, 2004).

Our study raises the interesting question of what type of context-based expectations help to disambiguate object representations in LOC. The scenes in the current study provided multiple cues that may help to recognize the degraded objects. For example, the scenes provided information about the approximate real-world size of the objects as well as the objects’ likely semantic category. Both of these cues may help to recognize objects (Biederman et al., 1982; Davenport and Potter, 2004; Munneke et al., 2013; Võ and Wolfe, 2013). Future experiments could test whether feedback to LOC is specifically related to one of these cues. For example, one could test whether similar effects are found when objects are presented in semantically uninformative scenes, with the scene only providing information about the approximate real-world size of the object.

To conclude, the current study provides causal evidence that context-based expectations facilitate object recognition by disambiguating object representations in the visual cortex. More generally, results reveal that distinct neural mechanisms support object recognition based on local features and global scene context. Future experiments may extend our approach to include other contextual features such as co-occurring objects, temporal context, and input from other modalities.

All data generated or analysed during this study are included in the manuscript and supporting files. Source data files have been provided for Figure 3.

1. 1. Bar M

   (2004) Visual objects in context

   Nature Reviews Neuroscience 5:617–629.

   https://doi.org/10.1038/nrn1476

   • PubMed
   • Google Scholar
2. 1. Biederman I
   2. Mezzanotte RJ
   3. Rabinowitz JC

   (1982) Scene perception: detecting and judging objects undergoing relational violations

   Cognitive Psychology 14:143–177.

   https://doi.org/10.1016/0010-0285(82)90007-X

   • PubMed
   • Google Scholar
3. 1. Brandman T
   2. Peelen MV

   (2017) Interaction between scene and object processing revealed by human fMRI and MEG decoding

   The Journal of Neuroscience 37:7700–7710.

   https://doi.org/10.1523/JNEUROSCI.0582-17.2017

   • PubMed
   • Google Scholar
4. 1. Camprodon JA
   2. Zohary E
   3. Brodbeck V
   4. Pascual-Leone A

   (2010) Two phases of V1 activity for visual recognition of natural images

   Journal of Cognitive Neuroscience 22:1262–1269.

   https://doi.org/10.1162/jocn.2009.21253

   • PubMed
   • Google Scholar
5. 1. Cichy RM
   2. Pantazis D
   3. Oliva A

   (2014) Resolving human object recognition in space and time

   Nature Neuroscience 17:455–462.

   https://doi.org/10.1038/nn.3635

   • PubMed
   • Google Scholar
6. 1. Davenport JL
   2. Potter MC

   (2004) Scene consistency in object and background perception

   Psychological Science 15:559–564.

   https://doi.org/10.1111/j.0956-7976.2004.00719.x

   • PubMed
   • Google Scholar
7. 1. de Lange FP
   2. Heilbron M
   3. Kok P

   (2018) How do expectations shape perception?

   Trends in Cognitive Sciences 22:764–779.

   https://doi.org/10.1016/j.tics.2018.06.002

   • PubMed
   • Google Scholar
8. 1. DiCarlo JJ
   2. Zoccolan D
   3. Rust NC

   (2012) How does the brain solve visual object recognition?

   Neuron 73:415–434.

   https://doi.org/10.1016/j.neuron.2012.01.010

   • PubMed
   • Google Scholar
9. 1. Dijkstra N
   2. Mostert P
   3. Lange FP
   4. Bosch S
   5. van Gerven MA

   (2018) Differential temporal dynamics during visual imagery and perception

   eLife 7:e33904.

   https://doi.org/10.7554/eLife.33904

   • PubMed
   • Google Scholar
10. 1. Dilks DD
    2. Julian JB
    3. Paunov AM
    4. Kanwisher N

    (2013) The occipital place area is causally and selectively involved in scene perception

    Journal of Neuroscience 33:1331–1336.

    https://doi.org/10.1523/JNEUROSCI.4081-12.2013

    • PubMed
    • Google Scholar
11. 1. Faivre N
    2. Dubois J
    3. Schwartz N
    4. Mudrik L

    (2019) Imaging object-scene relations processing in visible and invisible natural scenes

    Scientific Reports 9:4567.

    https://doi.org/10.1038/s41598-019-38654-z

    • PubMed
    • Google Scholar
12. 1. Friston K

    (2005) A theory of cortical responses

    Philosophical Transactions of the Royal Society B: Biological Sciences 360:815–836.

    https://doi.org/10.1098/rstb.2005.1622

    • Google Scholar
13. 1. Grill-Spector K

    (2003) The neural basis of object perception

    Current Opinion in Neurobiology 13:159–166.

    https://doi.org/10.1016/S0959-4388(03)00040-0

    • PubMed
    • Google Scholar
14. 1. Gronau N
    2. Neta M
    3. Bar M

    (2008) Integrated contextual representation for objects' identities and their locations

    Journal of Cognitive Neuroscience 20:371–388.

    https://doi.org/10.1162/jocn.2008.20027

    • PubMed
    • Google Scholar
15. 1. Haefner RM
    2. Berkes P
    3. Fiser J

    (2016) Perceptual Decision-Making as probabilistic inference by neural sampling

    Neuron 90:649–660.

    https://doi.org/10.1016/j.neuron.2016.03.020

    • PubMed
    • Google Scholar
16. 1. Henderson JM
    2. Hollingworth A

    (1999) High-level scene perception

    Annual Review of Psychology 50:243–271.

    https://doi.org/10.1146/annurev.psych.50.1.243

    • PubMed
    • Google Scholar
17. 1. Hochstein S
    2. Ahissar M

    (2002) View from the top: hierarchies and reverse hierarchies in the visual system

    Neuron 36:791–804.

    https://doi.org/10.1016/s0896-6273(02)01091-7

    • PubMed
    • Google Scholar
18. 1. Julian JB
    2. Ryan J
    3. Hamilton RH
    4. Epstein RA

    (2016) The occipital place area is causally involved in representing environmental boundaries during navigation

    Current Biology 26:1104–1109.

    https://doi.org/10.1016/j.cub.2016.02.066

    • Google Scholar
19. 1. Koivisto M
    2. Mäntylä T
    3. Silvanto J

    (2010) The role of early visual cortex (V1/V2) in conscious and unconscious visual perception

    NeuroImage 51:828–834.

    https://doi.org/10.1016/j.neuroimage.2010.02.042

    • PubMed
    • Google Scholar
20. 1. Koivisto M
    2. Railo H
    3. Revonsuo A
    4. Vanni S
    5. Salminen-Vaparanta N

    (2011) Recurrent processing in V1/V2 contributes to categorization of natural scenes

    Journal of Neuroscience 31:2488–2492.

    https://doi.org/10.1523/JNEUROSCI.3074-10.2011

    • PubMed
    • Google Scholar
21. 1. Korjoukov I
    2. Jeurissen D
    3. Kloosterman NA
    4. Verhoeven JE
    5. Scholte HS
    6. Roelfsema PR

    (2012) The time course of perceptual grouping in natural scenes

    Psychological Science 23:1482–1489.

    https://doi.org/10.1177/0956797612443832

    • PubMed
    • Google Scholar
22. 1. Lamme VA
    2. Roelfsema PR

    (2000) The distinct modes of vision offered by feedforward and recurrent processing

    Trends in Neurosciences 23:571–579.

    https://doi.org/10.1016/S0166-2236(00)01657-X

    • PubMed
    • Google Scholar
23. 1. Lee TS
    2. Mumford D

    (2003) Hierarchical bayesian inference in the visual cortex

    Journal of the Optical Society of America A 20:1434.

    https://doi.org/10.1364/JOSAA.20.001434

    • Google Scholar
24. 1. Liu H
    2. Agam Y
    3. Madsen JR
    4. Kreiman G

    (2009) Timing, timing, timing: fast decoding of object information from intracranial field potentials in human visual cortex

    Neuron 62:281–290.

    https://doi.org/10.1016/j.neuron.2009.02.025

    • PubMed
    • Google Scholar
25. 1. Malach R
    2. Reppas JB
    3. Benson RR
    4. Kwong KK
    5. Jiang H
    6. Kennedy WA
    7. Ledden PJ
    8. Brady TJ
    9. Rosen BR
    10. Tootell RB

    (1995) Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex

    PNAS 92:8135–8139.

    https://doi.org/10.1073/pnas.92.18.8135

    • PubMed
    • Google Scholar
26. 1. Munneke J
    2. Brentari V
    3. Peelen MV

    (2013) The influence of scene context on object recognition is independent of attentional focus

    Frontiers in Psychology 4:552.

    https://doi.org/10.3389/fpsyg.2013.00552

    • PubMed
    • Google Scholar
27. 1. Murray SO
    2. Kersten D
    3. Olshausen BA
    4. Schrater P
    5. Woods DL

    (2002) Shape perception reduces activity in human primary visual cortex

    PNAS 99:15164–15169.

    https://doi.org/10.1073/pnas.192579399

    • PubMed
    • Google Scholar
28. 1. O'Shea J
    2. Muggleton NG
    3. Cowey A
    4. Walsh V

    (2004) Timing of target discrimination in human frontal eye fields

    Journal of Cognitive Neuroscience 16:1060–1067.

    https://doi.org/10.1162/0898929041502634

    • Google Scholar
29. 1. Oliva A
    2. Torralba A

    (2007) The role of context in object recognition

    Trends in Cognitive Sciences 11:520–527.

    https://doi.org/10.1016/j.tics.2007.09.009

    • Google Scholar
30. 1. Opitz A
    2. Legon W
    3. Rowlands A
    4. Bickel WK
    5. Paulus W
    6. Tyler WJ

    (2013) Physiological observations validate finite element models for estimating subject-specific electric field distributions induced by transcranial magnetic stimulation of the human motor cortex

    NeuroImage 81:253–264.

    https://doi.org/10.1016/j.neuroimage.2013.04.067

    • Google Scholar
31. 1. Park S
    2. Brady TF
    3. Greene MR
    4. Oliva A

    (2011) Disentangling scene content from spatial boundary: complementary roles for the parahippocampal place area and lateral occipital complex in representing real-world scenes

    Journal of Neuroscience 31:1333–1340.

    https://doi.org/10.1523/JNEUROSCI.3885-10.2011

    • PubMed
    • Google Scholar
32. 1. Pascual-Leone A
    2. Walsh V

    (2001) Fast backprojections from the motion to the primary visual area necessary for visual awareness

    Science 292:510–512.

    https://doi.org/10.1126/science.1057099

    • Google Scholar
33. 1. Pitcher D
    2. Walsh V
    3. Yovel G
    4. Duchaine B

    (2007) TMS evidence for the involvement of the right occipital face area in early face processing

    Current Biology 17:1568–1573.

    https://doi.org/10.1016/j.cub.2007.07.063

    • PubMed
    • Google Scholar
34. 1. Pitcher D
    2. Charles L
    3. Devlin JT
    4. Walsh V
    5. Duchaine B

    (2009) Triple dissociation of faces, bodies, and objects in extrastriate cortex

    Current Biology 19:319–324.

    https://doi.org/10.1016/j.cub.2009.01.007

    • PubMed
    • Google Scholar
35. 1. Rao RPN
    2. Ballard DH

    (1999) Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects

    Nature Neuroscience 2:79–87.

    https://doi.org/10.1038/4580

    • Google Scholar
36. 1. Reddy L
    2. Tsuchiya N
    3. Serre T

    (2010) Reading the mind's eye: Decoding category information during mental imagery

    NeuroImage 50:818–825.

    https://doi.org/10.1016/j.neuroimage.2009.11.084

    • Google Scholar
37. 1. Rémy F
    2. Vayssière N
    3. Pins D
    4. Boucart M
    5. Fabre-Thorpe M

    (2014) Incongruent object/context relationships in visual scenes: Where are they processed in the brain?

    Brain and Cognition 84:34–43.

    https://doi.org/10.1016/j.bandc.2013.10.008

    • Google Scholar
38. 1. Riesenhuber M
    2. Poggio T

    (1999) Hierarchical models of object recognition in cortex

    Nature Neuroscience 2:1019–1025.

    https://doi.org/10.1038/14819

    • Google Scholar
39. 1. Scholte HS
    2. Jolij J
    3. Fahrenfort JJ
    4. Lamme VA

    (2008) Feedforward and recurrent processing in scene segmentation: electroencephalography and functional magnetic resonance imaging

    Journal of Cognitive Neuroscience 20:2097–2109.

    https://doi.org/10.1162/jocn.2008.20142

    • PubMed
    • Google Scholar
40. 1. Serre T
    2. Oliva A
    3. Poggio T

    (2007) A feedforward architecture accounts for rapid categorization

    PNAS 104:6424–6429.

    https://doi.org/10.1073/pnas.0700622104

    • Google Scholar
41. 1. Torralba A

    (2003) Contextual Priming for Object Detection

    International Journal of Computer Vision 53:169–191.

    https://doi.org/10.1023/A:1023052124951

    • Google Scholar
42. 1. van Koningsbruggen MG
    2. Peelen MV
    3. Downing PE

    (2013) A causal role for the extrastriate body area in detecting people in Real-World scenes

    Journal of Neuroscience 33:7003–7010.

    https://doi.org/10.1523/JNEUROSCI.2853-12.2013

    • Google Scholar
43. 1. Võ ML
    2. Wolfe JM

    (2013) Differential electrophysiological signatures of semantic and syntactic scene processing

    Psychological Science 24:1816–1823.

    https://doi.org/10.1177/0956797613476955

    • PubMed
    • Google Scholar
44. 1. Wischnewski M
    2. Peelen MV

    (2021) Causal evidence for a double dissociation between object- and Scene-Selective regions of visual cortex: a preregistered TMS replication study

    The Journal of Neuroscience 41:751–756.

    https://doi.org/10.1523/JNEUROSCI.2162-20.2020

    • PubMed
    • Google Scholar
45. 1. Wokke ME
    2. Vandenbroucke AR
    3. Scholte HS
    4. Lamme VA

    (2013) Confuse your illusion: feedback to early visual cortex contributes to perceptual completion

    Psychological Science 24:63–71.

    https://doi.org/10.1177/0956797612449175

    • PubMed
    • Google Scholar

Acceptance summary:

This study will be of interest to scientists involved in high-level vision. The data provide a compelling demonstration of the causal role of three key visual areas in context-based object recognition. The key claims of the manuscript are supported by the data, and are strengthened by the pre-registration of each of the three experiments.

Decision letter after peer review:

Thank you for submitting your article "Causal neural mechanisms of context-based object recognition" for consideration by eLife. Your article has been reviewed by 3 peer reviewers, including Redmond G O’Connell as the Reviewing Editor and Reviewer #1, and the evaluation has been overseen by Joshua Gold as the Senior Editor. The following individual involved in review of your submission has agreed to reveal their identity: Peter Kok (Reviewer #3).

The reviewers have discussed their reviews with one another, and the Reviewing Editor has drafted this to help you prepare a revised submission.

Essential revisions:

1) The preregistration for this study commits to including the scene-only condition in the statistical analyses for the main experiment however only the object-based and context-based conditions were considered. The authors should either provide a strong justification for this deviation or else run and report the statistics as originally planned.

2) More detail on the precise participant screening procedure is required. Exactly what tasks and stimuli were participants exposed to? What TMS SOAs were used? How many participants were excluded based on this procedure? A statement should be added to the main text flagging this procedure to the reader so that they are clear that the basic main effect of TMS to LOC object-based task performance was pre-ordained.

3) The authors should provide greater discussion of alternative interpretations of their results. Currently the authors only entertain the possibility that the results reflect feedback effects but they should also consider the possibility of persistent/recurrent activity within visual areas reflecting extended processing of stimuli held in iconic memory.

Reviewer #1 (Recommendations for the authors):

The study hypotheses could be articulated more clearly. Citations should be provided to back up the hypothesised TMS SOA effects in each region. The legend of Figure 3 would seem a good place to lay out the predictions in more detail.

The area identified as EVA should be clarified in the Introduction. What visual regions does this area encompass? Just V1? Others?

Reviewer #2 (Recommendations for the authors):

1. The authors argue that feedback based on more global representations (of the scene) serve to disambiguate more local representations (of the object).

The scene-only condition seems important to this interpretation, and I would suggest discussing this manipulation further in the main text. An alternative interpretation is that scene recognition simply reduces the epistemic priors of what the object could be (e.g., if the scene is grass, the object is unlikely to be a fish, ship, airplane, or train). Indeed, Figure S2 suggests that performance in the scene-only condition, although relatively poor (40-50%), is much higher than chance (12.5%). So scene recognition seems to filter the range of possible correct responses in the first place, and the effects that are observed may be separate to the capacity of scene recognition to disambiguate specific features of the object itself.

I take the authors' point that the absent effect of TMS on accuracy in the scene-only condition argues against this as a possibility. Nevertheless, an informative condition may have been one in which the image was simply occluded by a shape (e.g., circle) that provided only information about the relative size of the object, but not about its internal features. Such a manipulation would have maintained a similarity with the context-based condition while eliminating the intrinsic perceptual features of the object that could be disambiguated by the scene (other than its approximate size, which provides information about the possible object category, but not the object itself). This may have allowed the authors to more clearly determine whether context-based object recognition is specifically driven by disambiguation of the perceptual features intrinsic to each object.

I am not necessarily suggesting running an additional experiment, but further clarification/discussion on the above issues would be worthwhile.

Reviewer #3 (Recommendations for the authors):

– Why where these exact time windows chosen for stimulation (and hypotheses)? Can you provide a more concrete (neurophysiological) rationale?

– The EVC results are opposite to the hypothesised ones, with later involvement for objects in isolation than those in context. The authors acknowledge this, but do not discuss in much depth why this might be. Alternatively, they might simply acknowledge that we do not know why this is, and more research is needed on this point.

– Could you also draw the hypotheses that the parallel account yields, so that it can be clearly seen which data points distinguish the hypotheses? Is the late LOC stimulation effect in context vs. isolation the only crucial point?

– Regarding this datapoint, could it be that late LOC stimulation interferes with object recognition in the context condition because the objects were degraded, rather than because they were presented in a scene context? In other words, because there is more (local) recurrence in LOC required to resolve degraded objects? This seems important to rule out (or acknowledge).

– From Figure S1, it seems that participants were generally faster in the EVC experiment. Any idea why this might be?

Essential revisions:

1) The preregistration for this study commits to including the scene-only condition in the statistical analyses for the main experiment however only the object-based and context-based conditions were considered. The authors should either provide a strong justification for this deviation or else run and report the statistics as originally planned.

The pre-registered statistics for the scene-alone condition in the OPA experiment are now included in the manuscript (p.9-10). The relevant figure (Figure 3) has also been updated such that the scene-alone condition results are now in the main text rather than the Supplement. Results confirmed our predictions, showing a reduction of scene-alone performance when OPA was stimulated 160-200 ms after stimulus onset. Note that the scene-alone condition was only included in the pre-registration of the OPA experiment, which is why we had not reported the corresponding statistics previously. (This condition was not relevant for the LOC and EVC experiments.)

2) More detail on the precise participant screening procedure is required. Exactly what tasks and stimuli were participants exposed to? What TMS SOAs were used? How many participants were excluded based on this procedure? A statement should be added to the main text flagging this procedure to the reader so that they are clear that the basic main effect of TMS to LOC object-based task performance was pre-ordained.

We now introduce the screening procedure in the main text and point the reader to a recent publication that documents the methods and results of this experiment (Wischnewski and Peelen, J Neurosci 2021). The screening experiment followed the design of Dilks et al. (J Neurosci 2013), stimulating OPA and LOC using 5 TMS pulses at a rate of 10Hz (i.e., no SOAs were used). No participants were excluded – all participants were assigned to one of the three conditions (OPA, LOC, EVC). This is now more clearly explained in the manuscript.

3) The authors should provide greater discussion of alternative interpretations of their results. Currently the authors only entertain the possibility that the results reflect feedback effects but they should also consider the possibility of persistent/recurrent activity within visual areas reflecting extended processing of stimuli held in iconic memory.

We have added a paragraph to the Discussion section in which we discuss the alternative interpretation of local recurrence (p.13-14).

Reviewer #1 (Recommendations for the authors):

The study hypotheses could be articulated more clearly. Citations should be provided to back up the hypothesised TMS SOA effects in each region. The legend of Figure 3 would seem a good place to lay out the predictions in more detail.

Thanks for these suggestions. We have added citations to back up the hypothesized SOAs. We have also explained the previous fMRI/MEG study that led to these predictions in more detail. We now also included a more detailed explanation in the Figure 3 legend.

The area identified as EVA should be clarified in the Introduction. What visual regions does this area encompass? Just V1? Others?

We have added this clarification to the Introduction, citing previous work using the same procedures. EVC here primarily corresponds to V1.

Reviewer #2 (Recommendations for the authors):

1. The authors argue that feedback based on more global representations (of the scene) serve to disambiguate more local representations (of the object).

The scene-only condition seems important to this interpretation, and I would suggest discussing this manipulation further in the main text. An alternative interpretation is that scene recognition simply reduces the epistemic priors of what the object could be (e.g., if the scene is grass, the object is unlikely to be a fish, ship, airplane, or train). Indeed, Figure S2 suggests that performance in the scene-only condition, although relatively poor (40-50%), is much higher than chance (12.5%). So scene recognition seems to filter the range of possible correct responses in the first place, and the effects that are observed may be separate to the capacity of scene recognition to disambiguate specific features of the object itself.

I take the authors' point that the absent effect of TMS on accuracy in the scene-only condition argues against this as a possibility. Nevertheless, an informative condition may have been one in which the image was simply occluded by a shape (e.g., circle) that provided only information about the relative size of the object, but not about its internal features. Such a manipulation would have maintained a similarity with the context-based condition while eliminating the intrinsic perceptual features of the object that could be disambiguated by the scene (other than its approximate size, which provides information about the possible object category, but not the object itself). This may have allowed the authors to more clearly determine whether context-based object recognition is specifically driven by disambiguation of the perceptual features intrinsic to each object.

I am not necessarily suggesting running an additional experiment, but further clarification/discussion on the above issues would be worthwhile.

The scene-alone condition is now included in the main text, as well as in Figure 3. We have also included a more extensive summary of previous fMRI and MEG studies that formed the basis for the current study, where we focused on the question of whether scene and object information are additive or super-additive. We have explained this study in more detail in the Introduction to make the predictions clearer.

The reviewer also raises an interesting point about what aspects the scene helps to disambiguate. Object size is certainly one candidate. We have included a paragraph to the Discussion raising this possibility (p.14-15).

Reviewer #3 (Recommendations for the authors):

– Why where these exact time windows chosen for stimulation (and hypotheses)? Can you provide a more concrete (neurophysiological) rationale?

We have added citations to back up the hypothesized SOAs. We have also explained our previous fMRI and MEG work in more detail in the Introduction, as this led to the current predictions.

– The EVC results are opposite to the hypothesised ones, with later involvement for objects in isolation than those in context. The authors acknowledge this, but do not discuss in much depth why this might be. Alternatively, they might simply acknowledge that we do not know why this is, and more research is needed on this point.

We have extended the discussion of these results and mention that more work is needed to follow up on these findings (p.13).

– Could you also draw the hypotheses that the parallel account yields, so that it can be clearly seen which data points distinguish the hypotheses? Is the late LOC stimulation effect in context vs. isolation the only crucial point?

Yes, the late LOC effect is the most informative point to distinguish between the hypotheses. We have now made this clear in the Introduction, also based on a more detailed description of previous fMRI/MEG work.

– Regarding this datapoint, could it be that late LOC stimulation interferes with object recognition in the context condition because the objects were degraded, rather than because they were presented in a scene context? In other words, because there is more (local) recurrence in LOC required to resolve degraded objects? This seems important to rule out (or acknowledge).

We have added a paragraph to the Discussion section in which we discuss the alternative interpretation of local recurrence (p.13-14).

– From Figure S1, it seems that participants were generally faster in the EVC experiment. Any idea why this might be?

There were no significant differences between regions (main effect of Region: (F(2,69) = 0.82, p = 0.447)), which is now reported in the main text (p.8). Thus, while participants were numerically faster in the EVC experiment, this was not reliably different from the other regions (note that Region was manipulated across participants, unlike the other variables).

Author details

1. Miles Wischnewski

   1. Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, Netherlands
   2. Department of Biomedical Engineering, University of Minnesota, Minneapolis, United States

   Contribution

   Data curation, Formal analysis, Investigation, Methodology, Project administration, Writing - review and editing

   Competing interests

   No competing interests declared

2. Marius V Peelen

   Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, Netherlands

   Contribution

   Conceptualization, Supervision, Funding acquisition, Methodology, Writing - original draft, Project administration, Writing - review and editing

   For correspondence

   m.peelen@donders.ru.nl

   Competing interests

   Reviewing editor, eLife

   "This ORCID iD identifies the author of this article:" 0000-0002-4026-7303

• 1,503

  Page views

• 200

  Downloads

• 5

  Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.