Distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior

  1. Iris IA Groen  Is a corresponding author
  2. Michelle R Greene
  3. Christopher Baldassano
  4. Li Fei-Fei
  5. Diane M Beck
  6. Chris I Baker
  1. National Institutes of Health, United States
  2. Bates College, United States
  3. Princeton University, United States
  4. Stanford University, United States
  5. University of Illinois at Urbana-Champaign, United States

Abstract

Inherent correlations between visual and semantic features in real-world scenes make it difficult to determine how different scene properties contribute to neural representations. Here, we assessed the contributions of multiple properties to scene representation by partitioning the variance explained in human behavioral and brain measurements by three feature models whose inter-correlations were minimized a priori through stimulus preselection. Behavioral assessments of scene similarity reflected unique contributions from a functional feature model indicating potential actions in scenes as well as high-level visual features from a deep neural network (DNN). In contrast, similarity of cortical responses in scene-selective areas was uniquely explained by mid- and high-level DNN features only, while an object label model did not contribute uniquely to either domain. The striking dissociation between functional and DNN features in their contribution to behavioral and brain representations of scenes indicates that scene-selective cortex represents only a subset of behaviorally relevant scene information.

Article and author information

Author details

  1. Iris IA Groen

    Laboratory of Brain and Cognition, National Institutes of Health, Bethesda, United States
    For correspondence
    iris.groen@nih.gov
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-5536-6128
  2. Michelle R Greene

    Neuroscience Program, Bates College, Lewiston, United States
    Competing interests
    The authors declare that no competing interests exist.
  3. Christopher Baldassano

    Princeton Neuroscience Institute, Princeton University, Princeton, United States
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-3540-5019
  4. Li Fei-Fei

    Stanford Vision Lab, Stanford University, Stanford, United States
    Competing interests
    The authors declare that no competing interests exist.
  5. Diane M Beck

    Department of Psychology, University of Illinois at Urbana-Champaign, Urbana-Champaign, United States
    Competing interests
    The authors declare that no competing interests exist.
  6. Chris I Baker

    Laboratory of Brain and Cognition, National Institutes of Health, Bethesda, United States
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-6861-8964

Funding

National Institutes of Health (ZIAMH002909)

  • Iris IA Groen
  • Chris I Baker

Netherlands Organization for Scientific Research (Rubicon Fellowship)

  • Iris IA Groen

Office of Naval Research (Multidisciplinary Research Initiative Grant N000141410671)

  • Li Fei-Fei
  • Diane M Beck

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Ethics

Human subjects: All participants had normal or corrected-to-normal vision and gave written informed consent as part of the study protocol (93 M-0170, NCT00001360) prior to participation in the study. The study was approved by the Institutional Review Board of the National Institutes of Health and was conducted according to the Declaration of Helsinki.

Copyright

This is an open-access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.

Metrics

  • 5,922
    views
  • 758
    downloads
  • 130
    citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Iris IA Groen
  2. Michelle R Greene
  3. Christopher Baldassano
  4. Li Fei-Fei
  5. Diane M Beck
  6. Chris I Baker
(2018)
Distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior
eLife 7:e32962.
https://doi.org/10.7554/eLife.32962

Share this article

https://doi.org/10.7554/eLife.32962

Further reading

    1. Developmental Biology
    2. Neuroscience
    Changtian Ye, Ryan Ho ... James Q Zheng
    Research Article

    Environmental insults, including mild head trauma, significantly increase the risk of neurodegeneration. However, it remains challenging to establish a causative connection between early-life exposure to mild head trauma and late-life emergence of neurodegenerative deficits, nor do we know how sex and age compound the outcome. Using a Drosophila model, we demonstrate that exposure to mild head trauma causes neurodegenerative conditions that emerge late in life and disproportionately affect females. Increasing age-at-injury further exacerbates this effect in a sexually dimorphic manner. We further identify sex peptide signaling as a key factor in female susceptibility to post-injury brain deficits. RNA sequencing highlights a reduction in innate immune defense transcripts specifically in mated females during late life. Our findings establish a causal relationship between early head trauma and late-life neurodegeneration, emphasizing sex differences in injury response and the impact of age-at-injury. Finally, our findings reveal that reproductive signaling adversely impacts female response to mild head insults and elevates vulnerability to late-life neurodegeneration.

    1. Neuroscience
    Iustin V Tabarean
    Research Article

    Neurotensin (Nts) is a neuropeptide acting as a neuromodulator in the brain. Pharmacological studies have identified Nts as a potent hypothermic agent. The medial preoptic area, a region that plays an important role in the control of thermoregulation, contains a high density of neurotensinergic neurons and Nts receptors. The conditions in which neurotensinergic neurons play a role in thermoregulation are not known. In this study, optogenetic stimulation of preoptic Nts neurons induced a small hyperthermia. In vitro, optogenetic stimulation of preoptic Nts neurons resulted in synaptic release of GABA and net inhibition of the preoptic pituitary adenylate cyclase-activating polypeptide (Adcyap1) neurons firing activity. GABA-A receptor antagonist or genetic deletion of Slc32a1 (VGAT) in Nts neurons unmasked also an excitatory effect that was blocked by a Nts receptor 1 antagonist. Stimulation of preoptic Nts neurons lacking Slc32a1 resulted in excitation of Adcyap1 neurons and hypothermia. Mice lacking Slc32a1 expression in Nts neurons presented changes in the fever response and in the responses to heat or cold exposure as well as an altered circadian rhythm of body temperature. Chemogenetic activation of all Nts neurons in the brain induced a 4–5°C hypothermia, which could be blocked by Nts receptor antagonists in the preoptic area. Chemogenetic activation of preoptic neurotensinergic projections resulted in robust excitation of preoptic Adcyap1 neurons. Taken together, our data demonstrate that endogenously released Nts can induce potent hypothermia and that excitation of preoptic Adcyap1 neurons is the cellular mechanism that triggers this response.