1. Neuroscience
Download icon

Distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior

  1. Iris IA Groen  Is a corresponding author
  2. Michelle R Greene
  3. Christopher Baldassano
  4. Li Fei-Fei
  5. Diane M Beck
  6. Chris I Baker
  1. National Institutes of Health, United States
  2. Bates College, United States
  3. Princeton University, United States
  4. Stanford University, United States
  5. University of Illinois at Urbana-Champaign, United States
Research Article
  • Cited 31
  • Views 3,227
  • Annotations
Cite this article as: eLife 2018;7:e32962 doi: 10.7554/eLife.32962

Abstract

Inherent correlations between visual and semantic features in real-world scenes make it difficult to determine how different scene properties contribute to neural representations. Here, we assessed the contributions of multiple properties to scene representation by partitioning the variance explained in human behavioral and brain measurements by three feature models whose inter-correlations were minimized a priori through stimulus preselection. Behavioral assessments of scene similarity reflected unique contributions from a functional feature model indicating potential actions in scenes as well as high-level visual features from a deep neural network (DNN). In contrast, similarity of cortical responses in scene-selective areas was uniquely explained by mid- and high-level DNN features only, while an object label model did not contribute uniquely to either domain. The striking dissociation between functional and DNN features in their contribution to behavioral and brain representations of scenes indicates that scene-selective cortex represents only a subset of behaviorally relevant scene information.

Article and author information

Author details

  1. Iris IA Groen

    Laboratory of Brain and Cognition, National Institutes of Health, Bethesda, United States
    For correspondence
    iris.groen@nih.gov
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-5536-6128
  2. Michelle R Greene

    Neuroscience Program, Bates College, Lewiston, United States
    Competing interests
    The authors declare that no competing interests exist.
  3. Christopher Baldassano

    Princeton Neuroscience Institute, Princeton University, Princeton, United States
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-3540-5019
  4. Li Fei-Fei

    Stanford Vision Lab, Stanford University, Stanford, United States
    Competing interests
    The authors declare that no competing interests exist.
  5. Diane M Beck

    Department of Psychology, University of Illinois at Urbana-Champaign, Urbana-Champaign, United States
    Competing interests
    The authors declare that no competing interests exist.
  6. Chris I Baker

    Laboratory of Brain and Cognition, National Institutes of Health, Bethesda, United States
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-6861-8964

Funding

National Institutes of Health (ZIAMH002909)

  • Iris IA Groen
  • Chris I Baker

Netherlands Organization for Scientific Research (Rubicon Fellowship)

  • Iris IA Groen

Office of Naval Research (Multidisciplinary Research Initiative Grant N000141410671)

  • Li Fei-Fei
  • Diane M Beck

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Ethics

Human subjects: All participants had normal or corrected-to-normal vision and gave written informed consent as part of the study protocol (93 M-0170, NCT00001360) prior to participation in the study. The study was approved by the Institutional Review Board of the National Institutes of Health and was conducted according to the Declaration of Helsinki.

Reviewing Editor

  1. Doris Y Tsao, California Institute of Technology, United States

Publication history

  1. Received: October 19, 2017
  2. Accepted: March 2, 2018
  3. Accepted Manuscript published: March 7, 2018 (version 1)
  4. Version of Record published: March 20, 2018 (version 2)

Copyright

This is an open-access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.

Metrics

  • 3,227
    Page views
  • 485
    Downloads
  • 31
    Citations

Article citation count generated by polling the highest count across the following sources: Scopus, Crossref, PubMed Central.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Download citations (links to download the citations from this article in formats compatible with various reference manager tools)

Open citations (links to open the citations from this article in various online reference manager services)

Further reading

    1. Neuroscience
    Genjiro Suzuki et al.
    Research Article Updated

    Abnormal α-synuclein aggregation has been implicated in several diseases and is known to spread in a prion-like manner. There is a relationship between protein aggregate structure (strain) and clinical phenotype in prion diseases, however, whether differences in the strains of α-synuclein aggregates account for the different pathologies remained unclear. Here, we generated two types of α-synuclein fibrils from identical monomer and investigated their seeding and propagation ability in mice and primary-cultured neurons. One α-synuclein fibril induced marked accumulation of phosphorylated α-synuclein and ubiquitinated protein aggregates, while the other did not, indicating the formation of α-synuclein two strains. Notably, the former α-synuclein strain inhibited proteasome activity and co-precipitated with 26S proteasome complex. Further examination indicated that structural differences in the C-terminal region of α-synuclein strains lead to different effects on proteasome activity. These results provide a possible molecular mechanism to account for the different pathologies induced by different α-synuclein strains.

    1. Neuroscience
    Omer Faruk Gulban et al.
    Tools and Resources

    The human superior temporal plane, the site of the auditory cortex, displays high inter-individual macro-anatomical variation. This questions the validity of curvature-based alignment (CBA) methods for in vivo imaging data. Here, we have addressed this issue by developing CBA+, which is a cortical surface registration method that uses prior macro-anatomical knowledge. We validate this method by using cytoarchitectonic areas on 10 individual brains (which we make publicly available). Compared to volumetric and standard surface registration, CBA+ results in a more accurate cytoarchitectonic auditory atlas. The improved correspondence of micro-anatomy following the improved alignment of macro-anatomy validates the superiority of CBA+ compared to CBA. In addition, we use CBA+ to align in vivo and postmortem data. This allows projection of functional and anatomical information collected in vivo onto the cytoarchitectonic areas, which has the potential to contribute to the ongoing debate on the parcellation of the human auditory cortex.