'Artiphysiology' reveals V4-like shape tuning in a deep network trained for image classification
Abstract
Deep networks provide a potentially rich interconnection between neuroscientific and artificial approaches to understanding visual intelligence, but the relationship between artificial and neural representations of complex visual form has not been elucidated at the level of single-unit selectivity. Taking the approach of an electrophysiologist to characterizing single CNN units, we found many units exhibit translation-invariant boundary curvature selectivity approaching that of exemplar neurons in the primate mid-level visual area V4. For some V4-like units, particularly in middle layers, the natural images that drove them best were qualitatively consistent with selectivity for object boundaries. Our results identify a novel image-computable model for V4 boundary curvature selectivity and suggest that such a representation may begin to emerge within an artificial network trained for image categorization, even though boundary information was not provided during training. This raises the possibility that single-unit selectivity in CNNs will become a guide for understanding sensory cortex.
Data availability
No new datasets were generated in the course of this research. The model this research is based on is openly available from the Berkeley Artificial Intelligence Lab.
-
BVLC reference caffenetBAIR BVLC CaffeNet Model.
Article and author information
Author details
Funding
National Science Foundation (Graduate Research Fellowship)
- Dean A Pospisil
National Science Foundation (CRCNS Grant IIS-1309725)
- Anitha Pasupathy
- Wyeth Bair
Google (Google Faculty Research Award)
- Wyeth Bair
National Institutes of Health (Grant R01 EY-018839)
- Anitha Pasupathy
National Institutes of Health Office of Research Infrastructure Programs (Grant RR-00166 to the Washington National Primate Research Center)
- Anitha Pasupathy
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Ethics
Animal experimentation: All animal procedures for this study, including implants, surgeries and behavioral training, conformed to NIH and USDA guidelines and were performed under an institutionally approved protocol at the Johns Hopkins University (Pasupathy and Connor, 2001) protocol #PR98A63 and the University of Washington (El-Shamayleh and Pasupathy, 2016) UW protocol #4133-01.
Copyright
© 2018, Pospisil et al.
This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.
Metrics
-
- 2,970
- views
-
- 396
- downloads
-
- 64
- citations
Views, downloads and citations are aggregated across all versions of this paper published by eLife.
Download links
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Further reading
-
- Neuroscience
Although parallel processing has been extensively studied in the low-level geniculostriate pathway and the high-level dorsal and ventral visual streams, less is known at the intermediate-level visual areas. In this study, we employed high-resolution fMRI at 7T to investigate the columnar and laminar organizations for color, disparity, and naturalistic texture in the human secondary visual cortex (V2), and their informational connectivity with lower- and higher-order visual areas. Although fMRI activations in V2 showed reproducible interdigitated color-selective thin and disparity-selective thick ‘stripe’ columns, we found no clear evidence of columnar organization for naturalistic textures. Cortical depth-dependent analyses revealed the strongest color-selectivity in the superficial layers of V2, along with both feedforward and feedback informational connectivity with V1 and V4. Disparity selectivity was similar across different cortical depths of V2, which showed significant feedforward and feedback connectivity with V1 and V3ab. Interestingly, the selectivity for naturalistic texture was strongest in the deep layers of V2, with significant feedback connectivity from V4. Thus, while local circuitry within cortical columns is crucial for processing color and disparity information, feedback signals from V4 are involved in generating the selectivity for naturalistic textures in area V2.
-
- Neuroscience
The hippocampus is a complex structure critically involved in numerous behavior-regulating systems. In young adults, multiple overlapping spatial modes along its longitudinal and transverse axes describe the organization of its functional integration with neocortex, extending the traditional framework emphasizing functional differences between sharply segregated hippocampal subregions. Yet, it remains unknown whether these modes (i.e. gradients) persist across the adult human lifespan, and relate to memory and molecular markers associated with brain function and cognition. In two independent samples, we demonstrate that the principal anteroposterior and second-order, mid-to-anterior/posterior hippocampal modes of neocortical functional connectivity, representing distinct dimensions of macroscale cortical organization, manifest across the adult lifespan. Specifically, individual differences in topography of the second-order gradient predicted episodic memory and mirrored dopamine D1 receptor distribution, capturing shared functional and molecular organization. Older age was associated with less distinct transitions along gradients (i.e. increased functional homogeneity). Importantly, a youth-like gradient profile predicted preserved episodic memory – emphasizing age-related gradient dedifferentiation as a marker of cognitive decline. Our results underscore a critical role of mapping multidimensional hippocampal organization in understanding the neural circuits that support memory across the adult lifespan.