Sensory cortex is optimised for prediction of future input
Abstract
Neurons in sensory cortex are tuned to diverse features in natural scenes. But what determines which features neurons become selective to? Here we explore the idea that neuronal selectivity is optimised to represent features in the recent sensory past that best predict immediate future inputs. We tested this hypothesis using simple feedforward neural networks, which were trained to predict the next few video or audio frames in clips of natural scenes. The networks developed receptive fields that closely matched those of real cortical neurons in different mammalian species, including the oriented spatial tuning of primary visual cortex, the frequency selectivity of primary auditory cortex and, most notably, their temporal tuning properties. Furthermore, the better a network predicted future inputs the more closely its receptive fields resembled those in the brain. This suggests that sensory processing is optimised to extract those features with the most capacity to predict future input.
Data availability
All custom code used in this study was implemented in MATLAB and Python. We have uploaded the code to a public Github repository. The raw auditory experimental data is available at https://osf.io/ayw2p/. The movies and sounds used for training the models are all publicly available at the websites detailed in the Methods.
Article and author information
Author details
Funding
Clarendon Fund
- Yosef Singer
- Yayoi Teramoto
University Of Oxford
- Nicol S Harper
Action on Hearing Loss (PA07)
- Nicol S Harper
Biotechnology and Biological Sciences Research Council (BB/H008608/1)
- Nicol S Harper
Wellcome (WT10525/Z/14/Z)
- Yayoi Teramoto
Wellcome (WT076508AIA)
- Ben DB Willmore
Wellcome (WT108369/Z/2015/Z)
- Ben DB Willmore
Wellcome (WT076508AIA)
- Andrew J King
Wellcome (WT108369/Z/2015/Z)
- Andrew J King
Wellcome (WT082692)
- Nicol S Harper
Wellcome (WT076508AIA)
- Nicol S Harper
Wellcome (WT108369/Z/2015/Z)
- Nicol S Harper
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Reviewing Editor
- Jack L Gallant, University of California, Berkeley, United States
Ethics
Animal experimentation: Auditory RFs of neurons were recorded in the primary auditory cortex (A1) and anterior auditory field (AAF) of 5 pigmented ferrets of both sexes (all > 6 months of age) and used as a basis for comparison with the RFs of model units trained on auditory stimuli. These recordings were performed under license from the UK Home Office and were approved by the University of Oxford Committee on Animal Care and Ethical Review. Full details of the recording methods are described in earlier studies [45,90]. Briefly, we induced general anaesthesia with a single intramuscular dose of medetomidine (0.022 mg · kg−1 · h−1) and ketamine (5 mg · kg−1 · h−1), which was then maintained with a continuous intravenous infusion of medetomidine and ketamine in saline. Oxygen was supplemented with a ventilator, and we monitored vital signs (body temperature, end-tidal CO2, and the electrocardiogram) throughout the experiment. The temporal muscles were retracted, a head holder was secured to the skull surface, and a craniotomy and a durotomy were made over the auditory cortex. Extracellular recordings were made using silicon probe electrodes (Neuronexus Technologies) and acoustic stimuli were presented via Panasonic RPHV27 earphones, which were coupled to otoscope specula that were inserted into each ear canal, and driven by Tucker-Davis Technologies System III hardware (48 kHz sample rate).
Version history
- Received: October 4, 2017
- Accepted: June 16, 2018
- Accepted Manuscript published: June 18, 2018 (version 1)
- Accepted Manuscript updated: June 22, 2018 (version 2)
- Version of Record published: August 24, 2018 (version 3)
- Version of Record updated: September 18, 2018 (version 4)
Copyright
© 2018, Singer et al.
This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.
Metrics
-
- 7,675
- views
-
- 1,130
- downloads
-
- 53
- citations
Views, downloads and citations are aggregated across all versions of this paper published by eLife.
Download links
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Further reading
-
- Neuroscience
Deep neural networks have made tremendous gains in emulating human-like intelligence, and have been used increasingly as ways of understanding how the brain may solve the complex computational problems on which this relies. However, these still fall short of, and therefore fail to provide insight into how the brain supports strong forms of generalization of which humans are capable. One such case is out-of-distribution (OOD) generalization – successful performance on test examples that lie outside the distribution of the training set. Here, we identify properties of processing in the brain that may contribute to this ability. We describe a two-part algorithm that draws on specific features of neural computation to achieve OOD generalization, and provide a proof of concept by evaluating performance on two challenging cognitive tasks. First we draw on the fact that the mammalian brain represents metric spaces using grid cell code (e.g., in the entorhinal cortex): abstract representations of relational structure, organized in recurring motifs that cover the representational space. Second, we propose an attentional mechanism that operates over the grid cell code using determinantal point process (DPP), that we call DPP attention (DPP-A) – a transformation that ensures maximum sparseness in the coverage of that space. We show that a loss function that combines standard task-optimized error with DPP-A can exploit the recurring motifs in the grid cell code, and can be integrated with common architectures to achieve strong OOD generalization performance on analogy and arithmetic tasks. This provides both an interpretation of how the grid cell code in the mammalian brain may contribute to generalization performance, and at the same time a potential means for improving such capabilities in artificial neural networks.
-
- Neuroscience
The brain’s ability to appraise threats and execute appropriate defensive responses is essential for survival in a dynamic environment. Humans studies have implicated the anterior insular cortex (aIC) in subjective fear regulation and its abnormal activity in fear/anxiety disorders. However, the complex aIC connectivity patterns involved in regulating fear remain under investigated. To address this, we recorded single units in the aIC of freely moving male mice that had previously undergone auditory fear conditioning, assessed the effect of optogenetically activating specific aIC output structures in fear, and examined the organization of aIC neurons projecting to the specific structures with retrograde tracing. Single-unit recordings revealed that a balanced number of aIC pyramidal neurons’ activity either positively or negatively correlated with a conditioned tone-induced freezing (fear) response. Optogenetic manipulations of aIC pyramidal neuronal activity during conditioned tone presentation altered the expression of conditioned freezing. Neural tracing showed that non-overlapping populations of aIC neurons project to the amygdala or the medial thalamus, and the pathway bidirectionally modulated conditioned fear. Specifically, optogenetic stimulation of the aIC-amygdala pathway increased conditioned freezing, while optogenetic stimulation of the aIC-medial thalamus pathway decreased it. Our findings suggest that the balance of freezing-excited and freezing-inhibited neuronal activity in the aIC and the distinct efferent circuits interact collectively to modulate fear behavior.