Sensory cortex is optimised for prediction of future input

  1. Yosef Singer
  2. Yayoi Teramoto
  3. Ben DB Willmore
  4. Andrew J King
  5. Jan W H Schnupp
  6. Nicol S Harper  Is a corresponding author
  1. University of Oxford, United Kingdom
  2. City University of Hong Kong, Hong Kong

Abstract

Neurons in sensory cortex are tuned to diverse features in natural scenes. But what determines which features neurons become selective to? Here we explore the idea that neuronal selectivity is optimised to represent features in the recent sensory past that best predict immediate future inputs. We tested this hypothesis using simple feedforward neural networks, which were trained to predict the next few video or audio frames in clips of natural scenes. The networks developed receptive fields that closely matched those of real cortical neurons in different mammalian species, including the oriented spatial tuning of primary visual cortex, the frequency selectivity of primary auditory cortex and, most notably, their temporal tuning properties. Furthermore, the better a network predicted future inputs the more closely its receptive fields resembled those in the brain. This suggests that sensory processing is optimised to extract those features with the most capacity to predict future input.

Data availability

All custom code used in this study was implemented in MATLAB and Python. We have uploaded the code to a public Github repository. The raw auditory experimental data is available at  https://osf.io/ayw2p/. The movies and sounds used for training the models are all publicly available at the websites detailed in the Methods.

The following data sets were generated
    1. Jan Schnupp
    (2016) NetworkReceptiveFields
    Available at the Open Science Framework.

Article and author information

Author details

  1. Yosef Singer

    Department of Physiology, Anatomy and Genetics, University of Oxford, Oxford, United Kingdom
    Competing interests
    No competing interests declared.
  2. Yayoi Teramoto

    Department of Physiology, Anatomy and Genetics, University of Oxford, Oxford, United Kingdom
    Competing interests
    No competing interests declared.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-3419-0351
  3. Ben DB Willmore

    Department of Physiology, Anatomy and Genetics, University of Oxford, Oxford, United Kingdom
    Competing interests
    No competing interests declared.
  4. Andrew J King

    Department of Physiology, Anatomy and Genetics, University of Oxford, Oxford, United Kingdom
    Competing interests
    Andrew J King, Senior Editor, eLife.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-5180-7179
  5. Jan W H Schnupp

    Department of Biomedical Sciences, City University of Hong Kong, Hong Kong, Hong Kong
    Competing interests
    No competing interests declared.
  6. Nicol S Harper

    Department of Physiology, Anatomy and Genetics, University of Oxford, Oxford, United Kingdom
    For correspondence
    nicol.harper@dpag.ox.ac.uk
    Competing interests
    No competing interests declared.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-7851-4840

Funding

Clarendon Fund

  • Yosef Singer
  • Yayoi Teramoto

University Of Oxford

  • Nicol S Harper

Action on Hearing Loss (PA07)

  • Nicol S Harper

Biotechnology and Biological Sciences Research Council (BB/H008608/1)

  • Nicol S Harper

Wellcome (WT10525/Z/14/Z)

  • Yayoi Teramoto

Wellcome (WT076508AIA)

  • Ben DB Willmore

Wellcome (WT108369/Z/2015/Z)

  • Ben DB Willmore

Wellcome (WT076508AIA)

  • Andrew J King

Wellcome (WT108369/Z/2015/Z)

  • Andrew J King

Wellcome (WT082692)

  • Nicol S Harper

Wellcome (WT076508AIA)

  • Nicol S Harper

Wellcome (WT108369/Z/2015/Z)

  • Nicol S Harper

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Reviewing Editor

  1. Jack L Gallant, University of California, Berkeley, United States

Ethics

Animal experimentation: Auditory RFs of neurons were recorded in the primary auditory cortex (A1) and anterior auditory field (AAF) of 5 pigmented ferrets of both sexes (all > 6 months of age) and used as a basis for comparison with the RFs of model units trained on auditory stimuli. These recordings were performed under license from the UK Home Office and were approved by the University of Oxford Committee on Animal Care and Ethical Review. Full details of the recording methods are described in earlier studies [45,90]. Briefly, we induced general anaesthesia with a single intramuscular dose of medetomidine (0.022 mg · kg−1 · h−1) and ketamine (5 mg · kg−1 · h−1), which was then maintained with a continuous intravenous infusion of medetomidine and ketamine in saline. Oxygen was supplemented with a ventilator, and we monitored vital signs (body temperature, end-tidal CO2, and the electrocardiogram) throughout the experiment. The temporal muscles were retracted, a head holder was secured to the skull surface, and a craniotomy and a durotomy were made over the auditory cortex. Extracellular recordings were made using silicon probe electrodes (Neuronexus Technologies) and acoustic stimuli were presented via Panasonic RPHV27 earphones, which were coupled to otoscope specula that were inserted into each ear canal, and driven by Tucker-Davis Technologies System III hardware (48 kHz sample rate).

Version history

  1. Received: October 4, 2017
  2. Accepted: June 16, 2018
  3. Accepted Manuscript published: June 18, 2018 (version 1)
  4. Accepted Manuscript updated: June 22, 2018 (version 2)
  5. Version of Record published: August 24, 2018 (version 3)
  6. Version of Record updated: September 18, 2018 (version 4)

Copyright

© 2018, Singer et al.

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 7,675
    views
  • 1,130
    downloads
  • 53
    citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Yosef Singer
  2. Yayoi Teramoto
  3. Ben DB Willmore
  4. Andrew J King
  5. Jan W H Schnupp
  6. Nicol S Harper
(2018)
Sensory cortex is optimised for prediction of future input
eLife 7:e31557.
https://doi.org/10.7554/eLife.31557

Share this article

https://doi.org/10.7554/eLife.31557

Further reading

    1. Neuroscience
    Shanka Subhra Mondal, Steven Frankland ... Jonathan D Cohen
    Research Article

    Deep neural networks have made tremendous gains in emulating human-like intelligence, and have been used increasingly as ways of understanding how the brain may solve the complex computational problems on which this relies. However, these still fall short of, and therefore fail to provide insight into how the brain supports strong forms of generalization of which humans are capable. One such case is out-of-distribution (OOD) generalization – successful performance on test examples that lie outside the distribution of the training set. Here, we identify properties of processing in the brain that may contribute to this ability. We describe a two-part algorithm that draws on specific features of neural computation to achieve OOD generalization, and provide a proof of concept by evaluating performance on two challenging cognitive tasks. First we draw on the fact that the mammalian brain represents metric spaces using grid cell code (e.g., in the entorhinal cortex): abstract representations of relational structure, organized in recurring motifs that cover the representational space. Second, we propose an attentional mechanism that operates over the grid cell code using determinantal point process (DPP), that we call DPP attention (DPP-A) – a transformation that ensures maximum sparseness in the coverage of that space. We show that a loss function that combines standard task-optimized error with DPP-A can exploit the recurring motifs in the grid cell code, and can be integrated with common architectures to achieve strong OOD generalization performance on analogy and arithmetic tasks. This provides both an interpretation of how the grid cell code in the mammalian brain may contribute to generalization performance, and at the same time a potential means for improving such capabilities in artificial neural networks.

    1. Neuroscience
    Sanggeon Park, Yeowool Huh ... Jeiwon Cho
    Research Article

    The brain’s ability to appraise threats and execute appropriate defensive responses is essential for survival in a dynamic environment. Humans studies have implicated the anterior insular cortex (aIC) in subjective fear regulation and its abnormal activity in fear/anxiety disorders. However, the complex aIC connectivity patterns involved in regulating fear remain under investigated. To address this, we recorded single units in the aIC of freely moving male mice that had previously undergone auditory fear conditioning, assessed the effect of optogenetically activating specific aIC output structures in fear, and examined the organization of aIC neurons projecting to the specific structures with retrograde tracing. Single-unit recordings revealed that a balanced number of aIC pyramidal neurons’ activity either positively or negatively correlated with a conditioned tone-induced freezing (fear) response. Optogenetic manipulations of aIC pyramidal neuronal activity during conditioned tone presentation altered the expression of conditioned freezing. Neural tracing showed that non-overlapping populations of aIC neurons project to the amygdala or the medial thalamus, and the pathway bidirectionally modulated conditioned fear. Specifically, optogenetic stimulation of the aIC-amygdala pathway increased conditioned freezing, while optogenetic stimulation of the aIC-medial thalamus pathway decreased it. Our findings suggest that the balance of freezing-excited and freezing-inhibited neuronal activity in the aIC and the distinct efferent circuits interact collectively to modulate fear behavior.