Local online learning in recurrent networks with random feedback

  1. James M Murray  Is a corresponding author
  1. Columbia University, United States

Abstract

Recurrent neural networks (RNNs) enable the production and processing of time-dependent signals such as those involved in movement and working memory. Classic gradient-based algorithms for training RNNs have been available for decades, but are inconsistent with biological features of the brain, such as causality and locality. We derive an approximation to gradient-based learning that comports with these constraints by requiring synaptic weight updates to depend only on local information about pre- and postsynaptic activities, in addition to a random feedback projection of the RNN output error. In addition to providing mathematical arguments for the effectiveness of the new learning rule, we show through simulations that it can be used to train an RNN to perform a variety of tasks. Finally, to overcome the difficulty of training over very large numbers of timesteps, we propose an augmented circuit architecture that allows the RNN to concatenate short-duration patterns into longer sequences.

Data availability

Code implementing the RFLO learning algorithm for the example shown in Figure 2 has been included as a source code file accompanying this manuscript.

Article and author information

Author details

  1. James M Murray

    Zuckerman Mind, Brain, and Behavior Institute, Columbia University, New York, United States
    For correspondence
    jm4347@columbia.edu
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-3706-4895

Funding

National Institutes of Health (DP5 OD019897)

  • James M Murray

National Science Foundation (DBI-1707398)

  • James M Murray

Gatsby Charitable Foundation

  • James M Murray

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Reviewing Editor

  1. Peter Latham, University College London, United Kingdom

Version history

  1. Received: November 1, 2018
  2. Accepted: May 23, 2019
  3. Accepted Manuscript published: May 24, 2019 (version 1)
  4. Accepted Manuscript updated: May 31, 2019 (version 2)
  5. Version of Record published: June 12, 2019 (version 3)

Copyright

© 2019, Murray

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 5,355
    views
  • 861
    downloads
  • 45
    citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. James M Murray
(2019)
Local online learning in recurrent networks with random feedback
eLife 8:e43299.
https://doi.org/10.7554/eLife.43299

Share this article

https://doi.org/10.7554/eLife.43299

Further reading

    1. Neuroscience
    Nicola Masala, Manuel Mittag ... Tony Kelly
    Research Article

    Genetically encoded calcium indicators (GECIs) such as GCaMP are invaluable tools in neuroscience to monitor neuronal activity using optical imaging. The viral transduction of GECIs is commonly used to target expression to specific brain regions, can be conveniently used with any mouse strain of interest without the need for prior crossing with a GECI mouse line, and avoids potential hazards due to the chronic expression of GECIs during development. A key requirement for monitoring neuronal activity with an indicator is that the indicator itself minimally affects activity. Here, using common adeno-associated viral (AAV) transduction procedures, we describe spatially confined aberrant Ca2+ microwaves slowly travelling through the hippocampus following expression of GCaMP6, GCaMP7, or R-CaMP1.07 driven by the synapsin promoter with AAV-dependent gene transfer in a titre-dependent fashion. Ca2+ microwaves developed in hippocampal CA1 and CA3, but not dentate gyrus nor neocortex, were typically first observed at 4 wk after viral transduction, and persisted up to at least 8 wk. The phenomenon was robust and observed across laboratories with various experimenters and setups. Our results indicate that aberrant hippocampal Ca2+ microwaves depend on the promoter and viral titre of the GECI, density of expression, as well as the targeted brain region. We used an alternative viral transduction method of GCaMP which avoids this artefact. The results show that commonly used Ca2+-indicator AAV transduction procedures can produce artefactual Ca2+ responses. Our aim is to raise awareness in the field of these artefactual transduction-induced Ca2+ microwaves, and we provide a potential solution.

    1. Neuroscience
    John J Stout, Allison E George ... Amy L Griffin
    Research Article

    Functional interactions between the prefrontal cortex and hippocampus, as revealed by strong oscillatory synchronization in the theta (6–11 Hz) frequency range, correlate with memory-guided decision-making. However, the degree to which this form of long-range synchronization influences memory-guided choice remains unclear. We developed a brain-machine interface that initiated task trials based on the magnitude of prefrontal-hippocampal theta synchronization, then measured choice outcomes. Trials initiated based on strong prefrontal-hippocampal theta synchrony were more likely to be correct compared to control trials on both working memory-dependent and -independent tasks. Prefrontal-thalamic neural interactions increased with prefrontal-hippocampal synchrony and optogenetic activation of the ventral midline thalamus primarily entrained prefrontal theta rhythms, but dynamically modulated synchrony. Together, our results show that prefrontal-hippocampal theta synchronization leads to a higher probability of a correct choice and strengthens prefrontal-thalamic dialogue. Our findings reveal new insights into the neural circuit dynamics underlying memory-guided choices and highlight a promising technique to potentiate cognitive processes or behavior via brain-machine interfacing.