Gated recurrence enables simple and accurate sequence prediction in stochastic, changing, and structured environments

Abstract
Data availability
Article and author information
Metrics

Abstract

From decision making to perception to language, predicting what is coming next is crucial. It is also challenging in stochastic, changing, and structured environments; yet the brain makes accurate predictions in many situations. What computational architecture could enable this feat? Bayesian inference makes optimal predictions but is prohibitively difficult to compute. Here, we show that a specific recurrent neural network architecture enables simple and accurate solutions in several environments. This architecture relies on three mechanisms: gating, lateral connections, and recurrent weight training. Like the optimal solution and the human brain, such networks develop internal representations of their changing environment (including estimates of the environment's latent variables and the precision of these estimates), leverage multiple levels of latent structure, and adapt their effective learning rate to changes without changing their connection weights. Being ubiquitous in the brain, gated recurrence could therefore serve as a generic building block to predict in real-life environments.

Data availability

This paper presents no experimental data. All synthetic data are available in the code repository at https://github.com/cedricfoucault/networks_for_sequence_prediction and archived on Zenodo with DOI: 10.5281/zenodo.5707498.

The following data sets were generated

1. Foucault C
(2021) Networks for sequence prediction
Publicly available at Github (https://github.com).

https://github.com/cedricfoucault/networks_for_sequence_prediction
1. Foucault C
(2021) Networks for sequence prediction
Publicly available at Zenodo (https://zenodo.org/).

http://dx.doi.org/10.5281/zenodo.5707498

Article and author information

Author details

Cédric Foucault

INSERM, CEA, Université Paris-Saclay, Gif sur Yvette, France

Competing interests
The authors declare that no competing interests exist.

"This ORCID iD identifies the author of this article:" 0000-0002-7247-6927
Florent Meyniel

NeuroSpin, CEA, Sorbonne Université, Gif sur Yvette, France

For correspondence
florent.meyniel@cea.fr

Competing interests
The authors declare that no competing interests exist.

"This ORCID iD identifies the author of this article:" 0000-0002-6992-678X

Funding

Ecole normale superieure Paris-Saclay (PhD fellowship Contrat doctoral spécifique normalien"")

Cédric Foucault

Agence Nationale de la Recherche (18-CE37-0010-01 CONFI LEARN"")

Florent Meyniel

H2020 European Research Council (ERC StG 947105 NEURAL PROB"")

Florent Meyniel

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Copyright

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.