Reward-based training of recurrent neural networks for cognitive and value-based tasks

Abstract
Article and author information
Metrics

Abstract

Trained neural network models, which exhibit features of neural activity recorded from behaving animals, may provide insights into the circuit mechanisms of cognitive functions through systematic analysis of network activity and connectivity. However, in contrast to the graded error signals commonly used to train networks through supervised learning, animals learn from reward feedback on definite actions through reinforcement learning. Reward maximization is particularly relevant when optimal behavior depends on an animal's internal judgment of confidence or subjective preferences. Here, we implement reward-based training of recurrent neural networks in which a value network guides learning by using the activity of the decision network to predict future reward. We show that such models capture behavioral and electrophysiological findings from well-known experimental paradigms. Our work provides a unified framework for investigating diverse cognitive and value-based computations, and predicts a role for value representation that is essential for learning, but not executing, a task.

Article and author information

Author details

H Francis Song

Center for Neural Science, New York University, New York, United States

Competing interests
The authors declare that no competing interests exist.
Guangyu R Yang

Center for Neural Science, New York University, New York, United States

Competing interests
The authors declare that no competing interests exist.
Xiao-Jing Wang

Center for Neural Science, New York University, New York, United States

For correspondence
xjwang@nyu.edu

Competing interests
The authors declare that no competing interests exist.

"This ORCID iD identifies the author of this article:" 0000-0003-3124-8474

Funding

Office of Naval Research (N00014-13-1-0297)

H Francis Song
Guangyu R Yang
Xiao-Jing Wang

Google

H Francis Song
Guangyu R Yang
Xiao-Jing Wang

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Copyright

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.