Abstract
Convergent evidence suggeststhat the basal ganglia support reinforcement learning by adjusting action values according to reward prediction errors. However, adaptive behavior in stochasticenvironments requires the consideration of uncertainty to dynamically adjust the learning rate. We consider how cholinergic tonically active interneurons (TANs) may endow the striatum with such a mechanismin computational models spanning three Marr's levels of analysis. In the neural model, TANs modulate the excitability of spiny neurons, theirpopulation response to reinforcement, and hence the effective learning rate. Long TAN pauses facilitated robustness to spuriousoutcomes by increasing divergence in synaptic weights between neurons coding for alternative action values,whereas short TAN pauses facilitated stochastic behavior but increased responsiveness to change-points in outcome contingencies.A feedback control system allowed TAN pauses to be dynamically modulated by uncertainty across the spiny neuron population,allowing the system to self-tune and optimizeperformance across stochastic environments.
Article and author information
Author details
Reviewing Editor
- Upinder S Bhalla, National Centre for Biological Sciences, India
Publication history
- Received: October 3, 2015
- Accepted: December 24, 2015
- Accepted Manuscript published: December 25, 2015 (version 1)
- Accepted Manuscript updated: January 12, 2016 (version 2)
- Accepted Manuscript updated: January 13, 2016 (version 3)
- Version of Record published: February 10, 2016 (version 4)
Copyright
© 2015, Franklin & Frank
This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.
Metrics
-
- 2,181
- Page views
-
- 460
- Downloads
-
- 25
- Citations
Article citation count generated by polling the highest count across the following sources: Crossref, Scopus, PubMed Central.