1. Computational and Systems Biology
  2. Neuroscience
Download icon

A cholinergic feedback circuit to regulate striatal population uncertainty and optimize reinforcement learning

  1. Nicholas T Franklin
  2. Michael J Frank  Is a corresponding author
  1. Brown University, United States
Research Article
  • Cited 25
  • Views 2,188
  • Annotations
Cite this article as: eLife 2015;4:e12029 doi: 10.7554/eLife.12029

Abstract

Convergent evidence suggests that the basal ganglia support reinforcement learning by adjusting action values according to reward prediction errors. However, adaptive behavior in stochastic environments requires the consideration of uncertainty to dynamically adjust the learning rate. We consider how cholinergic tonically active interneurons (TANs) may endow the striatum with such a mechanism in computational models spanning three Marr's levels of analysis. In the neural model, TANs modulate the excitability of spiny neurons, their population response to reinforcement, and hence the effective learning rate. Long TAN pauses facilitated robustness to spurious outcomes by increasing divergence in synaptic weights between neurons coding for alternative action values,whereas short TAN pauses facilitated stochastic behavior but increased responsiveness to change-points in outcome contingencies.A feedback control system allowed TAN pauses to be dynamically modulated by uncertainty across the spiny neuron population,allowing the system to self-tune and optimize performance across stochastic environments.

Article and author information

Author details

  1. Nicholas T Franklin

    Department of Cognitive, Linguistic and Psychological Sciences, Brown Institute for Brain Science, Brown University, Providence, United States
    Competing interests
    No competing interests declared.
  2. Michael J Frank

    Department of Cognitive, Linguistic and Psychological Sciences, Brown Institute for Brain Science, Brown University, Providence, United States
    For correspondence
    Michael_Frank@brown.edu
    Competing interests
    Michael J Frank, Reviewing editor, eLife.

Reviewing Editor

  1. Upinder S Bhalla, National Centre for Biological Sciences, India

Publication history

  1. Received: October 3, 2015
  2. Accepted: December 24, 2015
  3. Accepted Manuscript published: December 25, 2015 (version 1)
  4. Accepted Manuscript updated: January 12, 2016 (version 2)
  5. Accepted Manuscript updated: January 13, 2016 (version 3)
  6. Version of Record published: February 10, 2016 (version 4)

Copyright

© 2015, Franklin & Frank

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 2,188
    Page views
  • 460
    Downloads
  • 25
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, Scopus, PubMed Central.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Download citations (links to download the citations from this article in formats compatible with various reference manager tools)

Open citations (links to open the citations from this article in various online reference manager services)

Further reading

    1. Chromosomes and Gene Expression
    2. Computational and Systems Biology
    Michael Roland Wolff et al.
    Research Article

    Chromatin dynamics are mediated by remodeling enzymes and play crucial roles in gene regulation, as established in a paradigmatic model, the S. cerevisiae PHO5 promoter. However, effective nucleosome dynamics, i.e. trajectories of promoter nucleosome configurations, remain elusive. Here, we infer such dynamics from the integration of published single-molecule data capturing multi-nucleosome configurations for repressed to fully active PHO5 promoter states with other existing histone turnover and new chromatin accessibility data. We devised and systematically investigated a new class of 'regulated on-off-slide' models simulating global and local nucleosome (dis)assembly and sliding. Only seven of 68145 models agreed well with all data. All seven models involve sliding and the known central role of the N-2 nucleosome, but regulate promoter state transitions by modulating just one assembly rather than disassembly process. This is consistent with but challenges common interpretations of previous observations at the PHO5 promoter and suggests chromatin opening by binding competition.

    1. Cell Biology
    2. Computational and Systems Biology
    Taraneh Zarin et al.
    Research Advance Updated

    In previous work, we showed that intrinsically disordered regions (IDRs) of proteins contain sequence-distributed molecular features that are conserved over evolution, despite little sequence similarity that can be detected in alignments (Zarin et al., 2019). Here, we aim to use these molecular features to predict specific biological functions for individual IDRs and identify the molecular features within them that are associated with these functions. We find that the predictable functions are diverse. Examining the associated molecular features, we note some that are consistent with previous reports and identify others that were previously unknown. We experimentally confirm that elevated isoelectric point and hydrophobicity, features that are positively associated with mitochondrial localization, are necessary for mitochondrial targeting function. Remarkably, increasing isoelectric point in a synthetic IDR restores weak mitochondrial targeting. We believe feature analysis represents a new systematic approach to understand how biological functions of IDRs are specified by their protein sequences.