Monkey plays Pac-Man with compositional strategies and hierarchical decision-making

  1. Qianli Yang
  2. Zhongqiao Lin
  3. Wenyi Zhang
  4. Jianshu Li
  5. Xiyuan Chen
  6. Jiaqi Zhang
  7. Tianming Yang  Is a corresponding author
  1. Chinese Academy of Sciences, China
  2. Brown University, United States

Abstract

Humans can often handle daunting tasks with ease by developing a set of strategies to reduce decision making into simpler problems. The ability to use heuristic strategies demands an advanced level of intelligence and has not been demonstrated in animals. Here, we trained macaque monkeys to play the classic video game Pac-Man. The monkeys' decision-making may be described with a strategy-based hierarchical decision-making model with over 90% accuracy. The model reveals that the monkeys adopted the take-the-best heuristic by using one dominating strategy for their decision-making at a time and formed compound strategies by assembling the basis strategies to handle particular game situations. With the model, the computationally complex but fully quantifiable Pac-Man behavior paradigm provides a new approach to understanding animals’ advanced cognition.

Data availability

The data and codes that support the findings of this study are provided at: https://github.com/superr90/Monkey_PacMan.

The following data sets were generated

Article and author information

Author details

  1. Qianli Yang

    Institute of Neuroscience, Chinese Academy of Sciences, Shanghai, China
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-4226-2319
  2. Zhongqiao Lin

    Institute of Neuroscience, Chinese Academy of Sciences, Shanghai, China
    Competing interests
    The authors declare that no competing interests exist.
  3. Wenyi Zhang

    Institute of Neuroscience, Chinese Academy of Sciences, Shanghai, China
    Competing interests
    The authors declare that no competing interests exist.
  4. Jianshu Li

    Institute of Neuroscience, Chinese Academy of Sciences, Shanghai, China
    Competing interests
    The authors declare that no competing interests exist.
  5. Xiyuan Chen

    Institute of Neuroscience, Chinese Academy of Sciences, Shanghai, China
    Competing interests
    The authors declare that no competing interests exist.
  6. Jiaqi Zhang

    Brown University, Providence, United States
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-1649-3378
  7. Tianming Yang

    Institute of Neuroscience, Chinese Academy of Sciences, Shanghai, China
    For correspondence
    tyang@ion.ac.cn
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-6976-9246

Funding

Chinese Academy of Sciences (XDB32070100)

  • Tianming Yang

Shanghai Municipal Science and Technology Major Project (2018SHZDZX05)

  • Tianming Yang

National Natural Science Foundation of China (32100832)

  • Qianli Yang

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Ethics

Animal experimentation: All procedures followed the protocol approved by the Animal Care Committee of Shanghai Institutes for Biological Sciences, Chinese Academy of Sciences (CEBSIT-2021004).

Copyright

© 2022, Yang et al.

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 3,788
    views
  • 496
    downloads
  • 9
    citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Qianli Yang
  2. Zhongqiao Lin
  3. Wenyi Zhang
  4. Jianshu Li
  5. Xiyuan Chen
  6. Jiaqi Zhang
  7. Tianming Yang
(2022)
Monkey plays Pac-Man with compositional strategies and hierarchical decision-making
eLife 11:e74500.
https://doi.org/10.7554/eLife.74500

Share this article

https://doi.org/10.7554/eLife.74500

Further reading

    1. Neuroscience
    Nico A Flierman, Sue Ann Koay ... Chris I De Zeeuw
    Research Article

    The role of cerebellum in controlling eye movements is well established, but its contribution to more complex forms of visual behavior has remained elusive. To study cerebellar activity during visual attention we recorded extracellular activity of dentate nucleus (DN) neurons in two non-human primates (NHPs). NHPs were trained to read the direction indicated by a peripheral visual stimulus while maintaining fixation at the center, and report the direction of the cue by performing a saccadic eye movement into the same direction following a delay. We found that single-unit DN neurons modulated spiking activity over the entire time course of the task, and that their activity often bridged temporally separated intra-trial events, yet in a heterogeneous manner. To better understand the heterogeneous relationship between task structure, behavioral performance, and neural dynamics, we constructed a behavioral, an encoding, and a decoding model. Both NHPs showed different behavioral strategies, which influenced the performance. Activity of the DN neurons reflected the unique strategies, with the direction of the visual stimulus frequently being encoded long before an upcoming saccade. Moreover, the latency of the ramping activity of DN neurons following presentation of the visual stimulus was shorter in the better performing NHP. Labeling with the retrograde tracer Cholera Toxin B in the recording location in the DN indicated that these neurons predominantly receive inputs from Purkinje cells in the D1 and D2 zones of the lateral cerebellum as well as neurons of the principal olive and medial pons, all regions known to connect with neurons in the prefrontal cortex contributing to planning of saccades. Together, our results highlight that DN neurons can dynamically modulate their activity during a visual attention task, comprising not only sensorimotor but also cognitive attentional components.

    1. Neuroscience
    Robert A Bruce, Matthew Weber ... Kumar Narayanan
    Research Article

    The role of striatal pathways in cognitive processing is unclear. We studied dorsomedial striatal cognitive processing during interval timing, an elementary cognitive task that requires mice to estimate intervals of several seconds and involves working memory for temporal rules as well as attention to the passage of time. We harnessed optogenetic tagging to record from striatal D2-dopamine receptor-expressing medium spiny neurons (D2-MSNs) in the indirect pathway and from D1-dopamine receptor-expressing MSNs (D1-MSNs) in the direct pathway. We found that D2-MSNs and D1-MSNs exhibited distinct dynamics over temporal intervals as quantified by principal component analyses and trial-by-trial generalized linear models. MSN recordings helped construct and constrain a four-parameter drift-diffusion computational model in which MSN ensemble activity represented the accumulation of temporal evidence. This model predicted that disrupting either D2-MSNs or D1-MSNs would increase interval timing response times and alter MSN firing. In line with this prediction, we found that optogenetic inhibition or pharmacological disruption of either D2-MSNs or D1-MSNs increased interval timing response times. Pharmacologically disrupting D2-MSNs or D1-MSNs also changed MSN dynamics and degraded trial-by-trial temporal decoding. Together, our findings demonstrate that D2-MSNs and D1-MSNs had opposing dynamics yet played complementary cognitive roles, implying that striatal direct and indirect pathways work together to shape temporal control of action. These data provide novel insight into basal ganglia cognitive operations beyond movement and have implications for human striatal diseases and therapies targeting striatal pathways.