A model of hippocampal replay driven by experience and environmental structure facilitates spatial learning
Abstract
Replay of neuronal sequences in the hippocampus during resting states and sleep play an important role in learning and memory consolidation. Consistent with these functions, replay sequences have been shown to obey current spatial constraints. Nevertheless, replay does not necessarily reflect previous behavior and can construct never-experienced sequences. Here we propose a stochastic replay mechanism that prioritizes experiences based on three variables: 1. Experience strength, 2. experience similarity, and 3. inhibition of return. Using this prioritized replay mechanism to train reinforcement learning agents leads to far better performance than using random replay. Its performance is close to the state-of-the-art, but computationally intensive, algorithm by Mattar & Daw (2018). Importantly, our model reproduces diverse types of replay because of the stochasticity of the replay mechanism and experience-dependent differences between the three variables. In conclusion, a unified replay mechanism generates diverse replay statistics and is efficient in driving spatial learning.
Data availability
The current manuscript is a computational study, so no data have been generated for this manuscript. Modelling code has been made publicly available at https://github.com/sencheng/-Mechanisms-and-Functions-of-Hippocampal-Replay.
Article and author information
Author details
Funding
Deutsche Forschungsgemeinschaft (419037518 - FOR 2812 P2)
- Sen Cheng
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Reviewing Editor
- Payam Piray, University of Southern California, United States
Publication history
- Received: July 29, 2022
- Accepted: March 13, 2023
- Accepted Manuscript published: March 14, 2023 (version 1)
Copyright
© 2023, Diekmann & Cheng
This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.
Metrics
-
- 298
- Page views
-
- 42
- Downloads
-
- 0
- Citations
Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.
Download links
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Further reading
-
- Neuroscience
Sensory neurons previously shown to optimize speed and balance in fish by providing information about the curvature of the spine show similar morphology and connectivity in mice.
-
- Neuroscience
Beta oscillations in human sensorimotor cortex are hallmark signatures of healthy and pathological movement. In single trials, beta oscillations include bursts of intermittent, transient periods of high-power activity. These burst events have been linked to a range of sensory and motor processes, but their precise spatial, spectral, and temporal structure remains unclear. Specifically, a role for beta burst activity in information coding and communication suggests spatiotemporal patterns, or travelling wave activity, along specific anatomical gradients. We here show in human magnetoencephalography recordings that burst activity in sensorimotor cortex occurs in planar spatiotemporal wave-like patterns that dominate along two axes either parallel or perpendicular to the central sulcus. Moreover, we find that the two propagation directions are characterised by distinct anatomical and physiological features. Finally, our results suggest that sensorimotor beta bursts occurring before and after a movement can be distinguished by their anatomical, spectral and spatiotemporal characteristics, indicating distinct functional roles.