Behavioral evidence for memory replay of video episodes in the macaque
- Shanghai Key Laboratory of Brain Functional Genomics, Key Laboratory of Brain Functional Genomics Ministry of Education, School of Psychology and Cognitive Science, East China Normal University, China
- Program of Brain and Cognitive Engineering, Korea Advanced Institute of Science and Technology, Republic of Korea
- Department of Bio and Brain Engineering, Korea Advanced Institute of Science and Technology, Republic of Korea
- Department of Computer Science, University of York, United Kingdom
- School of Psychology, Shenzhen University, China
- Shanghai Key Laboratory of Magnetic Resonance, East China Normal University, China
- NYU-ECNU Institute of Brain and Cognitive Science at NYU Shanghai, China
Figures
Figure 1 with 1 supplement
TOJ task schema and RT results.
(A) In each trial, the monkey watched a video (8–12 s, comprising two 4–6 s video clips), and following a 2 s retention delay, made temporal order judgement between two probe frames extracted from …
Figure 1—figure supplement 1
Performance of human participants and speed accuracy trade off results.
(A) (Left) Task performance of seven human participants. The proportions of correct responses for the across-context condition are significantly higher than those for the within-context condition …
Figure 2 with 1 supplement
Moving average analysis based on reciprocal latency and accuracy for monkeys (left panel) and human participants (right panel).
(A) Reciprocal latency for monkeys as a function of chosen frame location for the average of all animals (upper panel) in the within-context condition, with the results for six individual monkeys …
Figure 2—figure supplement 1
Relationship between temporal similarity and reciprocal latency for within-context trials in (A) monkeys and (B) humans.
Reciprocal latency as a function of temporal similarity for the average of all individuals (upper panel) and for each individual (bottom panel). The temporal similarity between two frames is …
Figure 3
Model comparison using representative similarity analysis.
(A) Visualization of two candidate models as representational dissimilarity matrices (RDMs). Patterns of reaction time (rank-transformed values) as a function of chosen frame location for the the …
Figure 4
The Strict forward model provides a better fit to the RT data in monkeys but not in humans.
(A) ‘Offsets’ are defined as the magnitude of reduced RT when the frames were in Clip 2. 11 hypothetical models with their reaction time patterns (top) and RDMs (bottom). We systemically varied the …
Figure 5
LATER model fitting of RT in across-context and within-context conditions for both species.
(A) Cartoon of the LATER model cartoon illustrating that a decision signal triggered by a stimulus rises from its start level, at a rate of information accumulation r, to the threshold. Once it …
Figure 6 with 1 supplement
Full GLM analysis including a number of variables that might affect reciprocal latency separately for within-context and across-context conditions.
(A) Monkey data. (B) Human data. We included ten regressors, namely, a binary regressor indicating whether the video category is primate or non-primate (video category), a binary regressor …
Figure 6—figure supplement 1
GLM results on the effects of image similarity measures on reciprocal latency for within-context and across-context conditions for (A) the monkeys and (B) humans, and (C) an example of SURF similarity.
Among the several indices (difference of distribution in RGB-histogram, Histogram of Oriented Gradients (HOG) similarity and SURF similarity), the SURF similarity measure was significantly …
Figure 7
Sessional accuracy data expressed as proportion correct for each individual.
(A) Monkey data. (B) Human data. No obvious increase in performance was observed over the course of testing days in the experiment for either monkeys or humans.
Videos
Video 1
A monkey performing an example trial.
The monkey performs an across-context trial with a correct response (rewarded with liquid).
Tables
Table 1
One sample t-tests of the slopes of reciprocal latency as a function of chosen frame location for each monkey after having entered a range of nuisance variables as regressor-of-no-interest (see also Figure 6).
The three panels correspond to analyses performed using all trials (top), only correct trials (middle), and only incorrect trials (bottom). The same slope patterns were observed irrespective of …
| Monkeys | Beta | SEM | t-statistics | p-value | 95% confidence interval Lower Upper | |
|---|---|---|---|---|---|---|
| Slope of reciprocal latency/chosen frame location tested against zero (all trials) | ||||||
| Jupiter | –0.203 | 0.021 | –9.751 | <0.001 | –0.244 | –0.163 |
| Mars | –0.369 | 0.025 | –14.950 | <0.001 | –0.417 | –0.320 |
| Saturn | –0.157 | 0.027 | –5.810 | <0.001 | –0.210 | –0.104 |
| Mercury | –0.207 | 0.052 | –3.958 | <0.001 | –0.309 | –0.104 |
| Uranus | –0.164 | 0.022 | –7.595 | <0.001 | –0.207 | –0.122 |
| Neptune | –0.197 | 0.031 | –6.361 | <0.001 | –0.257 | –0.136 |
| Slope of reciprocal latency/chosen frame location tested against zero (correct trials) | ||||||
| Jupiter | –0.185 | 0.025 | –7.393 | <0.001 | –0.234 | –0.136 |
| Mars | –0.272 | 0.028 | –9.879 | <0.001 | –0.326 | –0.218 |
| Saturn | –0.092 | 0.032 | –2.857 | 0.004 | –0.155 | –0.029 |
| Mercury | –0.246 | 0.065 | –3.777 | <0.001 | –0.374 | –0.118 |
| Uranus | –0.153 | 0.024 | –6.259 | <0.001 | –0.201 | –0.105 |
| Neptune | –0.150 | 0.039 | –3.858 | <0.001 | –0.226 | –0.074 |
| Slope of reciprocal latency/chosen frame location tested against zero (Incorrect trials) | ||||||
| Jupiter | –0.175 | 0.027 | –6.619 | <0.001 | –0.227 | –0.123 |
| Mars | –0.366 | 0.031 | –11.705 | <0.001 | –0.428 | –0.305 |
| Saturn | –0.191 | 0.035 | –5.386 | 0.002 | –0.261 | –0.122 |
| Mercury | –0.075 | 0.077 | –0.975 | 0.330 | –0.227 | 0.076 |
| Uranus | –0.140 | 0.029 | –4.816 | <0.001 | –0.197 | –0.083 |
| Neptune | –0.209 | 0.041 | –5.148 | <0.001 | –0.288 | –0.129 |
Table 2
Number of days accrued in each training stage.
When we trained the two additional monkeys (Uranus and Neptune), we made them skip the 4th and 5th stages entirely. The performance patterns of these two monkeys were not different from those of the …
| Monkeys | Stage 1 | Stage 2 | Stage 3 | Stage 4 | Stage 5 |
|---|---|---|---|---|---|
| Jupiter | 12 | 39 | 16 | 65 | 51 |
| Mars | 29 | 7 | 15 | 48 | 42 |
| Saturn | 25 | 42 | 31 | 64 | 95 |
| Mercury | 13 | 38 | 17 | 61 | 49 |
| Uranus | 14 | 17 | 11 | - | - |
| Neptune | 13 | 21 | 11 | - | - |
Additional files
-
Source data 1
Source data for all figures and tables.
- https://cdn.elifesciences.org/articles/54519/elife-54519-data1-v2.xlsx
-
Supplementary file 1
Data description tables used to illustrate all of the key variables contained in the ‘Source Data 1.xlsx’.
- https://cdn.elifesciences.org/articles/54519/elife-54519-supp1-v2.docx
-
Supplementary file 2
For monkeys: one sample t-test results for the slopes of reciprocal latency as a function of temporal similarity, having entered a range of nuisance variables as regressor-of-no-interest.
The three panels correspond to analyses performed using all trials (top), only correct trials (middle), and only incorrect trials (bottom). The same slope patterns were observed irrespective of correctness, as is consistent with the analyses of slopes of reciprocal latency as a function of chosen frame location for each monkey. Related to Figure 2—figure supplement 1A.
- https://cdn.elifesciences.org/articles/54519/elife-54519-supp2-v2.docx
-
Supplementary file 3
Hierarchical multiple regression results for individual monkeys (left panel) and for human participants (right panel) showing reaction time as a function of chosen frame location (CFL).
- https://cdn.elifesciences.org/articles/54519/elife-54519-supp3-v2.docx
-
Supplementary file 4
For human participants: one sample t-test results for the slopes of reciprocal latency as a function of temporal similarity (upper panel) and the slopes of reciprocal latency as a function of chosen frame location (bottom panel) against zero, after having entered a range of nuisance variables as regressor-of-no-interest.
Related to Figure 2—figure supplement 1B.
- https://cdn.elifesciences.org/articles/54519/elife-54519-supp4-v2.docx
-
Supplementary file 5
LATER model fitting results of six monkeys and seven human participants.
For ease of comparison, we computed the respective ΔBIC to index the strength of evidence for each model. Note that the model with the lowest BIC is the winning model. In all 6 monkeys, the shift model is superior to the other three models, whereas this effect is not consistent in the humans. Related to Figure 5.
- https://cdn.elifesciences.org/articles/54519/elife-54519-supp5-v2.docx
-
Transparent reporting form
- https://cdn.elifesciences.org/articles/54519/elife-54519-transrepform-v2.docx
