Successor-like representation guides the prediction of future events in human visual cortex and hippocampus
- Radboud University Nijmegen, Donders Institute for Brain, Cognition and Behaviour, Netherlands
Figures
Figure 1
Sequence paradigm to probe successor-like representations.
(a) Stimulus timing for full sequence trials (top) and partial sequence trials (bottom). During full sequence trials, four dots were presented in rapid succession in a fixed sequence order (A-B-C-D). During partial sequence trials, only one of the four dots was presented, omitting the remaining sequence dots. Here shown for -B - -, while A- - -, - -C-, and - - -D partial trials were also presented. (b) Sequences were randomized across subjects such that sequence locations were sampled from a total of eight possible locations with the constraint that every quadrant was stimulated once. Dot locations were evenly spaced around central fixation at a radius of 7 degrees visual angle (dva). (c) Independent stimulus localizer trials to map out stimulus representations.
Figure 2
V1 stimulus mapping.
(a) An independent stimulus localizer was used to identify V1 subpopulations that respond to individual dot locations (left). Stimulus-response profiles show tuning properties for selected V1 populations (middle). Visualizing stimulus activity by projecting group averaged BOLD activity (n=35) into stimulus space (right) shows focal activity at the stimulated location with minimal spreading to neighboring locations. (b) Identified V1 subpopulations during full sequence trials (left) show heightened BOLD activity compared to non-stimulated control locations (middle). Group averaged (n=7) sequence activity projected into stimulus space shows spatially specific activity at the stimulated locations (right).
Figure 3
Successor-like representation of future sequence events in V1.
(a) BOLD activity during full sequence trials. (b) Schematic of all partial sequence trials (left) illustrating the omission of different predecessor (purple), or successor (orange) sequence locations. Group averaged (n=35) V1 activity during partial sequence trials (right) shows enhanced activation of successor locations compared to predecessor locations. (c) Group averaged V1 activity for individual partial sequence trials. Error bars denote ± s.e.m.; two-tailed t test, ***p < 0.001; **p < 0.01; *p < 0.05, uncorrected for multiple comparisons.
Figure 4
Model comparison favors successor-like representation in V1.
(a) Probing predictions of the successor representation (SR) against the competing co-occurrence (CO) model. The relational structure of the full sequence A-B-C-D is translated into a transition matrix (top), where a non-zero value indicates a transition between two states in the sequence. The SR matrix (bottom) is computed from the transition matrix, here shown with a temporal discount factor of γ = 0.3 (see Materials and methods). (b) The relational structure in the CO model is non-directional, resulting in a constant prediction of past and future states weighted by a factor ω. (c) Competing model predictions were fitted to partial sequence trial V1 data of each individual participant with γ and ω as free parameters. Comparison of model errors showed that the data is most in line with the SR. A null model (bottom), resembling no prediction of past and future locations, was included in the model comparison as baseline. Error bars denote ± s.e.m.; BIC, Bayesian Information Criterion (taking into account that the H0 model has fewer parameters). Two-tailed t test, ***p < 0.001, uncorrected for multiple comparisons.
Figure 5 with 1 supplement
Hippocampus represents spatial locations and engages in future-directed predictions.
(a) Hippocampus region of interest (green). (b) A pattern classifier was trained to distinguish between the eight stimulus locations during a perceptual localizer. Resulting stimulus-response profiles reveal that hippocampus distinguishes between individual stimulus locations. (c) Averaged (n=35) tuning profiles shifted to one location. (d) A classifier that was trained on the perceptual localizer was applied to partial sequence trials during the main task to probe whether hippocampal representations skew toward predecessor locations (purple), or successor locations (orange). (e) Classifier evidence, averaged across possible successor and predecessor locations, shows that hippocampus predominantly represents future (successor) stimulus locations over predecessor locations. (f) Since the hemodynamic properties of hippocampal functions are not well understood, the decoding analysis was additionally performed in a time-resolved manner and fitted with a canonical hemodynamic function to estimate the time to peak. The difference time-course (successor minus predecessor) showed a temporally distinct peak around 4.7 s indicating that the future-directed prediction occurs as transient response to the partial stimulus input and not as a sustained signal throughout the trial. Error bars denote ± s.e.m.; **p < 0.01.
Figure 5—figure supplement 1
Successor-like representation of future sequence events in hippocampus.
(a) Group averaged (n=35) hippocampus classification evidence during partial sequence trials shows enhanced activation of successor locations compared to predecessor locations. (b) Model comparison favors successor-like representation in hippocampus. Competing model predictions were fitted to partial sequence trial data of each individual participant. Comparison of model fits showed that the data is most in line with the successor representation (lower BIC values correspond to a better fit). A null model (H0), resembling no prediction of past and future locations was included in the model comparison as baseline. Error bars denote ± s.e.m.; BIC, Bayesian Information Criterion (taking into account that the H0 model has fewer parameters); two-tailed t test, ***p < 0.001; **p < 0.01; *p < 0.05, uncorrected for multiple comparisons.
Figure 6
Stimulus localizer reveals complementary coactivation (tuning) properties in hippocampus and V1.
(a) Schematic of the localizer trial with the stimulated location ‘A’ and the non-stimulated locations (B, C, D, dashed circle) that were part of the sequence in the main task preceding the localizer. (b) Illustration of coactivation (tuning) of sequence locations based on spatial (Euclidean) distance from the stimulated location (left) and temporal distance in sequence space (right). Note how sequence locations A and B are far apart in the spatial (Euclidean) domain, but close in terms of temporal distance in sequence space. (c) Hypothetical activation pattern for representational tuning of spatial distance and temporal distance for illustration shown in (b). (d) Illustration of tuning pattern averaged across all localizer conditions for temporal tuning (top), spatial tuning (middle), Successor Representation (SR, middle), and no coactivation (H0, bottom). For visualization purposes the x-axis is sorted by time for all three tuning patterns. (e) Classifier evidence for current, future, and past locations for hippocampus (left) and V1 (right). (f) Comparing model errors (i.e., lower is better) show that hippocampal representations were best described by temporal coactivation (left), while V1 (right) was best described by spatial coactivation and the absence of coactivation (H0) of sequence locations. Error bars denote ± s.e.m.; two-tailed t test, ***p < 0.001; **p < 0.01; *p < 0.05, uncorrected for multiple comparisons; BIC, Bayesian Information Criterion (taking into account that the H0 model has fewer parameters).
Additional files
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Successor-like representation guides the prediction of future events in human visual cortex and hippocampus
eLife 12:e78904.
https://doi.org/10.7554/eLife.78904
