Knowledge of the temporal structure of events is central to our experience. We remember how a sequence of events unfolded and can recall when in time events occurred. Emphasizing both when in time and where in space events came to pass, episodic memories typically comprise event information linked to a spatiotemporal context. Space and time have been suggested to constitute fundamental dimensions along which our experience is organized (Konkel and Cohen, 2009; Ekstrom and Ranganath, 2018; Bellmund et al., 2018a). Consistently, the role of the hippocampus — a core structure for episodic memory (Scoville and Milner, 1957; Squire, 1982) — in coding locations in space (O'Keefe and Dostrovsky, 1971; Moser et al., 2017; Epstein et al., 2017) and moments in time (Pastalkova et al., 2008; MacDonald et al., 2011; Eichenbaum, 2014; Ranganath, 2019; Howard, 2018) is well-established. Human memory research has highlighted the role of the hippocampus in the encoding, representation and retrieval of temporal relations (Tubridy and Davachi, 2011; DuBrow and Davachi, 2014; Ezzyat and Davachi, 2014; Hsieh et al., 2014; Jenkins and Ranganath, 2010; Jenkins and Ranganath, 2016; Kyle et al., 2015; Copara et al., 2014). The similarity patterns of mnemonic representations suggest that the hippocampus forms integrated maps reflecting the temporal and spatial structure of event memories (Deuker et al., 2016; Nielson et al., 2015). Consistently, activity in the hippocampal-entorhinal region has been demonstrated to be sensitive to Euclidean distances as well as the lengths of shortest paths to goals during navigation (Spiers and Maguire, 2007; Viard et al., 2011; Sherrill et al., 2013; Howard et al., 2014; Chrastil et al., 2015; Spiers and Barry, 2015).

How do representations of temporal structure arise in the hippocampus? Evidence suggests that neural ensembles in the lateral entorhinal cortex (EC), which is strongly connected to the hippocampus (Witter et al., 2017), carry temporal information in freely moving rodents (Tsao et al., 2018). Specifically, temporal information could be decoded from population activity with high accuracy (Tsao et al., 2018). This temporal information was suggested to arise from the integration of experience rather than an explicit clocking signal (Tsao et al., 2018). Recently, the human anterior-lateral entorhinal cortex (alEC), the homologue region of the rodent lateral entorhinal cortex (Navarro Schröder et al., 2015; Maass et al., 2015), as well as the perirhinal cortex and a network of brain regions including the hippocampus, the medial prefrontal cortex, posterior cingulate cortex and angular gyrus have been implicated in the recall of temporal information (Montchal et al., 2019). These regions responded more strongly for high compared to low accuracy retrieval of when in time snapshots from a sitcom appeared over the course of the episode viewed in the experiment (Montchal et al., 2019). Together, these findings demonstrate that entorhinal population activity carries temporal information in navigating rodents and that its human homologue is activated during temporal memory recall. However, the contents of mnemonic representations in the alEC remains unclear.

We used representational similarity analysis of fMRI multi-voxel patterns in the entorhinal cortex to address the question how learning the structure of an event sequence shapes mnemonic representations in the alEC. Using this paradigm and data, we previously demonstrated that participants can successfully recall spatial and temporal relations of events defined by object encounters in a virtual city and that the change of hippocampal representations reflects an integrated event map of the remembered distance structure (Deuker et al., 2016). Here, we show that the change of multi-voxel pattern similarity through learning in the alEC specifically reflects the temporal structure of the event sequence.

We examined the effect of learning on object representations in the human entorhinal cortex using fMRI. In between two picture viewing tasks during which fMRI data were collected, participants acquired knowledge of temporal and spatial positions of objects in a familiar virtual city. Participants navigated repeated laps of a route along which they encountered chests containing different objects (Figure 1; Figure 1—figure supplement 1). We aimed to test whether entorhinal pattern similarity change from before to after learning related to experienced object relationships. Specifically, we presented object images twelve times in the picture viewing tasks before and after learning, using the same random order in both scanning runs. For both runs, we calculated the similarity of multi-voxel patterns for all object pairs and correlated changes in representational similarity with the temporal and spatial object relationships. The temporal distance structure of the object sequence can be quantified as the elapsed time between object encounters or as ordinal differences between their sequence positions, which are closely related in our task. Spatial distances on the other hand can be captured by Euclidean distances or geodesic distances between positions based on the shortest navigable paths between object positions. Importantly, we dissociated temporal from Euclidean and geodesic spatial object relationships through the use of teleporters along the route (Figure 1—figure supplement 2). Further, object relationships can be quantified by the distance traveled along the section of the route separating their positions (Figure 1—figure supplement 2). To assess whether entorhinal object representations change from before to after learning to map experienced object relationships, we compared changes in neural pattern similarity to the temporal and spatial structure of the task.

Figure 1 with 2 supplements see all

Download asset Open asset

The change in multi-voxel pattern similarity in alEC between pre- and post-learning scans was negatively correlated with the sequence structure (Figure 2A and Figure 2B, T(25)=- 3.75, p=0.001, alpha-level of 0.0125, Bonferroni-corrected for four comparisons), which was quantified as the median elapsed time between objects pairs along the route. After relative to before learning, objects encountered in temporal proximity were represented more similarly compared to object pairs further separated in time (Figure 2C). Pattern similarity change was negatively correlated with temporal distances after excluding comparisons of objects encountered in direct succession from the analysis (T(25)=-2.00, p=0.029, one-sided test, Figure 2—figure supplement 1A), speaking for holistic representations of temporal structure in the alEC and ruling out that the effect we observed is largely driven by increased similarity of temporally adjacent objects. Importantly, the strength of this effect was strongly related to behavior in the post-scan free recall test, where participants retrieved the objects from memory. Specifically, participants with stronger correlations between alEC pattern similarity change and the temporal task structure tended to recall objects together that were encountered in temporal proximity along the route (Pearson r = −0.53, p=0.006, CIs: −0.76,–0.19, Figure 2D).

Figure 2 with 4 supplements see all

Download asset Open asset

Figure 2—source data 1

Z-values of correlations between pattern similarity change in the entorhinal subregions and temporal and Euclidean spatial distances as shown in panel B.

https://doi.org/10.7554/eLife.45333.010

Download elife-45333-fig2-data1-v1.txt

Figure 2—source data 2

Pattern similarity changes in alEC for object pairs separated by low and high temporal distances as shown in panel C.

https://doi.org/10.7554/eLife.45333.011

Download elife-45333-fig2-data2-v1.txt

Figure 2—source data 3

Z-values of correlations between alEC pattern similarity change and temporal distances without comparisons of objects encountered in direct succession along the route and Pearson correlation coefficients quantifying temporal clustering during the free recall task (panel D).

https://doi.org/10.7554/eLife.45333.012

Download elife-45333-fig2-data3-v1.txt

Figure 2—source data 4

Z-value differences quantifying the difference in temporal and spatial mapping in alEC and pmEC as shown in panel E.

https://doi.org/10.7554/eLife.45333.013

Download elife-45333-fig2-data4-v1.txt

Pattern similarity change in alEC did not correlate significantly with Euclidean spatial distances (T(25)=0.81, p=0.420) and pattern similarity change in posterior-medial EC (pmEC) did not correlate with Euclidean (T(25)=0.58, p=0.583) or temporal (T(25)=1.73, p=0.089) distances. Temporal distances between objects during the first picture viewing task were not related to alEC pattern similarity change (Figure 2—figure supplement 1B; T(24)=-0.29 p=0.776, one outlier excluded, see Materials and methods) and correlations with elapsed time between objects during navigation were significantly more negative (T(24)=-1.76 p=0.045; one-sided test); strengthening our interpretation that pattern similarity changes reflected relationships experienced in the virtual city.

Can we reconstruct the timeline of events from pattern similarity change in alEC? Here, we used multidimensional scaling to extract coordinates along one dimension from pattern similarity change averaged across participants (Figure 3A–D). The reconstructed temporal coordinates, transformed into the original value range using Procrustes analysis (Figure 3A), mirrored the time points at which objects were encountered during the task (Figure 3B, Pearson correlation between reconstructed and true time points, r = 0.56, p=0.023, bootstrapped 95% confidence interval: 0.21, 0.79). Further, we contrasted the fit of the coordinates from multidimensional scaling between the true and randomly shuffled timelines (Figure 3C). Specifically, we compared the standardized sum of squared errors of the fit between the reconstructed and the true timeline, the Procrustes distance, to a surrogate distribution of deviance values. This surrogate distribution was obtained by fitting the coordinates from multidimensional scaling to randomly shuffled timelines of events. The Procrustes distance from fitting to the true timeline was smaller than the 5th percentile of the surrogate distribution generated via 10000 random shuffles (Figure 3D, p=0.026). Taken together, these findings indicate that alEC representations change through learning to reflect the temporal structure of the acquired event memories and that we can recover the timeline of events from this representational change.

Figure 3

Download asset Open asset

Figure 3—source data 1

True and reconstructed temporal coordinates of object positions as shown in panel B.

https://doi.org/10.7554/eLife.45333.015

Download elife-45333-fig3-data1-v1.txt

Figure 3—source data 2

Procrustes distance from mapping coordinates from multidimensional scaling based on alEC pattern similarity change to true temporal coordinates and surrogate distribution obtained from fitting to shuffled temporal coordinates (panel D).

https://doi.org/10.7554/eLife.45333.016

Download elife-45333-fig3-data2-v1.txt

What is the nature of regional specificity within entorhinal cortex? In a next step, we compared temporal and spatial mapping between the subregions of the entorhinal cortex. We conducted a permutation-based two-by-two repeated measures ANOVA (see Materials and methods) with the factors entorhinal subregion (alEC vs. pmEC) and relationship type (temporal vs. Euclidean spatial distance between events). Crucially, we observed a significant interaction between EC subregion and distance type (F(1,25)=7.40, p=0.011, Figure 2B and E). Further, the main effect of EC subregion was significant (F(1,25)=5.18, p=0.029), while the main effect of distance type was not (F(1,25)=0.84, p=0.367). Based on the significant interaction, we conducted planned post-hoc comparisons, which revealed significant differences (Bonferroni-corrected alpha-level of 0.025) between the mapping of temporal and spatial distances in alEC (T(25)=-2.91, p=0.007) and a significant difference between temporal mapping in alEC compared to pmEC (T(25)=-3.52, p=0.001).

Operationalizing the temporal structure in terms of the ordinal distances between object positions in the sequence yielded comparable results since our design did not disentangle time elapsed from ordinal positions as objects were encountered at regular intervals along the route. Pattern similarity change in alEC correlated significantly with ordinal temporal distances (Figure 4, T(25)=-3.37, p=0.002), an effect further qualified by a significant interaction in the two-by-two repeated measures ANOVA contrasting the effects of ordinal temporal and Euclidean spatial distances in the entorhinal subregions (interaction: F(1,25)=7.11, p=0.012; main effect of EC subregion: F(1,25)=4.97, p=0.033; main effect of distance type: F(1,25)=0.84, p=0.365). Alternative to the quantification of spatial relationships as Euclidean distances we calculated geodesic distances between object positions (Figure 1—figure supplement 2B–E). Entorhinal pattern similarity change was not correlated with geodesic distances based on shortest paths between locations using all positions not obstructed by buildings or other obstacles (Figure 2—figure supplement 2A, alEC: T(25)=0.82, p=0.436, pmEC: T(25)=0.73, p=0.479) or based on shortest paths using only the street network (Figure 2—figure supplement 2B, alEC: T(25)=0.36, p=0.715, pmEC: T(25)=0.92, p=0.375). Furthermore, the interaction of the two-by-two repeated measures ANOVA with the factors entorhinal subregion and distance type remained significant when using geodesic spatial distances based on shortest paths using all non-obstructed positions (interaction: F(1,25)=6.96, p=0.014; main effect of EC subregion: F(1,25)=5.18, p=0.031; main effect of distance type: F(1,25)=0.99, p=0.330) or the street network only (interaction: F(1,25)=4.30, p=0.048; main effect of EC subregion: F(1,25)=6.68, p=0.017; main effect of distance type: F(1,25)=0.81, p=0.376). Spatial and temporal signal-to-noise ratios did not differ between alEC and pmEC (Figure 2—figure supplement 3), ruling out that differences in signal quality might explain the observed effects.

Figure 4

Download asset Open asset

Figure 4—source data 1

Z-values of correlations between pattern similarity change in the entorhinal subregions and ordinal temporal and Euclidean spatial distances.

https://doi.org/10.7554/eLife.45333.018

Download elife-45333-fig4-data1-v1.txt

Does the presentation of object images during the post-learning picture viewing task elicit reactivations of similar representations from the pre-scan? For example, associations might be formed between objects encountered in succession along the route, which might result in the reactivation of neighboring objects after learning. To test this notion, we trained pattern classifiers to distinguish object representations on the pre-learning scan and tested these classifiers on the post-learning scan (see Materials and methods). We observed classifier accuracies exceeding chance levels in the lateral occipital cortex (LOC, Figure 2—figure supplement 4, T(25)=7.54, p<0.001, see Materials and methods) — known to be involved in visual object processing (Grill-Spector et al., 2001). In the entorhinal cortex, classification accuracies did not exceed chance levels (alEC: T(25)=-0.08, p=0.941; pmEC: T(25)=0.53, p=0.621). Next, we examined classifier predictions as a function of lag along the sequence. If effects in the alEC are driven by the reactivation of objects at neighboring sequence positions, then one might expect systematic classification errors, where an object might likely be confused with preceding or successive objects. In the entorhinal cortex, classifier evidence did not exceed chance levels for the three objects preceding (Figure 2—figure supplement 4B, alEC: most extreme T(25)=1.13; minimum p=0.270; pmEC: most extreme T(25)=1.00; min. p=0.332) or following (alEC: most extreme T(25)=-2.07; min. p=0.055; pmEC: most extreme T(25)=-0.83; min. p=0.414) an object. We also did not observe above-chance classifier evidence for nearby objects in the LOC, but rather classifier evidence was below chance levels for some lags, potentially due to high classification accuracies at no lag (preceding objects: most extreme T(25)=-2.51; min. p=0.018; successive objects: most extreme T(25)=-4.09; min. p<0.001).

Collectively, our findings demonstrate that, within the EC, only representations in the anterior-lateral subregion changed to resemble the temporal structure of the event sequence and that this mapping was specific to the temporal rather than the spatial dimension.

We examined the similarity of multi-voxel patterns to demonstrate that alEC representations reflect the experienced temporal event structure. Despite being cued in random order after learning, these representations related to a holistic temporal map of the sequence structure. Moreover, entorhinal pattern similarity change correlated with participants’ recall behavior and we recovered the timeline of events during learning from these representations.

Our hypothesis for temporal mapping in the alEC specifically was based on a recent finding demonstrating that population activity in the rodent lateral EC carries information from which time can be decoded at different scales ranging from seconds to days (Tsao et al., 2018). Time could be decoded with higher accuracies from the lateral EC than the medial EC and hippocampal subfield CA3. During a structured task in which the animal ran repeated laps on a maze separated into different trials, neural trajectories through population activity space were similar across trials, illustrating that the dynamics of lateral EC neural signals were more stable than during free foraging (Tsao et al., 2018). Consistently, temporal coding was improved for time within a trial during the structured task compared to episodes of free foraging. These findings support the notion that temporal information in the lateral EC might inherently arise from the encoding of experience (Tsao et al., 2018). In our task, relevant factors contributing to a similar experience of the route on each lap are not only the encounters of objects in a specific order at their respective positions, but also recognizing and passing salient landmarks as well as traveled distance and navigational demands in general. Changes in metabolic states and arousal presumably varied more linearly over time. Slowly drifting activity patterns have been observed also in the human medial temporal lobe (Folkerts et al., 2018) and EC specifically (Lositsky et al., 2016). A representation of time within a known trajectory in the alEC could underlie the encoding of temporal relationships between events in our task, where participants repeatedly navigated along the route to learn the positions of objects. Hence, temporal mapping in the alEC as we report here might help integrate hippocampal spatio-temporal event maps (Deuker et al., 2016).

Our findings demonstrate that alEC representations reflect the temporal structure of events after learning. This finding further dovetails with a recent fMRI study (Montchal et al., 2019), in which participants indicated when a still frame was encountered over the course of an episode of a sitcom. The alEC activated more strongly for the third of trials in which participants recalled temporal information most accurately compared to the third of trials in which temporal precision was lowest (Montchal et al., 2019). Going beyond the relationship of univariate activation differences to the precision of temporal memory recall, we focused on the content of alEC activation patterns and demonstrate that the alEC represents the temporal structure of events after learning.

One possibility for why alEC multi-voxel patterns resemble a holistic temporal map of the event structure in our task is the reactivation of temporal context information. If alEC neural populations traverse similar population state trajectories on each lap, they would carry information about time within a lap. A given object would be associated with a similar alEC population state on each lap. Associations with temporally drifting signals during the learning task would result in representational changes relative to the baseline scan that, if reactivated in the post-learning picture viewing task, reflect the experienced temporal structure of object encounters. This might explain the observed pattern similarity structure with relatively increased similarity for objects encountered in temporal proximity during learning and decreased similarity for items encountered after longer delays. While this interpretation is in line with data from rodent electrophysiology (Tsao et al., 2018) and the framework proposed by the temporal context model (Howard and Kahana, 2002; Howard et al., 2005) as well as evidence for neural contiguity effects in image recognition tasks (Howard et al., 2012; Folkerts et al., 2018), we cannot test the reinstatement of specific activity patterns from the learning phase directly since fMRI data were only collected during the picture viewing tasks in this study.

An alternative explanation for how the observed effects might arise is through associations between the objects. During learning, an object might become associated with preceding and successive objects, with stronger associations for objects closer in the sequence (Metcalfe and Murdock, 1981; Lewandowsky and Murdock, 1989; Jensen and Lisman, 2005). In this framework, the reactivation of associated objects during the post-learning picture viewing task could drive similarity increases for objects close together in the sequence. We tested for stable object representations from before to after learning and assessed classifier predictions to test the hypothesis that — if object reactivations underlie the effects — we might observe biased classifier evidence for the objects preceding or following a given object in the sequence. However, using classifiers trained on the picture viewing task before learning, we did not observe evidence for stable object representations in the entorhinal cortex or above-chance classifier evidence for objects nearby in the sequence after learning. Object representations in lateral occipital cortex (LOC) were stable between the picture viewing tasks. Previous studies have observed evidence for cortical reinstatement during memory retrieval (Nyberg et al., 2000; Wheeler et al., 2000; Polyn et al., 2005), modulated by hippocampal-entorhinal activity (Bosch et al., 2014). We did not observe classification accuracies exceeding chance levels for objects from nearby sequence positions in LOC, which one would expect if associative retrieval of objects accompanied by cortical reinstatement were to underlie our effects. Hence, these results fail to provide evidence for the notion that the reactivation of object representations drove our effects.

Importantly, the highly-controlled design of our study supports the interpretation that alEC representations change through learning to map the temporal event structure. The order of object presentations during the scanning sessions was randomized and thus did not reflect the order in which objects were encountered during the learning task. Since the assignment of objects to positions was randomized across participants and we analyzed pattern similarity change from a baseline scan, our findings do not go back to prior associations between the objects, but reflect information learned over the course of the experiment. Further, we presented the object images during the scanning sessions not only in the same random order, but also with the same presentation times and inter-stimulus intervals; thereby ruling out that the effects we observed relate to temporal autocorrelation of the BOLD-signal. Taken together, the high degree of experimental control of our study supports the conclusion that alEC representations change to reflect the temporal structure of acquired memories.

The long time scales of lateral EC temporal codes differ from the observation of time cells in the hippocampus and medial EC, which fire during temporal delays in highly trained tasks (Pastalkova et al., 2008; MacDonald et al., 2011; Eichenbaum, 2014; Kraus et al., 2015; Mau et al., 2018; Heys and Dombeck, 2018). Time cell ensembles change over minutes and days (Mau et al., 2018), but their firing has been investigated predominantly in the context of short delays in the range of seconds. One recent study did not find evidence for time cell sequences during a 60s-delay (Sabariego et al., 2019). In our task, one lap of the route took approximately 4.5 min on average; comparable to the 250s-duration of a trial in Tsao et al. (2018). How memories are represented at different temporal scales, which might be integrated in hierarchically nested sequences such as different days within a week, remains a question for future research.

Our assessment of temporal representations in the antero-lateral and posterior-medial subdivision of the EC was inspired by a recent report of temporal coding during free foraging and repetitive behavior in the rodent EC, which was most pronounced in the lateral EC (Tsao et al., 2018). In humans, local and global functional connectivity patterns suggest a preserved bipartite division of the EC, but along not only its medial-lateral, but also its anterior-posterior axis (Navarro Schröder et al., 2015; Maass et al., 2015). Via these entorhinal subdivisions, cortical inputs from the anterior-temporal and posterior-medial memory systems might converge onto the hippocampus (Ranganath and Ritchey, 2012; Ritchey et al., 2015). In line with hexadirectional signals in pmEC during imagination (Bellmund et al., 2016), putatively related to grid-cell population activity (Doeller et al., 2010), one might expect the pmEC to map spatial distances between object positions in our task. However, we did not observe an association of pattern similarity change in the entorhinal cortex with Euclidean or geodesic distances based on shortest paths between object positions. One potential explanation for the absence of evidence for a spatial distance signal in pmEC might be the way in which we cued participants’ memory during the picture viewing task. The presentation of isolated object images probed locations in their stored representation of the virtual city. Due to the periodic nature of grid-cell firing, different locations might not result in diverging patterns of grid-cell population activity. Hence, the design here was not optimized for the analysis of spatial representations in pmEC, if the object positions were encoded in grid-cell firing patterns as suggested by models of grid-cell function (Fiete et al., 2008; Mathis et al., 2012; Bush et al., 2015).

Our findings are in line with the role of the hippocampus in the retrieval of temporal information from memory (Copara et al., 2014; DuBrow and Davachi, 2014; Kyle et al., 2015; Nielson et al., 2015). Hippocampal pattern similarity has been shown to scale with temporal distances between events (Deuker et al., 2016; Nielson et al., 2015) and evidence for the reinstatement of temporally associated items from memory has been reported in the hippocampus (DuBrow and Davachi, 2014). Already at the stage of encoding, hippocampal and entorhinal activity have been related to later temporal memory (DuBrow and Davachi, 2014; DuBrow and Davachi, 2016; Ezzyat and Davachi, 2014; Jenkins and Ranganath, 2010; Jenkins and Ranganath, 2016; Lositsky et al., 2016; Tubridy and Davachi, 2011). For example, increased pattern similarity has been reported for items remembered to be close together compared to items remembered to be far apart in time, despite the same time having elapsed between these items (Ezzyat and Davachi, 2014). Similarly, changes in EC pattern similarity during the encoding of a narrative correlated with later duration estimates between events (Lositsky et al., 2016). Complementing these reports, our findings demonstrate that entorhinal activity patterns carry information about the temporal structure of memories at retrieval. Furthermore, the degree to which EC patterns reflected holistic representations of temporal relationships related to recall behavior characterized by the consecutive reproduction of objects encountered in temporal proximity; potentially through mental traversals of the route during memory recall. The central role of the hippocampus and entorhinal cortex in temporal memory (for review see Davachi and DuBrow, 2015; Howard, 2018; Ranganath, 2019; Wang and Diana, 2017) dovetails with the involvement of these regions in learning sequences and statistical regularities in general (Barnett et al., 2014; Garvert et al., 2017; Hsieh et al., 2014; Kumaran and Maguire, 2006; Schapiro et al., 2012; Schapiro et al., 2016; Thavabalasingam et al., 2018).

In this experiment, the paradigm was designed to disentangle temporal distances from Euclidean spatial distances between objects (Deuker et al., 2016), resulting also in a decorrelation of temporal distances and geodesic distances based on shortest paths between object positions. Since objects were encountered at regular intervals along the route, temporal distances quantified as elapsed time in seconds or, on an ordinal level of measurement, as the difference in sequence position were highly correlated measures of the sequence structure. To partially decorrelate elapsed time from ordinal temporal distances and distance traveled along the route, future studies could vary the speed of movement between different sections of the route. This might allow the investigation of the level of precision at which the hippocampal-entorhinal region stores temporal relations, in line with evidence for the integration of duration information in the representations of short sequences (Thavabalasingam et al., 2018; Thavabalasingam et al., 2019). Interestingly, a multi-scale ensemble of successor representations was recently suggested to estimate sequences of anticipated future states, including the order and distances between states (Stachenfeld et al., 2017; Momennejad and Howard, 2018); consistent with the sensitivity of neurons (Sarel et al., 2017; Gauthier and Tank, 2018; Qasim et al., 2018) and BOLD-responses (Spiers and Maguire, 2007; Viard et al., 2011; Sherrill et al., 2013; Howard et al., 2014; Chrastil et al., 2015) in the hippocampal-entorhinal region to distances and directions to navigational goals. Related to the effects of circumnavigation on travel time and Euclidean distance estimates (Brunec et al., 2017), experimental manipulations of route tortuosity could shed additional light on how, in the context of navigation, spatio-temporal event structures shape episodic memory.

In conclusion, our findings demonstrate that activity patterns in alEC, the human homologue region of the rodent lateral EC, carry information about the temporal structure of experienced events. The observed effects might be related to the reactivation of temporal contextual tags, in line with the recent report of temporal information available in rodent lateral EC population activity and models of episodic memory.

Source data files have been provided for Figures 2, 3 and 4. The virtual city Donderstown is available at https://osf.io/78uph/.

1. Report

   1. Anderson J
   2. Jenkinson M
   3. Smith S

   (2007)

   Non-Linear Registration, Aka Spatial Normalisation. FMRIB Technical Report TR07JA

   Oxford, United Kingdom: FMRIB Centre.

   • Google Scholar
2. 1. Barnett AJ
   2. O'Neil EB
   3. Watson HC
   4. Lee AC

   (2014) The human Hippocampus is sensitive to the durations of events and intervals within a sequence

   Neuropsychologia 64:1–12.

   https://doi.org/10.1016/j.neuropsychologia.2014.09.011

   • PubMed
   • Google Scholar
3. 1. Bellmund JLS
   2. Deuker L
   3. Navarro Schröder T
   4. Doeller CF

   (2016) Grid-cell representations in mental simulation

   eLife 5:e17089.

   https://doi.org/10.7554/eLife.17089

   • PubMed
   • Google Scholar
4. 1. Bellmund JLS
   2. Gärdenfors P
   3. Moser EI
   4. Doeller CF

   (2018a) Navigating cognition: spatial codes for human thinking

   Science 362:eaat6766.

   https://doi.org/10.1126/science.aat6766

   • PubMed
   • Google Scholar
5. Software

   1. Bellmund JLS
   2. Deuker L
   3. Doeller CF

   (2018b) Donderstown

   Open Science Framework.

   https://osf.io/78uph/
6. 1. Bosch SE
   2. Jehee JF
   3. Fernández G
   4. Doeller CF

   (2014) Reinstatement of associative memories in early visual cortex is signaled by the Hippocampus

   Journal of Neuroscience 34:7493–7500.

   https://doi.org/10.1523/JNEUROSCI.0805-14.2014

   • PubMed
   • Google Scholar
7. 1. Brunec IK
   2. Javadi AH
   3. Zisch FEL
   4. Spiers HJ

   (2017) Contracted time and expanded space: the impact of circumnavigation on judgements of space and time

   Cognition 166:425–432.

   https://doi.org/10.1016/j.cognition.2017.06.004

   • PubMed
   • Google Scholar
8. 1. Bush D
   2. Barry C
   3. Manson D
   4. Burgess N

   (2015) Using grid cells for navigation

   Neuron 87:507–520.

   https://doi.org/10.1016/j.neuron.2015.07.006

   • PubMed
   • Google Scholar
9. 1. Chrastil ER
   2. Sherrill KR
   3. Hasselmo ME
   4. Stern CE

   (2015) There and back again: hippocampus and retrosplenial cortex track homing distance during human path integration

   Journal of Neuroscience 35:15442–15452.

   https://doi.org/10.1523/JNEUROSCI.1209-15.2015

   • PubMed
   • Google Scholar
10. 1. Copara MS
    2. Hassan AS
    3. Kyle CT
    4. Libby LA
    5. Ranganath C
    6. Ekstrom AD

    (2014) Complementary roles of human hippocampal subregions during retrieval of spatiotemporal context

    Journal of Neuroscience 34:6834–6842.

    https://doi.org/10.1523/JNEUROSCI.5341-13.2014

    • PubMed
    • Google Scholar
11. 1. Davachi L
    2. DuBrow S

    (2015) How the Hippocampus preserves order: the role of prediction and context

    Trends in Cognitive Sciences 19:92–99.

    https://doi.org/10.1016/j.tics.2014.12.004

    • PubMed
    • Google Scholar
12. 1. Deuker L
    2. Bellmund JLS
    3. Navarro Schröder T
    4. Doeller CF

    (2016) An event map of memory space in the Hippocampus

    eLife 5:e16534.

    https://doi.org/10.7554/eLife.16534

    • PubMed
    • Google Scholar
13. 1. Doeller CF
    2. Barry C
    3. Burgess N

    (2010) Evidence for grid cells in a human memory network

    Nature 463:657–661.

    https://doi.org/10.1038/nature08704

    • PubMed
    • Google Scholar
14. 1. DuBrow S
    2. Davachi L

    (2014) Temporal memory is shaped by encoding stability and intervening item reactivation

    Journal of Neuroscience 34:13998–14005.

    https://doi.org/10.1523/JNEUROSCI.2535-14.2014

    • PubMed
    • Google Scholar
15. 1. DuBrow S
    2. Davachi L

    (2016) Temporal binding within and across events

    Neurobiology of Learning and Memory 134:107–114.

    https://doi.org/10.1016/j.nlm.2016.07.011

    • PubMed
    • Google Scholar
16. 1. Eichenbaum H

    (2014) Time cells in the Hippocampus: a new dimension for mapping memories

    Nature Reviews Neuroscience 15:732–744.

    https://doi.org/10.1038/nrn3827

    • PubMed
    • Google Scholar
17. 1. Ekstrom AD
    2. Ranganath C

    (2018) Space, time, and episodic memory: the Hippocampus is all over the cognitive map

    Hippocampus 28:680–687.

    https://doi.org/10.1002/hipo.22750

    • PubMed
    • Google Scholar
18. 1. Epstein RA
    2. Patai EZ
    3. Julian JB
    4. Spiers HJ

    (2017) The cognitive map in humans: spatial navigation and beyond

    Nature Neuroscience 20:1504–1513.

    https://doi.org/10.1038/nn.4656

    • PubMed
    • Google Scholar
19. 1. Ezzyat Y
    2. Davachi L

    (2014) Similarity breeds proximity: pattern similarity within and across contexts is related to later mnemonic judgments of temporal proximity

    Neuron 81:1179–1189.

    https://doi.org/10.1016/j.neuron.2014.01.042

    • PubMed
    • Google Scholar
20. 1. Fiete IR
    2. Burak Y
    3. Brookings T

    (2008) What grid cells convey about rat location

    Journal of Neuroscience 28:6858–6871.

    https://doi.org/10.1523/JNEUROSCI.5684-07.2008

    • PubMed
    • Google Scholar
21. 1. Folkerts S
    2. Rutishauser U
    3. Howard MW

    (2018) Human episodic memory retrieval is accompanied by a neural contiguity effect

    The Journal of Neuroscience 38:4200–4211.

    https://doi.org/10.1523/JNEUROSCI.2312-17.2018

    • PubMed
    • Google Scholar
22. 1. Garvert MM
    2. Dolan RJ
    3. Behrens TE

    (2017) A map of abstract relational knowledge in the human hippocampal-entorhinal cortex

    eLife 6:e17086.

    https://doi.org/10.7554/eLife.17086

    • PubMed
    • Google Scholar
23. 1. Gauthier JL
    2. Tank DW

    (2018) A dedicated population for reward coding in the Hippocampus

    Neuron 99:179–193.

    https://doi.org/10.1016/j.neuron.2018.06.008

    • PubMed
    • Google Scholar
24. 1. Grill-Spector K
    2. Kourtzi Z
    3. Kanwisher N

    (2001) The lateral occipital complex and its role in object recognition

    Vision Research 41:1409–1422.

    https://doi.org/10.1016/S0042-6989(01)00073-6

    • PubMed
    • Google Scholar
25. 1. Heys JG
    2. Dombeck DA

    (2018) Evidence for a subcircuit in medial entorhinal cortex representing elapsed time during immobility

    Nature Neuroscience 21:1574–1582.

    https://doi.org/10.1038/s41593-018-0252-8

    • PubMed
    • Google Scholar
26. 1. Howard MW
    2. Fotedar MS
    3. Datey AV
    4. Hasselmo ME

    (2005) The temporal context model in spatial navigation and relational learning: toward a common explanation of medial temporal lobe function across domains

    Psychological Review 112:75–116.

    https://doi.org/10.1037/0033-295X.112.1.75

    • PubMed
    • Google Scholar
27. 1. Howard MW
    2. Viskontas IV
    3. Shankar KH
    4. Fried I

    (2012) Ensembles of human MTL neurons "jump back in time" in response to a repeated stimulus

    Hippocampus 22:1833–1847.

    https://doi.org/10.1002/hipo.22018

    • PubMed
    • Google Scholar
28. 1. Howard LR
    2. Javadi AH
    3. Yu Y
    4. Mill RD
    5. Morrison LC
    6. Knight R
    7. Loftus MM
    8. Staskute L
    9. Spiers HJ

    (2014) The Hippocampus and entorhinal cortex encode the path and euclidean distances to goals during navigation

    Current Biology 24:1331–1340.

    https://doi.org/10.1016/j.cub.2014.05.001

    • PubMed
    • Google Scholar
29. 1. Howard MW

    (2018) Memory as perception of the past: compressed time inMind and brain

    Trends in Cognitive Sciences 22:124–136.

    https://doi.org/10.1016/j.tics.2017.11.004

    • PubMed
    • Google Scholar
30. 1. Howard MW
    2. Kahana MJ

    (2002) A distributed representation of temporal context

    Journal of Mathematical Psychology 46:269–299.

    https://doi.org/10.1006/jmps.2001.1388

    • Google Scholar
31. 1. Hsieh LT
    2. Gruber MJ
    3. Jenkins LJ
    4. Ranganath C

    (2014) Hippocampal activity patterns carry information about objects in temporal context

    Neuron 81:1165–1178.

    https://doi.org/10.1016/j.neuron.2014.01.015

    • PubMed
    • Google Scholar
32. 1. Jenkins LJ
    2. Ranganath C

    (2010) Prefrontal and medial temporal lobe activity at encoding predicts temporal context memory

    Journal of Neuroscience 30:15558–15565.

    https://doi.org/10.1523/JNEUROSCI.1337-10.2010

    • PubMed
    • Google Scholar
33. 1. Jenkins LJ
    2. Ranganath C

    (2016) Distinct neural mechanisms for remembering when an event occurred

    Hippocampus 26:554–559.

    https://doi.org/10.1002/hipo.22571

    • PubMed
    • Google Scholar
34. 1. Jenkinson M
    2. Bannister P
    3. Brady M
    4. Smith S

    (2002) Improved optimization for the robust and accurate linear registration and motion correction of brain images

    NeuroImage 17:825–841.

    https://doi.org/10.1006/nimg.2002.1132

    • PubMed
    • Google Scholar
35. 1. Jenkinson M
    2. Smith S

    (2001) A global optimisation method for robust affine registration of brain images

    Medical Image Analysis 5:143–156.

    https://doi.org/10.1016/S1361-8415(01)00036-6

    • PubMed
    • Google Scholar
36. 1. Jensen O
    2. Lisman JE

    (2005) Hippocampal sequence-encoding driven by a cortical multi-item working memory buffer

    Trends in Neurosciences 28:67–72.

    https://doi.org/10.1016/j.tins.2004.12.001

    • PubMed
    • Google Scholar
37. 1. Konkel A
    2. Cohen NJ

    (2009) Relational memory and the Hippocampus: representations and methods

    Frontiers in Neuroscience 3:166–174.

    https://doi.org/10.3389/neuro.01.023.2009

    • PubMed
    • Google Scholar
38. 1. Kraus BJ
    2. Brandon MP
    3. Robinson RJ
    4. Connerney MA
    5. Hasselmo ME
    6. Eichenbaum H

    (2015) During running in place, grid cells integrate elapsed time and distance run

    Neuron 88:578–589.

    https://doi.org/10.1016/j.neuron.2015.09.031

    • PubMed
    • Google Scholar
39. 1. Kriegeskorte N
    2. Mur M
    3. Ruff DA
    4. Kiani R
    5. Bodurka J
    6. Esteky H
    7. Tanaka K
    8. Bandettini PA

    (2008a) Matching categorical object representations in inferior temporal cortex of man and monkey

    Neuron 60:1126–1141.

    https://doi.org/10.1016/j.neuron.2008.10.043

    • PubMed
    • Google Scholar
40. 1. Kriegeskorte N
    2. Mur M
    3. Bandettini P

    (2008b) Representational similarity analysis - connecting the branches of systems neuroscience

    Frontiers in Systems Neuroscience 2:4.

    https://doi.org/10.3389/neuro.06.004.2008

    • PubMed
    • Google Scholar
41. 1. Kumaran D
    2. Maguire EA

    (2006) An unexpected sequence of events: mismatch detection in the human Hippocampus

    PLOS Biology 4:e424.

    https://doi.org/10.1371/journal.pbio.0040424

    • PubMed
    • Google Scholar
42. 1. Kyle CT
    2. Smuda DN
    3. Hassan AS
    4. Ekstrom AD

    (2015) Roles of human hippocampal subfields in retrieval of spatial and temporal context

    Behavioural Brain Research 278:549–558.

    https://doi.org/10.1016/j.bbr.2014.10.034

    • PubMed
    • Google Scholar
43. 1. Lewandowsky S
    2. Murdock BB

    (1989) Memory for serial order

    Psychological Review 96:25–57.

    https://doi.org/10.1037/0033-295X.96.1.25

    • Google Scholar
44. 1. Lositsky O
    2. Chen J
    3. Toker D
    4. Honey CJ
    5. Shvartsman M
    6. Poppenk JL
    7. Hasson U
    8. Norman KA

    (2016) Neural pattern change during encoding of a narrative predicts retrospective duration estimates

    eLife 5:e16070.

    https://doi.org/10.7554/eLife.16070

    • PubMed
    • Google Scholar
45. 1. Maass A
    2. Berron D
    3. Libby LA
    4. Ranganath C
    5. Düzel E

    (2015) Functional subregions of the human entorhinal cortex

    eLife 4:e06426.

    https://doi.org/10.7554/eLife.06426

    • Google Scholar
46. 1. MacDonald CJ
    2. Lepage KQ
    3. Eden UT
    4. Eichenbaum H

    (2011) Hippocampal "time cells" bridge the gap in memory for discontiguous events

    Neuron 71:737–749.

    https://doi.org/10.1016/j.neuron.2011.07.012

    • PubMed
    • Google Scholar
47. 1. Mathis A
    2. Herz AV
    3. Stemmler M

    (2012) Optimal population codes for space: grid cells outperform place cells

    Neural Computation 24:2280–2317.

    https://doi.org/10.1162/NECO_a_00319

    • PubMed
    • Google Scholar
48. 1. Mau W
    2. Sullivan DW
    3. Kinsky NR
    4. Hasselmo ME
    5. Howard MW
    6. Eichenbaum H

    (2018) The same hippocampal CA1 population simultaneously codes temporal information over multiple timescales

    Current Biology 28:1499–1508.

    https://doi.org/10.1016/j.cub.2018.03.051

    • PubMed
    • Google Scholar
49. 1. Metcalfe J
    2. Murdock BB

    (1981) An encoding and retrieval model of single-trial free recall

    Journal of Verbal Learning and Verbal Behavior 20:161–189.

    https://doi.org/10.1016/S0022-5371(81)90365-0

    • Google Scholar
50. 1. Milivojevic B
    2. Vicente-Grabovetsky A
    3. Doeller CF

    (2015) Insight reconfigures hippocampal-prefrontal memories

    Current Biology 25:821–830.

    https://doi.org/10.1016/j.cub.2015.01.033

    • PubMed
    • Google Scholar
51. Preprint

    1. Momennejad I
    2. Howard MW

    (2018) Predicting the future with Multi-scale successor representations

    bioRxiv.

    https://doi.org/10.1101/449470

    • Google Scholar
52. 1. Montchal ME
    2. Reagh ZM
    3. Yassa MA

    (2019) Precise temporal memories are supported by the lateral entorhinal cortex in humans

    Nature Neuroscience 22:284–288.

    https://doi.org/10.1038/s41593-018-0303-1

    • PubMed
    • Google Scholar
53. 1. Moser EI
    2. Moser MB
    3. McNaughton BL

    (2017) Spatial representation in the hippocampal formation: a history

    Nature Neuroscience 20:1448–1464.

    https://doi.org/10.1038/nn.4653

    • PubMed
    • Google Scholar
54. 1. Navarro Schröder T
    2. Haak KV
    3. Zaragoza Jimenez NI
    4. Beckmann CF
    5. Doeller CF

    (2015) Functional topography of the human entorhinal cortex

    eLife 4:e06738.

    https://doi.org/10.7554/eLife.06738

    • Google Scholar
55. 1. Nielson DM
    2. Smith TA
    3. Sreekumar V
    4. Dennis S
    5. Sederberg PB

    (2015) Human Hippocampus represents space and time during retrieval of real-world memories

    PNAS 112:11078–11083.

    https://doi.org/10.1073/pnas.1507104112

    • PubMed
    • Google Scholar
56. 1. Nyberg L
    2. Habib R
    3. McIntosh AR
    4. Tulving E

    (2000) Reactivation of encoding-related brain activity during memory retrieval

    PNAS 97:11120–11124.

    https://doi.org/10.1073/pnas.97.20.11120

    • PubMed
    • Google Scholar
57. 1. O'Keefe J
    2. Dostrovsky J

    (1971) The Hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat

    Brain Research 34:171–175.

    https://doi.org/10.1016/0006-8993(71)90358-1

    • Google Scholar
58. 1. Pastalkova E
    2. Itskov V
    3. Amarasingham A
    4. Buzsáki G

    (2008) Internally generated cell assembly sequences in the rat Hippocampus

    Science 321:1322–1327.

    https://doi.org/10.1126/science.1159775

    • PubMed
    • Google Scholar
59. 1. Pernet CR
    2. Wilcox R
    3. Rousselet GA

    (2012) Robust correlation analyses: false positive and power validation using a new open source matlab toolbox

    Frontiers in Psychology 3:.

    https://doi.org/10.3389/fpsyg.2012.00606

    • PubMed
    • Google Scholar
60. 1. Polyn SM
    2. Natu VS
    3. Cohen JD
    4. Norman KA

    (2005) Category-specific cortical activity precedes retrieval during memory search

    Science 310:1963–1966.

    https://doi.org/10.1126/science.1117645

    • PubMed
    • Google Scholar
61. Preprint

    1. Qasim SE
    2. Miller J
    3. Inman CS
    4. Gross R
    5. Willie JT
    6. Lega B
    7. Lin J-J
    8. Sharan A
    9. Wu C
    10. Sperling MR

    (2018) Single neurons in the human entorhinal cortex remap to distinguish individual spatial memories

    bioRxiv.

    https://doi.org/10.1101/433862

    • Google Scholar
62. 1. Ranganath C

    (2019) Time, memory, and the legacy of howard eichenbaum

    Hippocampus 29:146–161.

    https://doi.org/10.1002/hipo.23007

    • PubMed
    • Google Scholar
63. 1. Ranganath C
    2. Ritchey M

    (2012) Two cortical systems for memory-guided behaviour

    Nature Reviews Neuroscience 13:713–726.

    https://doi.org/10.1038/nrn3338

    • PubMed
    • Google Scholar
64. 1. Ritchey M
    2. Libby LA
    3. Ranganath C

    (2015) Cortico-hippocampal systems involved in memory and cognition: the PMAT framework

    Progress in Brain Research 219:45–64.

    https://doi.org/10.1016/bs.pbr.2015.04.001

    • PubMed
    • Google Scholar
65. 1. Sabariego M
    2. Schönwald A
    3. Boublil BL
    4. Zimmerman DT
    5. Ahmadi S
    6. Gonzalez N
    7. Leibold C
    8. Clark RE
    9. Leutgeb JK
    10. Leutgeb S

    (2019) Time cells in the Hippocampus are neither dependent on medial entorhinal cortex inputs nor necessary for spatial working memory

    Neuron 102:1235–1248.

    https://doi.org/10.1016/j.neuron.2019.04.005

    • PubMed
    • Google Scholar
66. 1. Sarel A
    2. Finkelstein A
    3. Las L
    4. Ulanovsky N

    (2017) Vectorial representation of spatial goals in the Hippocampus of bats

    Science 355:176–180.

    https://doi.org/10.1126/science.aak9589

    • PubMed
    • Google Scholar
67. 1. Schapiro AC
    2. Kustner LV
    3. Turk-Browne NB

    (2012) Shaping of object representations in the human medial temporal lobe based on temporal regularities

    Current Biology 22:1622–1627.

    https://doi.org/10.1016/j.cub.2012.06.056

    • PubMed
    • Google Scholar
68. 1. Schapiro AC
    2. Turk-Browne NB
    3. Norman KA
    4. Botvinick MM

    (2016) Statistical learning of temporal community structure in the Hippocampus

    Hippocampus 26:3–8.

    https://doi.org/10.1002/hipo.22523

    • PubMed
    • Google Scholar
69. 1. Schlichting ML
    2. Mumford JA
    3. Preston AR

    (2015) Learning-related representational changes reveal dissociable integration and separation signatures in the Hippocampus and prefrontal cortex

    Nature Communications 6:8151.

    https://doi.org/10.1038/ncomms9151

    • PubMed
    • Google Scholar
70. 1. Scoville WB
    2. Milner B

    (1957) Loss of recent memory after bilateral hippocampal lesions

    Journal of Neurology, Neurosurgery & Psychiatry 20:11–21.

    https://doi.org/10.1136/jnnp.20.1.11

    • Google Scholar
71. 1. Sherrill KR
    2. Erdem UM
    3. Ross RS
    4. Brown TI
    5. Hasselmo ME
    6. Stern CE

    (2013) Hippocampus and retrosplenial cortex combine path integration signals for successful navigation

    Journal of Neuroscience 33:19304–19313.

    https://doi.org/10.1523/JNEUROSCI.1825-13.2013

    • PubMed
    • Google Scholar
72. 1. Spiers HJ
    2. Barry C

    (2015) Neural systems supporting navigation

    Current Opinion in Behavioral Sciences 1:47–55.

    https://doi.org/10.1016/j.cobeha.2014.08.005

    • Google Scholar
73. 1. Spiers HJ
    2. Maguire EA

    (2007) A navigational guidance system in the human brain

    Hippocampus 17:618–626.

    https://doi.org/10.1002/hipo.20298

    • PubMed
    • Google Scholar
74. 1. Squire LR

    (1982) The neuropsychology of human memory

    Annual Review of Neuroscience 5:241–273.

    https://doi.org/10.1146/annurev.ne.05.030182.001325

    • PubMed
    • Google Scholar
75. 1. Stachenfeld KL
    2. Botvinick MM
    3. Gershman SJ

    (2017) The Hippocampus as a predictive map

    Nature Neuroscience 20:1643–1653.

    https://doi.org/10.1038/nn.4650

    • PubMed
    • Google Scholar
76. 1. Stelzer J
    2. Chen Y
    3. Turner R

    (2013) Statistical inference and multiple testing correction in classification-based multi-voxel pattern analysis (MVPA): random permutations and cluster size control

    NeuroImage 65:69–82.

    https://doi.org/10.1016/j.neuroimage.2012.09.063

    • PubMed
    • Google Scholar
77. 1. Thavabalasingam S
    2. O'Neil EB
    3. Lee ACH

    (2018) Multivoxel pattern similarity suggests the integration of temporal duration in hippocampal event sequence representations

    NeuroImage 178:136–146.

    https://doi.org/10.1016/j.neuroimage.2018.05.036

    • PubMed
    • Google Scholar
78. 1. Thavabalasingam S
    2. O'Neil EB
    3. Tay J
    4. Nestor A
    5. Lee ACH

    (2019) Evidence for the incorporation of temporal duration information in human hippocampal long-term memory sequence representations

    PNAS 116:6407–6414.

    https://doi.org/10.1073/pnas.1819993116

    • PubMed
    • Google Scholar
79. 1. Tsao A
    2. Sugar J
    3. Lu L
    4. Wang C
    5. Knierim JJ
    6. Moser MB
    7. Moser EI

    (2018) Integrating time from experience in the lateral entorhinal cortex

    Nature 561:57–62.

    https://doi.org/10.1038/s41586-018-0459-6

    • PubMed
    • Google Scholar
80. 1. Tubridy S
    2. Davachi L

    (2011) Medial temporal lobe contributions to episodic sequence encoding

    Cerebral Cortex 21:272–280.

    https://doi.org/10.1093/cercor/bhq092

    • PubMed
    • Google Scholar
81. 1. Viard A
    2. Doeller CF
    3. Hartley T
    4. Bird CM
    5. Burgess N

    (2011) Anterior Hippocampus and goal-directed spatial decision making

    Journal of Neuroscience 31:4613–4621.

    https://doi.org/10.1523/JNEUROSCI.4640-10.2011

    • PubMed
    • Google Scholar
82. 1. Wang F
    2. Diana RA

    (2017) Temporal context in human fMRI

    Current Opinion in Behavioral Sciences 17:57–64.

    https://doi.org/10.1016/j.cobeha.2017.06.004

    • Google Scholar
83. 1. Wheeler ME
    2. Petersen SE
    3. Buckner RL

    (2000) Memory's echo: vivid remembering reactivates sensory-specific cortex

    PNAS 97:11125–11129.

    https://doi.org/10.1073/pnas.97.20.11125

    • PubMed
    • Google Scholar
84. 1. Witter MP
    2. Doan TP
    3. Jacobsen B
    4. Nilssen ES
    5. Ohara S

    (2017) Architecture of the entorhinal cortex A review of entorhinal anatomy in rodents with some comparative notes

    Frontiers in Systems Neuroscience 11:46.

    https://doi.org/10.3389/fnsys.2017.00046

    • Google Scholar

Thank you for submitting your article "Structuring time in human lateral entorhinal cortex" for consideration by eLife. Your article has been reviewed by three peer reviewers, including Ida Momennejad as the Reviewing Editor and Reviewer #2, and the evaluation has been overseen by Timothy Behrens as the Senior Editor. The following individuals involved in review of your submission have agreed to reveal their identity: Marc Howard (Reviewer #1); H Freyja Ólafsdóttir (Reviewer #3).

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

Summary:

Bellmund and colleagues describe interesting findings pointing to a role of the alEC in temporal coding. Specifically, the authors show objects encountered with close temporal proximity during a spatiotemporal task show similar pattern similarity using an fMRI RSA analysis. Importantly, the degree of temporal coding in the alEC correlated with behavioural performance. This paper examines changes in the representation of stimuli presented in a virtual world as a function of the time between and the (virtual) distance between their presentations. Previous work from the same group (using the same dataset as a matter of fact) and others has shown that learning causes representations of stimuli presented close together in time to become more similar to one another. This paper reexamines these data to specifically examine the role of subregions of the entorhinal cortex. This is inspired by a recent rodent study from the Moser group (Tsao et al., 2018) that has received a great deal of attention. The primary novel result is that there is an interaction of distance accounted for by time and space in the anterior-lateral entorhinal cortex (alEC) as compared to the posterior-medial entorhinal cortex (pmEC).

This paper is an advancement to previous work, an fMRI paradigm published in eLife by the same group. The authors ask a timely question about the representation of temporal structures in the human anterior lateral entorhinal cortex. The paper is very well written and the reasoning is clear. All reviewers agree that the manuscript presents a novel finding, which adds to the nascent body of research dedicated to studying the coding of space and time in the brain. Moreover, the work agrees with findings recently reported in rodents (Tsao et al., 2018). Overall, the reviewers find the manuscript timely and of broad interest and would like to see it published. Comments and revision requests are summarized below.

Essential Revisions:

Broadly: Is the finding clearly about time or something else (e.g. ordinal information, object reactivation, distance, etc.)?

Different reviewers had questions about the temporal nature of the findings and other interpretations. One of the reviewers notes that unlike the Tsao paper, this manuscript does not show direct evidence for a temporal representation in alEC. Here temporal (or related) information is incorporated into the response to a repeated stimulus perhaps due to recovery of temporal information – a jump back in time. The contrast to spatial information pmEC is quite meaningful and validates the relevance of a large body of rodent work for studies of human memory. These results are just as well predicted by models of human memory and localize the temporal vs. spatial aspect to the alEC vs. pmEC. Many models of human memory (including the temporal context model as well as a Laplace transform of time) hypothesize gradually-decaying activation of preceding information. This is also just what the Tsao Nature paper found. While one can choose to call this decaying trace of an object representation or a temporal code, the authors are advised to clarify that the temporal nature is not exclusively established early on, throughout the manuscript and by including the ordinal figure in the main results, and in the Discussion.

The comments are listed below. Unless the authors are inclined to conduct control experiments that verifies the 'temporal' nature of the findings, they are advised to revise the title as the present findings do not clearly delineate a temporal code. Detailed comments are listed below.

– The authors state their findings corroborate those of Tsao and colleagues (2018), which show temporal coding in the EC in rats. Specifically, Tsao et al. say they find EC codes for moment-to-moment changes in experience and explain the temporal signal may be less of a clocking signal and more of an experience timestamp.

Do the authors interpret their finding along the same lines? Moreover, the authors show their effect is replicated when they do an ordinal analysis. Thus, do the authors claim the alEC signals temporal order specifically rather than time or experience? It would be useful for the authors to elaborate on the interpretation of their effect especially in light of the interpretation given by Tsao et al. for the EC temporal coding.

– Regarding the analysis which included the pmEC (which is meant to be the human homologue to the MEC in rats), should one not expect to see a big change in pattern similarity for objects spatially close together? i.e. pmEC is meant to contain grid cells which code for space, rather than time (or at least more robustly than time). Could the authors comment on why they think they do not observe the reverse effect in pmEC compared to alEC?

– Another reason this is important regards the time scale of temporal distances in the Tsao experiment compared to the present study. Given the large time scales here, and the fact that the rodent electrophysiology data related to smaller time scales, it is possible that this result has to do with other measures/scales of distance. Please discuss.

– Do the authors believe that short/long time scales are coded by the same region? Do they think there's a gradient in alEC that allows computational of long and short temporal/ordinal distances?

– The authors consider "spatial distance" to only denote Euclidean distance. While the teleport condition clearly can distinguish between Euclidean spatial distance and other distances, it is not sufficient to infer temporal distance. For instance, path distance and geodesic distance are relevant here. This is a crucial point, since other sorts of distance such as path distance and communicability distance could be correlated with representational similarity results that the authors take to be uniquely indicative of time distance – depending on the sequence the participants observed.

This is a major concern: dissociating both Euclidean and non-Euclidean spatial distances from temporal distance requires control experiments and a bit of computational work.

Figure 2—figure supplement 1: This figure regarding the relevance of ordinal distances should be in the main text. In addition, please discuss the relationship between time and ordinal distance and compare them to other types of distance (path/geodesic, communicability, etc.). Specifically, if the correlation to other types of distance turned out to be significant that should be shown in the main text as well. Do other distance measure not previously discussed in the paper also capture the patterns in the alEC?

On a related note: in the Materials and methods, the analysis does not consider spatial distance per se, but merely Euclidean distance. While the analysis can dissociate Euclidean distance from ordinal/temporal distance, the authors thus cannot conclude that they have dissociated space and time. This section can be revised depending on the results of the analyses/simulations the authors would conduct involving other forms of distance mentioned above.

As a potential future experiment, if the speed of motion was controlled (e.g. participants were taken from A to B with fixed geodesic or ordinal or path or goal or communicability distances with controlled varying speeds) then it would be easier to decide whether the distance denotes time or some form of non-Euclidean spatial distance such as path distance (which will probably correlate with ordinal distance). Changing speed would make it clear whether path distance or geodesic or communicability is relevant here or time distance can explain the variance observed in the region of interest.

– While it is not necessary that the authors run the following analyses, here are some thoughts. It is possible that the authors have the required data to test the demands of the speed-controlled experiment. Specifically, since the experiments are self-paced, it is possible that the authors can compare pattern similarity in temporal distances separated by geodesic distance x speed variations. If the present experiment does not include sufficient number of trials per subject to make this analysis work, perhaps a super-group analysis may be possible (which the authors also are not required to perform). For instance, one could use SRM or other methods of pooling individual data into a group space and explore the nature of the distance x speed x time interaction.

Whether the authors run the previous analysis or not, this is an important point also because they report similarity to ordinal results, which will also be related to path distance, communicability distance, and other forms of distance measures that can be derived from a graph of the sequence or successor predictive representations (and can still be dissociated from Euclidean distance).

The authors could discuss their reasoning about why the potentially object trace, ordinal, or distance code finding is temporal in nature in the Discussion. They could discuss future experiments and analyses that can discern whether this is an exclusively temporal representation or it could be object race or path distance or other spatially relevant distance in their future directions (e.g. time can be discerned by varying speed across similar distances in a controlled fashion). However, making a strong claim about "time" in the title given the present findings and the role of ordinal information seems unwarranted.

Essential Revisions

Broadly: Is the finding clearly about time or something else (e.g. ordinal information, object reactivation, distance, etc.)?

Different reviewers had questions about the temporal nature of the findings and other interpretations. One of the reviewers notes that unlike the Tsao paper, this manuscript does not show direct evidence for a temporal representation in alEC. Here temporal (or related) information is incorporated into the response to a repeated stimulus perhaps due to recovery of temporal information – a jump back in time. The contrast to spatial information pmEC is quite meaningful and validates the relevance of a large body of rodent work for studies of human memory. These results are just as well predicted by models of human memory and localize the temporal vs. spatial aspect to the alEC vs. pmEC. Many models of human memory (including the temporal context model as well as a Laplace transform of time) hypothesize gradually-decaying activation of preceding information. This is also just what the Tsao Nature paper found. While one can choose to call this decaying trace of an object representation or a temporal code, the authors are advised to clarify that the temporal nature is not exclusively established early on, throughout the manuscript and by including the ordinal figure in the main results, and in the Discussion.

The comments are listed below. Unless the authors are inclined to conduct control experiments that verifies the 'temporal' nature of the findings, they are advised to revise the title as the present findings do not clearly delineate a temporal code. Detailed comments are listed below.

We would like to thank the reviewers for their positive evaluation of our manuscript and the constructive feedback. We appreciate the helpful comments and suggestions and have taken the opportunity to clarify our views on the points raised by the reviewers. In line with the reviewers’ suggestions we have refined the interpretation of our results in the revised manuscript. As pointed out correctly, we do not measure temporal information during the learning task. Rather, our analysis capitalizes on changes in multi-voxel activity patterns elicited by the object cues during the picture viewing tasks before and after learning. In our design, we present stimuli in random order during the picture viewing task and yet observe activity patterns that reflect the temporal structure experienced during learning. Indeed, this is consistent with the notion of a “jump back in time” elicited by the object representations. For example, our results could be explained by a reinstatement of temporal contextual tags or by a reactivation of objects encountered nearby in time. We agree that these explanations are in line with models of human memory. The findings by Tsao et al. (2018), who decode temporal information from neural signals during ongoing navigation behavior, but do not focus on memory per se, are of relevance by emphasizing the role of the lateral entorhinal cortex, specifically. This served as our motivation to analyze pattern similarity change in the entorhinal subdivisions separately.

We have followed the suggestion to incorporate the analysis using ordinal temporal distances in the main manuscript. Further, we more prominently discuss this finding and describe that ordinal distances also reflect the temporal structure of the event sequence, but that with the current design we cannot disentangle the precise scale of temporal representations. We have further adapted the title as suggested by the reviewers. The title of the manuscript is now “Mapping sequence structure in the human lateral entorhinal cortex”. As we describe below and throughout the manuscript, both elapsed time and ordinal temporal distances capture the temporal structure of the sequence participants experienced during the learning task. Please find our detailed responses to the individual comments below.

– The authors state their findings corroborate those of Tsao and colleagues (2018), which show temporal coding in the EC in rats. Specifically, Tsao et al. say they find EC codes for moment-to-moment changes in experience and explain the temporal signal may be less of a clocking signal and more of an experience timestamp.

Do the authors interpret their finding along the same lines? Moreover, the authors show their effect is replicated when they do an ordinal analysis. Thus, do the authors claim the alEC signals temporal order specifically rather than time or experience? It would be useful for the authors to elaborate on the interpretation of their effect especially in light of the interpretation given by Tsao et al. for the EC temporal coding.

The reviewers ask the important question which role specifically the alEC might play for temporal coding. We appreciate the opportunity to offer our views on this issue. Tsao et al. (2018) emphasize that neural activity in the lateral subdivision of the entorhinal cortex in particular carries temporal information and suggest that this is due to the uniqueness of experience at every moment in time. In terms of the anatomical location of the effect, our data are in line with these findings in rodents as the anterior-lateral entorhinal cortex (alEC) is considered the human homologue region of the rodent LEC (Navarro Schröder et al., 2015; Maass et al., 2015). Further, they report cells with activity profiles that vary linearly over time. As alluded to above, no neural data was collected during the learning phase of our study. This precludes making strong interpretations about how specifically the effect we observe in the alEC relates to activity during the learning task. One explanation would be indeed that a slowly drifting signal, which might vary similarly on each lap due to the experience of navigating the same route (c.f. Tsao et al., 2018), provides timestamps for the object encounters during learning. These might in turn be reactivated in the alEC during the picture viewing task after learning, which could give rise to activity patterns reflecting the temporal distances of the task. This would be consistent with neural “jumps back in time” that have previously been observed in the human brain during image recognition tasks (Howard et al., 2012; Folkerts et al., 2018). Given the evidence for gradually drifting population activity in the human medial temporal lobe such an explanation seems more parsimonious to assume than an explicit clocking signal.

Our main analysis uses the median elapsed time between object encounters to show that alEC pattern similarity change correlates with the temporal structure of object relationships. We observe comparable results when quantifying the temporal structure of object relationships on an ordinal level of measurement using the difference in sequence position. We would like to stress that in our view such a representation of the serial order likely reflects a representation of the temporal structure of the encountered event sequence. One way how ordinal representations of the object sequence could arise is through object-to-object associations. For example, chaining models (e.g. Lewandowsky and Murdock, 1989; Jensen and Lisman, 2005) would predict that, during learning, an object is associated with preceding and successive objects in the sequence. Positions closer together in the sequence could result in stronger associations. During the picture viewing task after learning, object cues could result in the reactivation of associated objects. The strength of this reactivation might be more or less strong based on the strength of the association. This might give rise to activation patterns reflecting the serial order of events in a holistic fashion as observed in the alEC. Related to this question, we test in a new analysis in response to Comment 6 whether we can decode object representations in the entorhinal cortex and the lateral occipital cortex (LOC) using support vector machines. In brief, we do not observe stable object representations from the pre- to the post-scan picture viewing task in the entorhinal cortex.

While we could decode object identity from voxel patterns in the LOC, we did not observe evidence for the cortical reinstatement of temporally contiguous objects, which might be expected if the reactivation of object representations would underlie the effects we observed (see our response to Comment 6 for a detailed description of the analysis and results).

To dissociate whether temporal structure is represented on an ordinal as opposed to an interval or logarithmic level one would need an experimental design in which order and time/experience are at least partly decorrelated. Our experiment was designed to dissociate temporal and Euclidean spatial distances between objects and therefore the objects were spread fairly evenly along the route and participants’ movement speed was constant throughout the environment, which is why we cannot disentangle elapsed time from ordinal positions. In the revised manuscript, we have expanded our considerations of how a representation of the sequence structure might arise and how future studies might be able to dissociate ordinal temporal distances from time elapsed.

Please see below for the revised sections of the manuscript.

Introduction section

“This temporal information was suggested to arise from the integration of experience rather than an explicit clocking signal (Tsao et al., 2018).”

Discussion section

“While this interpretation is in line with data from rodent electrophysiology (Tsao et al., 2018) and the framework proposed by the temporal context model (Howard and Kahana, 2002; Howard et al., 2005) as well as evidence for neural contiguity effects in image recognition tasks (Howard et al., 2012; Folkerts et al., 2018), we cannot test the reinstatement of specific activity patterns from the learning phase directly since fMRI data were only collected during the picture viewing tasks in this study.”

“An alternative explanation for how the observed effects might arise is through associations between the objects. […] Hence, these results fail to provide evidence for the notion that the reactivation of object representations drove our effects.”

“In this experiment, the paradigm was designed to disentangle temporal distances from Euclidean spatial distances between objects (Deuker et al., 2016). […] This might allow the investigation of the level of precision at which the hippocampal-entorhinal region stores temporal relations, in line with evidence for the integration of duration information in the representations of short sequences (Thavabalasingam et al., 2018, 2019).”

– Regarding the analysis which included the pmEC (which is meant to be the human homologue to the MEC in rats), should one not expect to see a big change in pattern similarity for objects spatially close together? i.e. pmEC is meant to contain grid cells which code for space, rather than time (or at least more robustly than time). Could the authors comment on why they think they do not observe the reverse effect in pmEC compared to alEC?

The reviewers here ask the question why we did not observe reliable pattern similarity changes scaling with spatial distances in the posterior-medial subdivision of the entorhinal cortex in this task. This is a very interesting question given the wealth of literature describing spatial coding in the medial entorhinal cortex. Perhaps most prominently, grid cells have been discovered in the MEC (Hafting et al., Nature, 2005). Consistently, we have previously observed hexadirectional signals in the human pmEC (Bellmund et al., 2016), which are thought of as a proxy measure for activity in the entorhinal grid system in fMRI (Doeller et al., 2010). Models of grid-cell function suggest positions to be encoded by grid-cell population vectors in pmEC (e.g. Fiete et al., 2008; Mathis et al., 2012; Bush et al., 2015). Such spatial representations could be reactivated during the picture viewing task after learning in our study. However, cueing different positions might not result in BOLD activity patterns reflecting spatial distances. The analyses designed to detect grid-like entorhinal signals with fMRI are directional in nature, i.e. they contrast activity as a function of directions sampled in different trials (c.f. Doeller et al., 2010 for the univariate approach and Bellmund et al., 2016 for an adaptation of the analysis to multivoxel patterns). By presenting participants with isolated object images during picture viewing, we did not sample trajectories of different directions in this task and hence might not have been sensitive to spatial maps in pmEC if these were based on grid cell population codes.

We have made this notion explicit in the revised manuscript. The new section of the Discussion reads as follows:

“In line with hexadirectional signals in pmEC during imagination (Bellmund et al., 2016), putatively related to grid-cell population activity (Doeller et al., 2010), one might expect the pmEC to map spatial distances between object positions in our task. […] Hence, the design here was not optimized for the analysis of spatial representations in pmEC, if the object positions were encoded in grid-cell firing patterns as suggested by models of grid-cell function (Fiete et al., 2008; Mathis et al., 2012; Bush et al., 2015).”

– Another reason this is important regards the time scale of temporal distances in the Tsao experiment compared to the present study. Given the large time scales here, and the fact that the rodent electrophysiology data related to smaller time scales, it is possible that this result has to do with other measures/scales of distance. Please discuss.

The reviewers here ask the question how the time scale of our experiment compares to the experiment by Tsao et al. (2018), which decoded temporal information from LEC population signals. In our design, participants took around 264.6 ± 47.8s (mean ± standard deviation of median time per lap; see caption of Figure 1) to complete one lap of the route through the virtual city along which the 16 objects were encountered. In the rodent experiment, animals foraged for food in sessions consisting of a sequence of 12 trials with a trial length of 250s each. Tsao et al. show that they can not only decode trial identity, but also that they can decode shorter within-trial epochs. In their Figure 3F and Figure 3G, the authors show above chance decoding for epoch lengths of 20s, 10s and 1s. In our view, the trial length of 250s is comparable to the length of a lap of the route in our study. Further, epochs with a length of 10s and 20s constitute a similar temporal scale in comparison to our experiment, where objects were encountered on average every 16.6 ± 5.0s (mean ± standard deviation) on each lap. Therefore, we believe that the temporal scales can actually be regarded as comparable. Yet, of course, a key difference that remains is that Tsao et al. base their analyses on the period of ongoing activity, whereas our analyses focus on representations after learning. Nonetheless, the question of temporal scales in memory is intriguing as also discussed in response to the following comment. Further, in our view, this does not preclude other temporal scales in human memory, where temporal relations might also be represented on different levels or chunked in superordinate hierarchical structures such as different days or weeks.

We have made the match in the length of one lap in our design and the length of a trial in the paper by Tsao et al. explicit in the revised version of the manuscript.

Discussion section

“In our task, one lap of the route took approximately 4.5 minutes on average; comparable to the 250s-duration of a trial in Tsao et al. (2018).”

– Do the authors believe that short/long time scales are coded by the same region? Do they think there's a gradient in alEC that allows computational of long and short temporal/ordinal distances?

The reviewers here ask the question whether different time scales are represented by the same brain regions and whether there might be a gradient of temporal representation in the alEC. Relevant to the question how different time scales are encoded in the subregions of the hippocampal formation is the time cell literature. Time cells have been observed in the hippocampus (e.g. Pastalkova et al., 2008; MacDonald et al., 2011; Kraus et al., 2013; Mau et al., 2018) and medial entorhinal cortex (Kraus et al., Neuron, 2015) of the rodent brain during running in place. The length of the temporal intervals tiled by the sequential firing of time cells are typically in the range of seconds, constituting a neural code on a shorter temporal scale. Notably, the firing patterns of time cells with elevated firing at specific moments during the delay differ from firing patterns in the alEC where some cells’ activity varied linearly over time (Tsao et al., 2018).

Of note, studies investigating time cells typically employ highly-trained tasks including a repeated temporal delay, which constitutes a difference to temporal information derived from decaying traces of prior experience. Conceptually, we believe that slowly drifting population signals carrying temporal information arising from decaying traces of prior experience could more easily provide temporal context information for naturalistic episodic memory where temporal intervals are typically not repeated. Temporal information based on timestamps from a slowly varying signal might be a way to inherently encode temporal structure even without longer training procedures, a property desirable for episodic memory. Despite changes in the ensemble of time cells active over different sessions (Mau et al., 2018), it remains unclear whether time cells indeed also encode longer intervals. One recent study failed to detect time cells for a delay with a length of 60s (Sabariego et al., 2019).

A question that arises from this consideration is how time cells would behave in our task where one lap of the route is characterized by multiple intervals between the different events. One possibility might be that a long sequence of time cells could encode the temporal progression along the entire route. Alternatively, each interval between object encounters could be encoded by the same firing sequence since object encounters occurred at fairly regular temporal intervals along the route.

With respect to different granularities of representations there is evidence from studies in rodents for a granularity increase along the dorsoventral hippocampal axis in rodents. Place field size in the hippocampus and grid scale in the medial entorhinal cortex increase from more dorsal to ventral recording sites in the rodent brain (Kjelstrup et al., 2008; Barry et al., 2007; Stensola et al., 2012). Consistently, gradients in the scale of mnemonic networks (Collin et al., 2015) and fMRI voxel dynamics (Brunec, Bellana et al., 2018) have been documented along the hippocampal long axis in the human brain. To the best of our knowledge, there is no evidence for a gradient of representations in the lateral subdivision of the entorhinal cortex. Using fMRI, one possibility to investigate this question in future studies could be to scrutinize the particularly slow fluctuations of the BOLD signal in the EC (Lositsky et al., 2016) in more detail. Specifically, one could apply the analysis approach developed by Brunec et al. (Curr. Bio., 2018) and contrast time courses and temporal autocorrelation of voxels at different anatomical positions within the alEC. Given the small size of this region of interest, the increased spatial resolution of fMRI at 7T might be required. A gradient within the alEC could then be measured by greater similarity among voxel time courses and higher temporal autocorrelation in anterior compared to more posterior voxels (c.f. Brunec et al., 2018).

Discussion section

“Time cell ensembles change over minutes and days (Mau et al., 2018), but their firing has been investigated predominantly in the context of short delays in the range of seconds. […] How memories are represented at different temporal scales, which might be integrated in hierarchically nested sequences such as different days within a week, remains a question for future research.”

– The authors consider "spatial distance" to only denote Euclidean distance. While the teleport condition clearly can distinguish between Euclidean spatial distance and other distances, it is not sufficient to infer temporal distance. For instance, path distance and geodesic distance are relevant here. This is a crucial point, since other sorts of distance such as path distance and communicability distance could be correlated with representational similarity results that the authors take to be uniquely indicative of time distance – depending on the sequence the participants observed.

This is a major concern: dissociating both Euclidean and non-Euclidean spatial distances from temporal distance requires control experiments and a bit of computational work.

Figure 2—figure supplement 1: This figure regarding the relevance of ordinal distances should be in the main text. In addition, please discuss the relationship between time and ordinal distance and compare them to other types of distance (path/geodesic, communicability, etc.). Specifically, if the correlation to other types of distance turned out to be significant that should be shown in the main text as well. Do other distance measure not previously discussed in the paper also capture the patterns in the alEC?

On a related note: in the Materials and methods, the analysis does not consider spatial distance per se, but merely Euclidean distance. While the analysis can dissociate Euclidean distance from ordinal/temporal distance, the authors thus cannot conclude that they have dissociated space and time. This section can be revised depending on the results of the analyses/simulations the authors would conduct involving other forms of distance mentioned above.

As a potential future experiment, if the speed of motion was controlled (e.g. participants were taken from A to B with fixed geodesic or ordinal or path or goal or communicability distances with controlled varying speeds) then it would be easier to decide whether the distance denotes time or some form of non-Euclidean spatial distance such as path distance (which will probably correlate with ordinal distance). Changing speed would make it clear whether path distance or geodesic or communicability is relevant here or time distance can explain the variance observed in the region of interest.

The reviewers raise an important point and comment on different distance measures that might describe pattern similarity changes in the entorhinal cortex. Our paradigm was designed to dissociate Euclidean spatial distances and elapsed time between objects. We agree that there are other distance measures that can be used to quantify spatial relationships. In terms of the geodesic distance, we implemented two approaches of finding the shortest path between all pairs of object positions. First, we focused on all locations in the virtual city that were not obstructed by the collision volumes of virtual buildings, trees, or other objects distributed throughout the city and created a corresponding map of valid locations. Alternatively, one might consider only the streets as valid locations for navigation. Indeed, participants were instructed to stay on the streets during the learning phase and a prompt appearing on the screen reminded them to do so whenever they left the street network. Consequently, we created a second map in which only the streets of the virtual city were navigable positions. For either approach, we used a Matlab implementation of the A* search algorithm (https://mathworks.com/matlabcentral/fileexchange/56877) to find the shortest paths between all pairs of object positions. Examples of the resulting shortest paths are shown in Figure 1—figure supplement 2D and E. Geodesic distances were then quantified as the lengths of these shortest paths. Importantly, geodesic distances were not correlated with temporal distances measured as median elapsed time (Figure 1—figure supplement 2B and C; based on obstacles: mean and standard deviation of individual Pearson r=-0.061 ± 0.006, minimum p=0.414, correlation with averaged temporal distance: Pearson r=-0.061, p=0.505; based on streets: mean and standard deviation of individual Pearson r=-0.041 ± 0.006, minimum p=0.552, correlation with averaged temporal distance: Pearson r=-0.041 p=0.653). Entorhinal pattern similarity change was not significantly correlated with the geodesic distances based on obstacles in the virtual city (alEC: T(25)=0.82, p=0.436; pmEC: T(25)=0.73, p=0.479, Figure 2—figure supplement 2A) or the street network (alEC: T(25)=0.36, p=0.715; pmEC: T(25)=0.92, p=0.375, Figure 2—figure supplement 2B). Pattern similarity change in alEC was more strongly related to temporal than geodesic distances: akin to the main analyses, we conducted a 2x2 repeated measures ANOVA with the factors EC subregion and temporal vs. geodesic distances based on the obstacle map, revealing a significant interaction (main effect subregion: F(1,25)=5.18, p=0.031, main effect distance type:

F(1,25)=0.99, p=0.330, interaction: F(1,25)=6.96, p=0.014, post hoc comparison of temporal and geodesic distance in alEC: T(25)=-2.88, p=0.009). Likewise, we observed comparable results when using geodesic distances based on the street network (main effect subregion: F(1,25)=6.68, p=0.017, main effect distance type: F(1,25)=0.81, p=0.376, interaction: F(1,25)=4.30, p=0.048, post hoc comparison of temporal and geodesic distance in alEC: T(25)=-2.51, p=0.019). Taken together, these data highlight that pattern similarity change in alEC was not related to geodesic spatial distances between object positions.

The reviewers further point towards the path and communicability distance as a measure of spatial distances. In our analyses, the lengths of the paths between objects are almost perfectly correlated with the time elapsing between object encounters. This is due to the fact that travelled distances and elapsed time are identical unless participants take breaks from navigating. Since our measures of representational similarity are acquired before and after the navigation of the route, we are not sensitive to stops on individual laps of the route because we have to rely on measures describing central tendencies of participants’ behavior during the learning task. Because participants navigate the route repeatedly, consistent biases in stopping behavior across laps are unlikely. If one follows the notion that correlations between pattern similarity change in alEC and the temporal structure of the task arise not from a ticking clock, but from the association of objects with a slowly drifting contextual signal, e.g. through the decaying trace of prior experience, the travelled distances can be conceived of as an additional proxy measure of past experience that is closely related to elapsed time (mean and standard deviation of individual Pearson r=0.98 ± 0.005, all p<0.001; correlation with averaged temporal distance: Pearson r=0.98, p<0.001). However, there seems to be, to the best of our knowledge, little evidence in the literature that the human alEC or rodent LEC specifically would be involved in the mapping of distances travelled along a path in a spatial sense. Rather, keeping track of travelled distances is closely related to path integration for which grid cells found in the (posterior-) medial subdivision of the EC are thought to be of central importance in rodents and humans (Hafting et al., Nature, 2005; Gil et al., Nat. Neurosci. 2018; Chen et al., Curr. Bio., 2015; Stangl et al., Curr. Bio., 2018). In sum, the path distance along the route in our experiment and, more generally, travelled distances might be important contributors to the accumulation of experience in the context of spatial navigation. The reviewers discuss the experimental idea to dissociate the path distance from temporal distances (both elapsed time or ordinal distances) by controlling participants’ walking speed. As also discussed in response to Comments 2 and 7, we agree that variations of movement speed would be a suitable manipulation to disentangle the path distance from temporal relationships between positions in a future experiment.

In contrast, we do not think that the communicability distance offers a plausible measure for the object relationships in this task. While it would be possible to convert the street network and object positions into a graph structure, the communicability distance provides a suboptimal measure to quantify participants’ learning experience in this experiment in our view. This is because participants experienced the objects in a deterministic sequence by navigating along a fixed route that was designed to have little overlap. In fact, only a short section of one street was traversed twice on one lap of the route (see Figure 1, paths from chest 1 to 2 and chest 13 to 14) and no object was encountered on this stretch. BOLD-signals in the entorhinal cortex have been shown to be sensitive to communicability distances between nodes on a graph when stimulus sequences during learning reflect random walks along the underlying graph (Garvert et al., 2017). However, the ambiguity of the stimulus sequence constitutes a marked difference to the deterministic structure of the object sequence encountered along a route consisting almost exclusively of unique paths through the city. Therefore, we have not correlated communicability distances with pattern similarity change.

We have revised the manuscript according to the suggestions by the reviewers. We have incorporated the results of the analysis using ordinal temporal distances into the main manuscript. Further, we have included the analysis of geodesic distances in Figure 1—figure supplement 2 and Figure 2—figure supplement 2. We have carefully gone through the manuscript to specify where spatial distances are operationalized as Euclidean distances. Please see below for the revised sections of the manuscript.

See Figure 4, Figure 1—figure supplement 2B-E and Figure 2—figure supplement 2.

Introduction section

“We used representational similarity analysis of fMRI multi-voxel patterns in the entorhinal cortex to address the question how learning the structure of an event sequence shapes mnemonic representations in the alEC.”

Results section

“The temporal distance structure of the object sequence can be quantified as the elapsed time between object encounters or as ordinal differences between their sequence positions, which are closely related in our task. Spatial distances on the other hand can be captured by Euclidean or geodesic distances between positions. Importantly, we dissociated temporal from Euclidean and geodesic spatial object relationships through the use of teleporters along the route (Figure 1—figure supplement 2).”

“Pattern similarity change in alEC did not correlate significantly with Euclidean spatial distances (T(25)=0.81, p=0.420) and pattern similarity change in posterior-medial EC (pmEC) did not correlate with Euclidean (T(25)=0.58, p=0.583) or temporal (T(25)=1.73, p=0.089) distances.”

“Operationalizing the temporal structure in terms of the ordinal distances between object positions in the sequence yielded comparable results since our design did not disentangle time elapsed from ordinal positions as objects were encountered at regular intervals along the route. […]Furthermore, the interaction of the two-by-two repeated measures ANOVA with the factors entorhinal subregion and distance type remained significant when using geodesic spatial distances based on shortest paths using all non-obstructed positions (interaction: F(1,25)=6.96, p=0.014; main effect of EC subregion: F(1,25)=5.18, p=0.031; main effect of distance type: F(1,25)=0.99, p=0.330) or the street network only (interaction: F(1,25)=4.30, p=0.048; main effect of EC subregion: F(1,25)=6.68, p=0.017; main effect of distance type: F(1,25)=0.81, p=0.376).”

Discussion

“In our task, relevant factors contributing to a similar experience of the route on each lap are not only the encounters of objects in a specific order at their respective positions, but also recognizing and passing salient landmarks as well as travelled distance and navigational demands in general.”

Materials and methods

“The use of teleporters, which instantaneously moved participants to a different part of the city, enabled us to dissociate temporal from Euclidean and geodesic spatial distances between object positions (Figure 1—figure supplement 2).”

“Indeed, temporal distances across all comparisons of object pairs were not correlated with spatial relationships measured as Euclidean distances (Figure 1—figure supplement 2A).”

“An alternative way of capturing the spatial structure of the task is via geodesic distances. We quantified geodesic distances as the lengths of the shortest paths between object locations. Shortest paths were calculated using a Matlab implementation of the A* search algorithm (https://mathworks.com/matlabcentral/fileexchange/56877). First, we calculated shortest paths that were allowed to cross all positions not obstructed by buildings or other obstacles (see Figure 1—figure supplement 2D for example paths). Second, because participants were instructed to only navigate on the streets during the learning task, we found shortest paths restricted to the city’s street network (example paths are shown in Figure 1—figure supplement 2E). Neither form of geodesic distances between object positions was correlated with temporal distances (Figure 1—figure supplement 2BC).”

– While it is not necessary that the authors run the following analyses, here are some thoughts. It is possible that the authors have the required data to test the demands of the speed-controlled experiment. Specifically, since the experiments are self-paced, it is possible that the authors can compare pattern similarity in temporal distances separated by geodesic distance x speed variations. If the present experiment does not include sufficient number of trials per subject to make this analysis work, perhaps a super-group analysis may be possible (which the authors also are not required to perform). For instance, one could use SRM or other methods of pooling individual data into a group space and explore the nature of the distance x speed x time interaction.

Whether the authors run the previous analysis or not, this is an important point also because they report similarity to ordinal results, which will also be related to path distance, communicability distance, and other forms of distance measures that can be derived from a graph of the sequence or successor predictive representations (and can still be dissociated from Euclidean distance).

The authors could discuss their reasoning about why the potentially object trace, ordinal, or distance code finding is temporal in nature in the Discussion. They could discuss future experiments and analyses that can discern whether this is an exclusively temporal representation or it could be object race or path distance or other spatially relevant distance in their future directions (e.g. time can be discerned by varying speed across similar distances in a controlled fashion). However, making a strong claim about "time" in the title given the present findings and the role of ordinal information seems unwarranted.

The reviewers here offer interesting suggestions for additional analyses to attempt to dissociate elapsed time from spatial distances. Unfortunately, the data we have from this experiment does not allow us to analyze the relationship of pattern similarity change and temporal distances for different walking speeds. The learning task was indeed self-paced, so there are variations in navigation efficiency and potentially walking speeds across laps. However, the representational change is only assessed after the learning task. Hence, we do not have a lap-by-lap measure of object representations that would allow more fine-grained analysis. Rather, we have to select one variable from the learning task and relate representational change to it. Performing an analysis where temporal distances are assessed as a function of speed would require data in which speed variations are consistent across laps by a participant. This would require the explicit experimental manipulation of walking speeds between different objects in a new experiment. We describe this idea for a future study in the Discussion.

The reviewers suggest to additionally discuss the potential role of object traces or predictive representations as possible explanations for the results. Since objects were presented in random order during the picture viewing tasks, memory-based reactivations of objects preceding or following a cued object would be required to explain the observed effect. In new analyses, we aimed to test predictions that can be derived from the idea that the observed effects reflect the, potentially predictive, reactivation of neighboring object representations. If object cues during the post-learning picture viewing task were to reactivate neighboring objects from a learned sequence, we should be able to detect such object representations. To test this idea, we turned towards a classification analysis. Using the Matlab function fitcecoc, we trained support vector machines in a one-vs.-one coding design to distinguish activity patterns evoked by the different object cues in different regions of interest. In addition to the entorhinal cortex, these included the lateral occipital cortex (LOC) – known to be involved in the visual processing of objects (for review see Grill-Spector et al., 2001) – to test for cortical reinstatement of object representations. We trained the classifiers on the data from the pre-learning scan to test for similar representations during the post-learning scan.

Using this procedure, we were able to classify object identities significantly above chance levels determined through random permutations of trial labels in the LOC (T(25)=7.54, p<0.001, Figure 2—figure supplement 4). However, decoding accuracies were at chance level for both entorhinal ROIs (alEC: T(25)=-0.08, p=0.941; pmEC: T(25)=0.53, p=0.621). If object images during the post-learning scan serve as cues for the reactivation of neighboring objects, objects n-1 and n+1 might be activated when viewing object n. This might result in systematic errors of the classifier. Hence, we analyzed the classifier evidence as a function of sequence position lag to test for above-chance classifier confusion using one-sided t-tests. Focusing on lags ± 3, we did not observe any above-chance classifier evidence for preceding (alEC: most extreme T(25)=1.13; minimum p=0.270; pmEC: most extreme T(25)=1.00; min. p=0.332) or upcoming (alEC: most extreme T(25)=-2.07; min. p=0.055; pmEC: most extreme T(25)=-0.83; min. p=0.414) objects in the entorhinal cortex. Based on evidence for the reinstatement of cortical activity patterns during retrieval (Nyberg et al., 2000; Wheeler et al., 2000; Polyn et al., 2005), which is modulated by hippocampalentorhinal activity (Bosch et al., 2014), we also performed this analysis in the LOC, but again failed to observe above-chance evidence for preceding or following objects. Rather classifier evidence was below chance levels, potentially due to the high accuracy at no lag (preceding objects: most extreme T(25)=-2.51; min. p=0.018; successive objects: most extreme T(25)=-4.09; min. p<0.001). Taken together, the lack of stable object representations in the entorhinal cortex makes it unlikely that the effects we observed go back to a decaying object trace or the reactivation of objects preceding or following a presented object along the route.

As noted above already, we have followed the suggestion to revise the title of the manuscript, which now reads “Mapping sequence structure in the human lateral entorhinal cortex”. Throughout the manuscript, we describe that the temporal structure of the object sequence can be quantified at different levels of measurement. Again, we would like to emphasize that, in our view, ordinal-level representations of the sequence also reflect the temporal structure of the object sequence.

In the revised manuscript, we have included the classification analyses and discuss potential future experiments dissociating elapsed time from ordinal temporal distances and spatial distances such as the path distance. The new figures and revised sections of the manuscript are:

Title: Mapping sequence structure in the human lateral entorhinal cortex

Figure 2—figure supplement 4

Results section

“Does the presentation of object images during the post-learning picture viewing task elicit reactivations of similar representations from the pre-scan? […] We also did not observe abovechance classifier evidence for nearby objects in the LOC, but rather classifier evidence was below chance levels for some lags, potentially due to high classification accuracies at no lag (preceding objects: most extreme T(25)=-2.51; min. p=0.018; successive objects: most extreme T(25)=-4.09; min. p<0.001).”

“An alternative explanation for how the observed effects might arise is through associations between the objects. […] Hence, these results fail to provide evidence for the notion that the reactivation of object representations drove our effects.”

Materials and methods

“ROI masks for the bilateral lateral occipital cortex were defined based on the Freesurfer segmentation and intersected with the combined brain masks from the two fMRI runs since this ROI was located at the edge of our field of view.”

“Classification analysis

To examine whether object representations were stable between the pre- and the postlearning scan, we turned to pattern classification techniques and examined whether classifiers trained on the pre-learning scan exhibited systematic errors when tested on the post-learning data. […] Note that classifier performance is below chance for some preceding and upcoming sequence positions due to high accuracy at lag 0.

Author details

1. Jacob LS Bellmund

   1. Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
   2. Kavli Institute for Systems Neuroscience, Norwegian University of Science and Technology, Trondheim, Norway
   3. Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, Netherlands

   Contribution

   Conceptualization, Formal analysis, Investigation, Visualization, Writing—original draft, Writing—review and editing

   For correspondence

   bellmund@cbs.mpg.de

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-2098-4487

2. Lorena Deuker

   Department of Neuropsychology, Institute of Cognitive Neuroscience, Faculty of Psychology, Ruhr University Bochum, Bochum, Germany

   Contribution

   Conceptualization, Formal analysis, Investigation, Writing—review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-4939-5862

3. Christian F Doeller

   1. Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
   2. Kavli Institute for Systems Neuroscience, Norwegian University of Science and Technology, Trondheim, Norway

   Contribution

   Conceptualization, Supervision, Funding acquisition, Writing—review and editing

   For correspondence

   doeller@cbs.mpg.de

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0003-4120-4600

• 3,193

  Page views

• 476

  Downloads

• 33

  Citations

Article citation count generated by polling the highest count across the following sources: Scopus, Crossref, PubMed Central.