Flexible neural representations of abstract structural knowledge in the human entorhinal cortex
- Wellcome Centre for Human Neuroimaging, University College London, United Kingdom
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, United Kingdom
- Sainsbury Wellcome Centre for Neural Circuits and Behaviour, University College London, United Kingdom
Figures
Figure 1
Subspace generalization across environments in grid and place cells in data from Chen et al., 2018.
(a) Illustration of the subspace generalization method. The principal components (PCs) are calculated using the covariance matrix of the neuronal activity matrix. Then the activity matrix is projected on each PC (recorded when the animal was in the same or different environment/task) and the variance explained along each PC dimension is calculated. We calculate the area under the curve (AUC) of the cumulative sum of the variance explained on each PC’s dimension as our similarity measure. When the similarity in neuronal patterns during the two different tasks is higher, the AUC is higher (green AUC is added to the blue AUC). (b) The cumulative variance explained by the PCs calculated using the activity of grid (black) or place (green) cells, within (solid lines) and across (dotted lines) environments. Subspace generalization is calculated as the difference between the AUC of two lines. The difference between the black lines is small, indicating generalization of grid cells across environments. The difference between the green lines is larger, indicating remapping of place cells (p < 0.001, permutation test, see Methods). (c) The difference between the within and across (solid and dashed lines in (a), respectively) environments AUCs of the cumulative variance explained by grid or place cells (black or green lines in (a), respectively). Data shown for all mice with enough grid or place cells (>10 recorded cells of the same type, each bar is a mouse and a specific projection (i.e. projecting on environment one or two)). The differences between the grid cells AUCs are significantly smaller than the place cells (p < 0.001 permutation test, see Appendix 1 for more statistical analyses and specific examples). (d) An example of the cumulative variance explained by the PCs, calculated using the constructed low-resolution version of grid and place cells data. The solid and dotted lines are average over 10 samples and the shaded areas represent the standard error of the mean across samples. Here, as above, the solid lines are projection within environment and the dotted lines are projections between environments. (e) Subspace generalization in the low-resolution version of the data captures the same generalization properties of grid vs place cells. The distributions were created via bootstrapping over cells from the same animal, averaging their activity, concatenating the samples across all animals, and calculating the AUC difference between within and across environments projections (p < 0.001 Kolmogorov–Smirnov test).
Figure 2
Simulated voxels from simulated grid modules.
(a) Examples of simulated voxels activity map in the two environments, without noise. Upper: higher-frequency module, lower: lower-frequency module. Cells are grouped into voxels randomly. (b) Same as (a), but with cells grouped into voxels according to the grid phase. Note the different scale of the color bar between (a) and (b). (c) Subspace generalization plot for the 16 simulated voxels, where the grouping into voxels is either random (left) or according to phase (right). Legend as in (d), noise amplitude = 0.1. (d) Left: area under the curves (AUCs) of the subspace generalization plots in (c). as a function of the ratio of random vs phase-organized cells in the voxels, with no noise (black) or with high amplitude of noise (blue, noise amplitude = 0.1). Without noise (black lines), the subspace generalization measure (AUC) remains high even when the fraction of randomly sampled cells increases. However, in the presence of noise, the subspace generalization measure decreases with the fraction of randomly sampled cells. Right: p-value of the effect according to the permutation distribution (see methods, shaded area: standard error of the mean). In the presence of noise and when the cells are sampled randomly, AUCwithin-between becomes non-significant; see Appendix 1—figure 3 for the dependency of the permutation distributions on the presence of noise and sampling. (e) Same as (d), except the continuous X-axis variable is the noise amplitude, for either of phase-organized (black) or randomly organized voxels (red). AUC decreases sharply with noise amplitude when the cells are sampled randomly, while it decreases more slowly when the cells are sampled according to phase. The decrease in AUC to chance level (i.e. AUC = 0.5) with the increase in noise amplitude results in insignificant difference in subspace generalization measure (AUCwithin-between). See Appendix 1—figure 3 for the permutation distributions.
Figure 3
Experimental design and behavior.
(A) Example of an associative graph. Participants were never exposed to this top-down view of the graph – they learned the graph by viewing a series of pairs of neighboring images, corresponding to a walk on the graph. To aid memorization, we asked participants to internally invent stories that connect the images. (B) Each participant learned four graphs: two with a hexagonal lattice structure (both learned on days 1 and 2) and two with a community structure (both learned on days 3 and 4). For each structural form, there was one larger graph and one smaller graph. The nodes of graphs with approximately the same size were drawn from the same set of images. (C–F) In each day of training, we used four tests to probe the knowledge of the graphs, as well as to promote further learning. In all tests, participants performed above chance level on all days and improved their performance between the first and second days of learning a graph. (C) Participants were asked whether an image X can appear between images Y and Z (one-sided t-test against chance level (50%): hex day 1 t(27) = 31.2, p < 10–22; hex day 2 t(27) = 35.5, p < 10–23; comm day 3 t(27) = 26.9, p < 10–20; comm day 4 t(27) = 34.2, p < 10–23; paired one-sided t-test between first and second day for each structural form: hex t(27) = 4.78, p < 10–5; comm t(27) = 3.49, p < 10–3). (D) Participants were shown two three-long image sequences and were asked whether a target image can be the fourth image in the first, second, or both of the sequences (one-sided t-test against chance level (33.33%): hex day 1 t(27) = 39.9, p < 10–25; hex day 2 t(27) = 42.3, p < 10–25; comm day 3 t(27) = 44.8, p < 10–26; comm day 4 t(27) = 44.2, p < 10–26; paired one-sided t-test between first and second day for each structural form: hex t(27) = 3.97, p < 10–3; comm t(27) = 2.81, p < 10–2). (E) Participants were asked whether an image X is closer to image Y or images Z, Y and Z are not neighbors of X on the graph (one-sided t-test against chance level (50%): hex day 1 t(27) = 12.6, p < 10–12; hex day 2 t(27) = 12.5, p < 10–12; comm day 3 t(27) = 5.06, p < 10–4; comm day 4 t(27) = 7.42, p < 10–7; paired one-sided t-test between first and second day for each structural form: hex t(27) = 3.44, p < 10–3; comm t(27) = 2.88, p < 10–2). (F) Participants were asked to navigate from a start image X to a target image Y. In each step, the participant had to choose between two (randomly selected) neighbors of the current image. The participant repeatedly made these choices until they arrived at the target image (paired one-sided t-test between number of steps taken to reach the target in first and second day for each structural form. Left: trials with initial distance of two edges between start and target images: hex t(27) = 2.57, p < 10–2; comm t(27) = 2.41, p < 10–2; middle: initial distance of three edges: hex t(27) = 2.58, p < 10–2; comm t(27) = 4.67, p < 10–2; right: trials with initial distance of four edges: hex t(27) = 3.02, p < 10–2; comm t(27) = 3.69, p < 10–3). Note that while feedback was given for the local tests in panels C and D, no feedback was given for the tests in panels E and F to ensure that participants were not directly exposed to any non-local relations. The location of different options on the screen was randomized for all tests. Hex: hexagonal lattice graphs. Comm: community structure graphs.
Figure 4
fMRI experiment and analysis method (subspace generalization).
(a) Each fMRI block starts with 70 s of random walk on the graph: a pair of pictures appears on the screen, each time a participant presses enter a new picture appears on the screen and the previous picture appears behind (similar to the three pictures sequence, sell below). During this phase, participants are instructed to infer which ‘pictures set’ (i.e. graph) they are currently playing with. Note that fMRI data from this phase of the task is not included in the current manuscript. (b) The three pictures sequence: three pictures appear one after the other, while previous picture/s still appear on the screen. (c) Each block starts with the random walk (panel a). Following the random walk, sequences of three pictures appear on the screen. Every few sequences, there was a catch trial in which we asked participants to determine whether the questioned picture can appear next on the sequence. (d) Subspace generalization method on fMRI voxels. Each searchlight extracts a beta X voxels’ coefficients (of three-image sequences) matrix for each graph in each run (therefore, there are four such matrices). Then, using cross-validation across runs, the left-out run matrix of one graph is projected on the EVs from the (average of three runs of the) other graph. Following the projections, we calculate the cumulative percentage of variance explained and the area under this curve for each pair of graphs. This leads to a 4 × 4 subspace generalization matrix that is then being averaged over the four runs (see main text and methods for more details). The colors of this matrix indicate our original hypothesis for the study: that in EC, graphs with the same structure would have larger (brighter) area under the curves (AUCs) than graphs with different structures (darker).
Figure 5
Subspace generalization in visual and structural representations.
(a) Subspace generalization of visual representations in lateral occipital cortex (LOC). Left: difference in subspace generalization was computed between different blocks that included the same stimuli with subspace generalization computed between blocks of different stimuli while controlling for the graph structure, that is [HlHl +ClCl + HsHs + CsCs] − [HlHs + HsHl + ClCs + CsCl]. t(27)_peak = 4.96, p_tfce <0.05 over LOC. Right: visualization of the subspace generalization matrix averaged over all LOC voxels with t > 2 for the [HlHl +ClCl + HsHs + CsCs] − [HlHs + HsHl + ClCs + CsCl] contrast, that is green minus red entries. (b) Entorhinal cortex (EC) generalizes over the structure of hexagonal graphs. Left: the effect for the contrast [HlHl + HlHs + HsHl + HsHs] − [HlCl + HlCs + HsCl + HsCs], that is the difference between subspace generalization of hexagonal graphs data, when projected on principal components (PCs) calculated from (cross-validated) hexagonal graphs (green elements in right panel) vs community structure graphs (red elements). t(27)_peak = 4.2, p_tfce <0.01 over EC. Right: same as in (a) right but for the [HlHl + HlHs + HsHl + HsHs] − [HlCl + HlCs + HsCl + HsCs] contrast in EC. (c) The average effect in an ROI from Baram et al. (green cluster in Figure 3D of Baram et al., 2021) for each participant. Star denotes the mean, error bars are SEM.
Appendix 1—figure 1
Subspace generalization: grid cells generalize over different environments.
(a) Left: grid cells subspace generalization graph for two more mice (averaged over projection on environment 1 and 2). The lower and upper plots share the same legend. Right: permutation distributions of grid cells (number of permutations = 5000). We permuted the cells in the activity matrix (cells × bins) while the animal forages in one environment, then projected on the EVs from the activity matrix while the animal forages in the other environment and calculated the AUC (and vice versa). We then calculated the difference in AUC between the within-environment AUC and the AUC resulting from the permutation and calculated the cdf. Using these distributions, we can conclude that the difference in AUC of within and between environments is smaller than expected by random projections. p < 0.001 for both animals. (b) The place cells’ sampling distribution of the difference in AUC of within and between arenas (number of bootstraps = 1000, for more details see methods, permutation test 2). We treat these distributions as our NULL distributions to answer the question whether grid cells’ AUC within-between is smaller than place cells’ AUC within-between that is do grid cells generalize significantly better than place cells. Upper plot: 14 cells are being sampled, lower plot: 21 cells are being sampled (to match the number of grid cells that were recorded within each animal), here again the calculation repeated twice, each time we project on one of the environments and then averaged the results.
Appendix 1—figure 2
Subspace generalization of place cells.
Left: subspace generalization plots for the different mice. Solid: within environment; dotted: across environments. Right: the corresponding permutation distribution within the corresponding mouse place cells. Cells were permuted before projection (as before). p-values correspond to the significance of the difference in the AUC for within and across environments under this perturbation distribution.
Appendix 1—figure 3
Dependency of subspace generalization on number of voxels per module.
(a) Subspace generalization plot for voxels of the four modules, each module is segregated into two voxels as a function of the ratio of randomly sampled cells. Black – no noise, blue – with noise. Left: AUC (solid – within dash – across environments). Right: p-value of the effect according to the permutation distribution (see methods, shaded area – standard error of the mean). (b) Subspace generalization plot for voxels of 1095 of the 4 modules as a function of the number of voxels per module. Noise std = 50. Black – sampled according to phase, red – sampled randomly. (c) Left: AUC (solid – within dash – across environments). Middle: p-value of the effect according to the permutation distribution (see methods, shaded area – standard error of the mean). Right: signal-to-noise ratio (SNR) as a function of number of voxels per module, where the std (signal) is the averaged standard deviation over the voxels’ activity map across the environment. Permutation distributions (the Null distribution). The shift of the distributions to the left following the introduction of noise/random sampling explains the increase in p-values even though the difference in AUC is still small.
Appendix 1—figure 4
Learning a community structure.
(a) During the first community structure training day, during the navigation task, participants preferred to choose connecting nodes even though it was the wrong answer. (b) Subspace generalization matrix in the PFC (ROI taken from Baram et al., 2021, 125 voxels around 1171 the peak).
Appendix 1—figure 5
Partition of the graphs into three image sequences.
Behavior during the fMRI session.
Appendix 1—figure 6
Behavior during the fMRI session.
Left: Participants were able to detect whether the image in question can indeed follow the current three picture sequence significantly better than chance (t-test, p < 0.001, t[27]hex = 11.3, t[27]cluster = 10.6). Right: Fraction of correct answers to the recognition question at the end of each block (t-test, p < 0.001 for both structures, t[27]hex = 3.8, t[27]cluster = 9.96). Error bars denote SEMs.
Additional files
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Flexible neural representations of abstract structural knowledge in the human entorhinal cortex
eLife 13:RP101134.
https://doi.org/10.7554/eLife.101134.3
