Specific lexico-semantic predictions are associated with unique spatial and temporal patterns of neural activity
- Harvard Medical School, United States
- Massachusetts General Hospital, United States
- Tufts University, United States
- Centre for Human Brain Health, University of Birmingham, United Kingdom
Figures
Figure 1
The experimental procedure and approach for Representational Similarity Analyses.
(A) Trials began with a blank screen (1600 ms). Sentences were presented in Chinese (translated here into English), word-by-word (200 ms per word; 800 ms blank interval between words). Sentences were followed either by ‘NEXT’ (2000 ms) or by a probe question (1/6th of trials, randomly). We constructed sentences in pairs such that the same word could be predicted from the context (e.g. S1-A and S2-A’; S3-B and S4-B') (although during presentation, members of each pair were presented separately, with at least 30 other sentences in between). One member of each pair ended with the predicted word (e.g. S1–A, S3–B) and the other member ended with a plausible but unpredicted word (e.g. S2–A’, S4–B’). Before the onset of the predicted word, we compared brain activity associated with the prediction of the same word (within-pairs) and a different word (between-pairs). (B) Spatial representational similarity analysis. Left: The pattern of MEG data over sensors was correlated between each sentence pair (e.g. S1–A and S2–A’) at each time sample t(j).. Right: The average spatial correlation values of pairs (R1within, R2within, …) in which the same word was predicted formed the within-pair spatial correlation time series (, shown in red). The average spatial correlation values of pairs (R1between, R2between, …) in which different words were predicted formed the between-pair spatial correlation time series (, shown in blue). (C) Temporal representational similarity analysis. Left: The temporal pattern of MEG activity was correlated between sentence pairs, at each sensor (sensor space) or at each grid point (source space). Right: The average temporal correlation values of pairs (R1within, R2within, …) in which the same word was predicted formed the within-pair temporal correlation topographic/source maps. The average temporal correlation values of pairs (R1between, R2between, …) in which different words were predicted formed the between-pair temporal correlation topographic/source maps.
-
Figure 1—source data 1
Chinese stimuli together with their English translations, as well as some measures for the context up until the sentence final word (SFW), the word before the sentence final word (SFW-1) and the sentence final words (SFW).
The first tab shows the full list of the stimuli, the second tab shows a subset of sentence pairs for the control analysis.
- https://doi.org/10.7554/eLife.39061.003
Figure 2 with 4 supplements
Results of the Spatial Representational Similarity Analysis.
(A) The time series of spatial similarity R values combined across the within-pair and between-pair correlations. The horizontal line indicates a threshold of R = 0.04 where the general increase in spatial correlation was largest. (B) The time series of spatial similarity R values for pairs in which the same word was predicted (within-pairs, shown in red) and in which a different word was predicted (between-pairs, shown in blue). Both the within- and the between-pair spatial similarity time series showed a sharp increase at ~100 ms and a decrease at ~500 ms after the onset of each word. Between −880 and −485 ms before the onset of the final word, the spatial similarity was greater when the same word was predicted than when different words were predicted (within-pairs >between-pairs: t(25) = 3.751, p < 0.001). (C) Scatter plots of spatial similarity values averaged between −880 and −485 ms before the onset of the final word in 26 participants. In most participants (18/26) the within-pair spatial correlations were greater than the between-pair spatial correlations. (D) Cross-temporal spatial similarity matrices for the within- and between-pair correlations (Red: positive correlations; blue: negative correlations). Left and middle: Both sets of pairs showed increased spatial similarity along the diagonal with greater similarities for the within- than the between-pairs in the −900 – −500 ms interval prior to the onset of the final word. Right: The matrix shows the cluster with a statistically significant difference between the within-pair and between-pair spatial correlations (p = 0.002, cluster-randomization approach controlling for multiple comparisons over time). The absence of ‘off-diagonal’ correlations suggests that the spatial pattern of neural activity associated with the predicted word was reliable but changed over time.
-
Figure 2—source data 1
Data used for plotting Figure 2 as well as its supplementary Figures.
- https://doi.org/10.7554/eLife.39061.009
Figure 2—figure supplement 1
Results of the Spatial Representational Similarity Analysis after matching the number of pairs between the within-pair and between-pair correlations.
(A) The time series of spatial similarity R values for the pairs in which the same word was predicted (within-pair, shown in red) and in which a different word was predicted (between-pair, shown in blue). Within the −880 – −485 ms interval relative to the onset of the final word, the spatial similarity was greater when the same word was predicted than when different words were predicted (−880 – −485 ms before its onset; t(25) = 2.393, p = 0.025). (B) Scatter plots of the spatial similarity values averaged between −880 and −485 ms before the onset of final word in 26 participants. In most participants (17/26) the within-pair spatial correlations were greater than the between-pair spatial correlations.
Figure 2—figure supplement 2
Results of the Spatial Representational Similarity Analysis in a subset of sentence pairs that had the same pre-sentence-final word (SFW-1) but predicted a different SFW (a subset of between-pairs, shown in blue), and a subset of sentences that constrained for these same SFWs, but which differed in the SFW-1 (a subset of within-pairs, shown in red).
The spatial patterns produced by the sentence pairs that predicted the same SFW (i.e. within-pairs) appeared to be more similar than the sentence pairs that predicted different SFW (i.e. between-pairs), even though the between-pairs contained the same SFW-1 (t(25) = 1.81, p = 0.08). This strongly suggests that the observed effect reflects the prediction of the SFW rather than the lexical processing of the SFW-1.
Figure 2—figure supplement 3
Results of the Spatial Representational Similarity Analysis for two subsets of trials where (A) sentences ending with expected words were seen first or (B) sentences ending with unexpected words were seen first.
The time series of spatial similarity R values for the pairs in which the same word was predicted (within-pair) are shown in red, while the time series for the pairs in which a different word was predicted (between-pair) are shown in blue. The spatial similarity was greater when the same word was predicted than when different words were predicted in both subsets. No significant difference was found between the two subsets of trials, as indicated by the lack of a main effect of Order (Expected First, Unexpected First) (F(1,25) = 0.747, p = 0.396, η2 = 0.029) or an interaction between Order (Expected First, Unexpected First) and Pairs (Within-pair, Between-pair) (F(1,25) = 1.804, p = 0.191, η2 = 0.067).
Figure 2—figure supplement 4
Results of the Spatial Representational Similarity Analysis for pairs in which the same word was predicted (within-pair, shown in red) and in which the same syntactic category (e.g. nouns or verbs) of words (but not the same words) was predicted (within-category, shown in cyan).
The spatial similarity was greater when the same word was predicted than when different words belonging to the same syntactic category were predicted: t(25) = 3.559, p = 0.002. This suggests that the high within-pair spatial similarity relates to the specific representation of the predicted words over and above the syntactic category that the predicated words belong to.
Figure 3 with 2 supplements
Results of the Temporal Representational Similarity Analysis.
The Temporal Representational Similarity Analysis was carried out between −880 and −485 ms before the onset of the final word. (A) Temporal similarity topographic maps at the sensor level. Left and middle: Both the within- and between-pair correlations revealed increased temporal similarity over bilateral temporal and posterior sensors. Right: the difference map revealed greater temporal similarity when the same word was predicted (within-pairs) than when a different word was predicted (between-pairs) over central and posterior sensors. The sensors where this difference was significant at the cluster level are marked with black asterisks (p = 0.002; a cluster-randomization approach controlling for multiple comparisons over sensors). (B) Temporal similarity difference map in source space. The correlation values were interpolated on the MNI template brain and are shown both on the coronal plane (Talairach coordinate of peak: y = −19.5 mm) and the sagittal plane (Talairach coordinate of peak: x = −39.5 mm). This revealed significantly greater temporal similarity between sentence pairs that predicted the same word (within-pairs) than pairs that predicted a different word (between-pairs) within the left inferior temporal gyrus, extending into the medial temporal lobe including the left fusiform, hippocampus and parahippocampus as well as left cerebellum (p = 0.006; a cluster-randomization approach controlling for multiple comparisons over grid points).
-
Figure 3—source data 1
Data used for plotting Figure 3 as well as its supplementary Figures.
- https://doi.org/10.7554/eLife.39061.013
Figure 3—figure supplement 1
Results of the Temporal Representational Similarity Analysis after matching the number of pairs between the within-pair and between-pair correlations.
The Temporal Representational Similarity Analysis was carried out between −880 and −485 ms before the onset of the final word. (A) Temporal similarity topographic maps at the sensor level. Left and middle: Both the within- and between-pair correlations revealed increased temporal similarity over bilateral temporal and posterior sensors. Right: the difference map revealed greater temporal similarity when the same word was predicted than when different words were predicted over central and posterior sensors (marginally significant cluster: p = 0.0679; a cluster-randomization approach controlling for multiple comparisons over sensors). (B) Temporal similarity difference map in source space. The values were interpolated on the MNI template brain and are shown both on the coronal plane (Talairach coordinate of peak: y = −9.5 mm) and the sagittal plane (Talairach coordinate of peak: x = −29.5 mm). This revealed significantly greater temporal similarity between sentence pairs that predicted the same word than pairs that predicted a different word within the left inferior temporal region and extended into the left hippocampus, left fusiform, parahippocampus as well as left cerebellum (p = 0.034; a cluster-randomization approach controlling for multiple comparisons over grid points).
Figure 3—figure supplement 2
Results of the Temporal Representational Similarity Analysis showing the 85% maximum difference of the statistically significant cluster in source space.
The Temporal Representational Similarity Analysis was carried out between −880 and −485 ms before the onset of the final word. (A) Temporal similarity difference map in source space between the averaged N within-pair correlations and 2N(N-1) between-pair correlations. The values were interpolated on the MNI template brain and are shown both on the coronal plane (Talairach coordinate of peak: y = −19.5 mm) and the sagittal plane (Talairach coordinate of peak: x = −39.5 mm). The maximum difference between the within-pair and the between-pair correlations was found within the left inferior temporal gyrus, and the cluster extended into the medial temporal lobe including the left fusiform, hippocampus and parahippocampus. (B) Temporal similarity difference map in source space between the averaged N within-pair correlations and N between-pair correlations. The values were interpolated on the MNI template brain and are shown both on the coronal plane (Talairach coordinate of peak: y = −9.5 mm) and the sagittal plane (Talairach coordinate of peak: x = −29.5 mm). The maximum difference between the within-pair and the between-pair correlations was found within the left inferior temporal gyrus, and the cluster extended into the medial temporal lobe including the left fusiform, hippocampus and parahippocampus.
Author response image 1
Additional files
-
Source code 1
Matlab code for data analysis.
These scripts were used for spatial Representational Similarity Analysis, cross-temporal Spatial Similarity Analysis, as well as temporal Representational Similarity Analysis.
- https://doi.org/10.7554/eLife.39061.014
-
Transparent reporting form
- https://doi.org/10.7554/eLife.39061.015
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Specific lexico-semantic predictions are associated with unique spatial and temporal patterns of neural activity
eLife 7:e39061.
https://doi.org/10.7554/eLife.39061
