Understanding spoken language likely requires a multitude of processes (Friederici, 2011; Martin, 2020; Halle and Stevens, 1962). Although not always an exclusively bottom-up affair, acoustic patterns must be segmented and mapped onto internally-stored phonetic and syllabic representations (Halle and Stevens, 1962; Marslen-Wilson and Welsh, 1978; Martin, 2016). Phonemes must be combined and mapped onto words, which in turn form abstract linguistic structures such as phrases (e.g., Martin, 2020; Pinker and Jackendoff, 2005).In proficient speakers of a language, this process seems to happen so naturally that one might almost forget the complex parallel and hierarchical processing which occurs during natural speech and language comprehension.

It has been shown that it is essential to track the temporal dynamics of the speech signal in order to understand its meaning (e.g., Giraud and Poeppel, 2012; Peelle and Davis, 2012). In natural speech, syllables follow each other in the theta range (3–8 Hz; Rosen, 1992; Ding et al., 2017; Pellegrino et al., 2011), while higher-level linguistic features such as words and phrases tend to occur at lower rates (0.5–3 Hz; Rosen, 1992; Kaufeld et al., 2020; Keitel et al., 2018). Tracking of syllabic features is stronger when one understands a language (Luo and Poeppel, 2007; Zoefel et al., 2018a; Doelling et al., 2014) and tracking of phrasal rates is more prominent when the signal contains phrasal information (Kaufeld et al., 2020; Keitel et al., 2018; Ding et al., 2016; e.g., word lists versus sentences). Importantly, phrasal tracking even occurs when there are no distinct acoustic modulations at the phrasal rate (Kaufeld et al., 2020; Keitel et al., 2018; Ding et al., 2016). These results seem to suggest that tracking of relevant temporal timescales is critical for speech understanding.

An observation one could make regarding these findings is that tracking occurs only at the rates that are meaningful and thereby behaviourally relevant (Kaufeld et al., 2020; Ding et al., 2016). For example, in word lists, the word rate is the slowest rate that is meaningful during natural listening. Modulations at slower phrasal rates might not be tracked as they do not contain behaviourally relevant information. In contrast, in sentences, phrasal rates contain linguistic information and therefore these slower rates are also tracked. Thus, when listening to speech one automatically tries to extract the meaning, which requires extracting information at the highest linguistic level (Halle and Stevens, 1962; Martin, 2016). However, it remains unclear if tracking at these slower rates is a unique feature of language processing, or rather is dependent on attention to relevant temporal timescales.

As understanding language requires a multitude of processes, it is difficult to figure out what participants actually are doing when listening to natural speech. Moreover, designing a task in an experimental setting that does justice to this multitude of processing is difficult. This is probably why tasks in language studies vary vastly. Tasks include passively listening (e.g., Kaufeld et al., 2020), asking comprehension questions (e.g., Keitel et al., 2018), rating intelligibility (e.g., Luo and Poeppel, 2007; Doelling et al., 2014), working memory tasks (e.g., Kayser et al., 2015), or even syllable counting (e.g., Ding et al., 2016). It is unclear whether outcomes are dependent on the specifics of the task. There has so far not been a study that investigates if task instructions focusing on extracting information at different temporal rates or timescales have an influence on the tracking that occurs on these timescales. It is therefore not clear whether tracking phrasal timescales is unique for language stimuli which contain phrasal structures, or could also occur for other acoustic materials where participants are instructed to pay attention to information happening at these temporal rates or timescales.

To answer this question, we designed an experiment in which participants were instructed to pay attention to different temporal modulation rates while listening to the same stimuli. We presented participants with naturally spoken sentences and word lists and asked them to either passively listen, or perform a task on the temporal scales corresponding to syllables, words, or phrases. We recorded brain activity using magnetoencephalography (MEG) while participants performed these tasks and investigated tracking as well as power and connectivity at three nodes that are part of the language network: the superior temporal gyrus (STG), the middle temporal gyrus (MTG), and the inferior frontal gyrus (IFG). We hypothesized that if tracking is purely based on behavioural relevance, it should mostly depend on the task instructions, rather than the nature of the stimuli. In contrast, if there is something automatic and specific about language information, tracking should depend on the level of linguistic information available to the brain.

In the current study, we investigated the effects of ‘additional’ tasks on the neural tracking of sentences and word lists at temporal modulations that matched phrasal rates. Different nodes of the language network showed different tracking patterns. In STG, we found stronger tracking of phrase-timed dynamics in sentences compared to word lists, independent of task. However, in MTG we found this sentence-improved tracking only for active tasks. In IFG, we also found an overall increase of tracking for sentences compared to word lists. Additionally, stronger phrasal tracking was found for the phrasal-level word-combination task compared to the other tasks (independent of stimulus type; note that for the syllable and passive comparison we found a trend), which was paralleled with increased IFG–STG connectivity in the delta band for the word-combination task. Behavioural performance seemed to relate to MI tracking in the MTG and STG–MTG connections. This suggests that tracking at phrasal timescales depends both on the linguistic information present in the signal, and on the specific task that is performed.

The findings reported in this study are in line with previous results, with overall stronger tracking of low-frequency information in the sentences compared to the word list condition (Kaufeld et al., 2020). Crucially, for the stimuli used in our study it has been shown that the condition effects are not due to acoustic differences in the stimuli and also do not occur for reversed speech (Kaufeld et al., 2020). It is therefore most likely that our results reflect an automatic inference-based extraction of relevant phrase-level information in sentences, indicating automatic processing in participants as they understand the meaning of the speech they hear using stored, structural linguistic knowledge (Martin, 2020; Ding et al., 2016; Har-Shai Yahav and Zion Golumbic, 2021). Overall, it did not seem that making participants pay attention to the temporal dynamics at the same hierarchical level through an additional task – instructing them to remember word combinations at the phrasal rate during word list presentation – could counter this main effect of condition.

Even though there was an overall main effect of condition, task did influence neural responses. Interestingly, the task effects differed for the three ROIs. In the STG, we found no task effects, while in the MTG we found an interaction between task and condition. In the MTG increased phrasal-level tracking for sentences only occurred when participants were specifically instructed to perform an active task on the materials. It therefore seems that in MTG all levels of linguistic information are used to do an active language operation on the stimuli. Importantly, the tracking at the phrasal rate in MTG seemed relevant for behavioural performance when attending to phrasal timescales (Figure 7A). This is in line with previous theoretical and empirical research suggesting a strong top-down modulatory response of speech processing in which predictions flow from the highest hierarchical levels (e.g., syntax) down to lower levels (e.g., phonemes) to aid language understanding (Martin, 2016; Hagoort, 2017; Federmeier, 2007). As in the word list condition no linguistic information is present at the phrasal rate, this information cannot be used to provide useful feedback for processing lower-level linguistic information. Instead, it could have been expected that the same type of increased tracking should have happened at the word rate rather than the phrasal rate for word lists (i.e., stronger word-rate tracking for word lists for the active tasks versus passive task). This effect was not found; this could either be attributed to different computational operations occurring at different hierarchical levels or to signal-to-noise/signal detection issues.

We found that across participants both the MI and the connectivity in temporal cortex influenced behavioural performance. Specifically, MTG–STG connections were, independent of task, related to accuracy. There was higher connectivity between MTG and STG for sentences compared to word lists at low accuracies. At high accuracies, we found that stronger MTG tracking at phrasal rates (measured with MI) for sentences compared to word lists during the word-combination task. These results suggest that indeed tracking of phrasal structure in MTG is relevant to understand sentences compared to word lists. This was reflected in a general increase in delta connectivity differences when the task was difficult (Figure 7B). Participants might compensate for the difficulty using phrasal structure present in the sentence condition. When phrasal structure in sentences are accurately tracked (as measured with MI) performance is better when these rates are relevant (Figure 7A). These results point to a role for phrasal tracking for accurately understanding the higher-order linguistic structure in sentences, though more research is needed to verify this. It is evident that the connectivity and tracking correlations to behaviour do not explain all variation in the behavioural performance (compare Figure 1 with Figure 3). Plainly, temporal tracking does not explain everything in language processing. Besides tracking there are many other components important for our designated tasks, such as memory load and semantic context which are not captured by our current analyses.

It is interesting that MTG, but not STG, showed an interaction effect. Both MTG and STG are strong hubs for language processing and have been involved in many studies which contrasted pseudo-words and words (Hickok and Poeppel, 2007; Turken and Dronkers, 2011; Vouloumanos et al., 2001). It is likely that STG does the more lower-level processing of the two regions, as it is earlier in the cortical hierarchy, thereby being more involved in initial segmentation and initial phonetic abstraction rather than a lexical interface (Hickok and Poeppel, 2007). This could also explain why STG does not show task-specific tracking effects; STG could be earlier in a workload bottleneck, receiving feedback independent of task, while MTG feedback is recruited only when active linguistic operations are required. Alternatively, it is possible that either small differences in the acoustics are detected by STG (even though this effect was not previously found with the same stimuli, Kaufeld et al., 2020), or that our blocked designed put participants in a sentence or word list ‘mode’ which could have influenced the state of these early hierarchical regions.

The IFG was the only region that showed an increase in phrasal-rate tracking specifically for the word-combination task. Note, however, that this was a weak effect, as the comparison between the phrase task and the syllable and passive tasks only reached a trend towards significance. Nonetheless, this effect is interesting for understanding the role of IFG in language. Traditionally, IFG has been viewed as a hub for articulatory processing (Hickok and Poeppel, 2007), but its role during speech comprehension, specifically in syntactic processing, has also been acknowledged (Friederici, 2011; Hagoort, 2017; Nelson et al., 2017; Dehaene et al., 2015; Zaccarella et al., 2017). Integrating information across time and relative timing is essential for syntactic processing (Martin, 2020; Dehaene et al., 2015; Martin and Doumas, 2019), and IFG feedback has been shown to occur in temporal dynamics at lower (delta) rates during sentence processing (Park et al., 2015; Keitel and Gross, 2016). However, it has also been shown that syntactic-independent verbal working memory chunking tasks recruit the IFG (Dehaene et al., 2015; Osaka et al., 2004; Fegen et al., 2015; Koelsch et al., 2009). This is in line with our findings that show that IFG is involved when we need to integrate across temporal domains either in a language-specific domain (sentences versus word lists) or for language-unspecific tasks (word-combination versus other tasks). We also show increased delta connectivity with STG for the only temporal-integration tasks in our study (i.e., the word-combination task), independent of the linguistic features in the signal. Our results therefore support a role of the IFG as a combinatorial hub integrating information across time (Gelfand and Bookheimer, 2003; Schapiro et al., 2013; Skipper, 2015).

In the current study, we investigated power as a neural readout during language comprehension from speech. This was both to ensure that any tracking effects we found were not due to overall signal-to-noise (SNR) differences, as well as to investigate task-and-condition dependent computations. SNR is better for conditions with higher power, which therefore leads to more reliable phase estimations, critical for computing MI as well as connectivity (Zar, 1998). We will therefore discuss the power differences as well as their consequences for the interpretation of the MI and connectivity results. Generally, it seemed that there was stronger power in the sentence condition compared to the word list condition in the delta band. However, the pattern was very different than the MI pattern. For the power, the word list-sentence difference was the biggest in the passive condition. In contrast, for the MI there was either no task difference (in STG) or even a stronger effect for the active tasks (in MTG; note that the power interaction was trend significant STG and MTG). We therefore think it unlikely that our MI effects were purely driven by SNR differences, and our power control analysis is consistent with this interpretation. Instead, power seems to reflect a different computation than the tracking, where more complex tasks generally lead to lower power across almost all tested frequency bands. As most of our frequency bands are on the low side of the spectrum (up to beta), it is expected that more complex tasks reduce the low-frequency power (Jensen and Mazaheri, 2010; Klimesch, 1999). It is interesting to observe that this did not reduce the connectivity for the delta band between IFG and STG, but rather increased it. It has been suggested that low power can potentially increase the available computational space, as it increases the entropy in the signal (Hanslmayr et al., 2012; ten Oever and Sack, 2015). Note that even though we found increased connectivity, we did not see a clear power peak in the delta band. This suggests that we might not be looking at an endogenous oscillator, but rather at connections operation at that temporal scale (potentially being non-oscillatory in nature). Finally, in the power comparisons for the theta, alpha, and beta bands we found stronger power for the sentence compared to the word list condition, which could reflect that listening to a natural sentence is generally less effortful than listening to a word list.

In the current manuscript, we describe tracking of ongoing temporal dynamics. However, the neural origin of this tracking is unknown. While we can be sure that modulations in the phrasal-rate follow changes in the phrasal rate of the acoustic input, it is unclear what the mechanism behind this modulation is. It is possible that there is stronger alignment of neural oscillations with the acoustic input at the phrasal rate (Lakatos et al., 2008; Obleser and Kayser, 2019; Rimmele et al., 2021). However, it could as well be that there is a phrasal timescale or slower operation happening while processing the incoming input (which de facto is at the same timescale as the phrasal structure inferred from the input). This operation, in response to stimulus input, could just as well induce the patterns we observe (Meyer et al., 2019; Zoefel et al., 2018b). Finally, it is possible that there are specific responses as a consequence of the syntactic structure, task, or statistical regularities occurring as specific events at phrasal timescales (Obleser and Kayser, 2019; Ten Oever and Martin, 2021; Frank and Yang, 2018).

It is difficult to decide on the most natural task in an experimental setting, that best reflects how we use language in a natural setting. This is probably why such a vast number of different tasks have been used in the literature. Our study (and many before us) indicates that during passive listening, we naturally attend to all levels of linguistic hierarchy. This is consistent with the widely accepted notion that the meaning of a natural sentence requires composing words in a grammatical structure. For most research questions in language, it therefore is sensible to use a task that mimics this automatic natural understanding of a sentence. Here, we show that automatic understanding of linguistic information, and all the processing that this entails, cannot be countered to substantially change the consequences for neural readout, even when explicitly instructing participants to pay attention to particular timescales.

Data and analysis code are available at https://data.donders.ru.nl/collections/di/dccn/DSC_3027006.01_220 (doi: https://doi.org/10.34973/vjw9-0572).

1. 1. ten Oever S
   2. Carta S
   3. Kaufeld G
   4. Martin AE

   (2022) Donders Repository

   Task relevant tracking of hierarchical linguistic structure.

   https://doi.org/10.34973/vjw9-0572

1. 1. Bastos AM
   2. Schoffelen JM

   (2015) A Tutorial Review of Functional Connectivity Analysis Methods and Their Interpretational Pitfalls

   Frontiers in Systems Neuroscience 9:175.

   https://doi.org/10.3389/fnsys.2015.00175

   • PubMed
   • Google Scholar
2. 1. Dehaene S
   2. Meyniel F
   3. Wacongne C
   4. Wang L
   5. Pallier C

   (2015) The Neural Representation of Sequences: From Transition Probabilities to Algebraic Patterns and Linguistic Trees

   Neuron 88:2–19.

   https://doi.org/10.1016/j.neuron.2015.09.019

   • PubMed
   • Google Scholar
3. 1. Ding N
   2. Melloni L
   3. Zhang H
   4. Tian X
   5. Poeppel D

   (2016) Cortical tracking of hierarchical linguistic structures in connected speech

   Nature Neuroscience 19:158–164.

   https://doi.org/10.1038/nn.4186

   • PubMed
   • Google Scholar
4. 1. Ding N
   2. Patel AD
   3. Chen L
   4. Butler H
   5. Luo C
   6. Poeppel D

   (2017) Temporal modulations in speech and music

   Neuroscience and Biobehavioral Reviews 81:181–187.

   https://doi.org/10.1016/j.neubiorev.2017.02.011

   • PubMed
   • Google Scholar
5. 1. Doelling KB
   2. Arnal LH
   3. Ghitza O
   4. Poeppel D

   (2014) Acoustic landmarks drive delta-theta oscillations to enable speech comprehension by facilitating perceptual parsing

   NeuroImage 85 Pt 2:761–768.

   https://doi.org/10.1016/j.neuroimage.2013.06.035

   • PubMed
   • Google Scholar
6. 1. Donoghue T
   2. Haller M
   3. Peterson EJ
   4. Varma P
   5. Sebastian P
   6. Gao R
   7. Noto T
   8. Lara AH
   9. Wallis JD
   10. Knight RT
   11. Shestyuk A
   12. Voytek B

   (2020) Parameterizing neural power spectra into periodic and aperiodic components

   Nature Neuroscience 23:1655–1665.

   https://doi.org/10.1038/s41593-020-00744-x

   • PubMed
   • Google Scholar
7. 1. Federmeier KD

   (2007) Thinking ahead: the role and roots of prediction in language comprehension

   Psychophysiology 44:491–505.

   https://doi.org/10.1111/j.1469-8986.2007.00531.x

   • PubMed
   • Google Scholar
8. 1. Fegen D
   2. Buchsbaum BR
   3. D’Esposito M

   (2015) The effect of rehearsal rate and memory load on verbal working memory

   NeuroImage 105:120–131.

   https://doi.org/10.1016/j.neuroimage.2014.10.034

   • PubMed
   • Google Scholar
9. 1. Fischl B

   (2012) FreeSurfer

   NeuroImage 62:774–781.

   https://doi.org/10.1016/j.neuroimage.2012.01.021

   • PubMed
   • Google Scholar
10. 1. Frank SL
    2. Yang J

    (2018) Lexical representation explains cortical entrainment during speech comprehension

    PLOS ONE 13:e0197304.

    https://doi.org/10.1371/journal.pone.0197304

    • PubMed
    • Google Scholar
11. 1. Friederici AD

    (2011) The brain basis of language processing: from structure to function

    Physiological Reviews 91:1357–1392.

    https://doi.org/10.1152/physrev.00006.2011

    • PubMed
    • Google Scholar
12. 1. Gelfand JR
    2. Bookheimer SY

    (2003) Dissociating neural mechanisms of temporal sequencing and processing phonemes

    Neuron 38:831–842.

    https://doi.org/10.1016/s0896-6273(03)00285-x

    • PubMed
    • Google Scholar
13. 1. Giraud AL
    2. Poeppel D

    (2012) Cortical oscillations and speech processing: emerging computational principles and operations

    Nature Neuroscience 15:511–517.

    https://doi.org/10.1038/nn.3063

    • PubMed
    • Google Scholar
14. 1. Glasser MF
    2. Sotiropoulos SN
    3. Wilson JA
    4. Coalson TS
    5. Fischl B
    6. Andersson JL
    7. Xu J
    8. Jbabdi S
    9. Webster M
    10. Polimeni JR
    11. Van Essen DC
    12. Jenkinson M
    13. WU-Minn HCP Consortium

    (2013) The minimal preprocessing pipelines for the Human Connectome Project

    NeuroImage 80:105–124.

    https://doi.org/10.1016/j.neuroimage.2013.04.127

    • PubMed
    • Google Scholar
15. 1. Gross J
    2. Hoogenboom N
    3. Thut G
    4. Schyns P
    5. Panzeri S
    6. Belin P
    7. Garrod S
    8. Poeppel D

    (2013) Speech Rhythms and Multiplexed Oscillatory Sensory Coding in the Human Brain

    PLOS Biology 11:e1001752.

    https://doi.org/10.1371/journal.pbio.1001752

    • Google Scholar
16. 1. Hagoort P

    (2017) The core and beyond in the language-ready brain

    Neuroscience and Biobehavioral Reviews 81:194–204.

    https://doi.org/10.1016/j.neubiorev.2017.01.048

    • PubMed
    • Google Scholar
17. 1. Halle M
    2. Stevens K

    (1962) Speech recognition: A model and A program for research

    IEEE Transactions on Information Theory 8:155–159.

    https://doi.org/10.1109/TIT.1962.1057686

    • Google Scholar
18. 1. Hanslmayr S
    2. Staudigl T
    3. Fellner MC

    (2012) Oscillatory power decreases and long-term memory: the information via desynchronization hypothesis

    Frontiers in Human Neuroscience 6:74.

    https://doi.org/10.3389/fnhum.2012.00074

    • PubMed
    • Google Scholar
19. 1. Har-Shai Yahav P
    2. Zion Golumbic E

    (2021) Linguistic processing of task-irrelevant speech at a cocktail party

    eLife 10:e65096.

    https://doi.org/10.7554/eLife.65096

    • PubMed
    • Google Scholar
20. 1. Hickok G
    2. Poeppel D

    (2007) The cortical organization of speech processing

    Nature Reviews. Neuroscience 8:393–402.

    https://doi.org/10.1038/nrn2113

    • PubMed
    • Google Scholar
21. 1. Ince RAA
    2. Giordano BL
    3. Kayser C
    4. Rousselet GA
    5. Gross J
    6. Schyns PG

    (2017) A statistical framework for neuroimaging data analysis based on mutual information estimated via A gaussian copula

    Human Brain Mapping 38:1541–1573.

    https://doi.org/10.1002/hbm.23471

    • PubMed
    • Google Scholar
22. 1. Jensen O
    2. Mazaheri A

    (2010) Shaping functional architecture by oscillatory alpha activity: gating by inhibition

    Frontiers in Human Neuroscience 4:186.

    https://doi.org/10.3389/fnhum.2010.00186

    • PubMed
    • Google Scholar
23. 1. Kaufeld G
    2. Bosker HR
    3. Ten Oever S
    4. Alday PM
    5. Meyer AS
    6. Martin AE

    (2020) Linguistic Structure and Meaning Organize Neural Oscillations into a Content-Specific Hierarchy

    The Journal of Neuroscience 40:9467–9475.

    https://doi.org/10.1523/JNEUROSCI.0302-20.2020

    • PubMed
    • Google Scholar
24. 1. Kayser SJ
    2. Ince RAA
    3. Gross J
    4. Kayser C

    (2015) Irregular Speech Rate Dissociates Auditory Cortical Entrainment, Evoked Responses, and Frontal Alpha

    The Journal of Neuroscience 35:14691–14701.

    https://doi.org/10.1523/JNEUROSCI.2243-15.2015

    • PubMed
    • Google Scholar
25. Preprint

    1. Kazanina N
    2. Tavano A

    (2021) What Neural Oscillations Can(Not) Do for Syntactic Structure Building

    PsyArXiv.

    https://doi.org/10.31234/osf.io/yv2tm

    • Google Scholar
26. 1. Keitel A
    2. Gross J

    (2016) Individual Human Brain Areas Can Be Identified from Their Characteristic Spectral Activation Fingerprints

    PLOS Biology 14:e1002498.

    https://doi.org/10.1371/journal.pbio.1002498

    • PubMed
    • Google Scholar
27. 1. Keitel A
    2. Gross J
    3. Kayser C

    (2018) Perceptually relevant speech tracking in auditory and motor cortex reflects distinct linguistic features

    PLOS Biology 16:e2004473.

    https://doi.org/10.1371/journal.pbio.2004473

    • PubMed
    • Google Scholar
28. 1. Klimesch W

    (1999) EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis

    Brain Research. Brain Research Reviews 29:169–195.

    https://doi.org/10.1016/s0165-0173(98)00056-3

    • PubMed
    • Google Scholar
29. 1. Koelsch S
    2. Schulze K
    3. Sammler D
    4. Fritz T
    5. Müller K
    6. Gruber O

    (2009) Functional architecture of verbal and tonal working memory: an FMRI study

    Human Brain Mapping 30:859–873.

    https://doi.org/10.1002/hbm.20550

    • PubMed
    • Google Scholar
30. 1. Lakatos P
    2. Karmos G
    3. Mehta AD
    4. Ulbert I
    5. Schroeder CE

    (2008) Entrainment of neuronal oscillations as a mechanism of attentional selection

    Science 320:110–113.

    https://doi.org/10.1126/science.1154735

    • PubMed
    • Google Scholar
31. 1. Luo H
    2. Poeppel D

    (2007) Phase patterns of neuronal responses reliably discriminate speech in human auditory cortex

    Neuron 54:1001–1010.

    https://doi.org/10.1016/j.neuron.2007.06.004

    • PubMed
    • Google Scholar
32. 1. Marslen-Wilson WD
    2. Welsh A

    (1978) Processing interactions and lexical access during word recognition in continuous speech

    Cognitive Psychology 10:29–63.

    https://doi.org/10.1016/0010-0285(78)90018-X

    • Google Scholar
33. 1. Martin AE

    (2016) Language Processing as Cue Integration: Grounding the Psychology of Language in Perception and Neurophysiology

    Frontiers in Psychology 7:120.

    https://doi.org/10.3389/fpsyg.2016.00120

    • PubMed
    • Google Scholar
34. 1. Martin AE
    2. Doumas LA

    (2019) Predicate learning in neural systems: using oscillations to discover latent structure

    Current Opinion in Behavioral Sciences 29:77–83.

    https://doi.org/10.1016/j.cobeha.2019.04.008

    • Google Scholar
35. 1. Martin AE

    (2020) A Compositional Neural Architecture for Language

    Journal of Cognitive Neuroscience 32:1407–1427.

    https://doi.org/10.1162/jocn_a_01552

    • PubMed
    • Google Scholar
36. 1. Meyer L

    (2018) The neural oscillations of speech processing and language comprehension: state of the art and emerging mechanisms

    The European Journal of Neuroscience 48:2609–2621.

    https://doi.org/10.1111/ejn.13748

    • PubMed
    • Google Scholar
37. 1. Meyer L
    2. Sun Y
    3. Martin AE

    (2019) Synchronous, but not entrained: exogenous and endogenous cortical rhythms of speech and language processing

    Language, Cognition and Neuroscience 35:1089–1099.

    https://doi.org/10.1080/23273798.2019.1693050

    • Google Scholar
38. 1. Nelson MJ
    2. El Karoui I
    3. Giber K
    4. Yang X
    5. Cohen L
    6. Koopman H
    7. Cash SS
    8. Naccache L
    9. Hale JT
    10. Pallier C
    11. Dehaene S

    (2017) Neurophysiological dynamics of phrase-structure building during sentence processing

    PNAS 114:E3669–E3678.

    https://doi.org/10.1073/pnas.1701590114

    • PubMed
    • Google Scholar
39. 1. Obleser J
    2. Herrmann B
    3. Henry MJ

    (2012) Neural Oscillations in Speech: Don’t be Enslaved by the Envelope

    Frontiers in Human Neuroscience 6:250.

    https://doi.org/10.3389/fnhum.2012.00250

    • PubMed
    • Google Scholar
40. 1. Obleser J
    2. Kayser C

    (2019) Neural Entrainment and Attentional Selection in the Listening Brain

    Trends in Cognitive Sciences 23:913–926.

    https://doi.org/10.1016/j.tics.2019.08.004

    • PubMed
    • Google Scholar
41. 1. Oostenveld R
    2. Fries P
    3. Maris E
    4. Schoffelen JM

    (2011) FieldTrip: Open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data

    Computational Intelligence and Neuroscience 2011:156869.

    https://doi.org/10.1155/2011/156869

    • PubMed
    • Google Scholar
42. 1. Osaka N
    2. Osaka M
    3. Kondo H
    4. Morishita M
    5. Fukuyama H
    6. Shibasaki H

    (2004) The neural basis of executive function in working memory: an fMRI study based on individual differences

    NeuroImage 21:623–631.

    https://doi.org/10.1016/j.neuroimage.2003.09.069

    • PubMed
    • Google Scholar
43. 1. Park H
    2. Ince RAA
    3. Schyns PG
    4. Thut G
    5. Gross J

    (2015) Frontal top-down signals increase coupling of auditory low-frequency oscillations to continuous speech in human listeners

    Current Biology 25:1649–1653.

    https://doi.org/10.1016/j.cub.2015.04.049

    • PubMed
    • Google Scholar
44. 1. Peelle JE
    2. Davis MH

    (2012) Neural Oscillations Carry Speech Rhythm through to Comprehension

    Frontiers in Psychology 3:320.

    https://doi.org/10.3389/fpsyg.2012.00320

    • PubMed
    • Google Scholar
45. 1. Pellegrino F
    2. Coupé C
    3. Marsico E

    (2011) Across-Language Perspective on Speech Information Rate

    Language 87:539–558.

    https://doi.org/10.1353/lan.2011.0057

    • Google Scholar
46. 1. Pinker S
    2. Jackendoff R

    (2005) The faculty of language: what’s special about it?

    Cognition 95:201–236.

    https://doi.org/10.1016/j.cognition.2004.08.004

    • PubMed
    • Google Scholar
47. 1. Rimmele JM
    2. Poeppel D
    3. Ghitza O

    (2021) Acoustically Driven Cortical δ Oscillations Underpin Prosodic Chunking

    ENeuro 8:ENEURO.0562-20.2021.

    https://doi.org/10.1523/ENEURO.0562-20.2021

    • PubMed
    • Google Scholar
48. 1. Rosen S

    (1992) Temporal information in speech: acoustic, auditory and linguistic aspects

    Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences 336:367–373.

    https://doi.org/10.1098/rstb.1992.0070

    • PubMed
    • Google Scholar
49. 1. Schapiro AC
    2. Rogers TT
    3. Cordova NI
    4. Turk-Browne NB
    5. Botvinick MM

    (2013) Neural representations of events arise from temporal community structure

    Nature Neuroscience 16:486–492.

    https://doi.org/10.1038/nn.3331

    • PubMed
    • Google Scholar
50. 1. Skipper JI

    (2015) The NOLB model: A model of the natural organization of language and the brain

    APA Psycnet 1:e6.

    https://doi.org/10.1017/CBO9781107323667.006

    • Google Scholar
51. 1. Smith ZM
    2. Delgutte B
    3. Oxenham AJ

    (2002) Chimaeric sounds reveal dichotomies in auditory perception

    Nature 416:87–90.

    https://doi.org/10.1038/416087a

    • PubMed
    • Google Scholar
52. 1. Stolk A
    2. Todorovic A
    3. Schoffelen JM
    4. Oostenveld R

    (2013) Online and offline tools for head movement compensation in MEG

    NeuroImage 68:39–48.

    https://doi.org/10.1016/j.neuroimage.2012.11.047

    • PubMed
    • Google Scholar
53. 1. ten Oever S
    2. Sack AT

    (2015) Oscillatory phase shapes syllable perception

    PNAS 112:15833–15837.

    https://doi.org/10.1073/pnas.1517519112

    • PubMed
    • Google Scholar
54. 1. Ten Oever S
    2. Martin AE

    (2021) An oscillating computational model can track pseudo-rhythmic speech by using linguistic predictions

    eLife 10:e68066.

    https://doi.org/10.7554/eLife.68066

    • PubMed
    • Google Scholar
55. 1. Turken AU
    2. Dronkers NF

    (2011) The neural architecture of the language comprehension network: converging evidence from lesion and connectivity analyses

    Frontiers in Systems Neuroscience 5:1.

    https://doi.org/10.3389/fnsys.2011.00001

    • PubMed
    • Google Scholar
56. 1. Vouloumanos A
    2. Kiehl KA
    3. Werker JF
    4. Liddle PF

    (2001) Detection of sounds in the auditory stream: event-related fMRI evidence for differential activation to speech and nonspeech

    Journal of Cognitive Neuroscience 13:994–1005.

    https://doi.org/10.1162/089892901753165890

    • PubMed
    • Google Scholar
57. 1. Zaccarella E
    2. Meyer L
    3. Makuuchi M
    4. Friederici AD

    (2017) Building by Syntax: The Neural Basis of Minimal Linguistic Structures

    Cerebral Cortex 27:411–421.

    https://doi.org/10.1093/cercor/bhv234

    • PubMed
    • Google Scholar
58. Book

    1. Zar JH

    (1998)

    Biostatistical Analysis (4 ed)

    Englewood Cliffs, New Jersey: Prentice Hall.

    • Google Scholar
59. 1. Zoefel B
    2. Archer-Boyd A
    3. Davis MH

    (2018a) Phase Entrainment of Brain Oscillations Causally Modulates Neural Responses to Intelligible Speech

    Current Biology 28:401–408.

    https://doi.org/10.1016/j.cub.2017.11.071

    • PubMed
    • Google Scholar
60. 1. Zoefel B
    2. Ten Oever S
    3. Sack AT

    (2018b) The Involvement of Endogenous Neural Oscillations in the Processing of Rhythmic Input: More Than a Regular Repetition of Evoked Neural Responses

    Frontiers in Neuroscience 12:95.

    https://doi.org/10.3389/fnins.2018.00095

    • PubMed
    • Google Scholar

Decision letter after peer review:

Thank you for submitting your article "Task-dependent and automatic tracking of hierarchical linguistic structure" for consideration by eLife. Your article has been reviewed by 3 peer reviewers, one of whom is a member of our Board of Reviewing Editors, and the evaluation has been overseen by Barbara Shinn-Cunningham as the Senior Editor. The following individual involved in the review of your submission has agreed to reveal their identity: Johanna Rimmele (Reviewer #2).

The reviewers have discussed their reviews with one another, and the Reviewing Editor has drafted this to help you prepare a revised submission.

Essential revisions:

1. The relationship of participants' behavior to the observed MEG responses. One clear concern is the degree to which the accuracy differences between word lists and sentences (Figure 1) explain MI differences.

2. A number of important details regarding analysis choices were missing and should be added, including: justification of the frequency bands used for the MI analysis, task order (were sentences always first, and word lists second), whether ROIs were defined across hemispheres, details on signal filtering, and details regarding the PCA analysis.

3. The use of "hierarchical linguistic structure" should be clarified or tempered in the context of the current results. Although the attention was directed to syllables, words, and phrases through instructions, the significant MEG results were in the phrasal band (Figure 3) and not the others (supplemental figures).

Reviewer #1 (Recommendations for the authors):

The participants ranged in age from 18-59, which is a rather broad range. Given age-related changes in both hearing (not assessed) and language processing, it was surprising to not at least see age included in the models (though perhaps there were not enough participants?). Some comment would be useful here.

Figures: The y axis label seems frequently oddly positioned at the top – maybe consider just centering it on the axis?

I liked the inset panels for the main effects, but they are rather small. If you can make them any larger that would improve readability.

In general, I really like showing individual subjects in addition to means. However, it also can make the plots a bit busy. I imagine you've tried other ways of plotting but might be worth exploring ways to highlight the means a bit more (or simplify the main paper figures, and keep these more detailed figures for supplemental)? Totally up to you, but I had a hard time seeing the trends as the plots currently stand.

On my copy of the PDF, Figures 6 and 4 (p. 14) were overlapping and so really impossible to see.

Figure 1: In the text, the condition is referred to as "word list", but in the figure "wordlist". Having these match isn't strictly necessary but would be a nice touch.

Regarding accuracy differences: including accuracy in the models are good, but alternately, restricting analyses to only correct responses might also help with this?

Reviewer #2 (Recommendations for the authors):

– l. 122: does that mean the sentence was always presented first and the word-list second? The word-list did not contain the same words as the sentence, right? This might have affected differences in tracking and power.

– l. 197: why were the linear mixed-models for the MEG data performed separately per ROI? Also, were ROIs computed across hemispheres?

– l. 198: was the procedure described by Ince et al., 2017 using gaussian copula used for the MI analysis, if yes could you cite the reference?

– l. 191: could you give the details of the filter, what kind of filter etc?

– l. 191/l. 206: why were the frequency bands for the power analysis (δ: 0.5-3.0 Hz; theta: 3.0-8.0 Hz; α: 8.0-15.0 Hz; β: 15.0-25.0 Hz) different than for the mutual information analysis (l. 191: phrase (0.8-1.1 Hz), word (1.9-2.8 Hz), and syllable (3.5- 5.0 Hz)), could you explain the choice of frequency band? Particularly, if the authors want to check signal-to-noise differences in the mutual information analysis by using the power analysis, it seems relevant to match frequency bands.

– l. 207: why was the data averaged across trials and not the single-trial data fed into the mixed-models, such an analysis might strengthen the findings?

– l. 213: averaged across hemispheres or across trials?

– l. 216: all previous analyses were conducted across the hemispheres?

– l. 217: "for the four different frequency bands" this is referring to the frequency bands chosen for the power analysis, not those in the MI analysis?

– Figure 1: were only correct trials analyzed in the MEG analysis? If not, this might be a problem as there were more correct trials in the sentence compared to the word-list condition at the phrasal scale task. Could you add a control analysis on the correct trials only, to make sure that this was not a confound?

– l. 252 ff./Figure 2: As no power peak is observed in the δ (and α) band, this might result in spurious connectivity findings.

– l. 304: with higher connectivity in the phrasal compared to the passive task? Could you add this info?

– Something went wrong with figure 6.

– l. 403: alignment of neural oscillations with acoustics at phrasal scale, this reference seems relevant here: Rimmele, Poeppel, Ghitza, 2021.

– Wording: l. 387 "in STG".

Reviewer #3 (Recommendations for the authors):

1. In procedure settings, I am not sure whether the sentence condition is always presented before the word list condition (the misunderstanding comes from lines 126-128). Please add the necessary details and avoid this misunderstanding.

2. In the MEG analysis, PCA was performed to reduce computational load and increase the data relevance. However, the PCA procedure is not clear. For example, the authors extracted top *20 PCA components per region. Why 20 components? And why not 10 or 30 components? The details should be clarified.

Essential revisions:

1. The relationship of participants' behavior to the observed MEG responses. One clear concern is the degree to which the accuracy differences between word lists and sentences (Figure 1) explain MI differences.

2. A number of important details regarding analysis choices were missing and should be added, including: justification of the frequency bands used for the MI analysis, task order (were sentences always first, and word lists second), whether ROIs were defined across hemispheres, details on signal filtering, and details regarding the PCA analysis.

3. The use of "hierarchical linguistic structure" should be clarified or tempered in the context of the current results. Although the attention was directed to syllables, words, and phrases through instructions, the significant MEG results were in the phrasal band (Figure 3) and not the others (supplemental figures).

We thank the Editor and the Reviewers for their helpful comments and constructive feedback. We feel that the reviews have substantially improved our manuscript. We have now addressed these three core concerns, and detail at length in individual responses to each reviewer how and what can now additionally be shown. To summarize briefly here:

1. We now include a new analysis in which we add accuracy as a factor. None of the original statistical patterns changed. We still find that MI/ neural tracking is higher for phrases in sentences than in word lists and that tracking during spoken language processing is a largely automatic response. Task specific effect were still found in in MTG and IFG. Thus, accuracy differences did not explain MI difference and we therefore remain with our core messages of the paper. We would like to note that we do no use tasks in the same way that they are typically used in cognitive neuroscience (say in a working memory paradigm, where only neural activity on correct trials can be associated the cognitive processing in question; instead we use tasks to direct our participant’s attention to syllables, words, and phrases and the timescale that they occur in). This difference in design – in other words, that in our case, that behavioral performance on the syllable task was lower than in the word or phrase tasks, does not mean that the sentence or word list was not heard/comprehended – means that inclusion or exclusion of incorrect trials does not further isolate the cognitive process of spoken language comprehension in question. That said, we agree it is very important to show that MI differences do not stem from behavioral performance differences. In order to show this, we add the above mentioned analysis to the manuscript. As said, this analysis did not change any of our main findings or conclusions, and rather strengthened the argument that tracking of phrases in sentences vs. word lists is stronger. We report now report these results in both our response and the manuscript.

2. We believe that we have fully addressed these queries and thank the reviewers for encouraging us be more comprehensive and thorough. We note that due to our 2-condition by 4-task design, we did not achieve enough power to include ROI as an independent factor. This is obviously not ideal, but embodies a trade-off between testing these 8 conditions together, which we felt was crucial for the inferences we wanted to make about tracking, and discovering underlying sources. We thus stuck to the well-known ROIs/ sources widely used and defined in the speech and language processing literature.

3. We regret any confusion this word might have caused; we meant to refer to the fact that our 4 tasks focus participants’ attention to different timescales and linguistic representations occur on across the linguistic hierarchy. We have changed the title to more clearly reflect this and have removed reference to linguistic hierarchy throughout the paper.

Reviewer #1 (Recommendations for the authors):

The participants ranged in age from 18-59, which is a rather broad range. Given age-related changes in both hearing (not assessed) and language processing, it was surprising to not at least see age included in the models (though perhaps there were not enough participants?). Some comment would be useful here.

We indeed had quite a wide range of ages included in our study. Our wide age range was partly due to constraints in the MEG testing during covid times. It was difficult to find participants during the pandemic. On top of this, it is almost standard practice in the Netherlands to put dental wires behind the teeth after having braces and leave them in place for life. These participants cannot participate in MEG studies and therefore we widened the scope of our age range to reach enough participants. However, we also believe that it is good practice to not limit the participants age range to such a restricted range as is common practice in most cognitive neuroscience research. Otherwise, we end up only comparing brains of student populations with each other while we want to make conclusions for a much wider population. It is therefore rather an asset than a drawback that we have this wide age range. Regarding the hearing difficulties, we would like to note that we did ask participants whether they had any hearing problems (which none of them reported), but indeed did not assess did ourselves.

Nonetheless, we agree that it is valuable to investigate whether age had an influence in any of our results and therefore added age in the models of the main manuscript (i.e. for the lower frequencies). Again, we could only run fixed intercept models due to a reduction in the degrees of freedom.

For the MI we found no significant effect of age or interactions for any of our models. There was a trend significant effect in MTG for task*age interaction (F(3,123) = 2.5874, p = 0.0561). However, also the task*condition interaction remained significant and thus not change any conclusions.

Author response image 1

Download asset Open asset

For power we found a significant interaction between condition and age in STG (F(1, 28.88), p = 0.0192) and MTG (F(1,31) = 10.31, p = 0.003). In the MTG we additionally found an interaction between task and ages (F(3,31) = 5.02, p = 0.006). In IFG we only found a main effect of age (F(1,43) = 5.067, p = 0.030). For connectivity we found overall a main effect of age for all connections (STG-MTG, F(1,17.12) = 10.09, p = 0.005), (STG-IFG, F(1,16.96) = 17.42, p < 0.001, MTG-IFG, F(1,17.058) = 12.478, p = 0.002). The condition*age interaction in STG and MTG both suggested only for wordlist a change in power with age and not for the sentence condition (follow-up correlation age-MI per condition. STG: p = 0.076 (uncorrected) and MTG: p = 0.023 (uncorrected)). The task*age interaction in MTG showed only for the passive task a significant effect of age (follow-up correlation age-MI per task. p = 0.028 (uncorrected)).

Generally, we found it difficult to make strong conclusions about the main effect of age. Firstly, we did not have any baseline to assess whether main effects are specific to our language tasks or a more general age-related difference. Secondly, there could be anatomical age-related differences that spuriously drive the main effects of age. Therefore, we only report on the interaction effects. These analyses are now in the manuscript and there is an additional figure in the supplementary materials.

The results now read:

“Age control. Adding age to the analysis did not change any of the original findings (all original effects were still significant). We did however find for the power analysis age-specific interactions with condition and task. Specifically, for both the STG and the MTG we found an interaction between age and condition (F(1,28.87) = 6.156, p = 0.0192 and F(1,31) = 10.31, p = 0.003). In both ROIs there was a stronger difference between sentences and word lists (higher δ power for sentences) for the younger compared to the older participants (Supplementary Figure 4). In the MTG there was also an interaction between task and age (F(1,31) = 5.020, p = 0.006). Here, in a follow-up we found that only in the word task there was a correlation between age and power (p = 0.023 uncorrected), but not for the other tasks (p>0.1).”.

Figures: The y axis label seems frequently oddly positioned at the top – maybe consider just centering it on the axis?

We changed the y-axes of the figures accordingly and centered them all.

I liked the inset panels for the main effects, but they are rather small. If you can make them any larger that would improve readability.

We increased the size of the insets. Note however that as there is no space to add any y-labels these insets do not reflect real data, but just to quickly show the direction and presence of a main effect. We added this information now in the figure legend to ensure that this is clear.

In general, I really like showing individual subjects in addition to means. However, it also can make the plots a bit busy. I imagine you've tried other ways of plotting but might be worth exploring ways to highlight the means a bit more (or simplify the main paper figures, and keep these more detailed figures for supplemental)? Totally up to you, but I had a hard time seeing the trends as the plots currently stand.

There is a difficult balance to showing everything and highlighting the important bits. We were happy the reviewer pointed out that all trends were difficult to see and now change the figures to only include the mean and SEM. Additionally, when we found a main effect of task, we also added the condition means to highlight the differences on the significant effect more. The original plots are now added in the supplementary materials.

On my copy of the PDF, Figures 6 and 4 (p. 14) were overlapping and so really impossible to see.

We are so sorry about this! We realized this was the case due to the pdf conversion from our own word document. We now double-checked that this wasn’t the case.

Figure 1: In the text, the condition is referred to as "word list", but in the figure "wordlist". Having these match isn't strictly necessary but would be a nice touch.

We change the condition to word list throughout.

Regarding accuracy differences: including accuracy in the models are good, but alternately, restricting analyses to only correct responses might also help with this?

We have looked into this analysis, but there are some issues with doing the analysis in this way. First, there are clear differences in difficulty level of the trials within a condition. For example, if the target question was related to the last part of the audio fragment, the task was much easier than when it was at the beginning of the audio fragment. In the syllable task, if syllables also were (by chance) a part-word, the trial was also much easier. If we were to split up in correct and incorrect trials we would not really infer solely processes due to accurately processing the speech fragments, but also confounded the analysis by the individual difficulty level of the trials. Second, we end up with a very low trial amount after only looking at incorrect trials. In the worst case, some participants end up with only 22 trials which is too little to do much. We think it is therefore fair to compare the accuracy across participants (as done in the other analysis), but it is difficult to do this within participants.

To acknowledge this, we added this limitation to the methods. The method now reads:

“Note that different trials within a task were not matched for task difficulty. For example, in the syllable task syllables that make a word are much easier to recognize than syllables that do not make a word. Additionally, trials pertaining to the beginning of the sentence are more difficult than ones related to the end of the sentence due to recency effects.”

Reviewer #2 (Recommendations for the authors):

– l. 122: does that mean the sentence was always presented first and the word-list second? The word-list did not contain the same words as the sentence, right? This might have affected differences in tracking and power.

We regret the confusion. We randomized whether the sentence or wordlist would come first across participants, but kept it constant within the participants across the four tasks. So, an individual would always alternate between a sentence and a word list block, but which one is first is counterbalanced across participants. To control for acoustic differences, we have the same words in the wordlists and the sentences (see e.g. Kaufeld et al., 2020). But this cannot have affected the results as the order is counterbalanced across participants. We now made clearer in the text how we assign the blocks.

– l. 197: why were the linear mixed-models for the MEG data performed separately per ROI? Also, were ROIs computed across hemispheres?

We had a clear a-priori idea about choosing these ROIs, therefore we did the analysis separately. Post-hoc, there now is also the simple issue of having too little power to estimate a three-way interaction between ROI, task, and condition. This power situation arises because we needed to test 4 task * 2 condition together in a single experiment in order to be able to make the inferences that we wanted to about the effects on tracking. This unfortunately comes at the cost of the power we can allot to each condition which affects our ability to generate source models with ROIs as a factor. Ideally, this would have been done, but practically this is very difficult to achieve. We cannot do more than acknowledge this in the main text (now in line 195-199). It is simply very difficult to have any design with multiple factors on top of an anatomical constraint (most studies seem to limit themselves to two factors these days to avoid these issues). The ROIs were computed across hemisphere (we had no a-priori hypothesis about hemispheres). Note that the PCAs therefore we also calculated across hemispheres (except for the coherence analysis as cross-hemisphere coherence seemed unlikely to us). Please note that we do not use ROIs that deviate from the established speech and language processing literature.

We now write:

“All following analyses were done per ROI. With enough statistical power one would add ROI as a separate factor in the analyses, but unfortunately, we did not have enough power to find a potential three-way interaction (ROI*condition*task). We therefore cannot make strong conclusions about one ROI having a stronger effect than another.”

– l. 198: was the procedure described by Ince et al., 2017 using gaussian copula used for the MI analysis, if yes could you cite the reference?

We regret not citing it in the main text and do this now.

– l. 191: could you give the details of the filter, what kind of filter etc?

We used a third order (bi-directional) Butterworth filter separately on eight equidistant bands of the speech signal. This information is now added.

– l. 191/l. 206: why were the frequency bands for the power analysis (δ: 0.5-3.0 Hz; theta: 3.0-8.0 Hz; α: 8.0-15.0 Hz; β: 15.0-25.0 Hz) different than for the mutual information analysis (l. 191: phrase (0.8-1.1 Hz), word (1.9-2.8 Hz), and syllable (3.5- 5.0 Hz)), could you explain the choice of frequency band? Particularly, if the authors want to check signal-to-noise differences in the mutual information analysis by using the power analysis, it seems relevant to match frequency bands.

The frequency bands of the MI analyses were based on the stimuli. They reflect the syllabic, word, and phrasal rates (calculated in Kaufeld et al., 2020). The power analyses were based on generic frequency bands. We choose for these two different splits as for the tracking one would expect the tracking (or phase alignment) from the exact frequency ranges in the signal relating to linguistic content (Ding et al., 2016; Martin, 2020; Keitel et al., 2018). In contrast, if there is no alignment, it is still possible that oscillatory signals are important in processing language stimuli (Jensen et al., 2010; Benitez-Burraco and Murphy, 2019). As these hypotheses rather pertain to commonly known frequency bands present in the brain we think it is more appropriate here to use generic bands (also to promote comparisons). We therefore stick to the stimulus-driven frequencies for the tracking hypothesis and use for control frequencies the generic ones (as we do not match them with anything in the stimulus). Note however, that when controlling for power in both the MI and the connectivity analysis, we did use the power of the respective band used for that specific analysis.

We write this reasoning now clearer in the methods:

“Power analysis was performed to compare the MI results with absolute power changes. On the one hand, we did this analysis as MI differences could be a consequence of signal-to-noise differences in the original data (which would be reflected in power effects). On the other hand, generic δ power has been associated with language processing [27, 28]. Therefore, we choose to analyse classical frequency bands instead of the stimulus informed ones (as used in the MI analysis) in order to compare these results with other studies. Moreover, as this analysis does not measure tracking to the stimulus (like the MI analysis does) it did not seem appropriate to match the frequency content to the stimulus content.”

– l. 207: why was the data averaged across trials and not the single-trial data fed into the mixed-models, such an analysis might strengthen the findings?

We could do this for the power analysis, but not for the MI and connectivity analysis which were calculated incorporating all trials (for the MI by concatenating all trials/speech signals, and for the connectivity as connectivity is calculated across trials). This would mean that we would run a different model on the power as on the other two variables of interest. We choose not to do this. Previously, MI analyses have been done by concatenating the data to improve sensitivity of the measure (Keitel et al., 2020; Kaufeld et al., 2018). Theoretically one could calculate the MI per trial, however this would unnecessarily reduce the sensitivity of the analysis.

– l. 213: averaged across hemispheres or across trials?

Across hemispheres. The WPLI is calculated across trials, so cannot be averaged across trials. We now clarified this.

– l. 216: all previous analyses were conducted across the hemispheres?

Yes. The MI and the power used the principles components incorporating both hemispheres, but for the connectivity analysis we calculated the PCAs for the hemispheres separately and averages only after calculating the connectivity across hemispheres.

– l. 217: "for the four different frequency bands" this is referring to the frequency bands chosen for the power analysis, not those in the MI analysis?

Yes. This is now clarified. As stated above there it makes more sense to look at generic frequency bands when we do not directly link the analysis with the speech signal.

– Figure 1: were only correct trials analyzed in the MEG analysis? If not, this might be a problem as there were more correct trials in the sentence compared to the word-list condition at the phrasal scale task. Could you add a control analysis on the correct trials only, to make sure that this was not a confound?

We included all trials in the analysis; we do this for several reasons, first because we not use the tasks in the same way as a traditional working memory task, where only correct trials contain the cognitive process that is being studied. Second, because it is not possible to only include the correct trials as we end up with too few trials per condition to reliably estimate our effect. Moreover, trials are not controlled for difficulty within a condition. However, in order to show that our MI effects are not driven by task performance, we can include accuracy across participants as a factor. We refer to point 1 for the results of this analysis. We have added this information now in the methods:

“Note that different trials within a task were not matched for task difficulty. For example, in the syllable task syllables that make a word are much easier to recognize than syllables that do not make a word. Additionally, trials pertaining to the beginning of the sentence are more difficult than ones related to the end of the sentence due to recency effects.”

– l. 252 ff./Figure 2: As no power peak is observed in the δ (and α) band, this might result in spurious connectivity findings.

Indeed, the absence of a peak does result in difficulty to interpret connectivity findings as it is unknown whether we are truly investigating an endogenous oscillation or some other form of connections which could for example be related to stimulus evoked responses. If the latter, we could not speak of ‘true’ connectivity increases, but rather for differential processing of the stimulus. We acknowledge this, but still find also this type of change worth reporting. We therefore made clear cautious note in the Results section that this is the case.

The results now read:

“Note that not for all frequency bands we found a clear peak in the power signal (only clearly so for the α band), this indicates that the connectivity likely does not reflect endogenous oscillatory activity [29], but might still pertain to connected regions operating at those time scales.”

The discussion now reads:

“Note that even though we found increased connectivity, we did not see a clear power peak in the δ band. This suggest that we might not be looking at an endogenous oscillator, but rather at connections operation at that temporal scale (potentially being non-oscillatory in nature).”

– l. 304: with higher connectivity in the phrasal compared to the passive task? Could you add this info?

We have added this info.

– Something went wrong with figure 6.

We have realized this and regret it. The new manuscript should have the figures placed correctly.

– l. 403: alignment of neural oscillations with acoustics at phrasal scale, this reference seems relevant here: Rimmele, Poeppel, Ghitza, 2021.

Agreed at we have added this.

– Wording: l. 387 "in STG".

We have changed this wording accordingly.

Reviewer #3 (Recommendations for the authors):

1. In procedure settings, I am not sure whether the sentence condition is always presented before the word list condition (the misunderstanding comes from lines 126-128). Please add the necessary details and avoid this misunderstanding.

We regret the confusion. We randomized whether the sentence or wordlist would come first across participants, but kept it constant within the participants across the four tasks. So, an individual would always alternate between a sentence and a word list block, but which one is first is counterbalanced across participants. To control for acoustic differences, we have the same words in the wordlists and the sentences (see e.g. Kaufeld et al., 2020). But this cannot have affected the results as the order is counterbalanced across participants. We now made clearer in the text how we assign the blocks.

2. In the MEG analysis, PCA was performed to reduce computational load and increase the data relevance. However, the PCA procedure is not clear. For example, the authors extracted top *20 PCA components per region. Why 20 components? And why not 10 or 30 components? The details should be clarified.

As the reviewer also mentions, our main aim was to reduce the dimensions of our analysis (to reduce computational load), while keeping in enough variance relevant for the analysis. We are not aware of any established way to determine the amount of PCA components which is best to use and can therefore only look at the explained variance of the components. As for us the goal was to reduce computational load, we wanted to keep as much original variance in the analysis as possible. In Author response image 2 one can see the cumulative explained variance of the components. Around 20 components over 99.9 % of the data is explained, which seems sensible amount to keep in.

Author response image 2

Download asset Open asset

To ensure that all our results are also robust against changing the exact number of components we repeated the analysis using 10 and 30 components. No big qualitative differences were visible. It did seem that 10 components were not sufficient to show the original effects. The IFG task effect and MTG task*condition effect were only trend significant for 10 components (p = 0.06 and p = 0.1 respectively). The condition effect remained significant for all ROIs. Using 30 components all effects (including the main and interaction; IFG task effect: p = 0.034; MTG task*condition interaction p = 0.0064) effect remained significant. We therefore believe that the results are robust against choosing the exact amount of PCAs, but does require more than 10 components.

We now discuss the effect of PCA component number in the text. The text now reads:“Overall, the amount of PCA components did not influence any of the qualitative differences in the condition. It did seem however that 10 PCA components were not sufficient to show all original effects with the same power. Specifically, the IFG task and MTG task*condition effect were only trend significant for 10 components (p = 0.06 and p = 0.1 respectively). The other effects did remain significant with 10 components. Using 30 components made some of our effects stronger than with 20 components. Here, the IFG task and MTG task*condition effects had p-values of 0.034 and 0.006 respectively. We conclude that the amount of PCAs components did not qualitative change any of our reported effects.”

Author details

1. Sanne ten Oever

   1. Language and Computation in Neural Systems group, Max Planck Institute for Psycholinguistics, Nijmegen, Netherlands
   2. Language and Computation in Neural Systems group, Donders Centre for Cognitive Neuroimaging, Nijmegen, Netherlands
   3. Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands

   Contribution

   Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Supervision, Validation, Visualization, Writing - original draft, Writing – review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0001-7547-5842

2. Sara Carta

   1. Language and Computation in Neural Systems group, Max Planck Institute for Psycholinguistics, Nijmegen, Netherlands
   2. ADAPT Centre, School of Computer Science and Statistics, University of Dublin, Trinity College, Dublin, Ireland
   3. CIMeC - Center for Mind/Brain Sciences, University of Trento, Trento, Italy

   Contribution

   Data curation, Investigation, Methodology, Software, Writing – review and editing

   Competing interests

   No competing interests declared

3. Greta Kaufeld

   Language and Computation in Neural Systems group, Max Planck Institute for Psycholinguistics, Nijmegen, Netherlands

   Contribution

   Conceptualization, Data curation, Methodology, Software, Writing – review and editing

   Competing interests

   No competing interests declared

4. Andrea E Martin

   1. Language and Computation in Neural Systems group, Max Planck Institute for Psycholinguistics, Nijmegen, Netherlands
   2. Language and Computation in Neural Systems group, Donders Centre for Cognitive Neuroimaging, Nijmegen, Netherlands

   Contribution

   Conceptualization, Formal analysis, Funding acquisition, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – review and editing

   For correspondence

   Andrea.Martin@mpi.nl

   Competing interests

   Reviewing editor, eLife

   "This ORCID iD identifies the author of this article:" 0000-0002-3395-7234

• 660

  Page views

• 163

  Downloads

• 1

  Citations

Article citation count generated by polling the highest count across the following sources: PubMed Central, Crossref, Scopus.