1. Neuroscience

A tradeoff between acoustic and linguistic feature encoding in spoken language comprehension

  1. Filiz Tezcan  Is a corresponding author
  2. Hugo Weissbart
  3. Andrea E Martin
  1. Language and Computation in Neural Systems Group, Max Planck Institute for Psycholinguistics, Netherlands
  2. Donders Centre for Cognitive Neuroimaging, Radboud University, Netherlands
https://doi.org/10.7554/eLife.82386
Share this article
Cite this article
  1. Filiz Tezcan
  2. Hugo Weissbart
  3. Andrea E Martin
(2023)
A tradeoff between acoustic and linguistic feature encoding in spoken language comprehension
eLife 12:e82386.
https://doi.org/10.7554/eLife.82386
  2. Download BibTeX
  3. Download .RIS
7 figures, 19 tables and 1 additional file

Figures

Figure 1
Download asset Open asset
Schematic of TRF models and features used in models.

(A) All speech features are divided into high and low word entropy conditions (B) TRFs for each brain source are generated by the linear regression model that estimates the source localized MEG …

Figure 2
Download asset Open asset
Model accuracy comparison between Dutch and French stories (n=24).

(A) Accuracy improvement (averaged over the sources in whole brain) by each feature for Dutch Stories (B) Accuracy improvement (averaged over the sources in whole brain) by each feature for French …

Figure 3 with 1 supplement
Download asset Open asset
First 4 Dutch Story Parts (n=24).

Light orange and light green represent Low Word Entropy condition, dark orange and dark green represent High Word Entropy condition for Dutch and French stories, respectively. (A) Reconstruction …

Figure 3—figure supplement 1
Download asset Open asset
First 4 Dutch Stories (n=24).

(A) Phoneme Onset TRF (B) Phoneme Surprisal TRF (C) Phoneme Entropy TRF Lines on the graphs in Figure A-C show the mean and shaded areas show the standard error of the mean. (D) Sources where the …

Figure 4 with 1 supplement
Download asset Open asset
Second 4 Dutch Story Parts (n=24).

Light orange and light green represent Low Word Entropy condition, dark orange and dark green represent High Word Entropy condition for Dutch and French stories, respectively. (A) Reconstruction …

Figure 4—figure supplement 1
Download asset Open asset
Next 4 Dutch Stories (n=24).

(A) Phoneme Onset TRF (B) Phoneme Surprisal TRF (C) Phoneme Entropy TRF Lines on the graphs in Figure A-C show the mean and shaded areas show the standard error of the mean.(D) Sources where the …

Figure 5
Download asset Open asset
Accuracy improvement by each linguistic feature calculated by subtracting the model accuracy of previous model from the model which also has the feature of interest for each time window in High and Low Word Entropy Conditions (n=24).

(A) Phoneme Onset (Left Hemisphere – LH on the left, Right Hemisphere RH on the right). (B) Phoneme Surprisal. (C) Phoneme Entropy. Lines on the graphs in Figure A-C show the mean and shaded areas …

Figure 6
Download asset Open asset
Accuracy improvement by each linguistic feature calculated by subtracting the model accuracy of previous model from the model which also has the feature of interest for each time window (n=24).

High (Left) and Low (Right) Word Entropy Conditions for French (Below) and Dutch (Top) Stories.

Author response image 1
Download asset Open asset
Full Model.

Light orange and light green represent Low Word Entropy condition, dark orange and dark green represent High Word Entropy condition for Dutch and French stories, respectively. (Braces indicate the …

Tables

Table 1
LMM results of reconstruction accuracies for Dutch and French stories.
EstimateStd. Errort valuePr(>|t|)
(Intercept)2.86E-032.38E-0412.002.16E-11***
Language (French - Dutch)–7.70E-041.57E-04–4.915.64E-05***
Phon. Onset – Acoustic5.50E-051.14E-054.823.04E-06***
Phon. Surprisal – Phon. Onset7.90E-051.14E-056.917.51E-11***
Phon. Entropy – Phon. Surprisal1.33E-041.14E-0511.632.00E-16***
Word Frequency – Phon. Entropy1.48E-041.14E-0512.952.00E-16***
Language: Phon. Onset – Acoustic–3.64E-051.62E-05–2.252.55E-02*
Language: Phon. Surprisal – Phon. Onset–5.97E-051.62E-05–3.702.88E-04***
Language: Phon. Entropy – Phon. Surprisal–1.04E-041.62E-05–6.459.83E-10***
Language:Word Frequency – Phon. Entropy–1.30E-041.62E-05–8.079.12E-14***

  1. ****<0.0001, ***<0.001, **<0.01, *<0.05.

Table 2
LMM results of reconstruction accuracies for Dutch stories.
EstimateStd. ErrorDft valuePr(>|t|)
(Intercept)2.86E-032.38E-0423.111.992.14E-11***
Phoneme Onset – Acoustic5.50E-051.49E-0592.03.703.63E-04***
Phoneme Surprisal – Phoneme Onset7.90E-051.49E-0592.05.327.41E-07***
Phoneme Entropy – Phoneme Surprisal1.33E-041.49E-0592.08.943.82E-14***
Word Frequency - Phoneme Entropy1.48E-041.49E-0592.09.962.84E-16***
Table 3
LMM results of reconstruction accuracies for French stories.
EstimateStd. ErrorDft valuePr(>|t|)
(Intercept)2.09E-032.09E-0423.010.007.54E-10***
Phoneme Onset – Acoustic1.86E-056.34E-0692.02.944.15E-03**
Phoneme Surprisal – Phoneme Onset1.93E-056.34E-0692.03.043.09E-03**
Phoneme Entropy – Phoneme Surprisal2.88E-056.34E-0692.04.541.70E-05***
Word Frequency - Phoneme Entropy1.76E-056.34E-0692.02.786.54E-03**

  1. ****<0.0001, ***<0.001, **<0.01, *<0.05.

Table 4
LMM results of accuracy improvement by linguistic features for Dutch stories.
EstimateStd. Errordft valuePr(>|t|)
(Intercept)4.22E-056.83E-0631.586.186.83E-07***
hemisphere_right–1.06E-055.26E-06167.00–2.024.55E-02*

  1. ****<0.0001, ***<0.001, **<0.01, *<0.05.

Table 5
LMM results of accuracy improvement by linguistic features for French stories.
EstimateStd. Errordft valuePr(>|t|)
(Intercept)5.16E-063.05E-0641.241.699.80E-02.
hemisphere_right–1.52E-063.10E-06167.00–0.496.26E-01

  1. ****<0.0001, ***<0.001, **<0.01, *<0.05.

Table 6
LMM results of reconstruction accuracies for Phoneme features.
EstimateStd. Errordft valuePr(>|t|)
(Intercept)8.68E-048.17E-0526.410.625.11E-11***
French–8.49E-046.75E-0535.1–12.581.46E-14***
Low Word Entropy–2.69E-044.21E-05118.0–6.383.65E-09***
Right Hemisphere1.22E-049.96E-0523.01.232.33E-01
French: Low Word Entropy3.74E-045.96E-05118.06.295.69E-09***
Table 7
LMM results of reconstruction accuracies for Acoustic Edges.
EstimateStd. Errordft valuePr(>|t|)
(Intercept)6.83E-051.66E-0532.84.112.45E-04***
French8.85E-051.10E-05141.08.033.48E-13***
Low Word Entropy–6.08E-051.10E-05141.0–5.521.61E-07***
Right Hemisphere4.11E-051.94E-0523.02.124.49E-02*
French: Low Word Entropy–9.54E-051.56E-05141.0–6.128.76E-09***

  1. ****<0.0001, ***<0.001, **<0.01, *<0.05.

Table 8
LMM results of reconstruction accuracies for Phoneme Features (Next 4 Dutch Story Parts).
EstimateStd. ErrorDft valuePr(>|t|)
(Intercept)8.14E-057.03E-0639.111.573.36E-14***
French–8.00E-055.62E-06141.0–14.232.00E-16***
Low Word Entropy–5.00E-055.62E-06141.0–8.892.61E-15***
Right Hemisphere–5.32E-067.61E-0623.0–0.704.91E-01
French: Low Word Entropy3.67E-057.95E-06141.04.618.87E-06***
Table 9
LMM results of reconstruction accuracies for Acoustic Edges (Next 4 Dutch Story Parts).
EstimateStd. Errordft valuePr(>|t|)
(Intercept)8.38E-051.70E-0532.04.932.44E-05***
French7.70E-051.09E-05141.07.067.15E-11***
Low Word Entropy–8.18E-051.09E-05141.0–7.496.76E-12***
Right Hemisphere3.30E-051.92E-0523.01.729.89E-02
French: Low Word Entropy–7.44E-051.54E-05141.0–4.823.66E-06***

  1. ****<0.0001, ***<0.001, **<0.01, *<0.05.

Table 10
Stimuli.
Story PartLanguageDurationSpeakerParts used in analysis
Anderson_S01_P01NL4 min 58 sWoman 1Dutch Part 1
Anderson_S01_P02NL5 min 17 sWoman 1Dutch Part 1
Anderson_S01_P03NL4 min 49 sWoman 1Dutch Part 1
Anderson_S01_P04NL5 min 50 sWoman 1Dutch Part 1
Grimm_23_1NL5 min 3 sWoman 2Dutch Part 2
Grimm_23_2NL5 min 32 sWoman 2Dutch Part 2
Grimm_23_3NL5 min 2 sWoman 2Dutch Part 2
Grimm_20_1NL6 min 6 sWoman 2Dutch Part 2
ANGE_part1FR4 min 34 sWoman 3French Part 1
BALL_part1FR4 min 58 sWoman 3French Part 1
EAUV_part1FR5 min 43 sMan 1French Part 1
EAUV_part2FR6 min 1 sMan 1French Part 1
Table 11
Model names and speech features in models.
SpectrogramAcoustic EdgePhoneme OnsetPhoneme SurprisalPhoneme EntropyWord Frequency
Acoustic
Phoneme Onset
Phoneme Surprisal
Phoneme Entropy
Word Frequency
Author response table 1
LMM results of reconstruction accuracies for Phoneme Features.
EstimateStd. Errordft valuePr(>|t|)
(Intercept)3.29E-055.45E-0643.076.033.34E-07****
French-2.48E-054.06E-06164.00-6.126.75E-09****
Low Word Entropy-2.00E-054.06E-06164.00-4.922.10E-06****
Right Hemisphere-3.73E-062.87E-06164.00-1.301.95E-01
French: Low Word Entropy1.09E-055.74E-06164.001.905.96E-02.
Author response table 2
LMM results of reconstruction accuracies for Acoustic Edges.
EstimateStd. Errordft valuePr(>|t|)
(Intercept)4.01E-043.82E-0532.4110.505.85E-12****
French1.48E-052.16E-05140.000.694.93E-01
Low Word Entropy-5.78E-062.64E-05140.00-0.228.27E-01
Right Hemisphere9.76E-054.96E-0528.021.975.91E-02.
French: Low Word Entropy-5.88E-053.05E-05140.00-1.935.60E-02.
Author response table 3
LMM results of reconstruction accuracies for Dutch (first 2 parts) and French (first 2 parts) stories.
EstimateStd. Errort valuePr(>|t|)
(Intercept)2.92E-032.72E-0410.721.99E-10***
Language (French – Dutch)-6.44E-041.52E-04-4.233.13E-04***
Phon. Onset – Acoustic5.57E-051.23E-054.521.13E-05***
Phon. Surprisal – Phon. Onset8.10E-051.23E-056.575.04E-10***
Phon. Entropy – Phon. Surprisal1.25E-041.23E-0510.102.00E-16***
Word Frequency – Phon. Entropy1.22E-041.23E-059.882.00E-16***
Language: Phon. Onset – Acoustic-4.17E-051.74E-05-2.391.80E-02*
Language: Phon. Surprisal – Phon. Onset-6.73E-051.74E-05-3.861.58E-04***
Language: Phon. Entropy – Phon. Surprisal-1.06E-041.74E-05-6.096.30E-09***
Language:Word Frequency – Phon. Entropy-1.27E-041.74E-05-7.298.80E-12***
Author response table 4
LMM results of reconstruction accuracies for Dutch (first 2 parts) and French (second 2 parts) stories.
EstimateStd. Errort valuePr(>|t|)
(Intercept)2.92E-032.72E-0410.722.00E-10***
Language (French- Dutch)-1.09E-031.75E-04-6.202.45E-06***
Phon. Onset – Acoustic5.57E-051.18E-054.714.79E-06***
Phon. Surprisal – Phon. Onset8.10E-051.18E-056.861.03E-10***
Phon. Entropy – Phon. Surprisal1.25E-041.18E-0510.552.00E-16***
Word Frequency – Phon. Entropy1.22E-041.18E-0510.322.00E-16***
Language: Phon. Onset – Acoustic-4.38E-051.67E-05-2.629.49E-03**
Language: Phon. Surprisal – Phon. Onset-7.34E-051.67E-05-4.401.87E-05***
Language: Phon. Entropy – Phon. Surprisal-1.17E-041.67E-05-7.024.25E-11***
Language:Word Frequency – Phon. Entropy-1.39E-041.67E-05-8.321.91E-14***
Author response table 5
LMM results of reconstruction accuracies for Phoneme Features (First 2 parts of Dutch stories and first 2 parts of French stories).
EstimateStd. Errordft valuePr(>|t|)
(Intercept)4.92E-057.45E-0656.116.601.56E-08***
French-5.92E-056.46E-06164.00-9.172.00E-16***
Low Word Entropy-4.89E-056.46E-06164.00-7.572.62E-12***
Right Hemisphere-3.68E-064.57E-06164.00-0.814.21E-01
French: Low Word Entropy4.13E-059.13E-06164.004.521.19E-05***
Author response table 6
LMM results of reconstruction accuracies for Acoustic Edges (First 2 parts of Dutch stories and first 2 parts of French stories).
EstimateStd. Errordft valuePr(>|t|)
(Intercept)1.45E-043.40E-0545.164.271.00E-04***
French-2.16E-062.61E-05164.00-0.089.34E-01
Low Word Entropy3.58E-052.61E-05164.001.371.72E-01
Right Hemisphere6.05E-051.85E-05164.003.281.27E-03**
French: Low Word Entropy-1.20E-043.69E-05164.00-3.241.43E-03**
Author response table 7
LMM results of reconstruction accuracies for Phoneme Features (First 2 parts of Dutch stories and second 2 parts of French stories).
EstimateStd. Errordft valuePr(>|t|)
(Intercept)4.97E-056.73E-0669.147.392.55E-10***
French-6.57E-056.37E-06164.00-10.312.00E-16***
Low Word Entropy-4.89E-056.37E-06164.00-7.671.45E-12***
Right Hemisphere-4.77E-064.51E-06164.00-1.062.91E-01
French: Low Word Entropy3.78E-059.01E-06164.004.204.38E-05***
Author response table 8
LMM results of reconstruction accuracies for Acoustic Edges (First 2 parts of Dutch stories and second 2 parts of French stories).
EstimateStd. Errordft valuePr(>|t|)
(Intercept)1.41E-043.06E-0547.694.603.11E-05***
French-4.81E-062.43E-05164.00-0.208.44E-01
Low Word Entropy3.58E-052.43E-05164.001.471.43E-01
Right Hemisphere6.87E-051.72E-05164.004.009.69E-05***
French: Low Word Entropy-1.33E-043.44E-05164.00-3.871.55E-04***

Additional files

MDAR checklist
https://cdn.elifesciences.org/articles/82386/elife-82386-mdarchecklist1-v1.docx

Download links