Neural synchronization is strongest to the spectral flux of slow music and depends on familiarity and beat salience

  1. Kristin Weineck  Is a corresponding author
  2. Olivia Xin Wen
  3. Molly J Henry
  1. Research Group “Neural and Environmental Rhythms”, Max Planck Institute for Empirical Aesthetics, Germany
  2. Goethe University Frankfurt, Institute for Cell Biology and Neuroscience, Germany
  3. Department of Psychology, Toronto Metropolitan University, Canada
8 figures, 1 table and 1 additional file

Figures

Figure 1 with 2 supplements
Experimental design and musical features.

(A) Schematic of the experimental procedure. Each trial consisted of the presentation of one music segment, during which participants were instructed to listen attentively without moving. After a 1 s silence, the last 5.5 s of the music segment was repeated while participants tapped their finger along with the beat. At the end of each trial, participants rated their enjoyment and familiarity of the music segment, as well as the ease with which they were able to tap to the beat (Translated English example in Figure: “How much did you like the song?” rated from “not at all” to “very much”). (B) Exemplary traces of the four musical features of one music segment. (C) Z-scored mean amplitude spectrum of all 4 musical features. Light orange dashed boxes highlight when the FFT Frequency corresponds to the stimulation tempo or first harmonic. (D) Mutual information (MI) for all possible feature combinations (green) compared to a surrogate distribution (yellow, three-way ANOVA, *pFDR <0.001, rest: pFDR <0.05). Boxplots indicate the median, the 25th and 75th percentiles (n=52). (E) MI scores between all possible feature combinations (*pFDR <0.001, rest: pFDR <0.05).

Figure 1—source data 1

Source data for visualizing and analyzing the musical features.

https://cdn.elifesciences.org/articles/75515/elife-75515-fig1-data1-v1.zip
Figure 1—figure supplement 1
Shared Mutual Information (MI) between musical features across tempo conditions.

MI scores for all possible feature combinations as a function of tempo. Significance was evaluated based on a three-way ANOVA (with tempo as second factor, see Materials and methods for further information) and a follow-up pairwise comparison. Red boxes indicate significant tempo-dependent shared MI between musical features with a significance level of pFDR <0.05.

Figure 1—figure supplement 2
Tempo manipulations of original music segments.

(A) Histogram of the original tempo of the music segments prior to tempo manipulation binned between 1–4 Hz in steps of 0.25 Hz (n=72). Histograms of (B) the difference between the original tempo and the slowest possible tempo manipulation and (C) the difference between the fastest possible tempo manipulation and the original tempo per music segment.

Figure 2 with 2 supplements
Stimulus–response correlation and stimulus–response coherence are tempo dependent for all musical features.

(A) Projected topography of the first reliable component (RC1). (B) Average SRCorr of the aligned neural response and surrogate distribution (grey) across tempi for each musical feature (left) and the z-scored SRCorr based on a surrogate distribution (right) (± SEM; shaded area). Highest correlations were found at slow tempi (repeated-measure ANOVA, Greenhouse-Geiser correction where applicable). The slopes of regression models were used to compare the tempo-specificity between musical features. (C) Mean SRCorr across musical features. Highest correlations were found in response to spectral flux with significant differences between all possible feature combinations, pFDR <0.001, except between the envelope or derivative and beat onsets, pFDR <0.01 (n=34, repeated-measure ANOVA, Tukey’s test, median, 25th and 75th percentiles). Z-scored SRCoh in response to the (D) amplitude envelope, (E) first derivative, (F) beat onsets and (G) spectral flux. Each panel depicts the SRCoh as colorplot (left) and the pooled SRCoh values at the stimulation tempo and first harmonic (right, n=34, median, 25th and 75th percentile). (H) Same as (C) for the SRCoh with significant differences between all possible feature combinations (pFDR <0.001) apart between the envelope and beat onsets. Coherence values were averaged over the stimulus tempo and first harmonic. (I) Mean differences of SRCoh values at the stimulation tempo and first harmonic (n=34, negative values: higher SRCoh at harmonic, positive values: higher SRCoh at stimulation tempo, paired-sample t-test, pFDR <0.05). (J) Same as (I) based on the FFT amplitudes (pFDR <0.001).

Figure 2—source data 1

Source data for the RCA-based measures stimulus-response correlation (SRCorr) and stimulus-response coherence (SRCoh).

https://cdn.elifesciences.org/articles/75515/elife-75515-fig2-data1-v1.zip
Figure 2—source data 2

Output of the RCA-based analysis of the first two stimulation subgroups (based on Kaneshiro et al., 2020).

https://cdn.elifesciences.org/articles/75515/elife-75515-fig2-data2-v1.zip
Figure 2—source data 3

Output of the RCA-based analysis of the last two stimulation subgroups (based on Kaneshiro et al., 2020).

https://cdn.elifesciences.org/articles/75515/elife-75515-fig2-data3-v1.zip
Figure 2—figure supplement 1
SRCorr and SRCoh in response to the full-band amplitude envelope and derivative.

(A) Mean SRCorr across stimulation tempi and musical features (± SEM). Similar to Figure 2B of the main manuscript, the full-band amplitude envelope (Hilbert transform) and resultant first derivative were used. Significance between tempi was assessed using a repeated-measure ANOVA (with Greenhouse-Geiser correction if applicable). (B) SRCorr across musical features. Statistically significant differences were identified between all musical feature combinations except between the envelope and beat onsets using a repeated-measure ANOVA (pFDR <0.001, median, 25th and 75th percentiles). (C) Colormap of the fast Fourier Transform (FFT) of the first reliable component (RC1) across stimulation tempi. Note that the colorbar is in a logarithmic scale.

Figure 2—figure supplement 2
Individual data examples for the SRCorr and SRCoh.

(A) Each plot shows the mean z-scored SRCorr of one participant across stimulation tempi. Each line represents the average of one musical feature (± SEM). (B) Illustrative color plots of the normalized SRCoh in response to the spectral flux across stimulation tempi (1–4 Hz) of the same four participants as in (A). Z-scoring was based on the surrogate distribution.

Figure 3 with 3 supplements
TRFs are tempo dependent.

(A) Mean TRF (± SEM) correlations as a function of stimulation tempo per stimulus feature (p-values next to the legend correspond to a repeated-measure ANOVA across tempi for every musical feature and the p-value below to the slope comparison of a linear regression model). TRF correlations were highest for spectral flux and combined musical features for slow tempi. The TRF correlations were z-scored based on a surrogate distribution (right panel). (B) Violin plots of the TRF correlations across musical features. Boxplots illustrate the median, 25th and 75th percentiles (n=34). Significant pairwise musical feature comparisons were calculated using a repeated-measure ANOVA with follow-up Tukey’s test, *pFDR <0.001. (C) Top panel: Topographies of the TRF correlations and TRF time lags (0–400ms) in response to the amplitude envelope. Each line depicts one stimulation tempo (13 tempi between 1 Hz, blue and 4 Hz, green). Lower panel: Colormap of the normalized TRF weights of the envelope in the same time window across stimulation tempi. (D) Same as (C) for the first derivative, (E) beat onsets and (F) spectral flux. Cluster-based permutation testing was used to identify significant tempo-specific time windows (red dashed box, p<0.05). Inset: Mean TRF weights in response to the spectral flux for time lags between 102 and 211ms (n=34, median, 25th and 75th percentile).

Figure 3—source data 1

Source data of the TRF correlations and weights.

https://cdn.elifesciences.org/articles/75515/elife-75515-fig3-data1-v1.zip
Figure 3—figure supplement 1
TRFs in response to the full-band amplitude envelope and first derivative show similar patterns as the gammatone filtered musical features.

(A) Mean TRF correlations across stimulation tempi and musical features (± SEM). (B) TRF correlations across musical features. Violin plots indicate the median, 25th and 75th percentiles (n=34, repeated-measure ANOVA; *pFDR <0.001). Similarly to the gammatone filtered features, the full-band envelope and derivative show significantly smaller TRF correlations in comparison to the spectral flux and the combination of all features. (C) Auditory topographies of the TRF correlations in response to the full-band features. (D) TRF weights for time lags between 0 and 400ms of the Hilbert envelope. Cluster-based permutation testing was used to identify significant time lag windows (envelope: 250–400ms, p=0.01; derivative: 281–400ms, p=0.02). Each line represents one stimulation tempo (n=13, blue, 1 Hz - green, 4 Hz). (E) Colormaps of the TRF weights over the same time lags for the envelope. Red dashed lines highlight significant time windows. (F) Average TRF weights from the significant time lag window for the envelope (n=34, median, 25th and 75th percentile). (G) Latency of the P3 peak located in the significant time lag window across stimulation tempi for the full-band envelope TRFs. The latencies were divided into two subgroups (1–2.5 Hz and 2.75–4 Hz) and a regression fit into the data. (H)-(K) Same as (D)-(G) for the first derivative.

Figure 3—figure supplement 2
Corrected TRF weights of the spectral flux after removing the effects of the other musical features.

(A) TRF weights in response to the spectral flux. To calculate those weights, a multivariate TRF approach based on the amplitude envelope, first derivative and beat onsets was used and the resulting TRF predictions were subtracted from the ‘actual’ EEG data. The residual EEG data was used to compute the spectral TRF model. (B) Similarly to the main Figure 3F the TRF weights at the previously calculated significant time window were plotted as a function of tempo. The boxplots indicate the median, 25th and 75th percentile (n=34).

Figure 3—figure supplement 3
No differences in TRFs correlations between more vs. less modulated music.

(A) TRF correlations for up to three trials per participant when the original music tempo ≈ manipulated music tempo (labelled ‘Original’) vs. when the manipulated music tempo was faster than the original music segments (‘Slow’). For this analysis, different trials for the different stimulation subgroups from the same stimulation tempo condition (here: 2.25 Hz) were used (nori = 91; nslow = 96 trials). In the right plot the TRF correlation were z-scored based on the surrogate distribution on a per trials basis. No significant differences were observed between groups (repeated-measures ANOVA). (B) Same as (A), but here the original tempo was at a slower tempo and was contrasted against music segments that were originally faster and were manipulated to be played at a 1.5 Hz (nori = 57; nfast = 58 trials). The boxplots indicate the median, 25th and 75th percentile.

Figure 4 with 1 supplement
Significant relationships between SRCorr and TRF correlations for all musical features.

(A) Linear-mixed effects models of the SRCorr (predictor variable) and TRF correlations (response variable) in response to the amplitude envelope. Each dot represents the mean correlation of one participant (n=34) at one stimulation tempo (n=13) (=grouping variables; blue, 1 Hz-green, 4 Hz). Violin plots illustrate fixed effects coefficients (β). (B)-(D) same as (A) for the first derivative, beat onsets and spectral flux. For all musical features, the fixed effects were significant.

Figure 4—source data 1

Source data for comparing the results of the TRF and RCA-based measures.

https://cdn.elifesciences.org/articles/75515/elife-75515-fig4-data1-v1.mat
Figure 4—figure supplement 1
Significant relationships between SRCoh and TRF correlations for all musical features at the stimulation tempo and first harmonic.

(A) Linear-mixed effects models of the SRCoh (predictor variable) and TRF correlations (response variable) in response to the amplitude envelope at the intended stimulation tempo (left, dark grey) and first harmonic (right, light grey). Each dot represents the mean correlation of one participant (n=34) at one stimulation tempo (n=13) (=grouping variables; blue, 1 Hz-green, 4 Hz). Violin plots illustrate fixed effects coefficients (β). (B)-(D) same as (A) for the first derivative, beat onsets and spectral flux. For all musical features, the fixed effects were significant (pFDR <0.01). Model comparisons were implemented based in the Likelihood ratio test (LRStat, p<0.001).

Figure 5 with 3 supplements
TRF correlations are highest in response to familiar songs.

(A) Normalized (to the maximum value per rating/participant), averaged behavioral ratings of enjoyment, familiarity and easiness to tap to the beat (± SEM). No significant differences across tempo conditions were observed (repeated-measure ANOVA with Greenhouse-Geiser correction). (B) Mean TRF correlations topography across all ratings (based on the analysis of 15 trials with highest and lowest ratings per behavioral measure). (C) Violin plots of TRF correlations comparing low vs. highly enjoyed, low vs. highly familiar, and subjectively difficult vs. easy beat trials. Strongest TRF correlations were found in response to familiar music and music with an easy-to-perceive beat (n=34, paired-sample t-test, *pFDR <0.05). Boxplots indicate median, 25th and 75th percentile. (D) Mean TRFs (± SEM) for time lags between 0–400ms of more and less enjoyable music songs. (E)-(F) Same as (D) for trials with low vs. high familiarity and difficult vs. easy beat ratings.

Figure 5—source data 1

Source data of the behavioral ratings and TRF correlations.

https://cdn.elifesciences.org/articles/75515/elife-75515-fig5-data1-v1.zip
Figure 5—figure supplement 1
Significant differences of FFT amplitudes at stimulus-relevant frequencies between differently rated trials.

Z-scored average FFT amplitudes at the stimulation tempo and first harmonic of the 15 highest vs. lowest ratings per behavioral rating category (n=34, low vs. highly enjoyed, low vs. highly familiar, and subjectively difficult vs. easy beat trials). Significant differences were observed for all pairwise comparisons (paired-sample t-test, pFDR≈0.01).

Figure 5—figure supplement 2
Musical training did not have an effect on TRF correlations regardless of the musical feature.

Scatter plot between the general sophistication index (F7, Gold-MSI) and mean TRF correlations per participant (n=34). No significant correlations between the Gold-MSI and TRF correlations were observed in response to the (A) amplitude envelope, (B) first derivative, (C) beat onsets and (D) spectral flux.

Figure 5—figure supplement 3
Music tapping rate across participants.

(A)-(B) Illustrative histograms of the relative number of trials per tapped music rate of two participants with a fitted skewed Gaussian. The modes indicate the preferred music tapping rate and the width the shape of the fitted Gaussian. (C) Mean music-tapping histogram across all participants (n=29, 5 participants were excluded from the music tapping analysis due to inconsistent taps). The mean preferred tapping frequency was 1.55 Hz.

Author response image 1
Author response image 2
Author response image 3

Tables

Appendix 1—table 1
Overview over the music stimuli.

Parameters of stimulus creation for all 72 musical stimulus segments. The columns indicate 1. the stimulus number, 2. title of the musical piece, 3. the Artist of each musical piece, 4. the CD each piece was taken from (available at Qobuz Downloadstore), 5. timestamp of the music segment onset relative to the start of the recording [min.sec,ms], 6. duration of the music segment [sec] relative to the start of the music segment, 7. original tempo of excerpt [BPM; beats per minute] based on the taps of the authors and their colleagues and 8. frequency range [Hz] of the tempo-modulation (in 0.25 Hz steps) for each music piece.

No.TitleArtistCDStart [min]Duration [sec]Tempo [BPM/Hz]Range [Hz]
1Abba MedleySuper TroopersInstrumental Pop Hits7.30,7122,99136.09 /2.271.5–3.75
2Abba MedleySuper TroopersInstrumental Pop Hits8.59,5721,69135.92 /2.271.75–4
3All is AliveFrancesco P.Instrumental Hits, Vol.11.22,8521,41128.27 /2.141.5–4
4All is AliveFrancesco P.Instrumental Hits, Vol.12.06,7221,62127.98 /2.131.5–4
5ApacheThe ShadowsRock Story "Instrumental Versions"0.39,5914,78135.22 /2.251.75–4
6ApacheThe ShadowsRock Story "Instrumental Versions"0.54,2521,60133.86 /2.231.75–4
7La BikinaRubén Fuentes GassonBachata0.47,4914,99124.83 /2.081.75–4
8BulldogThe VenturesRock Story "Instrumental Versions"0.06,1518,13151.33 /2.522–4
9Careless WhisperMads HaaberInstrumental Pop Hits2.41,9325,5076.93 /1.281–2.75
10CocaineCorben CassavetteInstrumental Pop Hits1.39,2925,74105.13 /1.751.5–4
11Dark PlaceBeataddictzStreet Beatz, Vol.20.20,8322,2292.13 /1.541–3.75
12F.B. I.The ShadowsRock Story "Instrumental Versions"0.20,5915,31140.05 /2.331.25–4
13Five TripsTr3ntatr3 GiriInstrumental Hits, Vol.11.48,7916,48123.24 /2.051.5–4
14GuyboEddie CochranRock Story "Instrumental Versions"0.18,9916,28110.00 /1.831.5–3
15Gypsy Salsa, Cha Cha BeatCorp Latino Dance GroupHot Latin Dance1.57,0624,26100.04 /1.671.5–3
16Highway RiderzBeataddictzStreet Beatz, Vol.20.19,6130,3097.13 /1.621–4
17In GoChuck BerryRock Story "Instrumental Versions"0.26,1024,00116.30 /1.941.5–4
18Oh by Jingo!Chet AtkinsRock Story "Instrumental Versions"0.07,4723,59120.55 /2.011–3.5
19Keep It 1,000BeataddictzStreet Beatz, Vol.21.13,8725,1078.06 /1.301–3
20The Last DayBeataddictzStreet Beatz, Vol.21.27,4022,5288.13 /1.471–2.5
21For the Last TimeBeataddictzStreet Beatz, Vol.21.16,8826,2174.86 /1.251–3.25
22Lights OutBeataddictzStreet Beatz, Vol.20.32,1221,6089.12 /1.491.25–3.5
23I likeFrancesco P.Instrumental Hits, Vol.11.34,1927,95112.13 /1.871.25–3.75
24Dark LineAlex CundariInstrumental Hits, Vol.10.37,8119,75106.08 /1.771.25–3
25Live ForeverThe WonderwallsInstrumental Pop Hits1.16,2822,6190.11 /1.501.5–3.75
26Lucy in the Sky with DiamondsRicardo CalienteInstrumental Pop Hits2.43,5523,9381.11 /1.351–3.5
27MonalisaKen LaszloInstrumental Hits, Vol.11.13,3729,79129.97 /2.171.5–4
28MonalisaKen LaszloInstrumental Hits, Vol.12.13,0128,80130.02 /2.171.5–4
29Can't Fight the MoonlightJon CarranInstrumental Pop Hits1.10,0218,5697.95 /1.631.5–3.75
30Muy TranquiloGramatikMuy Tranquilo2.05,126,5990.04 /1.501–3.25
31No MercyBeataddictzStreet Beatz, Vol.20.12,1526,4176.11 /1.271–3.5
32I'm A PushaBeataddictzStreet Beatz, Vol.20.11,0722,6085.15 /1.421–3.25
33Rockin' the Blues AwayTiny Grimes QuintetRock Story "Instrumental Versions"0.05,6120,84141.05 /2.351.5–4
34The Rocking GuitarIni KamozeRock Story "Instrumental Versions"0.17,3916,21118.66 /1.981.5–3.25
35Country Rodeo SongMarco RinaldoCountry Instrumental Mix1.46,3527,70112.94 /1.881.5–3.75
36I Shot the SheriffCorben CassavetteInstrumental Pop Hits0.19,5225,5494.12 /1.571.25–4
37Sing Sing SingBenny GoodmanSing Sing Sing0.18,2336,01108.68 /1.811.5–3
38Si Una VezPete AstudilloBachata0.46,5416,95124.54 /2.081.5–3
39I'm Still StandingRicardo CalienteInstrumental Pop Hits0.39,1421,7286.04 /1.431.25–2.75
40Streets On FireBeataddictzStreet Beatz, Vol.20.23,3324,9981.16 /1.351–3.75
41TequilaThe ChampsRock Story "Instrumental Versions"1.02,8521,4589.40 /1.491–3
42Vegas DreamVegas ProjectInstrumental Hits, Vol.11.13,7322,73128.08 /2.131.5–3.25
43I Can't WaitAlex CundariInstrumental Hits, Vol.10.23,7324,0883.59 /1.391–2.5
44Who DatBeataddictzStreet Beatz, Vol.21.15,5824,5087.10 /1.451–3.25
45Abba MedleySuper TroopersInstrumental Pop Hits0.28,3827,05136.09 /2.271.25–4
46Abba MedleySuper TroopersInstrumental Pop Hits5.09,8921,71136.09 /2.271–3
47Abba MedleySuper TroopersInstrumental Pop Hits6.03,5920,64136.09 /2.271.5–4
48La BikinaRubén Fuentes GassonBachata1.18,3146,00124.83 /2.081.75–4
49BulldogThe VenturesRock Story "Instrumental Versions"0.44,9819,20151.33 /2.522–4
50BulldogThe VenturesRock Story "Instrumental Versions"1.21,2538,85151.33 /2.522–4
51Careless WhisperMads HaaberInstrumental Pop Hits3.09,4424,4176.93 /1.281–2.75
52Dark PlaceBeataddictzStreet Beatz, Vol.21.45,7335,1092.13 /1.541–3.75
53F.B. I.The ShadowsRock Story "Instrumental Versions"0.37,3020,91140.05 /2.331.5–4
54GuyboEddie CochranRock Story "Instrumental Versions"0.36,3117,53110.00 /1.831.5–3
55Highway RiderzBeataddictzStreet Beatz, Vol.20.59,1920,4097.13 /1.621–4
56In GoChuck BerryRock Story "Instrumental Versions"0.48,4226,80116.30 /1.941.5–4
57Oh by Jingo!Chet AtkinsRock Story "Instrumental Versions"0.40,8923,00120.55 /2.011.25–3.5
58Live ForeverThe WonderwallsInstrumental Pop Hits1.41,0234,0090.11 /1.501.25–3.5
59Live ForeverThe WonderwallsInstrumental Pop Hits3.35,7225,8090.11 /1.501.25–3.5
60Lucy in the Sky with DiamondsRicardo CalienteInstrumental Pop Hits3.06,4023,0081.11 /1.351–3.5
61Can't Fight the MoonlightJon CarranInstrumental Pop Hits1.42,2621,8297.95 /1.631.5–3.75
62No MercyBeataddictzStreet Beatz, Vol.20.50,1626,7476.11 /1.271–3.5
63No MercyBeataddictzStreet Beatz, Vol.22.33,5326,4176.11 /1.271–3.5
64Rockin' the Blues AwayTiny Grimes QuintetRock Story "Instrumental Versions"0.47,6525,50141.05 /2.351.5–4
65Rockin' the Blues AwayTiny Grimes QuintetRock Story "Instrumental Versions"1.12,7836,07141.05 /2.351.5–4
66The Rocking GuitarIni KamozeRock Story "Instrumental Versions"0.33,5815,08118.66 /1.981.5–3.25
67Country Rodeo SongMarco RinaldoCountry Instrumental Mix2.13,9924,03112.94 /1.881.5–3.75
68Sing Sing SingBenny GoodmanSing Sing Sing1.08,6518,87108.68 /1.811.5–3
69Sing Sing SingBenny GoodmanSing Sing Sing2.46,0336,29108.68 /1.811.5–3
70Si Una VezPete AstudilloBachata1.03,7029,27124.54 /2.081.5–3
71Streets on fireBeataddictzStreet Beatz, Vol.22.00,6338,0081.16 /1.351–3.75
72I Can't WaitAlex CundariInstrumental Hits, Vol.11.10,7134,4883.59 /1.391–2.5

Additional files

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Kristin Weineck
  2. Olivia Xin Wen
  3. Molly J Henry
(2022)
Neural synchronization is strongest to the spectral flux of slow music and depends on familiarity and beat salience
eLife 11:e75515.
https://doi.org/10.7554/eLife.75515