Detecting behavioural oscillations with increased sensitivity: A modification of Brookshire’s (2022) AR-surrogate method

Reviewer #1 (Public review):

Summary:

Some years ago, Brookshire proposed a method to identify oscillations in behavioural data that controls for effects of aperiodic trends. Such trends can produce false positive results if not controlled for. Although this method successfully controlled for this issue, it was also relatively insensitive to true effects, and it remained unclear whether it was unable to replicate published evidence for behavioural oscillations because they were false positives or the method could not detect them. In simulated data, Harris & Beale show that their revised version of the method proposed by Brookshire is more sensitive to effects and equally unsusceptible to false positives. When applied to available data, this new version indeed revealed evidence for behavioural oscillations. This paper is therefore an important piece in the puzzle of the ongoing debate on behavioural oscillations.

Strengths:

(1) The paper is well written and compact.

(2) The new method proposed is tested thoroughly, and its application in simulated data shows its properties.

(3) It is very important that the code is made publicly available.

(4) The fact that this new version identifies behavioural oscillations in available datasets can resolve the current debate on the existence of such oscillations.

Weaknesses:

I see the following weaknesses as minor.

(1) I wonder whether the frequency-dependent results (e.g., Figures 7 and 8) need to be seen in light of the sampling rate used in the simulations. For example, a lower sampling rate might be sufficient if only low frequencies are of interest in the data and lead to higher sensitivity as the number of trials (per time point) can be increased. Conversely, a higher sampling rate might lead to a higher sensitivity for the detection of effects at higher frequencies.

(2) The behavioural oscillations from individual participants do not need to have common phases for this analysis to reveal an effect. However, this also means that in a scenario where they do have common phases, this similarity remains "unused" by the analysis (e.g., due to similar phases, the oscillation could be easier to identify on the group level as signals that are not phase locked are averaged out). In such a scenario, it remains unclear whether the analysis proposed is the most sensitive one.

Reviewer #2 (Public review):

Summary

Dozens of published studies have investigated rhythms in behavior. These studies have typically tested for oscillations by shuffling the timestamps of the individual observations and comparing the resulting shuffled spectra with the empirical spectrum. However, that shuffling-in-time method leads to strongly inflated rates of false positives. Brookshire (2022) suggested a method that controls the rate of false positives (the "AR-surrogate method"). In the current study, Harris and Beale propose a modification of the AR-surrogate analysis method with the goal of increasing the sensitivity while maintaining a low rate of false positives.

This study is carefully conducted and it addresses an interesting question. However, the simulations were performed in a way that ignores one important source of temporal structure: non-oscillatory patterns that are consistent across subjects. In order to know whether the updated AR-surrogate method would control the rate of false positives in real behavioral data, we need to know whether it controls the rate of false positives when the data includes aperiodic patterns that are consistent across subjects.

Strengths

This study was constructed carefully and written up very clearly. It's a clever idea to analyze the time series separately for each participant. After examining how the updated AR-surrogate method behaves when the simulated data includes consistency across subjects, this will be a useful contribution to the field.

Weaknesses

When describing their simulations of behavioral data, the authors write: "Each participant's data was produced by creating an independent idealised time-course of 1-second length, sampled at 60 Hz."

Because these simulations generated a totally independent time-course for every subject, they don't capture an important source of aperiodic structure in real behavior: consistent non-oscillatory patterns that occur across subjects. In other words, these simulations do not account for any pattern that remains after averaging across subjects. The literature is rich with patterns that persist across subjects, including all the studies of behavioral oscillations that analyze their data after averaging across subjects (e.g., Landau & Fries, 2012; Fiebelkorn, Saalmann, & Kastner, 2013, etc). As a consequence, I suspect that the reported increase in power comes at the expense of a corresponding increase in false positives, but that the false positives aren't captured here due to the lack of consistency across simulated subjects.

It's therefore possible that the authors' updated AR-surrogate method would mistakenly conclude that behavior oscillates when it only includes aperiodic consistency across subjects. Since that kind of aperiodic structure is ubiquitous, this analysis could lead to very high rates of false positives. Luckily, it's easy to find out whether this is the case - the authors could simulate data using an idealized time-course that is consistent across subjects.

Reviewer #3 (Public review):

Summary:

This work revises the autoregressive surrogate (AR-surrogate) method proposed in 2022 by Brookshire to estimate the oscillatory content of behavioural time series. The main issue raised by Brookshire was the inadequacy of methods used in a series of papers that rely on shuffling the time axis of the behavioural data. Brookshire argued that while this approach tests for temporal structures, it does not differentiate between 1/f activity and true oscillatory signals. The AR-surrogate, on the other hand, removes aperiodic activity and should therefore provide a more accurate representation of oscillatory behaviour.

In this well-written paper, Harris and Beale clearly describe an improvement to Brookshire's method, which has been called into question for its low sensitivity.

Strengths:

The starting point of this work is that oscillatory patterns should be tested at the individual participant level rather than at the group level. This is critical because anyone working with behavioural data will know that averaging across participants generates distorted time series. Averaging also assumes phase consistency across participants, which may not always be valid.

Once freed from this limitation, the results presented here are exciting and convincingly demonstrate a significant improvement over the original implementation.

The authors have devised a series of tests that systematically assess the effects of participant and trial number, and effect size on the accuracy of AR-surrogate results. This is particularly useful, as it may guide researchers in designing appropriate behavioural experiments.

Weaknesses:

The method proposed here is undoubtedly an improvement on the original. However, its biggest limitation is the restriction on the frequencies that can be investigated. This is acknowledged by the authors, who rightly point out that there is still room for improvement. Another issue is that modulation depths below 10-15% may be difficult to detect.