EEG responses to visual stimulation are modulated by walking.

(A) Participant wearing the setup for wireless EEG recordings and a virtual reality headset. An Open BCI EEG electrode cap is combined with META QUEST 3. Inset: first person view of the virtual environment. Red fixation dot was enlarged in figure to make it visible. (B) A third person view of the virtual scene used to study visual responses to a reversing checkerboard pattern. The black line on the ground indicates the safety boundary within which the participant was instructed to walk around during the movement block of the paradigm. The grid indicates 1 by 1 m squares. (C) Participants viewed a reversing checkerboard in two conditions: during half of the trials, they remained seated, and during the other half they walked within a defined safety boundary. Checkerboard reversed colors at random intervals every 2 to 4 s. (D) 3D trajectory of an example participant during the visual experiment. Movement was recorded using accelerometers integrated into the VR headset. (E) Visually evoked potentials measured in the sitting condition on the occipital electrodes O1 and O2. Responses recorded on other electrodes are presented in Figure S2. Solid black line represents the mean, and shading indicates the SEM across participants. Dashed vertical red line indicates the time of the checkerboard reversal. P1 - positive peak at around 88 ms. (F) As in E, but while participants were walking. Inset: comparison of visual evoked potentials measured in the sitting E and walking F conditions. The horizontal bar above the plot marks time bins in which responses differ significantly (black: p < 0.05) or do not differ (gray: p > 0.05).

Visuomotor mismatch elicits strong EEG responses.

(A) Participant navigating a virtual tunnel. Inset: first person view of the tunnel. (B) Third person view of the virtual environment used to study visuomotor mismatch responses. The grid indicates 1 by 1 m squares. (C) In closed loop sessions, the walking speed of participants was coupled to movement in the virtual corridor. Visuomotor mismatches were introduced by briefly halting the visual flow for 0.5 s. Following each visuomotor mismatch event, the view was updated to the participants’ current position (brief peaks in visual flow speed after the mismatch event) and visuomotor coupling was resumed. (D) Trajectory of an example participant during the visuomotor mismatch paradigm. (E) Responses to visuomotor mismatches recorded from occipital electrodes (O1 and O2). Solid black line represents the mean, and shading indicates the SEM across participants. Dashed vertical red lines are onset and offset of the visuomotor mismatch. The gray shaded areas mark the analysis windows used to quantify response strength in G and H. Analysis windows of 100 ms were centered on the peak of the visuomotor mismatch response and the visual flow re-onset response. (F) As in E, but for visual flow playback halt responses recorded from occipital electrodes. (G) Comparison of the response strength to visuomotor mismatches and playback halts. Here and elsewhere: boxes mark median, quartiles, and range of data not considered outliers. Each circle represents data from one participant. ***: p<0.001. One data point not shown (Mismatch response, at −9.5 μV). See Table S1 for all statistical information. (H) Comparison of the response strength to visual flow re-onset events following visuomotor mismatches in the closed loop condition and playback halts in the open loop condition. n.s.: not significant. See Table S1 for all statistical information. (I) Average walking speed of participants during visuomotor mismatches. Inset: Temporally expanded view. The horizontal bar above the plot marks time bins where walking speed differs significantly (black: p < 0.05) or does not differ (gray: p > 0.05) from baseline.

Visuomotor mismatch responses are most prominent in occipital cortex.

(A) Top down view of EEG electrode locations on the head. (B) Visuomotor mismatch responses measured on electrodes shown in A. Solid black lines represent the mean, and shading indicates the SEM across participants. Dashed vertical red lines are onset and offset of the visuomotor mismatch. (C) Comparison of the response strength to visuomotor mismatches measured on electrodes shown in A. Average response strength was calculated within a 100 ms window centered on the peak of the average visuomotor mismatch response across all electrodes. Boxes mark median, quartiles, and range of data not considered outliers. Each circle represents data from individual participant. ***: p<0.001, **: p<0.01, *: p<0.05, n.s.: not significant. See Table S1 for all statistical information. (D) The responses shown in B, averaged over pairs of electrodes and overlaid. Solid black lines represent mean and shading SEM across electrodes. Dashed vertical red lines are onset and offset of the visuomotor mismatch. (E) Comparison of the latency to half maximum response for the 4 electrode pairs. n.s.: not significant. See Table S1 for all statistical information.

Visuomotor mismatch responses have reversed polarity and more power compared to visual responses.

(A) Comparison of visual (Figure 1E) and visuomotor mismatch (Figure 2E) responses recorded from occipital electrodes. Solid black line represents the mean, and shading indicates the SEM across participants. Dashed vertical red lines are onset (visual and mismatch) and offset (mismatch) of the stimuli. (B) Comparison of the power of visual and visuomotor mismatch responses, calculated within a 0 - 0.5 s time window following stimulus onset. Boxes mark median, quartiles, and range of data not considered outliers. Each circle represents data from individual participant. ***: p<0.001. See Table S1 for all statistical information.

Visuomotor mismatch responses are larger than auditory oddball mismatch responses but have similar temporal dynamics

(A) Design of the auditory oddball paradigm with examples of the silent films participants were exposed to. (B) Top: Auditory responses to the 1 kHz tone presented as a standard versus as a deviant. Bottom: Oddball mismatch response calculated by subtracting the average response over trials in which the tone was presented as a standard from those when the tone was presented as a deviant. Solid black lines represent the mean, and shading indicates the SEM across participants. Dashed vertical red line is the onset of the auditory stimulus. (C) As in B, but for the 1.2 kHz tone. (D) Comparison of visuomotor mismatch, oddball mismatch (average over data shown in panel B and C) and playback halt responses recorded from occipital electrodes. Solid black lines represent the mean response, with shading indicating SEM across electrodes. Dashed vertical red lines are onset (visual, mismatch) and offset (mismatch) of the stimuli. (E) Comparison of the power of visuomotor mismatch, oddball mismatch response (average over data shown in panel B and C) and playback halt responses, calculated within a 0 s - 0.5 s time window following stimulus onset. Boxes mark median, quartiles, and range of data not considered outliers. Each circle represents data from individual participant. ***: p<0.001; *p<0.05. See Table S1 for all statistical information.

Movement onsets result in increases in variance in EEG activity.

(A) We included only data in which the EEG signals remained below an exclusion threshold of 100 μV. Most of the movement related variance in the EEG activity is likely a movement artifact. Example of an EEG signal (black line) at movement onset that reached exclusion threshold (100 μV). Overlaid is the walking speed of the participant (green line). (B) As in A, but for an example of an EEG signal at movement onset that did not reach exclusion threshold (100 μV). (C, D) As in A, but for examples of EEG signals at movement onset with minimal movement contamination.

Visual responses are strongest in occipital cortex.

(A) Top down view of EEG electrode locations on the head. (B) Visual evoked responses measured on electrodes shown in A. Solid black lines represent the mean, and shading indicates the SEM across participants. Dashed vertical red line is the onset of the checkerboard inversion. (C) Comparison of the response strength of visual evoked potentials measured on electrodes shown in A. Average response strength was calculated within a 100 ms window centered on the peak of the average visual response across all electrodes. Boxes mark median, quartiles, and range of data not considered outliers. Each circle represents data from an individual participant. ***: p<0.001, **: p<0.01, *: p<0.05, n.s.: not significant. See Table S1 for all statistical information.

Playback halt responses do not depend on whether coupling is full.

(A) A subset of participants viewed 6DOF playback (Methods). Shown are visual flow playback halt responses recorded from occipital electrodes in these participants. Solid black lines represent the mean, and shading indicates the SEM across participants. Dashed vertical red lines are onset and offset of the visuomotor mismatch. (B) Comparison of the response strength to 6DOF and 4DOF playback halts. Average response strength was calculated within a 100 ms analysis window centered on the peak of the playback halt response in the 6DOF condition. Data were collected from four participants; one of them participated in two separate recording sessions. Each circle represents data from individual recording session. Boxes mark median, quartiles, and range of data not considered outliers. n.s.: not significant.

Mismatch and playback halt responses obtained from the same participants.

(A) Responses to visuomotor mismatches recorded from occipital electrodes (O1 and O2). Solid black line represents the mean, and shading indicates the SEM across participants. The gray shaded areas mark the analysis windows used to quantify response strength in C. Dashed vertical red lines are onset and offset of the visuomotor mismatch. As in Figure 2E, F, but only including data from participants for which we have both closed and open loop data. (B) As in A, but for visual flow playback halt responses recorded from occipital electrodes. (C) Comparison of the response strength to visuomotor mismatch and playback halts (23 participants 4DOF and 3 participants 6DOF). Boxes mark median, quartiles, and range of data not considered outliers. Each data point corresponds to one participant and lines connect mismatch and playback halt responses from the same participant. ***: p<0.001. One data point not shown (Mismatch response, at −9.5 μV). See Table S1 for all statistical information.

Examples of rejected and valid trials based on maximum signal amplitude in the visuomotor mismatch paradigm.

(A) Example of an EEG response contaminated by an eye blink artifact (arrow). Dashed vertical red lines are onset and offset of the visuomotor mismatch. This trial was removed. (B, C) As in A, but for examples of EEG responses contaminated by walking artifacts. These trials were removed. (D, E) As in A, but for an example of an EEG response that met the inclusion criteria (amplitude < 100 μV). (F) Histogram of maximum trial amplitudes. The red dashed line marks the threshold for inclusion. 46 trials with amplitudes exceeding 1500 μV are not shown.

Statistics All information on statistical tests used in this manuscript.

We used hierarchical bootstrap (Saravanan et al., 2020) for all comparisons. We had a total of 46 participants, the numbers in the table indicate the subset of these we could include for each analysis. Note, this differs for electrode location and condition. Exclusion reasons were a) recording too noisy, or b) participant aborted the recording (in the case of playback).

Key Resources Table