Neural signatures of vigilance decrements predict behavioural errors before they occur
- Perception in Action Research Centre, Faculty of Human Sciences, Macquarie University, Australia
- Medical Research Council Cognition and Brain Sciences Unit, University of Cambridge, United Kingdom
- Department of Cognitive Science, Faculty of Human Sciences, Macquarie University, Australia
Figures
Figure 1
The multiple-object monitoring (MOM) task and types of information decoded.
(A) At the start of a block, the relevant colour is cued (here, green; distractors in red). Over the on-task period (~30 min per task condition), multiple dots entered from either direction, each moving along a visible individual trajectory towards the middle object. Only attended dots that failed to deflect along the trajectories at the deflection point required a response (Target: bottom right display). Participants did not need to press the button for the unattended dot (Distractor: top right display) or the dots that kept moving on the trajectories (Event: middle right panel). Each dot took ~1226 ms from appearance to deflection. (B) Direction of approach information (left display: left vs. right as indicated by dashed and solid lines, respectively) and distance to object information (right display). Note the blue dashed lines and orange arrows were not present in the actual display. d1, d2, etc. denote the ‘distance units’ used to train the classifier for the key distance to object information. A demo of the task can be found here [https://osf.io/5aw8v/].
Figure 2
Behavioural performance on the MOM task.
The percentage of miss trials (A), and correct reaction times (B), as a function of block. Thick lines show the average across participants (shading 95% confidence intervals) for Active (blue) and Monitoring (red) conditions. Each block lasted for 110 s and had either 16 (Active) or 2 (Monitoring) targets out of 32 cued-colour and 32 non-cued colour dots. Bayes factors (BF) are shown in the bottom section of each graph: Filled circles show moderate/strong evidence for either hypothesis and empty circles indicate insufficient evidence when evaluating the contrast between Active and Monitoring conditions.
Figure 3 with 3 supplements
Impact of different conditions and their interactions on information on correct trials (all trials except those in which a target was missed or there was a false alarm).
(A) Decoding of direction of approach information (less task-relevant) and (B) decoding of distance to object information (most task-relevant). Left two columns: Attended dots; Right two columns: Unattended (‘distractor’) dots. Thick lines show the average across participants (shading 95% confidence intervals). Horizontal dashed line refers to theoretical chance-level decoding (50%). Vertical dashed lines indicate critical times in the trial. Bottom panels: Bayesian evidence for main effects and interactions, Bayes factors (BF): Filled circles show moderate/strong evidence for either hypothesis and empty circles indicate insufficient evidence. Main effects and interactions of conditions calculated using BF ANOVA analysis. Cyan, purple, green, and red dots indicate the main effects of Attention, Target frequency, Time on Task, and the interaction between Target frequency and Time on Task, respectively. The results of BF analysis (i.e., the main effects of the three conditions and their interactions) are from the same three-way ANOVA analysis and are therefore identical for attended and unattended panels. Early = data from the first five blocks (~10 min). Late = data from the last five blocks (~10 min). Note the different scales of the BF panels, and the down-sampling, for clearer illustration.
Figure 3—figure supplement 1
Impact of different conditions in the direction of approach and distance to object information coding and their Bayesian evidence for difference from chance.
Two left graphs: Attended dots; Two right graphs: Unattended (‘distractor’) dots. (A) Decoding of direction of approach information (less task-relevant). (B) Decoding of distance to object information (most task-relevant). Thick lines show the average across participants (shading 95% confidence intervals). The horizontal dashed lines refer to chance-level decoding. Vertical dashed lines indicate critical times in the trial. Bayes factors (BF) are shown in the bottom section of each graph: Filled circles show moderate/strong evidence for either hypothesis and empty circles indicate insufficient evidence. They show the results of BF analysis when evaluating the difference of the decoding values from chance as explained in Materials and methods. Early = data from the first five blocks (~10 min). Late = data from the last five blocks (~10 min). Note the different scales of the BF panels, and the down-sampling, for clearer illustration.
Figure 3—figure supplement 2
Impact of different conditions and their interactions on information processing on correct trials (all trials except those in which a target was missed or there was a false alarm) without and with standard eye-artefact removal.
Left and right panels show the results without (repeated from Figure 3) and with eye-standard eye-artefact removal, respectively. (A) Decoding of direction of approach information (less task-relevant) and (B) decoding of distance to object information (most task-relevant). Left two columns: Attended dots; Right two columns: Unattended (‘distractor’) dots. Thick lines show the average across participants (shading 95% confidence intervals). Horizontal dashed line refers to theoretical chance-level decoding (50%). Vertical dashed lines indicate critical times in the trial. Bottom panels: Bayesian evidence for main effects and interactions, Bayes factors (BF): Filled circles show moderate/strong evidence for either hypothesis and empty circles indicate insufficient evidence. Main effects and interactions of conditions calculated using BF ANOVA analysis. Cyan, purple, green, and red dots indicate the main effects of Attention, Target frequency, Time on Task, and the interaction between Target frequency and Time on Task, respectively. The results of BF analysis (i.e., the main effects of the three conditions and their interactions) are from the same three-way ANOVA analysis and are therefore identical for attended and unattended panels. Early = data from the first five blocks (~10 min). Late = data from the last five blocks (~10 min). Note the different scales of the BF panels, and the down-sampling, for clearer illustration.
Figure 3—figure supplement 3
The statistical relationship between the proportion of eye-related measures and Target Frequency, Attention, and Time on the task factors.
(A) Eye-related measures. The error bars show 95% confidence interval around the mean across participants. (B) Bayes factor evidence for the main effects and interactions between factors and eye-related measures.
Figure 4
Relationship between informational connectivity and Attention, Target Frequency, Time on Task, and the behavioural outcome of the trial (i.e., correct vs. miss).
(A) Calculation of connectivity using Spearman’s rank correlation between RDMs obtained from the peri-frontal and peri-occipital sensors as indicated by coloured boxes, respectively. RDMs include decoding accuracies obtained from testing the 105 classifiers trained to discriminate different distance to object categories. (B) Connectivity values for the eight different conditions of the task and the results of three-way Bayes factor ANOVA with factors Time on Task (Early, Late), Attention (Attended, Unattended), and Target Frequency (Active, Monitoring), using only correct trials. (C) Connectivity values for the Active and Monitoring, Early and Late blocks of each task for correct and miss trials (attended condition only), and the result of Bayes factor ANOVA with factors Target Frequency (Active, Monitoring), Time on Task (Early, Late), and behavioural outcome (correct, miss) as inputs. Number of trials are equalised across conditions in B and C separately. Bars show the average across participants (error bars 95% confidence intervals). Bold fonts indicate moderate or strong evidence for either the effect or the null hypothesis.
Figure 5 with 2 supplements
Decoding of information on correct vs miss trials.
(A) Decoding of direction of approach information (less task-relevant). (B) Decoding of distance to object information (most task-relevant). The horizontal dashed lines refer to theoretical chance-level decoding. Left panels: Decoding using correct trials; Right panels: Decoding using miss trials. In both right and left panels, the classifiers were trained on correct trials and tested on (left out) correct and all miss trials, respectively. Thick lines show the average across participants (shading 95% confidence intervals). Vertical dashed lines indicate critical events in the trial. Bayes factors (BF) are shown in the bottom section of each graph: Filled circles show moderate/strong evidence for either hypothesis and empty circles indicate insufficient evidence. They show the results of BF analysis when evaluating the difference of the decoding values from chance for Active (blue) and Monitoring (red) conditions separately, the comparison of the two conditions (green), and the comparison of correct and miss trials (black). Note that for the comparison of correct and miss trials, Active and Monitoring conditions were averaged separately. Note the different scales of the BF panels, and down-sampling, for clearer illustration.
Figure 5—figure supplement 1
Distribution of decoding accuracies for every individual correct and miss trial in the Active and Monitoring conditions for all 21 subjects.
A and B show the data for correct and miss trials, respectively. The percentages show the percentage of trials for which there was strong (BF >10) evidence for above-chance decoding in Active and Monitoring conditions. (C) Difference between the decoding accuracies in the correct (A) and miss (B) trials as evaluated using Cohen’s d.
Figure 5—figure supplement 2
Decoding of information on correct vs false alarm trials.
(A) Decoding of direction of approach information (less task-relevant). (B) Decoding of distance to object information (most task-relevant). The horizontal dashed lines refer to theoretical chance-level decoding. Left panels: Decoding using correct trials; Right panels: Decoding using false alarm trials. In both right and left panels, the classifiers were trained on correct trials and tested on (left out) correct and all false alarm trials, respectively. Thick lines show the average across participants (shading 95% confidence intervals). Vertical dashed lines indicate critical events in the trial. Bayes factors (BF) are shown in the bottom section of each graph: Filled circles show moderate/strong evidence for either hypothesis and empty circles indicate insufficient evidence. They show the results of BF analysis when evaluating the difference of the decoding values from chance for Active (blue) and Monitoring (red) conditions separately, the comparison of the two conditions (green) and the comparison of correct and false alarm trials (black). Note that for the comparison of correct and miss trials, Active and Monitoring conditions were averaged separately. Note the different scales of the BF panels, and the down-sampling, for clearer illustration.
Figure 6 with 1 supplement
Prediction of behavioural outcome (correct vs miss) trial-by-trial using decoding of distance to object information.
(A) Sample classifiers’ accuracies (correct or incorrect classification of current distance as indicated by colours) for a miss (left panel; average accuracy ≅ 43% when the dot reached the deflection point) and a correct trial (right panel; average accuracy ≅ 67% at the deflection point). The classifiers were trained on the data from correct trials and tested on the data from correct and miss trials. For the miss trials, around half the classifiers categorised the dot’s distance incorrectly by the time it reached the deflection point. (B) Accumulation of classifiers’ accuracies over decreasing dot distances/time to deflection. This shows stronger information coding of the crucial distance to object information on the correct trials over miss trials. A variable threshold used in (C) is shown as a green dashed line. (C) Prediction of behavioural outcome as a function of threshold and distance using a second-level behavioural outcome classification. Results show highest prediction accuracies on the participant set at around the threshold of 1.5SD under the decoding level for correct validation trials, increasing at closer distances. (D) Accuracy of predicting behavioural outcome for the left-out participant using the threshold obtained from all the other participants as function of distance/time from the deflection point. Results showed successful (~=70%) prediction of behavioural outcome of the trial as early as 80 ms after stimulus appearance. Thick lines and shading refer to average and one standard deviation around the mean across participants, respectively. Bayes factors (BF) are shown in the bottom section of each graph: Filled circles show moderate/strong evidence for either hypothesis and empty circles indicate insufficient evidence (black dots under B and D).
Figure 6—figure supplement 1
Accuracy of predicting behavioural outcome of trials in the early (first 5) vs late (last 5) blocks of trials before eye-blink removal.
Left column shows the result for the early and right shows the result for the late blocks. (A) Result of classifier accumulation of decoding accuracy. (B) Prediction accuracy and (C) same results (as in B) overlapping on the same plot with Bayes factor (BF) analysis of the difference. The details of this figure are identical to Figure 6B and D. BF in A and C show the evidence for and against the alternative of difference between the curves and the BFs in B reflect the evidence for and against the alternative for difference from chance.
Author response image 1
The accuracy of predicting behavioral outcome of trials without and with eye artefact removal.
The results are for the left-out participant (averaged over all participants) using the threshold obtained from all the other participants as function of distance/time from the deflection point. Figure 6D shows the result without eye artefact removal and Author response image 1 with eye artefact removal. Thick lines and shading refer to average and one standard deviation around the mean across participants, respectively. Bayes Factors are shown in the bottom section of each graph: Filled circles show moderate/strong evidence for either hypothesis and empty circles indicate insufficient evidence.
Tables
Author response table 1
Connectivity (correlation) matrix obtained from four sample areas.
| AREA | A | B | C | D |
|---|---|---|---|---|
| A | 1 | 0.4 | 0.8 | 1 |
| B | 0.4 | 1 | 0 | 0.4 |
| C | 0.8 | 0 | 1 | 0.8 |
| D | 1 | 0.4 | 0.8 | 1 |
Additional files
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Neural signatures of vigilance decrements predict behavioural errors before they occur
eLife 10:e60563.
https://doi.org/10.7554/eLife.60563
