Distinct neural mechanisms underlying perceptual and attentional impairments of conscious access

Samuel Noorman; Timo Stein; Johannes J Fahrenfort; Simon van Gaal

doi:10.7554/eLife.97900.1

eLife assessment

This important study provides new insights into the mechanisms that underlie perceptual and attentional impairments of conscious access. The paper presents convincing evidence of a dissociation between the early stages of low-level perception, which are impermeable to perceptual or attentional impairments, and subsequent stages of visual integration which are susceptible to perceptual impairment but resilient to attentional manipulations. This study will be of interest to scientists working on visual perception and consciousness.

https://doi.org/10.7554/eLife.97900.1.sa3

Significance of findings

important: Findings that have theoretical or practical implications beyond a single subfield

landmark
fundamental
important
valuable
useful

Strength of evidence

convincing: Appropriate and validated methodology in line with current state-of-the-art

exceptional
compelling
convincing
solid
incomplete
inadequate

During the peer-review process the editor and reviewers write an eLife assessment that summarises the significance of the findings reported in the article (on a scale ranging from landmark to useful) and the strength of the evidence (on a scale ranging from exceptional to inadequate). Learn more about eLife assessments

Abstract

This study investigates failures in conscious access resulting from either weak sensory input (perceptual blindness) or unattended input (attentional blindness). Participants viewed a Kanizsa stimulus with or without an illusory triangle within a rapid serial visual presentation of distractor stimuli. We designed a novel Kanizsa stimulus that contained additional ancillary features of different complexity (local contrast and collinearity) that were independently manipulated. Perceptual performance on the Kanizsa stimulus (presence vs. absence of an illusion) was equated between the perceptual (masking) and attentional (attentional blink) manipulation to circumvent common confounds related to conditional differences in task performance. We trained and tested classifiers on electroencephalogram (EEG) data to reflect the processing of specific stimulus features, with increasing levels of complexity. We show that late stages of processing (∼200-250 ms), reflecting the integration of complex stimulus features (collinearity, illusory triangle), were impaired by masking but spared by the attentional blink. In contrast, decoding of local contrast (the spatial arrangement of stimulus features) was observed early in time (∼80 ms) and was left largely unaffected by either manipulation. These results replicate previous work showing that feedforward processing is largely preserved under both perceptual and attentional blindness. Crucially however, under matched levels of performance, only attentional blindness left the processing of more complex visual features relatively intact, likely related to spared lateral and local feedback processes during inattention. These findings reveal distinct neural mechanisms associated with perceptual and attentional blindness and thus contribute to a comprehensive understanding of distinct neural stages leading to conscious access.

Introduction

Conscious access to sensory input can be impaired in two distinct ways (Dehaene et al., 2006; Lamme, 2010; Mashour et al., 2020; Northoff & Lamme, 2020). Sensory input may lack sufficient bottom-up strength, or top-down attention may be directed elsewhere. Despite both cases resulting in a failure to perceive a stimulus, their underlying neural mechanisms are thought to be remarkably different. Influential theories of consciousness such as global neuronal workspace and recurrent processing theory propose four stages of neural information processing associated with distinct levels of bottom-up signal strength and top-down attention. These four stages can be investigated empirically by crossing “perceptual” manipulations that degrade the strength of sensory input (e.g., reducing stimulus contrast, masking, continuous flash suppression) with “attentional” manipulations that affect top-down attention (e.g., attentional blink, inattentional blindness, Fig. 1A).

Experimental design and behavior.
(A) Perceptual vs. attentional blindness in the four-stage model. A stimulus with low bottom-up strength (masked) is thought to interrupt local recurrent processing in sensory areas while leaving feedforward processing largely intact, while inattention (induced by the attentional blink) is thought to interrupt global recurrent processing between frontoparietal areas and sensory areas, while leaving local recurrent processing within sensory areas largely intact. Reprinted from Dehaene et al. (2006) with permission from Elsevier. (B) Target stimulus set and decoding analyses. (C) Trial design. (D) Perceptual performance refers to participants’ ability to detect the Kanizsa illusion. Metacognition refers to participants’ ability to evaluate their own performance using confidence judgments. Both perceptual performance and metacognition are measured as the area under the receiver operating characteristic curve (AUC). Error bars are mean ± standard error of the mean. Individual data points are plotted using low contrast. Ns is not significant (P≥0.477, BF₀₁≥4.05). *P≤0.001.

According to these theoretical models, all stimuli elicit feedforward information transfer from lower- to higher-level brain regions (Fig. 1A, bottom row), but recurrent interactions are initiated only for stimuli with sufficient bottom-up strength (Fig. 1A, top row). If stimuli are sufficiently strong and top-down attention is available, neural processing crosses a threshold, triggering a process termed global ignition, facilitating widespread recurrent interactions between frontal, parietal and sensory cortices, yielding conscious access (Fig. 1A, top left). Crucially, when top-down attention is lacking, frontoparietal network ignition is prevented, while local recurrent interactions within sensory brain regions remain relatively intact (“attentional blindness”, Fig. 1A, top right) (Dehaene et al., 2003; Marti et al., 2015; Sergent et al., 2005; Zivony & Lamy, 2022). Weak stimuli result in the absence, or a substantial reduction, of local recurrent interactions (“perceptual blindness”, Fig. 1A, bottom left) (Fahrenfort et al., 2007; Joglekar et al., 2018; van Gaal et al., 2008; van Vugt et al., 2018).

Although this framework is at the heart of influential theories of consciousness, the four stages of the model and their underlying neural mechanisms have rarely been investigated simultaneously within the same study (for an exception see Fahrenfort et al., 2017). One challenge with comparing results across different studies, or even within a study, is that perceptual manipulations tend to impair overall task performance more than attentional manipulations, so that it may not be surprising to find that perceptual manipulations interrupt recurrent interactions to a greater extent than attentional manipulations. Given the right parameter settings, perceptual manipulations can be used to induce chance-level performance, while it is not possible to use attentional manipulations to drive behavioral performance down to chance, even when they are optimized fully. For this reason, attentional manipulations are often combined with post-hoc selection of a subset of “blind” trials (e.g., attentional blink) or subjects (e.g., inattentional blindness), a methodologically questionable practice that introduces criterion confounds, sampling bias, and underestimates consciousness (Peters & Lau, 2016; T. Schmidt, 2015; Shanks, 2017). Thus, when comparing perceptual to attentional manipulations, any (neural) effect could reflect differences in task performance rather than genuine differences between manipulations and hence stages in the model depicted in Figure 1A. While these issues with comparing conditions that differ in task performance in consciousness research have been acknowledged (Lau, 2022), they have rarely been addressed experimentally (Kanai et al., 2010; Lau & Passingham, 2006; Meuwese et al., 2014). To test and further refine the four-stage model of consciousness in humans, we compared all stages within the same experimental setup, matching task performance between a perceptual manipulation (masking) and an attentional manipulation (attentional blink).

To reveal neural mechanisms associated with perceptual and attentional blindness, we combined a novel visual stimulus with features of different complexity with time-resolved decoding of these visual features from electroencephalogram (EEG) data (Fig. 1B). The target stimulus differed along three dimensions (illusory triangle, non-illusory triangle, and local contrast) that were independently manipulated. First, “Pac-Man” stimuli could create either the perception of an illusory surface in the shape of a Kanizsa triangle when aligned, or not, when misaligned. Second, additional “two-legged white circles” could form either a non-illusory triangle when their line segments were aligned, or not, when the legs were misaligned. Third, for the local contrast manipulation, the whole stimulus was rotated by 180 degrees, so that the same retinotopic positions had high contrast in one spatial configuration and low contrast when flipped 180 degrees. Direct neural recordings in animals have shown that the Kanizsa illusion is supported by both lateral and feedback connections (Halgren et al., 2003; Kok et al., 2016; Kok & de Lange, 2014; Lee & Nguyen, 2001; Pak et al., 2020; Wokke et al., 2013), while the processing of collinear line elements (as in the non-illusory triangle) primarily relies on lateral connections (Bosking et al., 1997; Gilbert & Wiesel, 1979; Li, 1998; Liang et al., 2017; K. E. Schmidt et al., 1997; Stettler et al., 2002). Differences in local contrast are processed early in the visual system through feedforward connections and are resistant to masking (Fahrenfort et al., 2007, 2017; Kandel et al., 2000; Lamme & Roelfsema, 2000). Decoding the stimulus conditions of the illusory triangle, non-illusory triangle, and local contrast at different points in time, in combination with the associated topography, therefore served as putative markers of these distinct neural processes (Fig. 1B) and allowed us to test whether the effects of masking and the attentional blink followed the predictions of the four-stage model of consciousness.

Results

Masking and the attentional blink were matched in perceptual performance and metacognition

We recorded the EEG signal of 30 participants who identified the presence or absence of an illusory surface (triangle) in two black target stimuli (T1 and T2) that were presented amongst red distractors in a rapid serial visual presentation task (Fig. 1C). We manipulated the visibility of T2 in two ways: masking the stimulus and manipulating attention, resulting in a 2×2 factorial design (Fig. 1A). Specifically, T2 could be either masked or unmasked (perceptual manipulation), and T2 could be presented at either a long interval (900 ms) or a short interval (200 or 300 ms) after T1, inducing an attentional blink (AB) effect for the short T1-T2 intervals (Raymond et al., 1992). This design resulted in four conditions, which we from now on refer to as the masked condition (T2 masked at the long T1-T2 interval), AB condition (T2 unmasked at the short T1-T2 interval), no manipulations condition (T2 unmasked at the long T1-T2 interval), and both manipulations condition (T2 masked at the short T1-T2 interval). At the end of a trial, participants indicated whether each target (T1 and T2) contained an illusory surface or not. Importantly, mask contrast in the masked condition was individually adjusted using a staircasing procedure to match participants’ performance in the AB condition, ensuring comparable perceptual performance in the masked and the AB condition (see Methods for more details).

Conscious access can be assessed not only based on perceptual performance but also through metacognitive sensitivity, the ability to evaluate one’s own performance (Brown et al., 2019; Dienes, 2007; Fleming & Lau, 2014; Lau & Passingham, 2006; Merikle et al., 2001; Seth et al., 2008). Participants in our study provided confidence ratings on a 3-point scale (low, medium, high) for their responses to T2. To ensure that the distribution of confidence ratings was not influenced by overall differences in perceptual performance between conditions, conditions that were matched in perceptual performance (masked and AB condition) were presented in the same experimental block, while the other block type included the unmatched conditions (no and both manipulations condition).

We used area under the receiver operating characteristic (ROC) curve (AUC) as a shared metric for perceptual performance (detection of the Kanizsa illusion), metacognitive sensitivity, and EEG decoding (see Methods for details on the calculation of these measures). Repeated-measures (rm) ANOVA with the factors masking (present/absent) and T1-T2 lag (short/long) revealed, as expected, that both masking and the short T1-T2 lag impaired perceptual performance (F_1,29=344.24, P<10⁻¹⁵ and F_1,29=427.54, P<10⁻¹⁵) as well as metacognitive sensitivity (F_1,29=50.78, P<10⁻⁷ and F_1,29=47.83, P<10⁻⁶). Importantly, paired t-tests showed that we successfully matched the key conditions, the masked condition (masked, long lag) and the AB condition (unmasked, short lag) for perceptual performance (t₂₉=0.62, P=0.537, BF₀₁=4.30, Fig. 1D, left) as well as for metacognitive sensitivity (t₂₉=0.72, P=0.477, BF₀₁=4.05; Fig. 1D, right, see Fig. S1 for signal detection theory related measures of performance). Thus, the two performance matched conditions were indistinguishable from each other in both measures of conscious access.

Masking and the attentional blink leave local contrast decoding largely intact

To derive our markers of the different neural processes from our EEG data, for each stimulus feature we trained linear discriminant classifiers on the T1 data and tested them on the T2 data. Classifiers used raw EEG activity across all electrodes. To leverage the similarities between T1 and T2 in task and stimulus context, all main analyses used T1 training data for T2 decoding. This approach minimized possible differences in conscious access and working memory demands between the training and test datasets.

For local contrast decoding, the classifier categorized stimuli as either pointing upwards or pointing downwards, thereby effectively decoding the stimuli’s local differences in contrast at the top vs. bottom of the stimulus. Classification performance (AUC) over time was obtained, with peak decoding accuracy in a 75-95 ms time window (Fig. 2A and Fig. S2A, top). The peak in decoding accuracy was occipital in nature (see the covariance/class separability map of Fig. 2A) (Haufe et al., 2014), consistent with our previous findings (Fahrenfort et al., 2017). We focused our analyses on the averages of this time window. An rm ANOVA with the factors masking (present/absent) and T1-T2 lag (short/long) revealed only a marginal effect of masking on local contrast decoding (F_1,29=6.51, P=0.016), while the T1-T2 lag had no significant effect (F_1,29=0.32, P=0.578). A paired t-test yielded no evidence for a difference between the performance matched conditions (masked vs. AB; t₂₉=1.42, P=0.166, BF₀₁=2.08; Fig. 2C, “Local contrast: 75-95 ms”). These results are in line with theoretical proposals and empirical findings that suggest limited effects of masking and attentional manipulations on perceptual processes that rely on feedforward connections (Dehaene et al., 2006; Fahrenfort et al., 2007, 2017; Lamme, 2010).

Local contrast and illusory triangle decoding using first targets as training data.
(A) Local contrast decoding. (B) Illusory Kanizsa triangle decoding. For both features, covariance/class separability maps reflecting underlying neural sources are shown. Below these maps: mean decoding performance, area under the receiver operating characteristic curve (AUC), over time ± standard error of the mean (SEM). Thick lines differ from chance: P<0.05, cluster-based permutation test. (C) Normalized (Z-scored) AUC for every measure: mean decoding time windows and two types of behavior. Each measure is Z-scored separately. Perceptual performance refers to participants’ ability to detect the Kanizsa illusion. Metacognition refers to participants’ ability to evaluate their own performance using confidence judgments. See Figure S3 for the same analyses but then for off-diagonal decoding profiles. Error bars are mean ± SEM. Individual data points are plotted using low contrast. Ns is not significant (P≥0.166, BF₀₁≥2.07). *P≤0.002.

Stronger effect of masking than the attentional blink on early but not on late illusion decoding

Next, we trained a linear classifier on the T1 data to discriminate between the absence and presence of the Kanizsa illusion and tested it on each of the four conditions of the T2 data. The average of all four conditions revealed two prominent peaks in decoding accuracy, consistent with previous research (Fig. S2C, top) (Fahrenfort et al., 2017). Based on this previous study, our analyses focused on the averages of the two time windows that encompassed these two peaks: specifically, from 200 to 250 ms and from 375 to 475 ms after target stimulus onset. The covariance/class separability maps (Fig. 2B) indicated that during the earlier time window (200– 250 ms) classification mainly relied on occipital electrodes. Considering its timing, topology and previous findings, this neural event likely reflects early sensory processes and may thus represent a marker for local recurrent processing (Fahrenfort et al., 2017; Kok et al., 2016; Roelfsema, 2006; Wokke et al., 2013; Wyatte et al., 2014). The timing and topology of the later neural event (375–475 ms) overlapped with the event-related potential component P300, which is associated with conscious access (Fahrenfort et al., 2017; Sergent et al., 2005; Weaver et al., 2019) and may thus represent a marker for global recurrent processing.

We tested how the consciousness manipulations affected these putative markers of local and global recurrent processing (Fig. 2C), again conducting rm ANOVAs with the factors masking (present/absent) and T1-T2 lag (short/long) that we followed-up on with paired t-tests comparing the matched conditions (masked vs. AB). Importantly, we observed a distinct difference between the performance matched conditions in the first decoding peak, which was significantly impaired by both masking (F_1,29=162.62, P<10⁻¹²) and T1-T2 lag (F_1,29=78.07, P<10⁻ ⁸), but importantly, it was more affected by masking than T1-T2 lag (F_1,29=18.67, P<0.001) (Fig. 2C, “Illusory triangle: 200-250 ms”). Directly comparing the performance matched conditions, the first decoding peak was more strongly impaired by masking than by the AB (t₂₉=4.66, P<10⁻ ⁴, BF₀₁=0.003).

The pattern of results of the second peak was notably different. The second decoding peak was impaired by both masking (F_1,29=49.75, P<10⁻⁷) and T1-T2 lag (F_1,29=78.48, P<10⁻⁹), and the matched conditions (masked and AB condition) did not differ significantly from each other (t₂₉=0.21, P=0.837, BF₀₁=5.04) (Fig. 2C, “Illusory triangle: 375-475 ms”). Furthermore, another rm ANOVA comparing the first and second decoding peak between the matched conditions (masked/AB) revealed a significant interaction, reflecting a larger difference between the AB and the masked condition in the first than in the second decoding peak (F_1,29=31.53, P<10⁻⁵). Across the four conditions, the pattern of behavioral results, both for perceptual performance and metacognitive sensitivity, closely resembled the second decoding peak (Fig. 2C), indicating that global recurrent processing reflected conscious access to the Kanizsa illusion.

Additional rm ANOVAs comparing the effect of the consciousness manipulations on local contrast decoding with their effects on the first and second illusion decoding peak showed that, compared to contrast decoding, both manipulations had stronger effects on the first illusion decoding peak (masking: F_1,29=99.35, P<10⁻¹⁰; AB: F_1,29=38.95, P<10⁻⁶) and on the second illusion decoding peak (masking: F_1,29=22.25, P<10⁻⁴; AB: F_1,29=49.60, P<10⁻⁷). This suggests that masking and the AB specifically influenced local and global recurrent processing respectively, while early feedforward processing was less affected.

Distinguishing collinearity and illusion-specific processing

The performance matched masked and AB condition differed only in the first illusion decoding peak, our putative marker of local recurrent processing, which was markedly more impaired in the masked than the AB condition. Next, we determined whether this effect reflected relatively preserved processing of collinearity or the Kanizsa illusion during attentional blindness. In our target stimulus, collinearity was present when the Pac-Man stimuli aligned, inducing the illusory Kanizsa triangle. Notably, collinearity was also present when the line segments of the “two-legged white circles” of the stimulus aligned, forming the non-illusory triangle. Note that the line segments making up the triangle were equally long, and the spaces between them equally large, for the illusory and non-illusory triangles. Thus, by comparing non-illusory triangle decoding to illusory triangle decoding we distinguished between collinearity and illusion-specific processing.

However, in the main RSVP task the illusory triangle was task-relevant, while non-illusory triangles were always task-irrelevant. To equate the effect of task-relevance in the comparison, classifiers were trained on an independent training set in which each relevant stimulus feature was task-relevant. Specifically, in different experimental blocks, participants focused either on local contrast, the non-illusory triangle, or the illusory triangle (Fig. S4). We trained a classifier to distinguish between the presence and absence of the task-relevant non-illusory triangle (collinearity-only) and the same was done for the task-relevant illusory triangle (collinearity-plus-illusion). Then, both classifiers were used to decode the presence vs. absence of the illusory triangle in the main RSVP task (cross-task-decoding approach), which ensured that both training and testing were always performed on task-relevant stimuli. By comparing Kanizsa decoding performance in the RSVP task based on the collinearity-only classifier with decoding performance based on the collinearity-plus-illusion classifier, we effectively subtracted out the contribution of collinearity processing to illusion decoding, thereby isolating illusion-specific processing.

Preserved collinearity and illusion-specific processing during the attentional blink

To determine a time window for (the start of) collinearity-only processing, we first trained and tested classifiers to distinguish present vs. absent non-illusory triangles (training and testing on non-illusory triangles only). We trained two classifiers, one on the T1 in the RSVP task and one on the independent training set and tested their performance in decoding the non-illusory triangle in the T2 data, where this non-illusory triangle was always task-irrelevant. The results of both classifiers converged and these analyses revealed a peak in decoding accuracy at ∼164 ms, right before the 200-250 ms time window of the first illusion decoding peak (Fig. S2B). This peak was also evident when these classifiers were used to categorize the presence vs. absence of the Kanizsa illusion (Fig. S2C, first time window), as well as in previous research (Fahrenfort et al., 2017), suggesting that collinearity processing also contributes to Kanizsa decoding.

We examined how decoding the presence vs. absence of the Kanizsa illusion in the RSVP task was affected by the consciousness manipulations, while training classifiers either on the illusory (collinearity-plus-illusion) or non-illusory (collinearity-only) triangle from the independent training set. Figure 3A shows the decoding accuracies of these analyses across the entire time window (purple and green lines). Follow-up analyses were performed using the 140-190 ms window, the peak of collinearity-only processing (see the previous paragraph). An rm ANOVA with the factors masking (present/absent), T1-T2 lag (short/long), and training set (illusory/non-illusory triangle) revealed that both masking and the short T1-T2 lag impaired decoding accuracy (masking: F_1,29=58.95, P<10⁻⁷; T1-T2 lag: F_1,29=24.90, P<10⁻⁴). Furthermore, paired t-tests comparing the matched conditions (masked vs. AB condition) confirmed that decoding accuracy was more impaired in the masked than AB condition, both when training was done on the illusory (t₂₉=2.26, P=0.031, BF₀₁=0.58) and non-illusory triangle (t₂₉=2.78, P=0.009, BF₀₁=0.21; Fig. 3B, “140-190 ms”). Focusing on the performance matched conditions and the role of the training set, an rm ANOVA with the factors condition (masked/AB) and training set (illusory/non-illusory triangle) on T2 illusory triangle decoding revealed no significant effect of training set (F_1,29=0.09, P=0.766), i.e., no evidence for illusion-specific processing, and no significant interaction (F_1,29=0.04, P=0.837). This demonstrates that neural processing in the 140-190 ms window indeed reflects collinearity-only rather than illusion-specific processing.

Separating collinearity and illusion-specific processes using the independent training dataset.
(A) Illusory triangle decoding, after training classifiers on the independent training set on either the non-illusory (collinearity-only, purple lines) or illusory triangle (collinearity-plus-illusion, green lines). For comparison, training and testing on local contrast is shown in light blue. Mean decoding performance, area under the receiver operating characteristic curve (AUC), over time ± standard error of the mean (SEM) is shown. Thick lines differ from chance: P<0.05, cluster-based permutation test. The highlighted time windows are 75-95, 140-190, 200-250, and 375-475 ms, corresponding to separate panels in (B), which shows normalized (Z-scored) mean AUC for every time window. Each window is Z-scored separately. Error bars are mean ± SEM. Individual data points are plotted using low contrast. Ns is not significant (P≥0.084, BF₀₁≥1.26). *P≤0.048.

The marker for illusion-specific processing emerged later, namely in the 200-250 ms time window that encompassed the first illusion decoding peak reported above. As can be seen in Figure 3A, when no consciousness manipulations were applied (unmasked, long T1-T2 lag), there was significant illusion-specific processing, i.e., T2 illusory triangle decoding was better after training a classifier on the Kanizsa illusion (collinearity-plus-illusion, green line) than after training a classifier on the non-illusory triangle (collinearity-only, purple line) (t₂₉=4.22, P<0.001, BF₀₁=0.008). Turning to the performance matched conditions, an rm ANOVA with the factors condition (masked/AB) and training set (illusory/non-illusory triangle) on T2 illusory triangle decoding yielded a significant effect of condition (F_1,29=16.59, P<0.001), with overall better decoding for the AB than for the masked condition, and importantly, a significant interaction (F_1,29=4.65, P=0.039). Figure 3A shows that decoding after training on illusory triangles (collinearity-plus-illusion) was better than after training on non-illusory triangles (collinearity-only) for the AB (t₂₉=2.51, P=0.018, BF₀₁=0.36, Fig. 3A, top right) but not for the masked condition (t₂₉=-0.02, P=0.982, BF₀₁=5.14, Fig. 3A, bottom left). Thus, while illusion-specific processing was evident in the AB condition, it was fully abolished in the masked condition. Illusion-specific processing was not even affected by the AB, as an rm ANOVA with the factors condition (no manipulations/AB) and training set (illusory/non-illusory triangle) revealed no significant interaction (F_1,29=1.33, P=0.259). The classifier trained on non-illusory triangles (collinearity-only) also performed better during the AB than masking (t₂₉=2.85, P=0.008, BF₀₁=0.18; Fig. 3B, “200-250 ms”, purple line), hence both collinearity-only and illusion-specific processing were most strongly impaired by masking. Control analyses presented in the Supplementary information (Fig. S6) demonstrate that cross-feature-decoding can indeed isolate illusion-specific processes and does not reflect other, e.g., task- or attention-related, processes (for further control analyses testing the effect of task-relevance on local contrast processing, see Supplementary Information and Fig. S7).

Finally, we focused on the late 375-475 ms window, encompassing the second illusion decoding peak, which was directly linked to behavioral performance (see above, Fig. 3A, last time window). Similarly as above, illusory triangle decoding was now based on training the decoder on the illusory triangles from the independent training set. Replicating our main analysis, classifier performance was impaired by both masking (F_1,29=6.01, P=0.020) and T1-T2 lag (F_1,29=10.10, P=0.004), with no significant differences between the two performance matched conditions (t₂₉=-0.63, P=0.531, BF₀₁=4.27) (Fig. 3B, “375-475 ms”).

Discussion

We demonstrate that perceptual and attentional manipulations, despite similarly impairing conscious access, exhibit distinct neural profiles in the brain. To investigate this difference, we decoded different visual features targeting distinct stages of visual processing from human EEG activity, while carefully matching a masked condition and an attentional blink (AB) condition in perceptual and metacognitive performance. While decoding of local contrast was barely affected by the two consciousness manipulations, early (200–250 ms, occipital) decoding of the illusory Kanizsa triangle was markedly more impaired in the masked than the AB condition (Fahrenfort et al., 2017), even though task performance was matched. By contrast, later (375–475 ms, centroparietal) illusion decoding was similarly impaired by masking and the AB, closely resembling their matched effects on behavioral performance. Furthermore, we differentiated between collinearity-only and illusion-specific processing and found that both processes were more strongly impaired by masking than the AB. Notably, illusion-specific processing was unaffected by the AB, but completely abolished by masking.

Decoding of these different stimulus features at different points in time, together with their topography, may be regarded as markers of distinct neural processes. Based on neurophysiology and previous neuroimaging studies, our early decoding peak for local contrast decoding (75–95 ms) likely reflects feedforward processing (Fahrenfort et al., 2007, 2017; Kandel et al., 2000; Lamme & Roelfsema, 2000). The first illusion decoding peak may be regarded as a marker of local recurrent processing, while the second illusion decoding peak likely reflects global recurrent processing (Fahrenfort et al., 2017). Thus, our findings suggest that both perceptual and attentional blindness leave feedforward processing largely intact and similarly impair global recurrent processing, while local recurrent processing is markedly more impaired by perceptual than attentional blindness.

Furthermore, based on neurophysiological evidence that collinearity processing primarily relies on lateral connections (Bosking et al., 1997; Gilbert & Wiesel, 1979; Li, 1998; Liang et al., 2017; K. E. Schmidt et al., 1997; Stettler et al., 2002), while processing of the Kanizsa illusion involves both lateral and feedback connections (Halgren et al., 2003; Kok et al., 2016; Kok & de Lange, 2014; Lee & Nguyen, 2001; Pak et al., 2020; Wokke et al., 2013), the comparison of collinearity-only to illusion-specific processing may provide insight into the components of local recurrent processing: lateral and feedback connections (Lamme et al., 1998; Roelfsema, 2006). Both feedback and lateral processing were more strongly impaired by masking than the AB. Notably, illusion-specific feedback processes were unaffected by the AB, but completely abolished by masking. These findings confirm and enrich empirical and theoretical work on perceptual vs. attentional mechanisms of consciousness (Block, 2005; Dehaene et al., 2006; Hatamimajoumerd et al., 2022; Lamme, 2010; Pitts et al., 2018; Sergent & Dehaene, 2004), clearly distinguishing and specifying the neural profiles of each processing stage of the influential four-stage model of conscious experience.

To our knowledge, this is the first study to examine the neural mechanisms underlying conscious access in which behavioral measures of conscious perception are carefully matched between the attentional blink and masking within a single experimental design. Previous investigations have typically employed separate paradigms for perceptual and attentional manipulations, often using different stimuli associated with distinct neural mechanisms, which complicates direct comparisons between manipulations and across studies. Further, inattention approaches generally use stronger sensory input (e.g., stimuli of longer duration, higher contrast) than perceptual manipulations (Stein et al., 2021). Here, we introduced a novel stimulus that allowed us to isolate four distinct stages of visual processing by decoding different features while holding visual stimulation and task context constant. Furthermore, measurement of conscious perception often differs between perceptual and attentional manipulations. In particular, inattention approaches, which have previously tended to reveal more extensive neural processing, frequently involve post-hoc selection of “blind” trials or participants based on subjective awareness reports, which is susceptible to criterion confounds and introduces sampling biases that lead to underestimation of awareness in the selected sample (Peters & Lau, 2016; T. Schmidt, 2015; Shanks, 2017). In contrast, our study analyzed all trials and included all participants while carefully matching perceptual performance and metacognition between masking and the AB. Therefore, any observed neural difference between masking and the AB can be unequivocally attributed to differences between attentional and perceptual manipulations of conscious access.

Our results suggest that, compared to masking, the AB left local recurrent processing intact, while feedforward processing did not differ between the two manipulations. Local recurrent processing plays a critical role in perceptual integration, facilitating the organization of fragmented sensory information, such as lines, surfaces, and objects, into a coherent whole (Roelfsema, 2023). Our EEG decoding results support this notion, demonstrating that the AB allows for greater processing of collinearity and the illusion specifically within a time window spanning 140 to 250 ms after stimulus onset, likely reflecting sparing of local recurrent processes in visual cortex (Fahrenfort et al., 2017; Kok et al., 2016). This aligns with established models of the AB phenomenon, in which the AB reflects a late post-perceptual central bottleneck characterized by limited attentional capacity (Shapiro et al., 1997), so that sensory information presented during the AB can nevertheless undergo extensive processing, allowing for perceptual integration, possibly even leading up to semantic analysis (Luck et al., 1996).

Preserved local recurrent processing during the AB is also consistent with classic load theory (Lavie & Dalton, 2014), where increasing perceptual load (Lavie & de Fockert, 2003) more strongly reduces distractor processing than increasing cognitive load (e.g., by engaging working memory, as in our AB condition). According to this theory, perceptual and attentional manipulations serve as early and late filters for incoming sensory information, respectively, resulting in more extensive processing under inattention. Indeed, one of the few neuroimaging studies that included both manipulations found that only perceptual but not cognitive (working memory) load decreased fMRI activity in the parahippocampal place area in response to distractor scenes (Yi et al., 2004). However, not all neuroimaging evidence is consistent with a stronger effect of perceptual than cognitive load (Brockhoff et al., 2022). Furthermore, previous research has shown that the impact of inattention vs. masking can depend on the neural architecture required for the task at hand. For example, processes related to the detection of conflicting response tendencies, a hallmark of cognitive control and strongly associated with the prefrontal cortex (Ridderinkhof et al., 2004), are more susceptible to inattention, which reduces the depth of stimulus processing (Nuiten et al., 2021) than to masking, restricting recurrent interactions, but allowing for deep feedforward processing (all the way up to prefrontal cortex) (Jiang et al., 2018; van Gaal et al., 2008). Thus, the preservation of local recurrent interactions appears to be particularly important for perceptual integration, aligning with the influential notion that perceptual segmentation and organization may represent the mechanism of conscious experience (Lamme, 2020).

Local recurrent interactions in visual cortex encompass both lateral and feedback connections. The distinct roles of lateral and feedback connections to visual function have received limited attention in human cognitive neuroscience and remain unaddressed in theories of consciousness. Here we sought to distinguish between lateral processing reflecting basic collinearity processing and feedback processing reflecting illusion-specific processing. Our results suggest that lateral processing occurred earlier (between 140 and 190 ms after target stimulus onset) than illusion-specific feedback processing (between 200 and 250 ms), in line with animal research (Angelucci & Bressloff, 2006; Lamme et al., 1998; Roelfsema, 2006). Both lateral and feedback processing appeared to be more strongly affected by masking than by the AB, indicating that the “attentional blindness” stage of the four-stage model of consciousness (Fig. 1A) involves both lateral and feedback connections. Interestingly, masking had a stronger effect on illusion-specific feedback processing than on lateral processing. Along with the distinct temporal and spatial EEG decoding patterns associated with lateral and feedback processing, this suggests a processing sequence from feedforward processing to local recurrent interactions encompassing lateral-to-feedback connections, ultimately leading to global recurrency and conscious report.

Having delineated these distinct stages of feedforward, lateral, feedback and global recurrent processing, one important avenue for future research is to distinguish between unconscious and conscious perceptual processes at these stages. Having opted to equate performance across manipulations in our study, behavioral performance was above chance level for both consciousness manipulations. Follow-up research investigating perceptual integration of fully unconscious stimuli could address ongoing debates between influential theories of consciousness (Cogitate Consortium et al., 2023; Mudrik et al., 2014). The global neuronal workspace theory suggests a durable, yet unconscious processing stage (referred to as preconscious), where the input is not globally available, and amplification through top-down attention is required for conscious access and report (Dehaene et al., 2006). In contrast, others have argued that already local recurrent interactions reflect subjective phenomenal experience (Block, 2005; Lamme, 2010). Moreover, markers like the P300 and ours for global recurrent processing may reflect functions not directly related to conscious experience, like report or decision-making (Alilović et al., 2023; Canales-Johnson et al., 2023; Pitts et al., 2018). Another way forward therefore consists in combining no-report paradigms (Sergent et al., 2021; Tsuch iya et al., 2015) with our EEG markers to examine whether local or global recurrent processing more accurately reflects consciousness in the absence of report.

Methods

Participants

Thirty-three participants took part in the first two sessions (independent EEG training set and practice). Three of them met the practice session’s pre-established criteria for exclusion (see “Procedure”). The remaining 30 participants (22±3 years old, 10 men, 2 left-handed) took part in the final (main experimental) session. They all had normal or corrected-to-normal vision. The study was approved by the local ethics committee. Participants gave informed consent and received research credits or 15 euros per hour.

Stimuli

The target stimulus set had a 2 (illusory Kanizsa triangle: present/absent) × 2 (non-illusory triangle: present/absent) × 2 (rotation: present/absent) design, resulting in eight stimuli (Fig. 1B). Three aligned Pac-Man elements induced the Kanizsa illusion. The non-illusory triangle was present when the stimuli’s three other elements (the “two-legged white circles”) were aligned. The controls for both the illusory and non-illusory triangle were created by rotating their elements by 90 degrees. Differences in local contrast were created by rotating the entire stimulus by 180 degrees. The targets spanned 7.5 degrees by 8.3 degrees of visual angle. The distance between the three Pac-Man stimuli as well as between the three aligned two-legged white circles was 2.8 degrees of visual angle. Although neuronal responses to collinearity in primary visual cortex are most robust when this distance is smaller (Kapadia et al., 1995, 2000), longer-range lateral connections between neurons with similar orientation selectivity can span distances corresponding to visual angles considerably greater than 2.8 degrees (Bosking et al., 1997; Stettler et al., 2002).

The distractor stimulus set was the same as the target stimulus set, with two exceptions. First, the distractors were red instead of black. Second, the distractors’ six elements were rotated by 180 degrees relative to the targets’, so neither the illusory nor non-illusory triangle was ever present in the distractors. Masks consisted of six differently shaped elements, all capable of covering the targets’ elements. Six masks were created by rotating the original mask five times by 60 degrees. They spanned 8.5 degrees by 9.1 degrees of visual angle. The fixation cross, which was always present, was adapted from Thaler et al. (2013).

Procedure

The experiment consisted of three separate sessions conducted on different days: a three-hour session to collect EEG data for the independent training set, a 1.5-hour practice session, and a three-hour experimental session. Tasks were programmed in Presentation software (Neurobehavioral Systems) and displayed on a 23-inch, 60 Hz, 1920×1080 pixels monitor. On each trial of the experimental session, participants were shown two targets (T1 and T2) within a rapid serial visual presentation (RSVP) of distractors (Fig. 1C). The targets and distractors had a stimulus onset asynchrony of 100 ms. T2 and distractors were presented for 17 ms each. To improve the decoding analyses’ training dataset, T1 was presented for 67 ms. The longer presentation duration facilitated attending to T1, which should result in greater deployment of attentional resources and thereby increase the size of the AB. T1 was preceded by five distractors and T2 was followed by six distractors.

T2 visibility was manipulated in two ways, using a perceptual and an attentional manipulation (Fig. 1A). The perceptual manipulation consisted in masking T2 with three masks, each presented for 17 ms with an interstimulus interval of 0 ms. The three masks were selected randomly, but all differed from each other. Half of the T2s were masked; for the other half no masks were presented (unmasked condition). The attentional manipulation consisted in shortening the T1–T2 lag from a long interval of eight distractors (900 ms) to a short interval of one or two distractors (200 or 300 ms). Half the trials had a long lag, the other half had a short lag. The short lag duration was determined for each participant individually during the training session. Short lags were expected to result in an AB. Participants were instructed to fixate on the fixation cross. After the RSVP, they indicated for each target whether it contained the Kanizsa illusion or not. For T2, participants simultaneously reported their confidence in their response: low, medium, or high, resulting in six response options. To get accurate ratings, participants first responded to T2 and then to T1. Response screens lasted until the response. In short, the experimental session had an 8 (T1 stimulus conditions) × 8 (T2 stimulus conditions) × 2 (masked/unmasked) × 2 (short/long T1–T2 lag) task design, resulting in 256 conditions. Each condition was presented four times, totaling 1024 trials.

The experimental session was preceded by the practice session, in which participants were familiarized with the task. To proceed to the experimental session, participants had to score above 80% correct for both T1 and unmasked, long lag T2. One participant was excluded for failing to achieve this. The training session was also used to determine for each participant the duration of the short lag (200 or 300 ms T1–T2 interval) that induced the largest AB (lowest T2 accuracy) and that was used in the subsequent experimental session. Two participants were excluded due to their AB size falling below the predetermined criterion. Specifically, their T2 accuracy at both short lags did not exhibit a decrease of more than 5% compared to long lags.

One of the main goals of this study was to match perceptual performance between the perceptual and the attentional manipulation. We did this in two ways. First, during the training session, the matching was done by staircasing mask contrast using the weighted up-down method (Kaernbach, 1991). Contrast levels ranged from 0 (black) to 255 (white). Mask contrast started at level 220. Each correct response made the task more difficult: masks got darker by downward step size S_down. Each incorrect response made the task easier: masks got lighter by upward step size S_up. Step sizes were determined by S_up × p = S_down × (1 – p), where p is the accuracy at short lags. The smallest step size was always nine contrast levels. A reversal is making a mistake after a correct response, or vice versa. The staircase ended after 25 reversals. The mask contrast with which the experimental session started was the average contrast level of the last 20 reversals. Second, during the experimental session, after every 32 masked trials, mask contrast was updated in accordance with our goal to match performance over participants, while also matching performance within participants as well as possible.

To ensure that confidence ratings for these matched conditions (masked, long lag and unmasked, short lag) were not contaminated by differences in perceptual performance, one type of block only contained the matched conditions, while the other block type contained the two remaining, unmatched conditions (masked, short lag and unmasked, long lag). To ensure every confidence rating would have enough trials for creating receiver operating characteristic curves, participants were instructed to distribute their responses evenly over all ratings within a block. Participants received feedback about the distribution of their responses. The mask contrasts from a performance matched block were used in the subsequent non-performance matched block to ensure that masking remained orthogonal to the AB manipulation. The experimental session therefore always started with a performance matched block.

We wanted to compare illusory triangle decoding to non-illusory triangle decoding. However, during the experimental session, the non-illusory triangle was never task-relevant, only the illusory one was. During the independent EEG classification training session, we therefore made each visual feature, one after the other, task-relevant. A target was presented for 33 ms every 900-1100 ms (Fig. S4). Participants had to fixate on the fixation cross and indicate whether the current task-relevant feature was absent or present. For each feature, each target was presented 64 times, totaling 512 trials. The order of the task-relevant features was counterbalanced over participants. For all sessions, response button mapping was counterbalanced within tasks.

Behavioral analysis

To quantify perceptual performance, we constructed receiver operating characteristic (ROC) curves by plotting objective hit rates against objective false alarm rates. We used the six response options to get five inflection points (Green & Swets, 1966). We also quantified metacognitive sensitivity: the ability to know whether you were right or wrong. Performance is high when you are confident in objectively correct responses and not confident in objectively incorrect responses. We again constructed ROC curves, now by plotting the rate of high-confidence correct responses (subjective hit rates) against the rate of high-confidence incorrect responses (subjective false alarm rates). We used the three confidence ratings to get two inflection points. To ensure that T1 was attended, trials with incorrect T1 responses were excluded. Repeated measures ANOVAs and Bayesian t-tests were used to test the differences between experimental conditions.

EEG recording and preprocessing

EEG was recorded at 1024 Hz using a 64 channel ActiveTwo system (BioSemi). Four electrooculographic (EOG) electrodes measured horizontal and vertical eye movements. The data were analyzed with MATLAB (MathWorks). For most of the preprocessing steps, EEGLAB was used (Delorme & Makeig, 2004). The data were re-referenced to the earlobes. Poor channels were interpolated. High-pass filtering can cause artifacts in decoding analyses; we therefore removed slow drifts using trial-masked robust detrending (van Driel et al., 2021). Each target was epoched from −250 to 1000 ms relative to target onset. To improve the results from the independent component analysis (ICA), baseline correction was applied using the whole epoch as baseline (Groppe et al., 2009). ICA was used to remove blinks. Blink components were removed manually. Baseline correction was applied, now using a −250 to 0 ms window relative to target onset. Trials with values outside of a −300 to 300 microvolts range were removed. We used an adapted version of FieldTrip’s ft_artifact_zvalue function to detect and remove trials with muscle artifacts (Oostenveld et al., 2011). As in the behavioral analyses, trials with incorrect T1 responses were excluded. Finally, the data were downsampled to 128 Hz.

Multivariate pattern analyses

We decoded the different visual features (local contrast, non-illusory triangle, and illusory Kanizsa triangle, respectively; Fig. 1B) using the Amsterdam Decoding and Modeling (ADAM) toolbox (Fahrenfort et al., 2018). For each participant and each visual feature, a linear discriminant classifier was trained on the T1 data and tested on each condition of the T2 data. The classifier was trained to discriminate between the feature’s (e.g., the illusory triangle’s) absence and presence based on the raw EEG activity across all electrodes. AUC was again used as the performance measure. This procedure was executed for every time sample in a trial, yielding classification performance over time. For the time samples from −100 to 700 ms relative to target onset, we used a two-sided t-test to evaluate whether classifier performance differed from chance. We used cluster-based permutation testing (1000 iterations at a threshold of 0.05) to correct for multiple comparisons (Maris & Oostenveld, 2007). To obtain topographic maps showing the neural sources of the classifier performance, we multiplied the classifier weights with the data covariance matrix, yielding covariance/class separability maps (Haufe et al., 2014).

In the decoding analyses described in the results, we applied “diagonal decoding”: classifiers were tested on the same time sample they were trained on. We did the same analyses again, now by applying “off-diagonal decoding”: classifiers trained on a particular time point are tested on all time points (King & Dehaene, 2014). Off-diagonal decoding allowed us to investigate whether patterns of activity during the time windows of interest were stable over time (Fig. S3). For the illusory triangle, classifiers were trained on the 200-250 ms window and then averaged. The same was done for the local contrast 75-95 ms window.

To distinguish between collinearity-only and illusion-specific processing, we trained classifiers on independent data based on collinearity-only (the non-illusory triangle was task-relevant) or collinearity-plus-illusion (the illusory triangle was task-relevant) and then decoded the Kanizsa illusion in T2s of the main RSVP task. The rationale for this analysis is that collinearity is present both when the Pac-Man stimuli align to form the illusory Kanizsa triangle and when the two-legged white circles align to from a non-illusory triangle, but only in the case of the Kanizsa triangle do participants experience an illusion. The comparison of T2 illusion decoding between the classifiers trained on the illusion and on collinearity-only in the training set may therefore isolate illusion-specific (likely involving feedback processing) from basic collinearity processing (likely involving lateral connections; Fig. 3). In the Supplementary information (Fig. S5), we compare the independent training set to the training set used for the main analyses, the T1 data from the RSVP task.

As described in the Supplementary information as well, a tenfold cross-validation scheme was applied to the data from the independent training set to decode local contrast. Individual participants’ data were split into ten equal-sized folds after randomizing the task’s trial order. A classifier was then trained on nine folds and tested on the tenth one, ensuring independence of the training and testing sets. This procedure was repeated until each fold served as the test set once. Classifier performance, AUC, was averaged across all ten iterations (Fig. S7).

As in the behavioral analyses, repeated measures ANOVAs and Bayesian t-tests were used to test the differences between experimental conditions.

References

1. Alilović J.
2. Lampers E.
3. Slagter H. A.
4. van Gaal S.
2023Illusory object recognition is either perceptual or cognitive in origin depending on decision confidencePLoS Biology 21
1. Angelucci A.
2. Bressloff P. C.
2006Contribution of feedforward, lateral and feedback connections to the classical receptive field center and extra-classical receptive field surround of primate V1 neuronsProgress in Brain Research 154:93–120
1. Block N.
2005Two neural correlates of consciousnessTrends in Cognitive Sciences 9:46–52
1. Bosking W. H.
2. Zhang Y.
3. Schofield B.
4. Fitzpatrick D.
1997Orientation selectivity and the arrangement of horizontal connections in tree shrew striate cortexThe Journal of Neuroscience: The Official Journal of the Society for Neuroscience 17:2112–2127
1. Brockhoff L.
2. Schindler S.
3. Bruchmann M.
4. Straube T.
2022Effects of perceptual and working memory load on brain responses to task-irrelevant stimuli: Review and implications for future researchNeuroscience and Biobehavioral Reviews 135
1. Brown R.
2. Lau H.
3. LeDoux J. E.
2019Understanding the Higher-Order Approach to ConsciousnessTrends in Cognitive Sciences https://doi.org/10.1016/j.tics.2019.06.009
1. Canales-Johnson A.
2. Beerendonk L.
3. Chennu S.
4. Davidson M. J.
5. Ince R. A. A.
6. van Gaal S.
2023Feedback information transfer in the human brain reflects bistable perception in the absence of reportPLoS Biology 21
1. Cogitate Consortium
2. Ferrante O.
3. Gorska-Klimowska U.
4. Henin S.
5. Hirschhorn R.
6. Khalaf A.
7. Lepauvre A.
8. Liu L.
9. Richter D.
10. Vidal Y.
11. Bonacchi N.
12. Brown T.
13. Sripad P.
14. Armendariz M.
15. Bendtz K.
16. Ghafari T.
17. Hetenyi D.
18. Jeschke J.
19. Kozma C.
20. Melloni L.
2023An adversarial collaboration to critically evaluate theories of consciousnessbioRxiv https://doi.org/10.1101/2023.06.23.546249
1. Dehaene S.
2. Changeux J.-P.
3. Naccache L.
4. Sackur J.
5. Sergent C.
2006Conscious, preconscious, and subliminal processing: a testable taxonomyTrends in Cognitive Sciences 10:204–211
1. Dehaene S.
2. Sergent C.
3. Changeux J.-P.
2003A neuronal network model linking subjective reports and objective physiological data during conscious perceptionProceedings of the National Academy of Sciences of the United States of America 100:8520–8525
1. Delorme A.
2. Makeig S.
2004EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysisJournal of Neuroscience Methods 134:9–21
1. Dienes Z.
2. Banerjee R.
3. Chakrabarti B. K.
2007Subjective measures of unconscious knowledgeProgress in Brain Research Elsevier :49–269
1. Fahrenfort J. J.
2. Scholte H. S.
3. Lamme V. A. F.
2007Masking disrupts reentrant processing in human visual cortexJournal of Cognitive Neuroscience 19:1488–1497
1. Fahrenfort J. J.
2. van Driel J.
3. van Gaal S.
4. Olivers C. N. L.
2018From ERPs to MVPA Using the Amsterdam Decoding and Modeling Toolbox (ADAM)Frontiers in Neuroscience 12
1. Fahrenfort J. J.
2. van Leeuwen J.
3. Olivers C. N. L.
4. Hogendoorn H.
2017Perceptual integration without conscious accessProceedings of the National Academy of Sciences of the United States of America 114:3744–3749
1. Fleming S. M.
2. Lau H. C.
2014How to measure metacognitionFrontiers in Human Neuroscience 8
1. Gilbert C. D.
2. Wiesel T. N.
1979Morphology and intracortical projections of functionally characterised neurones in the cat visual cortexNature 280:120–125
1. Green D. M.
2. Swets J. A.
1966Signal detection theory and psychophysics
1. Groppe D. M.
2. Makeig S.
3. Kutas M.
2009Identifying reliable independent components via split-half comparisonsNeuroImage 45:1199–1211
1. Halgren E.
2. Mendola J.
3. Chong C. D. R.
4. Dale A. M.
2003Cortical activation to illusory shapes as measured with magnetoencephalographyNeuroImage 18:1001–1009
1. Hatamimajoumerd E.
2. Ratan Murty N. A.
3. Pitts M.
4. Cohen M. A.
2022Decoding perceptual awareness across the brain with a no-report fMRI masking paradigmCurrent Biology: CB https://doi.org/10.1016/j.cub.2022.07.068
1. Haufe S.
2. Meinecke F.
3. Görgen K.
4. Dähne S.
5. Haynes J.-D.
6. Blankertz B.
7. Bießmann F.
2014On the interpretation of weight vectors of linear models in multivariate neuroimagingNeuroImage 87:96–110
1. Jiang J.
2. Correa C. M.
3. Geerts J.
4. van Gaal S.
2018The relationship between conflict awareness and behavioral and oscillatory signatures of immediate and delayed cognitive controlNeuroImage 177:11–19
1. Joglekar M. R.
2. Mejias J. F.
3. Yang G. R.
4. Wang X.-J.
2018Inter-areal Balanced Amplification Enhances Signal Propagation in a Large-Scale Circuit Model of the Primate CortexNeuron 98:222–234
1. Kaernbach C.
1991Simple adaptive testing with the weighted up-down methodPerception & Psychophysics 49:227–229
1. Kanai R.
2. Walsh V.
3. Tseng C.-H.
2010Subjective discriminability of invisibility: a framework for distinguishing perceptual and attentional failures of awarenessConsciousness and Cognition 19:1045–1057
1. Kandel E. R.
2. Schwartz J. H.
3. Jessell T. M.
4. Siegelbaum S.
2000Kandel, E. R., Schwartz, J. H., Jessell, T. M., & Siegelbaum, S. (2000). Principles of neural science. academia.edu. https://www.academia.edu/download/30536508/neuroscience_syllabus.pdf
1. Kapadia M. K.
2. Ito M.
3. Gilbert C. D.
4. Westheimer G.
1995Improvement in visual sensitivity by changes in local context: parallel studies in human observers and in V1 of alert monkeysNeuron 15:843–856
1. Kapadia M. K.
2. Westheimer G.
3. Gilbert C. D.
2000Spatial distribution of contextual interactions in primary visual cortex and in visual perceptionJournal of Neurophysiology 84:2048–2062
1. King J.-R.
2. Dehaene S.
2014Characterizing the dynamics of mental representations: the temporal generalization methodTrends in Cognitive Sciences 18:203–210
1. Kok P.
2. Bains L. J.
3. van Mourik T.
4. Norris D. G.
5. de Lange F. P.
2016Selective Activation of the Deep Layers of the Human Primary Visual Cortex by Top-Down FeedbackCurrent Biology: CB 26:371–376
1. Kok P.
2. de Lange F. P.
2014Shape perception simultaneously up- and downregulates neural activity in the primary visual cortexCurrent Biology: CB 24:1531–1535
1. Lamme V. A. F.
2010How neuroscience will change our view on consciousnessCognitive Neuroscience 1:204–220
1. Lamme V. A. F.
2020Visual Functions Generating Conscious SeeingFrontiers in Psychology 11
1. Lamme V. A. F.
2. Roelfsema P. R.
2000The distinct modes of vision offered by feedforward and recurrent processingTrends in Neurosciences 23:571–579
1. Lamme V. A. F.
2. Supèr H.
3. Spekreijse H.
1998Feedforward, horizontal, and feedback processing in the visual cortexCurrent Opinion in Neurobiology 8:529–535
1. Lau H. C.
2022In Consciousness we Trust: The Cognitive Neuroscience of Subjective ExperienceOxford University Press
1. Lau H. C.
2. Passingham R. E.
2006Relative blindsight in normal observers and the neural correlate of visual consciousnessProceedings of the National Academy of Sciences of the United States of America 103:18763–18768
1. Lavie N.
2. Dalton P.
2014Load theory of attention and cognitive controlThe Oxford Handbook of Attention :56–75
1. Lavie N.
2. de Fockert J. W.
2003Contrasting effects of sensory limits and capacity limits in visual selective attentionPerception & Psychophysics 65:202–212
1. Lee T. S.
2. Nguyen M.
2001Dynamics of subjective contour formation in the early visual cortexProceedings of the National Academy of Sciences of the United States of America 98:1907–1911
1. Li Z.
1998A neural model of contour integration in the primary visual cortexNeural Computation 10:903–940
1. Liang H.
2. Gong X.
3. Chen M.
4. Yan Y.
5. Li W.
6. Gilbert C. D.
2017Interactions between feedback and lateral connections in the primary visual cortexProceedings of the National Academy of Sciences of the United States of America 114:8637–8642
1. Luck S. J.
2. Vogel E. K.
3. Shapiro K. L.
1996Word meanings can be accessed but not reported during the attentional blinkNature 383:616–618
1. Maris E.
2. Oostenveld R.
2007Nonparametric statistical testing of EEG- and MEG-dataJournal of Neuroscience Methods 164:177–190
1. Marti S.
2. King J.-R.
3. Dehaene S.
2015Time-Resolved Decoding of Two Processing Chains during Dual-Task InterferenceNeuron 88:1297–1307
1. Mashour G. A.
2. Roelfsema P.
3. Changeux J.-P.
4. Dehaene S.
2020Conscious Processing and the Global Neuronal Workspace HypothesisNeuron 105:776–798
1. Merikle P. M.
2. Smilek D.
3. Eastwood J. D.
2001Perception without awareness: perspectives from cognitive psychologyCognition 79:115–134
1. Meuwese J. D. I.
2. van Loon A. M.
3. Lamme V. A. F.
4. Fahrenfort J. J.
2014The subjective experience of object recognition: comparing metacognition for object detection and object categorizationAttention, Perception & Psychophysics 76:1057–1068
1. Mudrik L.
2. Faivre N.
3. Koch C.
2014Information integration without awarenessTrends in Cognitive Sciences 18:488–496
1. Northoff G.
2. Lamme V. A. F.
2020Neural signs and mechanisms of consciousness: Is there a potential convergence of theories of consciousness in sight?Neuroscience and Biobehavioral Reviews 118:568–587
1. Nuiten S. A.
2. Canales-Johnson A.
3. Beerendonk L.
4. Nanuashvili N.
5. Fahrenfort J. J.
6. Bekinschtein T.
7. van Gaal S.
2021Preserved sensory processing but hampered conflict detection when stimulus input is task-irrelevantELife 10https://doi.org/10.7554/eLife.64431
1. Oostenveld R.
2. Fries P.
3. Maris E.
4. Schoffelen J.-M.
2011FieldTrip: Open source software for advanced analysis of MEG, EEG, and invasive electrophysiological dataComputational Intelligence and Neuroscience 2011
1. Pak A.
2. Ryu E.
3. Li C.
4. Chubykin A. A.
2020Top-Down Feedback Controls the Cortical Representation of Illusory Contours in Mouse Primary Visual CortexThe Journal of Neuroscience: The Official Journal of the Society for Neuroscience 40:648–660
1. Peters M. A.
2. Lau H.
2016Correction: Human observers have optimal introspective access to perceptual processes even for visually masked stimuliELife 5https://doi.org/10.7554/eLife.16332
1. Pitts M. A.
2. Lutsyshyna L. A.
3. Hillyard S. A.
2018The relationship between attention and consciousness: an expanded taxonomy and implications for ‘no-report’ paradigmsPhilosophical Transactions of the Royal Society of London. Series B, Biological Sciences 373
1. Raymond J. E.
2. Shapiro K. L.
3. Arnell K. M.
1992Temporary suppression of visual processing in an RSVP task: an attentional blink?Journal of Experimental Psychology. Human Perception and Performance 18:849–860
1. Ridderinkhof K. R.
2. Ullsperger M.
3. Crone E. A.
4. Nieuwenhuis S.
2004The role of the medial frontal cortex in cognitive controlScience 306:443–447
1. Roelfsema P. R.
2006Cortical algorithms for perceptual groupingAnnual Review of Neuroscience 29:203–227
1. Roelfsema P. R.
2023Solving the binding problem: Assemblies form when neurons enhance their firing rate-they don’t need to oscillate or synchronizeNeuron 111:1003–1019
1. Schmidt K. E.
2. Goebel R.
3. Löwel S.
4. Singer W.
1997The perceptual grouping criterion of colinearity is reflected by anisotropies of connections in the primary visual cortexThe European Journal of Neuroscience 9:1083–1089
1. Schmidt T.
2015Invisible Stimuli, Implicit Thresholds: Why Invisibility Judgments Cannot be Interpreted in IsolationAdvances in Cognitive Psychology / University of Finance and Management in Warsaw 11:31–41
1. Sergent C.
2. Baillet S.
3. Dehaene S.
2005Timing of the brain events underlying access to consciousness during the attentional blinkNature Neuroscience 8:1391–1400
1. Sergent C.
2. Corazzol M.
3. Labouret G.
4. Stockart F.
5. Wexler M.
6. King J.-R.
7. Meyniel F.
8. Pressnitzer D.
2021Bifurcation in brain dynamics reveals a signature of conscious processing independent of reportNature Communications 12
1. Sergent C.
2. Dehaene S.
2004Is consciousness a gradual phenomenon? Evidence for an all-or-none bifurcation during the attentional blinkPsychological Science 15:720–728
1. Seth A. K.
2. Dienes Z.
3. Cleeremans A.
4. Overgaard M.
5. Pessoa L.
2008Measuring consciousness: relating behavioural and neurophysiological approachesTrends in Cognitive Sciences 12:314–321
1. Shanks D. R.
2017Regressive research: The pitfalls of post hoc data selection in the study of unconscious mental processesPsychonomic Bulletin & Review 24:752–775
1. Shapiro K. L.
2. Raymond J. E.
3. Arnell K. M.
1997The attentional blinkTrends in Cognitive Sciences 1:291–296
1. Stein T.
2. Kaiser D.
3. Fahrenfort J. J.
4. van Gaal S.
2021The human visual system differentially represents subjectively and objectively invisible stimuliPLoS Biology 19
1. Stettler D. D.
2. Das A.
3. Bennett J.
4. Gilbert C. D.
2002Lateral connectivity and contextual interactions in macaque primary visual cortexNeuron 36:739–750
1. Thaler L.
2. Schütz A. C.
3. Goodale M. A.
4. Gegenfurtner K. R.
2013What is the best fixation target? The effect of target shape on stability of fixational eye movementsVision Research 76:31–42
1. Tsuchiya N.
2. Wilke M.
3. Frässle S.
4. Lamme V. A. F.
2015No-Report Paradigms: Extracting the True Neural Correlates of ConsciousnessTrends in Cognitive Sciences 19:757–770
1. van Driel J.
2. Olivers C. N. L.
3. Fahrenfort J. J.
2021High-pass filtering artifacts in multivariate classification of neural time series dataJournal of Neuroscience Methods 352
1. van Gaal S.
2. Ridderinkhof K. R.
3. Fahrenfort J. J.
4. Scholte H. S.
5. Lamme V. A. F.
2008Frontal cortex mediates unconsciously triggered inhibitory controlThe Journal of Neuroscience: The Official Journal of the Society for Neuroscience 28:8053–8062
1. van Vugt B.
2. Dagnino B.
3. Vartak D.
4. Safaai H.
5. Panzeri S.
6. Dehaene S.
7. Roelfsema P. R.
2018The threshold for conscious report: Signal loss and response bias in visual and frontal cortexScience 360:537–542
1. Weaver M. D.
2. Fahrenfort J. J.
3. Belopolsky A.
4. van Gaal S.
2019Independent Neural Activity Patterns for Sensory- and Confidence-Based Information Maintenance during Category-Selective Visual ProcessingENeuro 6https://doi.org/10.1523/ENEURO.0268-18.2018
1. Wokke M. E.
2. Vandenbroucke A. R. E.
3. Scholte H. S.
4. Lamme V. A. F.
2013Confuse your illusion: feedback to early visual cortex contributes to perceptual completionPsychological Science 24:63–71
1. Wyatte D.
2. Jilk D. J.
3. O’Reilly R. C.
2014Early recurrent feedback facilitates visual object recognition under challenging conditionsFrontiers in Psychology 5
1. Yi D.-J.
2. Woodman G. F.
3. Widders D.
4. Marois R.
5. Chun M. M.
2004Neural fate of ignored stimuli: dissociable effects of perceptual and working memory loadNature Neuroscience 7:992–996
1. Zivony A.
2. Lamy D.
2022What processes are disrupted during the attentional blink? An integrative review of event-related potential researchPsychonomic Bulletin & Review 29:394–414

Article and author information

Author information

Samuel Noorman
Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands, Amsterdam Brain and Cognition, University of Amsterdam, Amsterdam, The Netherlands
- Correspondence: sgnoorman@gmail.com
Timo Stein
Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands, Amsterdam Brain and Cognition, University of Amsterdam, Amsterdam, The Netherlands
Johannes J Fahrenfort
Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands, Amsterdam Brain and Cognition, University of Amsterdam, Amsterdam, The Netherlands, Department of Applied and Experimental Psychology, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
ORCID iD: 0000-0002-9025-3436
- Shared senior authorship
Simon van Gaal
Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands, Amsterdam Brain and Cognition, University of Amsterdam, Amsterdam, The Netherlands
ORCID iD: 0000-0001-6628-4534
- Shared senior authorship

Author Notes

This research was supported by a grant from the H2020 European Research Council (ERC STG 715605, SVG).

Version history

Preprint posted: March 15, 2024
Sent for peer review: April 16, 2024
Reviewed Preprint version 1: July 8, 2024

Copyright

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Not revised: This Reviewed Preprint includes the authors’ original preprint (without revision), an eLife assessment, and public reviews.

Reviewing Editor
Peter Kok
University College London, London, United Kingdom
Senior Editor
Tirin Moore
Stanford University, Howard Hughes Medical Institute, Stanford, United States of America

Reviewer #1 (Public Review):

Summary:

In this work, Noorman and colleagues test the predictions of the "four-stage model" of consciousness by combining psychophysics and scalp EEG in humans. The study relies on an elegant experimental design to investigate the respective impact of attentional and perceptual blindness on visual processing.

The study is very well summarised, the text is clear and the methods seem sound. Overall, a very solid piece of work. I haven't identified any major weaknesses. Below I raise a few questions of interpretation that may possibly be the subject of a revision of the text.

(1) The perceptual performance on Fig1D appears to show huge variation across participants, with some participants at chance levels and others with performance > 90% in the attentional blink and/or masked conditions. This seems to reveal that the procedure to match performance across participants was not very successful. Could this impact the results? The authors highlight the fact that they did not resort to post-selection or exclusion of participants, but at the same time do not discuss this equally important point.

(2) In the analysis on collinearity and illusion-specific processing, the authors conclude that the absence of a significant effect of training set demonstrates collinearity-only processing. I don't think that this conclusion is warranted: as the illusory and non-illusory share the same shape, so more elaborate object processing could also be occuring. Please discuss.

(3) Discussion, lines 426-429: It is stated that the results align with the notion that processes of perceptual segmentation and organization represent the mechanism of conscious experience. My interpretation of the results is that they show the contrary: for the same visibility level in the attentional blind or masking conditions, these processes can be implicated or not, which suggests a role during unconscious processing instead.

(4). The two paradigms developed here could be used jointly to highlight non-idiosyncratic NCCs, i.e. EEG markers of visibility or confidence that generalise regardless of the method used. Have the authors attempted to train the classifier on one method and apply it to another (e.g. AB to masking and vice versa)? What perceptual level is assumed to transfer?

(5). How can the results be integrated with the attentional literature showing that attentional filters can be applied early in the processing hierarchy?

https://doi.org/10.7554/eLife.97900.1.sa2

Reviewer #2 (Public Review):

Summary:

This is a very elegant and important EEG study that unifies within a single set of behaviorally equated experimental conditions conscious access (and therefore also conscious access failures) during visual masking and attentional blink (AB) paradigms in humans. By a systematic and clever use of multivariate pattern classifiers across conditions, they could dissect, confirm, and extend a key distinction (initially framed within the GNWT framework) between 'subliminal' and 'pre-conscious' unconscious levels of processing. In particular, the authors could provide strong evidence to distinguish here within the same paradigm these two levels of unconscious processing that precede conscious access : (i) an early (< 80ms) bottom-up and local (in brain) stage of perceptual processing ('local contrast processing') that was preserved in both unconscious conditions, (ii) a later stage and more integrated processing (200-250ms) that was impaired by masking but preserved during AB. On the basis of preexisting studies and theoretical arguments, they suggest that this later stage could correspond to lateral and local recurrent feedback processes. Then, the late conscious access stage appeared as a P3b-like event.

Strengths:

The methodology and analyses are strong and valid. This work adds an important piece in the current scientific debate about levels of unconscious processing and specificities of conscious access in relation to feed-forward, lateral, and late brain-scale top-down recurrent processing.

Weaknesses:

- The authors could improve clarity of the rich set of decoding analyses across conditions.
- They could also enrich their Introduction and Discussion sections by taking into account the importance of conscious influences on some unconscious cognitive processes (revision of traditional concept of 'automaticity'), that may introduce some complexity in Results interpretation
- They should discuss the rich literature reporting high-level unconscious processing in masking paradigms (culminating in semantic processing of digits, words or even small group of words, and pictures) in the light of their proposal (deeper unconscious processing during AB than during masking).

https://doi.org/10.7554/eLife.97900.1.sa1

Reviewer #3 (Public Review):

Summary:

This work aims to investigate how perceptual and attentional processes affect conscious access in humans. By using multivariate decoding analysis of electroencephalography (EEG) data, the authors explored the neural temporal dynamics of visual processing across different levels of complexity (local contrast, collinearity, and illusory perception). This is achieved by comparing the decidability of an illusory percept in matched conditions of perceptual (i.e., degrading the strength of sensory input using visual masking) and attentional impairment (i.e., impairing top-down attention using attentional blink, AB). The decoding results reveal three distinct temporal responses associated with the three levels of visual processing. Interestingly, the early stage of local contrast processing remains unaffected by both masking and AB. However, the later stage of collinearity and illusory percept processing are impaired by the perceptual manipulation but remain unaffected by the attentional manipulation. These findings contribute to the understanding of the unique neural dynamics of perceptual and attentional functions and how they interact with the different stages of conscious access.

Strengths:

The study investigates perceptual and attentional impairments across multiple levels of visual processing in a single experiment. Local contrast, collinearity, and illusory perception were manipulated using different configurations of the same visual stimuli. This clever design allows for the investigation of different levels of visual processing under similar low-level conditions.

Moreover, behavioural performance was matched between perceptual and attentional manipulations. One of the main problems when comparing perceptual and attentional manipulations on conscious access is that they tend to impact performance at different levels, with perceptual manipulations like masking producing larger effects. The study utilizes a staircasing procedure to find the optimal contrast of the mask stimuli to produce a performance impairment to the illusory perception comparable to the attentional condition, both in terms of perceptual performance (i.e., indicating whether the target contained the Kanizsa illusion) and metacognition (i.e., confidence in the response).

The results show a clear dissociation between the three levels of visual processing in terms of temporal dynamics. Local contrast was represented at an early stage (~80 ms), while collinearity and illusory perception were associated with later stages (~200-250 ms). Furthermore, the results provide clear evidence in support of a dissociation between the effects of perceptual and attentional processes on conscious access: while the former affected both neuronal correlates of collinearity and illusory perception, the latter did not have any effect on the processing of the more complex visual features involved in the illusion perception.

Weaknesses:

The design of the study and the results presented are very similar to those in Fahrenfort et al. (2017), reducing its novelty. Similar to the current study, Fahrenfort et al. (2017) tested the idea that if both masking and AB impact perceptual integration, they should affect the neural markers of perceptual integration in a similar way. They found that behavioural performance (hit/false alarm rate) was affected by both masking and AB, even though only the latter was significant in the unmasked condition. An early classification peak was instead only affected by masking. However, a late classification peak showed a pattern similar to the behavioural results, with classification affected by both masking and AB.

The interpretation of the results mainly centres on the theoretical framework of the recurrent processing theory of consciousness (Lamme, 2020), which lead to the assumption that local contrast, collinearity, and the illusory perception reflect feedforward, local recurrent, and global recurrent connections, respectively. It should be mentioned, however, that this theoretical prediction is not directly tested in the study. Moreover, the evidence for the dissociation between illusion and collinearity in terms of lateral and feedback connections seems at least limited. For instance, Kok et al. (2016) found that, whereas bottom-up stimulation activated all cortical layers, feedback activity induced by illusory figures led to a selective activation of the deep layers. Lee & Nguyen (2001), instead, found that V1 neurons respond to illusory contours of the Kanizsa figures, particularly in the superficial layers. They all mention feedback connections, but none seem to point to lateral connections.

Moreover, the evidence in favour of primarily lateral connections driving collinearity seems mixed as well. On one hand, Liang et al. (2017) showed that feedback and lateral connections closely interact to mediate image grouping and segmentation. On the other hand, Stettler et al. (2002) showed that, whereas the intrinsic connections link similarly oriented domains in V1, V2 to V1 feedback displays no such specificity. Furthermore, the other studies mentioned in the manuscript did not investigate feedback connections but only lateral ones, making it difficult to draw any clear conclusions.

https://doi.org/10.7554/eLife.97900.1.sa0

Distinct neural mechanisms underlying perceptual and attentional impairments of conscious access

Significance of findings

Strength of evidence

Abstract

Introduction

Experimental design and behavior.

Results

Masking and the attentional blink were matched in perceptual performance and metacognition

Masking and the attentional blink leave local contrast decoding largely intact

Local contrast and illusory triangle decoding using first targets as training data.

Stronger effect of masking than the attentional blink on early but not on late illusion decoding

Distinguishing collinearity and illusion-specific processing

Preserved collinearity and illusion-specific processing during the attentional blink

Separating collinearity and illusion-specific processes using the independent training dataset.

Discussion

Methods

Participants

Stimuli

Procedure

Behavioral analysis

EEG recording and preprocessing

Multivariate pattern analyses

References

Article and author information

Author information

Samuel Noorman

Timo Stein

Johannes J Fahrenfort

Simon van Gaal

Author Notes

Version history

Copyright

Peer review process

Editors

Be the first to read new articles from eLife

Significance of findings

Strength of evidence

Abstract

Introduction

Experimental design and behavior.

Results

Masking and the attentional blink were matched in perceptual performance and metacognition

Masking and the attentional blink leave local contrast decoding largely intact

Local contrast and illusory triangle decoding using first targets as training data.

Stronger effect of masking than the attentional blink on early but not on late illusion decoding

Distinguishing collinearity and illusion-specific processing

Preserved collinearity and illusion-specific processing during the attentional blink

Separating collinearity and illusion-specific processes using the independent training dataset.

Discussion

Methods

Participants

Stimuli

Procedure

Behavioral analysis

EEG recording and preprocessing

Multivariate pattern analyses

References

Article and author information

Author information

Samuel Noorman

Timo Stein

Johannes J Fahrenfort**

Simon van Gaal**

Author Notes

Version history

Copyright

Peer review process

Editors

Johannes J Fahrenfort

Simon van Gaal