The involvement of the human prefrontal cortex in the emergence of visual awareness

  1. Zepeng Fang
  2. Yuanyuan Dang
  3. Zhipei Ling
  4. Yongzheng Han
  5. Hulin Zhao  Is a corresponding author
  6. Xin Xu  Is a corresponding author
  7. Mingsha Zhang  Is a corresponding author
  1. State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Division of Psychology, Beijing Normal University, China
  2. Department of Neurosurgery, Chinese PLA General Hospital, China
  3. Department of Anesthesiology, Peking University Third Hospital, China

Abstract

Exploring the neural mechanisms of awareness is a fundamental task of cognitive neuroscience. There is an ongoing dispute regarding the role of the prefrontal cortex (PFC) in the emergence of awareness, which is partially raised by the confound between report- and awareness-related activity. To address this problem, we designed a visual awareness task that can minimize report-related motor confounding. Our results show that saccadic latency is significantly shorter in the aware trials than in the unaware trials. Local field potential (LFP) data from six patients consistently show early (200–300ms) awareness-related activity in the PFC, including event-related potential and high-gamma activity. Moreover, the awareness state can be reliably decoded by the neural activity in the PFC since the early stage, and the neural pattern is dynamically changed rather than being stable during the representation of awareness. Furthermore, the enhancement of dynamic functional connectivity, through the phase modulation at low frequency, between the PFC and other brain regions in the early stage of the awareness trials may explain the mechanism of conscious access. These results indicate that the PFC is critically involved in the emergence of awareness.

eLife assessment

This paper reports valuable results regarding the potential role and time course of the prefrontal cortex in conscious perception. Although the sample size is small, the results are convincing, and strengths include the use of several complementary analysis methods. The behavioral test includes subject report such that the study does not allow for distinguishing between (phenomenal) awareness and conscious access; nevertheless, results do advance our understanding of the contribution of prefrontal cortex to conscious perception.

https://doi.org/10.7554/eLife.89076.3.sa0

Introduction

A conscious state means that a person has some kind of subjective experiences, such as seeing a painting, hearing a sound, thinking about a problem, or feeling an emotion (Koch et al., 2016). The neural mechanism of consciousness is one of the most fundamental, exciting and challenging pursuits in the 21st-century brain science (Mashour, 2018). Among the various aspects of consciousness, visual awareness, as a core of consciousness, has attracted enormous attention in consciousness research. Visual awareness seems quite simple: we see an object and seem to immediately know its shape, color and other properties. However, the neural processes behind the emergence of visual awareness are quite sophisticated and largely unknown (Dehaene et al., 2017).

The typical method to study the neural mechanism of visual awareness is to present external stimuli with similar physical properties but to elicit different conscious experiences, such as using near perceptual threshold stimuli, visual masking, or attentional blinks (Kim and Blake, 2005; Dehaene and Changeux, 2011). Then, the aware and unaware states are determined through the subjective reports of participants, and the awareness-related neural activity is dissociated by the brain activation difference between the two states. Using this approach, some studies have found that activities in broad brain regions are correlated with visual awareness. For example, an fMRI study of lexical visual masking found that only consciously perceived and reported words can induce activation of higher-order cortical regions such as the frontal and parietal lobes (Dehaene et al., 2001). Another study using a visual masking task found changes in the amplitude of intracranial electroencephalography (iEEG) signals and high-gamma power in occipital-temporal regions, which indicated that these regions played an important role in the emergence of awareness (Fisch et al., 2009).

However, there is still much debate about the origin of conscious awareness in the brain. Some hypotheses (global neuronal workspace theory, higher-order theory, etc.) argued that visual awareness arose from the frontoparietal network, in which the prefrontal cortex played a key role, and they argued that the earliest biomarker of awareness in EEG is the P3b wave onset approximately 300ms after stimulus onset (Mashour et al., 2020; Brown et al., 2019). For example, based on the electrophysiological study of a visual awareness-guided saccade selection paradigm in macaques, by comparing the action potential firing characteristics of neurons in the dorsolateral prefrontal cortex (dlPFC), V4 and V1, only dlPFC neurons were closely associated with visual awareness states. Therefore, the authors argued that stimuli must elicit a threshold level of prefrontal activity for conscious detection to occur (van Vugt et al., 2018). In contrast, other hypotheses (integrated information theory, recurrent processing theory, etc.) proposed that the ‘hot zone’ of the posterior cortex of the brain, including the parietal lobe, occipital lobe, and temporal lobe, is sufficient to generate visual awareness, and its biomarkers on EEG are the relatively earlier visual awareness negativity (VAN) at approximately 200ms after stimulus onset (Koch et al., 2016; Koch, 2018; Lamme, 2018; Dembski et al., 2021), whereas the PFC is not necessary for the emergence of conscious experience.

This controversy is partially caused by the confounding effect of report-related activity, such as motor planning and execution, on awareness-related activity. The reason is that the study of awareness usually requires subjective reports to determine different awareness states, while the prefrontal cortex is closely related to such report-related functions (Seth and Bayne, 2022). To solve this problem, some researchers use the no-report paradigm to rule out report-related impacts. For instance, a functional magnetic resonance imaging (fMRI) study employing human binocular rivalry paradigms found that when subjects need to manually report the changing of their awareness between conflict visual stimuli, the frontal, parietal, and occipital lobes all exhibited awareness-related activity. However, when report was not required, awareness-related activation was largely diminished in the frontal lobe but remained in the occipital and parietal lobes (Frässle et al., 2014). Nevertheless, the no-report paradigm may overestimate the neural correlates of awareness by including unconscious processing, because it infers the awareness state through other relevant physiological indicators, such as optokinetic nystagmus and pupil size (Tsuchiya et al., 2015). In the absence of subjective reports, it remains controversial regarding whether the presented stimuli are truly seen or not. Therefore, neither the typical report paradigm nor the no-report paradigm is ideal for visual awareness research, thus, it is difficult to solve the problem of whether the PFC participates in conscious perception.

In addition, when studying the role of the PFC in visual awareness, the noninvasive technologies mostly used in previous studies, that is, Functional magnetic resonance imaging (fMRI) and M/EEG have limitations. Although fMRI research can detect the whole brain activation pattern, it is difficult to describe the dynamic changes and sequences of different brain regions in the rapid process of visual awareness due to its limited time resolution (approximately several seconds). For example, even though a recent fMRI study using the no-reported paradigm found that there are still visual awareness-related activities in the PFC (Kronemer et al., 2022), it remains unclear when these activities begin and whether PFC activity occurs earlier or later than other brain regions. At the same time, although EEG and MEG have a very high temporal resolution, their low spatial resolution and low signal-to-noise ratio limit their interpretation of awareness-related activities, especially high-frequency activities (highly correlated with local population neuronal activities). For example, although increasing evidence shows that the earliest biomarker of visual awareness in EEG is more likely to be the VAN, which starts at approximately 180–300ms (may be delayed to 300–460ms for low-visible stimulus) and peaks at the parieto-occipital region (Northoff and Lamme, 2020; Koivisto and Revonsuo, 2010; Koivisto et al., 2008), rather than the P3b wave onset at approximately 300–500ms which peaks in the central frontoparietal region, it is still unclear whether there are local visual awareness-related activities in the PFC during the VAN period. In addition, a few recent studies have suggested that awareness may be related to the activities of brain networks, including the PFC, rather than that of a single brain region. However, most of these brain network studies used noninvasive research methods (Kronemer et al., 2022; Huang et al., 2023), and due to the limitations mentioned above, it is difficult to detect the dynamic changes in brain networks in a more precise spatial-temporal profile during the generation of visual awareness.

To address the questions mentioned above, we designed a novel visual awareness task (Figure 1) that can minimize motor-related confounding, while retaining the explicit subjective report. The contrast of the cue stimulus (grating) was roughly divided into three levels: well above the visual perceptual threshold, near the visual perceptual threshold and zero contrast (no stimulus). During each experiment, in nearly 80% of trials, the contrast of the grating was close to the subject’s visual perceptual threshold so that the subjects could sometimes see the grating and sometimes could not. Subjects were required to choose a saccade direction according to the visual awareness state (seeing the grating or not seeing the grating) and the color of the fixation point. Notably, subjects were unable to choose the saccade direction or effectively prepare for saccades until the color of the fixation point was changed. Moreover, the subjects needed to choose the saccade direction, both in the awareness and unawareness states, according to the color of the fixation point so that the report behaviors were matched between the two awareness states. Such a paradigm can effectively dissociate awareness-related activity from motor-related activity in terms of time (the fixation point changes color 650ms after the grating onset) and report behavior, thereby being report-independent of the awareness states (Merten and Nieder, 2012), thus minimizing the impact of motor-related factors on visual awareness. In addition, saccade parameters (such as reaction time, etc.) under different awareness states can be compared to explore the influence of visual awareness on saccade behavior. Then, we applied the above behavioral paradigm to six clinical patients with implanted electrodes (stereoelectroencephalography, sEEG) in prefrontal and other cortices and 10 healthy subjects. The behavior and local field potential (LFP) data of six patients were recorded while they performed the task, and we also collected the behavioral data of 10 healthy subjects for comparison with the patients’ behavioral data.

Schematic diagram of the visual awareness task.

A trial started when a fixation point (0.5°x0.5°, white cross) appeared at the center of the screen (radius of eye position check window is 4°, the dotted circle). After the subject fixated on the fixation point for 600ms, a cue stimulus (Gabor grating, 2x2° circle) was presented for 50ms at a fixed position (7°) on the left (or right, see Methods) side of the screen for all participants. In 70% of the trials, the grating contrast was maintained near the subject’s perceptual threshold by a staircase method; in 10% of the trials, the stimulus contrast was well above the threshold; and in the other 20% of the trials, the stimulus contrast was 0, namely, no stimulus appeared. After another 600ms delay, the color of the fixation point turned red or green, and two saccade targets (1x1°, white square) appeared at fixed positions (10°) on the left and right sides of the screen. If the grating was seen, the green fixation point was required to make a saccade to the right target, while the red fixation point was required to make a saccade to the left target. If the grating was not seen, the rule of saccadic direction was inverted.

Results

Behavioral results

To detect the effect of cue stimulus (grating) contrast on the emergence of visual awareness, we first analyzed the proportion of participants being aware of the grating as a function of its contrast. Figure 2A shows exemplified sessions from a patient and healthy subject, and Figure 2B shows the population results of the two participant groups. The results showed that with increasing grating contrast, the ratio of reporting ‘awareness’ of the grating gradually increased for all participants (including patients and healthy subjects), showing a classical psychometric curve. Such results showed that the grating contrast level had a direct impact on the emergence of visual awareness and thus is a reliable way to study visual awareness. In addition, in trials with no grating and grating contrast well above the perceptual threshold, subjects showed high accuracy (patients 94.75%±2.65; healthy subjects, 96.80%±0.71, mean ± SEM) and sensitivity (d’=1.81 ± 0.27 for patients and 2.12±0.37 for healthy subjects), indicating that the subjects understood and performed the task well following the task rules.

Behavioral performance and electrode location.

(A) Psychometric detection curve in a single session. Left panels from one patient and right from one healthy subject. Each black point in the graph represents the aware percent in a contrast level, and the black curve represents the fitted psychometric function. The contrast level that resulted in an awareness percentage greater than 25%, and less than 75% was defined as near-threshold, whereas an awareness percentage less than 25% was low and greater than 75% was high. (B) Psychometric detection curves for all participants. The same as Panel A except the contrast is aligned to the individual subject’s perceptual threshold, that is, the contrast 0 represents each subject’s perceptual threshold. (N, number of subjects; R2, coefficient of determination). (C) Saccadic reaction time in the aware and unaware trials under high +low and near-threshold-level contrasts of gratings. Bars show the mean values of saccadic reaction time. In the near-threshold condition, the mean reaction times of patients and healthy subjects in the aware trials were 676.55±42.38 and 556.16±28.45ms, respectively. The mean reaction times in the unaware trials were 892.57±43.28 and 702.30±30.93ms, respectively, and the p values were 0.03 and 9.77x10 –4, respectively, Wilcoxon signed-rank test. In the high +low condition, the mean reaction times for patients and healthy subjects under the aware trials were 577.78±55.28 and 524.08±31.06ms, and the mean reaction times of the unaware trials were 898.64±57.21 and 687.59±49.07ms, respectively, with p values of 0.002 and 9.77x10–4, respectively, Wilcoxon signed-rank test. Each black dot represents one participant. The gray solid line represents the paired dots. Error bars represent the standard error of the mean (SEM). (D) Left, right, and top views of all recording sites projected on an MNI brain template. Each color represents a participant. In all brain images, right and up side of the image represent the right and up side of the brain.

To explore whether visual awareness affects behavioral performance, we divided the trials into 4 groups according to the level of grating contrast and the participants’ reported awareness state, that is high-contrast aware (HA), near-threshold-contrast aware (NA), near-threshold-contrast unaware (NU), and low-contrast unaware (LU) (Figure 2A, also see Methods). We compared the saccadic parameters between the awareness and unawareness states in near-threshold contrast levels (NA vs. NU), as well as in high and low contrast levels (HA vs. LU), as a control. While saccadic reaction time showed a significant difference between the two awareness states (Figure 2B), the other parameters did not. (Figure 2C) shows the mean saccadic reaction time of patients and healthy subjects under different awareness states and contrast levels. The results showed that the average saccadic reaction time was significantly shorter under the awareness state than under the unawareness state at both the near-threshold contrast level and the high-low contrast level (p<0.05, Wilcoxon signed rank test), and it was also explicit and consistent at the individual level. The results showed that the saccadic reaction time in the aware trials was systematically shorter than that in the unaware trials. Such results demonstrate that visual awareness significantly affects the speed of information processing in the brain.

Local field potential results

While the patients were performing our visual awareness task, we recorded the local field potential (LFP) from 901 recording sites (Figure 2D, projected on Montreal Neurological Institute (MNI) brain template ICBM152), in which 245 sites were located in the PFC. The table in Figure 2D shows a summary of the number of recording sites and patients in different lobes and structures.

Early visual awareness-related iERP activity in the PFC

While looking through the event-related response of the LFP data, a striking phenomenon in all patients was that the amplitude of LFP showed remarkably different patterns between awareness and unawareness states. We show the grand average (left) and single-trial data (right) of LFP activity of recording sites in the PFC from each patient in Figure 3A, which represents the typical awareness-related neural activity of the PFC. There was a vigorous visual evoked response under the HA condition (light red line) compared to the LU (light blue line) condition. However, in the near-threshold condition, although the contrast of the grating was similar (see Results below), the LFP amplitude of the NA (red line) and NU (blue line) trials started to differ significantly at approximately 200–300ms (thick black line in panels, p<0.01 corrected, by independent t-test, details see Methods). This difference is also very explicit and consistent at the single-trial (>180 trials for NA and NU conditions in individual participants, see Methods) and single-subject level. The divergence onset time (DOT, see Methods) of different recording sites in the PFC is consistently within the period of 200~300ms after grating onset, that is the time window of VAN in scalp EEG mentioned above.

Visual awareness-related iERP activities in the PFC.

(A) LFP activity of example sites from each patient. Each panel represents one example site from one patient. In each panel, the left plot shows the grand average of LFP activity in different conditions. The pink line represents data under HA conditions, the red line represents data under NA conditions, the blue line represents data under NU conditions, and the light blue line represents data under LU conditions. The shaded area of the curve represents the SEM. The two black dotted lines at 0ms and 650ms represent the time when the grating and fixation point change color (the appearance of saccade targets), and the black thick solid line area represents the period when the LFP amplitude is significantly different under NA and NU conditions (p<0.01 corrected, independent sample t-test). The right figure in each panel shows the single-trial data in the NA (upper) and NU (lower) conditions. The color represents the voltage. (B) The spatial-temporal distribution of awareness-related ERP activities after the appearance of the grating. Each brain image showed a significant (p<0.01 corrected, see Methods) difference in local field potential in NA and NU trials at all visual awareness-related sites at a specific time point (lower left corner) after the grating appeared. The color represents the standardized voltage difference (see Methods). (C) Spatial distribution of divergence onset time. The color represents the normalized DOT value. (D) Population results of early phase ERP response in the prefrontal cortex. Different lines represent different conditions, as shown in Panel A. The right table shows the number of recording sites for different patients. The lower right is the location of these sites. The dots with different colors represent different subjects, as shown in Figure 2D.

Figure 3—source code 1

Source code files for pre-processing iEEG data and generating the results in Figure 3.

https://cdn.elifesciences.org/articles/89076/elife-89076-fig3-code1-v1.zip

As proposed above, the different activities between NA and NU are regarded as awareness-related activities. We found 89 sites with awareness-related iERP activity in the PFC (323 for all recording sites). Figure 3B shows the spatial-temporal dynamics of visual awareness-related activities in all recording sites during the 180–650ms period (sampled at eight time points that equally divided this period, for a clearer display, projected to the cortex, see the Video 1 for the complete video). Visual awareness-related activities clearly appeared in the PFC during 200–300ms, and these early activities were mainly concentrated in the middle lateral prefrontal cortex (LPFC) and then spread to other brain areas of the PFC. We further calculated the start time of awareness-related activity, that is DOT, of all recording sites. Figure 3C shows the spatial distribution of all recording sites DOT (standardized and convenient for visual display), and it can be found that the DOT of the PFC is not explicitly later than those in posterior brain regions, including the posterior parietal lobe, temporal lobe, occipital lobe, etc., and the earliest region in PFC is still located in the middle lateral prefrontal cortex.

Video 1
The spatial-temporal dynamics of visual awareness-related activities (event-related potential (ERP)) in all recording sites during the 180–650ms period (for a clearer display, projected to the cortex).

The curves in the upper left panel represent the significant (p<0.01 corrected, see Methods) difference (normalized) in averaged amplitude of local field potential in NA and NU trials at all visual awareness-related ERP sites. The brain image showed the difference projected on the surface at a specific time point after the grating appeared (top, left, and right views were showed at upper right, lower left, and lower right panels, respectively). The color represents the normalized voltage difference (see Methods).

Considering the possibility that the brain region with earlier awareness-related activity might be more important for visual awareness, we tried to detect awareness-related activity in the PFC during the ‘early’ period, that is the VAN time window mentioned above. We focused on the sites in PFC with earlier DOTs. Because of the low-visible stimulus used in this experiment, the onset time of VAN may be delayed to 300–350ms, and we took 350ms as the cutoff value and divided DOT into early (n=43, 48.31%) and late (n=46, 51.69%) phases. We focused on the early DOT sites and calculated the normalized average iERP response of the early DOT sites (Figure 3D). At the population level, visual awareness-related activities in the PFC were also significant and started at approximately 200–300ms, which is located in the VAN time window (Figure 3D, left). Importantly, there were sites with DOT <350ms from each patient and these sites from different patients were uniformly concentrated in the middle lateral prefrontal cortex (Figure 3D, right). That is, the early visual awareness-related activities in the prefrontal cortex are robust and consistent not only at the group level but also at the individual level.

Early visual awareness-related high-gamma activity in the PFC

Although the above iERP results indicate that there are early visual awareness-related neural activities in the prefrontal cortex, it is difficult to explain whether this response is the result of local processing in the PFC or transmission from other brain regions. Therefore, we calculated the LFP response in the high-gamma (HG) band, which is more representative of the local processing, of the prefrontal cortex. Figure 4A shows the exemplified high-gamma activity in the prefrontal cortex from each patient at the grand average (left), spectrogram (middle), and single-trial level (right). It is similar to the iERP results (Figure 3) except the data are the HG power and there was a vigorous visual evoked power increase under the HA condition compared to the LU condition. However, in the near-threshold condition, while the contrast of the grating was similar, the high-gamma power of the NA and NU trials started to differ significantly after ~230ms (thick black line in panels, P<0.01 corrected by independent t-test; for details, see Methods). This difference is also very explicit and consistent at the single-trial and single-subject levels. The HG activity divergence onset time (DOT) of different recording sites in the PFC was also consistent within the early stage, which was similar to the above iERP result.

Figure 4 with 1 supplement see all
Visual awareness-related high-gamma activities in the PFC.

(A) High-gamma activity of example sites from each patient. Each panel represents one example site from one patient. In each panel, the left figure shows the grand average of high-gamma power in different conditions. The pink line represents data under HA conditions, the red line represents data under NA conditions, the blue line represents data under NU conditions, and the light blue line represents data under LU conditions. The shaded area of the curve represents the SEM. The two black dotted lines at 0ms and 650ms represent the time when the grating and fixation point change color (the appearance of saccade targets), and the black thick solid line area represents the period when the high-gamma power is significantly different under NA and NU conditions (p<0.01 corrected, independent sample t-test). The right figure in each panel shows the single-trial data in the NA (upper) and NU (lower) conditions. The color represents the normalized power. (B) The spatial-temporal distribution of awareness-related high-gamma activities after the appearance of the grating. Each brain image showed a significant (p<0.01 corrected, see Methods) difference in high-gamma activity in NA and NU trials at all visual awareness-related sites at a specific time point (lower left corner) after the grating appeared. The color represents the standardized power difference. (C) Spatial distribution of divergence onset time for high-gamma activity. The color represents the normalized DOT value. (D) Population results of early phase high-gamma response in the prefrontal cortex. Different lines represent different conditions, as shown in Panel A. The middle panel shows the spectrogram in NA (upper) and NU (lower) trials. The right table shows the number of recording sites for different patients. The lower right is the location of these sites. The dots with different colors represent different subjects, as shown in Figure 2D. (E) Percentage of awareness-related sites in ERP and HG analysis. n, number of recording sites in PFC.

We found 31 sites with awareness-related HG activity in the PFC (74 for all recording sites). Figure 4B shows the spatial-temporal dynamics of visual awareness-related HG activities in all recording sites during the 0–650ms period (for a clearer display, projected to the cortex, see the Video 2 for the complete video). Visual awareness-related HG activities also clearly appeared in the PFC during 200–300ms, while they were rarely seen in lower-level visual areas (occipital and temporal lobe), and these early activities were mainly concentrated in the middle lateral prefrontal cortex (LPFC) and then spread to other brain areas of the PFC. We further calculated the start time of awareness-related HG activity, that is DOT, of all recording sites. Figure 4C shows the spatial distribution of all recording sites DOT (standardized and convenient for visual display), and it can be found that the DOT of the PFC is not explicitly later than those in posterior brain regions, and the earliest region is still located in the middle lateral prefrontal cortex.

Video 2
The spatial-temporal dynamics of visual awareness-related activities (high-gamma (HG) activity) in all recording sites during the 180–650ms period (for a clearer display, projected to the cortex).

The same as Video 1 but for high-gamma activity. The curves in the upper left panel represent the significant (p<0.01 corrected, see Methods) difference (normalized) in averaged magnitude of high-gamma activity in NA and NU trials at all visual awareness-related HG sites. The brain image showed the difference projected on the surface at a specific time point after the grating appeared (top, left and right views were showed at upper right, lower left and lower right panels, respectively). The color represents the normalized magnitude difference (see Methods).

We continue to take 350ms as the cutoff value and divide the HG DOT into early (n=18, 58.06%) and late (13, 41.94%). We focused on the early DOT sites and calculated the normalized mean response (Figure 4D) and spectrum of the early DOT sites. At the population level, early visual awareness-related HG activities of the prefrontal sites still exist, and the start time is approximately 200ms, which is within the VAN time window. Importantly, there are sites with HG DOT <350ms for each patient except one, and these sites are also uniformly concentrated in the middle lateral prefrontal cortex. In other words, this effect is significant and consistent at both the group and individual levels. It is worth noting that in the PFC recording sites, the early sites with a high-gamma visual response (7.35%) were explicitly less than those with ERP visual response (17.55%), and this is also evident in the temporal dynamics (Figure 4E), which may indicate that in the process of visual awareness generation, there may also be a variety of other bands activity for information interaction between multiple brain regions.

Decoding awareness state through ERP and HG activity in the PFC

In addition to the univariate analysis above, we also adopted the machine learning decoding method based on multivariate pattern analysis (MVPA) to test the ability of broadband and high-gamma activities in the PFC to predict the awareness state at the single-trial level. We applied a linear discriminant analysis (LDA) to classify aware versus unaware trials based on broadband and high-gamma LFP activities in the PFC.

Figure 5A shows the decoding performance through the ERP activity in the PFC. We found that the ERP activity of the PFC can accurately predict the state of visual awareness at the single-trial level in the delayed period after grating onset, and this result is also relatively consistent at the individual level. Moreover, this decoding accuracy has begun to be much higher than the chance level in 200–300ms, consistent with our above univariate analysis results. Furthermore, we applied the temporal generalization method to test the predictive ability across time. The results (Figure 5B) show that the decoding performance of population ERP is very limited in temporal generalization and cannot be well generalized over a long period. Importantly, this phenomenon is also very consistent at the individual level, which indicates that the representation of visual awareness via broadband activities in the PFC is relatively dynamic rather than static. Moreover, we also applied temporal generalization across the control conditions, that is train on NA/NU trials but test on HA/LU trials to test the predictive ability across conditions. The classifier trained on the NA/NU trials can be well generalized to the HA/LU trials, and the generalization across time is also very consistent. These results further indicated the robustness of the above decoding results.

Decoding the awareness state through broadband and HG activities.

(A) Decoding the awareness state through broadband LFP activity. The population result (left) is the average result of six individual subjects (right) after decoding analysis. (B) Time generalization of broadband activity decoding. The left panel represents generalization across time. The right panel represents generalization across different conditions, which means training on NA&NU data and testing on HA&LU data. In each panel, the population result (left) is the average result of six subjects (right) after decoding analysis. (C) The awareness state is decoded by HG activity. Same as A, except that the HG magnitude is used for decoding. (D) Time generalization of HG activity decoding. Same as B, except that the HG magnitude is used for decoding.

Figure 5C–D shows the results of a similar analysis of HG activity in the prefrontal cortex. (Figure 5C) shows the decoding performance through HG activity in the PFC. We found that the HG activity of the PFC can accurately predict the state of visual awareness at the single-trial level in the delayed period, and this result is also relatively consistent at the individual level. Moreover, this decoding accuracy began to be significantly higher than chance at approximately 200–300ms, which is also consistent with our above univariate analysis results. Furthermore, we applied the temporal generalization method to the HG activity. Although the group results still show that the decoding performance of HG activity is limited in time generalization, the generalization results are not consistent at the individual level, which may be attributed to the variability of electrode location in different patients. This may suggest that the representation of visual awareness-related information by population neurons at different regions in the PFC is inconsistent, that is some are relatively stable, and others are dynamic. Similarly, we applied temporal generalization across the HA/LU conditions, and the result was also very consistent.

Dynamic functional connectivity changes in the PFC associated with visual awareness

We further explored the dynamic functional connectivity between the prefrontal cortex and other brain regions in the generation of visual awareness. First, we calculated the time-across phase-locking value (PLV) (Tass et al., 1998) in NA and NU conditions at the sensor level (baseline removed, see Methods for details). Figure 6A–B shows the example results of two exemplified patients (see Figure 6—figure supplement 1. for other patients). We found that in the low-frequency band (1–8 Hz), overall, there was a significant difference in the PLV between the NA and NU trials. In the first 150ms after the appearance of the grating, there was no significant difference in the PLV between these two conditions. However, in the period of approximately 200–300ms, the PLV between many recoding sites in the NA trials began to increase explicitly and lasted until after 600ms. In contrast, there was no such phenomenon in the NU trials. Importantly, this phenomenon is strikingly explicit and consistent in each patient.

Figure 6 with 1 supplement see all
Functional connectivity analysis results.

(A–B) Phase-locking value (PLV) changes at the sensor level (NxN) in two example patients. From left to right, the changes in PLV between each recording site at the five time points of 0/150/300/450/600ms after the appearance of the grating in the near-threshold aware (upper) and near-threshold unaware (lower) trials are displayed (baseline removed, see the Methods). The color represents PLV. (C) The population results of functional connectivity analysis averaged according to brain regions. From left to right, PLV between different brain regions at the five time points of 0/150/300/450/600ms after the appearance of the grating in the near-threshold aware (upper) and near-threshold unaware (lower) trials (baseline removed, see the Methods). The color represents the PLV. (D) For the same data as C, only the strongest 15% PLV is displayed on the brain template for clear visualization. (E) PLV under different conditions at early-phase HG awareness-related sites in the PFC. The red line represents the average PLV in the NA trials, and the blue line represents the average PLV in the NU trials. The faded line represents the PLV of each site.

Furthermore, we grouped the recording sites by brain regions according to the DKT brain atlas Fischl, 2012 and averaged the PLV between different sites and different subjects according to brain regions (Figure 6C). The functional connectivity between the prefrontal cortex and subcortical structures, such as the thalamus, and the posterior cortex, such as the posterior parietal cortex, was significantly enhanced in the NA trials compared with the PLV in NU trials. In addition, the functional connectivity between the brain regions within the PFC was also significantly enhanced. For clearer visualization, we show the dynamics of PLV in the NA and NU trials on a topographic map (Figure 6D).

Interestingly, we found that at the single-subject level, the sites in the prefrontal cortex that showed early awareness-related PLV activity coincided with the above sites with early awareness-related HG activity. Figure 6E shows the averaged PLV between these sites (n=18) with early visual awareness-related HG activity in the PFC and other sites in the NA and NU trials. The averaged PLV also showed significant visual awareness-related activities. Moreover, at the population level, the start time of awareness-related PLV activity at these sites also started at approximately 200ms, which is very similar to that of HG activity.

The different neural activity between NA and NU trials is not caused by the grating contrast

Since there may be a slight difference in the grating contrast under the NA and NU trials, the aforementioned different activity between the NA and NU conditions may be caused by the different contrast of grating. To rule out this possibility, we compared the distribution of grating contrast in the NA and NU conditions for each patient (Figure 7A). Overall, we find that the distributions of grating contrasts largely overlap with each other (red and blue curves). The Kolmogorov‒Smirnov test resulted in no significant difference (p>0.05) between the two contrast distributions in all patients except patient 2. Although intuitively the trials with higher contrast contributed more to the NA condition and the trials with lower contrast contributed more to the NU condition, the contrast difference between the NA and NU conditions were not significantly different perhaps because the grating contrast was designed to be near the perceptual threshold (the ‘aware’ percent is 50%) in most trials. Moreover, for patient 2, the difference between the mean grating contrast in the NA and NU trials was also subtle (0.15%), much less than the difference between NA and HA (17.35%) or NU and LU (0.84%). However, it is clear in the results above that the different activities between NA and NU were much stronger than those between the same awareness states, that is NA vs. HA or NU vs. LU. Therefore, the difference in neural activity between aware and unaware trials cannot be explained by the contrast differences in the grating.

Grating contrast in NA and NU conditions.

(A) Distribution of grating contrast in the NA and NU conditions for each patient. Each panel represents one patient. The red and blue lines represent the distribution of grating contrast in NA and NU conditions, respectively.

Discussion

We employed a novel visual awareness task and found that, first, the saccadic reaction time of all subjects was shorter when they were aware of grating than when they were not aware. Second, there are ERP and HG activities related to visual awareness in the prefrontal LFP, and these activities begin at the early stage (200–300ms). Then, we found that the awareness state can be reliably decoded from the broadband and HG activities of the PFC at approximately 200–300ms after stimulus onset. Finally, the functional connectivity between the PFC and other brain regions in the low-frequency band (1–8 Hz) also appeared awareness-related changes, and the initial time and location of this change are consistent with the above HG results. The results above are not caused by the difference in the physical properties of external stimuli. Therefore, we propose that the prefrontal cortex also plays a key role in the early stage of visual awareness generation even if motor effects are excluded.

Advantages and limitations of the behavioral paradigm used in the present study

Compared with the regular report-dependent visual awareness task, we employed a task that effectively dissociated the activity of visual awareness from the activity of saccadic decision and execution in terms of time (the fixation point changes color 650ms after the grating onset) and reporting behavior. Since the choice of reporting behavior depends on the visual awareness state and the color of the fixation point, the subjects were unable to effectively prepare the subsequent saccadic reports during the delay period of the task; thus, the visual awareness-related activity evoked by the grating stimulus and report-preparation-related activity were temporally dissociated. Furthermore, both in the awareness and unawareness states, the subjects needed to choose the saccade direction after the fixation point changes its color so that the saccade directions in the two visual awareness states were fully matched to exclude the effects of saccade preparation- and execution-related activity on visual awareness-related activity. Compared with the no-report paradigms, our novel paradigm required subjects to make explicit reports of their awareness states so that it can be more accurate regarding the judgment of the awareness status.

Moreover, the intervals for the initial fixation period and delay period were both fixed at 600ms, instead of randomly varied, in order to keep subjects’ anticipation being similar in all trials. Also, the location of grating remained same in all trials for the same purpose.

It was also worth noting that our task does not remove the confound of report entirely, that is the post-perceptual confound might relate to planning to report perception, which is different for perceived and not perceived stimuli. Nevertheless, our task did minimize the motor-related confound to awareness-related activity.

Visual awareness states affect saccadic reaction time

Many factors affect saccadic reaction time, including the characteristics of external stimuli and the internal state of the brain. The physical properties of external visual stimuli have a direct impact on the performance of saccades; for example, saccadic reaction time decreases with increasing visual stimulus contrast (Allik and Kreegipuu, 1998; Deplancke et al., 2010). A widely accepted view is that the greater contrast of visual stimuli will automatically attract more bottom-up attention and speed up information processing in the brain, thereby shortening the saccadic reaction time (Posner and Petersen, 1990; Corbetta and Shulman, 2002). Other studies found that endogenous factors, such as top-down attention, can also significantly shorten the saccadic reaction time (Corbetta and Shulman, 2002; Posner, 1980).

One study briefly reported that the average reaction time (button press) for word semantic classification in the awareness state was approximately 300ms faster than in the unawareness state, but the statistical test did not reach a significant level (Gaillard et al., 2009). In present study, we found that the saccadic reaction time of all subjects in the aware trials was significantly smaller than that in the unaware trials (Figure 2C). More critically, the grating contrast near the threshold was not significantly different between the aware and unaware trials (Figure 7A); thus, such results cannot be merely caused by the contrast difference in the grating. Here, we showed that awareness states significantly affect saccadic reaction time, which supports our previous hypothesis.

An alternative interpretation for RT difference between aware and unaware condition in our study is that the difference in task-strategies used by subjects/patients to remember the response mapping rules between the perception and the color cue (e.g. if the YES +GREEN = RIGHT and YES +RED = LEFT rules were held in memory, while the NO mappings were inferred secondarily rather than being actively held in memory).

Another possibility is that the reaction time is strongly modulated by the confident level, which has been described in previous studies (Marzi et al., 2006; Broggin et al., 2012). However, in previous studies, the confident levels were usually induced by presenting stimulus with different physical property, such as spatial frequency, eccentricity and contrast. However, the dependence of visual process on the salience of visual stimulus confounds with the effect of visual awareness on the reaction time of responsive movements, which is hard to attribute the shorter reaction time in more salient condition purely to visual awareness. In contrast, we create a condition (near aware threshold) in the present study, in which the saliency (contrast) of visual stimulus is very similar in both aware and unaware conditions in order to eliminate the influence of stimulus saliency in reaction time. We think that the difference in reaction time in our study is mainly due to the modulation of awareness state, which was not reported previously.

Visual awareness-related activities appeared in the PFC at an early stage

The long debate about the neural mechanism of consciousness focuses on the location, that is ‘front’ vs. ‘back’, and the time, that is ‘early’ vs. ‘late’, of its origin in the brain (Seth and Bayne, 2022). Concerning the dispute of location, our results show that the prefrontal cortex still displays visual awareness-related activities even after minimizing the influence of the motor-related confounding variables related to subjective reports such as motion preparation, which indicates that the prefrontal cortex does participate in the information processing of visual awareness. Regarding the time dispute, our results show that at 200–300ms after stimulus onset, the VAN time window, the prefrontal cortex has activities related to visual awareness, and these activities include not only ERP activities but also HG activities, which are highly related to the firing of local population neurons (Fisch et al., 2009). Notably, the recording sites with early ERP and HG visual awareness-related activities are mostly concentrated in the middle lateral prefrontal cortex. This shows that the prefrontal cortex, especially the middle lateral prefrontal cortex, started visual awareness-related activities, including local activities, at a relatively early stage.

Several iEEG studies have shown the early prefrontal cortical involvement in visual perception, including ERP and HFA (Blanke et al., 1999; Khalaf et al., 2023; Vishne et al., 2023). However, in these studies, the differential activity between conscious and unconscious conditions was not investigated, thus, the activity in prefrontal cortex might be correlated with unconscious processing, rather than conscious processing. In present study, we compared the neural activity in PFC between conscious and unconscious trials, and found the correlation between PFC activity and conscious perception.

Although one iEEG study (Gaillard et al., 2009) reported awareness-specific PFC activation, the awareness-related activity started 300ms after the onset of visual stimuli, which was ~100ms later than the early awareness related activity in our study. Also, due to the limited number of electrodes in the previous study (2 patients with 19 recording sites mostly in mesiofrontal and peri-insular regions), it was restricted while exploring the awareness-related activity in PFC. In the present study, the number of recording sites (245) were much more than previous study and covered multiple areas in PFC. Our results further show earlier awareness-related activity (~200ms after visual stimuli onset), including ERP, HFA and PLV. These awareness-related activity in PFC occurred even earlier (~150ms after stimulus onset) for the salient stimulus trials (Figure 3A, D and Figure 4A, D, HA condition)’.

However, the proportions are much smaller than that reported by Gaillard et al, which peaked at ~60%. We think that one possibility for the difference may be due to the more sampled PFC subregions in present study and the uneven distribution of awareness-related activity in PFC. Meanwhile, we noticed that the peri-insula regions and middle frontal gyrus (MFG), which were similar with the regions reported by Gaillard et al, seemed to show more fraction of awareness-related sites than other subregions during the delay period (0–650ms after stimulus onset). To test such possibility and make comparison with the study of Gaillard et al., we calculated the proportion of awareness-related site in peri-insula and MFG regions (Figure 4—figure supplement 1). We found although the proportion of awareness-related site was larger in peri-insula and MFG than in other subregions, it was much lower than the report of Gaillard et al. One alternative possibility for the difference between these two studies might be due to the more complex task in Gaillard et al. Nevertheless, we think these new results would contribute to our understanding of the neural mechanism underlying conscious perception, especially for the role of PFC.

Some recent studies utilizing the no-report paradigm supported our results. For example, a recent electrophysiological study of rhesus monkeys found that even if monkeys do not need to report their conscious content under the binocular competition paradigm, there were also visual awareness-related spiking activities in the prefrontal cortex (Kapoor et al., 2022). In addition, some previous human intracranial studies also reported ERP, HG, and single-neuron activities related to visual perception in the higher visual areas of the temporal lobe, such as the superior temporal gyrus (Fisch et al., 2009; Reber et al., 2017). However, due to the limitations of the experimental paradigm, these studies cannot completely exclude the impact of subjective reports. Moreover, most of the electrodes in previous studies were located in the temporal lobe, and electrodes in the prefrontal cortex are rare. As our data show that the proportion of sites with earlier ERP and HG responses in the prefrontal cortex is low and relatively concentrated, it is not hard to explain that earlier visual awareness-related activities in the prefrontal cortex were not found in previous iEEG studies, especially in the HG band. Our results, on the one hand, support that the prefrontal cortex may play an important role in the emergence of visual awareness and thus support the 'front of the brain’ theories of consciousness in the origin brain regions. However, our data also support the prediction of the early emergence of consciousness proposed by the 'back of the brain’ theories. This means that our result also challenges some predictions of these mainstream theories.

Awareness state can be effectively decoded from PFC activities from an early stage

Compared with traditional univariate analysis, MVPA-based machine learning decoding may be more sensitive to the information contained in brain activities (Grootswagers et al., 2017), so it has gradually become a powerful tool for the study of neural correlates of consciousness (NCCs). Therefore, we used linear discriminant analysis (LDA) to decode the awareness states at the single-trial level through broadband and HG activities in the prefrontal cortex. Our decoding results show that whether through broadband or HG activities, the decoding performance began well above the chance level at approximately 200–300ms. This shows that the prefrontal cortex has begun to encode visual awareness-related information in 200–300ms, which is consistent with the above univariate analysis results. In addition, a recent electrophysiological study of rhesus monkeys using the no-report paradigm also found that even without the requirement of explicit reporting, awareness-related neurons can still be found in the lateral prefrontal cortex of rhesus monkeys, and conscious content can be accurately decoded by the MVPA method (Kapoor et al., 2022). Another human fMRI study also found similar results (Kronemer et al., 2022). These results supported our experimental results.

In addition, we also applied the temporal generalization method during decoding. Temporal generalization analysis shows how different brain activity patterns change and transform over time (King and Dehaene, 2014). Our results show that the representation pattern of neural activity in the prefrontal cortex for visual awareness-related information is dynamic, not static, and this result is very consistent at the individual level and across the control conditions. Such a result may be consistent with the nature of conscious perception. The process of conscious perception (visual awareness) includes a series of processes from phenomenal consciousness (‘qulia’) to conscious access. Therefore, at the level of large, high-order brain regions, the representation of this complex process in the brain should be continuously transmitted and transformed, that is, dynamically changed.

Evaluate the role of the prefrontal cortex in visual awareness at the brain network level

There is growing evidence showing that conscious experience may be represented at the brain network level (Kronemer et al., 2022; Huang et al., 2023). Thanks to the simultaneously recorded LFP data in multiple brain regions with a high signal-to-noise ratio and high spatial-temporal resolution, we can analyze the dynamic functional connectivity changes under the aware and unaware conditions at the millisecond scale. Since the signals in the low-frequency band are more involved in the information interaction between different brain regions and the activities in the high-frequency band are more representative of the local neural activities, it is easy to understand the functional connectivity changes in the low-frequency band between relatively distant brain regions. Some previous studies have found that the phase synchronization of neural activities in beta and gamma bands is related to visual awareness, but because most of these studies are based on fMRI, it is difficult to describe the dynamic changes in such brain networks on a more precise time scale. The only human iEEG study (Gaillard et al., 2009) reported that the phase synchronization of the beta band in the aware condition also occurred relatively late (>300ms) and mainly confined to posterior zones but not PFC. Moreover, most of these studies did not rule out the influence of motor-related processing, and these processes are often associated with changes in the functional connectivity of multiple brain regions. To the best of our knowledge, we are the first to report dynamic changes in functional connectivity on the millisecond time scale in the low-frequency band (1–8 Hz) associated with visual awareness, and this change is not caused by motor-related confounding factors, such as motion preparation. Moreover, the early sites of PLV and HG overlapped each other to a large extent. Such converging results may further suggest that the middle lateral prefrontal cortex, the brain regions where these sites are located, plays a crucial role in the emergence of visual awareness.

Our results may contribute to some theoretical concepts related to consciousness

Compared with the unaware trials, the high-frequency activity in the PFC and the functional connectivity between many brain regions in the aware trials increased. These results may confirm some theoretical concepts, such as ‘ignition’ and conscious access. Global neuronal workspace theory (GNWT) emphasizes ignition and conscious access (Dehaene and Changeux, 2011; Mashour et al., 2020). Ignition means nonlinear brain activation to small stimulus changes (such as perceptual threshold stimulation). For the near-threshold stimulus used in the current study, while the sensory information in the brain passes through a certain threshold in a specific brain area, such as the lateral prefrontal cortex, the population of neurons discharges (HG activity of local field potential), that is, ‘ignition’. This is consistent with many previous human intracranial studies (Fisch et al., 2009; Reber et al., 2017). However, as discussed above, in contrast with previous studies, our study detected earlier awareness-specific ‘ignition’ in the human PFC, while minimizing the motor-related confounding.

Conscious access means that conscious contents can be shared by many other local information processors in the brain (Dehaene and Changeux, 2011; Mashour et al., 2020). For example, when you see the grating, you can either report it orally or draw it down. Such a function may require the participation and coordination of many related brain regions or networks, but there is no clear biomarker for how these brain regions coordinate. The explicit increase in the PLV, specifically appearing in the aware trials, in the low-frequency band found in the current study may provide potential biological meaning for conscious access; that is, many brain regions interact with each other through phase synchronization in the low-frequency band to accomplish information sharing of conscious content.

Interestingly, the beginning time of prefrontal visual awareness-related activities found in ERP and HG activities is very consistent with the onset time of dynamic functional connectivity enhancement, both of which start at 200–300ms. In addition, the dynamic functional connectivity of the sites with early HG visual awareness-related activity is usually the earliest to enhance compared to other sites, and the two have good consistency. And this consistency suggested that conscious access and phenomenal awareness may be closely coupled, occurring initially in the lateral PFC at the early stage, and indicate a potential process of conscious perception: subtle change in the external stimulus or internal brain state - population neuron discharge in specific brain areas (ignition, phenomenal consciousness appeared) - phase synchronization of various brain areas induced by specific brain areas (conscious access). Further research is needed to regulate the activity of the middle lateral prefrontal cortex in the high-gamma band or the phase synchronization in the low-frequency band to explore whether the prefrontal cortex plays a causal role in the process of conscious perception.

Methods

Data acquisition

Six adult patients with drug-resistant epilepsy (6 males, 32.33±4.75 years old, mean ± SEM) and 10 healthy subjects (4 males, 6 females, age 27.80±3.92, mean ± SEM) participated in this study. All subjects had normal or corrected-to-normal vision. Electrophysiological signals (LFPs) were obtained from six patients. Stereotaxic EEG depth electrodes (SINOVATION MEDICAL TECHNOLOGY CO., LTD. Ltd., Beijing, China) containing 8–20 sites were implanted in patients in the Department of Neurosurgery, PLA General Hospital. Each site was 0.8 mm in diameter and 2 mm in length, with 1.5 mm spacing between adjacent sites. A few electrodes were segmented, and the distance between the two segments was 10 mm. Electrode placement was based on clinical requirements only. Recordings were referenced to a site in white matter. The sEEG signal was sampled at a rate of 1 kHz, filtered between 1 and 250 Hz, and notched at 50 Hz (NEURACLE Technology Co., Ltd., Beijing, China). Stimulus-triggered electrical pulses were recorded simultaneously with sEEG signals to precisely synchronize the stimulus with electrophysiological signals.

Ten healthy subjects performed the experiment in a quiet dark room. Stimuli were presented on a 27-inch LED screen (BenQ, refresh rate of 120 Hz, resolution of 1920 x 1080), and eye movement data were recorded by an infrared eye tracker (EyeLink 1000 Desktop; SR Research) with a sampling rate of 1 kHz. Patients performed the experiment in a quiet dark environment. The stimuli were presented on a 24-inch LED screen (Admiral Overseas Corporation, refresh rate of 144 Hz, resolution of 1920 x 1080), and eye position data were obtained by an infrared eye tracker (Jsmz EM2000C, Beijing Jasmine Science & Technology, Co. Ltd), sampling at 1 kHz. The experimental paradigm was presented by MATLAB (The MathWorks) and the toolbox Psychtoolbox-3 (PTB-3; Brainard and Pelli). Patients were regular recipients of standard medication for epilepsy treatment during the experimental period.

All subjects provided informed consent to participate in this study. The Ethics Committee of Chinese PLA General Hospital approved the experimental procedures (approval numbers S2022-457-01).

Electrode localization

Electrode locations were determined by registering postoperative computed tomography (CT) scans and preoperative T1 MRI using a standardized mutual information method in SPM12 (Ashburner and Friston, 1997). Cortical reconstruction and parcellation (Desikan-Killiany Atlas) were conducted by FreeSurfer (Fischl, 2012). To demonstrate the MNI coordinates of recording sites, a nonlinear surface registration was performed using a nonlinear registration in SPM12, aligning to the MNI ICBM152 template.

Experimental task

A trial started when a fixation point (0.5° x 0.5°, white cross) appeared at the center of the screen (radius of eye position check window is 4°, the dotted circle). After the subject fixated on the fixation point for 600ms, a cue stimulus (Gabor grating, 2x2° circle) was presented for 50ms at a fixed position (7°) on the left (or right) side of the screen for all participants. The side where the grating appears (left or right) is opposite to the hemisphere where the patients’ electrodes are implanted (left or right). For example, if the patient’s electrodes are implanted on the right side, the grating would appear on the left side of the screen. For patients 1, 5, and 6, the electrodes were implanted in both hemispheres, and the grating is set on the right side. In 70% of the trials, the grating contrast (weber contrast, see van Vugt et al., 2018) was maintained near the subject’s perceptual threshold by a 1 up/1 down staircase method, and the step was either 0.39% or 0.78% for each participant. In 10% of the trials, the stimulus contrast was well above the threshold, and in the other 20% of the trials, the stimulus contrast was 0, namely, no stimulus appeared. After another 600ms delay, the color of the fixation point turned red or green, and two saccade targets (1° x 1°, white square) appeared at fixed positions (10°) on the left and right sides of the screen. If the grating was seen, the green fixation point was required to make a saccade to the right target, while the red fixation point was required to make a saccade to the left target. If the grating was not seen, the rule of saccadic direction was inverted. Gratings with high contrast (well above the perceptual threshold) and zero contrast (grating absent) served as control conditions to evaluate the understanding and performance of the task. Before starting to collect data, subjects accepted 1–2 training sessions, and the contrast perceptual threshold was determined for each subject. In the formal experiments, each session consisted of 180 trials, and the intertrial interval (ITI) was 800ms.

Data analysis

Each healthy subject completed two sessions, and each patient completed five to seven sessions. Each session contained 180 trials in total. We excluded trials in which subjects broke fixation during the fixation period (eye position exceeded 4x4° check window by more than 40ms) or the saccadic latency exceeded max response duration (2000ms for most patients, 5000ms for patient 6 according to the patient’s performance) after the target appeared. For all subjects, more than 85% of the trials were available for further analysis.

In each session, according to the subjects' reports of being aware and unaware of the existence of grating, we divided the trials into awareness and unawareness states. According to the percentage of subjects reporting 'awareness', we further divided grating contrast into three levels: high contrast (aware percentage >75%), near-threshold contrast (25% ≤ aware percentage ≤ 75%), and low contrast (aware ratio <25%). Due to the few trials of low-contrast-aware and high-contrast-unaware trials, we next analyzed only trials under four conditions of awareness: low-contrast-unaware (LU), near-threshold-unaware (NU), near-threshold-aware (NA) and high contrast-awareness (HA). In total, under LU/NU/NA/HA conditions, there were 243/353/343/160 trials (22.1/32.1/31.2/15.6%, averaged across patients). Since we were more concerned with the differences in subjects' behavior and electrophysiological responses during the physical stimulus, we focused on the results under NA and NU conditions in the subsequent analysis. The results under LU and HA conditions were classified as the control group and were only used to verify and check the results during calculation.

Behavioral data analysis

Psychophysical curve fitting uses the following formula:

AP=a+b1+exp(d(SCc))

AP, ratio of reported awareness; SC, stimulus contrast; a-d, individual fitting parameters.

For population psychometric curve fitting, adjacent contrast levels were averaged, resulting in 8 contrast levels in each session.

Saccadic latency calculations

The saccadic latency was calculated according to a custom program. Eye position and saccade velocity were smoothed with 20ms sliding windows, and each step was 1ms. Saccade onset time was calculated as the time when the saccade velocity first exceeded 30°/s and lasted more than 20ms. Saccadic latency was defined as the time interval between the onset of the saccadic target and the onset of the saccade.

LFP data analysis

Data preprocessing

Data preprocessing was performed by the software package EEGLAB (Delorme and Makeig, 2004) in MATLAB. Data were filtered between 1–250 Hz and notch-filtered at 50, 100, 150, 200, and 250 Hz with an FIR filter. Epochs were extracted from –490ms before grating onset to 1299ms after grating onset. Bad recording sites were discarded for analysis based on visual inspection and power spectral density (PSD) analysis. each recording site was applied a bipolar reference, rereferenced to its direct neighbor.

Spectral analyses

Time-frequency decomposition in high-gamma bands (60–150 Hz) was achieved by Morlet wavelets in the Brainstorm toolbox (Tadel et al., 2011). Before computation, the mean ERP in each condition was removed. For visualization, we used a prestimulus baseline correction over –200–0ms, and the results in Figure 4 show the HG activity increases or decreases relative to this baseline. To avoid windowing artifacts, we only reported results of –200–800ms.

Quantitative definition of visual awareness-related activity in recording sites

We first identified visual awareness-related sites. For all recording sites, we compared the broadband amplitude (for ERP analysis) or high-gamma magnitude (for HG analysis) of LFP between the NA and NU conditions within 650ms after grating onset by statistical tests for each time point. We defined the site where there was a significant difference (p<0.01 FDR corrected for time points and channels, independent t-test) in the LFP between NA and NU conditions and persisted for more than 20ms as an awareness-related site. We also defined the start time of divergence as the divergence onset time (DOT) and defined the mean of the LFP amplitude difference in the first divergence period as the divergence amplitude (DA).

Normalized DOT:

Td=TtargetDOT
normDOT=TdTdminTdmaxTdmin

DOT, divergence onset time; Ttarget, time from target onset to grating onset, 650ms; Td, time from DOT to Ttarget; Tdmin, the minimum value of Td in all awareness-related sites within one patient; Tdmax, the maximum value of Td of all awareness-related sites within one patient; normDOT, normalized DOT value, a larger normDOT value means a shorter DOT.

For population analysis of recording sites, we conducted baseline normalization for each site.

Baseline normalization for ERP amplitude:

xnorm=x-μμ*100

xnorm, normalized value; x, LFP amplitude; μ, mean LFP amplitude in the baseline period (−200–0ms).

Since it is not clearly understood so far regarding the biological characteristics of LFP polarity (Einevoll et al., 2013), to simplify such complex issue, we consider the change in magnitude of LFP during delay period in our task is awareness related activity, regardless its actual value being positive or negative. Therefore, while analyzing the awareness related population activity, we first calculate the absolute value of activity difference between aware and unaware trials in individual recording site, then pool the data of 43 recording sites together and calculate the mean and standard error of mean (SEM) (Figure 3D). As in Figure 3A, the activity difference between aware (red) and unaware (blue) trials lasts until/after the end of delay period. Thus, the awareness related population activity in Figure 3D extends out to 600ms.

Baseline normalization for HG magnitude:

xnorm=xμσ100

xnorm, normalized value; x, high-gamma magnitude or power; μ, mean of high-gamma magnitude or power in the baseline period (−200–0ms); σ, standard error of high-gamma magnitude or power in the baseline period (−200–0ms).

Baseline normalization for spectrogram:

xnorm=10*log10x/μ

xnorm, normalized value; x, power value; μ, mean power in the baseline period (−200–0ms).

Topographic display of ERP and HG data

To visualize the topography of the ERP and HG activities, we used the Brainstorm (Tadel et al., 2011) package of MATLAB to map the results on the ICBM152 cortical surface. It is a simple 3D interpolation with Shepard’s method (weights decreasing with the square of the distance), similar to what is done when projecting subject sources to an anatomy template, where vertices of the cortical surface that are further than 15 mm from the center of the sEEG contact are ignored.

For ERP visualization, we normalized the activity in absolute value because it is not clearly understood so far regarding the biological characteristics of LFP polarity. To simplify such complex issue, we consider the change in magnitude of LFP during delay period in our task represents awareness related activity, regardless its actual value being positive or negative. Therefore, we first calculated the absolute value of activity difference between aware and unaware trials in individual recording site, then used Shepard’s method to calculate the activity in each vertex and projected on the surface of brain template as shown in Figure 3B.

Decoding Analysis

We implemented a machine learning framework for trial-by-trial classification using broadband amplitude and high-gamma magnitude of LFP activities in the PFC via the MVPA-Light toolbox (Guggenmos et al., 2018). We applied the LDA classification and temporal generalization on each participant separately, and all trials from each class were undersampled to balance the unequal trial number between different conditions. Then, the trials were randomly assigned into 10 folds, and all trials within each fold were subaveraged across every four trials to increase the signal-to-noise ratio. Decoding was then followed with a leave-one-out cross-validation procedure on the subaveraged trials. This procedure was repeated 10 times so that each fold in the dataset was used exactly once for testing, and at least once for training, but never at the same time. And the fold assignment was repeated five times, which resulted in total 50 iterations for the decoding analysis of each patient.

Functional Connectivity analysis

The time-across phase-locking value (PLV) was calculated in the brainstorm toolbox. Time-frequency transformation was conducted via the Hilbert transform method. The time-across PLV measures if the phase difference between two signals is consistent across trials with respect to the event that defines the trials (i.e. the output is a mean across trials per time point) (Mercier et al., 2022). PLV values range between 0 (for totally random) and 1 (for perfect phase locking). For clear demonstration here, the mean PLV in the baseline period (−200–0ms) of each pair of recording sites was subtracted. The group analysis of PLV was performed according to Betzel et al., 2019; Sadaghiani et al., 2022. The topographic display of the connectivity was conducted by the BrainNet Viewer toolbox (Xia et al., 2013).

Data availability

The raw data, including behavioral data, sEEG data, the deidentified brain imaging data, are available from the Dryad Digital Repository: http://dx.doi.org/10.5061/dryad.p8cz8w9xp. The code for data analysis is uploaded as source code files.

The following data sets were generated
    1. Fang Z
    2. Dang Y
    3. Ling Z
    4. Han Y
    5. Zhao H
    6. Xu X
    7. Zhang M
    (2024) Dryad Digital Repository
    The raw data, including behavioral data, sEEG data, the deidentified brain imaging data.
    https://doi.org/10.5061/dryad.p8cz8w9xp

References

  1. Software
    1. Ashburner J
    2. Friston K
    (1997)
    Spatial transformation of images
    Spm Course Notes.
    1. Posner MI
    (1980) Orienting of attention
    The Quarterly Journal of Experimental Psychology 32:3–25.
    https://doi.org/10.1080/00335558008248231

Peer review

Reviewer #1 (Public Review):

This is a clear and rigorous study of intracranial EEG signals in the prefrontal cortex during a visual awareness task. The results are convincing and worthwhile, and strengths include the use of several complementary analysis methods and clear results. The only methodological weakness is relatively small sample size of only 6 participants compared to other studies in the field. Interpretation weaknesses are claims that their task removes the confound of report (it does not), and claims of primacy in showing early prefrontal cortical involvement in visual perception using intracranial EEG (several studies already have shown this). Also the shorter reaction times for perceived vs not perceived stimuli (confident vs not confident responses) has been described many times previously and is not a new result.

https://doi.org/10.7554/eLife.89076.3.sa1

Reviewer #2 (Public Review):

The authors attempt to address a long-standing controversy in the study of the neural correlates of visual awareness, namely whether neurons in prefrontal cortex are necessarily involved in conscious perception. Several leading theories of consciousness propose a necessary role for (at least some sub-regions of) PFC in basic perceptual awareness (e.g., global neuronal workspace theory, higher order theories), while several other leading theories posit that much of the previously reported PFC contributions to perceptual awareness may have been confounded by task-based cognition that co-varied between the aware and unaware reports (e.g., recurrent processing theory, integrated information theory). By employing intracranial EEG in human patients and a threshold detection task on low-contrast visual stimuli, the authors assessed the timing and location of neural populations in PFC that are differentially activated by stimuli that are consciously perceived vs. not perceived. Overall, the reported results support the view that certain regions of PFC do contribute to visual awareness, but at time-points earlier than traditionally predicted by GNWT and HOTs.

Major strengths of this paper include the straightforward visual threshold detection task including the careful calibration of the stimuli and the separate set of healthy control subjects used for validation of the behavioral and eye tracking results, the high quality of the neural data in six epilepsy patients, the clear patterns of differential high gamma activity and temporal generalization of decoding for seen versus unseen stimuli, and the authors' interpretation of these results within the larger research literature on this topic. This study appears to have been carefully conducted, the data were analyzed appropriately, and the overall conclusions seem warranted given the main patterns of results.

Weaknesses include the saccadic reaction time results and the potential flaws in the design of the reporting task. As the authors acknowledge, this is not a "no report" paradigm, rather, it's a paradigm aimed at balancing the post-perceptual cognitive and motor requirements between the seen and unseen trials. On each trial, subjects/patients either perceived the stimulus or not, and had to briefly maintain this "yes/no" judgment until a fixation cross changed color, and the color change indicated how to respond (saccade to the left or right). Differences in saccadic RTs (measured from the time of the fixation color change to moving the eyes to the left or right response square) were evident between the seen and unseen trials (faster for seen). In the discussion, the authors summarize several alternative explanations of the saccade results and limitations of their report paradigm that will help guide future research.

The current results help advance our understanding of the contribution of PFC to visual awareness. These results, when situated within the larger context of the rapidly developing literature on this topic provide converging evidence that some sub-regions of PFC contribute to visual awareness, but at latencies earlier than originally predicted by proponents of, especially, global neuronal workspace theory. Three recent studies that used "no report paradigms", but with clearly visible stimuli, reported very similar results in PFC (Vishne et al., 2023; Broday-Dvir et al., 2023; Cogitate et al., 2023). The current study uses a report paradigm, but with consciously seen vs. unseen conditions, to fill the gap left by these previous studies, i.e., it remained unclear whether the PFC results from the previous studies were related to conscious or unconscious processing. Taken as a whole, evidence appears to be converging for a limited and early-in-time (200-300ms) contribution of PFC to visual awareness, after task and motor confounds are minimized.

https://doi.org/10.7554/eLife.89076.3.sa2

Reviewer #3 (Public Review):

The authors report a study in which they use intracranial recordings to dissociate subjectively aware and subjectively unaware stimuli, focusing mainly on prefrontal cortex.

The authors have dealt successfully with some of my previous concerns, especially the more direct link to the Gaillard et al., (2009) paper, and the associated analyses, has improved the manuscript. Some of my other concerns regarding the theoretical embedding of the findings have only been partially mitigated and some interesting results derived from suggestions for additional analyses will be used for future papers.

https://doi.org/10.7554/eLife.89076.3.sa3

Author response

The following is the authors’ response to the original reviews.

eLife assessment

This paper reports valuable results regarding the potential role and time course of the prefrontal cortex in conscious perception. Although the sample size is small, the results are clear and convincing, and strengths include the use of several complementary analysis methods. The behavioral test includes subject report so the results do not allow for distinguishing between theories of consciousness; nevertheless, results do advance our understanding of the contribution of prefrontal cortex to conscious perception.We appreciate very much for editor and reviewers encouraged review opinion. Particularly, we thank three reviewers very much for their professional and constructive comments that help us to improve the manuscript substantially.

Public Reviews:

Reviewer #1 (Public Review):

This is a clear and rigorous study of intracranial EEG signals in the prefrontal cortex during a visual awareness task. The results are convincing and worthwhile, and strengths include the use of several complementary analysis methods and clear results. The only methodological weakness is the relatively small sample size of only 6 participants compared to other studies in the field. Interpretation weaknesses that can easily be addressed are claims that their task removes the confound of report (it does not), and claims of primacy in showing early prefrontal cortical involvement in visual perception using intracranial EEG (several studies already have shown this). Also the shorter reaction times for perceived vs not perceived stimuli (confident vs not confident responses) has been described many times previously and is not a new result.

We appreciate very much for the reviewer’s encouraged opinion. We are going to address reviewer’s specific questions and comments point-by-point in following.

‘The only methodological weakness is the relatively small sample size of only 6 participants compared to other studies in the field.’

We agree that the sample size is relatively small in the present study. To compensate such shortcoming, we rigorously verified each result at both individual and population levels, resembling the data analysis method in non-human primate study.

Interpretation weaknesses that can easily be addressed are claims that their task removes the confound of report (it does not),

Thank you very much for your comment. We agree that our task does not remove the confound of report entirely. However, we believe that our task minimizes the motor confounds by dissociating the emergence of awareness from motor in time and balanced direction of motor between aware and unaware conditions. We have modified the text according to reviewer’s comment in the revised manuscript as following: “This task removes the confound of motor-related activity”.

..and claims of primacy in showing early prefrontal cortical involvement in visual perception using intracranial EEG (several studies already have shown this).

We agree that several iEEG studies, including ERP and HFA, have shown the early involvement of prefrontal cortical in visual perception. However, in these studies, the differential activity between conscious and unconscious conditions was not investigated, thus, the activity in prefrontal cortex might be correlated with unconscious processing, rather than conscious processing. In present study, we compared the neural activity in PFC between conscious and unconscious trials, and found the correlation between PFC activity and conscious perception. Although one iEEG study(Gaillard et al., 2009) reported awareness-specific PFC activation, the awareness-related activity started 300 ms after the onset of visual stimuli, which was ~100 ms later than the early awareness related activity in our study. Also, due to the limited number of electrodes in the previous study (2 patients with 19 recording sites mostly in mesiofrontal and peri-insular regions), it was restricted while exploring the awareness-related activity in PFC. In the present study, the number of recording sites (245) were much more than previous study and covered multiple areas in PFC. Our results further show earlier awareness-related activity (~ 200 ms after visual stimuli onset), including ERP, HFA and PLV, which sheds new light on understanding of the role of PFC in conscious perception.

We have added this discussion in the MS (lines 522-536);

Also the shorter reaction times for perceived vs not perceived stimuli (confident vs not confident responses) has been described many times previously and is not a new result.Thank you very much for your comment. We agree that the reaction time is strongly modulated by the confident level, which has been described previously (Broggin, Savazzi, & Marzi, 2012; Marzi, Mancini, Metitieri, & Savazzi, 2006). However, in previous studies, the confident levels were usually induced by presenting stimulus with different physical property, such as spatial frequency, eccentricity and contrast. It is well known that the more salient stimuli will induce the faster process of visual information and speed up the process of visuomotor transformation, eventually shorten the reaction time (Corbetta & Shulman, 2002; Posner & Petersen, 1990). Therefore, the dependence of visual processing on the salience of visual stimulus confounds with the effect of visual awareness on the reaction time, which is hard to attribute the shorter reaction time in more salient condition purely to visual awareness. In contrast, we create a condition (near perceptual threshold) in the present study, in which the saliency (contrast) of visual stimulus is very similar in both aware and unaware conditions in order to eliminate the influence of stimulus saliency in reaction time. We think that the difference in reaction time in our study is mainly due to the modulation of awareness state, which was not reported previously.

We have added the discussion in the MS (lines 497-507).

Reviewer #1 (Recommendations For The Authors):

Specific comments follow:

Abstract: "we designed a visual awareness task that can minimize report-related confounding" and in the Introduction lines 112-115: "Such a paradigm can effectively dissociate awareness-related activity from report-related activity in terms of time... and report behavior"; Discussion lines 481-483 "even after eliminating the influence of the confounding variables related to subjective reports such as motion preparation" and other similar statements in the manuscript should be removed. The task involves report using eye movements with every single stimulus. The fact that there is report for both perceived and not perceived stimuli, that the direction of report is not determined until the time of report, and that there is delay between stimulus and report, does not remove the report-related post-perceptual processing that will inevitably occur in a task where overt report is required for every single trial. For example, brain activity related to planning to report perception will only occur after perceived trials, regardless of the direction of eye movement later decided upon. This preparation to respond is different for perceived and not perceived stimuli, but is not part of the perception itself. In this way the current task is not at all unique and does not substantially differ from many other report-based tasks used previously.

The objective of present study is to assess whether PFC is involved in the emergence of visual awareness. To do so, it is crucial to determine the subjective awareness state as correct as possible. Considering the disadvantage of non-report paradigms in determining the subjective awareness state (Tsuchiya et al. TiCS, 2015; Mashour et al, Neuron, 2020), we employed a balanced report paradigm. It has been argued (Merten & Nieder, PNAS, 2011) that, in the balanced report paradigms, subjects could not prepare any motor response during the delay period because only the appearance of a rule cue (change color of fixation point at the end of delay period) informed subjects about the appropriate motor action. In this case, the post-perceptual processing during delay period might reflect the non-motor cognitive activity. Alternatively, as being mentioned by reviewer, the post-perceptual processing might relate to planning to report perception, which is different for perceived and not perceived stimuli. Therefore, up to date, the understanding of the post-perceptual processing remains controversial. According to reviewer’s comment, we have modified the description of our task as following: “we designed a visual awareness task that can minimize report-related motor confounding”. Also, have changed “report-related” to “motorrelated” in the text of manuscript.

Figures 3, 4 changes in posterior middle frontal gyri suggest early frontal eye field involvement in perception. This should be interpreted in the context of many previous studies showing FEF involvement in signal detection. The authors claim that "earlier visual awareness related activities in the prefrontal cortex were not found in previous iEEG studies, especially in the HG band" on lines 501-502 of the Discussion. This statement is not true and should be removed. The following statement in the Discussion on lines 563-564 should be removed for the same reasons: "our study detected 'ignition' in the human PFC for the first time." Authors should review and cite the following studies as precedent among others:

Blanke O, Morand S, Thut G, Michel CM, Spinelli L, Landis T, Seeck M (1999) Visual activity in the human frontal eye field. Neuroreport 10 (5):925-930. doi:10.1097/00001756-19990406000006

Foxe JJ, Simpson GV (2002) Flow of activation from V1 to frontal cortex in humans. A framework for defining "early" visual processing. Exp Brain Res 142 (1):139-150. doi:10.1007/s00221-001-0906-7

Gaillard R, Dehaene S, Adam C, Clemenceau S, Hasboun D, Baulac M, Cohen L, Naccache L (2009) Converging intracranial markers of conscious access. Plos Biology 7 (3):e61

Gregoriou GG, Gotts SJ, Zhou H, Desimone R (2009) High-frequency, long-range coupling between prefrontal and visual cortex during attention. Science 324:1207-1210

Herman WX, Smith RE, Kronemer SI, Watsky RE, Chen WC, Gober LM, Touloumes GJ, Khosla M, Raja A, Horien CL, Morse EC, Botta KL, Hirsch LJ, Alkawadri R, Gerrard JL, Spencer DD, Blumenfeld H (2019) A Switch and Wave of Neuronal Activity in the Cerebral Cortex During the First Second of Conscious Perception. Cereb Cortex 29 (2):461-474.

Khalaf A, Kronemer SI, Christison-Lagay K, Kwon H, Li J, Wu K, Blumenfeld H (2022) Early neural activity changes associated with stimulus detection during visual conscious perception. Cereb Cortex. doi:10.1093/cercor/bhac140

Kwon H, Kronemer SI, Christison-Lagay KL, Khalaf A, Li J, Ding JZ, Freedman NC,Blumenfeld H (2021) Early cortical signals in visual stimulus detection. Neuroimage 244:118608.

We agree that several iEEG studies, including ERP and HFA, have shown the early involvement of prefrontal cortical in visual perception. However, in these studies, the differential activity between conscious and unconscious conditions was not investigated, thus, the activity in prefrontal cortex might be correlated with unconscious processing, rather than conscious processing. In present study, we compared the neural activity in PFC between conscious and unconscious trials, and found the correlation between PFC activity and conscious perception. Although one iEEG study reported awareness-specific PFC activation, the awareness-related activity started 300 ms after the onset of visual stimuli, which was ~100 ms later than the early awareness related activity in our study. Also, due to the limited number of electrodes in the previous study (2 patients with 19 recording sites mostly in mesiofrontal and peri-insular regions), it was restricted while exploring the awareness-related activity in PFC. In the present study, the number of recording sites (245) were much more than previous study and covered multiple areas in PFC. Our results further show earlier awareness-related activity (~ 200 ms after visual stimuli onset), including ERP, HFA and PLV, which sheds new light on understanding of the role of PFC in conscious perception.

We have added this discussion in the MS (lines 522-533);

Minor weakness that should be mentioned in the Discussion: The intervals for the FP (fixation period) and Delay period were both fixed at 600 ms instead of randomly jittered, so that subjects likely had anticipatory activity predictably occurring with each grating and cue stimulus.

Thank you very much for your comment. We agree that subjects might have anticipatory activity during experiment. Actually, the goal for us to design the task in this way is to try to balance the effect of attention and anticipation between aware and unaware conditions. We have added this discussion in the MS (lines 467-469);

The faster reaction times for perceived/confident responses vs not perceived/unconfident responses has been reported many times previously in the literature and should be acknowledged rather than being claimed as a novel finding. Authors should modify p. 163 lines 160-162, first sentence of the Discussion lines 445-446 "reaction time.. shorter" claiming this was a novel finding; same for lines 464-467. Please see the following among others:

Broggin E, Savazzi S, Marzi CA (2012) Similar effects of visual perception and imagery on simple reaction time. Q J Exp Psychol (Hove) 65 (1):151-164. doi:10.1080/17470218.2011.594896

Chelazzi L, Marzi CA, Panozzo G, Pasqualini N, Tassinari G, Tomazzoli L (1988) Hemiretinal differences in speed of light detection in esotropic amblyopes. Vision Res 28 (1):95-104Marzi CA, Mancini F, Metitieri T, Savazzi S (2006) Retinal eccentricity effects on reaction time to imagined stimuli. Neuropsychologia 44 (8):1489-1495. doi:10.1016/j.neuropsychologia.2005.11.012

Posner MI (1994) Attention: the mechanisms of consciousness. Proceedings of the National Academy of Sciences of the United States of America 91 (16):7398-7403

Sternberg S (1969) Memory-scanning: mental processes revealed by reaction-time experiments. Am Sci 57 (4):421-457

Thanks. We have cited some of these papers in the revised manuscript due to the restricted number of citations.

Methods lines 658-659: "results under LU and HA conditions were classified as the control group and were only used to verify and check the results during calculation." However the authors show these results in the figures and they are interesting. HA stimuli show earlier responses than NA stimuli. This is a valuable result which should be discussed and interpreted in light of the other findings.

We thank very much for reviewer’s comment. We have made discussion accordingly in the revised MS (lines 535-536).

General comment on figures: Many of the figure elements are tiny and the text labels and details can't be seen at all, especially single trial color plots, and the brain insets showing recording sites.

We have modified the figures accordingly.

Other minor comments:Typo: Figure 2 legend, line 169 "The contrast level resulted in an awareness percentage greater than 25%..." is missing a word and should say instead something like "The contrast level that resulted in an awareness percentage greater than 25%..."

Thanks. We have corrected the typo accordingly.

Figure 2 Table description in text line 190 says "proportions of recording sites" but the Table only shows number of recording sites and number of subjects, not "proportions." This should be corrected in the text.

Thanks. We have corrected the error.

Figure 3, and other figures, should always label the left and right hemispheres to avoid ambiguity.

Thanks. We have made correction accordingly. In caption of Figure 2D (line 189), we modified the sentence as ‘In all brain images, right side of the image represents the right side of the brain’.

Methods line 666. The saccadic latency calculations paragraph should have a separate heading before it, to separate it from the Behavioral data analysis section.

Thanks. It has been corrected in line 725.

Reviewer #2 (Public Review):

The authors attempt to address a long-standing controversy in the study of the neural correlates of visual awareness, namely whether neurons in prefrontal cortex are necessarily involved in conscious perception. Several leading theories of consciousness propose a necessary role for (at least some sub-regions of) PFC in basic perceptual awareness (e.g., global neuronal workspace theory, higher order theories), while several other leading theories posit that much of the previously reported PFC contributions to perceptual awareness may have been confounded by task-based cognition that co-varied between the aware and unaware reports (e.g., recurrent processing theory, integrated information theory). By employing intracranial EEG in human patients and a threshold detection task on low-contrast visual stimuli, the authors assessed the timing and location of neural populations in PFC that are differentially activated by stimuli that are consciously perceived vs. not perceived. Overall, the reported results support the view that certain regions of PFC do contribute to visual awareness, but at time-points earlier than traditionally predicted by GNWT and HOTs.

Reply: We appreciate very much for the reviewer’s encouraged opinion.

Major strengths of this paper include the straightforward visual threshold detection task including the careful calibration of the stimuli and the separate set of healthy control subjects used for validation of the behavioral and eye tracking results, the high quality of the neural data in six epilepsy patients, the clear patterns of differential high gamma activity and temporal generalization of decoding for seen versus unseen stimuli, and the authors' interpretation of these results within the larger research literature on this topic. This study appears to have been carefully conducted, the data were analyzed appropriately, and the overall conclusions seem warranted given the main patterns of results.

Reply: We appreciate very much for the reviewer’s encouraged opinion.

Weaknesses include the saccadic reaction time results and the potential flaws in the design of the reporting task. This is not a "no report" paradigm, rather, it's a paradigm aimed at balancing the post-perceptual cognitive and motor requirements between the seen and unseen trials. On each trial, subjects/patients either perceived the stimulus or not, and had to briefly maintain this "yes/no" judgment until a fixation cross changed color, and the color change indicated how to respond (saccade to the left or right). Differences in saccadic RTs (measured from the time of the fixation color change to moving the eyes to the left or right response square) were evident between the seen and unseen trials (faster for seen). If the authors' design achieved what they claim on page 3, "the report behaviors were matched between the two awareness states ", then shouldn't we expect no differences in saccadic RTs between the aware and unaware conditions? The fact that there were such differences may indicate differences in post-perceptual cognition during the time between the stimulus and the response cue. Alternatively, the RT difference could reflect task-strategies used by subjects/patients to remember the response mapping rules between the perception and the color cue (e.g., if the YES+GREEN=RIGHT and YES+RED=LEFT rules were held in memory, while the NO mappings were inferred secondarily rather than being actively held in memory). This saccadic RT result should be better explained in the context of the goals of this particular reporting-task.

The objective of present study is to assess whether PFC is involved in the emergence of visual awareness. To do so, it is crucial to determine the subjective awareness state as correct as possible. Considering the disadvantage of non-report paradigms in determining the subjective awareness state (Tsuchiya et al, TiCS, 2015; Mashour et al, Neuron, 2020), we employed a balanced report paradigm. It has been argued (Merten & Nieder, PNAS, 2011) that, in the balanced report paradigms, subjects could not prepare any motor response during the delay period because only after the appearance of a rule cue (change color of fixation point at the end of delay period) subjects were informed about the appropriate motor action. In this case, the post-perceptual processing during delay period might reflect the non-motor cognitive activity, such as working memory (Mashour et al. Neuron, 2020). Alternatively, as being mentioned by reviewer, the postperceptual processing might relate to planning to report perception, which is different for perceived and not perceived stimuli (Aru et al. Neurosci Biobehav Rev, 2012 ). Therefore, up to date, the understanding of the post-perceptual processing remains controversial. Considering reviewer’s comment together with other opinions, we have modified the description of our task as following: “we designed a visual awareness task that can minimize report-related motor confounding”. Also, we have changed “report-related” to “motor-related” in the rest of manuscript.

Regarding the question whether the saccadic RT in our balanced response paradigm should be expected to be similar between aware and unaware condition, we think that the RT should be similar in case if the delay period is long enough for the decision of “no” to be completed. In fact, in a previous study (Merten & Nieder, PNAS, 2011), the neuronal encoding of “no” decision didn’t appear until 2s after the stimulus cue onset. However, in our task, the delay period lasted only 600 ms that was long enough to form the “yes” decision, but was not enough to form the “no” decision. It might be the reason that our data show shorter RT in aware condition than in unaware condition.

We totally agree reviewer’s comment about the alternative interpretation for RT difference between aware and unaware condition in our study, i.e., reflecting task-strategies used by subjects/patients to remember the response mapping rules between the perception and the color cue (e.g., if the YES+GREEN=RIGHT and YES+RED=LEFT rules were held in memory, while the NO mappings were inferred secondarily rather than being actively held in memory).We have made additional discussion about these questions in the revised manuscript (lines 492496).

Nevertheless, the current results do help advance our understanding of the contribution of PFC to visual awareness. These results, when situated within the larger context of the rapidly developing literature on this topic (using "no report" paradigms), e.g., the recent studies by Vishne et al. (2023) Cell Reports and the Cogitate consortium (2023) bioRxiv, provide converging evidence that some sub-regions of PFC contribute to visual awareness, but at latencies earlier than originally predicted by proponents of, especially, global neuronal workspace theory.

We appreciate very much for the reviewer’s encouraged opinion.

Reviewer #2 (Recommendations For The Authors):

Abstract: "the spatiotemporal overlap between the awareness-related activity and the interregional connectivity in PFC suggested that conscious access and phenomenal awareness may be closely coupled." I strongly suggest revising this sentence. The current results cannot be used to make such a broad claim about p-consciousness vs. a-consciousness. This study used a balanced trial-by-trial report paradigm, which can only measure conscious access.

We thank reviewer for this comment. We have withdrawn this sentence from the revised manuscript.

Task design: A very similar task was used previously by Schröder et al. (2021) J Neurosci. See specifically, their Figure 1, and Figure 4B-C. Using almost the exact same "matching task", the authors of this previous study show that they get a P3b for both the perceived and not-perceived conditions, confirming that post-perceptual cognition/report confounds were not eliminated, but instead were present in (and balanced between) both the perceived/not-perceived trials due to the delayed matching aspect of the design. This previous paper should be cited and the P3b result should be considered when assessing whether cognition/report confounds were addressed in the current study.

Thank you very much for your reminding about the study of Schröder et al. We are sorry for not citing this closely related study in our previous manuscript. Schröder et al. found while P3b showed significant difference between perceived and not-perceived trials in direct report task, the P3b was presented in both perceived/not-perceived trials and not significantly different in the matched task. Based on these findings, Schröder et al. argued that P3b represented the task specific post-perceptual cognition/report rather than the emergence of awareness per se. Considering the similarity of tasks between Schröder et al. and ours, we agree that our task is not able to totally eliminate the confound of post-perceptual cognition/report related activity with awareness related activity. Nevertheless, our task is able to minimize the confound of motorrelated activity with the emergence of awareness by separating them in time and balancing the direction of responsive movements. Therefore, we modified the term of “report-related” to “motor-related” in the text of revised manuscript.

On page 2, lines 71-75, the authors' review of the Frassle et al. (2014) experiment should be revised for accuracy. In this study, all PFC activity did not disappear as the authors claim. Also, the main contrast in the Frassle et al. study was rivalry vs. replay. However, in both of these conditions, visual awareness was changing with the main difference being whether there was sensory conflict between the two eyes or not. Such a contrast would presumably subtract out the common activity patterns related to visual awareness changes, while isolating rivalry (and the resulting neural competition) vs. non-rivalry (and the lack of such competition) which is not broadly relevant for the goal of measuring neural correlates of visual awareness which are present in both sides of the contrast (rivalry and replay).

Thank you very much for your suggestion. We agree that and revised in the MS (lines 71-76).

‘For instance, a functional magnetic resonance imaging (fMRI) study employing human binocular rivalry paradigms found that when subjects need to manually report the changing of their awareness between conflict visual stimuli, the frontal, parietal, and occipital lobes all exhibited awareness-related activity. However, when report was not required, awareness-related activation was largely diminished in the frontal lobe but remained in the occipital and parietal lobes’

On page 2, lines 76-78, the authors write, "no-report paradigm may overestimate unconscious processing because it cannot directly measure the awareness state". This should be reworded for clarity, as report paradigms also do not "directly measure the awareness state". All measures of awareness are indirect, either via subjects verbal or manual reports, or via behaviors or other physiological measures like OKN, pupillometry, etc. It's also not clear as written why no-report paradigms might overestimate unconscious processing.

Thank you very much for your suggestion. We agreed and modified the description.In lines 76-80:

‘Nevertheless, the no-report paradigm may overestimate the neural correlates of awareness by including unconscious processing, because it infers the awareness state through other relevant physiological indicators, such as optokinetic nystagmus and pupil size(Tsuchiya, Wilke, Frassle, & Lamme, 2015). In the absence of subjective reports, it remains controversial regarding whether the presented stimuli are truly seen or not.’

However, the no-report paradigm may overestimate the neural correlates of awareness, because it infers the awareness state through other relevant physiological indicators, such as optokinetic nystagmus and pupil size(Tsuchiya et al., 2015) , in the absence of subjective reports and it remains controversial that whether the stimuli presented in such paradigm are truly seen as opposed to being merely potentially visible but unattended.

On page 5, line 155, there is a typo. This should be Figure 2C, not 2B.

Thanks. We have modified the description.

On page 5, lines 160-162, the authors state, "The results showed that the saccadic reaction time in the aware trials was systematically shorter than that in the unaware trials. Such results demonstrate that visual awareness significantly affects the speed of information processing in the brain." I don't understand this. If subjects can never make a saccade until the fixation cross changes color, both for Y and N decisions, why would a difference in saccadic reaction times indicate anything about visual awareness affecting the speed of information processing in the brain? Doesn't this just show that the Red/Green x Left/Right response contingencies were easier to remember and execute for the Yes-I-did-see-it decisions compared to the No-I-didn't-see-it decisions?

We agree and have made additional discussion about these questions in the revised manuscript (lines 492-496).

‘An alternative interpretation for RT difference between aware and unaware condition in our study is that the difference in task-strategies used by subjects/patients to remember the response mapping rules between the perception and the color cue (e.g., if the YES+GREEN=RIGHT and YES+RED=LEFT rules were held in memory, while the NO mappings were inferred secondarily rather than being actively held in memory).’

In Figure 3B (and several other figures) due to the chosen view and particular brain visualization used, many readers will not know whether the front of brain is up and back of brain down or vise versa (there are no obvious landmarks like the cerebellum, temporal sulcus, etc.). I suggest specifying this in the caption or better yet on the figure itself.

Thanks. We have added these descriptions in the caption of Figure 2D.

Line 189 ‘In all brain images, right and up sides of each image represent the right and up sides of the brain’.

In Figure 3B, the color scale may confuse some readers. When I first inspected this figure, I immediately thought the red meant positive voltage or activation, while the blue meant negative voltage or deactivation. Only later, I realized that any color here is meaningful. Not sure if an adjustment of the color scale might help, or perhaps not normalizing (and not taking absolute values of the voltage diffs, but maintaining the +/- diffs)?

Thanks for reviewer’s comment. We are sorry for not clearly describing the reason why we normalized the activity in absolute value and chose the color scale from 0 to 20. The major reason is that it is not clearly understood so far regarding the biological characteristics of LFP polarity (Einevoll et al, Nat Rev Neurosci, 2013). To simplify such complex issue, we consider the change in magnitude of LFP during delay period in our task represents awareness related activity, regardless its actual value being positive or negative. Therefore, we first calculated the absolute value of activity difference between aware and unaware trials in individual recording site, then used Shepard's method (see Method for detailed information) to calculate the activity in each vertex and projected on the surface of brain template as shown in Fig. 3B.

We have added the description in the MS (lines 794-800).

We have tried to adjust the color scale from -20 to 20 according to reviewer’s suggestion. However, the topographic heatmap showed less distinguishable between brain regions with different strength of awareness related activity. Thus, we would like to keep the way as we used to analyze and present these results.

Figure 3B: Why choose seemingly arbitrary time points in this figure? What's the significance of 247 and 314 and 381ms (why not show 200, 250, 300, etc.)? Also, are these single time-points or averages within a broader time window around this time-point, e.g., 225-275ms for the 250ms plot?

Thank reviewer for this helpful comment. We are sorry for not clearly describing why we chose the 8 time points to demonstrate the spatiotemporal characteristics of awareness related activity in Fig. 3B. To identify the awareness related activity, we analyzed the activity difference between aware and unaware trials during delay period (180-650 ms after visual stimulus onset). The whole dynamic process has been presented in SI with a video (video S1). Here, we just sampled the activity at 8 time points (180 ms, 247 ms, 314 ms, etc.) that equally divided the 430 ms delay period.

We have added the description in the MS (lines 213-215).

Figure 3D: It's not clear how this figure panel is related to the data shown in Fig3A. In Fig3A, the positive amplitude diffs all end at around 400ms, but in Fig3D, these diffs extend out to 600+ms. I suggest adding clarity about the conversion being used here.

Thanks for reviewer’s comment. We are sorry for not clearly describing the way to analyze the population activity (Fig. 3D) in the previous version of manuscript. Since it is not clearly understood so far regarding the biological characteristics of LFP polarity, to simplify such complex issue, we consider the change in magnitude of LFP during delay period in our task is awareness related activity, regardless its actual value being positive or negative. Therefore, while analyzing the awareness related population activity, we first calculate the absolute value of activity difference between aware and unaware trials in individual recording site, then pool the data of 43 recording sites together and calculate the mean and standard error of mean (SEM)(Fig. 3D). As you can see in Fig. 3A, the activity difference between aware (red) and unaware (blue) trials lasts until/after the end of delay period. Thus, the awareness related population activity in Fig 3D extends out to 600 ms.

We have added the description in the MS (lines 769-777).

Figure 6D could be improved by making the time labels much bigger, perhaps putting them on the time axis on the bottom rather than in tiny text above each brain.

Thanks for reviewer’s comment. We have modified it accordingly.

Page 18, line 480: "our results show that the prefrontal cortex still displays visual awareness-related activities even after eliminating the influence of the confounding variables related to subjective reports such as motion preparation" This is too strong of a statement. It's not at all clear whether confounding variables related to subjective reports (especially the cognition needed to hold in mind the Y/N decision about seeing the stimulus prior to the response cue) were eliminated with the design used here. In other places of the manuscript, the authors use "minimized" which is more accurate.

Thanks for reviewer’s comment. We have modified it accordingly.

Page 19, section starting on line 508: The authors should consider citing the study by Vishne et al. (2023), which was just accepted for publication recently, but has been posted on bioRxiv for almost a year now: https://www.biorxiv.org/content/10.1101/2022.08.02.502469v1 . And on page 20, line 563, the authors claim that to the best of their knowledge, they were the first to detect "ignition" in PFC in human subjects. Consider revising this statement, now that you know about the Vishne et al. paper.

We agree.

Thanks for your reminding about these papers. We have cited this study and made discussion in the revised manuscript (line 522-533). We agree that several iEEG studies have shown the early involvement of PFC in visual perception (Vishne et al. 2023; Khalaf et al. 2023; Kwon et al. 2021). However, in these studies, authors did not compare the neural activity between conscious and unconscious conditions, leaving the possibility that the ERP and HFA were correlated with the unconscious information processing rather than awareness-specific processing. In the present study, we compared the neural activity in PFC between conscious and unconscious trials, and found that the activity of PFC specifically correlated with conscious perception. As we mentioned in the previous version of manuscript, there is one iEEG study (Gaillard et al. 2009) that reported awareness-specific activity in PFC. However, the awareness related activity started more than 300 ms after the onset of visual stimuli, which was about 100 ms longer than the early awareness related activity in our study. Nevertheless, according to reviewer’s comment, we modified our argument as following in lines 621-623:

‘However, as discussed above, in contrast with previous studies, our study detected earlier awareness-specific ‘ignition’ in the human PFC, while minimizing the motor-related confounding.’

Experimental task section of Methods: Were any strategies for learning the response cue matching task suggested to patients/subjects, and/or did any patients/subjects report which strategy they ended up using? For example, if I were a subject in this experiment, I would remember and mentally rehearse the rules: "YES+GREEN = RIGHT" and "YES+RED = LEFT". For trials in which I didn't see anything, I wouldn't need to hold 2 more rules in mind, as they can be inferred from the inverse of the YES rules (and it's much harder to hold 4 things in mind than 2). This extra inference needed to get to the NO+GREEN = LEFT and NO+RED = RIGHT rules would likely cause me to respond slightly slower to the NO trials compared to the YES trials, leading to saccadic RT effects in the same direction the authors found. More information about the task training and strategies used by patients/subjects would be helpful.

We agree and discussed this in lines 492-496.

Reviewer #3 (Public Review):

The authors report a study in which they use intracranial recordings to dissociate subjectively aware and subjectively unaware stimuli, focusing mainly on prefrontal cortex. Although this paper reports some interesting findings (the videos are very nice and informative!) the interpretation of the data is unfortunately problematic for several reasons. I will detail my main comments below. If the authors address these comments well, I believe the paper may provide an interesting contribution to further specifying the neural mechanisms important for conscious access (in line with Gaillard et al., Plos Biology 2009).

Reply: We appreciate very much for the reviewer’s encouraged opinion.

The main problem with the interpretation of the data is that the authors have NOT used a so called "no-report paradigm". The idea of no report paradigms is that subjects passively view a certain stimulus without the instruction to "do something with it", e.g., detect the stimulus, immediately or later in time. Because of the confusion of this term, specifically being related to the "act of reporting", some have argued we should use the term no-cognition paradigm instead (Block, TiCS, 2019, see also Pitts et al., Phil Trans B 2018). The crucial aspect is that, in these types of paradigms, the critical stimulus should be task-irrelevant and thus not be associated with any task (immediately or later). Because in this experiment subjects were instructed to detect the gratings when cued 600 ms later in time, the stimuli are task relevant, they have to be reported about later and therefore trigger all kinds of (known and potentially unknown) cognitive processes at the moment the stimuli are detected in real-time (so stimulus-locked). You could argue that the setup of this delayed response task excludes some very specific report related processes (e.g., the preparation of an eye-movement), which is good, however this is usually not considered the main issue. For example when comparing masked versus unmasked stimuli (Gaillard et al., 2009 Plos Biology), these conditions usually also both contain responses but these response related processes are "averaged out" in the specific contrasts (unmasked > masked). In this paper, RT differences between conditions (that are present in this dataset) are taken care of by using this delayed response in this paper, which is a nice feature for that and is not the case for the above example set-up.

Given the task instructions, and this being merely a delayed-response task, it is to be expected that prefrontal cortex shows stronger activity for subjectively aware versus subjectively unaware stimuli. Unfortunately, given the nature of this task, the novelty of the findings is severely reduced. The authors cannot claim that prefrontal cortex is associated with "visual awareness", or what people have called phenomenal consciousness (this is the goal of using no-cognition paradigms). The only conclusion that can be drawn is that prefrontal cortex activity is associated with accessing sensory input: and hence conscious access. This less novel observation has been shown many times before and there is also little disagreement about this issue between different theories of consciousness (e.g., global workspace theory and local recurrency theories both agree on this).

We totally agree that the no-report/no-cognition paradigms contain less cognition within the post-perceptual processing than the report paradigms. We designed the balanced response task in order to minimize the motor related component from post-perceptual processing, even though this task does not eliminate the entire cognition from post-perceptual processing. Regarding reviewer’s comment that our task is not able to assess the involvement of PFC in the emergence of awareness, we have different opinion. As we mentioned in the manuscript, the findings of early awareness related activity (~200 ms) in PFC, which resemble the VAN activity in EEG studies, indicate the association of PFC with the emergence of visual awareness (phenomenal consciousness).

The best solution at this point seems to rewrite the paper entirely in light of this. My advice would be to state in the introduction that the authors investigate conscious access using iEEG and then not refer too much to no-cognition paradigm or maybe highlight some different strategies about using task-irrelevant stimuli (see Canales-Johnson et al., Plos Biology 2023; Hesse et al., eLife 2020; Hatamimajoumerd et al Curr Bio 2022; Alilovic et al., Plos Biology 2023; Pitts et al., Frontiers 2014; Dwarakanth et al., Neuron 2023 and more). Obviously, the authors should then also not claim that their results solve debates about theories regarding visual awareness (in the "no-cognition" sense, or phenomenal consciousness), for example in relation to the debate about the "front or the back of the brain", because the data do not inform that discussion. Basically, the authors can just discuss their results in detail (related to timing, frequency, synchronization etc) and relate the different signatures that they have observed to conscious access.

The objective of present study is to assess whether PFC is involved in the emergence of visual awareness (i.e., phenomenal consciousness). Interestingly, we found the early awareness related activity (~200 ms after visual stimulus onset), including ERP, high gamma activity and phase synchronization, in PFC, which indicate the association of PFC with the emergence of visual awareness. Therefore, we would like to keep the basic context of manuscript and make revision according to reviewers’ comments.

On the other hand, we totally agree reviewer’s argument that the report paradigm is more suitable to study the access consciousness. Indeed, we have found that the awareness related activity in PFC could be separated into two subgroups, i.e., early activity with shorter latency (~200 ms after stimulus onset) and late activity with longer latency (> 350 ms after stimulus onset). In addition, the early activity was declined to the baseline level within ~200 ms during delay period, whereas the late activity lasted throughout the delay period and reached to the next stage of task (change color of the fixation point). Moreover, the early activity occurs primarily within the contralateral PFC of the visual stimulus, whereas the late activity occurs within both contralateral and ipsilateral PFC. While the early awareness related activity resembles the VAN activity in EEG studies (associating with p-consciousness), the late awareness related activity resembles the P3b activity (associating with a-consciousness). We are going to report these results in a separated paper soon.

I think the authors have to discuss the Gaillard et al PLOS Biology 2009 paper in much more detail. Gaillard et al also report a study related to conscious access contrasting unmasked and masked stimuli using iEEG. In this paper they also report ERP, time frequency and phase synchronization results (and even Granger causality). Because of the similarities in approach, I think it would be important to directly compare the results presented in that paper with results presented here and highlight the commonalities and discrepancies in the Discussion.

Thanks for reviewer’s comment. We have made additional analysis and detailed discussion accordingly. In addition, we also extended discussion with other relevant studies in the revised manuscript.

In lines 528-549,

‘Although one iEEG study reported awareness-specific PFC activation, the awareness-related activity started 300 ms after the onset of visual stimuli, which was ~100 ms later than the early activity in our study. Also, due to the limited number of electrodes in PFC (2 patients with 19 recording sites mostly in mesiofrontal and peri-insular regions), their experiments were restricted while exploring the awareness-related activity in PFC. In the present study, the number of recording sites (245) were much more than previous study and covered more areas in PFC. Our results further show earlier awareness-related activity (~ 200 ms after visual stimuli onset), including ERP, HFA and PLV. These awareness-related activity in PFC occurred even earlier (~150 ms after stimulus onset) for the salient stimulus trials (Fig. 3A\D and Fig. 4A\D, HA condition).

However, the proportions are much smaller than that reported by Gaillard et al, which peaked at ~60%. We think that one possibility for the difference may be due to the more sampled PFC subregions in present study and the uneven distribution of awareness-related activity in PFC. Meanwhile, we noticed that the peri-insula regions and middle frontal gyrus (MFG), which were similar with the regions reported by Gaillard et al, seemed to show more fraction of awarenessrelated sites than other subregions during the delay period (0-650 ms after stimulus onset). To test such possibility and make comparison with the study of Gaillard et al. we calculated the proportion of awareness-related site in peri-insula and MFG regions. We found although the proportion of awareness-related site was larger in peri-insula and MFG than in other subregions, it was much lower than the report of Gaillard et al. One alternative possibility for the difference between these two studies might be due to the more complex task in Gaillard et al. Nevertheless, we think these new results would contribute to our understanding of the neural mechanism underlying conscious perception, especially for the role of PFC.’ In lines 601-603:

‘The only human iEEG study reported that the phase synchronization of the beta band in the aware condition also occurred relatively late (> 300 ms) and mainly confined to posterior zones but not PFC.’

As for the Granger Causality analysis between PFC and occipital lobe, while the aim of this study focused mainly on PFC and there were few recoding sites in occipital lobe, we would like to do this analysis in later studies after we collect more data.

In the Gaillard paper they report a figure plotting the percentage of significant frontal electrodes across time (figure 4A) in which it can be seen that significant electrodes emerge after approximately 250 ms in PFC as well. It would be great if the authors could make a similar figure to compare results. In the current paper there are much more frontal electrode contacts than in the Gaillard paper, so that is interesting in itself.

Thanks reviewer for this constructive comment. We made similar analysis as Gaillard et al. and plotted the results in the figure bellow. As you can see, the awareness related sites started to emerge about 200 ms after visual stimulus onset according to both ERP and HG activity. The proportion of awareness related sites reached peak at ~14% (8% for HG) in 300-400ms. However, the proportions are much smaller than that reported by Gaillard et al, which peaked at ~60%. We think that one possibility for the difference may be due to the more sampled PFC subregions in present study and the uneven distribution of awareness-related activity in PFC. Meanwhile, we noticed that the peri-insula regions and middle frontal gyrus (MFG), which were similar with the regions reported by Gaillard et al, seemed to show more fraction of awareness-related sites than other subregions during the delay period (0-650 ms after stimulus onset). To test such possibility and make comparison with the study of Gaillard et al. we calculated the proportion of awareness-related site in peri-insula and MFG regions. We found although the proportion of awareness-related site was larger in peri-insula and MFG than in other subregions, it was much lower than the report of Gaillard et al. One alternative possibility for the difference between these two studies might be due to the more complex task in Gaillard et al.

We have added this figure and discussion to the revised manuscript as a new result (Figure 4E & S2 and lines 537-549).

Author response image 1
Percentage of awareness-related sites in ERP and HG analysis.

n, number of recording sites in PFC.

Author response image 2
Percentage of awareness-related sites in ERP and HG analysis at parsopercularis and middle frontal gyrus (MFG).

n, number of recording sites.

In my opinion, some of the most interesting results are not highlighted: the findings that subjectively unaware stimuli show increased activations in the prefrontal cortex as compared to stimulus absent trials (e.g., Figure 4D). Previous work has shown PFC activations to masked stimuli (e.g., van Gaal et al., J Neuroscience 2008, 2010; Lau and Passigngham J Neurosci 2007) as well as PFC activations to subjectively unaware stimuli (e.g., King, Pescetelli, and Dehaene, Neuron 2016) and this is a very nice illustration of that with methods having more detailed spatial precision. Although potentially interesting, I wonder about the objective detection performance of the stimuli in this task. So please report objective detection performance for the patients and the healthy subjects, using signal detection theoretic d'. This gives the reader an idea of how good subjects were in detecting the presence/absence of the gratings. Likely, this reveals far above chance detection performance and in that case I would interpret these findings as "PFC activation to stimuli indicated as subjectively unaware" and not unconscious stimuli. See Stein et al., Plos Biology 2021 for a direct comparison of subjectively and objectively unaware stimuli.

We gratefully appreciate for reviewer’s helpful and valuable comments. We do notice that the activity of PFC in subjectively unawareness condition (stimulus contrast near perceptual threshold) is significantly higher than stimulus absent condition. Such results, by using sEEG recordings with much higher spatial resolution than brain imaging and scalp EEG, support findings of previous studies (citations). Considering the question of neural correlation of unawareness processing is a hot and interesting topic, after carefully considering, we would like to report these results in a separate paper, rather than add these results in the current manuscript in order to avoid the distraction.

According to reviewer’s comment about the objective detection performance of the stimuli in our task, we analyzed the signal detection theoretic d’. The values of d’ in patients and healthy subjects are similar (1.81±0.27 in patients and 2.12±0.37 in healthy subjects). Such results indicate that the objective detection performance of subjects in our task is well above the chance level. Since our task merely measures the subjective awareness, we agree reviewer’s comment about the interpretation of our results as “PFC activation to stimuli indicated the subjective unawareness rather than objective unawareness”. We will emphasize this point in our next paper.

We have added the d prime in the MS (lines149-150).

In Figure 7 of the paper the authors want to make the case that the contrast does not differ between subjectively aware stimuli and subjectively unaware stimuli. However so far they've done the majority of their analyses across subjects, and for this analysis the authors only performed within-subject tests, which is not a fair comparison imo. Because several P values are very close to significance I anticipate that a test across subjects will clearly show that the contrast level of the subjectively aware stimuli is higher than of the subjectively unaware stimuli, at the group level. A solution to this would be to sub-select trials from one condition (NA) to match the contrast of the other condition (NU), and thereby create two conditions that are matched in contrast levels of the stimuli included. Then do all the analyses on the matched conditions.

Thank reviewer for the helpful comment. Regarding reviewer’s comment “However so far they've done the majority of their analyses across subjects, and for this analysis the authors only performed within-subject tests, which is not a fair comparison imo”, if we understand correctly, reviewer considered that it was fair if the analysis of neural activity in PFC was done across subjects but the stimulus contrast analysis between NA and NU was done individually. Actually, it is not the case. In neural activity analysis, the significant awareness-related sites were identified firstly in each individual subject (Fig. 3A and Fig 4A, and Methods), same as the analysis of stimulus contrast (see Methods). Only in the neural population activity analysis, the activity of awareness-related sites was pooled together and made further analysis.

To further evidence the awareness related activity in PFC is not highly correlated with stimulus contrast, we compared the activity difference between two different stimulus contrast conditions, i.e., stimulus contrast difference between high-contrast aware (HA) and NA conditions (large difference, ~14%), and between NA and NU conditions (slight difference, ~0.2%). The working hypothesis is that, if PFC activity is closely correlated with the contrast of stimulus contrast, we expect to see the activity difference between HA and NA conditions is much larger than that between NA and NU conditions. To test this hypothesis, we analyzed data of two patients in which the previous analysis showed significant or near significant difference of stimulus contrast between NA and NU conditions (Author response image 1, below, patient #2 and 1). The results (Author response image 1) show that the averaged activity difference (0-650 ms after visual stimulus onset) between HA and NA was similar as the averaged activity difference between NA and NU trials, even though the stimulus contrast difference was much larger between HA and NA conditions than between NA and NU conditions. Such results indicate that the awareness-related activity in PFC cannot be solely explained by the contrast difference between NA and NU conditions. Based on these results, we think that it is not necessary to perform the analysis as reviewer’s comment “A solution to this would be to sub-select trials from one condition (NA) to match the contrast of the other condition (NU), and thereby create two conditions that are matched in contrast levels of the stimuli included. Then do all the analyses on the matched conditions”. Another reason that impedes us to do this analysis is due to the limited trial numbers in our dataset.

Author response image 3
Relationship between stimulus contract and PFC activity.

X axis represents the stimulus contrast difference between two paired conditions, i.e., aware versus unaware in near perceptual threshold conditions (NA – NU, red dots); aware in high contrast condition versus aware in near perceptual threshold condition (HA – NA, blue dots). Y axis represents the activity difference between paired stimulus conditions. The results show that activity difference is similar between two paired conditions regardless the remarkable contrast difference between two paired conditions. Such results indicate that the greater activity in NA trials than in NU trials (Fig. xx-xx) could not be interpreted by the slight difference in stimulus contrast between NA and NU trials.

Related, Figure 7B is confusing and the results are puzzling. Why is there such a strong below chance decoding on the diagonal? (also even before stimulus onset) Please clarify the goal and approach of this analysis and also discuss/explain better what they mean.

We have withdrawn Figure7B for the confusing decoding results on the diagonal.

I was somewhat surprised by several statements in the paper and it felt that the authors may not be aware of several intricacies in the field of consciousness. For example, a statement like the following "Consciousness, as a high-level cognitive function of the brain, should have some similar effects as other cognitive functions on behavior (for example, saccadic reaction time). With this question in mind, we carefully searched the literature about the relationship between consciousness and behavior; surprisingly, we failed to find any relevant literature." This is rather problematic for at least two reasons. First, not everyone would agree that consciousness is a highlevel cognitive function and second there are many papers arguing for a certain relationship between consciousness and behavior (Dehaene and Naccache, 2001 Cognition; van Gaal et al., 2012, Frontiers in Neuroscience; Block 1995, BBS; Lamme, Frontiers in Psychology, 2020; Seth, 2008 and many more). Further, the explanation for the reaction time differences in this specific case is likely related to the fact that subjects' confidence in that decision is much higher in the aware trials than in the unaware trials, hence the speeded response for the first. This is a phenomenon that is often observed if one explores the "confidence literature". Although the authors have not measured confidence I would not make too much out of this RT difference.

We agree that and modified accordingly in lines 492-507.

‘An alternative interpretation for RT difference between aware and unaware condition in our study, i.e., reflecting task-strategies used by subjects/patients to remember the response mapping rules between the perception and the color cue (e.g., if the YES+GREEN=RIGHT and YES+RED=LEFT rules were held in memory, while the NO mappings were inferred secondarily rather than being actively held in memory).

Another possibility is that the reaction time is strongly modulated by the confident level, which has been described in previous studies(Broggin et al., 2012; Marzi et al., 2006). However, in previous studies, the confident levels were usually induced by presenting stimulus with different physical property, such as spatial frequency, eccentricity and contrast. However, the dependence of visual process on the salience of visual stimulus confounds with the effect of visual awareness on the reaction time of responsive movements, which is hard to attribute the shorter reaction time in more salient condition purely to visual awareness. In contrast, we create a condition (near aware threshold) in the present study, in which the saliency (contrast) of visual stimulus is very similar in both aware and unaware conditions in order to eliminate the influence of stimulus saliency in reaction time. We think that the difference in reaction time in our study is mainly due to the modulation of awareness state, which was not reported previously.’

I would be interested in a lateralized analysis, in which the authors compare the PFC responses and connectivity profiles using PLV as a factor of stimulus location (thus comparing electrodes contralateral to the presented stimulus and electrodes ipsilateral to the presented stimulus). If possible this may give interesting insights in the mechanism of global ignition (global broadcasting), supposing that for contralateral electrodes information does not have to cross from one hemisphere to another, whereas for ipsilateral electrodes that is the case (which may take time). Gaillard et al refer to this issue as well in their paper, and this issue is sometimes discussed regarding to Global workspace theory. This would add novelty to the findings of the paper in my opinion.

We gratefully appreciate reviewer’s helpful and available suggestions. We have made the analysis accordingly. We find that the awareness-related ERP activation in PFC occurs earlier only in the contralateral PFC with latency about 200 ms and then occurs in both contralateral and ipsilateral PFC about 100 ms later. In addition, the magnitude of awareness-related activity is stronger in the contralateral PFC than in ipsilateral PFC during the early phase (200-400 ms), then the activity becomes similar between contralateral and ipsilateral PFC. Moreover, the awareness related HG activity only appears in the contralateral PFC. Such results show the spatiotemporal characteristics of visual awareness related activity between two hemispheres. We are going to report these results in a separate paper soon.

Reviewer #3 (Recommendations For The Authors):

Some of the font sizes in the figures are too small.

We have modified accordingly.

To me, the abbreviations are confusing, (NA/NU etc). I would try to come up with easier ones or just not use abbreviations.

We have modified accordingly and try to avoid to use the abbreviations.

The data/scripts availability statement states "available upon reasonable request". I would suggest that the authors make the data openly available when possible, and I believe eLife requires that as well.

Thanks for reviewer’s suggestions. Due to several ongoing studies based on this dataset, we would like to open our data after complete these studies if there is no restriction from national policy.

https://doi.org/10.7554/eLife.89076.3.sa4

Article and author information

Author details

  1. Zepeng Fang

    State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Division of Psychology, Beijing Normal University, Beijing, China
    Contribution
    Conceptualization, Resources, Data curation, Software, Formal analysis, Validation, Investigation, Visualization, Methodology, Writing – original draft, Writing – review and editing
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-0396-6463
  2. Yuanyuan Dang

    Department of Neurosurgery, Chinese PLA General Hospital, Beijing, China
    Contribution
    Resources, Data curation, Investigation, Methodology, Writing – original draft
    Competing interests
    No competing interests declared
  3. Zhipei Ling

    Department of Neurosurgery, Chinese PLA General Hospital, Beijing, China
    Contribution
    Resources, Investigation, Methodology, Project administration
    Competing interests
    No competing interests declared
  4. Yongzheng Han

    Department of Anesthesiology, Peking University Third Hospital, Beijing, China
    Contribution
    Data curation, Investigation, Methodology
    Competing interests
    No competing interests declared
  5. Hulin Zhao

    Department of Neurosurgery, Chinese PLA General Hospital, Beijing, China
    Contribution
    Resources, Data curation, Supervision, Investigation, Methodology, Writing – original draft, Project administration, Writing – review and editing
    For correspondence
    zhaohulin_90@sohu.com
    Competing interests
    No competing interests declared
  6. Xin Xu

    Department of Neurosurgery, Chinese PLA General Hospital, Beijing, China
    Contribution
    Resources, Data curation, Supervision, Investigation, Methodology, Writing – original draft, Project administration, Writing – review and editing
    For correspondence
    xuxinmm@hotmail.com
    Competing interests
    No competing interests declared
  7. Mingsha Zhang

    State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Division of Psychology, Beijing Normal University, Beijing, China
    Contribution
    Conceptualization, Resources, Formal analysis, Supervision, Funding acquisition, Validation, Methodology, Writing – original draft, Project administration, Writing – review and editing
    For correspondence
    mingsha.zhang@bnu.edu.cn
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-5407-7770

Funding

Ministry of Science and Technology of the People's Republic of China (STI2030-Major Projects+2021ZD0204300)

  • Mingsha Zhang

National Natural Science Foundation of China (32061143004)

  • Mingsha Zhang

National Natural Science Foundation of China (32030045)

  • Mingsha Zhang

State Key Laboratory of Cognitive Neuroscience and Learning (Open Research Fund CNLZD2202)

  • Yongzheng Han

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Acknowledgements

We thank the participants for volunteering to take part in the study. This study was funded by STI2030-Major Projects +2021ZD0204300. National Natural Science Foundation of China (32061143004, 32030045). Funded by Open Research Fund of the State Key Laboratory of Cognitive Neuroscience and Learning (CNLZD2202).

Ethics

All subjects provided informed consent to participate in this study. The Ethics Committee of Chinese PLA General Hospital approved the experimental procedures (approval numbers S2022-457-01).

Senior Editor

  1. Floris P de Lange, Donders Institute for Brain, Cognition and Behaviour, Netherlands

Reviewing Editor

  1. Marius V Peelen, Radboud University, Netherlands

Version history

  1. Sent for peer review: May 30, 2023
  2. Preprint posted: June 9, 2023 (view preprint)
  3. Preprint posted: August 10, 2023 (view preprint)
  4. Preprint posted: December 28, 2023 (view preprint)
  5. Version of Record published: January 24, 2024 (version 1)

Cite all versions

You can cite all versions using the DOI https://doi.org/10.7554/eLife.89076. This DOI represents all versions, and will always resolve to the latest one.

Copyright

© 2023, Fang et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 209
    Page views
  • 47
    Downloads
  • 0
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Zepeng Fang
  2. Yuanyuan Dang
  3. Zhipei Ling
  4. Yongzheng Han
  5. Hulin Zhao
  6. Xin Xu
  7. Mingsha Zhang
(2024)
The involvement of the human prefrontal cortex in the emergence of visual awareness
eLife 12:RP89076.
https://doi.org/10.7554/eLife.89076.3

Share this article

https://doi.org/10.7554/eLife.89076

Further reading

    1. Neuroscience
    Eyal Y Kimchi, Anthony Burgos-Robles ... Kay M Tye
    Research Article

    Basal forebrain cholinergic neurons modulate how organisms process and respond to environmental stimuli through impacts on arousal, attention, and memory. It is unknown, however, whether basal forebrain cholinergic neurons are directly involved in conditioned behavior, independent of secondary roles in the processing of external stimuli. Using fluorescent imaging, we found that cholinergic neurons are active during behavioral responding for a reward – even prior to reward delivery and in the absence of discrete stimuli. Photostimulation of basal forebrain cholinergic neurons, or their terminals in the basolateral amygdala (BLA), selectively promoted conditioned responding (licking), but not unconditioned behavior nor innate motor outputs. In vivo electrophysiological recordings during cholinergic photostimulation revealed reward-contingency-dependent suppression of BLA neural activity, but not prefrontal cortex. Finally, ex vivo experiments demonstrated that photostimulation of cholinergic terminals suppressed BLA projection neuron activity via monosynaptic muscarinic receptor signaling, while also facilitating firing in BLA GABAergic interneurons. Taken together, we show that the neural and behavioral effects of basal forebrain cholinergic activation are modulated by reward contingency in a target-specific manner.

    1. Neuroscience
    Olgerta Asko, Alejandro Omar Blenkmann ... Anne-Kristin Solbakk
    Research Article Updated

    Orbitofrontal cortex (OFC) is classically linked to inhibitory control, emotion regulation, and reward processing. Recent perspectives propose that the OFC also generates predictions about perceptual events, actions, and their outcomes. We tested the role of the OFC in detecting violations of prediction at two levels of abstraction (i.e., hierarchical predictive processing) by studying the event-related potentials (ERPs) of patients with focal OFC lesions (n = 12) and healthy controls (n = 14) while they detected deviant sequences of tones in a local–global paradigm. The structural regularities of the tones were controlled at two hierarchical levels by rules defined at a local (i.e., between tones within sequences) and at a global (i.e., between sequences) level. In OFC patients, ERPs elicited by standard tones were unaffected at both local and global levels compared to controls. However, patients showed an attenuated mismatch negativity (MMN) and P3a to local prediction violation, as well as a diminished MMN followed by a delayed P3a to the combined local and global level prediction violation. The subsequent P3b component to conditions involving violations of prediction at the level of global rules was preserved in the OFC group. Comparable effects were absent in patients with lesions restricted to the lateral PFC, which lends a degree of anatomical specificity to the altered predictive processing resulting from OFC lesion. Overall, the altered magnitudes and time courses of MMN/P3a responses after lesions to the OFC indicate that the neural correlates of detection of auditory regularity violation are impacted at two hierarchical levels of rule abstraction.