To detect the effect of cue stimulus (grating) contrast on the emergence of visual awareness, we first analyzed the proportion of participants being aware of the grating as a function of its contrast. Figure 2A shows exemplified sessions from a patient and healthy subject, and Figure 2B shows the population results of the two participant groups. The results showed that with increasing grating contrast, the ratio of reporting ‘awareness’ of the grating gradually increased for all participants (including patients and healthy subjects), showing a classical psychometric curve. Such results showed that the grating contrast level had a direct impact on the emergence of visual awareness and thus is a reliable way to study visual awareness. In addition, in trials with no grating and grating contrast well above the perceptual threshold, subjects showed high accuracy (patients 94.75%±2.65; healthy subjects, 96.80%±0.71, mean ± SEM) and sensitivity (d’=1.81 ± 0.27 for patients and 2.12±0.37 for healthy subjects), indicating that the subjects understood and performed the task well following the task rules.

Figure 2

Download asset Open asset

Figure 2—source code 1

Source code files for generating the results in Figure 2.

https://cdn.elifesciences.org/articles/89076/elife-89076-fig2-code1-v1.zip

Download elife-89076-fig2-code1-v1.zip

To explore whether visual awareness affects behavioral performance, we divided the trials into 4 groups according to the level of grating contrast and the participants’ reported awareness state, that is high-contrast aware (HA), near-threshold-contrast aware (NA), near-threshold-contrast unaware (NU), and low-contrast unaware (LU) (Figure 2A, also see Methods). We compared the saccadic parameters between the awareness and unawareness states in near-threshold contrast levels (NA vs. NU), as well as in high and low contrast levels (HA vs. LU), as a control. While saccadic reaction time showed a significant difference between the two awareness states (Figure 2B), the other parameters did not. (Figure 2C) shows the mean saccadic reaction time of patients and healthy subjects under different awareness states and contrast levels. The results showed that the average saccadic reaction time was significantly shorter under the awareness state than under the unawareness state at both the near-threshold contrast level and the high-low contrast level (p<0.05, Wilcoxon signed rank test), and it was also explicit and consistent at the individual level. The results showed that the saccadic reaction time in the aware trials was systematically shorter than that in the unaware trials. Such results demonstrate that visual awareness significantly affects the speed of information processing in the brain.

While the patients were performing our visual awareness task, we recorded the local field potential (LFP) from 901 recording sites (Figure 2D, projected on Montreal Neurological Institute (MNI) brain template ICBM152), in which 245 sites were located in the PFC. The table in Figure 2D shows a summary of the number of recording sites and patients in different lobes and structures.

While looking through the event-related response of the LFP data, a striking phenomenon in all patients was that the amplitude of LFP showed remarkably different patterns between awareness and unawareness states. We show the grand average (left) and single-trial data (right) of LFP activity of recording sites in the PFC from each patient in Figure 3A, which represents the typical awareness-related neural activity of the PFC. There was a vigorous visual evoked response under the HA condition (light red line) compared to the LU (light blue line) condition. However, in the near-threshold condition, although the contrast of the grating was similar (see Results below), the LFP amplitude of the NA (red line) and NU (blue line) trials started to differ significantly at approximately 200–300ms (thick black line in panels, p<0.01 corrected, by independent t-test, details see Methods). This difference is also very explicit and consistent at the single-trial (>180 trials for NA and NU conditions in individual participants, see Methods) and single-subject level. The divergence onset time (DOT, see Methods) of different recording sites in the PFC is consistently within the period of 200~300ms after grating onset, that is the time window of VAN in scalp EEG mentioned above.

Figure 3

Download asset Open asset

Figure 3—source code 1

Source code files for pre-processing iEEG data and generating the results in Figure 3.

https://cdn.elifesciences.org/articles/89076/elife-89076-fig3-code1-v1.zip

Download elife-89076-fig3-code1-v1.zip

As proposed above, the different activities between NA and NU are regarded as awareness-related activities. We found 89 sites with awareness-related iERP activity in the PFC (323 for all recording sites). Figure 3B shows the spatial-temporal dynamics of visual awareness-related activities in all recording sites during the 180–650ms period (sampled at eight time points that equally divided this period, for a clearer display, projected to the cortex, see the Video 1 for the complete video). Visual awareness-related activities clearly appeared in the PFC during 200–300ms, and these early activities were mainly concentrated in the middle lateral prefrontal cortex (LPFC) and then spread to other brain areas of the PFC. We further calculated the start time of awareness-related activity, that is DOT, of all recording sites. Figure 3C shows the spatial distribution of all recording sites DOT (standardized and convenient for visual display), and it can be found that the DOT of the PFC is not explicitly later than those in posterior brain regions, including the posterior parietal lobe, temporal lobe, occipital lobe, etc., and the earliest region in PFC is still located in the middle lateral prefrontal cortex.

Video 1

Download asset

Considering the possibility that the brain region with earlier awareness-related activity might be more important for visual awareness, we tried to detect awareness-related activity in the PFC during the ‘early’ period, that is the VAN time window mentioned above. We focused on the sites in PFC with earlier DOTs. Because of the low-visible stimulus used in this experiment, the onset time of VAN may be delayed to 300–350ms, and we took 350ms as the cutoff value and divided DOT into early (n=43, 48.31%) and late (n=46, 51.69%) phases. We focused on the early DOT sites and calculated the normalized average iERP response of the early DOT sites (Figure 3D). At the population level, visual awareness-related activities in the PFC were also significant and started at approximately 200–300ms, which is located in the VAN time window (Figure 3D, left). Importantly, there were sites with DOT <350ms from each patient and these sites from different patients were uniformly concentrated in the middle lateral prefrontal cortex (Figure 3D, right). That is, the early visual awareness-related activities in the prefrontal cortex are robust and consistent not only at the group level but also at the individual level.

Although the above iERP results indicate that there are early visual awareness-related neural activities in the prefrontal cortex, it is difficult to explain whether this response is the result of local processing in the PFC or transmission from other brain regions. Therefore, we calculated the LFP response in the high-gamma (HG) band, which is more representative of the local processing, of the prefrontal cortex. Figure 4A shows the exemplified high-gamma activity in the prefrontal cortex from each patient at the grand average (left), spectrogram (middle), and single-trial level (right). It is similar to the iERP results (Figure 3) except the data are the HG power and there was a vigorous visual evoked power increase under the HA condition compared to the LU condition. However, in the near-threshold condition, while the contrast of the grating was similar, the high-gamma power of the NA and NU trials started to differ significantly after ~230ms (thick black line in panels, P<0.01 corrected by independent t-test; for details, see Methods). This difference is also very explicit and consistent at the single-trial and single-subject levels. The HG activity divergence onset time (DOT) of different recording sites in the PFC was also consistent within the early stage, which was similar to the above iERP result.

Figure 4 with 1 supplement see all

Download asset Open asset

Figure 4—source code 1

Source code files for generating the results in Figure 4.

https://cdn.elifesciences.org/articles/89076/elife-89076-fig4-code1-v1.zip

Download elife-89076-fig4-code1-v1.zip

We found 31 sites with awareness-related HG activity in the PFC (74 for all recording sites). Figure 4B shows the spatial-temporal dynamics of visual awareness-related HG activities in all recording sites during the 0–650ms period (for a clearer display, projected to the cortex, see the Video 2 for the complete video). Visual awareness-related HG activities also clearly appeared in the PFC during 200–300ms, while they were rarely seen in lower-level visual areas (occipital and temporal lobe), and these early activities were mainly concentrated in the middle lateral prefrontal cortex (LPFC) and then spread to other brain areas of the PFC. We further calculated the start time of awareness-related HG activity, that is DOT, of all recording sites. Figure 4C shows the spatial distribution of all recording sites DOT (standardized and convenient for visual display), and it can be found that the DOT of the PFC is not explicitly later than those in posterior brain regions, and the earliest region is still located in the middle lateral prefrontal cortex.

Video 2

Download asset

We continue to take 350ms as the cutoff value and divide the HG DOT into early (n=18, 58.06%) and late (13, 41.94%). We focused on the early DOT sites and calculated the normalized mean response (Figure 4D) and spectrum of the early DOT sites. At the population level, early visual awareness-related HG activities of the prefrontal sites still exist, and the start time is approximately 200ms, which is within the VAN time window. Importantly, there are sites with HG DOT <350ms for each patient except one, and these sites are also uniformly concentrated in the middle lateral prefrontal cortex. In other words, this effect is significant and consistent at both the group and individual levels. It is worth noting that in the PFC recording sites, the early sites with a high-gamma visual response (7.35%) were explicitly less than those with ERP visual response (17.55%), and this is also evident in the temporal dynamics (Figure 4E), which may indicate that in the process of visual awareness generation, there may also be a variety of other bands activity for information interaction between multiple brain regions.

In addition to the univariate analysis above, we also adopted the machine learning decoding method based on multivariate pattern analysis (MVPA) to test the ability of broadband and high-gamma activities in the PFC to predict the awareness state at the single-trial level. We applied a linear discriminant analysis (LDA) to classify aware versus unaware trials based on broadband and high-gamma LFP activities in the PFC.

Figure 5A shows the decoding performance through the ERP activity in the PFC. We found that the ERP activity of the PFC can accurately predict the state of visual awareness at the single-trial level in the delayed period after grating onset, and this result is also relatively consistent at the individual level. Moreover, this decoding accuracy has begun to be much higher than the chance level in 200–300ms, consistent with our above univariate analysis results. Furthermore, we applied the temporal generalization method to test the predictive ability across time. The results (Figure 5B) show that the decoding performance of population ERP is very limited in temporal generalization and cannot be well generalized over a long period. Importantly, this phenomenon is also very consistent at the individual level, which indicates that the representation of visual awareness via broadband activities in the PFC is relatively dynamic rather than static. Moreover, we also applied temporal generalization across the control conditions, that is train on NA/NU trials but test on HA/LU trials to test the predictive ability across conditions. The classifier trained on the NA/NU trials can be well generalized to the HA/LU trials, and the generalization across time is also very consistent. These results further indicated the robustness of the above decoding results.

Figure 5

Download asset Open asset

Figure 5—source code 1

Source code files for generating the results in Figure 5.

https://cdn.elifesciences.org/articles/89076/elife-89076-fig5-code1-v1.zip

Download elife-89076-fig5-code1-v1.zip

Figure 5C–D shows the results of a similar analysis of HG activity in the prefrontal cortex. (Figure 5C) shows the decoding performance through HG activity in the PFC. We found that the HG activity of the PFC can accurately predict the state of visual awareness at the single-trial level in the delayed period, and this result is also relatively consistent at the individual level. Moreover, this decoding accuracy began to be significantly higher than chance at approximately 200–300ms, which is also consistent with our above univariate analysis results. Furthermore, we applied the temporal generalization method to the HG activity. Although the group results still show that the decoding performance of HG activity is limited in time generalization, the generalization results are not consistent at the individual level, which may be attributed to the variability of electrode location in different patients. This may suggest that the representation of visual awareness-related information by population neurons at different regions in the PFC is inconsistent, that is some are relatively stable, and others are dynamic. Similarly, we applied temporal generalization across the HA/LU conditions, and the result was also very consistent.

We further explored the dynamic functional connectivity between the prefrontal cortex and other brain regions in the generation of visual awareness. First, we calculated the time-across phase-locking value (PLV) (Tass et al., 1998) in NA and NU conditions at the sensor level (baseline removed, see Methods for details). Figure 6A–B shows the example results of two exemplified patients (see Figure 6—figure supplement 1. for other patients). We found that in the low-frequency band (1–8 Hz), overall, there was a significant difference in the PLV between the NA and NU trials. In the first 150ms after the appearance of the grating, there was no significant difference in the PLV between these two conditions. However, in the period of approximately 200–300ms, the PLV between many recoding sites in the NA trials began to increase explicitly and lasted until after 600ms. In contrast, there was no such phenomenon in the NU trials. Importantly, this phenomenon is strikingly explicit and consistent in each patient.

Figure 6 with 1 supplement see all

Download asset Open asset

Figure 6—source code 1

Source code files for generating the results in Figure 6 and Figure 6—figure supplement 1.

https://cdn.elifesciences.org/articles/89076/elife-89076-fig6-code1-v1.zip

Download elife-89076-fig6-code1-v1.zip

Furthermore, we grouped the recording sites by brain regions according to the DKT brain atlas Fischl, 2012 and averaged the PLV between different sites and different subjects according to brain regions (Figure 6C). The functional connectivity between the prefrontal cortex and subcortical structures, such as the thalamus, and the posterior cortex, such as the posterior parietal cortex, was significantly enhanced in the NA trials compared with the PLV in NU trials. In addition, the functional connectivity between the brain regions within the PFC was also significantly enhanced. For clearer visualization, we show the dynamics of PLV in the NA and NU trials on a topographic map (Figure 6D).

Interestingly, we found that at the single-subject level, the sites in the prefrontal cortex that showed early awareness-related PLV activity coincided with the above sites with early awareness-related HG activity. Figure 6E shows the averaged PLV between these sites (n=18) with early visual awareness-related HG activity in the PFC and other sites in the NA and NU trials. The averaged PLV also showed significant visual awareness-related activities. Moreover, at the population level, the start time of awareness-related PLV activity at these sites also started at approximately 200ms, which is very similar to that of HG activity.

Since there may be a slight difference in the grating contrast under the NA and NU trials, the aforementioned different activity between the NA and NU conditions may be caused by the different contrast of grating. To rule out this possibility, we compared the distribution of grating contrast in the NA and NU conditions for each patient (Figure 7A). Overall, we find that the distributions of grating contrasts largely overlap with each other (red and blue curves). The Kolmogorov‒Smirnov test resulted in no significant difference (p>0.05) between the two contrast distributions in all patients except patient 2. Although intuitively the trials with higher contrast contributed more to the NA condition and the trials with lower contrast contributed more to the NU condition, the contrast difference between the NA and NU conditions were not significantly different perhaps because the grating contrast was designed to be near the perceptual threshold (the ‘aware’ percent is 50%) in most trials. Moreover, for patient 2, the difference between the mean grating contrast in the NA and NU trials was also subtle (0.15%), much less than the difference between NA and HA (17.35%) or NU and LU (0.84%). However, it is clear in the results above that the different activities between NA and NU were much stronger than those between the same awareness states, that is NA vs. HA or NU vs. LU. Therefore, the difference in neural activity between aware and unaware trials cannot be explained by the contrast differences in the grating.

Figure 7

Download asset Open asset

Figure 7—source code 1

Source code files for generating the results in Figure 7.

https://cdn.elifesciences.org/articles/89076/elife-89076-fig7-code1-v1.zip

Download elife-89076-fig7-code1-v1.zip

Compared with the regular report-dependent visual awareness task, we employed a task that effectively dissociated the activity of visual awareness from the activity of saccadic decision and execution in terms of time (the fixation point changes color 650ms after the grating onset) and reporting behavior. Since the choice of reporting behavior depends on the visual awareness state and the color of the fixation point, the subjects were unable to effectively prepare the subsequent saccadic reports during the delay period of the task; thus, the visual awareness-related activity evoked by the grating stimulus and report-preparation-related activity were temporally dissociated. Furthermore, both in the awareness and unawareness states, the subjects needed to choose the saccade direction after the fixation point changes its color so that the saccade directions in the two visual awareness states were fully matched to exclude the effects of saccade preparation- and execution-related activity on visual awareness-related activity. Compared with the no-report paradigms, our novel paradigm required subjects to make explicit reports of their awareness states so that it can be more accurate regarding the judgment of the awareness status.

Moreover, the intervals for the initial fixation period and delay period were both fixed at 600ms, instead of randomly varied, in order to keep subjects’ anticipation being similar in all trials. Also, the location of grating remained same in all trials for the same purpose.

It was also worth noting that our task does not remove the confound of report entirely, that is the post-perceptual confound might relate to planning to report perception, which is different for perceived and not perceived stimuli. Nevertheless, our task did minimize the motor-related confound to awareness-related activity.

Many factors affect saccadic reaction time, including the characteristics of external stimuli and the internal state of the brain. The physical properties of external visual stimuli have a direct impact on the performance of saccades; for example, saccadic reaction time decreases with increasing visual stimulus contrast (Allik and Kreegipuu, 1998; Deplancke et al., 2010). A widely accepted view is that the greater contrast of visual stimuli will automatically attract more bottom-up attention and speed up information processing in the brain, thereby shortening the saccadic reaction time (Posner and Petersen, 1990; Corbetta and Shulman, 2002). Other studies found that endogenous factors, such as top-down attention, can also significantly shorten the saccadic reaction time (Corbetta and Shulman, 2002; Posner, 1980).

One study briefly reported that the average reaction time (button press) for word semantic classification in the awareness state was approximately 300ms faster than in the unawareness state, but the statistical test did not reach a significant level (Gaillard et al., 2009). In present study, we found that the saccadic reaction time of all subjects in the aware trials was significantly smaller than that in the unaware trials (Figure 2C). More critically, the grating contrast near the threshold was not significantly different between the aware and unaware trials (Figure 7A); thus, such results cannot be merely caused by the contrast difference in the grating. Here, we showed that awareness states significantly affect saccadic reaction time, which supports our previous hypothesis.

An alternative interpretation for RT difference between aware and unaware condition in our study is that the difference in task-strategies used by subjects/patients to remember the response mapping rules between the perception and the color cue (e.g. if the YES +GREEN = RIGHT and YES +RED = LEFT rules were held in memory, while the NO mappings were inferred secondarily rather than being actively held in memory).

Another possibility is that the reaction time is strongly modulated by the confident level, which has been described in previous studies (Marzi et al., 2006; Broggin et al., 2012). However, in previous studies, the confident levels were usually induced by presenting stimulus with different physical property, such as spatial frequency, eccentricity and contrast. However, the dependence of visual process on the salience of visual stimulus confounds with the effect of visual awareness on the reaction time of responsive movements, which is hard to attribute the shorter reaction time in more salient condition purely to visual awareness. In contrast, we create a condition (near aware threshold) in the present study, in which the saliency (contrast) of visual stimulus is very similar in both aware and unaware conditions in order to eliminate the influence of stimulus saliency in reaction time. We think that the difference in reaction time in our study is mainly due to the modulation of awareness state, which was not reported previously.

The long debate about the neural mechanism of consciousness focuses on the location, that is ‘front’ vs. ‘back’, and the time, that is ‘early’ vs. ‘late’, of its origin in the brain (Seth and Bayne, 2022). Concerning the dispute of location, our results show that the prefrontal cortex still displays visual awareness-related activities even after minimizing the influence of the motor-related confounding variables related to subjective reports such as motion preparation, which indicates that the prefrontal cortex does participate in the information processing of visual awareness. Regarding the time dispute, our results show that at 200–300ms after stimulus onset, the VAN time window, the prefrontal cortex has activities related to visual awareness, and these activities include not only ERP activities but also HG activities, which are highly related to the firing of local population neurons (Fisch et al., 2009). Notably, the recording sites with early ERP and HG visual awareness-related activities are mostly concentrated in the middle lateral prefrontal cortex. This shows that the prefrontal cortex, especially the middle lateral prefrontal cortex, started visual awareness-related activities, including local activities, at a relatively early stage.

Several iEEG studies have shown the early prefrontal cortical involvement in visual perception, including ERP and HFA (Blanke et al., 1999; Khalaf et al., 2023; Vishne et al., 2023). However, in these studies, the differential activity between conscious and unconscious conditions was not investigated, thus, the activity in prefrontal cortex might be correlated with unconscious processing, rather than conscious processing. In present study, we compared the neural activity in PFC between conscious and unconscious trials, and found the correlation between PFC activity and conscious perception.

Although one iEEG study (Gaillard et al., 2009) reported awareness-specific PFC activation, the awareness-related activity started 300ms after the onset of visual stimuli, which was ~100ms later than the early awareness related activity in our study. Also, due to the limited number of electrodes in the previous study (2 patients with 19 recording sites mostly in mesiofrontal and peri-insular regions), it was restricted while exploring the awareness-related activity in PFC. In the present study, the number of recording sites (245) were much more than previous study and covered multiple areas in PFC. Our results further show earlier awareness-related activity (~200ms after visual stimuli onset), including ERP, HFA and PLV. These awareness-related activity in PFC occurred even earlier (~150ms after stimulus onset) for the salient stimulus trials (Figure 3A, D and Figure 4A, D, HA condition)’.

However, the proportions are much smaller than that reported by Gaillard et al, which peaked at ~60%. We think that one possibility for the difference may be due to the more sampled PFC subregions in present study and the uneven distribution of awareness-related activity in PFC. Meanwhile, we noticed that the peri-insula regions and middle frontal gyrus (MFG), which were similar with the regions reported by Gaillard et al, seemed to show more fraction of awareness-related sites than other subregions during the delay period (0–650ms after stimulus onset). To test such possibility and make comparison with the study of Gaillard et al., we calculated the proportion of awareness-related site in peri-insula and MFG regions (Figure 4—figure supplement 1). We found although the proportion of awareness-related site was larger in peri-insula and MFG than in other subregions, it was much lower than the report of Gaillard et al. One alternative possibility for the difference between these two studies might be due to the more complex task in Gaillard et al. Nevertheless, we think these new results would contribute to our understanding of the neural mechanism underlying conscious perception, especially for the role of PFC.

Some recent studies utilizing the no-report paradigm supported our results. For example, a recent electrophysiological study of rhesus monkeys found that even if monkeys do not need to report their conscious content under the binocular competition paradigm, there were also visual awareness-related spiking activities in the prefrontal cortex (Kapoor et al., 2022). In addition, some previous human intracranial studies also reported ERP, HG, and single-neuron activities related to visual perception in the higher visual areas of the temporal lobe, such as the superior temporal gyrus (Fisch et al., 2009; Reber et al., 2017). However, due to the limitations of the experimental paradigm, these studies cannot completely exclude the impact of subjective reports. Moreover, most of the electrodes in previous studies were located in the temporal lobe, and electrodes in the prefrontal cortex are rare. As our data show that the proportion of sites with earlier ERP and HG responses in the prefrontal cortex is low and relatively concentrated, it is not hard to explain that earlier visual awareness-related activities in the prefrontal cortex were not found in previous iEEG studies, especially in the HG band. Our results, on the one hand, support that the prefrontal cortex may play an important role in the emergence of visual awareness and thus support the 'front of the brain’ theories of consciousness in the origin brain regions. However, our data also support the prediction of the early emergence of consciousness proposed by the 'back of the brain’ theories. This means that our result also challenges some predictions of these mainstream theories.

Compared with traditional univariate analysis, MVPA-based machine learning decoding may be more sensitive to the information contained in brain activities (Grootswagers et al., 2017), so it has gradually become a powerful tool for the study of neural correlates of consciousness (NCCs). Therefore, we used linear discriminant analysis (LDA) to decode the awareness states at the single-trial level through broadband and HG activities in the prefrontal cortex. Our decoding results show that whether through broadband or HG activities, the decoding performance began well above the chance level at approximately 200–300ms. This shows that the prefrontal cortex has begun to encode visual awareness-related information in 200–300ms, which is consistent with the above univariate analysis results. In addition, a recent electrophysiological study of rhesus monkeys using the no-report paradigm also found that even without the requirement of explicit reporting, awareness-related neurons can still be found in the lateral prefrontal cortex of rhesus monkeys, and conscious content can be accurately decoded by the MVPA method (Kapoor et al., 2022). Another human fMRI study also found similar results (Kronemer et al., 2022). These results supported our experimental results.

In addition, we also applied the temporal generalization method during decoding. Temporal generalization analysis shows how different brain activity patterns change and transform over time (King and Dehaene, 2014). Our results show that the representation pattern of neural activity in the prefrontal cortex for visual awareness-related information is dynamic, not static, and this result is very consistent at the individual level and across the control conditions. Such a result may be consistent with the nature of conscious perception. The process of conscious perception (visual awareness) includes a series of processes from phenomenal consciousness (‘qulia’) to conscious access. Therefore, at the level of large, high-order brain regions, the representation of this complex process in the brain should be continuously transmitted and transformed, that is, dynamically changed.

There is growing evidence showing that conscious experience may be represented at the brain network level (Kronemer et al., 2022; Huang et al., 2023). Thanks to the simultaneously recorded LFP data in multiple brain regions with a high signal-to-noise ratio and high spatial-temporal resolution, we can analyze the dynamic functional connectivity changes under the aware and unaware conditions at the millisecond scale. Since the signals in the low-frequency band are more involved in the information interaction between different brain regions and the activities in the high-frequency band are more representative of the local neural activities, it is easy to understand the functional connectivity changes in the low-frequency band between relatively distant brain regions. Some previous studies have found that the phase synchronization of neural activities in beta and gamma bands is related to visual awareness, but because most of these studies are based on fMRI, it is difficult to describe the dynamic changes in such brain networks on a more precise time scale. The only human iEEG study (Gaillard et al., 2009) reported that the phase synchronization of the beta band in the aware condition also occurred relatively late (>300ms) and mainly confined to posterior zones but not PFC. Moreover, most of these studies did not rule out the influence of motor-related processing, and these processes are often associated with changes in the functional connectivity of multiple brain regions. To the best of our knowledge, we are the first to report dynamic changes in functional connectivity on the millisecond time scale in the low-frequency band (1–8 Hz) associated with visual awareness, and this change is not caused by motor-related confounding factors, such as motion preparation. Moreover, the early sites of PLV and HG overlapped each other to a large extent. Such converging results may further suggest that the middle lateral prefrontal cortex, the brain regions where these sites are located, plays a crucial role in the emergence of visual awareness.

Compared with the unaware trials, the high-frequency activity in the PFC and the functional connectivity between many brain regions in the aware trials increased. These results may confirm some theoretical concepts, such as ‘ignition’ and conscious access. Global neuronal workspace theory (GNWT) emphasizes ignition and conscious access (Dehaene and Changeux, 2011; Mashour et al., 2020). Ignition means nonlinear brain activation to small stimulus changes (such as perceptual threshold stimulation). For the near-threshold stimulus used in the current study, while the sensory information in the brain passes through a certain threshold in a specific brain area, such as the lateral prefrontal cortex, the population of neurons discharges (HG activity of local field potential), that is, ‘ignition’. This is consistent with many previous human intracranial studies (Fisch et al., 2009; Reber et al., 2017). However, as discussed above, in contrast with previous studies, our study detected earlier awareness-specific ‘ignition’ in the human PFC, while minimizing the motor-related confounding.

Conscious access means that conscious contents can be shared by many other local information processors in the brain (Dehaene and Changeux, 2011; Mashour et al., 2020). For example, when you see the grating, you can either report it orally or draw it down. Such a function may require the participation and coordination of many related brain regions or networks, but there is no clear biomarker for how these brain regions coordinate. The explicit increase in the PLV, specifically appearing in the aware trials, in the low-frequency band found in the current study may provide potential biological meaning for conscious access; that is, many brain regions interact with each other through phase synchronization in the low-frequency band to accomplish information sharing of conscious content.

Interestingly, the beginning time of prefrontal visual awareness-related activities found in ERP and HG activities is very consistent with the onset time of dynamic functional connectivity enhancement, both of which start at 200–300ms. In addition, the dynamic functional connectivity of the sites with early HG visual awareness-related activity is usually the earliest to enhance compared to other sites, and the two have good consistency. And this consistency suggested that conscious access and phenomenal awareness may be closely coupled, occurring initially in the lateral PFC at the early stage, and indicate a potential process of conscious perception: subtle change in the external stimulus or internal brain state - population neuron discharge in specific brain areas (ignition, phenomenal consciousness appeared) - phase synchronization of various brain areas induced by specific brain areas (conscious access). Further research is needed to regulate the activity of the middle lateral prefrontal cortex in the high-gamma band or the phase synchronization in the low-frequency band to explore whether the prefrontal cortex plays a causal role in the process of conscious perception.

Six adult patients with drug-resistant epilepsy (6 males, 32.33±4.75 years old, mean ± SEM) and 10 healthy subjects (4 males, 6 females, age 27.80±3.92, mean ± SEM) participated in this study. All subjects had normal or corrected-to-normal vision. Electrophysiological signals (LFPs) were obtained from six patients. Stereotaxic EEG depth electrodes (SINOVATION MEDICAL TECHNOLOGY CO., LTD. Ltd., Beijing, China) containing 8–20 sites were implanted in patients in the Department of Neurosurgery, PLA General Hospital. Each site was 0.8 mm in diameter and 2 mm in length, with 1.5 mm spacing between adjacent sites. A few electrodes were segmented, and the distance between the two segments was 10 mm. Electrode placement was based on clinical requirements only. Recordings were referenced to a site in white matter. The sEEG signal was sampled at a rate of 1 kHz, filtered between 1 and 250 Hz, and notched at 50 Hz (NEURACLE Technology Co., Ltd., Beijing, China). Stimulus-triggered electrical pulses were recorded simultaneously with sEEG signals to precisely synchronize the stimulus with electrophysiological signals.

Ten healthy subjects performed the experiment in a quiet dark room. Stimuli were presented on a 27-inch LED screen (BenQ, refresh rate of 120 Hz, resolution of 1920 x 1080), and eye movement data were recorded by an infrared eye tracker (EyeLink 1000 Desktop; SR Research) with a sampling rate of 1 kHz. Patients performed the experiment in a quiet dark environment. The stimuli were presented on a 24-inch LED screen (Admiral Overseas Corporation, refresh rate of 144 Hz, resolution of 1920 x 1080), and eye position data were obtained by an infrared eye tracker (Jsmz EM2000C, Beijing Jasmine Science & Technology, Co. Ltd), sampling at 1 kHz. The experimental paradigm was presented by MATLAB (The MathWorks) and the toolbox Psychtoolbox-3 (PTB-3; Brainard and Pelli). Patients were regular recipients of standard medication for epilepsy treatment during the experimental period.

All subjects provided informed consent to participate in this study. The Ethics Committee of Chinese PLA General Hospital approved the experimental procedures (approval numbers S2022-457-01).

Electrode locations were determined by registering postoperative computed tomography (CT) scans and preoperative T1 MRI using a standardized mutual information method in SPM12 (Ashburner and Friston, 1997). Cortical reconstruction and parcellation (Desikan-Killiany Atlas) were conducted by FreeSurfer (Fischl, 2012). To demonstrate the MNI coordinates of recording sites, a nonlinear surface registration was performed using a nonlinear registration in SPM12, aligning to the MNI ICBM152 template.

A trial started when a fixation point (0.5° x 0.5°, white cross) appeared at the center of the screen (radius of eye position check window is 4°, the dotted circle). After the subject fixated on the fixation point for 600ms, a cue stimulus (Gabor grating, 2x2° circle) was presented for 50ms at a fixed position (7°) on the left (or right) side of the screen for all participants. The side where the grating appears (left or right) is opposite to the hemisphere where the patients’ electrodes are implanted (left or right). For example, if the patient’s electrodes are implanted on the right side, the grating would appear on the left side of the screen. For patients 1, 5, and 6, the electrodes were implanted in both hemispheres, and the grating is set on the right side. In 70% of the trials, the grating contrast (weber contrast, see van Vugt et al., 2018) was maintained near the subject’s perceptual threshold by a 1 up/1 down staircase method, and the step was either 0.39% or 0.78% for each participant. In 10% of the trials, the stimulus contrast was well above the threshold, and in the other 20% of the trials, the stimulus contrast was 0, namely, no stimulus appeared. After another 600ms delay, the color of the fixation point turned red or green, and two saccade targets (1° x 1°, white square) appeared at fixed positions (10°) on the left and right sides of the screen. If the grating was seen, the green fixation point was required to make a saccade to the right target, while the red fixation point was required to make a saccade to the left target. If the grating was not seen, the rule of saccadic direction was inverted. Gratings with high contrast (well above the perceptual threshold) and zero contrast (grating absent) served as control conditions to evaluate the understanding and performance of the task. Before starting to collect data, subjects accepted 1–2 training sessions, and the contrast perceptual threshold was determined for each subject. In the formal experiments, each session consisted of 180 trials, and the intertrial interval (ITI) was 800ms.

We first identified visual awareness-related sites. For all recording sites, we compared the broadband amplitude (for ERP analysis) or high-gamma magnitude (for HG analysis) of LFP between the NA and NU conditions within 650ms after grating onset by statistical tests for each time point. We defined the site where there was a significant difference (p<0.01 FDR corrected for time points and channels, independent t-test) in the LFP between NA and NU conditions and persisted for more than 20ms as an awareness-related site. We also defined the start time of divergence as the divergence onset time (DOT) and defined the mean of the LFP amplitude difference in the first divergence period as the divergence amplitude (DA).

Normalized DOT:

DOT, divergence onset time; T_target, time from target onset to grating onset, 650ms; T_d, time from DOT to T_target; T_dmin, the minimum value of T_d in all awareness-related sites within one patient; _Tdmax, the maximum value of T_d of all awareness-related sites within one patient; normDOT, normalized DOT value, a larger normDOT value means a shorter DOT.

For population analysis of recording sites, we conducted baseline normalization for each site.

Baseline normalization for ERP amplitude:

x_norm, normalized value; x, LFP amplitude; μ, mean LFP amplitude in the baseline period (−200–0ms).

Since it is not clearly understood so far regarding the biological characteristics of LFP polarity (Einevoll et al., 2013), to simplify such complex issue, we consider the change in magnitude of LFP during delay period in our task is awareness related activity, regardless its actual value being positive or negative. Therefore, while analyzing the awareness related population activity, we first calculate the absolute value of activity difference between aware and unaware trials in individual recording site, then pool the data of 43 recording sites together and calculate the mean and standard error of mean (SEM) (Figure 3D). As in Figure 3A, the activity difference between aware (red) and unaware (blue) trials lasts until/after the end of delay period. Thus, the awareness related population activity in Figure 3D extends out to 600ms.

Baseline normalization for HG magnitude:

x_norm, normalized value; x, high-gamma magnitude or power; μ, mean of high-gamma magnitude or power in the baseline period (−200–0ms); σ, standard error of high-gamma magnitude or power in the baseline period (−200–0ms).

Baseline normalization for spectrogram:

x_norm, normalized value; x, power value; μ, mean power in the baseline period (−200–0ms).

To visualize the topography of the ERP and HG activities, we used the Brainstorm (Tadel et al., 2011) package of MATLAB to map the results on the ICBM152 cortical surface. It is a simple 3D interpolation with Shepard’s method (weights decreasing with the square of the distance), similar to what is done when projecting subject sources to an anatomy template, where vertices of the cortical surface that are further than 15 mm from the center of the sEEG contact are ignored.

For ERP visualization, we normalized the activity in absolute value because it is not clearly understood so far regarding the biological characteristics of LFP polarity. To simplify such complex issue, we consider the change in magnitude of LFP during delay period in our task represents awareness related activity, regardless its actual value being positive or negative. Therefore, we first calculated the absolute value of activity difference between aware and unaware trials in individual recording site, then used Shepard’s method to calculate the activity in each vertex and projected on the surface of brain template as shown in Figure 3B.

We implemented a machine learning framework for trial-by-trial classification using broadband amplitude and high-gamma magnitude of LFP activities in the PFC via the MVPA-Light toolbox (Guggenmos et al., 2018). We applied the LDA classification and temporal generalization on each participant separately, and all trials from each class were undersampled to balance the unequal trial number between different conditions. Then, the trials were randomly assigned into 10 folds, and all trials within each fold were subaveraged across every four trials to increase the signal-to-noise ratio. Decoding was then followed with a leave-one-out cross-validation procedure on the subaveraged trials. This procedure was repeated 10 times so that each fold in the dataset was used exactly once for testing, and at least once for training, but never at the same time. And the fold assignment was repeated five times, which resulted in total 50 iterations for the decoding analysis of each patient.

The time-across phase-locking value (PLV) was calculated in the brainstorm toolbox. Time-frequency transformation was conducted via the Hilbert transform method. The time-across PLV measures if the phase difference between two signals is consistent across trials with respect to the event that defines the trials (i.e. the output is a mean across trials per time point) (Mercier et al., 2022). PLV values range between 0 (for totally random) and 1 (for perfect phase locking). For clear demonstration here, the mean PLV in the baseline period (−200–0ms) of each pair of recording sites was subtracted. The group analysis of PLV was performed according to Betzel et al., 2019; Sadaghiani et al., 2022. The topographic display of the connectivity was conducted by the BrainNet Viewer toolbox (Xia et al., 2013).

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

We thank the participants for volunteering to take part in the study. This study was funded by STI2030-Major Projects +2021ZD0204300. National Natural Science Foundation of China (32061143004, 32030045). Funded by Open Research Fund of the State Key Laboratory of Cognitive Neuroscience and Learning (CNLZD2202).

All subjects provided informed consent to participate in this study. The Ethics Committee of Chinese PLA General Hospital approved the experimental procedures (approval numbers S2022-457-01).

1. Sent for peer review: May 30, 2023
2. Preprint posted: June 9, 2023 (view preprint)
3. Preprint posted: August 10, 2023 (view preprint)
4. Preprint posted: December 28, 2023 (view preprint)
5. Version of Record published: January 24, 2024 (version 1)

You can cite all versions using the DOI https://doi.org/10.7554/eLife.89076. This DOI represents all versions, and will always resolve to the latest one.

© 2023, Fang et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.