Prior studies have shown that high-frequency activity (HFA) is modulated by the phase of low-frequency activity. This phenomenon of phase-amplitude coupling (PAC) is often interpreted as reflecting phase coding of neural representations, although evidence for this link is still lacking in humans. Here, we show that PAC indeed supports phase-dependent stimulus representations for categories. Six patients with medication-resistant epilepsy viewed images of faces, tools, houses, and scenes during simultaneous acquisition of intracranial recordings. Analyzing 167 electrodes, we observed PAC at 43% of electrodes. Further inspection of PAC revealed that category specific HFA modulations occurred at different phases and frequencies of the underlying low-frequency rhythm, permitting decoding of categorical information using the phase at which HFA events occurred. These results provide evidence for categorical phase-coded neural representations and are the first to show that PAC coincides with phase-dependent coding in the human brain.https://doi.org/10.7554/eLife.07886.001
Electrocorticography, or ECoG, is a technique that is used to record the electrical activity of the brain via electrodes placed inside the skull. This electrical activity repeatedly rises and falls, and can therefore be represented as a series of waves. All waves have three basic properties: amplitude, frequency and phase. Amplitude describes the height of a wave's peaks (and the depth of its troughs), and frequency defines how many waves are produced per second. The phase of a wave changes from 0° to 360° between two consecutive peaks of that wave and then repeats, similar to the phases of the moon.
Previous studies have shown that brain activity at different frequencies can interact. For instance, neural firing (when nerve impulses are sent from one neuron to the next) is related to ‘high frequency activity’; and the amplitude of high frequency activity can be altered by the phase of other, lower frequency brain activity. It has been suggested that this phenomenon, called ‘phase-amplitude coupling’, might be one way that the brain uses to represent information. This ‘phase coding’ hypothesis has been demonstrated in rodents but is largely untested in humans.
Now, Watrous et al. have explored this hypothesis in epilepsy patients who had ECoG electrodes implanted in their brains for a diagnostic procedure before surgery. These electrodes were used to record brain activity while the patients viewed images from four different categories (houses, scenes, tools and faces).
Watrous et al. found that phase-amplitude coupling occurred in over 40% of the recordings of brain activity. The analysis also revealed that the phase of the lower frequency activity at which the high frequency activity occurred was different for each of the four image categories. This provides support for the phase-coding hypothesis in humans. Furthermore, it suggests that not only how much neural firing occurs but also when (or specifically at what phase) it occurs is important for how the brain represents information. Future studies could now build on this analysis to see if phase-amplitude coupling also supports phase coding and neural representations in other thought processes, such as memory and navigation.https://doi.org/10.7554/eLife.07886.002
Perceptual representations of the environment are critical to an animal's survival and are believed to occur through coactivated neuronal groups known as cell assemblies. Human neuronal firing (Ekstrom et al., 2007; Kraskov et al., 2007; Chan et al., 2011; Rey et al., 2014) and increases in high-frequency activity (HFA) in the gamma range (above 30 Hz; Jacobs and Kahana, 2009; Jacobs et al., 2012; van Gerven et al., 2013) carry information about perceptual and mnemonic representations. Several recent studies have shown that these two signals are positively correlated (Ray et al., 2008; Manning et al., 2009; Whittingstall and Logothetis, 2009; Miller et al., 2014; Rey et al., 2014; Burke et al., 2015) and are each modulated by the phase of low frequency oscillations (LFO) (O'Keefe and Recce, 1993; Bragin et al., 1995; Skaggs et al., 1996; Canolty et al., 2006; Jacobs et al., 2007; Tort et al., 2009; Axmacher et al., 2010; Rutishauser et al., 2010; McGinn and Valiante, 2014). This modulation is detectable as phase-amplitude coupling (PAC) of gamma amplitude to LFO phase (Buzsaki, 2010; Miller et al., 2014; Aru et al., 2015).
Together, these findings have motivated models positing that LFO phase may organize cell assemblies (Kayser et al., 2012; Lisman and Jensen, 2013; Jensen et al., 2014; Watrous et al., 2015), a form of phase coding (O'Keefe and Recce, 1993). Supporting this view, LFO phase can be used to decode behaviorally relevant information (Belitski et al., 2008, 2010; Fell et al., 2008; Schyns et al., 2011; Lopour et al., 2013; Ng et al., 2013) and phase coded neural activity has been demonstrated in rodents (O'Keefe and Recce, 1993; Skaggs et al., 1996) and monkeys (Kayser et al., 2009; Siegel et al., 2009). Although the PAC observed in humans (Canolty et al., 2006; Axmacher et al., 2010) has been thought to reflect phase-coding, this assumption has yet to be validated because prior studies have not investigated the relation between PAC and decoding from LFO phases.
We have recently proposed that the frequency-specific phase of LFO coordinates neural firing to support neural representations (Watrous and Ekstrom, 2014; Watrous et al., 2015). Here, we tested this prediction, a form of the phase-coding hypothesis in humans, by examining the relation between PAC and neural representations for categories. We analyzed intracranial recordings from 167 electrodes in six patients with pharmaco-resistant epilepsy as they viewed pictures of houses, tools, scenes, and faces. First, we identified PAC on individual electrodes by using a recently developed metric which allows for the characterization of PAC across individual HFA events. On electrodes exhibiting PAC, we then assessed the distinctiveness of each category's phase-coded representation during periods with and without pronounced HFA. Our results suggest that during periods with pronounced HFA, categorical representations can be recovered based on the phase of low-frequency oscillations, supporting the idea of phase-coded neural representations in humans.
We analyzed data from a total of 167 intracranially-recorded EEG electrodes from six patients with pharmaco-resistant epilepsy as they viewed pictures of houses, faces, tools, and outdoor scenes (Figure 1A), testing whether these categories may be represented based on HFA activity at different phases of the LFO (Figure 1B). We first sought to identify electrodes exhibiting PAC and used a data-driven method which allows for the identification of predominant modulating and modulated frequencies (Dvorak and Fenton, 2014; see Figure 1—figure supplement 1 for analysis schematic). Figure 2A–D shows the PAC modulation profile of an example electrode from the basal temporal lobe of patient 3. Figure 2A shows the magnitude of the modulatory signal relative to HFA events (time 0) at different frequencies in the HFA band. PAC is evident as red and blue vertical striping, with maximal modulation of activity at 84 Hz (Figure 2B; ‘HFA event’ marked by arrow in Figure 2C) occurring near the trough of the 2.5 Hz oscillation (see also Figure 2D). Notably, PAC was visible in the raw trace (Figure 2C), the modulatory signal showed rhythmicity (Figure 2D), and there was a clear peak in the power spectrum of both the raw signal and the modulatory signal (Figure 2E, see Figure 2—figure supplement 1 for more examples).
Next, we investigated the prevalence of PAC and HFA on each electrode. We found robust evidence for PAC, with at least 20% of electrodes in each patient showing significant PAC (n = 72/167 ‘PAC+’ electrodes, see ‘Materials and methods’ for statistical assessment and inclusion criteria). On PAC+ electrodes, HFA was broadly distributed across trials and time points. Calculating the proportion of trials showing a period of significantly increased HFA (95th percentile, see ‘Materials and methods’ for ‘HFA windows’) on each PAC+ electrode and category, we found that HFA occurred throughout the period of stimulus presentation but increased ∼150 ms after stimulus onset (Figure 2F). 66% of trials had at least one HFA window and this prevalence did not vary by category (Figure 2—figure supplement 2; one-way ANOVA, F(3,284) = 0.6, p > 0.61). These findings converge with prior studies demonstrating increased neural firing and HFA during stimulus presentation and demonstrate pronounced PAC in our paradigm (Canolty et al., 2006; Mormann et al., 2008; Axmacher et al., 2010; Cichy et al., 2014; Rey et al., 2014).
We then determined the frequencies and phases at which PAC is maximal on each PAC+ electrode. Slow-modulating (‘Fphase’) frequencies were significantly clustered in the delta band (0.5–4 Hz; Figure 2G) and HFA modulated frequencies (‘Famp’) were significantly clustered around slow (∼32 Hz) and fast (∼110 Hz) gamma frequencies (Figure 2H, chi-square goodness of fit test across gamma frequencies, p < 0.004, χ2(22) = 43.6, Cohen's d = 0.77). Furthermore, we found that HFA was typically maximal near the trough of the oscillation (i.e., at 180°; Figure 2I; p < 0.05, Rayleigh test; see Figure 2—figure supplement 1 for additional examples and modulation at other phases).
We next tested if PAC occurs for all four categories, which would be necessary if PAC was related to the representation of categorical information. To this end, we tested each category separately for phase clustering of HFA events at the electrode-specific peak modulatory frequency (‘FMAX’). This analysis revealed significant clustering for all four categories on 87% (63/72) of PAC+ electrodes (Rayleigh test p < 0.00004, Bonferroni-corrected p < 0.01 for PAC+ electrodes and categories). Phase clustering was observed in each patient and did not vary across categories at FMAX (one-way ANOVA on resultant vector lengths, F(3,284) = 0.14, p > 0.93). In sum, we found evidence for widespread PAC in each patient at several frequencies and phases of the LFO, similar to single neuron and field potential studies in monkeys (Kayser et al., 2009; Siegel et al., 2009) and humans (Canolty et al., 2006; Jacobs et al., 2007; Axmacher et al., 2010; Maris et al., 2011; Jacobs et al., 2012; van der Meij et al., 2012; Voytek et al., 2015).
Testing the phase-coding hypothesis, we asked if high frequency activity occurred during category-specific phases of the modulatory LFO. Figure 3A shows two traces from an example electrode which are color-coded by the instantaneous phase at Fmax. HFA windows (boxes color coded by 1 Hz phase) occurred during different modulatory phases depending on stimulus category. On this electrode, phases extracted during HFA windows were clustered for each category to different phases, resulting in category-specific phase-clustering (Figure 3B). Similar findings were observed in other patients (Figure 3C), and appeared distinct from representations using power or phase (Figure 3—figure supplements 1, 2).
These findings imply that representations might occur by the category-specific phase at which HFA events occur. In order to further quantify this effect, we developed a simple metric, the difference score (‘DS’), which allowed us to identify the distinctiveness of each category's phase distribution during HFA windows. We applied this metric to the subset of 63 PAC+ electrodes showing significant phase-clustered HFA for each category. This was necessary in order to exclude spurious phase differences between categories occurring in the absence of phase clustering. Across all patients, 78% (49/63) of PAC+ electrodes showed a unique phase-clustering profile for one category compared with each other category (e.g., Figure 1B; DS = 3 for at least one category, p < 10−9, Watson Williams test, Bonferroni corrected across comparisons). This pattern was consistent both within and across patients, with at least 15% of electrodes in each patient showing these effects (Figure 3D).
We next calculated the average phase difference between categories, expecting this measure to increase with increasing DS. Indeed, categories with larger DSs exhibited larger phase differences with other categories such that maximally distinct representations were 35° phase offset from all other categories (Figure 3E).
As described above, PAC was most likely to occur at the oscillatory trough (Figure 2I and Figure 2—figure supplement 1). Nonetheless, on individual electrodes or for individual categories, HFA could occur at different phases. In fact, across electrodes, phase-coding was equally likely to occur at all phases and for all categories; phase-coded categories were not clustered at particular phases at any level of DS (Rayleigh test, all p > 0.19; Figure 3F) and phase-coding was equally likely for each category (χ2(3) = 1.6, p = 0.64). Thus, a large proportion of PAC+ electrodes also show category-specific phase clustering of HFA events to different phases (Video 1), suggesting that PAC is related to phase-coding (Figure 1B, middle).
To link these findings more directly to neural coding, we used pattern classification to determine if the phase at which HFA events occur is sufficient to recover categorical information (see ‘Materials and methods’). As expected from the analysis using DS, 42 (25% of all) electrodes showed significant decoding accuracy (using LFO phases during HFA windows as features) compared to category label shuffled surrogates and this proportion was significantly higher than would be expected by chance (p < 10−10, binomial test, chance level: 8.3 electrodes, Cohen's d = 0.6). Next, we assessed whether phase-coding of categorical information indeed depended on HFA, as would be expected if PAC supports phase-coding. We compared decoding accuracy during HFA events to decoding accuracy during randomly selected surrogate events. 19 (11%) electrodes showed significantly higher decoding accuracy during HFA events as compared to random event surrogates, and this proportion was significantly higher than would be expected by chance (p < 0.0003, binomial test, chance = 8.3 electrodes, Cohen's d = 0.22). Moreover, 17 (10%) electrodes showed significant enhancements of decoding accuracy during HFA events relative to both label and event shuffled surrogates, with at least two electrodes in each patient showing this pattern. This proportion of electrodes far exceeded that expected by chance (p < 10−10, binomial test, chance = 0.41 electrodes, Cohen's d = 0.46). These findings complement the above results using DS and indicate that the phase at which HFA events occur carries sufficient information to decode image category, suggesting such information may be a relevant component of the neural code.
We performed several control analyses to rule out alternative explanations. First, if slow oscillatory phase relates to category-specific representations, we expect phase-locking across trials to different categories. We observed significant phase locking on many electrodes to specific categories (Figure 3—figure supplement 3, Rayleigh test, p < 0.000001), similar to previous studies which have identified phase-locked activity (e.g., Fell et al., 2008). Second, we excluded the possibility that our PAC+ or phase-clustering inclusion criteria biased our findings by computing a composite measure of phase representation (PR) on each electrode (see ‘Supplement results’). This analysis again revealed that phase coding is largest on PAC+ electrodes and is enhanced during HFA windows. Third, for comparison with prior PAC methods, we recomputed PAC using the modulation index (MI, Tort et al., 2009) in different low-frequency bands, again finding PAC that was most prevalent in the delta band (Figure 2—figure supplement 3).
Lastly, several models predict that neural processes forming representations will show frequency-specificity (Siegel et al., 2012; Watrous and Ekstrom, 2014; Womelsdorf et al., 2014). We therefore recalculated phase-clustering and DS at the minimum modulatory frequency (FMIN; see ‘Materials and methods’ and Figure 4—figure supplement 1 for individual subject values) using the same criteria detailed above. As one would expect, on PAC+ electrodes, phase clustering was larger at FMAX compared to at FMIN, both on individual electrodes (Figure 4A, B) and at the group level (Figure 4C; paired t-test on resultant vector lengths, t(287) = 8, p < 10−10, Cohen's d = 0.32). Moreover, only 20% (15/72) of PAC+ electrodes showed significant phase-clustering at FMIN for all 4 categories and only 1 electrode showed category-selective phase-clustering of HFA events. Given that the phase of slower frequencies varies less over time and that we primarily identified Fmax at slow frequencies, this result might be biased towards finding enhanced phase clustering at Fmax. We therefore recalculated phase clustering across the full range of frequencies (1–12 Hz, 0.1 Hz steps). Again, we found enhanced phase-clustering around 0.5 and 1 Hz (Figure 4—figure supplement 2), but not at adjacent frequencies as would be expected from this alternative account. Taken together, these results support the conclusion that HFA at distinct phases and frequencies reflect representations for different categories.
We tested the hypothesis that PAC reflects a phase-coding mechanism, measuring both PAC and categorical PR in intracranial recordings from six patients who viewed pictures from different categories. Our analyses show that on a large subset of electrodes showing PAC, the frequency-specific phase at which HFA occurs varies with categorical information. Therefore, to the extent that HFA reflects increases in local neuronal activity (Crone et al., 1998; Miller et al., 2014), our results suggest that neural representations for categories might occur by the phase at which neurons fire. These findings thus provide a novel link between PAC and phase-coded neural representations in humans.
Critically, although PAC and phase-coded representations share some attributes, such as phase-clustering of activity, they are not necessarily identical processes. High frequency activity could occur at particular phases of LFOs, as reflected by PAC, but these phases may not vary with stimulus category (Figure 1B, left). In other words, there could be PAC without phase coding. This is in fact the null hypothesis we have tested and would manifest as PAC with DSs of zero. On the other hand, categorical information may be represented by specific low-frequency phases independent of HFA, leading to DS without phase-clustering across HFA events (PAC; Figure 1B, right). We did not find a complete overlap between PAC+ and phase-coding electrodes, indicating that each can occur in isolation, but instead found a compromise between these extremes (Figure 1B, middle). These results suggest that PAC in many cases reflects phase coding because of the significant overlap between the two phenomena (Video 1).
Phase-coding, in the form of phase-modulated neuronal firing, has been identified in rodents, monkeys, and humans (O'Keefe and Recce, 1993; Skaggs et al., 1996; Jacobs et al., 2007; Kayser et al., 2009; Rutishauser et al., 2010). Although the mechanisms which guide such a neuronal phase preference remain poorly understood, previous studies have found enhanced PAC during learning and memory tasks (Tort et al., 2008; Axmacher et al., 2010; Kendrick et al., 2011; Friese et al., 2013; Lega et al., 2014). Our findings provide a potentially unifying account of these observations, suggesting that PAC may be promoting the formation of phase-coded neural assemblies (Canolty and Knight, 2010; Watrous and Ekstrom, 2014; Watrous et al., 2015). Follow-up studies will need to test this account of PAC as it relates to other putative roles for PAC (Canolty and Knight, 2010; Voytek et al., 2015).
While epilepsy is marked by increased synchronized neuronal activity which could potentially manifest as HFA or PAC, we believe several factors weigh against this interpretation. First, we only analyzed electrodes overlying putatively healthy tissue, typically from the hemisphere contralateral to the epileptic focus, as assessed by our clinical team. Electrodes showing epileptic spiking were systematically removed from our analysis and all analyzed trials were visually inspected for artifacts related to epilepsy. Next, the PAC metric allows for assessment of the modulatory signal. Visual inspection of these signals did not reveal a similarity to epileptic spikes (Figure 2—figure supplement 1). Finally, it seems unlikely that epileptic activity at different phases would systematically differ by category. Similar reasoning excludes saccade-related artifacts (Yuval-Greenberg et al., 2008; Kovatch et al., 2011) as a parsimonious account of our results. We therefore conclude that similar findings would translate into healthy human populations.
Another caveat is that our results provide evidence for categorical phase-coding based on a restricted image set. This was necessary in the present study to maximize the chances of identifying category-selective responses while still ensuring that these responses were generalizable across a few exemplars. Follow-up studies should test the generalizability of these findings using more exemplars within a category and using other categories.
PAC has typically been investigated using pre-defined low and high-frequency filters which may optimize statistical power for detecting PAC but do not adequately deal with the time-resolved nature of cognition (Aru et al., 2015). Here, we leveraged a recent method which can identify PAC and subsequently test mechanistically interesting questions related to the modulation of HFA, such as its temporal profile and its dependence on phase, frequency, and behavioral requirements. Notably, this method may conservatively estimate PAC because it is based on transient increases in HFA, which do not necessarily occur in all cases of PAC. Our findings demonstrate that PAC and large HFA events can be identified and subsequently linked to categorically distinct representations. These results thus extend previous research which has decoded neural representations using either low or high frequency activity (Jacobs and Kahana, 2009; Schyns et al., 2011; van Gerven et al., 2013) and may provide new avenues for decoding the human representational system.
Intriguingly, phase-coding of categorical information extended beyond brain areas associated with higher-order vision. Thus, our findings of category-specificity do not appear to exclusively relate to perception but may also involve other more complex, and idiosyncratic, associations to these stimuli. Our findings are nonetheless in line with prior work (Majima et al., 2014; Yaffe et al., 2014; Zhang et al., 2015) which has found spatially-distributed content-specific representations.
We identified frequency-specific PRs in humans, consistent with a growing body of evidence implicating the relevance of frequency-specific oscillatory activity to human cognition (Daitch et al., 2013; Watrous et al., 2013; Fontolan et al., 2014; Freudenburg et al., 2014). These findings are therefore consistent with models implicating frequency-specific oscillations as central to higher-order cognition (Siegel et al., 2012; Watrous and Ekstrom, 2014; Watrous et al., 2015). It has recently been shown that the frequency of LFOs contributes to several neuronal properties such that relatively slower LFOs lead to decreased firing threshold and increased spike timing variability (Cohen, 2014). It is not immediately clear how this relates to our finding that PAC predominantly occurs with modulating frequencies in the delta band, particularly around 1 Hz. It is possible that our findings reflect the activation of assemblies during ‘up’ states which show a similar frequency profile (Destexhe et al., 2007) or that the applied method of identifying peaks in the spectrum biased our findings to find PAC at lower frequencies.
A third possibility, more likely in our view based on our results indicating multiple modulating frequencies per electrode (Figure 2G), is that the timing of our task (1 image per second with a jittered inter-stimulus interval) partially entrained slow oscillations forming an oscillatory hierarchy (Lakatos et al., 2005). Similarly, our results showing PAC at a variety of phases and frequencies (Maris et al., 2011; van der Meij et al., 2012), particularly near 32 Hz, might reflect a form of ‘nested coupling’ (Kopell et al., 2010) distinct from ‘broadband’ high gamma, which has been suggested to reflect population spiking (Manning et al., 2009; Miller et al., 2014). Future research may clarify this issue by comparing single neuron activity and HFA modulation during different perceptual tasks and by investigating their relation to hierarchical cross-frequency coupling.
To summarize, by identifying electrodes exhibiting both PAC and phase-coded neural representations for categories, our results employing direct brain recordings explicitly link phase-coupled neural activity to phase coding in humans.
Six right handed patients with pharmacoresistant epilepsy (mean age 31.8 years; 3 female) participated in the study. All patients were stereotactically implanted for diagnostic purposes. Medial temporal depth electrodes (AD-Tech, Racine, WI, USA) with 10 cylindrical platinum-iridium contacts (diameter: 1.3 mm) were implanted in 1 patient, and 5 patients were implanted with subdural grid and strip electrodes with stainless-steel contacts (diameter: 4 mm) at temporal, frontal, and parietal sites (Video 2). Recordings were performed using a Stellate recording system (Stellate GmbH, Munich, Germany) at the Department of Epileptology, University of Bonn, Germany. The study was conducted according to the latest version of the Declaration of Helsinki and approved by the ethical committee of the medical faculty at the University of Bonn (approval identifier 280/08). All patients provided written informed consent to participate in the study and for the results to be published in a pseudonymized manner.
Patients performed an object–location association task, though here we focus on neural representations for categories independent of memory encoding per se. Patients viewed greyscale images taken from four different categories (houses, tools, scenes, and faces) and each category had four unique stimuli, resulting in a stimulus set of 16 unique images. Example images from each category are shown in Figure 1A. Each image was presented 30 times in pseudo random order (total of 480 trials) and was followed by a white square in a fixed location. Patients were instructed to form object–location associations and to rate if they liked or disliked each image, thus ensuring that they were attending to each image presentation. Images were presented on a laptop placed in front of the patient. Each image was presented for 1 s, followed by the white square presented for 1 s, and finally a jittered inter-stimulus interval ranging from 1800–2200 ms. A fixation cross was presented between images.
Intracranial EEG recordings (sampled at 1000 Hz) were referenced to linked mastoids and band-pass filtered (0.01 Hz [6 dB/octave] to 300 Hz [12 dB/octave]). Recordings from the hemisphere contralateral to the epileptogenic focus were analyzed. To boost our electrode sampling, an additional 32 electrodes from an ipsilateral left lateral temporal grid were included from patient 5 based on the physicians' report, which indicated a left hippocampal focus and no evidence of neocortical lesion based on an magnetic resonance imaging (MRI). Signals from this grid were carefully visually inspected for artifacts and did not show increased artifacts associated with epilepsy. Qualitatively similar results were observed when excluding these electrodes from the analysis, with the proportions of electrodes showing any reported effect changing by no more than 3%.
Electrode locations were determined by post-implantation MRI such that electrodes were mapped by co-registering pre- and post-implantation MRIs, normalizing the pre-implantation MRI and applying the normalization matrix to the post-implantation MRI. The anatomical locations of contacts were then identified by comparison with standardized anatomical atlases and using custom software (published at http://pylocator.thorstenkranz.de/). In total, 167 implanted electrode contacts were analyzed across all patients (Video 2).
Raw EEG signals were extracted from 750 ms before to 1500 ms after image onset. EEG trials were visually inspected for artifacts (e.g., epileptiform spikes), and trials with artifacts were excluded from further analysis (15% of all trials on average). Trial epochs were then concatenated for subsequent analysis described below. We analyzed an average of 103 trials per category and subject and there were no differences in total number of trials analyzed across categories (F(3,20) = 0.38, p = 0.76).
PAC was detected using the methods described by Dvorak and Fenton (2014). This method is conceptually similar to an event-locked analysis around periods of enhanced HFA. All analyses were conducted using standard routines in EEGLab (Delorme and Makeig, 2004) and Matlab based on previously published algorithms (Rizzuto et al., 2006; Berens, 2009; Tort et al., 2009; Dvorak and Fenton, 2014). In brief, the power and phase of the signal on each electrode was computed in the low frequency (Fphase, 0.5–12 Hz, 0.5 Hz steps) and gamma (Famp, center frequencies at 32–120 Hz, 4 Hz steps) bands using Morlet wavelet convolution with 7 cycles. At each center HFA frequency, the time course of power values was z-scored and time periods exceeding the 95th percentile of these values were identified (we refer to these as ‘HFA windows’, red shaded area in Figure 2C). The time point of the largest power value within each window was identified and taken as the time-locking ‘HFA event’ for OTC analyses (arrow, Figure 2C). Two-second segments (1 s before to 1 s after each HFA event) of the raw signal were extracted around these timestamps and raw signal segments were summed at each time point across segments, resulting in the modulatory signal at each center HFA frequency.
The strength of modulation was determined based on the peak to trough height of the modulatory signal. Surrogate modulatory signals (n = 100) were constructed at each modulated frequency based on choosing an equal number of pseudo-HFA events at random timestamps and repeating the above procedure. Surrogate modulation strengths were extracted and used to z-score the observed modulation strength. PAC+ electrodes were identified as electrodes (1) with a modulation strength z-score >4.35 for at least one gamma frequency and (2) with a clear peak in the power spectrum of the raw signal at Fphase (Aru et al., 2015). This z-score threshold was calculated by identifying the z value equivalent to a Bonferroni corrected (across 23 gamma frequencies and 167 electrodes) alpha threshold of p < 0.05 and corresponded to p < 0.00005. Peaks were identified by normalizing the power spectrum of both the raw and modulatory signal to their respective maxima and ensuring that both normalized signals were maximal (i.e., 1) at the same frequency.
We identified the peak HFA modulation frequency as the frequency with the largest z-score and extracted the modulatory signal (see Figure 2B). The phase and frequency content of the modulatory signal was determined using a Hilbert transform and fast Fourier transform, respectively. The modulatory signal was mean-centered prior to FFT in order to remove DC components. The maximum (‘FMAX’) and minimum (‘FMIN’) of this FFT output indicate the strongest and weakest slow-modulating frequencies in the 0.5–12 Hz band, respectively.
Phase comparisons were conducted using a Watson Williams test following Rizzuto et al (2006) and using code taken from these eegtoolbox available at (http://memory.psych.upenn.edu/Software). Statistical testing was performed between the phases extracted during HFA windows for all pairs of conditions (4 categories; 6 total category pairs). DSs were computed for each category as the total number of significant differences (p < 0.001) between the phase distribution for one category and the remaining categories and thus ranged from 0 (no difference to any other category) to 3 (significant difference to all other categories). Phase clustering scores were defined as the resultant vector length for each category's phase distribution.
We used a pattern classification approach for comparison with our DS metric, classifying image category based on the phase at which HFA events occur. To this end, we trained support vector machines using a linear kernel and fivefold cross-validation. Phase values at Fmax were extracted at moments in time when HFA events occurred during image presentation and were used as input features for the classifier. Similar to previous approaches (Lopour et al., 2013; Majima et al., 2014), the sine and cosine of the phase values were used as input features for phase. Classifiers were run separately on each electrode and the classifier output was a prediction of the category label for each HFA event. Classification accuracy was defined as the average proportion of correctly classified HFA events across folds.
Chance classification performance varies across electrodes because we classified the category label associated with each HFA event and the number of HFA events per category varied across electrodes. We thus opted to report significance based on permutation testing which accounts for the varying chance level across electrodes and assessed the significance of classification using two separate analyses which both utilized permutation testing. First, we randomized the category labels associated with HFA events and assessed classification accuracy. Second, we used random time points (also corresponding to random phases) as surrogate HFA events and assessed classification accuracy. Each type of permutation test was performed 50 times, resulting in a distribution of pseudo classification accuracy values for each test. Observed classification accuracies at or above the 95th percentile of each of these distributions were deemed significant. Thus, we fixed the type 1 error rate for each test at 5% and, assuming independence between tests, we would therefore expect 0.0025 × 167 = 0.42 electrodes to show significance by chance for both permutation tests.
Conducting an alternative analysis, we aimed to determine if phase coding was more prominent during HFA windows. To this end, we created a composite measure of PR on each electrode. Direct comparison of the observed DS value to a surrogate distribution is problematic because DS assumes values between 0–3. Therefore, in the case of an observed DS of 3, it is impossible to identify any surrogates larger than our observation. We thus created a composite and continuous measure of PR by multiplying DS and phase clustering values on each category–electrode pair.
Permutation tests were then performed by shuffling the temporal position of HFA windows. This method is similar to a method used previously (Axmacher et al., 2008). Specifically, we randomly reordered the positions of HFA and non-HFA windows and then recalculated DS, phase-clustering, and PR values for each category. Notably, this method maintains the distribution of HFA and non-HFA window durations while shuffling these windows relative to the phase series. Surrogate PR values were calculated 200 times per electrode and the observed PR value was compared against the 95th percentile of this surrogate distribution. PR values were also extracted during non-HFA windows as a second control condition.
We computed a composite measure of PR by weighting each category's DS by its phase-clustering value. This measure combines two intuitive features of phase-coding, namely that information is represented at different phases and that activity is concentrated at these phases (captured by DS and phase-clustering values, respectively). Testing the specificity of HFA windows for phase coding, we found that 56% (94/167) of electrodes showed PR that was larger during HFA windows compared to both time-shifted surrogates and non-HFA windows for at least one category. Moreover, PR values were significantly larger on PAC+ electrodes compared to electrodes without significant PAC (two-sample t-test, p < 0.000005). Thus, we find that phase coding is largest on PAC+ electrodes and is enhanced during HFA windows.
We calculated the MI (Tort et al., 2009) by binning HFA (51–200 Hz) amplitude according to phase in either the delta (0.5–4 Hz), theta (4–8 Hz), alpha (8–13 Hz) or the entire low frequency (0.5–12 Hz) bands. Following surrogate control analyses, in which we randomly shuffled gamma values 500 times prior to calculating MI, we observed significant PAC in each band relative to shuffled gamma MI values. Consistent with our primary results, PAC was most prominent in the delta band using these methods. Furthermore, the magnitude of gamma band activity was maximal at a similar phase of delta oscillations (180°, oscillatory trough) as when assessed using the OTC method. Based on the MI based metric, we found that 102/167 electrodes exhibited significant PAC with delta band phase. These results are shown in Figure 2—figure supplement 3.
Untangling cross-frequency coupling in neuroscienceCurrent Opinion in Neurobiology 31:51–61.https://doi.org/10.1016/j.conb.2014.08.002
Cross-frequency coupling supports multi-item working memory in the human hippocampusProceedings of the National Academy of Sciences of USA 107:3228–3233.https://doi.org/10.1073/pnas.0911531107
Low-frequency local field potentials and spikes in primary visual cortex convey independent visual informationJournal of Neuroscience 28:5696–5709.
Sensory information in local field potentials and spikes from visual and auditory cortices: time scales and frequency bandsJournal of Computational Neuroscience 29:533–545.https://doi.org/10.1007/s10827-010-0230-y
Circstat: a matlab toolbox for circular statisticsJournal of Statistical Software 31.
Gamma (40–100 Hz) oscillation in the hippocampus of the behaving ratJournal of Neuroscience 15:47–60.
Human intracranial high-frequency activity during memory processing: neural oscillations or stochastic volatilityCurrent Opinion in Neurobiology 31:104–110.https://doi.org/10.1016/j.conb.2014.09.003
First-pass selectivity for semantic categories in human anteroventral temporal lobeJournal of Neuroscience 31:18119–18129.https://doi.org/10.1523/JNEUROSCI.3122-11.2011
Resolving human object recognition in space and timeNature Neuroscience 17:455–462.https://doi.org/10.1038/nn.3635
Fluctuations in oscillation frequency control spike timing and coordinate neural networksJournal of Neuroscience 34:8988–8998.https://doi.org/10.1523/JNEUROSCI.0261-14.2014
Frequency-specific mechanism links human brain networks for spatial attentionProceedings of the National Academy of Sciences of USA 110:19585–19590.https://doi.org/10.1073/pnas.1307947110
EEGLAB: an open source toolbox for analysis of single- trial EEG dynamics including independent component analysisJournal of Neuroscience Methods 134:9–21.https://doi.org/10.1016/j.jneumeth.2003.10.009
Are corticothalamic ‘up’ states fragments of wakefulness?Trends in Neuroscience 30:334–342.https://doi.org/10.1016/j.tins.2007.04.006
Toward a proper estimation of phase-amplitude coupling in neural oscillationsJournal of Neuroscience Methods 225:42–56.https://doi.org/10.1016/j.jneumeth.2014.01.002
Fast-scale network dynamics in human cortex have specific spectral covariance patternsProceedings of the National Academy of Sciences of USA 111:4602–4607.https://doi.org/10.1073/pnas.1311716111
Neural representations of individual stimuli in humans revealed by gamma-band electrocorticographic activityJournal of Neuroscience 29:10203–10214.https://doi.org/10.1523/JNEUROSCI.2187-09.2009
Brain oscillations control timing of single-neuron activity in humansJournal of Neuroscience 27:3839–3844.https://doi.org/10.1523/JNEUROSCI.4636-06.2007
Explaining how brain stimulation can evoke memoriesJournal of Cognitive Neuroscience 24:553–563.https://doi.org/10.1162/jocn_a_00170
Temporal coding organized by coupled alpha and gamma oscillations prioritize visual processingTrends in Neurosciences 37:357–369.https://doi.org/10.1016/j.tins.2014.04.001
Are different rhythms good for different functions?Frontiers in Human Neuroscience 4:1–9.https://doi.org/10.3389/fnhum.2010.00187
Local field potentials and spikes in the human medial temporal lobe are selective to image categoryJournal of Cognitive Neuroscience 19:479–492.https://doi.org/10.1162/jocn.2007.19.3.479
An oscillatory hierarchy controlling neuronal excitability and stimulus processing in the auditory cortexJournal of Neurophysiology 94:1904–1911.https://doi.org/10.1152/jn.00263.2005
Slow-theta-to-gamma phase-amplitude coupling in human hippocampus supports the formation of new episodic memoriesCerebral Cortex, 10.1093/cercor/bhu232.
Broadband shifts in local field potential power spectra are correlated with single-neuron spiking in humansJournal of Neuroscience 29:13613–13620.https://doi.org/10.1523/JNEUROSCI.2041-09.2009
Phase-amplitude coupling and interlaminar synchrony are correlated in human neocortexThe Journal of Neuroscience 34:15923–15930.https://doi.org/10.1523/JNEUROSCI.2771-14.2014
Latency and selectivity of single neurons indicate hierarchical processing in the human medial temporal lobeThe Journal of Neuroscience 28:8865–8872.https://doi.org/10.1523/JNEUROSCI.1640-08.2008
EEG phase patterns reflect the selectivity of neural firingCerebral Cortex 23:389–398.https://doi.org/10.1093/cercor/bhs031
Spectral fingerprints of large-scale neuronal interactionsNature Reviews Neuroscience 13:121–134.https://doi.org/10.1038/nrn3137
Phase-dependent neuronal coding of objects in short-term memoryProceedings of the National Academy of Sciences of USA 106:21341–21346.https://doi.org/10.1073/pnas.0908193106
Theta-gamma coupling increases during the learning of item-context associationsProceedings of the National Academy of Sciences of USA 106:20942–20947.https://doi.org/10.1073/pnas.0911331106
Oscillatory dynamics coordinating human frontal networks in support of goal maintenanceNature Neuroscience 18:1318–1324.https://doi.org/10.1038/nn.4071
The spectro-contextual encoding and retrieval theory of episodic memoryFrontiers in Human Neuroscience 8:75.https://doi.org/10.3389/fnhum.2014.00075
More than spikes: common oscillatory mechanisms for content specific neural representations during perception and memoryCurrent Opinion in Neurobiology 31:33–39.https://doi.org/10.1016/j.conb.2014.07.024
Dynamic circuit motifs underlying rhythmic gain countrol, gating and integrationNature Reviews Neuroscience 17:1031–1039.https://doi.org/10.1038/nn.3764
Reinstatement of distributed cortical oscillations occurs with precise spatiotemporal dynamics during successful memory retrievalProceedings of the National Academy of Science of USA 111:18727–18732.https://doi.org/10.1073/pnas.1417017112
Gamma power reductions accompany stimulus-specific representations of dynamic eventsCurrent Biology 25:635–640.
Howard EichenbaumReviewing Editor; Boston University, United States
eLife posts the editorial decision letter and author response on a selection of the published articles (subject to the approval of the authors). An edited version of the letter sent to the authors after peer review is shown, indicating the substantive concerns or comments; minor concerns are not usually shown. Reviewers have the opportunity to discuss the decision before the letter is sent (see review process). Similarly, the author response typically shows only responses to the major concerns raised by the reviewers.
Thank you for submitting your work entitled “Phase-amplitude coupling supports phase coding in human ECoG” for peer review at eLife. Your submission has been favorably evaluated by Timothy Behrens (Senior Editor), a Reviewing Editor, and three reviewers.
The reviewers have discussed their reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.
All of the reviewers felt the paper was strong, but there were several concerns that need to be addressed:
Watrous et al. tackle the important question of whether phase coding is a mechanism for information encoding in the human brain. They test whether cross-frequency coupling (CFC) subserves phase dependent coding of distinct visual categories. The study is highly relevant for the field of systems neuroscience.
1) The authors may be unintentionally inflating their findings by defining PAC by power increases in HFA through the oscillation triggered coupling (OTC) procedure. PAC does not require a change in power, but a change in the distribution of power relative to the phase of the low-frequency oscillation. Evoked activity might distort CFC results. Did the authors attempt to study evoked and non-stimulus locked activity separately?
2) OTC needs more explanation in this paper. Given the major claims of the paper, the authors should compare the OTC to more established PAC methods, so that the reader can better assess the results. The findings may be inflated by the chosen metric.
3) There are reports (Mathewson, 2009; Schyns, 2011) that indicate that phase coding only becomes evident in stages of high power. It is necessary to test the phase vs. power coding and then assess phase coding separately for high and low power trials. This is important, since the decoding results imply that HFA alone might explain some of the findings. FMAX and FMIN values per subject should be reported. In order to establish PAC as a putative mechanism, one needs to rule out that neither HFA nor low frequency phase/power alone can explain the findings.
4) Could the distribution of phase angles at stimulus onset exhibit a systematic bias with respect to the visual stream? To rule out entrainment to the rhythmic stream the authors could employ ITC or phase-locking to the sensory stream (e.g. Thut et al., 2011).
5) A major concern is the use of wavelets for extracting oscillatory phase. Wavelets constitute an acausal approach to determine the phase of a signal, since an underlying oscillation is assumed. This leads to frequency, but also phase smoothing, e.g. that a transient ERP contaminates the pre-stimulus phase estimates by backward ‘smearing’. The paper would benefit from a causal approach. Please consult Zoefel and Heil (2013, FIPSY).
6) The difference score is introduced in the second paragraph of the subsection “HFA occurs at different phases for different categories” and requires earlier explanation. A schematic for OTC and DS would be preferable.
7) Were the phase and power extracted from the same electrode? Was PAC calculated on epoched data? Both things could possibly lead to spurious findings.
8) Why did the authors only analyze the frequency spectrum up to 120 Hz? Often findings in HG in ECoG only start at 100 Hz and then extend up to 250 Hz.
9) Were eye movements recorded in any of the subjects? The authors argue that this is an unlikely source of artifactual data but eye-tracking data would be more assuring.
A growing literature in systems neuroscience has observed a consistent relationship between the timing of spikes in local ensembles, relative to the phase position of local oscillatory field potentials. This putative phase-coding mechanism provides an interesting means of coordinating population coding, and makes clear predictions about temporal constraints on computation. To date, phase-coding has been studied in rodent models and non-human primates. In their submission, Watrous et al. seek to quantify a correlate of phase-coding in human cortex using direct intracranial recordings. As a means of quantifying phase-coding, the authors focus on phase-amplitude coupling (PAC) in the cortical surface potential, specifically low frequency phase modulation of high-frequency amplitude (HFA). This approach is supported by the correlation between HFA and local population spiking, and previous evidence of phase modulation of HFA. In this regard, HFA PAC reflects a macro-scale measure of population phase-coding.
The role of PAC in cognition is an active area of investigation and the author's attempts to link this phenomenon to phase-coding using human intracranial recordings is of great interest. At a general level, the relationship between phase-coding and local field potential PAC is unclear given the population level readout of spiking in the HFA. In their study, Watrous et al. attempt to identify unique PAC based phase-coding for different visual categories. The authors provide a clear motivation, and employ sophisticated analysis methods towards this endeavor.
1) Anatomy: While the authors provide evidence for PAC, and phase based coding of visual categories, their conclusions make no reference to functional neuroanatomy. Recording sites across subjects come from a wide variety of regions; this strikes me as a challenge when interpreting the specific coding of visual categories. The supplemental movie suggests a range of distant regions, not typically associated with higher-level vision, display some degree of phase-coding/representation of visual categories. This seems inconsistent with previous electrophysiology and functional imaging work (which is not cited or discussed in any detail). On a related note, the diversity of recording sites also makes the use of count statistics as somewhat arbitrary. Percentage of electrodes displaying an effect of interest can be equally meaningful for large or small percentages (e.g. low % for anatomically specific effect or high % for a trivial global effect). The authors should clarify why macro-scale phase-coding exists for visual categories across many cortical regions, rather than focused to the more classical regions of categorical selectivity.
2) Task/decoding: In their task, the authors present only four exemplars for each category, repeating each 33 times (if I understand correctly). While this may serve to aid mnemonic encoding, it does limit the claim of categorical decoding. Specifically, in developing a category decoder, the large number of stimulus repetitions limits insight, given the similarity between any training and test set. On a related note, I found the reporting of the basic decoding results unclear (is the decoder working above expected chance levels? >25%). Given the authors’ aims, it seems that a better use of the data would be quantifying the consistency of phase-coding metrics across repetitions of stimuli, as well as within/between class comparison. This approach would focus more on extracting single trial features and testing similarity across repeated trials (I note issues of repetition suppression come into play here). This approach of displaying consistency of stimulus phase-coding would provide more robust evidence for the authors’ claims.
The manuscript by Watrous and colleagues is an interesting look at phase coding in the human cortex and medial temporal lobe. While the authors have a great deal of experience with ECoG analyses, including phase coding and PAC, and their manuscript is generally of interest, I have a number of questions and concerns.
1) In Figure 2H, Figure 2–figure supplement 1, the authors show “low gamma” PAC and “high gamma” PAC. This 32 Hz coupling mode seems striking, because it's likely that coupling extends even further below this range into the beta range. This low gamma has been argued to be distinct from more “broadband” high gamma (Kai Miller and Dora Hermes' work), which is correlated with population spiking (Mukamel, Science; Manning, J. Neurosci.), in contrast to low gamma, which is more oscillatory. Thus, the low gamma effect may be more a form of “nested” coupling as has been argued by Nancy Kopell.
2) There appears to be a disproportionate PAC effect at 0.5Hz and 1.0Hz, but with surprising specificity, and not between those two frequencies as seen in Figure 4–figure supplement 1. Why do the authors believe this occurs, and why do they believe their PAC effects are so restricted to this delta range, in contrast to what others have observed in ECoG?
3) How sensitive is detection of HFA event times to the filtering method?
4) With regards to electrode choice, the rationale for only using electrodes in the contralateral hemisphere is unclear. Why systematically reject an entire hemisphere (except for 1 subject, oddly) when you visually inspect channels for epileptic activity anyway? Additionally, what is the medical justification for implanting patients with electrodes in what is putatively a healthy hemisphere?
1) Watson-Williams test assumes a von Mises distribution. Is this true for the distributions studied here? If not, use the Wheeler-Watson test.
2) For the resampling statistics: the images were shown in groups of four, but the resampling seems to use random permutation. Resampling should be performed such that the labels for the “chunks” should be shuffled, but within these 4-trial chunks, the labels should be kept the same. This would control for any effect of this chunking.
3) Are there still significant differences between categories? How many electrodes have a category with DS=3?
4) It would be nice to also be given an estimate of effect size wherever a p-value is given.
5) For the SVM bootstrapping estimates, are the two bootstrapping experiments actually independent in order to support the expected false alarm rate of 0.42 electrodes?
1) Are there spatial clusters among the electrodes that have phase coding for each of the different categories (c.f. Vidal et al, 2010)?
2) Please make all rose phase plots opaque as in Figure 3C so that we can see the phase distributions for each category.
3) For these phase plots, it would be nice to see the true number of high frequency activity events within each phase bin.
4) It is unclear how Figure 3–figure supplement 1 should be interpreted. For example, the primary effect in the paper is in the delta range, but this figure seems to show poor delta phase clustering. Why?https://doi.org/10.7554/eLife.07886.018
- Nikolai Axmacher
- Juergen Fell
- Nikolai Axmacher
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
NA was supported by funding from an Emmy Noether grant by the DFG (AX82/2). NA and JF received funding via SFB 1089. We thank Hui Zhang and Marcin Leszczynski for comments on an initial manuscript version.
Human subjects: The study was conducted according to the latest version of the Declaration of Helsinki and approved by the local ethics committee, and all patients provided written informed consent.
- Howard Eichenbaum, Reviewing Editor, Boston University, United States
© 2015, Watrous et al.
This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.