1. Neuroscience
Download icon

Fast-backward replay of sequentially memorized items in humans

  1. Qiaoli Huang  Is a corresponding author
  2. Jianrong Jia
  3. Qiming Han
  4. Huan Luo  Is a corresponding author
  1. Peking University, China
Research Article
  • Cited 9
  • Views 3,392
  • Annotations
Cite this article as: eLife 2018;7:e35164 doi: 10.7554/eLife.35164

Abstract

Storing temporal sequences of events (i.e., sequence memory) is fundamental to many cognitive functions. However, it is unknown how the sequence order information is maintained and represented in working memory and its behavioral significance, particularly in human subjects. We recorded electroencephalography (EEG) in combination with a temporal response function (TRF) method to dissociate item-specific neuronal reactivations. We demonstrate that serially remembered items are successively reactivated during memory retention. The sequential replay displays two interesting properties compared to the actual sequence. First, the item-by-item reactivation is compressed within a 200 – 400 ms window, suggesting that external events are associated within a plasticity-relevant window to facilitate memory consolidation. Second, the replay is in a temporally reversed order and is strongly related to the recency effect in behavior. This fast-backward replay, previously revealed in rat hippocampus and demonstrated here in human cortical activities, might constitute a general neural mechanism for sequence memory and learning.

https://doi.org/10.7554/eLife.35164.001

eLife digest

Have you ever played the ‘Memory Maze Challenge’ game, or its predecessor from the 1980s, ‘Simon’? Players must memorize a sequence of colored lights, and then reproduce the sequence by tapping the colors on a pad. The sequence becomes longer with each trial, making the task more and more difficult. One wrong response and the game is over.

Storing and retrieving sequences is key to many cognitive processes, from following speech to hitting a tennis ball to recalling what you did last week. Such tasks require memorizing the order in which items occur as well as the items themselves. But how do we hold this information in memory? Huang et al. reveal the answer by using scalp electrodes to record the brain activity of healthy volunteers as they memorize and then recall a sequence.

Memorizing, or encoding, each of the items in the sequence triggered a distinct pattern of brain activity. As the volunteers held the sequence in memory, their brains replayed these activity patterns one after the other. But this replay showed two non-intuitive features. First, it was speeded up relative to the original encoding. In fact, the brain compressed the entire sequence into about 200 to 400 milliseconds. Second, the brain replayed the sequence backwards. The activity pattern corresponding to the last item was replayed first, while that corresponding to the first item was replayed last. This ‘fast-backward’ replay may explain why we tend to recall items at the end of a list better than those in the middle, a phenomenon known as the recency effect.

The results of Huang et al. suggest that when we hold a list of items in memory, the brain does not replay the list in its original form, like an echo. Instead, the brain restructures and reorganizes the list, compressing and reversing it. This process, which is also seen in rodents, helps the brain to incorporate the list of items into existing neuronal networks for memory storage.

https://doi.org/10.7554/eLife.35164.002

Introduction

Storing and retrieving temporal sequences of events (i.e., sequence memory), a capacity shared across species, is crucial to many cognitive functions, including speech recognition, movement planning, and episodic memory (Doyon et al., 2003; Giraud and Poeppel, 2012). For example, we retrieve a stream of serially ordered events when recalling past personal experience, and we memorize numbers in order when calling a friend on the phone. Swinging the racket to hit an incoming ball in a tennis match similarly involves the planning and controlling of a chain of movement elements over time.

To accomplish sequence memory, two core components – the content (items) and the ordinal information (temporal order) – of a sequence are vital to be encoded and maintained in working memory. There exists ample evidence that neural responses during the retention interval show a sustained load-dependent enhancement (Buschman et al., 2011; Jensen et al., 2002; Klimesch et al., 1999; Sauseng et al., 2009; Vogel and Machizawa, 2004; Xu and Chun, 2006), suggesting that the maintenance of mnemonic contents is implemented by recurrent feedback loops (Luck and Vogel, 2013), as well as suppression of irrelevant information (Sauseng et al., 2009; Bonnefond and Jensen, 2012). That being said, retention of the sequence order information in memory cannot solely rely on an overall enhancement of neural activity and presumably requires the temporally segregated representations of individual items. Theoretical models postulate that sequence memory is mediated by a theta-gamma coupled neuronal oscillatory mechanism (Jensen and Lisman, 2005; Lisman and Idiart, 1995), such that individual items of the list/sequence, encoded in gamma-band activities, occur at the different phases of a theta-band rhythm. A recent MEG study provides important evidence supporting this theta-gamma coupling model during the memory encoding period (Heusser et al., 2016). By asking subjects to mentally replay short sound or video clips, the temporal phase patterns of the neural activities during memory retrieval are found to be similar to that during memory encoding, further advocating preservation of temporal information of the stimulus in working memory (Michelmann et al., 2016). Working memory contents have also been suggested to be maintained in rapid transition in dynamic hidden states (Stokes, 2015; Wolff et al., 2017). However, it remains unknown how the human brain represents and maintains temporal order information during the memory retention period, when the stimulus sequence is not present.

Despite this gap, it is well-established in animal studies that place cells in rodent hippocampus fire in the same or reverse order in which they appeared during previous spatial navigation, in both awake and asleep status (Diba and Buzsáki, 2007; Foster and Wilson, 2006; Louie and Wilson, 2001; Skaggs and McNaughton, 1996). In contrast to rodent studies in which sequential trajectory of place cell firing could be explicitly assessed to explore reactivation profiles during memory maintenance, human studies have provided more limited access to the item-specific reactivation profiles at the neuronal ensemble level. To overcome this difficulty, in the present study, we employed sequence memory paradigms in combination with a temporal response function (TRF) method (Crosse et al., 2016; Jia et al., 2017; Lalor et al., 2006) to dissociate the item-specific neuronal response during the memory retention interval.

Our results demonstrate that each item in a memory list, characterized by an alpha-band neural profile, is successively reactivated to mediate the representation and preservation of ordinal information in memory. Importantly, this item-by-item neuronal profile does not simply echo the sequence in the external world but instead displays two intriguing properties compared to the actual stimulus sequence: reversal in order and compression in time. Specifically, the item that occupies a late position in the list is reactivated earlier (e.g., for sequence 1 – 2, the reactivation profile is 2 – 1 in order), and the whole list encompassing all items is compacted within a 200 – 400 ms temporal window. Most importantly, this backward reactivation profile is directly related to the subsequent recency effect in memory behavioral performance. Our study provides novel neural evidence in humans that the sequence order information is encoded and preserved in a fast and reverse reactivation sequence.

Results

Probing memory-related reactivations during retention period

Previous studies have provided evidence supporting the object-based nature of working memory such that all features of an object, even the task-irrelevant ones, will be stored automatically in working memory (Gao et al., 2011; Hollingworth and Luck, 2009; Hyun et al., 2009; Luck and Vogel, 1997). Most importantly, color features have been found to be the type of feature having the strongest conjunction with other features within an object (Gao et al., 2011; Johnson et al., 2008; Wheeler and Treisman, 2002). Based on these findings, in Experiment 1, we used task-irrelevant color probes that were either memory-related or non-memory-related to examine whether we could tag the memory reactivations during the retention period when the to-be-remembered features (i.e., orientation features) were not present.

We recorded 64-channel EEG signals from human participants performing an adapted working memory task (Sawaki and Luck, 2011). Each trial consisted of three periods: encoding, maintaining, and recalling (Figure 1A). In short, subjects were instructed to memorize the orientation of the cued bar (‘encoding’) and then held the information in working memory (‘maintaining’), and finally performed a memory test on the orientation feature (‘recalling’). During the maintaining period, participants performed a central fixation task (to control eye movements), with two task-irrelevant color discs presented simultaneously. Crucially, the two probes were either memory-related (i.e., with memory-matching color, WM) or non-memory-related (i.e., with memory-nonmatching color, NWM), and neither contained the to-be-memorized orientation features. They were displayed at either the left or right side of the fixation cross to exclude spatial memory effects.

Paradigm for Experiment 1(1-item memory task) and illustration of the TRF approach.

(A) A central arrow cue was first presented to indicate which of the bars the participants should examine. A sample array containing two bars then appeared and participants were asked to memorize the orientation of the cued bar (‘encoding period’). Next, a probe array containing two task-irrelevant disc probes with memory-matching color and non-memory-matching color was presented for 5 s (‘maintaining period’), during which participants performed a central fixation task by monitoring an abrupt luminance change of the central fixation cross, while simultaneously holding the orientation information in memory. In the final ‘recalling period’, participants were asked to judge which bar (upper or lower) had the same orientation as the cued bar. (B) During the 5 s maintaining period, the luminance of the two discs was continuously modulated according to two random temporal sequences respectively. At the same time, electroencephalography (EEG) responses were recorded. (C) The TRF approach was used to calculate the impulse brain response for the memory-related (left, WM-TRF) and non-memory-related (NWM-TRF) disc probes. TRF response characterizes the brain response to a unit increase in luminance of a stimulus sequence, as a function of time lag after each transient. Note that the WM-TRF and NWM-TRF were derived from the same EEG responses but were separated based on the corresponding stimulus luminance sequence. As a result, the computed WM-TRF and NWM-TRF would represent the elicited response for each unit transient at the WM and NWM probes, respectively, throughout the memory retention interval.

https://doi.org/10.7554/eLife.35164.003

Next, a TRF approach (Jia et al., 2017; Lalor and Foxe, 2010; Ding and Simon, 2012), which has been used to assess brain response that tracks ongoing changes in sound envelope (Lalor and Foxe, 2010; Ding and Simon, 2012), visual luminance (Lalor et al., 2006; VanRullen and Macdonald, 2012), and even high-level properties (Liu et al., 2017; Broderick et al., 2018), was employed to extract and isolate neuronal responses for the ongoing luminance change of the two probes (WM and NWM), respectively, throughout the maintaining period. Specifically, the luminance of the two discs was modulated continuously according to two 5 s random temporal sequences that were generated anew in each trial (Figure 1B). The TRF responses (Figure 1C) for the WM disc probe (WM-TRF) and NWM disc probe (NWM-TRF) were then calculated and dissociated from the same 5 s EEG recordings, based on their corresponding stimulus temporal sequences (Figure 1B). Note that the TRF response is defined as the brain response to a unit luminance in the stimulus sequence, as a function of temporal lag after each transient. As a result, the two calculated TRF responses (i.e., WM-TRF and NWM-TRF) would represent the impulse response for the luminance sequence of the WM and NWM probes, respectively, throughout the memory retention interval (see details of TRF approach in Materials and methods). If the color feature is not specifically bound to the associated orientation features held in memory, we would expect no difference between the WM-TRF and NWM-TRF responses. On the other hand, if the two probes show distinct responses, it suggests that the task-irrelevant color probes could be used to tag the memory-related activations during retention.

Experiment 1: Alpha-band memory effects (one-item memory)

Eighteen participants participated in Experiment 1, and their memory performance was well above chance (N = 18, mean accuracy = 0.888, s.e. = 0.029; One-sample t-test, df = 17, t = 26.34, p < 0.001, CI: [0.357, 0.419], Cohen’s d = 6.208). First, the overall TRF response for the color probes showed clear alpha-band patterns and was robust at single-subject level (Figure 2—figure supplement 1), consistent with previous findings (Jia et al., 2017; VanRullen and Macdonald, 2012). We next performed a spectrotemporal analysis on the WM-TRF and NWM-TRF responses (see the corresponding TRF waveforms in Figure 2—figure supplement 2A) to examine their fine dynamic structures as a function of frequency (0 – 30 Hz) and time (0 – 0.6 s), and this was done on each channel and in each participant separately. As shown in Figure 2A, both WM-TRF (left panel) and NWM-TRF (right panel) demonstrated clear sustained alpha-band activations (8 – 11 Hz; permutation test, p < 0.05, corrected, see Figure 2—figure supplement 2C), consistent with previous findings (VanRullen and Macdonald, 2012). Given the prominent alpha-band activations in the TRF responses, we then first selected channels that showed overall significant larger WM + NWM alpha-band responses (one-sample t-test, p < 0.05; compared to the average of all channels). Among the resulting channels, three electrodes (red dots in Figure 2B) passed the statistical test on the WM-NWM alpha-band difference (bootstrap test, p < 0.06, see details in Materials and methods; FDR corrected across channels and time) and were defined as a ‘channel-of-interest’ for allsubsequent analyses, including those in Experiment 2. As shown in Figure 2C, a ‘channel-of-interest’ showed enhanced alpha-band power for WM-TRF compared to NWM-TRF within the latency of around 200 – 500 ms. We then extracted the alpha-band time courses within the channel-of-interest. As shown in Figure 2D, WM (red) showed significant alpha-band enhancement over NWM (black) from 310 ms to 370 ms (bootstrap test, p = 0.046, FDR corrected across time), supporting the alpha-band memory effect.

Figure 2 with 4 supplements see all
Results for Experiment 1 (1-item memory task).

(A) Grand average (N = 18) time-frequency power profile for memory-related (WM, left) and non-memory-related (NWM, right) TRFs as a function of latency (0 – 0.6 s) and frequency (0 – 30 Hz) for all EEG channels. (B) Grand average distribution map for the normalized WM – NWM alpha-band (8 – 11 Hz) power difference in combination with the normalized WM + NWM alpha-band power within the first 0.6 s. Red dots indicate channel-of-interest (Pz, P1, P2) (see details of selection criteria in Materials and methods), which were analyzed for all subsequent analyses, including those in Exp. 2. (C) Grand average time–frequency plots for WM – NWM power difference as a function of latency and frequency in the channel-of-interest. (D) Grand average alpha (8 – 11 Hz) power time courses (mean ±SEM) for WM-TRF and NWM-TRF responses in the channel-of-interest. Black dots at the bottom indicate time points showing significant WM-NWM differences (bootstrap test, p < 0.05, FDR corrected across time points). See also Figure 2—figure supplements 14. The data are provided in source data 1.

https://doi.org/10.7554/eLife.35164.004

The WM and NWM probes were presented in two visual fields, and therefore the observed alpha memory effect could be a result of spatial attention (Worden et al., 2000). We performed two additional analyses to address this issue. First, we examined the alpha-band memory effects (WM – NWM) for the left and right discs separately and observed a similar spatial distribution without statistically significant lateralization effects (paired t-test, N = 18; see Figure 2—figure supplement 3A). Second, we compared the alpha power between channels that are contralateral to the WM probe and channels contralateral to the NWM probe and did not find any significant difference (paired t-test, N = 18; see Figure 2—figure supplement 3B). Thus, the alpha-mediated memory effects during maintenance could not be interpreted by spatial attention. This is also consistent with recent findings supporting memory-related hidden states in the absence of attention (Wolff et al., 2017).

It is noteworthy that during the central fixation task (see eye movement profile in Figure 2—figure supplement 4B), WM and NWM probes were both task-irrelevant, and the only difference between them regarded color (i.e., memory-related or non-memory-related). Therefore, the observed difference between the WM-TRF and NWM-TRF responses indicates that the task-irrelevant color probes did carry memory-related information, consistent with previous findings advocating the object-based nature of working memory (Gao et al., 2011; Hollingworth and Luck, 2009; Hyun et al., 2009). The results also suggest that we could use task-irrelevant color probes to tag the memory-associated reactivations during maintenance, using the alpha-band temporal profile as a neural signature (no effects in other frequency bands, Figure 2—figure supplement 3C). See further analysis on the alpha-band in TRF responses in Figure 2—figure supplement 4A.

Experiment 2: Sequential alpha-band response for sequence memory

We next examined how a temporal sequence and the associated order information is maintained in working memory. Nineteen participants participated in Experiment 2. As shown in Figure 3A, participants were first presented with a sample array containing three bars with different orientations and different colors. Next, two of the three bars were randomly selected and serially cued, and importantly, participants were asked to memorize not only the orientation of the two cued bars but also their temporal order (i.e., the 1st and the 2nd orientation). During the following 5 s ‘maintaining period’, participants performed a central fixation task and held the orientation sequence in memory. In the final ‘recalling period’, participants compared the orientation of the bar in the test array with the memorized orientations (e.g. is it more similar to the orientation of the 1st or the 2nd cued bars?). Therefore, subjects should be memorizing the temporal order of the orientation sequence.

Figure 3 with 2 supplements see all
Paradigm and results for Experiment 2 (two-item sequence memory task)(A) Participants were first presented with a sample array containing three bars with different orientations and different colors.
https://doi.org/10.7554/eLife.35164.009

Crucially, the probe array during the maintaining period consisted of three disc probes – one 1st-memory-related probe (i.e., matching color with the 1st cued bar), one 2nd-memory-related probe (i.e., matching color with the 2nd cued bar), and one non-memory-related probe (i.e., matching color with the non-cued bar). The three discs were presented at three random spatial locations on a ring to exclude spatial memory effects. Again, the luminance of the three disc probes was modulated continuously according to three random sequences respectively. The corresponding TRF responses (1st-WM-TRF, 2nd-WM-TRF, and NWM-TRF) were then computed from the same 5 s EEG recordings, and a spectrotemporal analysis was performed on them for each channel and in each participant separately (see TRF waveforms in Figure 2—figure supplement 2B). Notably, all the analysis was performed in the channel-of-interest, which was independently defined in Experiment 1 (see Figure 2B).

First, significant recency effect (i.e., 2nd item better than 1st item) in behavioral performance was found (N = 19, 1st item: mean = 0.794, s.e. = 0.020; 2nd item: mean = 0.840, s.e = 0.014; recency effect: mean = 0.047, s.e. = 0.020; paired t-test, df = 18, t = 2.378, p = 0.029, CI: [0.005, 0.089], Cohen’s d = 0.532). Moreover, the alpha-band memory effects (i.e., WM – NWM difference averaged within the whole 600 ms time range, bootstrap test, p = 0.047) revealed in Experiment 1 were still present(Figure 3B). Most interestingly, the 1st-WM-TRF and the 2nd-WM-TRF displayed distinct alpha-band temporal courses (Figure 3C): the 1st-WM-TRF alpha-band response (red line) was temporally delayed by approximately 200 ms relative to that of the 2nd-WM-TRF (blue line) response (alpha-band peak latency, Wilcoxon signed-rank test, p = 0.036). We next normalized the alpha-band power temporal profiles by subtracting the averaged alpha-band time courses across all the three TRFs from that for 1st-WM-TRF, 2nd-WM-TRF, and NWM-TRF, respectively. Figure 3D illustrates the normalized alpha-band temporal profile, showing a reverse sequential activation profile (i.e., alpha-band activation for the 1st item followed by that for the 2nd item). Specifically, the 2nd-WM-TRF showed a response at around 80–100 ms, whereas the 1st-WM-TRF showed a response at around 260–300 ms (p < 0.05, one-sided t-test). The backward sequential response profiles were consistent across electrodes and items (see control analysis for each channel and for different item combinations in Figure 3—figure supplement 1CD).

To examine the statistical significance of the sequential activation, we calculated the cross-correlation coefficient between the normalized alpha-band profile for the 1st-WM-TRF and the 2nd-WM-TRF responses and then performed a permutation test by shuffling condition labels. As shown in Figure 3E (top panel), the 1st-to-2nd cross-correlation coefficient at a temporal lag of around –170 ms was significant (permutation test, p < 0.05, corrected), supporting that the 2nd item alpha-band activations essentially preceded those of the 1st item (i.e., negative temporal lag) by around 170 ms. Notably, the sequential activation pattern was still present for direct comparisons between the 1st-WM-TRF and 2nd -WM-TRF responses (Figure 3—figure supplement 1A; Figure 3—figure supplement 1B), supporting that it was not involvement of the NWM item (i.e., normalization) that caused the results. Furthermore, we observed a similar but weaker trend of reverse serial reactivation profile in a three-item sequence memory task (Figure 3—figure supplement 2).

Finally, in addition to the item-by-item sequential responses, we also observed a trend of repeated activations for the same item (Figure 3D). We thus calculated the within-item autocorrelation coefficient to examine the temporal period of the recurrent activations. As shown in Figure 3E (bottom panel), the temporal lag of ~390 ms was statistically significant (permutation test, shuffling between condition labels, p < 0.05, corrected). This suggests that the two items in the memory list are further organized into a recurring temporal chunk of approximately 400 ms, within which sequentially memorized items reactivate one after another in a reversed order.

Relation to sequence memory behavior

We further evaluated whether the alpha-mediated memory reactivation during the retention period might have behavioral relevance. We first examined the relationship between the WM – NWM alpha-band enhancement and the one-item memory performance in Experiment 1. Participants were grouped based on their memory accuracy into ‘high-performance’ (N = 9, mean = 0.907, s.e. = 0.018) and ‘low-performance’ (N = 9, mean = 0.796, s.e. = 0.053) (Figure 4A). We then used the same analysis to assess the alpha-band courses for the two groups separately. Similar to the grand average results (Figure 2D), the high-performance group showed enhanced WM – NWM alpha-band power difference within 300–400 ms (Figure 4B), whereas the low-performance group showed weaker enhancement (Figure 4C). As shown in Figure 4D, the high- and low-performance groups displayed non-overlapping confidence intervals in the alpha-band memory effects (WM – NWM alpha power difference averaged from 310 to 370 ms, a time range selected based on group results, see Figure 2D). Thus, the stronger the alpha-band memory effects (i.e., Alpha WM > Alpha NWM), the better the corresponding memory behavioral performances.

Figure 4 with 1 supplement see all
Relationship between memory behavior and alpha-band TRF profiles.

(A) Participants in Experiment 1 were grouped based on the memory behavioral accuracy into ‘high-performance’ and ‘low-performance’. Grand average (mean ±SEM) memory accuracy (MA) for all participants (N = 18), high-performance group (N = 9), and low-performance group (N = 9). (B) Grand average alpha-band power time courses (mean ±confidence interval) for the high-performance group (red: WM-TRF; black: NWN-TRF). (C) Grand average alpha-band power time courses (mean ±confidence interval) for the low-performance group (red: WM; black: NWN). The shaded box (310-370 ms) representing the time range for the alpha-band memory effect (WM – NWM), was defined based on group results (see Figure 2D). (D) Averaged alpha-band memory effect (WM – NWM) within the time range (shaded box in Figure 4BC) for the high- and low-performance groups (mean ±confidence interval). (E) Participants in Experiment 2 were grouped into two groups based on memory recency effects (MA for 2nd item –MA for 1st item). Grand average (mean ±SEM) memory recency effect for high-recency group (N = 9) and low-recency group (N = 9). (F) Grand average normalized alpha-band power time courses (mean ±confidence interval) for the high-recency group (red: 1st WM; blue: 2nd WM). The three shaded boxes (phase 1: 140 – 180 ms; phase 2: 280 – 320 ms; phase 3: 470 – 510 ms), representing the time ranges for the sequential effect (2nd WM − 1st WM), were defined based on the group results (see Figure 3D). (G) Grand average normalized alpha-band power time courses (mean ±confidence interval) for the low-recency group (red: 1st WM; blue: 2nd WM). (H) 2nd − 1st alpha-band difference averaged within phase 1, phase 2, and phase 3 (shaded box in Figure 4FG) for the high-recency (grey) and low-recency (white) groups (mean ±confidence interval). The two groups showed non-overlapping confidence intervals for the 2nd −1st alpha-band difference in phase 1 and phase 3. All the confidence intervals were calculated using a jackknife procedure. See also Figure 4—figure supplement 1. The data are provided in source data 3.

https://doi.org/10.7554/eLife.35164.012

Participants in Experiment 2 (2-item sequence memory) were divided into two groups based on their memory recency effects (memory performance 2nd item – memory performance 1st item): the ‘high-recency’ group (N = 9, mean = 0.116, s.e. = 0.020) and the ‘low-recency’ group (N = 9, mean = –0.021, s.e. = 0.017) (Figure 4E). First, both groups (Figure 4FG) showed similar reverse sequential responses profiles, similar to those in the group results (see Figure 3D). Notably, the ‘high-recency’ group displayed a strong and significant backward-sequential alpha profile (Figure 4F; permutation test, 1 – 2 cross-correlation coefficient at a lag of −170 ms, p < 0.05, Figure 4—figure supplement 1A), whereas the ‘low-recency’ group displayed weak and nonsignificant sequential responses (Figure 4G; permutation test, 1 – 2 cross-correlation coefficient at lag of −170 ms, p = 0.28; Figure 4—figure supplement 1). To further compare the difference in reactivation profiles between the two groups, we examined the 2nd-1st alpha power difference for the two groups at three time ranges (phase 1: 140 – 180 ms; phase 2: 280 – 320 ms; phase 3: 470 – 510 ms; shaded regions in Figure 4FG), which were selected based on group results (Figure 3D). As shown in Figure 4H, interestingly, the main difference between the high- and low-recency groups was on the 2nd TRF response (Figure 4FGH). Specifically, the high-recency group elicited a stronger 2nd item response than the low-recency group (phase 1 and phase 3, non-overlapping confidence intervals, Figure 4H; permutation test by shuffling between high- and low-recency groups, p = 0.08), whereas both groups elicited similar 1st item responses (phase 2, Figure 4H; permutation test by shuffling between high- and low-recency groups, p = 0.48). Thus, our results support an essential association between the backward sequential reactivations and recency effect in sequence memory behavior. We also performed an additional analysis by dividing subjects based on their memory accuracy rather than the recency, and the two groups elicited quite similar backward sequential responses (Figure 3—figure supplement 1E), excluding the possibility that it is memory strength rather than the recency effect that leads to the observed backward sequential responses.

Taken together, the alpha-mediated reactivation profile during the memory retention period is related to subsequent memory behavioral performance. Importantly, there exists an essential relationship between the backward item-by-item reactivation during maintenance and the subsequent recency effect, a behavioral index for sequence memory.

Response bias control

In Experiment 2, subjects were instructed to decide whether the orientation of the bar in the test array was similar to the orientation of the 1st or the 2nd memorized orientations, and the behavioral performance showed a significant recency effect (i.e., 2nd item better than 1st item). Meanwhile, although the accuracy for both items was high, response bias might still be a confounding factor for the observed recency effect (e.g., biased to report the probe to better resemble the 2nd item). To address the issue, we ran a control study (N = 11) employing a new behavioral procedure that would not be contaminated by response bias. Specifically, the experiment paradigm was the same as that for Experiment 2 except that rather than making a binary response in the recalling period, subjects were instructed to rotate a bar to the memorized orientation. By calculating the angular deviation between the reported orientation and the true orientation of the memorized bar, in combination with a probabilistic model (Bays et al., 2009), we could estimate the target response probability for the 1st and the 2nd WM items separately, independent of response bias. As shown in Figure 5A, the target response probability was significantly larger for the 2nd item than for the 1st item (df = 10, one-tail paired t-test, t = 2.052, p = 0.034, CI: [−0.003, 0.083], Cohen’s d = 0.618), confirming the recency effect and excluding the response bias interpretation.

Response bias control and pooled results.

(A) Grand average (mean ±SEM) target probability estimation in the control study (N = 11) for the 1st (red) and 2nd (blue) memorized orientation. The 2nd WM showed larger target probability than the 1st WM (*: p < 0.05, one-sided paired t-test), confirming the recency effect and excluding the response bias interpretation. Note that the procedure was the same as that for Experiment 2 except during the recalling period, subjects were instructed to rotate a bar to the memorized orientation rather than making a binary response. We next pooled the neural data from the control study and that from Experiment 2, resulting in a 30-subject dataset. (B) Grand average (N = 30) normalized alpha power time courses (mean ±SEM) for 1st-WM-TRF (red) and 2nd-WM-TRF (blue) of the pooled data. Black dots indicate points at which the alpha power was significantly above zero (p < 0.05, one-sided paired t-test, uncorrected). (C) Cross-correlation coefficient between the normalized alpha-band profile for the 1st-WM-TRF and the 2nd-WM-TRF responses of the pooled data. The dashed lines represent the corresponding statistical significance threshold (p < 0.05, permutation test, corrected). The data are provided in source data 4.

https://doi.org/10.7554/eLife.35164.014

Furthermore, we combined the neural recordings in the control study (N = 11) with those from Experiment 2 (N = 19), resulting in a 30-subject dataset. As shown in Figure 5B, the normalized alpha-band reactivation profile (N = 30) showed a similar reverse sequential pattern. The 1st-to-2nd cross-correlation coefficient at a temporal lag of around –170 ms was significant (permutation test, p < 0.05, corrected; Figure 5C), again supporting that the 2nd item alpha-band activations essentially preceded those for the 1st item (i.e., negative temporal lag) by around 170 ms.

Discussion

We recorded EEG responses in human participants performing sequence memory tasks and employed a TRF approach to probe the neuronal response that specifically tracks each item of the temporal sequence during memory retention. Our results consistently demonstrate that individual items are successively reactivated, characterized by a sequence of alpha-band activities. Compared to the actual stimulus sequence, the serial neural replay is temporally compacted within a 200 –400 ms chunk and is reversed in temporal order. Crucially, the backward reactivation is associated with the recency effect, a behavioral index signifying backward memory priority. Taken together, our results constitute novel neural evidence in humans that the sequence order information is encoded and maintained in short-term memory by fast and backward serial reactivations.

Temporally sequenced activations have also been reported in several recent human studies. For example, after being trained in a reasoning task that involves selecting a statistically related object path, the brain spontaneously displays a fast successive representation of states (Kurth-Nelson et al., 2016). Recently, an fMRI study demonstrated that flashing only the starting point of a learned sequence would trigger a prediction wave of responses in primary visual cortices (Ekman et al., 2017). Finally, a MEG study examined the theta-gamma coupling strength as participants were presented with picture sequences, and found that the gamma power shifts along the phase of a theta rhythm as more items are added to memory, thus supporting the ‘phase coding’ model for episodic memory formation (Heusser et al., 2016). Our results are different from this important work and reveal distinct mechanisms. First, we examined the memory retention period when items were not presented, whereas the previous work studied the encoding period when items appeared serially. Second, we demonstrate backward reactivations whereas the previous study supports a forward-playing profile, further suggesting a fundamental distinction between memory encoding and retention.

We demonstrate that the item-by-item reactivation during memory retention period is temporally reversed in order. Previous neurophysiological recordings in monkeys have revealed forward serial activations in prefrontal population activity (Siegel et al., 2009). Sequences within the hippocampus in rats have been observed in both forward and reverse order (Diba and Buzsáki, 2007; Foster and Wilson, 2006; Louie and Wilson, 2001; Skaggs and McNaughton, 1996), and interestingly, the backward sequence has been shown during awake periods immediately after spatial experience (Foster and Wilson, 2006). A recent MEG work found a backward trajectory of representations of states (Kurth-Nelson et al., 2016) during a reasoning task. Why is the sequential reactivation reversed in temporal order during memory maintenance? One interpretation comes from a reinforcement learning model, which proposes that the recent item would be the starting and anchoring point for propagating information backward along incoming trajectories to build a recency-weighted running average of the rewards received for each action taken (Foster and Wilson, 2006; Bornstein et al., 2017; Gershman and Daw, 2017). Another interesting possibility is derived from the ‘activity-silent’ model (Stokes, 2015), which proposes that the contents of working memory are mediated by dynamic hidden states. These memory-related hidden states could be pinged and inferred out during maintenance in the absence of attention and lingering delay activity (Wolff et al., 2017). In our experiments, the TRF response could be regarded as the emerging item-specific activation after each pinging (i.e., each unit luminance transient in the probe stimulus). Consequently, the item associated with stronger excitability (i.e., the recent item) would appear with shorter latency compared to the item associated with weaker excitability (i.e., the early item), resulting in reversed sequential reactivation as we observed here. Furthermore, instead of presenting each item sequentially (Siegel et al., 2009), here all the items in the list were displayed simultaneously followed by serial cuing (to exclude the possibility that serial presentation might result in different encoding strengths). How the sequence is presented (sequential or simultaneous) could be another factor accounting for the order of reactivations.

Memory reactivations during the rehearsal period are compressed in time, in agreement with previous neurophysiological recordings in animals (Foster and Wilson, 2006; Euston et al., 2007; Ji and Wilson, 2007; Skaggs et al., 1996; Xu et al., 2012). Reorganization of events with regard to the internal temporal framework, presumably mediated by neuronal oscillations of various rhythms (e.g., theta, alpha, gamma, etc.), may contribute to memory consolidation. Discrete epochs of events that are temporally separated in the external world can be successfully associated in the brain within an appropriate time frame, which could be vital for operation of the neuronal plasticity mechanism (Jensen and Lisman, 2005; Nabavi et al., 2014), and a similar time compression mechanism could mediate sequence maintenance in working memory.

A recent EEG study revealed that attention samples multiple visual objects in a temporally sequential manner even when subjects maintained sustained attention on one object (Jia et al., 2017). We would argue that sequential attentional sampling could not account for the present observations. First, a previous study showed that attention would first sample the item associated with highest attentional priority and then would shift to an item associated with less attentional priority. However, here, all items in the sequence should be attended to with the same priority and thus would not be sequentially sampled by attention. Second, a previous sequential pattern (attended before unattended) could not reasonably account for the backward reactivation. Furthermore, the backward sequential pattern and its association with the recency effect could be explained by a ‘memory strength’ interpretation. For example, the better the memory strength is of the item (e.g., better memory strength for the 2nd item over the 1st item), the earlier would be the response latency (e.g., earlier response for the 2nd item compared to the 1st item). To test the account, we performed a control analysis by grouping subjects in Experiment 2 into two groups based on their memory accuracy rather than the recency, and examined the respective response profiles. If the alternative interpretation were correct, we would expect to see a temporal shift in the response pattern between the two groups such that the high-accuracy group would show earlier response compared to the low-accuracy group. However, the two groups elicited quite similar backward sequential responses (Figure 3—figure supplement 1E), excluding the possibility that it is memory strength rather than a recency effect that leads to the observed backward sequential responses

Finally, our results exhibit alpha-mediated responses during memory retention, consistent with previous findings on working memory (Jensen et al., 2002). It is well acknowledged that alpha-band neuronal activities have a key function in working memory by efficiently suppressing distracting information (Bonnefond and Jensen, 2012). For instance, increased memory load is associated with enhanced alpha-band activities (Jensen et al., 2002; Klimesch et al., 1999; Tuladhar et al., 2007), and repeated transcranial magnetic stimulation (TMS) at alpha frequency modulates the short-term memory capacity (Sauseng et al., 2009). Memory reinstatement has been found to be accompanied by alpha-band activities, which carry a temporal phase signature of the replayed stimulus (Michelmann et al., 2016). In addition to the traditionally posited inhibitory function in many cognitive processes (Sauseng et al., 2009; Bonnefond and Jensen, 2012; Klimesch, 2012; Händel et al., 2011), alpha-band neuronal oscillations have recently been found to implement an ‘‘echo’’ or reverberation of the inputs. These alpha echoes are indeed enhanced by visual attention, supporting an additional role of the alpha-band rhythm in the maintenance of sensory representations over time (VanRullen and Macdonald, 2012). Taken together, we reason that the memory-related alpha response in our results reflects memory reactivations rather than inhibition, based on several facts. First, previous studies using a TRF approach have revealed enhancement in alpha-echoes under attended conditions (Jia et al., 2017; VanRullen and Macdonald, 2012), which could not be interpreted by inhibitory account. Second, the memory-related alpha responses did not show spatial attention effects (see Figure 2—figure supplement 3A), which are known to be mediated by inhibitory alpha activities (Worden et al., 2000). Finally, WM showed stronger alpha responses than NWM here and were strongly associated with memory behavior, further excluding the inhibitory interpretation. We propose that the alpha-band activity in TRF responses might represent the stimulus-induced phase resetting of the intrinsic alpha-band oscillation in EEG activities (Makeig et al., 2002), given their close associations (Figure 2—figure supplement 4A).

In conclusion, our results provide converging evidence that sequence order information is represented and consolidated in short-term memory by an item-by-item serial reactivation mechanism. Crucially, the serial reactivation, temporally compressed (i.e., within a 200 – 400 ms window) and reversed in temporal order (i.e., backward reactivation), is essentially associated with the subsequent memory behavioral performance. This fast-backward replay during maintenance, previously revealed in rat hippocampus for spatial navigation and shown here in humans for a non-spatial task, might therefore constitute a general neural mechanism for sequence memory and learning.

Materials and methods

Participants

Twenty subjects participated in Experiment 1. Twenty subjects participated in Experiment 2. Eleven subjects participated in the control study. During data collection and preprocessing, two subjects in Experiment 1, and one subject in Experiment 2 were excluded because of overall low behavioral performance, excessive eye movements, or not finishing the whole experiment. All participants had normal or corrected-to-normal vision and had no history of psychiatric or neurological disorders. All experiments were carried out in accordance with the Declaration of Helsinki. All participants provided written informed consent prior to the start of the experiment, which was approved by the Research Ethics Committee at Peking University (2015-03-05c2).

Stimuli and tasks

Participants sat in a dark room in front of a CRT monitor with 100 Hz refresh rate, and their heads were stabilized using a chin rest. In each trial of all experiments, there were three periods: the encoding period, maintaining period, and recalling period (Figure 1A). Participants were asked to only memorize the orientation of the cued bars (0.38°×1.15° visual angle).

During each trial, all participants were instructed to keep the number of eye blinks to a minimum. Eye movements were monitored using an EyeLink 1000 eye tracker (SR Research). The experimental trial was initiated only when fixation was located within 1° visual angle of the fixation cross. Results showed that the participants maintained good fixation on the central cross (within 1°) throughout each trial (Figure 2—figure supplement 4B).

Experiment 1 (one-item memory)

Request a detailed protocol

In Experiment 1, participants were asked to memorize the orientation of the cued bar (Figure 1A). At the beginning of each trial, a central arrow cue (1 s duration) pointing to upper or lower locations was presented to indicate which of the following bars the participants should examine. A sample array containing two bars then appeared at 3° visual angle above and below the fixation for 1 s. One bar was red and the other was blue. The orientation of one bar was chosen randomly within a range between 25° and 65° and the other was between 115° and 155° (0° is vertical). The colors and orientations at each location were counterbalanced across trials. Participants were instructed to memorize the orientation of the bar at the cued location (‘encoding period’). After a blank interval (0.6 ~ 1 s), a probe array containing two task-irrelevant discs (3° in radius) with either memory-matching color or non-memory-matching color were presented to the left or right of the fixation (7° in eccentricity) for 5 s (‘maintaining period’). The colors of the two disc probes were counterbalanced across trials. During this 5 s maintaining period, participants performed a central fixation task by continuously monitoring an abrupt luminance change of the central fixation cross, while simultaneously holding the orientation information of the cued bar in memory. In 25% of trials, luminance of the fixation cross would dim for 0.2 s during 1 s to 3.8 s, and participants were asked to make a self-paced response afterward. The purpose of this fixation task was to control eye movements during memory maintenance, given the presence of two luminance-changing color probes. In the final ‘recalling period’, a test array consisting of two bars appeared at 3° visual angle above and below the fixation. One bar was in the same orientation as the memorized bar, whereas the other was tilted +20° or −20° relative to the memorized orientation. The locations (upper or lower) of the matching and nonmatching bars were counterbalanced across trials. Participants would judge which bar (upper or lower) had the same orientation as that of the cued bar. Participants completed 128 trials in total.

Experiment 2 (two-item sequence memory)

Request a detailed protocol

Experiment 2 was a two-item sequence memory task. Different from Experiment 1, participants were asked to memorize not only the orientations of two serially cued bars but also their temporal order (Figure 3A). At the beginning of each trial, three bars with different orientations and different colors (red, green or blue) were presented at the same eccentricity to the fixation (3° visual angle). Next, two of the three bars were randomly selected and serially cued by dimmed luminance to inform participants to memorize their orientations and temporal order (orientation of the 1st cued bar and the orientation of the 2nd cued bar). The orientation of the first memorized bar was randomly selected between 30° and 150°. The second memorized bar was tilted +25° or −25° relative to the first memorized bar. The non-memorized bar was tilted +12.5° or −12.5° relative to the first memorized bar. Next, three disc probes with the same colors as that of the three bars were presented for 5 s. The locations of three probes were randomly selected on a ring with a radius of 7° to exclude spatial memory effects (‘maintaining period’). During this period, participants were asked to do a fixation task by continuously detecting whether the luminance of fixation cross changed. At the end of each trial, a test bar was presented at the center of the screen with orientation tilted +5° or −5° relative to the 1st memorized or 2nd memorized bar. Participants should compare the orientation of the bar with the memorized orientations (e.g. is it more similar to the orientation of the 1st or the 2nd cued bars?). Participants completed 96 trials in total.

Control study (two-item sequence memory)

Request a detailed protocol

In the control study, all procedures were the same as in Experiment 2 except that during the recalling period, rather than making a binary response, subjects were instructed to rotate a test bar to either the 1st memorized orientation or the 2nd memorized orientation, and the orientations of the two bars were randomly selected between 0° and 180°. By calculating the angular deviation between the reported orientation and the true orientation of the memorized bar, in combination with a probabilistic model (Bays et al., 2009), we could estimate the target response probability for the 1st and the 2nd WM item separately, independent of response bias.

Luminance modulation

Request a detailed protocol

A TRF approach was employed to examine and dissociate the probe-specific impulse responses during the 5 s memory maintaining period. To enable the TRF estimation, the luminance of each of the disc probes was independently modulated in time at each refreshing frame (100 Hz monitor refresh rate), according to the corresponding temporal sequence that was generated anew in each trial (two temporal sequences in Experiment 1; three temporal sequences in Experiment 2). Each random sequence was tailored to have equal power at all frequencies by normalizing the amplitudes of its Fourier components before applying an inverse Fourier transform. Luminance sequences ranged from dark (0 cd/m2) to bright (13 cd/m2).

EEG recordings

Request a detailed protocol

EEG was recorded continuously using a 64-channel EasyCap and two BrainAmp amplifiers (BrainProducts). Vertical and horizontal electrooculograms were recorded by two additional electrodes around the participants’ eyes. EEG data were offline band-pass filtered between 2 and 50 Hz. Independent component analysis was performed independently for each subject to remove eye-movement and artifact components, and the remaining components were back-projected onto the EEG electrode space. All channels were then referenced to the average value of all channels. The EEG was downsampled to 100 Hz before TRF calculation to be matched with the temporal resolution of the luminance sequences. To avoid the influence of the onset EEG response in each trial, which may bias the estimated TRF results, we extracted the middle part of the 5 s EEG trial responses (0.5 – 4.5 s) for further TRF calculation.

Data analysis

TRF computation

Request a detailed protocol

The mapping between the stimulus luminance sequence and the recorded EEG data was characterized by TRF response using the multivariate temporal response function (mTRF) toolbox (Crosse et al., 2016; Lalor et al., 2006). The TRF response describes the brain’s response for unit transient in the stimulus temporal sequence and characterizes the brain’s linear transformation of a stimulus input, S(t), to the neural response output, R(t), as R(t) = TRF*S(t), where * denotes the convolution operator. Specifically, the TRF response for each probe (e.g., 1st WM TRF, 2nd WM TRF, and NWM TRF), as a function of temporal lag, was calculated from the same EEG recordings, based on the corresponding stimulus temporal sequences (Figure 1B). Specifically in the present experiment, the resulted TRF responses would characterize the neural response for each unit transient of the luminance sequence at each probe (e.g., 1st WM, 2nd WM, and NWM TRF) throughout the memory retention interval. In each experiment, the TRF analysis was performed for each probe, on each channel, and in each subject respectively.

The TRF analysis, based on the linear relationships between the luminance sequence and the brain response, has been previously used and validated in both visual and auditory studies to examine the brain response that specifically tracks sound envelope or luminance sequence (Jia et al., 2017; Lalor and Foxe, 2010; Ding and Simon, 2012; VanRullen and Macdonald, 2012). Recently, TRF has even been revealed to track high-level features, such as global form coherence (Liu et al., 2017) and semantic dissimilarity (Broderick et al., 2018). In fact, the TRF approach is quite a robust and conservative way to assess the neural response to luminance change. Previous studies have also demonstrated that applying a quadratic extension of the linear TRF approach only resulted in slight improvement compared to a linear model (Lalor et al., 2008; Power et al., 2011), further advocating the linear assumption in TRF calculation.

Reliable estimation of TRF response requires neural signals with enough temporal length as well as the corresponding length of stimulus sequences, from which the TRF response characterizing their linear relationship could be calculated. Previous study has proposed that neural signals of at least 120 s are needed to obtain TRF responses with reasonable SNR (Handy, 2009). Theoretically, dissociating multiple TRFs from brain signals would require even longer signals. Back to our studies, each trial was 5 s, and we only analyzed the middle 4 s to avoid influences from onset and offset response. Therefore, to obtain TRF responses for multiple probes (e.g., two TRFs in Experiment 1 and three TRFs in Experiment 2), we would need at least 40 trials for the calculation. Therefore, TRF calculation at single-trial level, unless the trial is designed to be very long, could not meet the signal length requirement.

Time-frequency analysis

Request a detailed protocol

The obtained TRF responses were then analyzed with MATLAB (MathWorks, Inc., Natick, Massachusetts), using the FieldTrip toolbox (Oostenveld et al., 2011) and wavelet toolbox functions to examine the spectrotemporal power profiles. Specifically, the TRF temporal profile was transformed using the continuous complex Gaussian wavelet (order = 4; for example, FWHM = 1.32 s for 1 Hz wavelet) transform (Wavelet toolbox, MATLAB), with frequencies ranging from1 to 30 Hz in increments of 1 Hz, for each probe, on each channel, and in each subject respectively. The alpha-band (8 – 11 Hz) power profiles were then extracted from the output of the wavelet transform for further analysis.

Channel-of-interest

Request a detailed protocol

Given the prominent alpha-band activations in the TRF responses, we first selected channels that showed overall significant larger WM + NWM alpha-band TRF responses (one-sample t-test, two-tailed, p < 0.05; compared to the average of all channels) within the 0.6 s in Experiment 1. Among the resultant 17 channels (P7, P5, P3, P1, Pz, P2, P4, P6, P8, PO7, PO3, POz, PO4, PO8, O1, Oz, O2), we next searched those showing significant memory effect (difference between WM and NWM conditions) using a bootstrap procedure with multiple comparison correction. Three parietal electrodes (Pz, P1, and P2) passed the test and were selected as a ‘channel-of-interest’ for all subsequent analyses, including those in Exp. 2 and the control study. The alpha-band power profiles were then averaged within the channel-of-interest for all subsequent analyses. It is notable that two channels (Pz, P2) passed the test with p ≤ 0.05, and three channels (Pz, P1, P2) passed the test for p ≤ 0.06 (marginally significant). We then set the p threshold as 0.06 to incorporate three channels to increase the signal-to-noise ratio. Furthermore, selection first based on the overall EEG alpha power led to the same channel selection.

Normalization

Request a detailed protocol

The alpha-band power profiles of the TRF responses were normalized by subtracting the averaged alpha-band time courses across all the TRFs (e.g., WM and NWM in Experiment 1; 1st WM, 2nd WM, and NWM in Experiment 2) from each alpha-band TRF profile and in each subject respectively. It is notable that in Experiment 1, given there was only one WM item, the normalization WM – (WM + NWM)/2 was equal to (WM – NWM)/2. We were more interested in examining the item-specific activation profile relative to the general TRF response (e.g., how item one was different from all the other probes), and therefore chose the averaged alpha-band profile across all probes rather than that of the NWM probe only.

Correlation coefficient calculation

Request a detailed protocol

To examine the sequential activation in Experiment 2, we calculated the cross-correlation coefficient between the normalized alpha-band profile for the 1st-WM-TRF and the 2nd-WM-TRF responses (Experiment 2).

Bootstrapping test

Request a detailed protocol

A bootstrap procedure with multiple comparison correction was used to search channels-of-interest showing memory effects (i.e., WM – NWM alpha-band difference). Specifically, for each of the 17 channels showing prominent alpha-band response, we randomly resampled the WM – NWM alpha-band activities with replacement 18 times for the 18 participants in Experiment 1 and computed the means of the estimated measures for the resampled participants. This procedure was repeated 1000 times. As a result, the distribution of the resampled WM – NWM alpha difference data was obtained and was compared to 0 to calculate the corresponding p-value for each time point and each channel. Next, we used FDR to correct the multiple comparison across channels (17 channels) and time (0 – 600 ms).

To examine the time range for the WM – NWM alpha-band difference within the channel-of-interest in Experiment 1, a bootstrapping procedure (bootstrapping number = 1000) was used to resample the WM – NWM alpha-band difference time course with replacement and the corresponding p-value was calculated for each time point. Next, a FDR was used to correct the multiple comparison across time. To examine the overall WM – NWM alpha-band difference within the channel-of-interest in Experiment 2, a bootstrapping procedure (bootstrapping number = 1000) was used to resample the WM – NWM alpha-band difference averaged over the first 600 ms with replacement, and the corresponding p-value was calculated.

Permutation test

Request a detailed protocol

To examine the spectral contents of the TRF responses across all channels in Experiment 1, we did a permutation test by shuffling the relationship between stimulus sequence and EEG signals 500 times, from which the 0.05 threshold level as a function of frequency was estimated. Multiple comparison was corrected by setting the maximum threshold across frequencies as the corrected threshold.

To test the statistical significance of the sequential activations (cross-correlation coefficient) in Experiment 2, we performed a permutation test by shuffling the labels across conditions (i.e., across 1st-WM-related, 2nd-WM-related, and NWM-related probes). This shuffling was performed in each subject separately. We then used the same analysis for the original data to recalculate the normalized TRF responses and the corresponding cross-correlation coefficients as a function of time lag. The analysis was repeated 500 times, and a distribution for the cross-correlation coefficient at each time lag was obtained, from which the 0.05 permutation threshold was estimated. The largest threshold across all temporal lags was set as the multiple-comparison corrected threshold (corrected). The same permutation was performed to test the within-item autocorrelation coefficients. To test the statistical significance of the sequential activations (cross-correlation coefficient) in three-item sequence memory, we performed a permutation test by shuffling the labels across conditions (i.e., across 1st-WM-related, 2nd-WM-related, 3nd-WM-related, and NWM-related probes). To test the statistical significance of the sequential activations in the raw data (unnormalized responses) in Experiment 2, we performed a permutation test by shuffling the labels across the 1st-WM-related and 2nd-WM-related conditions.

As to the statistical test for the cross-correlation coefficient in the high-recency and low-recency subject groups in Experiment 2, we did the same permutation test within each group respectively. To compare the difference between groups, we performed a permutation test by shuffling data between high and low groups for 500 times. The original high-low group difference (the averaged 2nd −1st alpha power in phase 1 and phase 3) was then compared to the permutation distribution, resulting in its corresponding p-value.

Confidence interval estimation

Request a detailed protocol

Considering the small sample size of each group (e.g., high- and low-performance groups in Experiment 1; high- and low-recency groups in Experiment 2), we used a jackknife procedure (Efron and Tibshirani, 1993), during which one subject was removed at a time, to estimate the confidence interval of the mean of the alpha-band responses in each group (e.g., high- and low-performance groups in Experiment 1; high- and low-recency groups in Experiment 2).

Time range selection

Request a detailed protocol

In Experiment 1, the time range (phase 1: 310 – 370 ms), defined based on group results (N = 18, Figure 2D), was used to examine the memory effects (WM – NWM alpha-band difference) for the high- and low-performance groups. In Experiment 2, the three time ranges (phase 1: 140 – 180 ms; phase 2: 280 – 320 ms; phase 3: 470 – 510 ms), defined based on group results (N = 19, Figure 3D), were used to examine the sequential reactivation profiles (2nd WM- 1st WM alpha-band difference) for the high- and low-recency groups.

Data availability

All data generated during this study are included in the manuscript and supporting files.

References

    1. Power AJ
    2. Reilly RB
    3. Lalor EC
    (2011) Comparing linear and quadratic models of the human auditory system using EEG
    Conference Proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference 2011:4171–4174.
    https://doi.org/10.1109/IEMBS.2011.6091035

Decision letter

  1. Lila Davachi
    Reviewing Editor; New York University, United States
  2. Michael J Frank
    Senior Editor; Brown University, United States
  3. Saskia Haegens
    Reviewer; Columbia University College of Physicians and Surgeons, United States

In the interests of transparency, eLife includes the editorial decision letter and accompanying author responses. A lightly edited version of the letter sent to the authors after peer review is shown, indicating the most substantive concerns; minor comments are not usually included.

Thank you for submitting your article "Fast-backward replay of sequentially memorized items in humans" for consideration by eLife. Your article has been reviewed by three peer reviewers, and the evaluation has been overseen by a Reviewing Editor and Michael Frank as the Senior Editor. The following individual involved in review of your submission has agreed to reveal her identity: Saskia Haegens (Reviewer #3).

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

This paper addresses an important question about how multi-item sequences are maintained in the brain. The paper is clear and well written and the analyses and results are novel. However, the paper rests on an assumption about probe tagging and WM maintenance, for which the case is not convincingly made in the paper. Furthermore, the bulk of the data is from group data. In order to be considered further, data from individual trials should be shown and analyzed.

For a revision to be successful, you must address the following three major issues and I am appending a list of other comments below that you can use to improve the paper as well:

1) All reviewers either did not follow the logic of the tagging idea or believed it was based on a number of assumptions that were not clearly demonstrated in the paper. The interpretation of the primary conclusions of this paper rest on the assumption that the TRF method can reliably 'tag' unique activation patterns for distinct stimuli. However, there is concern that a number of steps/assumptions that must be true in order for the author's interpretations of the results to be sound. First, there must be a relationship between the neural representation of the items during encoding and the neural representation of just the color feature of the item during maintenance, and the unique representations of these item/features must be dissociable. Second, there must be a linear relationship between the modulation of the luminance of the disc and the neural representation of the color of the disc. Third, the temporal dynamics of alpha power in the resulting TRF must serve as a unique proxy for the neural representation of each item. In order to draw the conclusions, it seems to me that each of these must be true. Please provide a more thorough explanation of the logic and assumptions behind using the TRF method to tag item reactivations and evidence that these assumptions are reasonable and supported by the data.

2) All reviewers felt strongly trial-by-trial evidence for item maintenance should be presented. For the reported patterns to truly reflect a physiological pattern akin to replay, it would be important to demonstrate that individual electrodes/subjects sequentially represent each of the two different stimuli on individual trials. Otherwise, there is the concern that the results may be the effect of averaging such that the observed patterns were caused by different items, electrodes, or subjects showing activations at different latencies according to list position. I think it is vital that a true replay analysis be performed and the authors need to explain this analysis very clearly. The authors can satisfy this concern by ensuring that they report and document an analysis that is focused on identifying replay at the single trial level.

3) Could the timing differences between the item reactivations be completely explained by the recency effect, or differences in memory strength by encoding position? For example, if we manipulated the sequence such that the item in the first position was always more salient than the item in the second position, would the temporal coding pattern during the maintenance period also flip (item 1 reactivated first, item 2 reactivated second)? A finding like this would suggest that the temporal reactivation differences are less about the encoding of order and more about the strength of encoding, or the 'attentional priority' of the items.

[Editors’ note: this article was subsequently rejected after discussions between the reviewers, but the authors were invited to resubmit after an appeal against the decision.]

Thank you for submitting your work entitled "Fast-backward replay of sequentially memorized items in humans" for consideration by eLife. Your article has been reviewed by three peer reviewers, and the evaluation has been overseen by a Reviewing Editor and a Senior Editor. The following individuals involved in review of your submission have agreed to reveal their identity: Andrew Heusser (Reviewer #1); Saskia Haegens (Reviewer #3); Rasa Gulbinaite (Reviewer #4).

Our decision has been reached after consultation between the reviewers. Based on these discussions and the individual reviews below, we regret to inform you that your work cannot be considered further for publication in eLife at this time.

The reviewers all agreed that the revision was very responsive and served to increase confidence in the results as well as clarified the methodological approach. However, there were still significant questions about the validity of the TRF approach especially given that the methods and analyses are so complex. Thus, as you can see, we reached out to another reviewer, an expert in the method. The comments of this reviewer echoed the concerns of the other reviewers and highlight the important need to validate the method with chromatic stimuli as suggested. Furthermore, for the reasons highlighted, this needs to be done on a single subject level. Because this would require new data collection and analyses, it is not clear how these analyses will turn out. However, if you are able to validate the paradigm with chromatic stimuli in the manner suggested by reviewer 4, as well as address all of the other concerns reviewer 4 brings up, we could consider a new submission at that time.

We have appended the reviews below but please note that reviewers 1 and 3 made their comments as an initial response to the revision which was very responsive. However it was upon further consultation with these reviewers that it was clear we needed to consult with an expert on the TRF methods.

Reviewer #1:

The authors did an excellent job at responding to my concerns and I have no additional major concerns.

Reviewer #3:

I believe the authors adequately addressed most of the concerns raised. The only remaining issue is whether these results hold on a single-trial level. While I understand that the TRF (by definition) cannot be computed for single trials, I would have liked to see an intermediate solution: recompute the analysis for binned trials (i.e., smaller subsets). However, I do understand the SNR concerns raised, and it may have to remain for future work. Maybe the authors can at least add a couple sentences to the Discussion to that extent?

Related to this, I did not understand how Figure 3D proves consistency over subjects? Because it is normalized? Maybe I am missing the point, but plotting the single subject results in addition to the grand average would have been more compelling.

Reviewer #4:

I have read the manuscript by Huang et al., the reviewing editor's summary of the reviewers' comments, and the authors' response to these comments. Although the authors extensively revised the manuscript based on the reviewer's comments, I agree with my predecessors that the assumptions that TRF approach allows to track memory activation of distinct stimuli is not sufficiently validated.

To my knowledge, there has not been yet a single paper validating the TRF approach for color stimuli (only achromatic stimuli have been used by several groups so far, e.g. Lalor and VanRullen), thus the authors first need to provide evidence that color stimuli can elicit a reliable TRF. This can be done by presenting a single colored stimulus and comparing the observed TRF with that obtained by shuffling the pairing between luminance sequences and concurrently recorded EEG time series. Crucially, this has to be demonstrated on a single-subject level. The latter is particularly important, because periodicity of TRF for achromatic stimuli has been shown to differ across participants, and the peak frequency of TRF is highly correlated with individual alpha peak frequency (VanRullen and Macdonald, 2012). Thus, group average analyses of TRF time courses (as done here), unless demonstrated that between-subject variability in TRF was not significant, can be driven by the subjects that have higher prominent TRF amplitude.

Based on the example TRFs in Figure 1, TRFs for chromatic stimuli differ substantially from the previously reported TRFs for achromatic stimuli (e.g. VanRullen and Macdonald, 2012), which reverberate up to 1 sec. Moreover, given the previous reports on difference in alpha-band power for different color stimuli (e.g. Yoto et al., 2007; DOI: 10.2114/jpa2.26.373), it is likely that TRF for different color stimuli would also be different. However, it is an empirical question that needs to be addressed prior to using TRFs for chromatic stimuli to tag items held in memory.

It has been previously reported that the phases of two TRFs elicited by achromatic flickering stimuli on the left and right hemifield exhibit phase differences across many channels (Lozano-Doldevilla and VanRullen, DOI: https://doi.org/10.1101/190595). Thus, it is possible that the phase differences reported here are not the property of the items stored in memory, but rather reflects the differences in propagation of different TRFs across cortex. Thus, again it is important to validate the TRF approach for several chromatic stimuli and to demonstrate that the phase differences are not related to the location of the stimuli in the retinotopic space.

Removing grand-average TRF responses from condition-specific TRF responses defeats the purpose of subject- and condition-specific TRF (subsection “Normalization”), and more importantly creates artificial oscillatory pattern (Figure 3D).

The electrode selection procedure is arguably biased to find condition differences. The authors selected the electrodes of interest based on the significant differences in TRFs between the WM and NWM conditions (subsection "Channel-of-interest", as well as authors' response: "According to the suggestion, we first selected channels based on the EEG alpha power, from which we then seek channels showing significant memory effects in TRFs."), and then show that there are indeed differences in TRFs in all subsequent analyses. To be statistically appropriate, channel selection procedure cannot involve data from condition differences, and should only be selected based on condition average or other criteria that are orthogonal to the subsequent analyses.

Overall, despite the interesting topic and clever experimental approach taken by the authors, the manuscript in its current form does not feel up to the empirical and analytic standards one typically associates with eLife.

[Editors’ note: what now follows is the decision letter after the authors submitted for further consideration.]

Thank you for choosing to send your work entitled "Fast-backward replay of sequentially memorized items in humans" for consideration at eLife. Your letter of appeal has been considered by a Senior Editor and a Reviewing Editor, and we are prepared to consider a revised submission with no guarantees of acceptance.

The appeal provided additional useful information that increased the confidence in your reported effects. However, there were still issues brought up by the reviewers that were misunderstood and/or not clearly addressed. These issues (outlined below) must be addressed in full in a revision in order for the paper to be acceptable either with data or added discussion.

1) The additional information you provided about the counterbalancing of color and location was very useful. An additional control that would strengthen your argument would be to directly compare the preferred phase of the TRF for different color/location combinations. The appeal included an additional figure illustrating that there is no effect of location in the alpha power timeseries. If possible with the existing data, it would useful to see the same analysis by color. If there are no reliable differences in the alpha power time course by color, then this cannot be a likely explanation for the phase differences for the different items maintained in memory.

2) Based on the new data, and re-analysis of another dataset, chromatic TRFs seem to be indeed oscillatory and comparable to the previously reported achromatic TRFs at the single-subject level. This is reassuring and adds credibility to the use of white-noise chromatic stimuli for tagging memory items. However, no information was included about the number of trials used in this control study to derive TFRs? Is it comparable to what has been used in the original study? Please add this information into the paper.

3) Also, it is not clear whether the plots provided in the appeal letter come from a single-color stimulus (which color?) or average of 3 colors? Please make this clear in the paper. Thus, this leaves open the original question asked by reviewers – are different color TRFs comparable in amplitude and phase (the property of TRFs that the results of this study hinge on)? Please show the data for each color separately.

4) Several statistical decisions remain unclear and need to be clearly motivated and described. For example, the paper stated "In Experiment 1, the "channel-of-interest" has passed statistical test after multiple-comparison correction across time and channels”. In other words, we used a non-biased approach (statistical test after multiple comparison correction) to determine channels showing memory effects ("channel of interest")." Thus, it appears that a multi-step procedure was used to select the channels of interest? The logic for this is not explained. Please explain the logic carefully and clearly. Although seemingly minor, why did you choose a p value of p =.058 instead of the conventional.05? (In the previous version reviewers also noted that details of a 'bootstrap' or 'permutation' test were also not clearly described). If you choose to submit a revised manuscript, please ensure that all statistics are clearly explained, motivated and reported. Otherwise, we cannot consider the paper suitable for publication.

5) A question was raised about whether the observed sequential replay was due to the normalization procedure. In the appeal, you noted 'First, we could see clear sequential activation pattern in the raw data (Figure 3C)." However, there is a concern that the normalization procedure itself (subtracting the average of the 3 conditions) seems to introduce an artificial oscillatory pattern. Looking at Figure 3C (non-normalized data), non-working memory item is arguably the only one that is oscillatory (most likely noise), and by removing the average of the three-curves one would end up with an oscillatory pattern in Figure 3D – the figure that summarizes the main finding of the paper. Please address this issue more directly by assessing whether there is indeed any evidence for oscillations in the raw data – without the normalization procedure.

https://doi.org/10.7554/eLife.35164.023

Author response

This paper addresses an important question about how multi-item sequences are maintained in the brain. The paper is clear and well written and the analyses and results are novel. However, the paper rests on an assumption about probe tagging and WM maintenance, for which the case is not convincingly made in the paper. Furthermore, the bulk of the data is from group data. In order to be considered further, data from individual trials should be shown and analyzed.For a revision to be successful, you must address the following three major issues and I am appending a list of other comments below that you can use to improve the paper as well:1) All reviewers either did not follow the logic of the tagging idea or believed it was based on a number of assumptions that were not clearly demonstrated in the paper. The interpretation of the primary conclusions of this paper rest on the assumption that the TRF method can reliably 'tag' unique activation patterns for distinct stimuli. However, there is concern that a number of steps/assumptions that must be true in order for the author's interpretations of the results to be sound.

Thanks for raising the crucial concern and we apologize for failing to make the underlying logic and assumptions clear in the previous manuscript. According to the suggestions, we have provided a more thorough explanation of all the logic and assumption as below, and have added more clarifications and references in main texts (see details as below). We have also performed new control analysis (see Figure 2—figure supplement 1C and Figure 2—figure supplement 2C).

First, there must be a relationship between the neural representation of the items during encoding and the neural representation of just the color feature of the item during maintenance, and the unique representations of these item/features must be dissociable.

As pointed out by the reviewer, we probed the neural representation of the memorized items by tagging the neural response of the associated color feature. This assumption is actually based on previous findings and Experiment 1 was actually designed to test this hypothesis. Specifically, previous studies have provided evidence supporting the object-based nature of visual working memory such that all features of an object, even the task-irrelevant features, will be stored automatically in working memory (Luck and Vogel, 1997; Hollingworth & Luck, 2009; Hyun et al., 2009; Shen et al., JEP: HPP, 2015). Most importantly, color feature has been found to be the type of features that have the strongest conjunction with other features within an object (Johnson, Hollingworth and Luck, 2008; Wheeler and Treisman, 2002; Shen et al., JEP: HPP, 2015). Based on these findings, we chose the color feature to probe the neural representation of memory-related information during maintenance. It is noteworthy that we do not propose that any features could be employed to probe memory reactivations; indeed, we made use of the strong conjunction characteristic of color features to probe the memory-associated activations here. We have added more rationales and references in the Results (subsection “Probing memory-related reactivations during retention period”, first paragraph).

Furthermore and most crucially, we did not take the assumption for granted and indeed ran Experiment 1 to test the idea by examining whether the neural response for the task-irrelevant color probe during maintenance would be modulated by whether or not it was associated with memorized orientation. If the color feature is not bound to the corresponding memorized feature, we would expect no difference in response for the memory-associated and non-memory-associated color probes. Crucially, we did find the difference (Figure 2), supporting that the task-irrelevant color probe carries memory-related information and could be used to tag the neural representation of memorized features. We have added more clarifications in the manuscript (subsection “Probing memory-related reactivations during retention period”, last paragraph; subsection “Experiment 1: Alpha-band memory effects (one-item memory)”, last paragraph).

As to the concern about unique dissociable representations of the item/features, it is exactly our motivation to employ TRF approach so that we could dissociate brain response specifically for different probes from the same EEG recordings. In particular, because the luminance of each probe was modulated independently according to a random temporal sequence, we could calculate the TRF response for each probe based on its specific luminance sequence. We have added more clarifications in texts (subsection “Probing memory-related reactivations during retention period”, last paragraph).

Second, there must be a linear relationship between the modulation of the luminance of the disc and the neural representation of the color of the disc.

The reviewer is correct that TRF response is calculated in terms of the linear relationship between stimulus luminance and neural response. In fact, TRF approach has been used and validated in previous visual and auditory studies, which assessed brain response that specifically tracks sound envelope or luminance sequence (Lalor and Foxe, 2010; Ding and Simon, 2012; VanRullen and Macdonald, 2012; Jia et al., 2017). Luminance tracking is actually one of the most robust findings in visual studies. Recently, the TRF has even be revealed to be able to track high-level features, such as global form coherence (Liu et al., 2017) and semantic dissimilarity (Broderick et al., 2018). Therefore, TRF approach would be a quite robust and conservative way to assess the neural response to luminance change. Furthermore, previous studies have also shown that applying a quadratic extension of the linear TRF approach only resulted in slight improvement compared to linear model (Lalor et al., 2008; Power et al., 2011), further advocating the linear assumption for TRF method. To make the points more clear, we have added more references and clarifications about the linear assumptions and validations in Results (subsection “Probing memory-related reactivations during retention period”, last paragraph) and Materials and methods (subsection “TRF computation”, second paragraph).

Nonetheless, we reanalyzed our data to assess the linear concern in our results. Specifically, we divided the whole luminance sequences into three quartiles (low-luminance, middle-luminance, high-luminance), and recalculated their corresponding TRF responses separately. As shown in Author response image 1, the power of the TRF responses showed a decreasing pattern as luminance decreased, again supporting the trend of linear relationship between stimulus luminance and neuronal responses.

Third, the temporal dynamics of alpha power in the resulting TRF must serve as a unique proxy for the neural representation of each item.

Again, we are sorry for not making the alpha-band points clear in our previous version. In fact, using the alpha-band profiles to tag neural representation of each item derives from the results in Experiment 1. The rationales are as follows. First, the TRF responses for both probes and across all EEG channels showed prominent alpha-band responses (see Figure 2A). We next performed a permutation analysis to examine the spectral contents of the TRF responses and the alpha-band showed significant activations (Figure 2—figure supplement 2C), consistent with previous findings using TRF approach (VanRullen and Macdonald, 2012; Jia et al., 2017). Second, only the alpha-band activity showed memory effects (Figure 2C). To further confirm the specificity of alpha-bands, we extracted the temporal dynamics within other frequency bands (delta, 1-2 Hz; theta, 3-6 Hz, beta, 15-25 Hz) in Experiment 1, none of which showed significant memory effects (Figure 2—figure supplement 3C). It is noteworthy that we used Experiment 1 as a pre-test to determine the neural marker for memory reactivations, and thus only assessed alpha-band temporal profiles in further experiments.

The spectrum results have been added (subsection “Experiment 1: Alpha-band memory effects (one-item memory)”, first paragraph; subsection “Correlation coefficient calculation and permutation test”, last paragraph; Figure 2—figure supplement 2C). The results within other frequency bands have been added (subsection “Experiment 1: Alpha-band memory effects (one-item memory)”, last paragraph; Figure 2—figure supplement 3C).

In order to draw the conclusions, it seems to me that each of these must be true. Please provide a more thorough explanation of the logic and assumptions behind using the TRF method to tag item reactivations and evidence that these assumptions are reasonable and supported by the data.

Again, we thank the reviewer for raising the important criticism and apologize for failing to make the underlying logic and assumptions clear in previous version. We have now provided a thorough explanation of all the logic and assumptions in the new version. The new control analysis results have also been added in supplementary figures (Figure 2—figure supplement 1C, Figure 2—figure supplement 2C).

2) All reviewers felt strongly trial-by-trial evidence for item maintenance should be presented. For the reported patterns to truly reflect a physiological pattern akin to replay, it would be important to demonstrate that individual electrodes/subjects sequentially represent each of the two different stimuli on individual trials. Otherwise, there is the concern that the results may be the effect of averaging such that the observed patterns were caused by different items, electrodes, or subjects showing activations at different latencies according to list position. I think it is vital that a true replay analysis be performed and the authors need to explain this analysis very clearly. The authors can satisfy this concern by ensuring that they report and document an analysis that is focused on identifying replay at the single trial level.

We fully agree with the reviewers that to essentially support the sequential reactivation profiles during maintenance, it is important to exclude the possibility that different electrodes, different subjects, and different items might elicit activations at different temporal latencies, and it might be the averaging that caused the observed sequential profiles. Replicating similar effects in single trials would thus exclude the alternative interpretation.

Meanwhile, we would like to point out a technical limitation about TRF approach that unfortunately could not be met in single-trial data in the current design. In particular, reliable estimation of TRF response would require neural signals with enough temporal length as well as the same duration of the corresponding stimulus sequences, from which the TRF response characterizing their linear relationship could be calculated. Previous study has proposed that neural signals of at least 120 secs would be needed to obtain TRF responses with reasonable SNR (Lalor et al., Brain Signal Analysis, 2009). Theoretically, dissociating multiple TRFs from brain signals would require even longer signals. Back to our studies, each trial was 5 s, and we only analyzed the middle 4 s to avoid influences from onset and offset response. Therefore, to obtain TRF responses for multiple probes (e.g., 2 TRFs in Experiment 1 and 3 TRFs in Experiment 2) from the same EEG signals, we would need at least more than 40 trials for the calculation. Single-trial analysis, unless it was designed to be very long, could not meet the signal length requirement. We have added the limitations of TRF approach in Materials and methods (subsection “TRF computation”, last paragraph).

Furthermore, we calculated the SNR of TRF responses for Experiment 1 and Experiment 2 as a function of signal length. As shown in Author response image 2, in both experiments, the SNR of TRF responses reached a steady level when the signal length was around 320 secs (equal to 80 trials). The overall SNR in Experiment 2 was much lower than that in Experiment 1, given that more TRF responses needed to be estimated in Experiment 2. The results further support that we would need adequate signal length (i.e., more trials here) for a reliable TRF estimation.

Although we could not perform single-trial TRF analysis, we fully agree the reviewer that it is critical to exclude the possibility that the results might be due to the combination of responses across subjects, channels and items, which might elicit distinct temporal activation patterns. We thus performed additional control analyses to address the three important concerns.

First, as shown in Figure 3D, we did a t-test on the normalized alpha-band time courses for the WM-TRF responses, demonstrating that the 2nd -WM-TRF showed significant response around 80-100 ms, whereas the 1st -WM-TRF elicited significant response around 260-300 ms. The results thus suggest that the observed backward sequential activations were consistent across individual subjects. Clarifications have been added in text (subsection “Experiment 2: Sequential alpha-band response for sequence memory”, third paragraph).

Second, we replotted the alpha power time courses for the 1st-WM-TRF and 2nd-WM-TRF for each of the ROI electrodes (Pz, P1, P2). As shown in Figure 3—figure supplement 1A, they revealed similar temporal profiles as that of the averaged pattern, supporting that the observed sequential activations were also consistent across ROI electrodes. New results and discussion have been added (subsection “Experiment 2: Sequential alpha-band response for sequence memory”, third paragraph; Figure 3—figure supplement 1A).

Finally, to examine whether different items would elicit distinct temporal patterns (e.g., red probes would be associated with responses with earlier latency, etc.), we replotted the TRF responses for different combinations of colors (i.e., red, blue, and green). Each time we removed one color condition and recalculated the TRF responses (we need enough trials for TRF estimation) for the remaining items. The logic is that if probes with different colors elicited different temporal patterns, we would expect to see inconsistent response profiles for different items. Meanwhile, as shown in Figure 3—figure supplement 1B, all the three conditions showed similar backward sequential activations, supporting that the observed sequential activations were also consistent across items, and were not biased by certain types of item. New results and discussion have been added (see the aforementioned subsection; Figure 3—figure supplement 1B).

Taken together, we thank the reviewer for raising the crucial concern. We have clarified why we could not perform single-trial TRF analysis (due to the technical requirement in TRF estimation) and instead needed sufficient number of trials (given the 4 s length per trial in our experiment) to obtain reliable TRF responses. Furthermore, to further address the concerns raised by the reviewers, we have performed several new control analyses. Our results demonstrate that the observed backward sequential profiles were not due to individual subject, or single channel, or combination of different items. In fact, we observed consistent response patters across subjects, channels, and items. We have added more clarifications and interpretation in Results (subsection “Experiment 2: Sequential alpha-band response for sequence memory”) and Materials and methods (subsection “TRF computation”). We have also added the new control analysis results in supplementary figures (Figure 3—figure supplement 1A, B).

3) Could the timing differences between the item reactivations be completely explained by the recency effect, or differences in memory strength by encoding position? For example, if we manipulated the sequence such that the item in the first position was always more salient than the item in the second position, would the temporal coding pattern during the maintenance period also flip (item 1 reactivated first, item 2 reactivated second)? A finding like this would suggest that the temporal reactivation differences are less about the encoding of order and more about the strength of encoding, or the 'attentional priority' of the items.

We appreciate the interesting alternative interpretation for our results. Specifically, the backward sequential pattern and its association with recency effect might be explained by the account that the better the memory strength or the attentional priority is, the earlier the latency would be. We thus performed a new control analysis in Experiment 2 by dividing subjects into two groups based on their memory accuracy instead of the recency effect and then examined the respective response profiles. If the ‘memory strength’ or ‘attentional priority’ interpretation were correct, we would expect to see temporal shift in the response pattern between the two groups given their different memory strength. In particular, the high-performance group would show earlier response compared to the low-accuracy group.

However, as shown in Figure 3—figure supplement 1C, high-accuracy (solid line) and low-accuracy (dashed line) groups did not show shift in temporal latency and actually elicited quite similar backward sequential responses as before. The new results thus exclude the possibility that it is the memory strength rather than the recency effect that leads to the backward sequential responses. Moreover and interestingly, we observed a trend that the high-accuracy group elicited stronger response than the low-accuracy group, consistent with findings in Experiment 1. The new control analysis and associated discussions have been added (Discussion, fifth paragraph; Figure 3—figure supplement 1C).

As to the interesting question about whether the saliency manipulation would change the order of the reactivation during maintenance, we believe it would depend on whether the recency behavior would be modulated correspondingly. If the 1st item is far more salient than the 2nd item, the recency effect probably would also be influenced and become weaker. As a result, it is quite possible that the reactivation sequence would be reversed. Nevertheless, it is noteworthy that in our experiments, items in the memory list were matched in both physical saliency (simultaneously presented and equally salient) and attentional priority (equally task-relevant) by presenting them in one display and employing sequential cuing to instruct subject to form sequence memory (Figure 3). We thank the reviewer for the interesting idea to test whether physical saliency manipulation could modulate the reactivation sequence during maintenance and believe further studies should be performed to fully address the issue.

Taken together, we appreciate the proposed interpretation for our results. We hope that our new control analysis could convince the reviewers that the observed backward sequential response could not be simply accounted for by the memory strength or attentional priority interpretation. We have added more interpretation and discussion in Discussion (fifth paragraph). We have also added the new control analysis results in Figure 3—figure supplement 1C.

[Editors’ note: the author responses to the re-review follow.]

The reviewers all agreed that the revision was very responsive and served to increase confidence in the results as well as clarified the methodological approach. However, there were still significant questions about the validity of the TRF approach especially given that the methods and analyses are so complex. Thus, as you can see, we reached out to another reviewer, an expert in the method. The comments of this reviewer echoed the concerns of the other reviewers and highlight the important need to validate the method with chromatic stimuli as suggested. Furthermore, for the reasons highlighted, this needs to be done on a single subject level. Because this would require new data collection and analyses, it is not clear how these analyses will turn out. However, if you are able to validate the paradigm with chromatic stimuli in the manner suggested by reviewer 4, as well as address all of the other concerns reviewer 4 brings up, we could consider a new submission at that time.

As summarized by the editor, the key concern raised by the reviewers is that we should provide more evidence validating the TRF approach with chromatic stimuli at single subject level. First, as suggested by reviewer 4, we have collected new data from additional subjects who were presented with a single chromatic disc stimulus. The results showed clear and robust TRF response at single subject level (see results as below), consistent with previous work (VanRullen and Macdonald, 2012) and our own study (Jia et al., 2017). Second, we have analyzed a new dataset in another study (N=13) that also used TRF to tag color probes, again confirming the approach.

That being done, we would like to point out that the skepticism raised by the reviewers might derive from some misunderstanding about our approach. We failed to make it clear enough in the previous manuscript for which we apologize. To clarify, we actually employed a well-established approach to track luminance sequence of color probes rather than the color change of the probes (the color of probes actually kept constant). Luminance modulation is widely used to tag item-specific brain response. For example, by applying different luminance modulation frequencies at different features (e.g., red or blue dot patterns), the corresponding steady state response (SSR) for different features could be dissociated to further examine attentional effects (Muller et al., PNAS, 2006). Here, rather than a sinusoidal luminance modulation in SSR, TRF approach employed a random luminance sequence to tag different color probes. To make the points more clear, we have modified the method description and added clarification, as well as adding a new figure to illustrate the TRF responses (new Figure 2—figure supplement 1). We have also made point-by-point responses and clarifications to other criticisms raised by reviewer 4 (see details in response to reviewer 4).

Author response image 3
TRF responses for the single color probe experiment (as suggested by reviewer 4, two subjects).
https://doi.org/10.7554/eLife.35164.019
Author response image 4
TRF response for a new dataset in another experiment (N = 13).
https://doi.org/10.7554/eLife.35164.020

We have appended the reviews below but please note that reviewers 1 and 3 made their comments as an initial response to the revision which was very responsive. However it was upon further consultation with these reviewers that it was clear we needed to consult with an expert on the TRF methods.

Reviewer #3:

I believe the authors adequately addressed most of the concerns raised. The only remaining issue is whether these results hold on a single-trial level. While I understand that the TRF (by definition) cannot be computed for single trials, I would have liked to see an intermediate solution: recompute the analysis for binned trials (i.e., smaller subsets). However, I do understand the SNR concerns raised, and it may have to remain for future work. Maybe the authors can at least add a couple sentences to the Discussion to that extent?

Thanks for the suggestion. Actually, we did the analysis on smaller subsets, as the reviewer suggested. As shown in Figure 3—figure supplement 1B, which plots the response profiles for different color probe combinations (a subset of all trials), and the results were consistent across different combinations.

Related to this, I did not understand how Figure 3D proves consistency over subjects? Because it is normalized? Maybe I am missing the point, but plotting the single subject results in addition to the grand average would have been more compelling.

Sorry for the confusion. Figure 3D plots the grand average results (mean ± SEM) and corresponding paired t-test results, based on which we claimed that the sequential profiles were consistent over subjects.

Reviewer #4:

I have read the manuscript by Huang et al., the reviewing editor's summary of the reviewers' comments, and the authors' response to these comments. Although the authors extensively revised the manuscript based on the reviewer's comments, I agree with my predecessors that the assumptions that TRF approach allows to track memory activation of distinct stimuli is not sufficiently validated.

To my knowledge, there has not been yet a single paper validating the TRF approach for color stimuli (only achromatic stimuli have been used by several groups so far, e.g. Lalor and VanRullen), thus the authors first need to provide evidence that color stimuli can elicit a reliable TRF. This can be done by presenting a single colored stimulus and comparing the observed TRF with that obtained by shuffling the pairing between luminance sequences and concurrently recorded EEG time series. Crucially, this has to be demonstrated on a single-subject level. The latter is particularly important, because periodicity of TRF for achromatic stimuli has been shown to differ across participants, and the peak frequency of TRF is highly correlated with individual alpha peak frequency (VanRullen and Macdonald, 2012). Thus, group average analyses of TRF time courses (as done here), unless demonstrated that between-subject variability in TRF was not significant, can be driven by the subjects that have higher prominent TRF amplitude.

We appreciate the reviewer’s thoughtful concerns. We agree with the reviewer that previous studies only used TRF approach on achromatic stimuli, and there seems to lack convincing evidence for color tracking in TRF response.

As suggested by the reviewer, we have collected new data from two subjects who were presented with a single chromatic disc stimulus. As shown in Author response image 3, the results showed clear and robust TRF response at single subject level, consistent with previous findings (VanRullen and Macdonald, 2012) as well as our own TRF studies (e.g., Jia et al., 2017). Furthermore, we analyzed a new dataset that also used TRF approach to tag color probes (N=13) and the results are illustrated in Author response image 4. All the results consistently support that TRF approach could be used to track chromatic stimuli at single-subject level.

Meanwhile, we would also like to point out that there might exist some misunderstanding about the approach we used in the current study. The confusion might derive from our unclear presentations in previous manuscript for which we apologize. To clarify, here we used TRF to track ongoing luminance change of the color probes rather than the color change of the probes (the color of the probes kept constant in our study). We believe the reviewer would agree with us that luminance modulation is a well established method to tag item-specific brain response. For example, by applying different luminance modulation frequencies at different features (red or blue dots), the corresponding steady state response (SSR) could be dissociated for different features to further examine feature-based attention (Muller et al., PNAS, 2006). Here, rather than a sinusoidal luminance modulation in SSR, TRF employed a random luminance sequence to tag different color probes. We thank the reviewer for raising the confusing concerns. We now have added more clarifications in main texts.

Furthermore, rather than averaging TRF time course across subjects (“group average analyses of TRF time courses (as done here)”) as the reviewer thought, we actually calculated the TRF time course and the corresponding alpha power profile in each subject separately before grand average (see the method in the second paragraph of the subsection 2 TRF computation”). Our subsequent memory effect (Figure 2D, Figure 3D) was also based on paired t-test, and thus the effect could not be driven by single subject as worried by the reviewer.

In addition, we agree with the reviewer that previous studies have demonstrated different alpha peak frequencies across individuals. However, here we extracted the alphaband (8-11 Hz) power profiles for each subject, an approach that could tolerate different peak frequencies.

Based on the example TRFs in Figure 1, TRFs for chromatic stimuli differ substantially from the previously reported TRFs for achromatic stimuli (e.g. VanRullen and Macdonald, 2012), which reverberate up to 1 sec. Moreover, given the previous reports on difference in alpha-band power for different color stimuli (e.g. Yoto et al., 2007; DOI: 10.2114/jpa2.26.373), it is likely that TRF for different color stimuli would also be different. However, it is an empirical question that needs to be addressed prior to using TRFs for chromatic stimuli to tag items held in memory.

We thank the reviewer for pointing out the unclear issue. Actually, the TRF responses obtained in the present study were quite consistent with previous ones (VanRullen and Macdonald, 2012), and the TRF response, especially the alpha echoes, were clear and robust at single subject level (see new Figure 2—figure supplement 1). The impression might derive from our previous rough and unclear illustration for which we apologize. To address the issue, we have added a new Figure 2—figure supplement 1, which plots the TRF waveforms, spectrum, as well as those for shuffled data. It is clear that our results largely replicated previous findings (VanRullen and Macdonald, 2012). Although the alpha echoes in our results did not last up to 1 sec as previous study showed, they were still quite similar to our own recordings in another study using achromatic stimuli (Jia et al., 2017).

The review also raised concerns about possible color effect on TRF response. This is an important question, but it is noteworthy that here we mainly focused on the memory effects and we have carefully counterbalanced the location and color effects. We have added more clarifications in the main texts.

It has been previously reported that the phases of two TRFs elicited by achromatic flickering stimuli on the left and right hemifield exhibit phase differences across many channels (Lozano-Doldevilla and VanRullen, DOI: https://doi.org/10.1101/190595). Thus, it is possible that the phase differences reported here are not the property of the items stored in memory, but rather reflects the differences in propagation of different TRFs across cortex. Thus, again it is important to validate the TRF approach for several chromatic stimuli and to demonstrate that the phase differences are not related to the location of the stimuli in the retinotopic space.

We appreciate the raised concern about possible location effect. First, as stated in the response to the last point, we mainly focused on the memory effects and have carefully counterbalanced the location of the probes. Second, we extracted the alpha power profiles for each condition and each channel, thus already discarding the phase information in the TRF responses. Finally, our control analysis demonstrated that probes at different locations showed similar memory-related temporal profiles (Author response image 5).

Author response image 5
Alpha power time courses for left and right probe.
https://doi.org/10.7554/eLife.35164.021

Removing grand-average TRF responses from condition-specific TRF responses defeats the purpose of subject- and condition-specific TRF (subsection “Normalization”), and more importantly creates artificial oscillatory pattern (Figure 3D).

We appreciate the concern. We do not agree that the observed sequential replay was due to the normalization procedure. First, we could see clear sequential activation pattern in the raw data (Figure 3C). Second, we did permutation test on the data, and the same normalization procedure was performed on the surrogate data as well. If the normalization introduced artificial sequential pattern, the data would not pass the test.

Moreover, the normalization was performed for each condition and in each subject separately, and therefore it would not remove subject- and condition-specific TRF, as worried by the reviewer.

The electrode selection procedure is arguably biased to find condition differences. The authors selected the electrodes of interest based on the significant differences in TRFs between the WM and NWM conditions (subsection "Channel-of-interest", as well as authors' response: "According to the suggestion, we first selected channels based on the EEG alpha power, from which we then seek channels showing significant memory effects in TRFs."), and then show that there are indeed differences in TRFs in all subsequent analyses. To be statistically appropriate, channel selection procedure cannot involve data from condition differences, and should only be selected based on condition average or other criteria that are orthogonal to the subsequent analyses.

We agree with the reviewer that the main purpose of Experiment 1 was to seek whether there were channels showing significant memory effects. Meanwhile, we do not agree with the reviewer that the ‘channel-of-interest’ was a biased selection. In Experiment 1, the “channel-of-interest” has passed statistical test after multiple-comparison correction across time and channels (subsection “Experiment 1: Alpha-band memory effects (one-item memory)”, first paragraph). In other words, we used a non-biased approach (statistical test after multiple comparison correction) to determine channels showing memory effects (“channel of interest”). In Experiment 2, we used channels that were independently established in Experiment 1 to examine response patterns for sequence memory, and we don’t think the selection was biased. We have added more clarifications about this point.

Overall, despite the interesting topic and clever experimental approach taken by the authors, the manuscript in its current form does not feel up to the empirical and analytic standards one typically associates with eLife.

We appreciate the reviewer’s insightful and careful criticisms and suggestions. We have added new data, as suggested by the reviewer, and a new dataset to validate the TRF approach on chromatic stimuli. We have also added a new Figure 2—figure supplement 1 to better illustrate the original TRF responses in our study. We hope our new data would convince the reviewer that TRF could be used to tag brain response for color probes and the TRF responses is consistent with previous findings and robust at single subject level. Moreover, based on the reviewer’s criticism, we are aware that some presentations in our previous manuscript were unclear and confusing and would introduce possible misunderstandings. We have modified and clarified several parts, emphasizing the luminance modulation point.

[Editors’ note: the author responses to the re-review follow.]

The appeal provided additional useful information that increased the confidence in your reported effects. However, there were still issues brought up by the reviewers that were misunderstood and/or not clearly addressed. These issues (outlined below) must be addressed in full in a revision in order for the paper to be acceptable either with data or added discussion.

1) The additional information you provided about the counterbalancing of color and location was very useful. An additional control that would strengthen your argument would be to directly compare the preferred phase of the TRF for different color/location combinations. The appeal included an additional figure illustrating that there is no effect of location in the alpha power timeseries. If possible with the existing data, it would useful to see the same analysis by color. If there are no reliable differences in the alpha power time course by color, then this cannot be a likely explanation for the phase differences for the different items maintained in memory.

We thank the editors for the comments. Unfortunately, our previous dataset (Experiment 1) did not tag the color of the probes and therefore we could not perform the control analysis as we did for locations. The reason is that, as stated in our previous response to reviews, we mainly focused on the memory effect in the original experimental design, and have carefully counterbalanced color and locations (for the same reason, in the additional 13-subject dataset experiment as shown before, we did not tag the color either).

Meanwhile, to examine the color effect as asked by the editor, we acquired “new data” from an additional 4 subjects who were presented with a single chromatic disc stimulus (blue or red color) and analyzed the 6-subject data (adding the original 2 subjects, see our response to reviews in the last round) for the two color probes respectively. As shown in Author response image 6, the alpha power time course for the two colors displayed similar temporal profiles(Author response image 6).

To test whether the two colors elicited alpha response with different latencies or different power, we calculated the alpha-band peak latency and the spectrum for the two colors respectively. Neither the peak latency (Author response image 6B) nor the alpha-band power (Author response image 6C) showed significant difference (peak latency: Wilcoxon signed-rank test, p = 0.14; power: paired t-test, p = 0.13).

Author response image 6
Color Control experiment (N = 6).

Six subjects were presented with single color probes (blue or red), while performing a central fixation task. The TRF response was calculated for red and blue color probes respectively (Red-TRF and Blue-TRF). Each condition was ran for 64 trials. (A) Grand average alpha (8-12 Hz) power time courses (mean ±SEM) of blue-TRF and red-TRF. (B) Peak latency of the alpha-band response profiles of blue-TRF and red-TRF (Wilcoxon signed-rank test, p = 0.14). (C) Power spectrum of blue-TRF and red-TRF responses (alpha-band power: paired t test, p = 0.13).

https://doi.org/10.7554/eLife.35164.022

</Author response image 6 title/legend>

Moreover, we would like to point out that we have done a control analysis in the last revision by assessing the response profiles for different color combinations in Experiment 2 (Figure 3—figure supplement 1D). The results showed similar sequential-backward profiles as found before, further confirming that color could not be an explanation for the observed sequential reactivation.

That being done, we totally agree that the possible influences of color on brain responses is an important question and needs to be systematically investigated in future studies.

2) Based on the new data, and re-analysis of another dataset, chromatic TRFs seem to be indeed oscillatory and comparable to the previously reported achromatic TRFs at the single-subject level. This is reassuring and adds credibility to the use of white-noise chromatic stimuli for tagging memory items. However, no information was included about the number of trials used in this control study to derive TFRs? Is it comparable to what has been used in the original study? Please add this information into the paper.

Thanks for the question. The number of trial for each color condition in the control recording was 64, comparable to that used in the original study. The information has been added in the figure legends.

3) Also, it is not clear whether the plots provided in the appeal letter come from a single-color stimulus (which color?) or average of 3 colors? Please make this clear in the paper. Thus, this leaves open the original question asked by reviewers – are different color TRFs comparable in amplitude and phase (the property of TRFs that the results of this study hinge on)? Please show the data for each color separately.

Thanks for the question and we are sorry for the lack of descriptions in the previous version. In the control recordings, we actually used the same colors as we did in Experiment 1 (red and blue probes) and the results previously shown were the average of the responses for the two colors. The information has been added in the figure legend. As to the color effect concern and the request for separate color plotting and comparison, please see our response to the 1st comment.

4) Several statistical decisions remain unclear and need to be clearly motivated and described. For example, the paper stated "In Experiment 1, the "channel-of-interest" has passed statistical test after multiple-comparison correction across time and channels”. In other words, we used a non-biased approach (statistical test after multiple comparison correction) to determine channels showing memory effects ("channel of interest")." Thus, it appears that a multi-step procedure was used to select the channels of interest? The logic for this is not explained. Please explain the logic carefully and clearly. Although seemingly minor, why did you choose a p value of p =.058 instead of the conventional.05? (In the previous version reviewers also noted that details of a 'bootstrap' or 'permutation' test were also not clearly described). If you choose to submit a revised manuscript, please ensure that all statistics are clearly explained, motivated and reported. Otherwise, we cannot consider the paper suitable for publication.

Thanks for the comments regarding statistical descriptions. First, as to the channel-of-interest selection, the editor is correct that it was a multi-step procedure. Based on the prominent alpha-band activations in the TRF responses as observed, we first selected occipital channels showing significantly strong overall alpha activations (independent criteria). Among these channels, we then searched ones showing memory effect (WM-NWM alpha-band difference) after multiple-comparison correction across time and channels. The procedure has been described in the main texts (Results) as well as in Materials and methods (subsection “Channel-of-interest”) in previous version. To further clarify the logic as requested by the editors, we have added texts to explain why we first selected channels showing strong alpha responses.

Second, the reason for the seemingly strange 0.058 p-value was due to our original understanding of the journal policy that specific p value rather than the p threshold should be stated. Actually, when we did the channel-of-interest search based on the bootstrap test with multiple comparison correction, two channels (Pz, P2) passed the test with p <= 0.05, and three channels (Pz, P1, P2) passed the test for p <= 0.06 (marginally significant). We set the p threshold as 0.06 to incorporate more channels to increase the signal-to-noise ratio. We have now added the rationale in Materials and methods (subsection “Channel-of-interest”).

We have now carefully gone through all the statistical tests throughout the manuscript and have added more details for each test in Materials and methods.

5) A question was raised about whether the observed sequential replay was due to the normalization procedure. In the appeal, you noted 'First, we could see clear sequential activation pattern in the raw data (Figure 3C)." However, there is a concern that the normalization procedure itself (subtracting the average of the 3 conditions) seems to introduce an artificial oscillatory pattern. Looking at Figure 3C (non-normalized data), non-working memory item is arguably the only one that is oscillatory (most likely noise), and by removing the average of the three-curves one would end up with an oscillatory pattern in Figure 3D – the figure that summarizes the main finding of the paper. Please address this issue more directly by assessing whether there is indeed any evidence for oscillations in the raw data – without the normalization procedure.

We appreciate the concern and we are sorry for not correctly understanding the question by the reviewer. To directly assess the activation pattern in the raw data (i.e., without normalization), we directly compared the alpha response profiles for the 1st and 2nd memory-related items. As shown in Figure 3—figure supplement 1A, the 1st -2nd showed a clear negative-positive-negative oscillatory pattern. Further correlation analysis on the 1st -2nd responses supports the backward reactivation (negative correlation around lag of 200 ms) as well as the recurrent activation for the 2nd item (positive correlation around lag of 400 ms). It is noteworthy that this analysis did not include the NWM items but showed the same results as before, thus excluding the interpretation that the observed backward-sequential reactivation was simply due to the subtraction of the NWM response. We have added the control analysis results in Figure 3—figure supplement 1 and main texts (subsection “Experiment 2: Sequential alpha-band response for sequence memory”, third paragraph).

Moreover, to directly assess the backward profile in the raw data, we extracted the alpha-band peak latency for the 1st and 2nd unnormalized TRF responses, and they showed significant difference (Wilcoxon signed-rank test, p = 0.036). The results further support that the observed backward reactivation was not due the involvement of the NWM item and was present in the raw data. We have also added the new results in the revised manuscript (see the aforementioned subsection).

https://doi.org/10.7554/eLife.35164.024

Article and author information

Author details

  1. Qiaoli Huang

    1. School of Psychological and Cognitive Sciences, Peking University, Beijing, China
    2. PKU-IDG/McGovern Institute for Brain Research, Peking University, Beijing, China
    3. Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, China
    Contribution
    Conceptualization, Data curation, Formal analysis, Validation, Investigation, Visualization, Methodology, Writing—original draft
    Contributed equally with
    Jianrong Jia
    For correspondence
    qiaolihuang@pku.edu.cn
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-4592-9270
  2. Jianrong Jia

    1. School of Psychological and Cognitive Sciences, Peking University, Beijing, China
    2. PKU-IDG/McGovern Institute for Brain Research, Peking University, Beijing, China
    3. Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, China
    4. Peking-Tsinghua Center for Life Sciences, Peking University, Beijing, China
    Contribution
    Data curation, Formal analysis, Investigation, Methodology, Writing—original draft
    Contributed equally with
    Qiaoli Huang
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-7665-1182
  3. Qiming Han

    1. School of Psychological and Cognitive Sciences, Peking University, Beijing, China
    2. PKU-IDG/McGovern Institute for Brain Research, Peking University, Beijing, China
    3. Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, China
    4. Peking-Tsinghua Center for Life Sciences, Peking University, Beijing, China
    Contribution
    Formal analysis, Methodology
    Competing interests
    No competing interests declared
  4. Huan Luo

    1. School of Psychological and Cognitive Sciences, Peking University, Beijing, China
    2. PKU-IDG/McGovern Institute for Brain Research, Peking University, Beijing, China
    3. Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, China
    Contribution
    Conceptualization, Resources, Supervision, Validation, Visualization, Writing—original draft, Project administration
    For correspondence
    huan.luo@pku.edu.cn
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-8349-9796

Funding

National Natural Science Foundation of China (31522027)

  • Huan Luo

National Natural Science Foundation of China (31571115)

  • Huan Luo

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Acknowledgements

We thank Dr. David Poeppel, Dr. Weiwei Zhang, Dr. Fang Fang, Dr. Lucia Melloni, Dr. Zaifeng Gao, Dr. Ling Liu, and Dr. Nai Ding for helpful comments. This work was supported by National Natural Science Foundation of China Grants 31522027 and 31571115 to LH.

Ethics

Human subjects: All participants provided written informed consent prior to the start of the experiment, which was approved by the Research Ethics Committee at Peking University (2015-03-05c2).

Senior Editor

  1. Michael J Frank, Brown University, United States

Reviewing Editor

  1. Lila Davachi, New York University, United States

Reviewer

  1. Saskia Haegens, Columbia University College of Physicians and Surgeons, United States

Publication history

  1. Received: January 17, 2018
  2. Accepted: October 18, 2018
  3. Accepted Manuscript published: October 18, 2018 (version 1)
  4. Version of Record published: November 12, 2018 (version 2)

Copyright

© 2018, Huang et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 3,392
    Page views
  • 595
    Downloads
  • 9
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Download citations (links to download the citations from this article in formats compatible with various reference manager tools)

Open citations (links to open the citations from this article in various online reference manager services)

Further reading

    1. Genetics and Genomics
    2. Neuroscience
    Li Hou et al.
    Research Article Updated

    Long-term flight depends heavily on intensive energy metabolism in animals; however, the neuroendocrine mechanisms underlying efficient substrate utilization remain elusive. Here, we report that the adipokinetic hormone/corazonin-related peptide (ACP) can facilitate muscle lipid utilization in a famous long-term migratory flighting species, Locusta migratoria. By peptidomic analysis and RNAi screening, we identified brain-derived ACP as a key flight-related neuropeptide. ACP gene expression increased notably upon sustained flight. CRISPR/Cas9-mediated knockout of ACP gene and ACP receptor gene (ACPR) significantly abated prolonged flight of locusts. Transcriptomic and metabolomic analyses further revealed that genes and metabolites involved in fatty acid transport and oxidation were notably downregulated in the flight muscle of ACP mutants. Finally, we demonstrated that a fatty-acid-binding protein (FABP) mediated the effects of ACP in regulating muscle lipid metabolism during long-term flight in locusts. Our results elucidated a previously undescribed neuroendocrine mechanism underlying efficient energy utilization associated with long-term flight.

    1. Neuroscience
    Krishna N Badhiwala et al.
    Research Article

    Hydra vulgaris is an emerging model organism for neuroscience due to its small size, transparency, genetic tractability, and regenerative nervous system; however, fundamental properties of its sensorimotor behaviors remain unknown. Here, we use microfluidic devices combined with fluorescent calcium imaging and surgical resectioning to study how the diffuse nervous system coordinates Hydra's mechanosensory response. Mechanical stimuli cause animals to contract, and we find this response relies on at least two distinct networks of neurons in the oral and aboral regions of the animal. Different activity patterns arise in these networks depending on whether the animal is contracting spontaneously or contracting in response to mechanical stimulation. Together, these findings improve our understanding of how Hydra’s diffuse nervous system coordinates sensorimotor behaviors. These insights help reveal how sensory information is processed in an animal with a diffuse, radially symmetric neural architecture unlike the dense, bilaterally symmetric nervous systems found in most model organisms.