Author response:
The following is the authors’ response to the original reviews
eLife Assessment
This important study shows that orientation tuning of V1 neurons is suppressed during a continuous flash suppression paradigm, especially when the neurons have a binocular receptive field. However, the evidence presented is incomplete and, in particular, does not distinguish whether this suppression is due to reduced contrast or due to masking.
This assessment is primarily based on the critique of Reviewer 2 that our results do not distinguish whether the impact of CFS is due to reduced contrast or due to masking. Reviewer 2 referred to Yuval-Greenberg and Heeger (2013), noting that: “V1 activity is, in fact, reduced during CFS … the mask reduces the gain of neural responses to the grating stimulus … making it invisible in the same way that reducing contrast makes a stimulus invisible.” To be precise, Yuval-Greenberg and Heeger (2013) used “akin to”, instead of “the same way”, in their abstract.
We agree that CFS masking and contrast reduction can both lower the signal-to-noise ratio and thereby reducing visibility. However, these two factors operate in fundamentally different ways. According to gain control models by Heeger and others, reducing the physical contrast of a stimulus decreases the excitatory drive, while dichoptic masking increases the normalization pool. Our findings therefore reflect genuine masking-induced suppression and are not attributable to stimulus contrast reduction.
Public Reviews:
Reviewer #1 (Public review):
Disclaimer: While I am familiar with the CFS method and the CFS literature, I am not familiar with primate research or two-photon calcium imaging. Additionally, I may be biased regarding unconscious processing under CFS, as I have extensively investigated this area but have found no compelling evidence in favor of unconscious processing under CFS.
This manuscript reports the results of a nonhuman-primate study (N=2 behaving macaque monkeys) investigating V1 responses under continuous flash suppression (CFS). The results show that CFS substantially suppressed V1 orientation responses, albeit slightly differently in the two monkeys. The authors conclude that CFS-suppressed orientation information "may not suffice for high-level visual and cognitive processing" (abstract).
The manuscript is clearly written and well-organized. The conclusions are supported by the data and analyses presented (but see disclaimer). However, I believe that the manuscript would benefit from a more detailed discussion of the different results observed for monkeys A and B (i.e., inter-individual differences), and how exactly the observed results are related to findings of higher-order cognitive processing under CFS, on the one hand, and the "dorsal-ventral CFS hypothesis", on the other hand.
Thanks for reviewer’s helpful comments and suggestions. We added new contents discussing the inter-individual differences and the "dorsal-ventral CFS hypothesis" in the revision, and made other changes, which are detailed below.
Major Comments:
(1) Some references are imprecise. For example, l.53: "Nevertheless, two fMRI studies reported that V1 activity is either unaffected or only weakly affected (Watanabe et al., 2011; Yuval-Greenberg & Heeger, 2013)". "To the best of my understanding, the second study reaches a conclusion that is entirely opposite to that of the first, specifically that for low-contrast, invisible stimuli, stimulus-evoked fMRI BOLD activity in the early visual cortex (V1-V3) is statistically indistinguishable from activity observed during stimulus-absent (mask-only) trials. Therefore, high-level unconscious processing under CFS should not be possible if Yuval-Greenberg & Heeger are correct. The two studies contradict each other; they do not imply the same thing.
Sorry we did not make our point clear. Our original concern was that the effects of CFS on V1 activity were underestimated, even in Yuval-Greenberg & Heeger (2013), as both studies compared monocular and dichoptic masking to estimate the influence of visibility. In contrast, in original psychophysical studies, the CFS effect was compared with or with dichoptic masking, which is expected to be stronger. We rewrote the paragraph to clarify.
“Two prominent fMRI studies have examined the impact of CFS on V1 activity (Watanabe et al., 2011; Yuval-Greenberg & Heeger, 2013). Watanabe et al. (2011) compared monocular CFS masking (stimulus visible) and dichoptic CFS masking (stimulus invisible), and reported that V1 BOLD responses were largely insensitive to stimulus visibility when attention was carefully controlled. However, using similar experimental design, Yuval-Greenberg and Heeger (2013) observed reduced BOLD responses in V1 under dichoptic masking, suggesting that V1 activity changed with stimulus visibility. They attributed the difference of results between two studies mainly to differences in statistical power (~250 trials per condition vs. ~90 trials per condition). Nevertheless, these studies were not designed to quantify the pure effect of CFS on stimulus-evoked V1 responses, as they contrasted monocular and dichoptic masking conditions to equate stimulus input while manipulating perceptual visibility. In contrast, original psychophysical studies (Tsuchiya & Koch, 2005; Tsuchiya, Koch, Gilroy, & Blake, 2006) demonstrated CFS masking by contrasting the visibility of the target stimulus with and without the presence of dichoptic mask. It is apparent that the pure CFS impact in above fMRI studies would be the difference of BOLD signals between binocular masking and stimulus alone conditions. In other words, the impact of CFS on V1 activity should be larger than what has been reported by Yuval-Greenberg and Heeger (2013).” (lines 55-71)
(2) Line 354: "The flashing masker was a circular white noise pattern with a diameter of 1.89°, a contrast of 0.5, and a flickering rate of 10 Hz. The white noise consisted of randomly generated black and white blocks (0.07 × 0.07 each)." Why did the authors choose a white noise stimulus as the CFS mask? It has previously been shown that the depth of suppression engendered by CFS depends jointly on the spatiotemporal composition of the CFS and the stimulus it is competing with (Yang & Blake, 2012). For example, Hesselmann et al. (2016) compared Mondrian versus random dot masks using the probe detection technique (see Supplementary Figure S4 in the reference below) and found only a poor masking performance of the random dot masks.
Yang, E., & Blake, R. (2012). Deconstructing continuous flash suppression. Journal of Vision, 12(3), 8. https://doi.org/10.1167/12.3.8
Hesselmann, G., Darcy, N., Ludwig, K., & Sterzer, P. (2016). Priming in a shape task but not in a category task under continuous flash suppression. Journal of Vision, 16, 1-17.
In a previous human psychophysical study, we also used the same noise pattern and the CFS effect appeared to be robust (Xiong et al., 2016, https://doi.org/10.7554/eLife.14614). However, we believe that the reviewer made a good point, and weaker suppression due to the use of our stimulus pattern may have contributed to the weaker suppression in Monkey B. This issue is now discussed in the revision regarding the individual variability in our results.
“In addition, the random-noise masker we used might not be as effective as Mondrian patterns (G. Hesselmann, Darcy, Ludwig, & Sterzer, 2016). If reduced stimulus contrast and a Mondrian masker were used, we predict that CFS suppression in Monkey B would strengthen, potentially approaching the level observed in Monkey A. Nevertheless, it is worth emphasizing that our main conclusions are primarily based on data from Monkey A, who exhibited much stronger CFS suppression.” (lines 321-327)
(3) Related to my previous point: I guess we do not know whether the monkeys saw the CF-suppressed grating stimuli or not? Therefore, could it be that the differences between monkey A and B are due to a different individual visibility of the suppressed stimuli? Interocular suppression has been shown to be extremely variable between participants (see reference below). This inter-individual variability may, in fact, be one of the reasons why the CFS literature is so heterogeneous in terms of unconscious cognitive processing: due to the variability in interocular suppression, a significant amount of data is often excluded prior to analysis, leading to statistical inconsistencies.
Yamashiro, H., Yamamoto, H., Mano, H., Umeda, M., Higuchi, T., & Saiki, J. (2014). Activity in early visual areas predicts interindividual differences in binocular rivalry dynamics. Journal of Neurophysiology, 111(6), 1190-1202. https://doi.org/10.1152/jn.00509.2013
The individual difference issue is now explicitly addressed in the Discussion:
“Interocular suppression under CFS is known to vary substantially across individuals (Blake, Goodman, Tomarken, & Kim, 2019; Gayet & Stein, 2017; Yamashiro et al., 2013). This inter-individual variability may contribute to the heterogeneity observed in the CFS literature. We also found that the strength of V1 response suppression during CFS differed between two monkeys, as reflected by population orientation tuning functions (Fig. 2C), Fisher information (Fig. 2F), and reconstruction performance by the transformer (Fig. 3E). Several experimental factors may have contributed to the relatively weaker suppression observed in Monkey B. Because monkeys viewed the stimuli passively, we could not determine the dominant eye for each monkey (instead we switched the eyes and averaged the results), and the target was presented at relatively high contrast. Both factors are known to reduce the effectiveness of CFS suppression (Yang, Blake, & McDonald, 2010; Yuval-Greenberg & Heeger, 2013). In addition, the random-noise masker we used might not be as effective as Mondrian patterns (G. Hesselmann, Darcy, Ludwig, & Sterzer, 2016). If reduced stimulus contrast and a Mondrian masker were used, we predict that CFS suppression in Monkey B would strengthen, potentially approaching the level observed in Monkey A. Nevertheless, it is worth emphasizing that our main conclusions are primarily based on data from Monkey A, who exhibited much stronger CFS suppression.” (lines 311-327)
Moreover, the authors' main conclusion (lines 305-307) builds on the assumption that the stimuli were rendered invisible, but isn't this speculation without a measure of awareness?
We agree. To correct, we have removed the original lines 305-307 discussing the consciousness perception and reframed the manuscript throughout to focus on the impact of CFS on neural coding rather than on perceptual awareness. For example, the title has been changed to:
“Continuous flashing suppression of neural responses and population orientation coding in macaque V1”,
and the ending line of Introduction was changed to:
“This approach enabled us to investigate the potentially differential impacts of CFS on the responses of V1 neurons with varying ocular preferences, as well as apply machine learning tools to understand the impacts of CFS on V1 stimulus coding at the population level.” (lines 81-83)
(4) The authors refer to the "tool priming" CFS studies by Almeida et al. (l.33, l.280, and elsewhere) and Sakuraba et al. (l.284). A thorough critique of this line of research can be found here:
Hesselmann, G., Darcy, N., Rothkirch, M., & Sterzer, P. (2018). Investigating Masked Priming Along the "Vision-for-Perception" and "Vision-for-Action" Dimensions of Unconscious Processing. Journal of Experimental Psychology. General. https://doi.org/10.1037/xge0000420
This line of research ("dorsal-ventral CFS hypothesis") has inspired a significant body of behavioral and fMRI/EEG studies (see reference for a review below). The manuscript would benefit from a brief paragraph in the discussion section that addresses how the observed results contribute to this area of research.
Ludwig, K., & Hesselmann, G. (2015). Weighing the evidence for a dorsal processing bias under continuous flash suppression. Consciousness and Cognition, 35, 251-259. https://doi.org/10.1016/j.concog.2014.12.010
In the revision, we added a new paragraph to discussion issues related to the dorsal-ventral CFS hypothesis.
“A related issue is the dorsal-ventral CFS hypothesis, which proposes that CFS suppression may disproportionately affect ventral visual processing while relatively preserving dorsal pathways involved in visuomotor functions, potentially allowing category- or action-related information to remain accessible under suppression (Fang & He, 2005). However, subsequent fMRI studies have failed to provide consistent support for this dissociation, reporting either stream-invariant awareness effects (Guido Hesselmann & Malach, 2011; Ludwig et al., 2015; Tettamanti et al., 2017), residual signal in ventral rather than dorsal regions (Fogelson et al., 2014; Guido Hesselmann et al., 2011), or residual low-level feature information/partial visibility rather than preserved dorsal processing (Ludwig et al., 2015). Although our study does not directly test dorsal-ventral dissociations, our V1 results provide a constraint on what information downstream visual pathways could access under suppression. When CFS- induced interocular suppression was strong enough and stimuli reconstruction was markedly reduced, as in the case of Monkey A, the information required for category-level or action-related processing may not be sufficient for high-level cortical representation.” (lines 297-310)
Reviewer #2 (Public review):
Summary:
The goal of this study was to investigate the degree to which low-level stimulus features (i.e., grating orientation) are processed in V1 when stimuli are not consciously perceived under conditions of continuous flash suppression (CFS). The authors measured the activity of a population of V1 neurons at single neuron resolution in awake fixating monkeys while they viewed dichoptic stimuli that consisted of an oriented grating presented to one eye and a noise stimulus to the other eye. Under such conditions, the mask stimulus can prevent conscious perception of the grating stimulus. By measuring the activity of neurons (with Ca2+ imaging) that preferred one or the other eye, the authors tested the degree of orientation processing that occurs during CFS.
Strengths:
The greatest strength of this study is the spatial resolution of the measurement and the ability to quantify stimulus representations during CSF in populations of neurons, preferring the eye stimulated by either the grating or the mask. There have been a number of prominent fMRI studies of CFS, but all of them have had the limitation of pooling responses across neurons preferring either eye, effectively measuring the summed response across ocular dominance columns. The ability to isolate separate populations offers an exciting opportunity to study the precise neural mechanisms that give rise to CFS, and potentially provide insights into nonconscious stimulus processing.
Weaknesses:
While this is an impressive experimental setup, the major weakness of this study is that the experiments don't advance any theoretical account of why CFS occurs or what CFS implies for conscious visual perception. There are two broad camps of thinking with regard to CFS. On the one hand, Watanabe et al. (2011) reported that V1 activity remained intact during CFS, implying that CFS interrupts stimulus processing downstream of V1. On the other hand, Yuval-Greenberg and Heeger (2013) showed that V1 activity is, in fact, reduced during CFS. By using a parametric experimental design, they measured the impact of the mask on the stimulus response as a function of contrast and concluded that the mask reduces the gain of neural responses to the grating stimulus. They presented a theoretical model in which the mask effectively reduced the SNR of the grating, making it invisible in the same way that reducing contrast makes a stimulus invisible.
We used multi-class SVM (as suggested by reviewer 3) and a transformer-based model to examine the impact of CFS on the classification of 12 orientations spaced in 15o gaps, which resembles coarse orientation discrimination, as well as on stimulus reconstruction, which resembles stimulus perception necessary for high-level cognitive tasks, respectively. The results suggest that under CFS, an observer may still be able to perform coarse orientation discrimination but not high-level cognitive tasks. These findings provide new insights into the implications of CFS for conscious visual perception from a population decoding perspective.
In the revision, we also added a new paragraph discussing the implications of our findings for the dorsal-ventral CFS hypothesis, as suggested by reviewer 1. We previously presented a gain control model for our neuronal data in a VSS talk. However, we later decided that, since there are already nice models by Heeger and others, it would be better present something more unique and novel (i.e., machine learning results), which has now become a major component of the manuscript. We welcome the reviewer’s comments on this part.
An important discussion point of Yuval-Greenberg and Heeger is that null results (such as those presented by Watanabe et al.) are difficult to interpret, as the lack of an effect may be simply due to insufficient data. I am afraid that this critique also applies to the present study.
We are very much puzzled by the reviewer’s critique. First, our main result is not a null effect. A null effect would mean that CFS masking had no impact on population orientation responses. Instead, we observed a significant suppression or abolished tuning, which clearly indicates a strong effect of dichoptic masking. Second, our findings are based on large neural populations recorded using two-photon imaging, providing extensive sampling and statistical power. Thus, we believe that the reviewer’s critique about “insufficient data” are not applicable to our study.
Here, the authors report that CFS effectively 'abolishes' tuning for stimuli in neurons preferring the eye with the grating stimulus. The authors would have been in a much stronger position to make this claim if they had varied the contrast of the stimulus to show that the loss of tuning was not simply due to masking.
We are sorry that we cannot follow the logic here either. Even if “the mask effectively reduced the SNR of the grating, making it invisible in the same way that (“akin to”, to be more precise according to the abstract of Yuval-Greenberg and Heeger (2013)) reducing contrast makes a stimulus invisible”, it does not necessarily mean that dichoptic masking and contrast reduction are the same process or are based on the same neuronal mechanisms. According to gain control models by Heeger and others, reducing the stimulus contrast decreases the excitatory drive, while dichoptic masking increases the normalization pool via interocular suppression, both of which lower SNR, but are two fundamentally distinct processes.
Therefore, varying the stimulus contrast might reveal a main effect of contrast, and possibly an interaction between contrast and dichoptic masking, but it would neither prove nor disprove the main effect of dichoptic masking.
So, while this is an incredibly impressive set of measurements that in many ways raises the bar for in vivo Ca2+ imaging in behaving macaques, there isn't anything in the results that constitutes a real theoretical advance.
We sincerely hope that the reviewer would have a better judgment after reading our responses.
Reviewer #3 (Public review):
Summary:
In this study, Tang, Yu & colleagues investigate the impact of continuous flash suppression (CFS) on the responses of V1 neurons using 2-photon calcium imaging. The report that CFS substantially suppressed V1 orientation responses. This suppression happens in a graded fashion depending on the binocular preference of the neuron: neurons preferring the eye that was presented with the marker stimuli were most suppressed, while the neurons preferring the eye to which the grating stimuli were presented were least suppressed. The binocular neuron exhibited an intermediate level of suppression.
Strengths:
The imaging techniques are cutting-edge, and the imaging results are convincing and consistent across animals.
Weaknesses:
I am not totally convinced by the conclusions that the authors draw based on their machine learning models.
Thanks for pointing this issue. We have used a new multi-class SVM suggested by the reviewer to reanalyze the data and found similar results, which is detailed later.
Recommendations for the authors:
Reviewer #1 (Recommendations for the authors):
(1) Lines 56-63: "As a result, the dichoptic CFS masking, which is cortical, could be substantially stronger than monocular masking when accounting for the pre-cortical effects of monocular masking." I don't quite understand this argument. Could you please elaborate?
We have revised our writing to address the reviewer’s first major comment, which the current issue is related. The elaboration is highlighted in the paragraph below.
“Two prominent fMRI studies have examined the impact of CFS on V1 activity (Watanabe et al., 2011; Yuval-Greenberg & Heeger, 2013). Watanabe et al. (2011) compared monocular CFS masking (stimulus visible) and dichoptic CFS masking (stimulus invisible), and reported that V1 BOLD responses were largely insensitive to stimulus visibility when attention was carefully controlled. However, using similar experimental design, Yuval-Greenberg and Heeger (2013) observed reduced BOLD responses in V1 under dichoptic masking, suggesting that V1 activity changed with stimulus visibility. They attributed the difference of results between two studies mainly to differences in statistical power (~250 trials per condition vs. ~90 trials per condition). Nevertheless, these studies were not designed to quantify the pure effect of CFS on stimulus-evoked V1 responses, as they contrasted monocular and dichoptic masking conditions to equate stimulus input while manipulating perceptual visibility. In contrast, original psychophysical studies (Tsuchiya & Koch, 2005; Tsuchiya, Koch, Gilroy, & Blake, 2006) demonstrated CFS masking by contrasting the visibility of the target stimulus with and without the presence of dichoptic mask. It is apparent that the pure CFS impact in above fMRI studies would be the difference of BOLD signals between binocular masking and stimulus alone conditions. In other words, the impact of CFS on V1 activity should be larger than what has been reported by Yuval-Greenberg and Heeger (2013).” (lines 55-71)
(2) Line 13 low-level stimulus (properties).
Fixed, thanks.
Reviewer #3 (Recommendations for the authors):
Major comments:
(1) My main comment is regarding the SVM classifiers. The pair-wise (adjacent orientation pairs) decoding approach is unrealistic in my opinion and likely explains the very high accuracies that are reported. I believe that a multi-way classification approach - Linear Discriminant Analysis, Decision Trees, etc. - is needed to draw reasonable conclusions. Even SVMs can be adapted for multi-way classification (e.g., Allwein et al., 2000, J. Machine Learning Research).
Following the reviewer’s advice, we reanalyzed the data using a multi-class SVM with a one-vs-one (OvO) scheme to classify 12 orientations (Allwein et al., 2000), which yielded similar results.
“For orientation classification, we trained an all-pair multiclass support vector machine (SVM) classifier to discriminate 12 orientations based on trial-by-trial population neural responses from all trials (Allwein, Schapire, & Singer, 2000). Decoders for different FOVs, ipsilateral/contralateral target presentations, and baseline vs. CFS conditions were trained separately. Under the baseline condition, the decoders achieved mean classification accuracies of 89.5 ± 2.0% and 91.5 ± 2.1% across ipsilateral and contralateral eye conditions in Monkeys A and B, respectively, in contrast to a chance level of 8.3% (1 out of 12). Under CFS, decoding accuracy slightly decreased in Monkey A (81.7 ± 1.9%) but remained stable in Monkey B (90.4 ± 2.1%, Fig. 3A). These results suggest that under CFS, there is still sufficient information for coarse orientation discrimination, even for Monkey A whose V1 neuronal responses were substantially suppressed.” (lines 171-181)
(2) The inconsistent modeling results (Figure 3E,F) are puzzling and need to be adequately addressed.
SSIM and orientation error in original Fig. 3E, F measured the same reconstruction quality, but these two indices go in opposite directions for the same modeling results. To avoid confusion, we have removed the orientation error metric and now only report SSIM.
“We used a structural similarity index (SSIM) (Brunet, Vrscay, & Wang, 2012) to quantify the reconstruction performances. Across the grating-presenting ipsilateral and contralateral eyes, the baseline models reconstructed the grating with median SSIMs of 0.52 and 0.61 for the two FOVs of Monkey A, and 0.57 and 0.63 for the two FOVs of Monkey B, respectively, while the corresponding SSIMs for the CFS models were 0.16 and 0.19 for Monkey A, and 0.55 and 0.53 for Monkey B (Fig. 3E).” (lines 200-206)
Minor points:
(1) The phrase "perceptual consequences" in the title is somewhat strong and possibly misleading, since there are no behavioral measures in this study.
To address this concern from this reviewer and reviewer 1, we now focus on the impact of CSF on population orientation coding rather than perceptual consequences, which is more appropriate describing our modeling results. For example, we changed the title to: “Continuous flashing suppression of neural responses and population orientation coding in macaque V1“. Other changes are also made throughout the manuscript accordingly.
(2) Figure 4: Panel "F" is not marked in the figure.
Fixed, thanks.
