A generative model of electrophysiological brain responses to stimulation

Reviewer #1 (Public Review):

With genephys, the author provides a generative model of brain responses to stimulation. This generative model allows mimicking of specific parameters of a brain response at the sensor level, to test the impact of those parameters on critical analytic methods utilized on real M/EEG data. Specifically, they compare the decoding output for differently set parameters to the decoding pattern observed in a classical passive viewing study in terms of the resulting temporal generalization matrix (TGM). They identify that the correspondence between the mimicked and the experimental TGM depends on an oscillatory component that spans multiple channels, frequencies, and latencies of response; and an additive, slower response with a specific (cross-frequency) relation to the phase of the oscillatory, faster component.

A strength of the article is that it considers the complexity of neural data that contributes to the findings obtained in stimulation experiments. An additional strength is the provision of a Python package that allows scientists to explore the potential contribution of different aspects of neural signals to obtained experimental data and thereby to potentially test their theoretical assumptions critical parameters that contribute to their experimental data.

A weakness of the paper is that the power of the model is illustrated for only one specific set of parameters, added in a stepwise manner and the comparison to on specific empirical TGM, assumed to be prototypical; And that this comparison remains descriptive. (That is could a different selection of parameters lead to similar results and is there TGM data which matches these settings less well.) It further remained unclear to me, which implications may be drawn from the generative model, following from the capacities to mimic this specific TGM (i) for more complex cases, such as the comparison between experimental conditions, and (ii) about the complex nature of neural processes involved.

Towards this end, I would appreciate (i) a more profound explanation of the conclusions that can be drawn from this specific showcase, including potential limitations, as well as wider considerations of how scientists may empower the generative model to (ii) understand their experimental data better and (iii) which added value the model may have in understanding the nature of underlying brain mechanism (rather than a mere technical characterization of sensor data).

Reviewer #2 (Public Review):

This paper introduces a new model that aims to explain the generators of temporal decoding matrices (TGMs) in terms of underlying signal properties. This is important because TGMs are regularly used to investigate neural mechanisms underlying cognitive processes, but their interpretation in terms of underlying signals often remains unclear. Furthermore, neural signals are often variant over different instances of stimulation despite behaviour being relatively stable. The author aims to tackle these concerns by developing a generative model of electrophysiological data and then showing how different parameterizations can explain different features of TGMs. The developed technique is able to capture empirical observations in terms of fundamental signal properties. Specifically, the model shows that complexity is necessary in terms of spatial configuration, frequencies and latencies to obtain a TGM that is comparable to empirical data.

The major strength of the paper is that the novel technique has the potential to further our understanding of the generators of electrophysiological signals which are an important way to understand brain function. Furthermore, the used techniques are state-of-the-art and the developed model is publicly shared in open source code.

On the other hand, the results of comparisons between simulations and real data are not always clear for an inexperienced reader. For example, the comparisons are qualitative rather than quantitative, making it hard to draw firm conclusions. Relatedly, it is unclear whether the chosen parameterizations are the only/best ones to generate the observed patterns or whether others are possible. In the case of the latter, it is unclear what we can actually conclude about underlying signal generators. It would have been different if the model was directly fitted to empirical data, maybe of different cognitive conditions. Finally, the neurobiological interpretation of different signal properties is not discussed. Therefore, taken together, in its currently presented form, it is unclear how this method could be used exactly to further our understanding of the brain.

Author Response:

I appreciate the time and effort of both Reviewers, who have raised important points that I would like to briefly discuss before I start working on a full revision of the paper.

Generality. First, there is the question of how much these conclusions broadly apply across experimental paradigms and subjects, which could give rise to potentially very different TGMs. As the Reviewers mention, I have focussed on one specific TGM that I assumed prototypical, and it could be that these conclusions fit other TGMs less well. Further, the model has quite a few hyperparameters so that it can flexibly accommodate a broad span of scenarios. This flexibility comes at a price, as pointed out by Reviewer 1: that “a different selection of parameters could lead to similar results”, i.e. that other configurations could fit this specific TGM just as well. This is related to the next point, so I will address them jointly.

Lack of quantitative evaluation, “making it hard to draw firm conclusions”. Indeed, I have not explicitly quantified the fit of the hyperparameters to this empirical TGM using a specific measure, and (related to the previous point) I have not made a systematic search through the space of model configurations based on such measure.

There is here a trade-off between generality and specificity. In fact, it is intentional that I did not optimise the hyperparameters to this specific TGM, and that I chose not to show a quantitative measure of fitness. This is because the TGM that I show in the paper is only meant as an example. Instead of focussing on fitting a specific TGM, I aimed at characterising some prominent general features that we often see throughout the literature, which this specific TGM shows in its own specific way. That is, if the paper was meant to focus on a specific paradigm (e.g. passive vision), then the use of a specific metric to fit the model to one or various empirical TGMs would have perhaps been more adequate, but this was not the case here. In future work, when focussing on specific paradigms, I will adapt methods of Bayesian optimisation (Lorenz et al., 2017) for this purpose, as mentioned in the Discussion. Note that doing this right is not trivial and would complicate the paper significantly; for this reason, I feel it should belong to a different piece of work.

I would also like to note that evaluating the different features of the data one by one (“in a stepwise manner”) was necessary for interpretation. One can loosely think of it as a sort of F-test: one is showing how important a feature is by comparing the full model vs. a nested model that does not have that feature. While the Reviewer is right that there might be interactions between the features that we can only unveil through a joint evaluation, my approach is at least valid as a first approximation. I will discuss this limitation in an updated version of the paper in more detail.

In a future revision of the paper, I will argue more specifically why and how these model configurations are, in general terms, necessary to produce these main effects in the TGM, and why other alternative configurations could not easily generate them.

Practical guidelines for researchers. It was suggested to make it clearer how researchers could leverage this model in their own studies to understand their data better and to help relating their TGMs to specific neurobiological mechanisms.

In a future revision of the paper, I will introduce a new section explaining how to use genephys practically, emphasising both opportunities and current limitations.

Neurobiological interpretation. It was criticised that the results were a mere characterisation of sensor space data, and that these were not related clearly to any neurobiological aspect.

In a future revision, I will work toward relating the main findings to existing literature in order to strengthen the neurobiological interpretation of the results, and toward a better justification of how genephys can help shed light on specific brain mechanisms.

Above and beyond these specific points, I intend to restructure the text so that the main goals of the study become clearer. This includes clarifying in the Introduction more unambiguously what is the gap of knowledge this work is specifically tackling.

Again, I would like to thank the Reviewers for helping me realise the limitations of the current version of the paper.