Inferring the time-varying coupling of dynamical systems with temporal convolutional autoencoders

Reviewer #1 (Public review):

Summary:

The authors make a new contribution with careful computational validation/exploration of their method on synthetic and real-world datasets. Overall, I find their results significant and their presentation compelling.

Strengths:

The authors provide extensive computational validation of their approach to synthetic and real-world datasets of increasing complexity.

Weaknesses:

The authors should provide a comparison of their approach to other state-of-the-art neural network-based methods. Without this, it is difficult to tell which aspects of their approach (novel coupling metric, or network architecture) are most important for their results.

Reviewer #2 (Public review):

Summary:

This paper introduces a new methodology for probing time-varying causal interactions in complex dynamical systems using a novel machine-learning architecture of Temporal Autoencoders for Causal Inference (TACI) combined with a novel metric (CSGI) for assessing causal interactions using surrogate data. This is a timely contribution in the field of causal inference from temporal data which has been largely restricted to stationary time series so far. However, the benchmarking of the proposed methods could be improved.

Strength:

The method's capacity to uncover piecewise time-varying non-linear dynamic systems is demonstrated on synthetic datasets as well as on two real-world applications on climate and brain activity data. A particular advantage of the approach is to train a single model capturing the dynamics of the whole time series, thereby allowing for time-varying interactions to be found without retraining over different time periods.

Weaknesses:

(1) It is not clear why the new metric Comparative Surrogate Granger Index CSGI (Eq.6) should be better than the Extended Granger Causality Index EGCI (Eq.5), which can also be used to compare the information about y(t) contained in the actual data x(t) versus in a randomized surrogate x^s(t), as implemented in the proposed metric (Eq.6).

(2) The benchmarking of the new approach TACI against earlier metrics (ie Surrogate Linear Granger, Convergent Cross Mapping, and Transfer Entropy) should be revised:

(a) The details of the computation should be provided to clarify how the different metrics are estimated notably between multidimensional variables [for instance to estimate Ty->x for x=(x_1,x_2,x_3) and y=(y_1,y_2,y_3)].

(b) Reliable implementations of the different metrics should be used, as some of the reported results do not seem right. In particular, the unidirectional examples, Eq.9 (Figure 2) and Eq.12 (Figure 5), are expected to lead to vanishing transfer entropies from Y to X, ie Ty->x =0, for all values of the coupling parameter below the synchronization threshold. This can be verified by computing transfer entropies as conditional mutual information using MIIC R package, i.e. Ty->x = I(x(t);y(t-1)|x(t-1)).

(c) Besides, some reported benchmarks focus on peculiar non-linear systems displaying somewhat "pathological" behaviors. For instance, the two Hénon maps with unidirectional coupling Eq.12 (Figure 5) lead to an equality between the two variables, i.e. y(t)=x(t) for all t, above the synchronization threshold C>0.7. This leads mathematically to zero transfer entropy upon synchronization, as I(x(t);y(
d) By contrast, Eq.9 (Fig.2) leads to strongly coupled, yet non-identical variables above the synchronization threshold. This strong coupling can be shown to yield non-vanishing transfer entropies in both directions, as observed in Figure 2c, and does not correspond to "incorrect prediction of non-existent interactions", as stated in the "Summary of Results on Artificial Test Systems". Clearly synchronized variables do interact and their bidirectional transfer entropies are actually consistent with a non-causal (or bidirectional) relationship. Only a vanishing transfer entropy in one direction implies a causal relation (in the opposite direction). Likewise, vanishing transfer entropies in both directions imply either independent variables or a spurious dependency between them due to an unobserved common cause L, i.e. X<--(L)-->Y. This is usually represented with a bidirected edge (X<-->Y), which is different from a bidirectional relation corresponding to two opposite unidirectional edges (ie X-->Y and X<--Y). It is therefore surprising that TACI metric vanishes in both directions upon synchronization in this case (Eq.9, Figure 2), as one would expect to learn variable y(t) more reliably using the actual data x(
e) In order to assess TACI performance on non-stationary time series, it might be more informative to benchmark it on datasets displaying intermittency rather than synchrony. In particular, the change of causal directions over time, presented as one of the motivations for the new approach, should be more thoroughly benchmarked in the paper. For instance, it would be nice to demonstrate the tracking of the spontaneous reversal of causal relation in a simple 'toggle switch' regulatory network between two mutually repressing genes + expression noise. This is something that causal inference methods assuming stationarity cannot do.

(3) Concerning the real-world applications, the analysis of the electrocorticography (ECoG) data does not seem to be in strong disagreement with the general trends of the original more detailed study by Tajima et al 2015. Could the authors better delineate what are the common versus conflicting findings between the two approaches? The main difference appears to be the near loss of interaction in the anesthetized state, which might be linked to TACI's tendency to report no interaction between synchronized variables as discussed in d) above. Does the anesthetized state correspond to a global synchrony of the brain regions? This could be easily validated by a more direct analysis of synchrony.