We consider a canonical experimental setting in which the same external stimulus, denoted by ${\mathbf{𝐬}}_t$, is repeatedly presented across $L$ independent trials and the spiking activity of a population of $N$ neurons are indirectly measured using two-photon calcium fluorescence imaging. Figure 1 (forward arrow) shows the generative model that is used to quantify this procedure. The fluorescence observation in the $l^{\mathrm{𝗍𝗁}}$ trial from the $j^{\mathrm{𝗍𝗁}}$ neuron at time frame $t$, denoted by $y_{t,l}^{(j)}$, is a noisy surrogate of the intracellular calcium concentrations. The calcium concentrations in turn are temporally blurred surrogates of the underlying spiking activity $n_{t,l}^{(j)}$, as shown in Figure 1.

Figure 1

Download asset Open asset

In modeling the spiking activity, we consider two main contributions: (1) the common known stimulus ${\mathbf{𝐬}}_t$ affects the activity of the $j^{\mathrm{𝗍𝗁}}$ neuron via an unknown kernel ${\mathbf{𝐝}}_j$, akin to the receptive field; (2) the trial-to-trial variability and other intrinsic/extrinsic neural covariates that are not time-locked to the stimulus ${\mathbf{𝐬}}_t$ are captured by a trial-dependent latent process $x_{t,l}^{(j)}$. Then, we use a Generalized Linear Model to link these underlying neural covariates to spiking activity (Truccolo et al., 2005). More specifically, we model spiking activity as a Bernoulli process:

where $\varphi (\cdot )$ is a mapping function, which could in general be non-linear.

The signal correlations aim to measure the correlations in the temporal response that are time-locked to the repeated stimulus, ${\mathbf{𝐬}}_t$. On the other hand, noise correlations in our setting quantify connectivity arising from covariates that are unrelated to the stimulus, including the trial-to-trial variability (Keeley et al., 2020). Based on the foregoing model, we propose to formulate the signal $\left({({\mathbf{𝚺}}_s)}_{i,j}\right)$ and noise $\left({({\mathbf{𝚺}}_x)}_{i,j}\right)$ covariance between the $i^{\mathrm{𝗍𝗁}}$ neuron and $j^{\mathrm{𝗍𝗁}}$ neuron as:

where ${\displaystyle \mathrm{cov}(\cdot ,\cdot )}$ is the empirical covariance function defined for two vector time series ${\displaystyle {\mathbf{u}}_t}$ and ${\displaystyle {\mathbf{v}}_t}$ as ${\displaystyle \mathrm{cov}\left({\mathbf{u}}_t,{\mathbf{v}}_t\right):=\frac{1}{T}\sum _{t=1}^T\left({\mathbf{u}}_t-\frac{1}{T}\sum _{t^{\mathrm{\prime }}=1}^T{\mathbf{u}}_{t^{\mathrm{\prime }}}\right){\left({\mathbf{v}}_t-\frac{1}{T}\sum _{t^{\mathrm{\prime }}=1}^T{\mathbf{v}}_{t^{\mathrm{\prime }}}\right)}^{\mathrm{\top }}}$, for a total observation duration of $T$ time frames.

Our main contribution is to provide an efficient solution for the so-called inverse problem: direct estimation of ${\mathbf{𝚺}}_s$ and ${\mathbf{𝚺}}_x$ from the fluorescence observations, without requiring intermediate spike deconvolution (Figure 1, backward arrow). The signal and noise correlation matrices, denoted by $\mathbf{𝐒}$ and $\mathbf{𝐍}$, can then be obtained by standard normalization of ${\mathbf{𝚺}}_s$ and ${\mathbf{𝚺}}_x$:

We note that when spiking activity is directly observed using electrophysiology recordings, the conventional signal $\left({({\mathbf{𝚺}}_s^{\mathrm{𝖼𝗈𝗇}})}_{i,j}\right)$ and noise $\left({({\mathbf{𝚺}}_x^{\mathrm{𝖼𝗈𝗇}})}_{i,j}\right)$ covariances of spiking activity between the $i^{\mathrm{𝗍𝗁}}$ and $j^{\mathrm{𝗍𝗁}}$ neuron are defined as (Lyamzin et al., 2015):

which after standard normalization in Equation 2 give the conventional signal $\left({({\mathbf{𝐒}}^{\mathrm{𝖼𝗈𝗇}})}_{i,j}\right)$ and noise $\left({({\mathbf{𝐍}}^{\mathrm{𝖼𝗈𝗇}})}_{i,j}\right)$ correlations. While at first glance, our definitions of signal and noise covariances in Equation 1 seem to be a far departure from the conventional ones in Equation 3, we show that the conventional notions of correlation indeed approximate the same quantities as in our definitions:

under asymptotic conditions (i.e. $T$ and $L$ sufficiently large). We prove this assertion of asymptotic equivalence in Appendix 1, which highlights another facet of our contributions: our proposed estimators are designed to robustly operate in the regime of finite (and typically small) $T$ and $L$, aiming for the very same quantities that the conventional estimators could only recover accurately under ideal asymptotic conditions.

In order to compare the performance of our proposed method with existing work, we consider three widely available methods for extracting neuronal correlations. In simulation studies, we additionally benchmark these estimates with respect to the known ground truth. The existing methods considered are the following:

• Pearson correlations from the two-photon data

• In this method, fluorescence observations are assumed to be the direct measurements of spiking activity, and thus empirical Pearson correlations of the two-photon data are used to compute the signal and noise correlations (Rothschild et al., 2010; Winkowski and Kanold, 2013; Francis et al., 2018; Bowen et al., 2020). Explicitly, these estimates are obtained by simply replacing $n_{t,l}^{(j)}$ in Equation 3 by $y_{t,l}^{(j)}$, without performing spike deconvolution.

• Two-stage Pearson estimation

• Unlike the previous method, in this case spikes are first inferred using a deconvolution technique. Then, following temporal smoothing via a narrow Gaussian kernel the Pearson correlations are computed using the conventional definitions of Equation 3. For spike deconvolution, we primarily used the FCSS algorithm (Kazemipour et al., 2018). In order to also demonstrate the sensitivity of these estimates to the deconvolution technique that is used, we provide a comparison with the f-oopsi deconvolution algorithm (Pnevmatikakis et al., 2016) in Figure 2—figure supplement 1.

• Two-stage GPFA estimation

• Similar to the previous method, spikes are first inferred using a deconvolution technique. Then, a latent variable model called Gaussian Process Factor Analysis (GPFA) (Yu et al., 2009) is applied to the inferred spikes in order to estimate the latent covariates and receptive fields. Based on those estimates, the signal and residual noise correlations are derived through a formulation similar to Equation 1 and Equation 2 (Ecker et al., 2014).

We simulated calcium fluorescence observations according to the proposed generative model given in Proposed forward model, from an ensemble of $N=8$ neurons for a duration of $T=5000$ time frames. We considered $L=20$ repeated trials driven by the same external stimulus, which we modeled by an autoregressive process (see Guidelines for model parameter settings for details). Figure 2 shows the corresponding estimation results.

Figure 2 with 6 supplements see all

Download asset Open asset

The first column of Figure 2A shows the ground truth noise (top) and signal (bottom) correlations (diagonal elements are all equal to one and omitted for visual convenience). The second column shows estimates of the noise and signal correlations using our proposed method, which closely match the ground truth. The third, fourth and fifth columns, respectively, show the results of the Pearson correlations from the two-photon data, two-stage Pearson, and two-stage GPFA estimation methods. Through a qualitative visual inspection, it is evident that these methods incur high false alarms and mis-detections of the ground truth correlations.

To quantify these comparisons, the normalized mean square error (NMSE) of different estimates with respect to the ground truth are shown below each of the subplots (Figure 2A). Our proposed method achieves the lowest NMSE compared to the others. Furthermore, we observed a significant mixing between signal and noise correlations in these other estimates. To quantify this leakage effect, we first classified each of the correlation entries as in-network or out-of-network, based on being non-zero or zero in the ground truth, respectively (see Performance evaluation). We then computed the ratio between the power of out-of-network components and the power of in-network components as a measure of leakage. The leakage ratios are also reported in Figure 2A. The leakage of our proposed estimates is the lowest of all four techniques, in estimating both the signal and noise correlations. In order to further probe the performance of our proposed method, the simulated external stimulus s_t, latent trial-dependent process $x_{t,1}^{(1)}$ , simulated observations $y_{t,1}^{(1)}$ , estimated calcium concentration ${\widehat{z}}_{t,1}^{(1)}$ , the putative spikes ${\widehat{n}}_{t,1}^{(1)}:={\widehat{z}}_{t,1}^{(1)}-\alpha {\widehat{z}}_{t-1,1}^{(1)}$ , and the estimated mean of the latent state $m_{{\mathbf{𝐱}}_{t,1}}^{(1)}$ , for the first trial of the first neuron are shown in Figure 2B. These results demonstrate the ability of the proposed estimation framework in accurately identifying the latent processes, which in turn leads to an accurate estimation of the signal and noise correlations as shown in Figure 2B.

The main sources of the observed performance gap between our proposed method and the existing ones are the bias incurred by treating the fluorescence traces as spikes, low spiking rates, non-linearity of spike generation with respect to intrinsic and external covariates, and sensitivity to spike deconvolution. For the latter, we demonstrated the sensitivity of the two-stage Pearson estimates to the choice of the deconvolution technique in Figure 2—figure supplement 1. Furthermore, in order to isolate the effect of said non-linearities on the estimation performance, we applied the two-stage methods to ground truth spikes in Figure 2—figure supplement 2. Our analysis showed that both two-stage estimates incur significant estimation errors even if the spikes were recovered perfectly, mainly due to the limited number of trials ($L=20$ here). In accordance with our theoretical analysis of the asymptotic behavior of the conventional signal and noise correlation estimates given in Appendix 1, we also showed in Figure 2—figure supplement 2 that the performance of the two-stage Pearson estimates based on ground truth spikes, but using $L=1000$ trials, dramatically improves. Our proposed method, however, was capable of producing reliable estimates with the number of trials as low as $L=20$, which is typical in two-photon imaging experiments.

Next, we present the results of a simulation study in the absence of external stimuli (i.e. ${\displaystyle {\mathbf{s}}_t=\mathbf{0}}$), pertaining to the spontaneous activity condition. It is noteworthy that the proposed method can readily be applied to estimate noise correlations during spontaneous activity, by simply setting the external stimulus ${\mathbf{𝐬}}_t$ and the receptive field ${\mathbf{𝐝}}_j$ to zero in the update rules (see Proposed forward model for details). We simulated the ensemble spiking activity based on a Poisson process (Smith and Brown, 2003) using a discrete time-rescaling procedure (Brown et al., 2002; Smith and Brown, 2003), so that the data are generated using a different model than that used in our inference framework (i.e. Bernoulli process with a logistic link as outlined in Proposed forward model). As such, we eliminated potential performance biases in favor of our proposed method by introducing the aforementioned model mismatch. We simulated $L=20$ independent trials of spontaneous activity of $N=30$ neurons, observed for a time duration of $T=5000$ time frames. The number of neurons in this study is notably larger than that used in the previous one, to examine the scalability of our proposed approach with respect to the ensemble size.

Figure 3 shows the comparison of the noise correlation matrices estimated by our proposed method, Pearson correlations from two-photon recordings, two-stage Pearson, and two-stage GPFA estimates, with respect to the ground truth. The Pearson and the two-stage estimates are highly variable and result in excessive false detections. Our proposed estimate, however, closely follows the ground truth, which is also reflected by the comparatively lower NMSE and leakage ratios, in spite of the mismatch between the models used for data generation and inference. In addition, our proposed method exhibits favorable scaling with respect to the ensemble size, thanks to the underlying low-complexity variational updates (see Low-complexity parameter updates for details).

Figure 3

Download asset Open asset

We next applied our proposed method to experimentally recorded two-photon observations from the mouse primary auditory cortex (A1). The dataset consisted of recordings from 371 excitatory neurons in layer 2/3 A1, from which we selected $N=16$ responsive neurons (i.e. neurons that exhibited at least one spiking event in at least half of the trials considered; see Guidelines for model parameter settings). A random sequence of four tones was presented to the mouse, with the same sequence being repeated for $L=10$ trials. Each trial consisted of $T=3600$ time frames, and each tone was 2 s long followed by a 4 s silent period (see Experimental procedures for details). We considered an integration window of $R=25$ frames for stimulus encoding (see Guidelines for model parameter settings for details). The comparison of the noise and signal correlation estimates obtained by our proposed method, Pearson correlations from two-photon recordings, two-stage Pearson and two-stage GPFA methods is shown in Figure 4A. The spatial map of the 16 neurons considered in the analysis in the field of view is shown in Figure 4B. Figure 4C shows the stimulus tone sequence s_t , two-photon observations $y_{t,1}^{(1)}$ , estimated calcium concentration ${\widehat{z}}_{t,1}^{(1)}$ , putative spikes ${\widehat{n}}_{t,1}^{(1)}:={\widehat{z}}_{t,1}^{(1)}-\alpha {\widehat{z}}_{t-1,1}^{(1)}$ and the estimated mean of the latent state $m_{{\mathbf{𝐱}}_{t,1}}^{(1)}$ , for the first trial of the first neuron.

Figure 4 with 2 supplements see all

Download asset Open asset

We estimated the Best Frequency (BF) of each neuron as the tone that resulted in the highest level of fluorescence activity. The results in Figure 4A are organized such that the neurons with the same BF are neighboring, with the BF increasing along the diagonal. Thus, expectedly (Bowen et al., 2020) our proposed method as well as the Pearson and two-stage Pearson estimates show high signal correlations along the diagonal. However, the two-stage GPFA estimates do not reveal such a structure. By visual inspection, as also observed in the simulation studies, the Pearson correlations from two-photon recordings, two-stage Pearson and two-stage GPFA estimates have significant leakage between the signal and noise correlations, whereas our proposed signal and noise correlation estimates in Figure 4A suggest distinct spatial structures.

To quantify this visual comparison, we used a statistic based on the Tanimoto similarity metric (Lipkus, 1999), denoted by $T_s(\mathbf{𝐗},\mathbf{𝐘})$ for two matrices $\mathbf{𝐗}$ and $\mathbf{𝐘}$. As a measure of dissimilarity, we used $T_d(\mathbf{𝐗},\mathbf{𝐘}):=1-T_s(\mathbf{𝐗},\mathbf{𝐘})$ (see Performance evaluation for details). The comparison of $T_d(\widehat{\mathbf{𝐒}},\widehat{\mathbf{𝐍}})$ for the four estimates is presented in the second column of Table 1. To assess statistical significance, for each comparison we obtained null distributions corresponding to chance occurrence of dissimilarities using a shuffling procedure as shown in Figure 4D, and then computed one-tailed $p$-values from those distributions (see Performance evaluation for details). Table 1 and Figure 4D includes these p-values, which show that the proposed estimates (boldface numbers in Table 1, second column) indeed have the highest dissimilarity between signal and noise correlations. The higher leakage effect in the other three estimates is also reflected in their smaller $T_d(\widehat{\mathbf{𝐒}},\widehat{\mathbf{𝐍}})$ values.

To further investigate this effect, we have depicted the scatter plots of signal vs. noise correlations estimated by each method in Figure 4E. To examine the possibility of the leakage effect on a pairwise basis, we performed linear regression in each case. The slope of the model fit, the p-value for the corresponding t-test, and the $R^2$ values are reported in the third and fourth columns of Table 1 (the slope and p-values are also shown as insets in Figure 4E). Consistent with the results of Winkowski and Kanold, 2013, the Pearson estimates suggest a significant correlation between the signal and noise correlation pairs (as indicated by the higher slope in Figure 4E). However, none of the other estimates (including the proposed estimates) in Figure 4E register a significant trend between signal and noise correlations. This further corroborates our assessment of the high leakage between signal and noise correlations in Pearson estimates, since such a leakage effect could result in overestimation of the trend between the signal and noise correlation pairs. The signal and noise correlations estimated by our proposed method show no pairwise trend, suggesting distinct patterns of stimulus-dependent and stimulus-independent functional connectivity (Kohn et al., 2016; Montijn et al., 2014; Rothschild et al., 2010; Keeley et al., 2020).

A key advantage of our proposed method over the Pearson and two-stage approaches is the explicit modeling of stimulus integration. The relevant parameter in this regard is the length of the stimulus integration window $R$. While in our simulation studies the value of $R$ was known, it needs to be set by the user in real data applications. To this end, domain knowledge or data-driven methods such as cross-validation and model order selection can be utilized (see Guidelines for model parameter settings for details). Noting that the number of parameters to be estimated linearly scales with $R$, it must be chosen large enough to capture the stimulus effects, yet small enough to result in favorable computational complexity. Here, given that the typical tone response duration of mouse A1 neurons is ${\displaystyle <1}$ s (Linden et al., 2003; DeWeese et al., 2003; Petrus et al., 2014), with a sampling frequency of $f_s=30$ Hz, we surmised that a choice of $R\sim 30$ suffices to capture the stimulus effects. We further examined the effect of varying $R$ on the proposed correlation estimates in Figure 4—figure supplement 1. As shown, small values of $R$ (e.g. $R=1$ or 10) may not be adequate to fully capture stimulus integration effects. By considering values of $R$ in the range $25-50$, we observed that the correlation estimates remain stable. We thus chose $R=25$ for our analysis.

Careful inspection of the second panel in Figure 4C shows that the fluorescence activity often saturates to ∼4 times its baseline value. This effect is due to successive closely spaced spikes, which implies the occurrence of more than one spike per frame and thus violates our Bernoulli modeling assumption. To inspect the performance of our method more carefully under this scenario, we show in Figure 4—figure supplement 2 a zoomed-in view of the estimated latent processes ${\widehat{z}}_{t,1}^{(1)}$ (calcium concentration) and ${\widehat{n}}_{t,1}^{(1)}$ (putative spikes) for a sample data segment with high fluorescence activity. The estimated latent processes reveal two mechanisms leveraged by our inference method to mitigate the aforementioned model mismatch: first, our proposed method predicts spiking events in adjacent time frames to compensate for rapid increase in firing rate and thus infers calcium concentration levels that match the observed fluorescence; secondly, even though our generative model assumes that there is only one spiking event in a given time frame, this restriction is mitigated in our inference framework by relaxing the constraint ${\widehat{n}}_{t,l}^{(j)}:={\widehat{z}}_{t,l}^{(j)}-\alpha {\widehat{z}}_{t-1,l}^{(j)}\le 1$, as explained in Low-complexity parameter updates. While this relaxation was performed for the sake of tractability of the inverse solution, it in fact leads to improved estimation results under episodes of rapid increase in firing rate, by allowing the putative spike magnitudes ${\widehat{n}}_{t,l}^{(j)}$ to be greater than 1. The latter is evident in the magnitude of the inferred spikes in Figure 4—figure supplement 2, following the rise of fluorescence activity.

Given that the ground truth correlations are not available for a direct comparison, we instead performed a test of specificity that reveals another key limitation of existing methods. Fluorescence observations exhibit structured dynamics due to the exponential intracellular calcium concentration decay (as shown in Figure 4C, for example), which are in turn related to the underlying spikes that are driven non-linearly by intrinsic/extrinsic stimuli as well as the properties of the indicator used. As such, an accurate inference method is expected to be specific to this temporal structure. To test this, we randomly shuffled the $T$ time frames consistently in the same order in all trials, in order to fully break the temporal structure governing calcium decay dynamics, and then estimated correlations from these shuffled data using the different methods. The resulting estimates of noise correlations are shown in Figure 5A for one instance of such shuffled data. The average NMSE for a total of 50 shuffled samples with respect to the original un-shuffled estimates (in Figure 4A) are tabulated in the fifth and sixth columns of Table 1, and are also indicated below each panel in Figure 5A.

Figure 5

Download asset Open asset

A visual inspection of Figure 5A shows that the Pearson correlations from two-photon recordings expectedly remain unchanged. Since this method treats each time frame to be independent, temporal shuffling does not impact the correlations in anyway. On the other extreme, both of the two-stage estimates seem to detect highly variable and large correlation values, despite operating on data that lacks any relevant temporal structure. Our proposed method, however, remarkably produces negligible correlation estimates. Although both the two-stage and proposed estimates show variability with respect to the shuffled data (Table 1, fifth column), the standard deviation of the NMSE values of our proposed method are considerably smaller than those of the two-stage methods (Table 1, fifth column). For further inspection, the histograms of a single element (${(\widehat{\mathbf{𝐍}})}_{1,3}$) of the estimated correlation matrices across the 50 shuffling trials are shown in Figure 5B. The original un-shuffled estimates are marked by the dashed vertical lines in each case. The proposed estimate in Figure 5B is highly concentrated around zero, even though the un-shuffled estimate is non-zero. However, the two-stage estimates produce correlations that are widely variable across the shuffling trials. This analysis demonstrates that our proposed method is highly specific to the temporal structure of fluorescence observations, whereas the Pearson correlations from two-photon recordings, two-stage Pearson and two-stage GPFA methods fail to be specific.

To further validate the utility of our proposed methodology, we applied it to another experimentally-recorded dataset from the mouse A1 layer 2/3. This experiment pertained to trials of presenting a sequence of short white noise stimuli, randomly interleaved with silent trials of the same duration. Figure 6A shows a sample trial sequence. The two-photon recordings thus contained episodes of stimulus-driven and spontaneous activity (see Experimental procedures for details). Under this experimental setup, it is expected that the noise correlations are invariant across the spontaneous and stimulus-driven conditions. In accordance with the foregoing results of real data study 1, we also expect the signal and noise correlation patterns to be distinct. Each trial considered in the analysis consisted of $T=765$ frames (see Experimental procedures for details). We selected $N=10$ responsive neurons (according to the criterion described in Guidelines for model parameter settings), each with $L=10$ trials. Similar to real data study 1, we chose a stimulus integration window of length $R=25$ frames.

Figure 6 with 1 supplement see all

Download asset Open asset

Figure 6B shows the resulting noise and signal correlation estimates under the spontaneous (${\widehat{\mathbf{𝐍}}}_{\mathrm{𝗌𝗉𝗈𝗇}}$, top) and stimulus-driven (${\widehat{\mathbf{𝐍}}}_{\mathrm{𝗌𝗍𝗂𝗆}}$ and ${\widehat{\mathbf{𝐒}}}_{\mathrm{𝗌𝗍𝗂𝗆}}$, bottom) conditions. Figure 6C shows the spatial map of the 10 neurons considered in the analysis in the field of view. A visual inspection of the first column of Figure 6B indeed suggests that ${\widehat{\mathbf{𝐍}}}_{\mathrm{𝗌𝗉𝗈𝗇}}$ and ${\widehat{\mathbf{𝐍}}}_{\mathrm{𝗌𝗍𝗂𝗆}}$ are saliently similar, and distinct from ${\widehat{\mathbf{𝐒}}}_{\mathrm{𝗌𝗍𝗂𝗆}}$. The Pearson correlations obtained from two-photon data (second column) and the two-stage Pearson and GPFA estimates (third and fourth columns, respectively), however, evidently lack this structure. As in the previous study, we quantified this visual comparison using the similarity metric $T_s(\mathbf{𝐗},\mathbf{𝐘})$ and the dissimilarity metric $T_d(\mathbf{𝐗},\mathbf{𝐘})$ (see Performance evaluation for details). These statistics are reported in Table 2 along with the p-values (null distributions are shown in Figure 6—figure supplement 1), which show that the only significant outcomes (boldface numbers) are those of our proposed method. While it is expected from the experiment design for the noise correlations under the two settings to be similar, the only method that detects this expected outcome with statistical significance is our proposed method. Moreover, the statistically significant dissimilarity between the signal and noise correlations of our proposed estimates corroborate the hypothesis that signal and noise are encoded by distinct functional networks (Kohn et al., 2016; Montijn et al., 2014; Rothschild et al., 2010; Keeley et al., 2020).

Furthermore, Figure 6D shows the time course of the stimulus, observations, estimated calcium concentrations and putative spikes for the first trial from two pairs of neurons with high signal correlation ($j=2,8$, top) and high noise correlation ($j=3,5$, bottom). As expected, the putative spiking activity of the neurons with high signal correlation (top) are closely time-locked to the stimulus onsets. The activity of the two neurons with high noise correlation (bottom), however, is not time-locked to the stimulus onsets, even though the two neurons exhibit highly correlated activity. The correlations estimated via the proposed method thus encode substantial information about the inter-dependencies of the spiking activity of the neuronal ensemble.

Lastly, we applied our proposed method to examine the spatial distribution of signal and noise correlations in the mouse A1 layers 2/3 and 4 (data from Bowen et al., 2020). The dataset included fluorescence activity recorded during multiple experiments of presenting sinusoidal amplitude-modulated tones, with each stimulus being repeated across several trials (see Experimental procedures and Bowen et al., 2020 for experimental details). In each experiment, we selected on average around 20 responsive neurons for subsequent analysis (according to the criterion described in Guidelines for model parameter settings). For brevity, we compare the estimates of signal and noise correlations using our proposed method only with those obtained by Pearson correlations from the two-photon data. The latter method was also used in previous analyses of data from this experimental paradigm (Winkowski and Kanold, 2013).

In parallel to the results reported in Winkowski and Kanold, 2013, Figure 7A and Figure 7B illustrate the correlation between the signal and noise correlations in layers 2/3 and 4, respectively. Consistent with the results of Winkowski and Kanold, 2013, the signal and noise correlations exhibit positive correlation in both layers, regardless of the method used. However, the correlation coefficients (i.e. slopes in the insets) identified by our proposed method are notably smaller than those obtained from Pearson correlations, in both layer 2/3 (Figure 7A) and layer 4 (Figure 7B). Comparing this result with our simulation studies suggests that the stronger linear trend between the signal and noise correlations observed using the Pearson correlation estimates is likely due to the mixing between the estimates of signal and noise correlations. As such, our method suggests that the signal and noise correlations may not be as highly correlated with one another as indicated in previous studies of layer 2/3 and 4 in mouse A1 (Winkowski and Kanold, 2013).

Next, to evaluate the spatial distribution of signal and noise correlations, we plotted the correlation values for pairs of neurons as a function of their distance for layer 2/3 (Figure 7C) and layer 4 (Figure 7D). The distances were discretized using bins of length ${\displaystyle 10\mu m}$. The scatter of the correlations along with their median at each bin are shown in all panels. Then, to examine the spatial trend of the correlations, we performed linear regression in each case. The slope of the model fit, the p-value for the corresponding t-test, and the $R^2$ values are reported in Table 3 (the slope and p-values are also shown as insets in Figure 7C and D).

From Table 3 and Figure 7C and D (upper panels), it is evident that the signal correlations show a significant negative trend with respect to distance, using both methods and in both layers. However, the slope of these negative trends identified by our method (boldface numbers in Table 3) is notably steeper than those identified by Pearson correlations. On the other hand, the trends of the noise correlations with distance (bottom panels) are different between our proposed method and Pearson correlations: our proposed method shows a significant negative trend in layer 2/3, but not in layer 4, whereas the Pearson correlations of the two-photon data suggest a significant negative trend in layer 4, but not in layer 2/3. In addition, the slopes of these negative trends identified by our method (boldface numbers in Table 3) are steeper than or equal to those identified by Pearson correlations.

Our proposed estimates also indicate that noise correlations are sparser and less widespread in layer 4 (Figure 7D) than in layer 2/3 (Figure 7C). To further investigate this observation, we depicted the two-dimensional spatial spread of signal and noise correlations in both layers and for both methods in Figure 7E and F, by centering each neuron at the origin and overlaying the individual spatial spreads. The horizontal and vertical axes in each panel represent the relative dorsoventral and rostrocaudal distances, respectively, and the heat-maps represent the magnitude of correlations. Comparing the proposed noise correlation spread in Figure 7E with the corresponding spread in Figure 7F, we observe that the noise correlations in layer 2/3 are indeed more widespread and abundant than in layer 4, as can be expected by more extensive intralaminar connections in layer 2/3 vs. 4 (Watkins et al., 2014; Meng et al., 2017a; Meng et al., 2017b; Kratz and Manis, 2015).

Figure 7 with 2 supplements see all

Download asset Open asset

The spatial spreads of signal and noise correlations based on the Pearson estimates are remarkably similar in both layers (Figure 7E and F, right panels), whereas they are saliently different for our proposed estimates (Figure 7E and F, left panels). This further corroborates our hypothesis on the possibility of high mixing between the signal and noise correlation estimates obtained by the Pearson correlation of two-photon data. To further examine the differences between the signal and noise correlations, the marginal distributions along the dorsoventral and rostrocaudal axes are shown in Figure 7E and F, selectively overlaid for ease of visual comparison. To quantify the differences between the spatial distributions of signal and noise correlations estimated by each method, we performed Kolmogorov-Smirnov (KS) tests on each pair of marginal distributions, which are summarized in Figure 7—figure supplement 1. Although the marginal distributions of signal and noise correlations are significantly different in all cases from both methods, the effect sizes of their difference (KS statistics) are higher for our proposed estimates compared to those of the Pearson estimates.

Finally, the spatial spreads of correlations for either method and in each layer suggest non-uniform angular distributions with possibly directional bias. To test this effect, we computed the angular marginal distributions and performed KS tests for non-uniformity, which are reported in Figure 7—figure supplement 2. These tests indicate that all distributions are significantly non-uniform. In addition, the angular distributions of both signal and noise correlations in layer 4 exhibit salient modes in the rostrocaudal direction, whereas they are less directionally selective in layer 2/3 (Figure 7—figure supplement 2).

In summary, the spatial trends identified by our proposed method are consistent with empirical observations of spatially heterogeneous pure-tone frequency tuning by individual neurons in auditory cortex (Winkowski and Kanold, 2013). The improved correspondence of our proposed method compared to results obtained using Pearson correlations could be the result of the demixing of signal and noise correlations in our method. As a result of the demixing, our proposed method also suggests that noise correlations have a negative trend with distance in layer 2/3, but are much sparser and spatially flat in layer 4. In addition, the spatial spread patterns of signal and noise correlations are more structured and remarkably more distinct for our proposed method than those obtained by the Pearson estimates.

Finally, we present a theoretical analysis of the bias and variance of the proposed estimator. Note that our proposed estimation method has been developed as a scalable alternative to the intractable maximum likelihood (ML) estimation of the signal and noise covariances (see Overview of the proposed estimation method). In order to benchmark our estimates, we thus need to evaluate the quality of said ML estimates. To this end, we derived bounds on the bias and variance of the ML estimators of the kernel ${\mathbf{𝐝}}_j$ for $j=1,\mathrm{\cdots },N$ and the noise covariance ${\mathbf{𝚺}}_x$. In order to simplify the treatment, we posit the following mild assumptions:

Assumption (1). We assume a scalar time-varying external stimulus (i.e. ${\mathbf{𝐬}}_t=s_t$, and hence ${\mathbf{𝐝}}_j=d_j,\mathbf{𝐝}={[d_1,d_2,\mathrm{\cdots },d_N]}^{\top }$ ). Furthermore, we set the observation noise covariance to be ${\mathbf{𝚺}}_w={\sigma }_w^2\mathbf{𝐈}$, for notational convenience.

Assumption (2). We derive the performance bounds in the regime where $T$ and $L$ are large, and thus do not impose any prior distribution on the correlations, which are otherwise needed to mitigate overfitting (see Preliminary assumptions).

Assumption (3). We assume the latent trial-dependent process and stimulus to be slowly varying signals, and thus adopt a piece-wise constant model in which these processes are constant within consecutive windows of length $W$ (i.e. ${\mathbf{𝐱}}_{t,l}={\mathbf{𝐱}}_{W_k,l}$ and $s_t=s_{W_k}$, for $(k-1)W+1\le t<kW$ and $k=1,\mathrm{\cdots },K$ with $W_k=(k-1)W+1$ and $KW=T$) for our theoretical analysis, as is usually done in spike count calculations for conventional noise correlation estimates.

Our main theoretical result is as follows:

Theorem 1 (Performance Bounds) Let $q\mathrm{>}\frac{\mathrm{1}}{\mathrm{64}}$, $\mathrm{0}\mathrm{<}ϵ\mathrm{<}\mathrm{1}\mathrm{/}\mathrm{2}$, and $\mathrm{0}\mathrm{<}\eta \mathrm{\le }\mathrm{1}\mathrm{/}\mathrm{2}$ be fixed constants, ${\sigma }_m^{\mathrm{2}}\mathrm{:=}{\max }_i{\mathrm{(}{\mathrm{\Sigma }}_x\mathrm{)}}_{i\mathrm{,}i}$ and ${\sigma }_s^{\mathrm{2}}\mathrm{:=}\frac{\mathrm{1}}{K}\mathit{}{\mathrm{\sum }}_{k\mathrm{=}\mathrm{1}}^K{s}_{W_k}^{\mathrm{2}}$. Then, under Assumptions (1 - 3), the bias and variance of the maximum likelihood estimators $\widehat{\mathrm{d}}$ and ${\widehat{\mathrm{\Sigma }}}_x$, conditioned on an event ${\mathrm{A}}_W$ with $\mathrm{P}\mathit{}\mathrm{\left(}{\mathrm{A}}_W\mathrm{\right)}\mathrm{\ge }\mathrm{1}\mathrm{-}\eta$ satisfy:

for all $i\mathrm{,}j\mathrm{=}\mathrm{1}\mathrm{,}\mathrm{2}\mathrm{,}\mathrm{\cdots }\mathrm{,}N$, if

where ${\tau }_j$ and ${\stackrel{\mathrm{~}}{\tau }}_j$ denote bounded terms that are $\mathrm{O}\mathit{}\mathrm{(}{\sigma }_w^{\mathrm{2}}\mathrm{)}$ or $\mathrm{O}\mathit{}\mathrm{\left(}\frac{\mathrm{1}}{W}\mathrm{\right)}$, ${\xi }_{i\mathrm{,}j}$ and ${\stackrel{\mathrm{~}}{\xi }}_{i\mathrm{,}j}$ denote bounded terms that are $\mathrm{O}\mathit{}\mathrm{(}{\sigma }_w^{\mathrm{2}}\mathrm{)}$ or $\mathrm{O}\mathit{}\mathrm{\left(}\frac{\mathrm{1}}{W^{\mathrm{1}\mathrm{-}\mathrm{2}\mathit{}ϵ}}\mathrm{\right)}$ and $C_{\mathrm{1}}\mathrm{,}{C}_{\mathrm{2}}\mathrm{,}{C}_{\mathrm{3}}$ and $C_{\mathrm{4}}$ are bounded constants given in Appendix 2.

Proof. The proof of Theorem 1 is provided in Appendix 2.

■

In order to discuss the implications of this theoretical result, several remarks are in order:

• Remark 1: Achieving near Oracle performance

• A common benchmark in estimation theory is the performance of the idealistic oracle estimator, in which an oracle directly observes the true latent process ${\mathbf{𝐱}}_{t,l}$ and the true kernel d_j and forms the correlation estimates. In this case, the oracle would incur zero bias and variance of order $𝒪\left(1/KL\right)$ in estimating d_j, and outputs an estimate of ${\mathbf{𝚺}}_x$ with bias and variance in the order of $𝒪\left(1/KL\right)$. Theorem 1 indeed states that for sufficiently large $W$ and small ${\sigma }_w$, the bias and variance of the ML estimators are arbitrarily close to those of the oracle estimator. Recall that our variational inference framework is in fact a solution technique for the regularized ML problem. Hence, the bounds in Theorem 1 provide a benchmark for the expected performance of the proposed estimators, by quantifying the excess bias and variance over the performance of the oracle estimator.

• Remark 2: Effect of the observation noise and observation duration

• As the assumed window of stationarity $W\to \mathrm{\infty }$ (and hence the observation duration $T\to \mathrm{\infty }$), the loss of performance of the proposed estimators only depends on ${\sigma }_w^2$, the variance of the observation noise. As a result, at a given observation noise variance ${\sigma }_w^2$, these bounds provide a sufficient upper bound on the time duration of the observations required for attaining a desired level of estimation accuracy. It is noteworthy that ${\sigma }_w^2$ is typically small in practice, as it pertains to the effective observation noise and is significantly diminished by pixel averaging of the fluorescence traces following cell segmentation.

• Remark 3: Effect of the number of trials

• Finally, note that the bounds in Theorem 1 have terms that also drop as the number of trials $L$ grows. These terms in fact pertain to the performance of the oracle estimator. As the number of trials grows ($L\to \mathrm{\infty }$), the oracle estimates become arbitrarily close to the true parameters ${\mathbf{𝚺}}_x$ and ${\mathbf{𝐝}}_j$. Thus, our theoretical performance bounds also provide a sufficient upper bound on the number of trials $L$ required for the oracle estimator to attain a desired level of estimation accuracy.

Suppose we observe fluorescence traces of $N$ neurons, for a total duration of $T$ discrete-time frames, corresponding to $L$ independent trials of repeated stimulus. Let ${\displaystyle {\mathbf{y}}_{t,l}:=[y_{t,l}^{(1)},y_{t,l}^{(2)},\cdots ,y_{t,l}^{(N)}{]}^{\mathrm{\top }}}$, ${\displaystyle {\mathbf{z}}_{t,l}:=[z_{t,l}^{(1)},z_{t,l}^{(2)},\cdots ,z_{t,l}^{(N)}{]}^{\mathrm{\top }}}$, and ${\displaystyle {\mathbf{n}}_{t,l}:=[n_{t,l}^{(1)},n_{t,l}^{(2)},\cdots ,n_{t,l}^{(N)}{]}^{\mathrm{\top }}}$ be the vectors of noisy observations, intracellular calcium concentrations, and ensemble spiking activities, respectively, at trial $l$ and frame $t$. We capture the dynamics of ${\mathbf{𝐲}}_{t,l}$ by the following state-space model:

where $\mathbf{𝐀}\in {ℝ}^{N\times N}$ represents the scaling of the observations, ${\mathbf{𝐰}}_{t,l}$ is zero-mean i.i.d. Gaussian noise with covariance ${\mathbf{𝚺}}_w$, and ${\displaystyle 0\le \alpha <1}$ is the state transition parameter capturing the calcium dynamics through a first order model. Note that this state-space is non-Gaussian due to the binary nature of the spiking activity, that is, ${\displaystyle n_{t,l}^{(j)}\in \{0,1\}}$. We model the spiking data as a point process or Generalized Linear Model with Bernoulli statistics (Eden et al., 2004; Paninski, 2004; Smith and Brown, 2003; Truccolo et al., 2005):

where ${\lambda }_{t,l}^{(j)}$ is the conditional intensity function (Truccolo et al., 2005), which we model as a non-linear function of the known external stimulus ${\mathbf{𝐬}}_t$ and the other latent intrinsic and extrinsic trial-dependent covariates, ${\displaystyle {\mathbf{x}}_{t,l}:={\displaystyle [x_{t,l}^{(1)},x_{t,l}^{(2)},\cdots ,x_{t,l}^{(N)}{]}^{\mathrm{\top }}}}$. While we assume the stimulus ${\mathbf{𝐬}}_t\in {ℝ}^M$ to be common to all neurons, we model the distinct effect of this stimulus on the $j^{\mathrm{𝗍𝗁}}$ neuron via an unknown kernel ${\mathbf{𝐝}}_j\in {ℝ}^M$, akin to the receptive field.

The non-linear mapping of our choice is the logistic link, which is also the canonical link for a Bernoulli process in the point process and Generalized Linear Model frameworks (Truccolo et al., 2005). Thus, we assume:

Finally, we assume the latent trial dependent covariates to be a Gaussian process ${\mathbf{𝐱}}_{t,l}\sim 𝒩({\mathit{𝝁}}_x,{\mathbf{𝚺}}_x)$, with mean ${\displaystyle {\mathit{\mu }}_x:={\displaystyle [{\mu }_x^{(1)},{\mu }_x^{(2)},\cdots ,{\mu }_x^{(N)}{]}^{\mathrm{\top }}}}$ and covariance ${\mathbf{𝚺}}_x$.

The probabilistic graphical model in Figure 8 summarizes the main components of the aforementioned forward model. According to this forward model, the underlying noise covariance matrix that captures trial-to-trial variability can be identified as ${\mathbf{𝚺}}_x$. The signal covariance matrix, representing the covariance of the neural activity arising from the repeated application of the stimulus ${\mathbf{𝐬}}_t$, is given by ${\mathbf{𝚺}}_s:={\mathbf{𝐃}}^{\top }cov({\mathbf{𝐬}}_t,{\mathbf{𝐬}}_t)\mathbf{𝐃}$, where ${\displaystyle \mathbf{D}:=[{\displaystyle {\mathbf{d}}_1,{\mathbf{d}}_2,\cdots ,{\mathbf{d}}_N}]\in {\mathbb{R}}^{M\times N}}$. The signal and noise correlation matrices, denoted by $\mathbf{𝐒}$ and $\mathbf{𝐍}$, can then be obtained by standard normalization of ${\mathbf{𝚺}}_s$ and ${\mathbf{𝚺}}_x$:

The main problem is thus to estimate $\{{\mathbf{𝚺}}_x,\mathbf{𝐃}\}$ from the noisy and temporally blurred data ${\left\{{\mathbf{𝐲}}_{t,l}\right\}}_{t=1,l=1}^{T,L}$ .

Figure 8

Download asset Open asset

First, given a limited number of trials $L$ from an ensemble with typically low spiking rates, we need to incorporate suitable prior assumptions to avoid overfitting. Thus, we impose a prior $p_{\mathrm{𝗉𝗋}}({\mathbf{𝚺}}_x)$ on the noise covariance, to compensate sparsity of data. A natural estimation method to estimate $\{{\mathbf{𝚺}}_x,\mathbf{𝐃}\}$ in a Bayesian framework is to maximize the observed data likelihood ${\displaystyle p({\displaystyle \{{\mathbf{y}}_{t,l}{\}}_{t,l=1}^{T,L}|{\mathbf{\Sigma }}_x,\mathbf{D}})}$, that is maximum likelihood (ML). Thus, we consider the joint likelihood of the observed data and latent processes to perform Maximum a Posteriori (MAP) estimation:

Inspecting this MAP problem soon reveals that estimating ${\mathbf{𝚺}}_x$ and $\mathbf{𝐃}$ is a challenging task: (1) standard approaches such as Expectation-Maximization (EM) (Shumway and Stoffer, 1982) are intractable due to the complexity of the model, arising from the hierarchy of latent processes and the non-linearities involved in their mappings and (2) the temporal coupling of the likelihood in the calcium concentrations makes any potential direct solver scale poorly with $T$.

Thus, we propose an alternative solution based on Variational Inference (VI) (Beal, 2003; Blei et al., 2017; Jordan et al., 1999). VI is a method widely used in Bayesian statistics to approximate unwieldy posterior densities using optimization techniques, as a low-complexity alternative strategy to Markov Chain Monte Carlo sampling (Hastings, 1970) or empirical Bayes techniques such as EM. To this end, we treat ${\{{\mathbf{𝐱}}_{t,l}\}}_{t,l=1}^{T,L}$ and ${\mathbf{𝚺}}_x$ as latent variables and ${\{{\mathbf{𝐳}}_{t,l}\}}_{t,l=1}^{T,L}$ and $\mathbf{𝐃}$ as unknown parameters to be estimated. We introduce a framework to update the latent variables and parameters sequentially, with straightforward update rules. We will describe the main ingredients of the proposed framework in the following subsections. Hereafter, we use the shorthand notations $\mathbf{𝐲}:={\{{\mathbf{𝐲}}_{t,l}\}}_{t,l=1}^{T,L}$, $\mathbf{𝐳}:={\{{\mathbf{𝐳}}_{t,l}\}}_{t,l=1}^{T,L}$, and $\mathbf{𝐱}:={\{{\mathbf{𝐱}}_{t,l}\}}_{t,l=1}^{T,L}$.

For the sake of simplicity, we assume that the constants α, $\mathbf{𝐀}$, ${\mathbf{𝚺}}_w$ and ${\mathit{𝝁}}_x$ are either known or can be consistently estimated from pilot trials. Next, we take $p_{\mathrm{𝗉𝗋}}({\mathbf{𝚺}}_x)$ to be an Inverse Wishart density:

which turns out to be the conjugate prior in our model. Thus, ${\mathit{𝝍}}_x$ and ${\rho }_x$ will be the hyper-parameters of our model. Procedures for hyper-parameter tuning and choosing the key model parameters are given in subsections Hyper-parameter tuning and Guidelines for model parameter settings, respectively.

Direct application of VI to problems containing both discrete and continuous random variables results in intractable densities. Specifically, finding a variational distribution for ${\mathbf{𝐱}}_{t,l}$ in our model with a standard distribution is not straightforward, due to the complicated posterior arising from co-dependent Bernoulli and Gaussian random variables. In order to overcome this difficulty, we employ Pólya-Gamma (PG) latent variables (Pillow and Scott, 2012; Polson et al., 2013; Linderman et al., 2016). We observe from Equation 4 that the posterior density, $p(\mathbf{𝐱}|\mathbf{𝐳},\mathbf{𝐃},{\mathbf{𝚺}}_x)$ is conditionally independent in $t,l$ with: 

Thus, upon careful inspection, we see that this density has the desired form for the PG augmentation scheme (Polson et al., 2013). Accordingly, we introduce a set of auxiliary PG-distributed i.i.d. latent random variables ${\displaystyle {\mathit{\omega }}_{t,l}:=[{\omega }_{t,l}^{(1)},{\omega }_{t,l}^{(2)},\cdots ,{\omega }_{t,l}^{(N)}{]}^{\mathrm{\top }}}$, ${\displaystyle {\omega }_{t,l}^{(j)}\sim \mathrm{PG}(1,0)}$ for $1\le j\le N$, $1\le t\le T$ and $1\le l\le L$, to derive the complete data log-likelihood:

where $\mathit{𝝎}:={\left\{{\mathit{𝝎}}_{t,l}\right\}}_{t,l=1}^{T,L}$ and $C$ accounts for terms not depending on $\mathbf{𝐲},\mathbf{𝐳},\mathbf{𝐱},\mathit{𝝎}$, ${\mathbf{𝚺}}_x$ and $\mathbf{𝐃}$. The complete data log-likelihood is notably quadratic in ${\mathbf{𝐳}}_{t,l}$, which as we show later admits efficient estimation procedures with favorable scaling in $T$.

In this section, we will outline the procedure of applying VI to the latent variables $\mathbf{𝐱}={\left\{{\mathbf{𝐱}}_{t,l}\right\}}_{t,l=1}^{T,L},\mathit{𝝎}={\left\{{\mathit{𝝎}}_{t,l}\right\}}_{t,l=1}^{T,L}$ and ${\mathbf{𝚺}}_x$, assuming that the parameter estimates $\widehat{\mathbf{𝐳}}$ and $\widehat{\mathbf{𝐃}}$ of the previous iteration are available. The methods that we propose to update the parameters $\widehat{\mathbf{𝐳}}$ and $\widehat{\mathbf{𝐃}}$ subsequently, will be discussed in the next section.

The objective of variational inference is to posit a family of approximate densities $𝒬$ over the latent variables, and to find the member of that family that minimizes the Kullback-Leibler (KL) divergence to the exact posterior:

However, evaluating the KL divergence is intractable, and it has been shown (Blei et al., 2017) that an equivalent result to this minimization can be obtained by maximizing the alternative objective function, called the evidence lower bound (ELBO):

Further, we assume $𝒬$ to be a mean-field variational family (Blei et al., 2017), resulting in the overall variational density of the form:

Under the mean field assumptions, the maximization of the ELBO can be derived using the optimization algorithm ‘Coordinate Ascent Variational Inference’ (CAVI) (Bishop, 2006; Blei et al., 2017). Accordingly, we see that the optimal variational densities in Equation 6 take the forms:

Upon evaluation of these expectations, we derive the optimal variational distributions as:

whose parameters ${\displaystyle {\mathbf{m}}_{{\mathbf{x}}_{t,l}}:={{\displaystyle [m_{{\mathbf{x}}_{t,l}}^{(1)},m_{{\mathbf{x}}_{t,l}}^{(2)},\cdots ,m_{{\mathbf{x}}_{t,l}}^{(N)}]}}^T}$, ${\mathbf{𝐐}}_{{\mathbf{𝐱}}_{t,l}}$, $c_{t,l}^{(j)}$, ${\mathbf{𝐏}}_x$, and ${\gamma }_x$ can be updated given parameter estimates $\widehat{\mathbf{𝐃}}$ and $\widehat{\mathbf{𝐳}}$:

and ${\gamma }_x:={\rho }_x+TL$, with ${\displaystyle {\tilde{\mathbf{\Omega }}}_{t,l}\in {\mathbb{R}}^{N\times N}}$ denoting a diagonal matrix with entries ${\displaystyle ({\tilde{\mathbf{\Omega }}}_{t,l}{)}_{j,j}:={\displaystyle \frac{1}{2c_{t,l}^{(j)}}\tanh (\frac{c_{t,l}^{(j)}}{2})}}$ and $\mathrm{𝟏}\in {ℝ}^N$ denoting the vector of all ones.

Note that even though $\mathbf{𝐳}$ is composed of the latent processes ${\mathbf{𝐳}}_{t,l}$, we do not use VI for its inference, and instead consider it as an unknown parameter. This choice is due to the temporal dependencies arising from the underlying state-space model in Equation 4, which hinders a proper assignment of variational densities under the mean field assumption. We thus seek to estimate both $\mathbf{𝐳}$ and $\mathbf{𝐃}$ using the updated variational density $q^{*}(\mathbf{𝐱},\mathit{𝝎},{\mathbf{𝚺}}_x)$.

First, note that the log-likelihood in Equation 5 is decoupled in $l$, which admits independent updates to ${\{{\mathbf{𝐳}}_{t,l}\}}_{t=1}^T$, for $l=1,\mathrm{\cdots },L$. As such, given an estimate $\widehat{\mathbf{𝐃}}$, we propose to estimate ${\{{\mathbf{𝐳}}_{t,l}\}}_{t=1}^T$ as:

${\displaystyle \begin{array}{ll}{\displaystyle \{{\hat{\mathbf{z}}}_{t,l}{\}}_{t=1}^T}& =\underset{\{{\mathbf{z}}_{t,l}{\}}_{t=1}^T}{\text{argmax}}{\mathbb{E}}_{q^{\ast }(\mathbf{x},\mathit{\omega },{\mathbf{\Sigma }}_x)}\left[\log p\left(\mathbf{y},\mathbf{z},\mathbf{x},\mathit{\omega },{\mathbf{\Sigma }}_x|\hat{\mathbf{D}}\right)\right]\\ & =\underset{\{{\mathbf{z}}_{t,l}{\}}_{t=1}^T}{\text{argmin}}\sum _{t=1}^T\left\{\frac{1}{2}{\left({\mathbf{y}}_{t,l}-\mathbf{A}{\mathbf{z}}_{t,l}\right)}^{\mathrm{\top }}{\mathbf{\Sigma }}_w^{-1}\left({\mathbf{y}}_{t,l}-\mathbf{A}{\mathbf{z}}_{t,l}\right)-\sum _{j=1}^N\left(m_{{\mathbf{x}}_{t,l}}^{(j)}+{{\hat{\mathbf{d}}}_j}^{\mathrm{\top }}{\mathbf{s}}_t\right)\left(z_{t,l}^{(j)}-\alpha z_{t-1,l}^{(j)}\right)\right\},\end{array}}$ under the constraints ${\displaystyle 0\le {\displaystyle z_{t,l}^{(j)}-\alpha z_{t-1,l}^{(j)}}\le 1}$, for $t=1,\mathrm{\cdots },T$ and $j=1,\mathrm{\cdots },N$. These constraints are a direct consequence of $n_{t,l}^{(j)}=z_{t,l}^{(j)}-\alpha z_{t-1,l}^{(j)}$ being a Bernoulli random variable with $𝔼\left[n_{t,l}^{(j)}\right]\in [0,1]$. While this problem is a quadratic program and can be solved using standard techniques, it is not readily decoupled in $t$, and thus standard solvers would not scale favorably in $T$.

Instead, we consider an alternative solution that admits a low-complexity recursive solution by relaxing the constraints. To this end, we relax the constraint ${\mathbf{𝐳}}_{t,l}-\alpha {\mathbf{𝐳}}_{t-1,l}⪯\mathrm{𝟏}$ and replace the constraint ${\mathbf{𝐳}}_{t,l}-\alpha {\mathbf{𝐳}}_{t-1,l}⪰\mathrm{}$ by penalty terms proportional to $\left|z_{t,l}^{(j)}-\alpha z_{t-1,l}^{(j)}\right|$. The resulting relaxed problem is thus given by: