Neural dynamics of causal inference in the macaque frontoparietal circuit

  1. Guangyao Qi
  2. Wen Fang  Is a corresponding author
  3. Shenghao Li
  4. Junru Li
  5. Liping Wang  Is a corresponding author
  1. Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, China
  2. University of Chinese Academy of Sciences, China

Abstract

Natural perception relies inherently on inferring causal structure in the environment. However, the neural mechanisms and functional circuits essential for representing and updating the hidden causal structure and corresponding sensory representations during multisensory processing are unknown. To address this, monkeys were trained to infer the probability of a potential common source from visual and proprioceptive signals based on their spatial disparity in a virtual reality system. The proprioceptive drift reported by monkeys demonstrated that they combined previous experience and current multisensory signals to estimate the hidden common source and subsequently updated the causal structure and sensory representation. Single-unit recordings in premotor and parietal cortices revealed that neural activity in the premotor cortex represents the core computation of causal inference, characterizing the estimation and update of the likelihood of integrating multiple sensory inputs at a trial-by-trial level. In response to signals from the premotor cortex, neural activity in the parietal cortex also represents the causal structure and further dynamically updates the sensory representation to maintain consistency with the causal inference structure. Thus, our results indicate how the premotor cortex integrates previous experience and sensory inputs to infer hidden variables and selectively updates sensory representations in the parietal cortex to support behavior. This dynamic loop of frontal-parietal interactions in the causal inference framework may provide the neural mechanism to answer long-standing questions regarding how neural circuits represent hidden structures for body awareness and agency.

Editor's evaluation

This study investigates the neural basis of the hidden causal structure between visual and proprioceptive signals in the primate premotor and parietal circuit during reaching tasks executed in a virtual reality environment, where information between the two modalities can be dissociated. The key novel result is that premotor neurons represent the integration of bimodal information for small disparities and the segregation for large disparities between the proprioceptive and visual information, while parietal cells show reaching tuning changes that support the updating sensory uncertainty between tasks.

https://doi.org/10.7554/eLife.76145.sa0

Introduction

The brain is constantly confronted with a myriad of sensory signals. Natural perception relies inherently on inferring the environment’s hidden causal structure (Deroy et al., 2016; French and DeAngelis, 2020; Lochmann and Deneve, 2011). For instance, in the ventriloquism illusion, when the audience is presented with a temporally synchronous but spatially discrepant audiovisual stimulus (e.g., a speech sound from the speaker and a visibly moving mouth of the puppet), they usually infer these audiovisual stimuli are coming from a common source and illusive perceive the speech coming from the puppet. In the process of building representation of the bodily self, the brain combines, in a near-optimal manner, information from multiple sensory inputs. When a single entity (e.g., the bodily self) evokes correlated noisy signals, our brain combines the information to infer the properties of this entity based on the quality and uncertainty of the sensory stimuli. As a result, behavioral performance often benefits from combining information using uncertainty-based weighting across sensory systems (Stein and Stanford, 2008). However, in a natural environment, multiple sensory cues are typically produced by more than one source (e.g., two entities), which should not be integrated in the brain, especially when the superposing cues are sufficiently dissimilar and uncorrelated. Instead, the brain’s inferential process of integration fades out, leading to the perception that these cues originate from distinct entities. This process of inferring the causes of sensory inputs for perception is known as causal inference (Körding et al., 2007).

Thus far, most of the neurobiological studies of multisensory processing have operated under the assumption that different streams of sensory information can arise from the same source. For example, previous neurophysiological research in monkeys showed that neurons implement reliability-weighted integration on the premise that visual and vestibular signals are from a common source (Fetsch et al., 2013; Morgan et al., 2008; Porter et al., 2007). Therefore, despite the ubiquity of the phenomenon of causal inference and many psychophysical and theoretical research (Acerbi et al., 2018; Dokka et al., 2019; Kayser and Shams, 2015; Körding et al., 2007; Mohl et al., 2020; Rohe and Noppeney, 2015; Sato et al., 2007), its neural mechanisms and functional circuits remain largely unknown. Recent neuroimaging studies have started to show the sequential causal inference process in the human brain (Aller and Noppeney, 2019; Cao et al., 2019; Rohe et al., 2019; Rohe and Noppeney, 2015; Rohe and Noppeney, 2016). However, little was known about the process at the single-neuron level in animals (Fang et al., 2019). In particular, the updating of prior and sensory information during causal inference has not been examined in animals.

In the present study, we established an objective and quantitative signature of causal inference at a single-trial level using a reaching task and a virtual reality system in macaque monkeys. We showed that monkeys combined previous experience and current multisensory signals to estimate the hidden common source and, more importantly, subsequently updated both the causal structure and sensory representation during the inference. We then further recorded from the premotor and parietal (area 5 and area 7) cortices of three monkeys to investigate the neural dynamics and functional circuits of causal inference in multisensory processing. Our behavioral and neural results reveal the neural computation that appears to mediate causal inference behavior, including inferring a hidden common source and updating prior and sensory representations at different hierarchies.

Results

Behavioral paradigm and hierarchical Bayesian causal inference model

Using a virtual-reality system, we trained three monkeys (monkeys H, N, and S) to reach for a visual target with their nonvisible (proprioceptive) arm while viewing a virtual arm moving in synchrony with a preset spatial visual-proprioceptive (VP) disparity (Figure 1A). On each trial of the experiment, the monkeys were required to initiate the trial by placing their hand on the starting position (blue dot) for 1 s and were instructed not to move. After the initiation period, the starting point disappeared, and the visual virtual arm was rotated; this mismatch arm was maintained for 0.5 s as the preparation period. The reaching target was presented as a ‘go’ signal, and monkeys had to reach toward the visual target within 2.5 s and place their hand in the target area for 0.5 s, referred to as the target-holding period, to receive a reward. Any arm movement during the target-holding period automatically terminated the trial. The proprioceptive drift due to the disparity between visual and proprioceptive inputs was measured at the endpoint of the reach and was defined as the angle difference between the proprioceptive arm and the visual target (the estimated arm) (Figure 1B, see details of animal training and reward in Materials and methods). In addition to this VP conflict (VPC) task, two control experiments were conducted: (i) where the visual and proprioceptive signals were perfectly aligned (VP task) and (ii) where there was only a proprioceptive signal (P task). The procedures of the three tasks (VPC, VP, and P) were identical, except that the visual or proprioceptive information presented to monkeys varied according to the context of the experiment (see Materials and methods). Using a block design, the order of three different blocks (tasks) in each training or recording session was randomized.

Figure 1 with 2 supplements see all
Behavioral task, the dynamic hierarchical causal inference model, and proprioceptive drift results.

(A) Overview of the behavioral task. The monkey was instructed to hold its proprioceptive arm over the starting position (blue dot) to initiate one trial. After the virtual visual arm rotation, a virtual red dot was presented, and the monkey was required to place its proprioceptive arm on the target and hold it to get a reward. (B) Schematic drawing of reward area, proprioceptive drift, and the different types of arms (proprioceptive and virtual/visual). Here, proprioceptive drift was defined as the rotated degree from the proprioceptive arm position to the estimated arm position (the same as the target location) measured from the shoulder. The reward area is defined by the green area, which ensures the monkey performed the task rationally and without visual feedback (see animal training in Materials and methods). (C–F) Schematic drawing of the dynamic hierarchical causal inference model. V-arm, visual arm signal; P-arm, proprioceptive arm signal; C=1: both V-arm and P-arm come from a common source; C=2: V-arm and P-arm come from different sources. (G) Example behavioral results from one session of one monkey (also see Figure 1—figure supplement 1 and Figure 1—figure supplement 2). CCW, counterclockwise; CW, clockwise. The black line represents raw data. The gray line represents the Bayesian causal inference (BCI) model fitting result. The black dots represent experimental trials and the gray dots represent simulated trials by the BCI model.

We used this paradigm to test the hypothesis of the causal inference process, which predicts how the brain infers and updates hidden structures on the basis of multiple sensory inputs.

First, the BCI model encodes probability distributions over the sensory (visual and proprioceptive) signals and incorporates rules that govern how a prior belief about the sensory causal structure is combined with incoming information to judge the event probability in proprioception (Figure 1C–F). Thus, the monkey’s behavior output (the proprioceptive drift distribution under each disparity) should show the dynamics of integration and segregation, which is the hallmark of causal inference. That is, the drift should increase for small disparities and decrease when the disparities become larger (Fang et al., 2019; Körding et al., 2007).

Second, according to the model, the prior of common source in the current trial should be modulated by the experience of the environmental structure. Thus, at a single trial level, the prior of a common source in the current trial should be updated based on the posterior of a common source from previous trials (Figure 1C and D).

Third, the sensory uncertainty is also proposed to update to maintain consistency with the prior beliefs of the causal structure of the world (French and DeAngelis, 2020). Therefore, the sensory uncertainty should increase when there is a conflict between the proprioceptive and visual signals (e.g., the VPC task) (Figure 1D and E).

To test these hypotheses, we adopted the Bayesian causal inference (BCI) model to assess monkeys’ behavior and investigated whether the neural activities in multiple brain regions correlate to proposed components in the behavior.

The probability of common source in monkey’s behavior

To examine whether the monkeys inferred the causal structure during multisensory processing, we first examined the proprioceptive drift as a function of disparity in the VPC task. Overall, the three monkeys showed a very consistent behavioral pattern, with the proprioceptive drift increasing for small levels of disparity and plateauing or even decreasing when the disparity became larger (e.g., exceeded 20°) (Figure 1G; for data on individual monkeys, see Figure 1—figure supplement 1). The BCI model qualitatively explains the nonlinear dependence of drift as a function of disparity. For small disparities, there is a high probability that the proprioceptive and visual signals came from the same source. Hence, the visual information is fully integrated with the proprioceptive information. For large disparities, however, the proprioceptive and visual signals are likely from different sources, leading to a breakdown of integration and consideration of only the proprioceptive information (segregation). In this case, visual information has a weaker weight for perception. Consequently, the effect of disparity on the drift is reduced by shifting integration to segregation. The BCI model quantified the nonlinear dependence between disparity and proprioceptive drift to measure the posterior probability of a common source (Pcom), the consequence of causal inference. We fitted the behavioral data using the BCI model. The results showed two signatures of the Pcom pattern: (i) the averaged Pcom decreased as the disparity increased (Figure 2A, left) and (ii) within each disparity, especially the large ones, the Pcom decreased as the proprioceptive drift decreased (Figure 2A, right) (see individual monkeys’ behavior in Figure 1—figure supplement 1).

Figure 2 with 2 supplements see all
The causal inference model predicts the dynamic updating of monkey behavior.

(A) Left: The average Pcom as a function of disparity. The black line represents the average Pcom across monkeys. The dashed lines represent the average Pcoms across sessions of three monkeys separately. Error bars indicate standard errors of the means (SEMs). Right: Model prediction of the Pcom. Each point represents the average Pcom in each cluster grouped by specific disparity and proprioceptive drift according to the clustering of disparity and drift (see Materials and methods). (B) The transition probability from the previous trial’s Pcom to the current trial’s Pprior. Left: The transition probability of an example session. Right: The transition probabilities across all sessions from three monkeys (Wilcoxon signed-rank test, W=6996.0, df = 242, p<0.001, rrb = 0.52). Each circle represents a behavior session. The most right insert scatter represents a single session’s transition probability. (C) After-trial effect of sensory updating. Left: The distribution of arm locations in P blocks after visual-proprioceptive (VP) and VP conflict (VPC) tasks in an example session. The solid lines represent fitted Gaussian distributions. Right: The standard deviations of drift in P blocks after VP and VPC tasks across all sessions from three monkeys in early trials (Wilcoxon signed-rank test, W=851.0, df = 72, p=0.012, false-discovery rate [FDR] corrected, see Materials and methods, rrb = 0.38) and in latter trials (W=1024.0, df = 72, p=0.073, FDR corrected, rrb = 0.24). The uncertainty of P trials after the VPC task in the early part of the session was significantly larger than that in the later part (W=917.0, df = 72, p=0.035, FDR corrected, rrb = 0.29); this is not the case for P trials after the VP task (W=1086.0, df = 72, p=0.15, FDR corrected, rrb = 0.20). (D) Within-trial effect of sensory updating. Left: The distribution of arm locations in VP and VPC (0°) tasks. The solid lines represent fitted Gaussian distributions. Right: The standard deviation of drift in VPC (0°) trials was significantly higher than that in VP trials (Wilcoxon signed-rank test, W=10,035.0, df = 237, p<0.001, rrb = 0.29). In (C) and (D), each circle represents a behavior session. The effect sizes (rrb) were performed using the rank-biserial correlation (Kerby, 2014). *p<0.05; ***p<0.001; n.s., not significant.

Pcom in the current trial depended on the experience

More importantly, the model posits that not only the inference of the causal structure is based on visual and proprioceptive inputs but also the subsequent updating of (i) the prior belief of causal structure based on the experience (e.g., probability of a common source in the previous trials) and (ii) the uncertainty of sensory signals for the visual and proprioceptive recalibration (Figure 1C–F). To test these hypotheses, we first implemented the Markov analysis of the prior belief and Pcom (see Materials and methods) to see whether the prior probability of a common source (Pprior) in the current trial depended on the previous Pcom (Figure 2B). The Markov model included the transition probability of Pprior between the current (nth) and previous (nth− 1) trial to account for the trial-by-trial variability in spatial drifts observed in the three monkeys (Figure 2B, left). The fit to the model demonstrated that the Pcom observed in the nth trial was significantly affected by that in the previous (nth− 1) trial (Figure 2B, right, Wilcoxon signed-rank test, p<0.001), indicating that the Pcom was computed based on both Pprior from the previous trial and the sensory inputs, with their disparity, from the current trial. Note that the transition probabilities (P(C=1|C=1) and P(C=1|C=2)) remained relatively high (larger than 0.8 in three monkeys) because overall, the number of high Pcom trials was much more than low Pcom trials in either training or recording sessions. This was consistent with high baseline Pprior in three monkeys (Supplementary file 1).

The common-source belief modulated the sensory uncertainty

We next examined whether the sensory representation is updated to maintain consistency with the causal structure of the environment. That is, the estimates of physical arm locations should tradeoff systematically depending on the current common-source belief (e.g., Pcom in different tasks: VP, P, and VPC). For example, when the monkey incorrectly infers that the visual and proprioceptive arms come from the same source when a disparity is presented, the uncertainty of proprioception should increase to ‘explain away’ the conflict between the two inputs. According to this idea, since the block design in the current experiment resulted in P trials (in the P task) sometimes following the VPC task and other times following the VP task, we then reasoned that because the overall Pcom was lower in the VPC task than in the VP task, the uncertainty of proprioception (i.e., the distribution of proprioceptive drifts in the P trials) would be larger after the VPC task than after the VP task. We analyzed the drift variation in P trials and found that, in the early trials (first third of each P block), the uncertainty of P trials following the VPC task was significantly larger than that following the VP task (Figure 2C, right, Wilcoxon signed-rank test, p=0.012). The increase in the uncertainty of proprioception was recovered in the late trials (last third of each P block), evident by a significant difference in the uncertainty between early and late P trials (Figure 2C, right, Wilcoxon signed-rank test, p=0.035). The decrease in the uncertainty of proprioception was reasonable, as the tradeoff effect in the VPC task gradually recovered.

Furthermore, we hypothesized that if a tradeoff of sensory representation occurs during the process of causal inference, the tradeoff would also affect the uncertainty of VP integration in both VP and VPC tasks. We examined the distribution of proprioceptive drifts using the trials with 0° disparity in the VPC task, in which the V and P information were congruent, and compared it with the distribution in the VP task. As predicted, we found that the variance of the proprioceptive drift was significantly larger in the VPC task than in the VP task (Figure 2D, right, Wilcoxon signed-rank test, p<0.001). Note that the difference between VP and VPC (0°) tasks could not be explained by the divergence of eye fixation positions (Figure 2—figure supplement 2). As a control, we also investigated whether the mean of drift, representing the perceptual accuracy of the proprioceptive arm, was affected by the causal structure of the environment. We found there was no significant difference between the mean of drift for P trials following the VPC task and that following the VP task in both early parts (Figure 2—figure supplement 1, left, Wilcoxon signed-rank test, p=0.37, false-discovery rate [FDR] corrected) and late parts (Figure 2—figure supplement 1, right, Wilcoxon signed-rank test, p=0.37, FDR corrected). Besides these, we also found that the mean of proprioceptive drift was not updated in the VPC task compared with the VP task (Figure 2—figure supplement 1, right, Wilcoxon signed-rank test, p=0.29). Thus, these results supported the notion of a tradeoff in proprioception according to causal inference environments; that is, sensory representation’s uncertainty, not accuracy, is updated dynamically based on the task environment (Pcom).

To summarize the above-described behavioral results, we found that monkeys’ proprioceptive drift shows a nonlinear dependency on the disparity between proprioceptive and visual input, which was well explained by the causal inference model. Second, we showed that the Pcom integrated with VP sensory inputs and is updated by previous experience on a trial-by-trial basis. Third, to maintain a consistency of causal inference, sensory uncertainty, reflected by the variance of proprioceptive drift, is updated in the inference along with the change of Pcom. Taken together, we established the behavioral paradigm in which monkeys infer the hidden cause by integrating prior information and sensory inputs while dynamically updating both Pcom and sensory representation. The behavioral responses of the monkeys enabled us to examine the underlying neural mechanisms and functional circuits.

Causal inference in individual premotor and parietal neurons

Previous studies showed that the premotor and parietal cortices were highly involved in body representation and multisensory perception (see reviews in Blanke, 2012; Graziano and Botvinick, 2002). In monkeys, bimodal neurons with visual and somatosensory receptive fields were found in both premotor (including F2vr in dorsal premotor and F4/F5 in ventral premotor) and posterior parietal cortices (including area 5 and area 7) (Fogassi et al., 1999; Graziano et al., 2000; Graziano and Gross, 1993; Graziano and Gross, 1998; Graziano et al., 1994). Specifically, ventral premotor neurons responded to visual stimuli in the space adjacent to the arm (Graziano and Gross, 1998; Graziano et al., 1994). The bimodal neurons in the parietal cortex (area 5 and area 7) showed to respond to both the real arm position and the seen position of a dummy arm (Graziano et al., 2000), which have a significant projection of the premotor cortex (Graziano and Gross, 1998). Consistently, human fMRI studies found that the posterior parietal and premotor (dorsal and ventral) cortices selectively respond to visual stimulation near the hand (Brozzoli et al., 2011) or the dummy hand near one’s corresponding hand (Blanke et al., 2015; Ehrsson et al., 2004). A human MEG study also revealed that the activities in the prefrontal and intraparietal sulcus were related to the causal inference computation in visual-auditory integration (Cao et al., 2019; Rohe et al., 2019). Therefore, we determined to record from two brain regions, the premotor cortex (dorsal and ventral, 412 neurons) and parietal cortex (area 5 and area 7; 238 neurons), in the three monkeys performing the reaching tasks (Figure 3A, for details, see Materials and methods). We first examined whether neurons in the premotor and parietal cortices during the target-holding period (Figure 3B) were selective to basic task components, including condition (VP or P), arm location, and visual disparity. In the premotor cortex, 40% (163/412) of neurons were selective to condition, 23% to arm location, and 37% to visual disparity (Figure 3—figure supplement 1A, upper panel). In the parietal cortex, 35% (83/238) of neurons were selective to condition, 27% to arm location, and 31% to visual disparity (Figure 3—figure supplement 1A, lower panel, ANOVA, main effect, p<0.05). We also examined the neural representations of the visual and proprioceptive arm locations in each trial during the target-holding period in the VPC, VP, and P tasks, measured by a bias-corrected percent explained variance (ωPEV) (Figure 3C). Both brain regions conveyed vital information about the arm location in the three tasks. In the VP and P tasks with no VP disparities, both premotor and parietal cortices showed similar visual and proprioceptive arm information (Figure 3C). However, when disparities were introduced in the VPC task, the premotor cortex showed a more robust signal for visual arm information (Figure 3C). In contrast, the parietal cortex showed stronger signals for information related to the proprioceptive arm (Figure 3C).

Figure 3 with 3 supplements see all
Casual inference neurons in premotor and parietal cortices.

(A) Recording sites. Left: Two regions of interest were recorded through single electrodes in macaque monkeys. Middle and right: Specific recording sites in three monkeys. AS, arcuate sulcus; CS, central sulcus; IPS, intraparietal sulcus; SPD, superior precentral dimple. L, left hemisphere; R, right hemisphere; A, anterior; P, posterior; M, medial. The straight dash gray line separated the dorsal and ventral part of the premotor cortex in the middle panel. The straight dash gray line indicates the middle of IPS and CS. The circular dash lines indicate the recording chambers. (B) Temporal structure of a single trial for the visual-proprioceptive conflict (VPC) task. (C) Neural information of arm locations in premotor and parietal cortices. Upper: No significant difference between the brain regions for the neural information of VP arm (Wilcoxon rank-sum test, W=0.64, dfpremotor = 411, dfparietal = 237, p=0.52, false-discovery rate [FDR] corrected, rrb = 0.030) and P arm (W=0.51, dfpremotor = 411, dfparietal = 237, p=0.52, FDR corrected, rrb = 0.031), respectively. Bottom: There were significant differences between the brain regions for both the neural information of proprioceptive arm (Wilcoxon rank-sum test, W=–3.92, dfpremotor = 411, dfparietal = 237, p<0.001, FDR corrected, rrb = 0.18) and visual arm (W=6.34, dfpremotor = 411, dfparietal = 237, p<0.001, FDR corrected, rrb = 0.30) in VPC task, respectively. Both brain regions conveyed significant information about the arm location in the three tasks (premotor: VP arm, Wilcoxon signed-rank test, W = 27,712.0, df = 474, p<0.001, FDR corrected, rrb = 0.35; P arm, W = 25,614.0, df = 411, p<0.001, FDR corrected, rrb = 0.40; proprioceptive arm (VPC), W=22,316.0, df = 411, p<0.001, FDR corrected, rrb = 0.48; visual arm (VPC), W=14,874.0, df = 411, p<0.001, FDR corrected, rrb = 0.65. Parietal: VP arm, W=9466.0, df = 237, p<0.001, FDR corrected, rrb = 0.33; P arm, W=7414.0, df = 237, p<0.001, FDR corrected, rrb = 0.48; proprioceptive arm (VPC), W=3745.0, df = 237, p<0.001, FDR corrected, rrb = 0.74; visual arm (VPC), W=10,138.0, df = 237, p<0.001, FDR corrected, rrb = 0.29). Each circle indicates a neuron. The effect sizes (rrb) were performed using the rank-biserial correlation. (D) Raster plots and mean firing rates from an example neuron in the parietal cortex that exhibited responses varied with visual disparity, showing the preference for the P task during the target-holding period (gray zones). The yellow curve was fitted with a von Mises distribution. (E) Schematic drawing of VP weight analysis (see Materials and methods) in one example trial for the VPC task. In brief, we first mapped the tuning curves of arm position in VP (left red curve) and P (left blue curve) tasks as integration and segregation templates, respectively. Then, during the VPC task, for a single trial, we mapped the visual and proprioceptive arm position onto the these templates to get the probabilities of integration and segregation. Then, we normalized the probability to get the VP weight. (F) Two examples of causal inference neurons in premotor and parietal cortices during the target-holding period (the same neurons shown in Figure 3—figure supplement 2 and (D), respectively). Each point represents one single trial, and the color represents the value of VP weight. The color bar represents VP weight, larger values indicate higher VP weights (higher probability of integration). (G) Population causal inference patterns in two brain regions. Each point was a pseudo-trial that was generated through bootstrapping, and the color represents the value of VP weight. (H) An example neuron in the parietal cortex shows the causal inference pattern defined by a significant positive correlation between VP weight and Pcom (Pearson correlation). Each point represents the average Pcom and VP weight in a cluster from the behavioral Pcom pattern. The solid line was fitted with linear regression, and the shaded area indicates the 95% confidence interval. The bar plot represents the fraction of causal inference neurons in the premotor cortex and parietal cortex. ***p<0.001.

Next, to define causal inference response in the VPC task at the single-neuron and single-trial levels, we utilized the VP and P tasks to characterize neural responses, as these tasks involve expected stereotypical behaviors in the two extreme regimes: full integration and segregation. Thus, neurons that are more active during the P task are likely candidates for ‘segregation (P) neurons’, which exhibited increased activity under the large disparities in the VPC task (Figure 3D). By contrast, neurons that are more active during the VP task reflect a preference for integrating congruent VP information and, hence, constitute a natural candidate for ‘integration (VP) neurons’ (example in Figure 3—figure supplement 2). We then implemented a linear probabilistic model which combined how the neural response pattern aligned with the VP and P response profiles and used this model to implement a probabilistic decoding analysis to calculate the probability of VP or P (VP weight = Pvp/[Pvp + Pp]) based on the firing rate in each trial (Figure 3E; also see Materials and methods). Thus, a larger VP weight for a single trial denotes a higher probability of integration (high Pcom). We first focused on the target-holding period in a trial, as the neurons could well display their spatial tunings when monkeys holding their arms on the target. We found that both premotor and parietal cortices carry information about Pcom at the single-neuron (Figure 3F; the same example neurons in Figure 3D) and population levels (Figure 3G; see Materials and methods) during the target-holding period. That is, the VP weight of the neuron or population progressively decreased along with the disparity, and in trials with large disparity (e.g., 35° and 45°), the neuron(s) had a higher VP weight when the drift was large (i.e., the monkey integrated the visual information; thus, a high Pcom predicted by the BCI model) and shifted gradually toward higher P weights when the drift shifted to 0 (i.e., the monkey segregated the visual information; thus, a low Pcom predicted by the BCI model). The VP weight was highly correlated with the Pcom from behavior (Figure 3H). Note that the premotor cortex had a slightly higher proportion of causal inference neurons (11.7%) than the parietal cortex (7.6%, Pearson chi-square test, χ2=2.33, p=0.063).

As neuronal activities in the premotor and the parietal cortices are reported to correlate with the eye position in the reaching task (Buneo and Andersen, 2006; Pesaran et al., 2006), one might ask whether the Pcom signals can be explained by the eye position. However, the result showed that the VP weights in the population could not be predicted by eye fixation positions during the target-holding period (Figure 3—figure supplement 3).

Population states encode Pcom during causal inference

We next focused on the overall populations of neurons in both regions and asked whether and how their population states reflect the uncertainty of causal structure, Pcom. We were guided by the results from single-neuron analyses during the target-holding period described above, in which neurons responsive to high Pcom (prefer integration) are more likely to show neural tuning similar to that during the VP task, and neurons responsive to low Pcom (prefer segregation) show a tuning profile similar to that in the P task. We thus hypothesized that neural components or subspaces embedded in the population activity represent the dynamic change in the coding of Pcom in the VPC task, which would lie between the components representing the VP and P profiles. Furthermore, the computation of Pcom in the BCI model is determined by the relation and disparities between the visual information from the artificial arm and proprioceptive information from the monkey’s actual arm. In other words, according to the model, the causal inference can be constructed before the visual target appears, and the participant uses this information to guide the reach. We thus further hypothesized that the dynamics of the population states also reflect the Pcom during the preparation period, during which there is no motor planning or preparation.

Thus, we grouped trials from each neuron into high and low Pcom classes according to the drift under each disparity (high, the top third of the trials [in red]; low, bottom third of the trials [in blue]) (Figure 4A). We conducted demixed principal component analysis (dPCA) to visualize any neural component that represents the Pcom in the VPC task in relation to that in the VP and P tasks (see Materials and methods). dPCA decomposes population activity into a set of dimensions that each explain the variance of one factor of the data (Kobak et al., 2016). We included the factors of time, arm location, and Pcom (Figure 4B). In the analysis, VP and P trials were included, which served as the templates of integration and segregation, respectively. As shown in the schema (Figure 4B), if the decomposed neural components indeed represent the Pcom, the population activity of high and low classes in this subspace should lie between that of the VP and P classes and the four classes (high, low, VP, and P) should be separated from each other. The dPCA results indicated that the Pcom components, unrelated to the arm location, represented 29.9% and 20.5% of the total firing rate variance in the premotor and parietal cortices, respectively (Figure 4C, in red). Notably, the activity in Pcom dimensions seems consistent with our hypothesis, demonstrating the dynamics of Pcom between integration (VP) and segregation (P). In addition, compared to the activity in the parietal cortex, the neural trajectories of the premotor populations showed an earlier divergence in Pcom dimensions (Figure 4D).

Figure 4 with 3 supplements see all
Dynamic population decoding of Pcom.

(A) Schematic drawing of the high Pcom group (top third of trials) and the low Pcom group (bottom third of trials) based on the relative drift (drift/disparity). (B) Schematic drawing of the demixed principal component analysis (dPCA). All trials of each neuron were grouped into 20 classes (5 targets × 4 conditions, including visual-proprioceptive (VP) and P tasks and high and low groups in the VP conflict [VPC] task). The marginalization matrix was generated by averaging all trials in each class. (C) dPCA decomposes population activity into a set of components given the task parameters of interest. (D) Temporal evolution of dPCA components of Pcom. The gray points represent the disparity onset; the black points represent the target onset. (E) Population decoding of Pcom. The decoding accuracy was plotted as a function of time. The gray shaded area represents the preparation period. The horizontal dashed black line represents the chance level. The horizontal solid-colored bars at the top represent the time of significant decoding accuracy (cluster-based permutation test, p<0.05). Shaded areas indicate 95% confidence intervals. (F) Joint peri-event canonical correlation (jPECC) results averaged across all sessions. Left: x-axis represents the time of parietal from target onset; y-axis: defines the time of premotor from target onset. The color bar represents the cross-validated correlation coefficient. Right: Lead-lag interactions as a function of time relative to target onset. The horizontal black bar represents the time of significant jPECC asymmetry index versus shuffled data (cluster-based permutation test, p<0.05).

To further quantify their dynamics statistically, we trained a linear support vector machine (SVM) using pooled activities in each brain region throughout the entire trial. The dynamic decoding results showed that the Pcom information is correctly predicted by neuronal population activities in both areas after target onset but is decoded only by premotor neurons during the preparation period when there was no visual target or motor preparation (Figure 4E, cluster-based permutation test, p<0.05). Randomization test confirmed the time difference that the Pcom information occurred significantly earlier in the premotor cortex than the parietal cortex (Figure 4—figure supplement 3, randomization test, p<0.01, see Materials and methods). This may suggest that the premotor cortex is where causal inference is computed and sends the information to the parietal cortex during the reaching period.

Next, we tested the relationship between the population activities in the two areas. We performed a joint peri-event canonical correlation (jPECC) analysis, which detects correlations in a ‘communication subspace’ between two brain regions (Steinmetz et al., 2019). In brief, we conducted a canonical correlation analysis for every pair of time points containing the population neural firing rates from the two regions. If the shared neural activity emerges at different times in the two areas, that is, activity in one region potentially leads to activity in the other, then we should observe a temporal offset between them. The jPECC results revealed a significant time lag for activity correlations between premotor and parietal areas in Pcom dimensions (Figure 4F, cluster-based permutation test, p<0.05), suggesting a potential feedback signal of Pcom from the premotor cortex to the parietal cortex. As a control, we performed the same procedure with misalignment trials (see Materials and methods) to exclude the probability that the observed time lag resulted from the intrinsic temporal property of neuronal activities in these regions. There was no significant time lag between premotor and parietal areas when the trials were misaligned (Figure 4—figure supplement 1).

Experience-dependent Pcom in the premotor cortex

The behavioral experiments showed that the Pcom could be updated by previous sensory experience on a trial-by-trial basis. To test the effect of the previous Pcom on the causal inference in each trial, we examined neural activities during the baseline period in the VPC task before a disparity in the visual and proprioceptive arm was introduced (Figure 5A). We again classified the trials according to high and low Pcom. Figure 5A depicts the results from an example premotor neuron, showing that during the baseline period, the neural activity exhibited selectivity toward the previous trial’s Pcom, and at the same time, its neural trajectories in high and low prior classes lay between the VP and P templates. Of 412 neurons in the premotor cortex, 29 (7.0%) showed such selectivity in the previous trial (Figure 5—figure supplement 1).

Figure 5 with 2 supplements see all
Premotor neurons encode prior information (previous trial’s Pcom) during the baseline period.

(A) Example neuron in the premotor cortex showing selectivity to prior information during the baseline period. The trials in the raster plot were sorted by the Pcom in the previous trial and grouped into high (red dots) and low (blue dots) groups. Bottom: temporal evolution of the average firing rate of ‘high prior’ and ‘low prior’ groups. The black horizontal line at the top represents the time window with a significant difference (two-sided t-test, t=2.36, p=0.019). Shaded areas indicate SEMs. (B) Dynamic population decoding of prior information (nth–1 trial). The gray shaded window represents the baseline period. The horizontal solid colored bar at the top represents the time with significant decoding accuracy with a cluster-based permutation test (p<0.05). Shaded areas indicate 95% confidence intervals. The horizontal dashed black line represents the chance level. (C) Decoding accuracy of prior trials (nth−1 to nth−4). Lag 0 represents the decoding of Pcom in the current (nth) trial. The horizontal dashed black line represents the chance level (permutation test, p<0.001). The solid lines were fitted with exponential functions. Error bars indicate 95% confidence intervals. (D) Schematic drawing of orthogonal subspaces of Pprior and Pcom. The solid-line circles represent Pcom and dotted circles represent Pprior. Red represents high Pcom, blue represents low Pcom. (E) Left: Percentage of baseline-period (Pprior) data variance (black bars, explained variance: about 99.63%) and target-holding period data variance (gray bars, explained variance: about 8.34%) explained by the top 10 prior PCs. Right: Percentage of baseline-period (Pprior) data variance (black bars, explained variance: about 11.30%) and target-holding (Pcom) period data variance (gray bars, explained variance: about 99.99%) explained by the top 10 Pcom PCs. (F) Premotor encoded prior information during the baseline period quickly decreased after the disparity onset while the Pcom information emerged. The orange line represents the population decoding accuracy of Pprior (nth–1 trial). The black line represents the population decoding accuracy of Pcom. The orange and black horizontal solid-colored bars at the top represent the time with significant decoding accuracy with a cluster-based permutation test (p<0.05) for prior information and Pcom information, respectively. *p<0.05.

To further test the relation between baseline neural activity and behavior quantitatively, we examined whether the population activities of these neurons can predict the Pcom from previous trials. We trained an SVM using pooled activities across recording sessions. The previous Pcom was only correctly decoded from the baseline activity in the premotor cortex (Figure 5B, cluster-based permutation test, p<0.05). Moreover, only recent experience (nth−1 trial) had a significant impact on the current trial (Figure 5C, permutation test, p<0.001).

As both Pprior and Pcom were represented in premotor neural activities, we wanted to examine their relationship in the neural states. We first found that very few neurons responded to both information types (see Figure 5—figure supplement 1). We then hypothesized that Pprior and Pcom might be represented independently at a population level. To validate this hypothesis, we conducted PCA on the population activities during baseline and target-holding periods for Pprior and Pcom, respectively. If they are independent, the subspaces of Pprior and Pcom will be near orthogonal, and the PCs of Pprior and Pcom will capture little variance from each other (Elsayed et al., 2016). To quantify this, we projected the Pprior data onto the Pcom subspace to calculate the percent variance explained by the Pcom PCs and repeated the same procedure for the Pcom data (Figure 5D). The results show that the top 10 Pprior PCs captured very little Pcom variance; similarly, the top 10 Pcom PCs captured very little Pprior variance (Figure 5E). These results support the hypothesis that the two information types are represented independently in the premotor cortex. However, such independence between Pcom and Pprior could also be caused by their different temporal structures in the task. Thus, we examined their neural dynamics within a trial. Figure 5F shows the time course of decoding results of prior and posterior information, where the Pprior quickly decreased after the disparity onset. At the same time, the Pcom information increased and was retained until the end of the trial. These results demonstrated the dynamics in the computation of causal inference, where the information from the last trial is only preserved transiently and then used to integrate with sensory inputs to generate Pcom information.

Update sensory uncertainty of arm location in the parietal cortex

Finally, we investigated the neural activities associated with updating sensory uncertainty. The behavior results revealed a significantly greater uncertainty of proprioception in VP trials in the VPC task (low belief of a common source) than in the VP task (high belief of a common source) (Figure 2D). We hypothesized that the sensory signals, which were used to make causal inference, in turn, updated their neuronal tunings to match inferred causal structure. We first examined the difference in neural tuning for arm location using the VP trials in the VP and VPC (VPC (0°), trials with no disparity) tasks. To test whether the tuning functions of arm location selective neurons changed between the VP condition and VPC (0°) condition at the single-neuron level, we fitted the tuning curve with the von Mises distribution by using the neuron response in different arm locations (five levels: [−30°, −20°, 0°, 20°, and 30°]) for these two conditions respectively (see Materials and methods). We found that the averaged firing rates during the target-holding period under the VP condition were higher than that under the VPC condition (0°) in the parietal cortex (Figure 6—figure supplement 1, left, Wilcoxon signed-rank test, p=0.017) but not in the premotor cortex (p=0.71). The gain index under VP condition were higher than the VPC condition (0°) in the parietal cortex (Figure 6—figure supplement 1, middle, Wilcoxon signed-rank test, p=0.0016, FDR corrected) but not the premotor cortex (p=0.11, FDR corrected). Figure 6A (right) shows an example neuron from the parietal cortex tuned to the center (0°) of arm location in the VP task, and the tuning range/uncertainty of the arm location was broader/lower in the VPC task. Here, for visualization purposes, we selected the time point when this neuron demonstrated the highest difference of ωPEV in the VP trials between VP and VPC tasks for the tuning calculation (Figure 6A, left, peak delta ωPEV). The averaged dynamic spatial selectivity of all neurons revealed a significant decrease of the total spike rate variance explained by the arm location in the parietal cortex but not in the premotor cortex (Figure 6B, cluster-based permutation test, p<0.05). Note that the updating of sensory uncertainty was not correlated with the uncertainty of eye position between VP and VPC (0°) tasks (Figure 2—figure supplement 2).

Figure 6 with 1 supplement see all
Representation of arm location is updated in the parietal cortex.

(A) Left: The difference of ωPEV between visual-proprioceptive (VP) and VP conflict (VPC) (0°) tasks for an example neuron in the parietal cortex. Right: Snapshot of the arm location tuning for VP and VPC (0°) tasks at the time point showed in the left panel (peak delta ωPEV). At the given time point of this neuron, there is no significant main effect for condition (two-way ANOVA, condition VP and VPC (0°)×hand location; condition, F(1,154)=1.450, p=0.23), but for hand location (F(4,154)=6.736, p<0.001). The solid curves were fitted with von Mises distributions. (B) Dynamic average ωPEV for VP and VPC (0°) tasks. The horizontal bar at the top represents the time bins in which the ωPEV for the VPC (0°) task was significantly lower than that for the VP task (cluster-based permutation test, p<0.05). (C) Dynamic population decoding of arm locations. The horizontal bar at the top represents the time bins in which the decoding accuracy for the VPC (0°) task was significantly lower than that for the VP task (cluster-based permutation test, p<0.05). Shaded areas indicate 95% confidence intervals. The horizontal dashed black lines represent the chance level.

Furthermore, at the population level, we performed the SVM decoding analysis of arm locations and found that only the parietal cortex showed a significantly decreased decoding accuracy in the VPC task (Figure 6C, cluster-based permutation test, p<0.05). We also confirmed that the change of decoding accuracy in the parietal cortex was significantly larger than the change in the premotor cortex (two-way ANOVA, Condition (VP and VPC (0°))×Region (premotor and parietal), significant interaction effect, p<0.05).

Discussion

Our data of behavior and multi-area neural recordings revealed, for the first time, the dynamic computation of causal inference in the frontal and parietal regions at single-neuron resolution during multisensory processing. Complementary to the previous findings focused on the feedforward sequential processing of BCI, the present results demonstrate parallel top-down processing of the hidden variable of Pcom from the premotor cortex, which monitors the weights of sensory combinations in the parietal cortex. By resolving the experience and causal belief, the hidden causal structure and sensory representation are dynamically updated in the premotor and parietal cortices, respectively.

In the last 15 years, the BCI model has been extended to account for a large number of perceptual and sensorimotor phenomena and a vast behavioral data (Shams and Beierholm, 2010). Recent studies have begun to map the algorithms and neural implementation in the human brain. Noninvasive human functional magnetic resonance imaging studies revealed a neural correlation to causal inference in the parietal cortex, and magnetoencephalography showed that frontal neural activities are also involved in the causal inference (Cao et al., 2019; Rohe et al., 2019; Rohe and Noppeney, 2015; Rohe and Noppeney, 2016). However, at the single-neuron level, very few studies have examined the neural mechanism in animals. More importantly, none of the human studies have investigated the neural representation of the hidden variable, Pcom. How the frontoparietal circuit contributes to the encoding and updating of Pcom has not been explored. Our results reconciled and extended previous findings by showing that Pcom is successively represented by premotor and parietal neural activities (Cao et al., 2019; Fang et al., 2019; Rohe et al., 2019). Unlike previous human imaging studies, which used the final behavioral estimation as the index of the causal inference (Cao et al., 2019; Rohe et al., 2019), our study directly examined the neural representation and dynamics of the hidden variable Pcom at single-neuron and neural population levels. We showed that, even within a trial, the inference of a common source was dynamic. We thus propose a dynamic flow of information processing during causal inference, where the Pcom is estimated from the information of sensory uncertainties and the disparity between them in the premotor cortex and then used for later sensory integration or segregation (model-weighted average) (Körding et al., 2007); finally, these signals are maintained in the frontoparietal circuit to guide the reaching behavior (Archambault et al., 2011; Caminiti et al., 2017; Cisek and Kalaska, 2005; Gail and Andersen, 2006).

Experience creates our prior beliefs of the surrounding environment. It was proposed that various cognitive functions, such as sensory perception, motor control, and working memory, can be modulated by experience (Akrami et al., 2018; Ernst and Banks, 2002; Rao et al., 2012). Computationally, the prior updating and its modulation of behavior can be well understood within the Bayesian framework (Badde et al., 2020; Beierholm et al., 2020; Körding and Wolpert, 2004; Rohe et al., 2019). For instance, by imposing the BCI model in the present study, we showed that prior knowledge of a common source is updated by the hidden probability of the common source (Pcom) in the previous trial and then integrated with the sensory inputs in a Bayesian manner. Such prior updating was also reported in a recent sensorimotor study, in which the posterior signals in the frontal cortex were used to update the prior (Darlington et al., 2018). Intriguingly, the empirical findings in this study could be reproduced by a biologically plausible recurrent neural network, which suggests that using the feedback of posterior from a Bayesian computation to update prior is an essential feature of a hierarchical recurrent Bayesian model (Darlington et al., 2018). From this perspective, the prior updating and its modulation of behavior may also serve as a plausible computational mechanism of multisensory recalibration in various sensorimotor behaviors (Badde et al., 2020; Bruns and Röder, 2015; Park and Kayser, 2019; Van der Burg et al., 2013).

The frontoparietal circuit, including the premotor and parietal cortices, has long been recognized as a central area in sensorimotor representations (Caminiti et al., 2017; Caminiti et al., 1991). Although the present experiments shared many movement features in the reaching task, the key findings of causal inference processing are unlikely to be explained by the kinematical components. First, previous studies have demonstrated that the neuronal activities in the premotor cortex are related to hand kinematics (e.g., hand position, speed, and direction) in the motor planning and execution (Caminiti et al., 1991; Churchland et al., 2006), which lead the neural activities in the parietal cortex (Archambault et al., 2011). However, in our study, the early activities of Pcom in the premotor cortex cannot be purely induced by the sequential activities of kinematics in the premotor and parietal cortices. Because the Pcom is abstract information, and its activity pattern is not correlated with any kinematical components. Expressly, under a given value of Pcom, the reaching kinematics can be varied (e.g., the hand position can be anywhere on the table according to the target position and disparity in a given trial). Moreover, the neural signals about Pcom in the premotor cortex were observed before the target onset, where no motor planning was possible during this period. Thus, our results are consistent with the idea that the high-level information, such as abstract and hidden structures, potential probability of multiple motor options, and VP integration, are encoded in the frontoparietal circuit, which could later integrate with the low-level sensory representations to guide the desired movement (Cisek and Kalaska, 2005; Gail and Andersen, 2006; Limanowski and Blankenburg, 2016).

Second, the dynamic updating of prior and sensory representation proposed a putative mechanism for multisensory recalibration in sensorimotor tasks. At the behavioral level, our results are in accord with the observations that sensory perception is modulated by a multisensory context with sensory conflicts. The BCI theory thus provides a framework to explain how the multisensory context (e.g., the prior of common source) modulates the sensory representations, such as sensory uncertainty in our study and sensory estimation (e.g., spatial localizations) in previous sensorimotor studies (Badde et al., 2020; Bruns and Röder, 2015; Park and Kayser, 2019; Van der Burg et al., 2013). The results support the notion of dynamic representations of Pcom in the present study – the top-down signal of common source from the premotor cortex modulates the spatial tuning in the parietal cortex and then guides hand estimation.

Previous research over the past two decades has revealed that even the perceptions of body ownership and agency are remarkably malleable and involve continuous processing of multisensory information and causal inference (Kilteni et al., 2015; Legaspi and Toyoizumi, 2019). Thus, our study provides unique data for understanding self-relative awareness (e.g., bodily self-consciousness) in macaque monkeys, showing neural implementation of causal inference at the neural circuit level. Using a VP task, we also identified the hidden components of causal inference in macaque monkeys’ parietal and premotor cortices. This is important because, unlike most sensory and cognitive functions, the subjective perceptions of body ownership and agency cannot be directly measured from explicit reports from animals. Using the BCI model and neural activities recorded from multiple brain areas, we can now begin exploring body ownership and agency qualitatively by examining the hidden variable in both behavior and neural representations.

In the BCI framework, there are two key components, inferring the hidden variables (e.g., Pcom) and updating the causal structure and sensory representation. First, our results suggested that the representation and core computation of the hidden common source most likely takes place in the premotor cortex (Ehrsson and Chancel, 2019; Fang et al., 2019), which is consistent with findings in the body awareness (Blanke et al., 2015; Ehrsson et al., 2004). Our results were also consistent with previous finding in monkeys that the higher order representations (e.g., the multisensory response of body recognition) of the body were encoded in both dorsal and ventral premotor cortex and posterior parietal cortex (Fogassi et al., 1999; Graziano et al., 2000; Graziano and Gross, 1993; Graziano and Gross, 1998; Graziano et al., 1994). Intriguingly, our results seem complementary to previous findings of mirror neuron systems in the premotor and parietal cortices in both humans and monkeys. Typically, a mirror neuron fires both when individual acts and when the individual observes the same action performed by another. That is, the mirror neuron is believed to mediate the understanding of others’ behavior (Jerjian et al., 2020; Jiang et al., 2020; Pezzulo et al., 2022). By contrast, the role of causal inference neurons in our study was putatively participating in self-identification and self-other discrimination. Future studies are needed to examine how these two systems work together to identify both self and foreign agents.

Second, the posterior belief of a common source is calculated using a Bayesian approach by integrating prior knowledge and sensory entities, and theoretically, these components should be dynamically updated at different time hierarchies. For example, the prior configuration of the body, known as the body schema in psychology, constrains the possible distribution of the body states but is dynamically updated when the context changes to maintain consistency between the internal body model and sensory inputs (e.g., rubber hand illusion or body illusion) (Botvinick and Cohen, 1998; Kilteni et al., 2015). Pathological impairment in inferring the sensory source can result in somatoparaphrenia, in which the patient declares that their body part belongs to another person despite the visual and proprioceptive signals from the common source of their body (Keromnes et al., 2019). Similarly, schizophrenia patients suffering from delusions of the agency have shown impairments in updating their internal causal structures. They show a deficit in detecting the source of their thoughts and actions and thus incorrectly attribute them to external agents (Haggard, 2017). Therefore, although we demonstrated the neural representations and their updating by using the multisensory and reaching task in monkeys, the computational mechanism and underlying neural circuits might contribute to learning and inference in any task that relies on causal inference.

Materials and methods

Experimental model and subject details

Request a detailed protocol

All animal procedures were approved by the Animal Care Committee of the Center for Excellence in Brain Science and Intelligence Technology, Institute of Neuroscience, Chinese Academy of Sciences (Permit Number: CEBSIT-2020034), and were described previously in detail (Fang et al., 2019). Three male adult rhesus monkeys (Macaca mulatta; Monkeys H, N, and S, weighting 6–10 kg) participated in the experiment. During the experiment, the monkeys were seated comfortably in the monkey chairs, and their heads were fixed. All monkeys were implanted with chambers for recordings.

Method details

Request a detailed protocol

Some of the following methods are similar to those previously published (Fang et al., 2019).

Apparatus

The monkeys were seated in front of a chest-height table on which a lab-made virtual reality system was placed (Fang et al., 2019). During the experiment, the monkey’s left arm (and the right arm in the case of Monkey H, who was right-handed) was placed in the system and blocked from sight. A CCD camera (MV-VEM120SC; Microvision Co., China) captured the image of the monkey’s arm reflected in a 45° mirror. This image was projected to the rear screen by a high-resolution projector (BenQ MX602, China). Therefore, when the monkey looked in the horizontal mirror suspended between the screen and the table, the visual arm image appeared to be its real arm on the table. The lower edge of the screen was aligned to the table edge. The monkey’s trunk was close to the edge of the table, and the left shoulder was aligned with the midline of the screen. Using the OpenCV graphics libraries in C++ (Visual Studio 2010; Microsoft Co., Redmond, WA, USA), the arms image and the visual target were generated and manipulated. Using CinePlex Behavioral Research Systems (Plexon Inc, Dallas, TX, USA), sampled at 80 Hz, the hand position was tracked and recorded. The tracking color marker was painted onto the monkey’s first segment of the middle finger, which was not visible after adjusting the light exposure settings of the video.

Behavioral task procedures

Request a detailed protocol

The monkey was trained to report its proprioceptive arm location by reaching for a target in a VPC causal inference task (Figure 1A; Fang et al., 2019). The monkey initiated a trial by placing its hand on the starting point (a blue dot with a 1.5 cm diameter) for 1000 ms and was instructed not to move. After the initiation period, the starting point disappeared, and the visual arm was rotated (within one video frame, 16.7 ms) for the VPC task. The rotation was maintained for 500 ms (the preparation period). After that, the reaching target was presented as a ‘go’ signal. The monkey had to reach the target (chosen from T1 to T5 randomly trial by trial [Figure 1A]) within 2500 ms and hold its hand in the target area (see as follows) for 500 ms to receive a drop of juice as the reward. Any arm movement during the target-holding period automatically terminated the trial. The rotated arm was maintained throughout the entire trial along with the arm movement. The intertrial interval (ITI) was ~1.5–2 s, after which the monkey was allowed to start the subsequent trial. During the ITI, the visual scene was blank. Under the VPC task, across trials, the visual arm was randomly presented with a disparity of 0°, ±10°,± 20°, ±35°, or ±45° (+, clockwise [CW]; −, counterclockwise [CCW] direction) from the subject’s proprioceptive arm, with its shoulder as the center point. The starting point was fixed 25 cm away from the monkey’s shoulder. The target position was selected randomly trial by trial from one of five possible positions located on an arc (a ±4° jitter was added to the original position trial by trial to ensure the monkey did not perform the task by memorizing all the target positions).

Besides the VPC task, the monkey was also instructed to perform a VP congruent and P task during the recording session. The only difference between the VPC and VP task was that during the entire trial under the VP task, the visual arm was always congruent with the proprioceptive arm. The only difference between VP and P tasks was that during the single trial for the P task, the visual arm information was blocked starting from the onset of the preparation period.

Each VPC block contained 55 trials in which the nine disparities and five targets were randomly combined. Each VP and P block had 27 trials in which five targets randomly occurred in every single trial. In one recording session, typically, one or two P blocks were given first to ensure that the monkey performed the task with its proprioceptive arm, and then in the following blocks, VP, P, and VPC tasks were randomly mixed. One recording session contained more than three VP and P blocks and more than eight VPC blocks.

Target (with reward) area

Request a detailed protocol

To ensure the monkeys indeed performed the reaching-to-target task with their proprioceptive hand, under the VPC task, the reaching target area (with reward) was defined as follows: the radial distance from the hand to the center of the target was less than 5 cm to ensure that the monkey did reach out to the target; with the target as the center, the azimuth range was set from [−7 (8 for some sessions, same below) + rotation degree/disparity] to +7° when the rotation degree was negative (counter-clockwise), and from –7° to [+7 + rotation degree/disparity] when the rotation degree was positive (clockwise). As shown in Figure 1B (green zone), the reward area ensured the monkey performed the task rationally and without visual feedback. That is monkey’s reaching position between two extreme conditions: one is that the monkey reaches the target purely relying on the visible arm (the drift is equal to the disparity); the other is that the monkey relies on the proprioceptive arm (the drift is equal to zero). Only the correct trials (when the monkey’s arm was located within the reward zone) were used in the subsequent analysis.

Electrophysiology

Request a detailed protocol

Extracellular single-unit recordings were performed as described previously (Fang et al., 2019; Merchant et al., 2013) from three hemispheres in three monkeys. Briefly, under strictly sterile tasks and general anesthesia with isoflurane, a cylindrical recording chamber (Crist Instrument Co., Inc, Hagerstown, MD, USA) of 22 mm diameter was implanted in the premotor cortex and the parietal cortex (area 5 and area 7). We collected the structural magnetic resonance images (MRI) of three monkeys (3T, Center for Excellence in Brain Science and Intelligence Technology, Institute of Neuroscience, Chinese Academy of Sciences), while they were in an MRI-compatible Horsley-Clarke stereotaxic apparatus. The location of the recording chamber on each animal was determined by the atlas with the origin at the Ear Bar Zero (Saleem and Logothetis, 2012). The centers of implanting recording chambers were [right: 20.0 mm; forward: 10.0 mm] for the premotor cortex in Monkey N, [left: 21.9 mm; forward: 24.9 mm] for the premotor cortex in Monkey H, [right: 14.7 mm; forward: 1.1 mm] for the parietal cortex in Monkey N, and, [left: 17.0 mm; forward: 3.5 mm] for the parietal cortex in Monkey S. During the recording session, glass-coated tungsten electrodes (1–2 MΩ; Alpha Omega, Israel) were inserted into the cortex via a guide tube using a multi-electrode driver (NAN electrode system; Plexon Inc, Dallas, TX, USA). All isolated neurons were recorded regardless of their activity during the task, with the recording locations varying from session to session. At each location, the raw extracellular membrane potential was sampled at 40 kHz. On-line raw neural signals were processed offline to obtain a single unit by Offline Sorter (Plexon Inc, Dallas, TX, USA). All spike data were re-sorted using off-line spike sorting clustering algorithms (Offline Sorter, PCA) (Merchant et al., 2013). With manual adjustments, only well-isolated units were considered for further analysis (signal-to-noise is larger than 3). The sorted files were then exported in MATLAB format for further analysis in MATLAB (Mathworks, Natick, MA, USA) and Python (The Python Software Foundation).

Quantification and statistical analysis

Request a detailed protocol

All statistical analyses were implemented with scripts written in MATLAB or Python. In the premotor cortex, 412 neurons were recorded from two monkeys (231 neurons from Monkey H and 181 neurons from Monkey N); in the parietal cortex (area 5 and area 7), 238 neurons were recorded from two monkeys (116 neurons from Monkey N and 122 neurons from Monkey S). As all monkeys’ behavior and model fitting results were similar, for all analyses, data were combined across monkeys. All related statistics are reported in the figure legends.

Analysis of behavior data

BCI model

Request a detailed protocol

To capture the uncertainty of causal structure, the core of causal inference, the BCI model described in a previous study (Fang et al., 2019) was adopted. In the present study, the BCI framework included three models: (i) the full-segregation model, which assumes that visual and proprioceptive estimates of the arm’s locations are drawn independently from different sources (C=2) and processed independently; (ii) the forced-fusion model, which assumes that visual and proprioceptive estimates of the arm’s locations are drawn from a common source (C=1) and integrated optimally, weighted by their reliabilities; and (iii) the BCI model, which computes the final proprioceptive estimate by averaging the spatial estimates under full-segregation and forced-fusion assumptions weighted by the posterior probabilities of a common source. Here, the BCI model assumes that both visual and proprioceptive location information (SV and SP) are represented as xV and xP in the neural system, respectively, which are drawn from the normal distribution with sensory noise [NSV,σV, NSP,σP ]. The causal inference structure is determined by the joint distribution of two sensory signals (sensory likelihood) and the prior probability of a common source (Pprior). Thus, according to the Bayesian rule, the posterior probability of a common source (Pcom) is calculated as follows:

p(C=1|xV,xP)=p(xV,xP|C=1)Ppriorp(xV,xP|C=1)Pprior+p(xV,xP|C=2)(1Pprior)

and the two sources of probability are pC=2xV,xP=1-pC=1xV,xP . Here, the likelihood of observed data (xV,xP) given common source [pxV,xPC=1 ] is calculated as follows Körding et al., 2007:

p(xV,xP|C=1)=exp[12(xVxP)2σPr2+(xVμPr)2σP2+(xPμPr)2σV2σV2σP2+σV2σPr2+σV2σP2]2πσV2σP2+σV2σPr2+σP2σPr2

where NμPr,σPr represents a prior distribution of arm locations. In this experiment, the μPr was set to 0 and σPr was set to 10,000 to approximate a uniform distribution.

If the system completely ‘believes’ the two sensory signals are from different sources (full-segregation situation), the proprioceptive arm position is estimated independently from the visual information, as follows:

S^P,C=2=xPσP2+μPrσPr21σP2+1σPr2

If the system completely ‘believes’ there is only a common source for the two sensory signals (forced-fusion situation), then the estimate of arm position is determined by the optimal integration rule, as follows:

S^VP,C=1=xVσV2+xPσP2+μPrσPr21σV2+1σP2+1σPr2

Here, we used the model average decision function to estimate final arm location (Fang et al., 2019):

S^P=p(C=1|xV,xP)S^VP,C=1+(1p(C=1|xV,xP)S^P,C=2)

In the model simulation, the proprioceptive arm position at the end of the trial was set to zero (SP=0), so that the visual arm position is the VP (SV=disparity). In the task, monkeys were required to report their proprioceptive arm position; thus, only the proprioceptive estimate was simulated.

Model fitting

Request a detailed protocol

To estimate the best-fitting model parameters in the BCI model, for each recording session, an optimization search was implemented that maximized the log-likelihood of each model given the monkey’s data under the VPC task. The prior probability of a common source (Pprior) and visual and proprioceptive standard deviations, σV and σP, respectively, were set as free parameters to be optimized. For each optimization step, 5000 trials per disparity were simulated to obtain the distribution, and the sum log-likelihood of the observations given the model was calculated for each disparity. Then, the parameters were optimized by minimizing the sum log-likelihood using a genetic algorithm (ga function in MATLAB). The procedure was the same as for the optimal integration model, except that there were no causal structures and only two free parameters (σV and σP) needed to be optimized. All simulation and optimization processes were performed in MATLAB. Only correct trials were included.

Model comparison

Request a detailed protocol

To determine the model that best explained the data at the group level using the Bayesian information criterion (BIC), a Bayesian random-effects model comparison was used (Rigoux et al., 2014). BIC=2LL+k×ln(n) , where LL denotes the log-likelihood, k is the number of free parameters, n is the total number of data points, and ln is the natural logarithm. The BIC is a criterion for model selection among a finite set of models; models with lower BIC are generally preferred. Finally, the better model was identified at the group level by the exceedance of the probability based on all sessions of monkeys’ BICs (Wozny et al., 2010). We used the exceedance probability to evaluate how likely it is that any given model is more frequent than all other models in the comparison set.

The models’ goodness-of-fit was reported using the coefficient of determination (R2) (Fang et al., 2019),

R2=1exp[2n{LL(β^)LL(0)}]

where LL(β^) and LL(0) denote the log-likelihoods of the fitted and the null model, respectively, and n is the number of observations. The null model assumes that monkeys report the perceived arm position randomly over the disparity range from the leftmost to the rightmost. Thus, a uniform distribution over this span was predicted.

Pprior updating in causal inference

Request a detailed protocol

To evaluate how the previous posterior probability of a common source (Pcom) influences the prior probability of a common source (Pprior), a Markov process was adopted to model the updating of Pprior. That is,

p(C=1)n=p(C=1|C=1)p(C=1|Data)n1+p(C=1|C=2)(1p(C=1|Data)n1)

where pC=1 and p(C=1|Data) denote Pprior and Pcom respectively, and n denotes the nth trial under the VPC task. Two prior states were included: C=1 (a common source) and C=2 (two different sources) at each trial. p(C=1|C=1) denotes the transition probability from a common source (C=1) to a common source (C=1), and p(C=1|C=2) denotes the transition probability from different sources (C=2) to a common source (C=1). For statistical significance analysis between p(C=1|C=1) and p(C=1|C=2) , the Wilcoxon signed-rank test was used for paired data.

Note both Pprior and Pcom are latent variables. During the model fitting, we first used the BCI model (as mentioned before) to search the overall Pprior, σP, and σV for each session/day, which were used as initial parameters in the subsequent Markov model. The σP and σV were fixed during the Markov model fitting. For all subsequent trials (except the first trial), both Pprior and Pcom are unknown. As time goes on, starting from the first trial, the Pcom of the current trial is obtained through the BCI model, and the Pprior of the next trial is obtained through the integration probability (Pcom) or separation probability (1 − Pcom) which are multiplied and added by the corresponding transition probability. Here, we fitted the observed data-drift to get the two free parameters transition probability. Through the transition probability, we define the influence of the Pcom of the previous trial on the Pprior of the next trial.

Updating of proprioceptive representation

Request a detailed protocol

To evaluate whether the primary sensory representation was modulated by the belief of causal structure, the proprioceptive variance within and after VPC tasks was compared to the baseline condition. For the within effect, the proprioceptive drift was calculated using the trials with 0° disparity in the VPC task and trials in the VP task (baseline condition). Here, the standard deviation (SD) of proprioceptive drift was used as a measurement for the uncertainty of proprioceptive representation, in which higher SD indicates higher uncertainty and vice versa. The mean of the proprioceptive drift for each target was normalized to zero. For the after-effect, the SDs of proprioceptive drift under the P task were compared between after the VP task and after the VPC task. To characterize the temporal dynamic of the proprioceptive updating (after-effect), trials in the first third and trials in the last third of the P task were compared. As a control, a similar analysis was conducted for the raw mean of proprioceptive drift (Figure 2—figure supplement 1). For statistical significance analysis, Wilcoxon signed-rank test was used for paired data.

Eye movement analysis

Request a detailed protocol

We trained the monkeys to perform the task without their eye fixed, but the eye movement during the recording sessions was recorded. To examine whether the updating of sensory uncertainty was correlated with the uncertainty of eye position between VP and VPC (0°) tasks. We identified the eye fixation position at the target-holding period. We examined the divergence of eye fixation position in VP and VPC (0°) tasks (see below). The average distance from the central point was used to measure the divergence at each target for each session. Each session’s divergence was obtained by averaging all the trials (see follows). The eye-tracking data were imported into MATLAB using EDF Converter (SR Research). The fixations and saccades in eye movements were separated with the default algorithm of the software with the velocity (30°/s), acceleration (8000°/s2), and motion thresholds (0.1°), respectively. The fixation positions were averaged during the target-holding period of each trial.

The normalized divergence of 2D eye fixation positions at each target was determined as follows:

Divergence=1ni=1nzic

where n is the sample size, c=c1,c2 is center of the eye fixation position, and zi={xi,yi} is ith eye fixation position. The divergence was averaged across different target positions for each session. zi-c indicates the Euclidean distance between c and zi . For statistical significance analysis, Wilcoxon signed-rank test was used for paired data.

Moreover, to examine whether the neural activity of Pcom was correlated with the eye position, we calculated the Pearson correlation coefficients between eye fixation position and VP weight. We found that there was no correlation between the VP weight and the eye fixation position at the population level for both regions (the premotor and parietal cortices) at both horizontal and vertical directions (Figure 3—figure supplement 3, Wilcoxon signed-rank test, premotor (horizontal): p=0.11; premotor (vertical): p=0.86; parietal (horizontal): p=0.35; parietal (vertical): p=0.87). Note that the recorded eye movement data used in this analysis included 78 sessions for the premotor cortex and 45 sessions for the parietal cortex.

Correction for FDR

Request a detailed protocol

In all cases, we used the Benjamini-Hochberg procedure (Benjamini and Hochberg, 1995) to control FDR at an α=0.05 level, as follows. The p-values of a given set of hypothesis tests were sorted in ascending order, {p1, p2, …, pn}, and we found the first rank iα such that piαiα×0.05/n . Then we considered tests to be significantly above chance (rejecting null hypotheses) for all p<piα .

Preprocessing of single-unit data

Request a detailed protocol

To estimate continuous time-dependent firing rates, timestamps of spiking events were resampled at 1 kHz and converted into binary spikes for single trials. Spike trains were then convolved with a symmetric Hann kernel (MATLAB, MathWorks),

convolvedw(n)=A(1cos(2πnN)),0nN(N=L1)

where A is a normalization factor ensuring the sum of the kernel values equals 1. Window width L was set to 300 ms. Single neurons were included in the analysis only if they had been recorded for a full set of tasks (VP, P, and VPC tasks with nine disparities: 0°, ±10°, ±20°, ±35°, and ±45°).

Peri-stimulus time histograms (PSTHs) were then calculated for four periods of interest in a trial: (i) the baseline period (500 ms before the onset of visual arm rotation), (ii) the preparation period (500 ms after the onset of the visual arm rotation), (iii) the target-onset period (1000 ms after the onset of target onset), and (iv) the target-holding period (500 ms after the onset of target-holding). To smooth the firing rate at each time point, the neural firing rate was calculated by averaging in sliding windows (window size, 400 ms; step size, 100 ms) in a single trial (Fried et al., 2011; Gu et al., 2016), resulting in 22 time bins of mean firing rate for every single trial for subsequent dynamic analysis.

Task selective neurons

Request a detailed protocol

To examine whether neurons in the premotor and parietal cortices during the target-holding period were selective to basic task components, including condition (VP or P task), arm location, and visual disparity. For each neuron, we conducted a two-way ANOVA in two datasets. One dataset contains the VP and P tasks (condition (two levels: VP and P tasks)×arm location (five levels: [–30°, –20°, 0°, 20°, and 30°]); the response variable is the mean firing rate during the holding period of each neuron). If a main effect of condition (or arm location) in the two-way ANOVA was found (p<0.05), this neuron was classified as a condition (or arm location) selective neuron. The other dataset is the VPC task (the visual disparity (nine levels: [–45°, –35°, –20°, –10°, 0°, 10°, 20°, 35°, 45°])×target position (five levels: [–30°, –20°, 0°, 20°, and 30°]); the response variable is the mean firing rate during the holding period of each neuron). If the main effect of visual disparity in the two-way ANOVA was found (p<0.05), this neuron was classified as a visual disparity selective neuron.

Tuning curve analysis of arm location selective neurons

Request a detailed protocol

To investigate whether the tuning functions of arm location selective neurons (Figure 3—figure supplement 1) changed between VP condition and VPC (0°) condition at single-neuron level, we fitted the tuning curve with a reduced von Mises function by using the neuron response in different arm location (five levels: [−30°, −20°, 0°, 20°, and 30°]) for these two conditions separately. Here, VPC (0°) condition represents the trials in VPC condition where the disparity equals to 0. And the fitting function was defined as:

fr(x)=b+acos(xμ)

where b is the spontaneous firing rate of the neuron, a is defined as the gain index, and μ is preferred arm location. frx represents the firing rate when the arm location is x. We analyzed the spontaneous firing rate of the neuron, gain index, and preferred arm location between VP condition and VPC (0°) condition in both premotor and parietal cortices (Figure 6—figure supplement 1).

Causal inference neuron

Request a detailed protocol

To measure the representation of a single neuron for causal inference on a single trial, the probability that a single neuron would integrate or segregate the sensory information on a single trial was calculated (Fang et al., 2019). The basic assumption here is that in a single trial under the VPC task, if the neuron is more inclined to represent integrated information, then its firing rate will be closer to its response under VP tasks and farther away from the response under P tasks, and vice versa. The normalized weight of integration (VP weight) was calculated as follows:

(1) First, obtain the neuron response to the arm position under P and VP tasks and fit the von Mises distribution to get the tuning curve.

(2) Under VPC tasks, obtain the current visual arm and the real arm positions, and at the same time, obtain the neuron’s firing rate when the arm is in the corresponding position under VP and P tasks, λVP , and λP, respectively.

(3) The VP and P templates can be generated through the Poisson distribution:

PrVP(X=k)=λVPkeλVPk!
PrP(X=k)=λPkeλPk!

(4) According to the corresponding probabilities, PrVP and PrP in the two templates are obtained, and the integration weights for this neuron in the VPC task can be obtained through standardization:

VPweight=PrVP(PrVP+PrP)

To quantitatively describe whether a single neuron is encoding causal inference, the correlation between Pcom and VP weight is calculated. The logic is as follows: the Pcom can be used to measure the degree of integration or segregation of sensory information at the behavioral level, whereas VP weight can measure this characteristic at the electrophysiological level. Therefore, if a neuron is performing causal inference, there should be a significant positive correlation between the Pcom and VP weight for the corresponding behavior. Neurons that (i) respond to VP/P tasks and (ii) for which Pcom and VP weight are significantly positively correlated in the final holding stage are called causal inference neurons. The specific algorithm was as follows:

  1. First, obtain neurons with significant selectivity under VP and P tasks (condition selective neuron, see Materials and methods: Task selective neurons).

  2. According to proprioception drift, all trials were divided into 29 classes. Continuous drift values were grouped into nine clusters: < –35°, [–35° –25°], [–25° –15°], [–15° –6°], [–6°+6°], [+6°+15°], [+15°+25°], [+25°+35°], >+35°. To be noticed, ±6° covers approximately 99% of drift distribution under the VP and P task. Thus, for the disparity of 0°, there was only one cluster [–6°+6°]. Since the distribution of drift becomes wider (higher variance) the larger the disparity, the more clusters would be assigned for the big disparity. For example, for the disparity ±45°, there were five clusters of drifts. Pcom and VP weight were assigned for each class by averaging all trials within it. The Pearson correlation coefficient was then calculated between Pcom and VP weight. If the Pcom and VP weight were correlated significantly and positively (p<0.05 and r>0), the neuron was called a causal inference neuron.

Population pattern of causal inference

Request a detailed protocol

To visualize the VP weight pattern at the brain region level, the VP weight of each trial of a single neuron under VPC tasks was calculated and then divided into 29 clusters as described above. Then, the bootstrap method was used to randomly select 50 trials from each cluster for averaging. This was repeated 50 times to obtain the VP weight (50×29) of a neuron for visualization. This results in a 50 × 29 × N matrix, where N indicates the number of neurons in each brain region (all neurons were used). The trial corresponding to each neuron was averaged to obtain a 50×29 matrix. The VP weights of a brain region were visualized in a heatmap.

High/low Pcom groups

Request a detailed protocol

To characterize the dynamic representation of the Pcom in the entire session, all trials in a recording session were divided into high Pcom trials and low Pcom trials based on the relative proprioception drift (RD). Each trial’s relative proprioception drift (RD = drift/disparity) was calculated. The basic idea was that the larger the Pcom, the more likely the monkey would integrate the visual and proprioceptive information, and the corresponding RD is closer to 1. The top third and bottom third of the trials were designated the high Pcom class and the low Pcom class, respectively. These grouping methods were verified by the dPCA.

dPCA

Request a detailed protocol

The method for dPCA was adopted from that published in a previous study (Kobak et al., 2016). Time, target position/arm location (−30°, −20°, 0°, 20°, and 30°), and Pcom (VP, P, high Pcom, and low Pcom) were combined to obtain the marginalized covariance matrix of the three. The neurons whose trial number was not less than five under a single condition were selected for dPCA. Population activity was then projected on the decoding axes and ordered by their explained total variance for each marginalization.

Information encoded by individual neurons

Request a detailed protocol

The percentage of explained variance (PEV) (Buschman et al., 2011) was used to measure the basic task components encoded by a single neuron, in which PEV reflected the degree to which the variance of a single neuron can be explained for a specific task component. Generally, PEV can be expressed as a statistical value of η2 , that is, the variance ratio between groups to the total variance. As the statistical value of η2 has a strong positive bias for a small sample, the unbiased ω2 statistical value (ωPEV) (Olejnik and Algina, 2003) was used.

To evaluate the information about the locations of the proprioceptive arm, visual arm, and estimated arm encoded by a single neuron in the VPC task, an analysis of covariance was used to decompose the variance, and the ωPEV was calculated. In detail, for a single neuron, ωPEV was calculated for each type of arm when setting the other two types of arm locations as covariates. The whole reaching space was divided into 11 parts from −45° to 45° to transform it from a continuous variable to a discrete variable. A nonparametric Wilcoxon rank-sum test was used for unpaired data for statistical significance analysis comparing two brain regions.

The ωPEV was calculated in each time bin to characterize the temporal dynamics of neural information under VP and VPC (0°) tasks. The baseline was defined as the period 500 ms before the onset of visual arm rotation. A one-sided, paired Wilcoxon signed-rank with FDR correction determined the time bins significantly different from the baseline. The time bins showing significant differences between VP and VPC (0°) tasks were determined by a cluster-based permutation test (Gramfort et al., 2013).

Population decoding analysis

Decoding of Pcom

Request a detailed protocol

The population decoding analysis of Pcom was performed by the linear SVM classifiers with the scikit-learn toolbox (Pedregosa et al., 2011). All neurons were included in this analysis without considering their Pcom selectivity. The classifier was trained to classify the Pcom (high/low Pcom) with neural activity (PSTHs) from each brain region. All recording sessions were pooled to form a pseudo-population. Neurons with more than 50 trials in each Pcom group were included in this analysis. Tenfold cross-validation was then implemented by splitting the neural data into 10 subsamples, each randomly drawn from the entire dataset. Decoders were then trained on nine of the subsamples and tested on the remaining one. This process was repeated 10 times to obtain the decoding accuracy by averaging across all 10 decoders. This cross-validation process was repeated 1000 times, and the overall decoding accuracy was taken as the mean across the 1000 repetitions. The decoding analysis was conducted for all time points. The significance of decoding accuracy was determined by comparing the mean decoding accuracy to the null distribution from the shuffled data. The significant time duration was determined using a cluster-based permutation test for multiple comparisons across time intervals (permutations = 5000; cluster-level statistic: sum of the t values in a cluster; auxiliary cluster defining threshold t=3) (Gramfort et al., 2013). For visualization, we plotted the mean of decoding accuracy with 95% confidence interval using 50 repetitions.

To test whether the premotor cortex neurons encode Pcom earlier than parietal cortex, a randomization test was performed between them. Neurons with more than 50 trials in each Pcom group were included in this analysis. The corresponding numbers (here, 200 neurons per region) of neurons were randomly exchanged between the paired regions 1000 times to generate a null distribution (chance level) of time lags, and the significance was determined by a permutation test of the true time lag from the original data and the null distribution (Panichello and Buschman, 2021).

Decoding of Pprior

Request a detailed protocol

Neurons with more than 50 trials in each Pcom group (high and low Pcom groups, same as for the Pcom decoding analysis described above) were selected for the Pprior updating decoding. The decoding procedure was the same as described for ‘Decoding of Pcom ’ unless the trials were sorted and labeled by the previous trial’s Pcom (nth−1 to nth−4) under the VPC task. The statistical significance was determined by a cluster-based permutation test (Gramfort et al., 2013).

Subspace overlap analysis

Request a detailed protocol

PCA was performed on neural activities during the baseline period and during the target-holding period. The first 10 principal components (PCs) during each period were used to obtain the Pprior and Pcom subspaces. To test the overlap of these subspaces, the baseline-period activity was projected onto the Pprior subspace, and the percent variance explained relative to the total variance of the baseline period data was quantified; similarly, the target-holding period activity was projected onto the Pcom subspace, and the percent variance explained relative to the total variance of the target-holding period data was quantified (Elsayed et al., 2016).

Decoding of arm locations

Request a detailed protocol

All arm locations were separated into five spatial bins: −30°, −20°, 0°, 20°, and 30°. The basic decoding procedure was the same as described above for ‘Decoding of Pcom’. Neurons with more than six trials in each arm location bin were selected. Leave-one-out cross-validation was then implemented, and this process was repeated 1000 times to obtain the averaged decoding accuracy. The decoding analysis was conducted for all time points. Statistical significance for decoding accuracy was determined by comparing the mean decoding accuracy to the null distribution from shuffled data. The time bins with significant differences between tasks (VP and VPC (0°)) were determined by the cluster-based permutation test for multiple comparisons across time intervals (Gramfort et al., 2013).

jPECC analysis

Request a detailed protocol

To test the relationship between population activities in the two brain regions, the jPECC method described in a previous study (Steinmetz et al., 2019) was utilized. First, the neuronal responses in two brain regions under the same behavior conditions, namely, high Pcom and low Pcom, were aligned. Then, a PCA was conducted across time and trials to reduce the dimensionality to obtain the first 10 PCs for each brain region. The trials were then divided into 10 equal parts (training set and testing set) for cross-validation (10-fold cross-validation). The PCs of the training set of each brain region were used to perform canonical correlation analysis to obtain the first pair of canonical correlation components (L2 regularization, λ=0.5). Then, the PCs of the testing set from each brain region were projected onto the first pair of canonical correlation components, and the correlation was determined by the Pearson correlation coefficient between these projections from each region. This analysis was performed for each pair of time bins to construct a cross-validated correlation coefficient matrix. Fifty trials for each group (high Pcom and low Pcom) from each brain region were randomly selected by bootstrapping in this analysis. Finally, a heatmap was obtained by averaging the correlation coefficient matrix repeated 1000 times.

To quantify the lead-lag relationship of information exchange between brain regions, an asymmetric index was calculated by diagonally slicing the jPECC matrix from +300 ms to +300 ms relative to each time point (Steinmetz et al., 2019). For time point t, the average correlation coefficient across the left half of this slice (i.e., the average along a vector from [t − 300, t + 300] to [t, t]) was subtracted from the right half of this slice (from [t, t] to [t + 300, t − 300]) to yield the asymmetry index. To test the leading significant time point across brain regions, the data from neurons in these brain regions were exchanged, and the above-described analysis was repeated 1000 times to obtain the null distribution of the asymmetric index. Then, a cluster-based permutation test was performed to test whether the symmetric index was significantly greater than the chance level (Gramfort et al., 2013).

To further exclude the possibility that the observed lead-lag relationship resulted from the intrinsic properties of neuronal activities rather than the encoded information in these regions, all trials in each brain region were shuffled to ensure that the inter-region trials were not aligned. Then, the analysis was repeated as described above to obtain the asymmetric index.

Note that, due to the limitations of the asynchronous recording (the premotor and parietal neurons were grouped from different individual animals and only Monkey N was recorded in both areas), further studies are required to clarify the dynamics and functional interactions between regions using a simultaneous recording.

Resource availability

Lead contact

Request a detailed protocol

Further information and requests for resources should be directed to and will be fulfilled by the Lead Contact, Liping Wang (liping.wang@ion.ac.cn).

Materials availability

Request a detailed protocol

This study did not generate new unique reagents.

Data availability

Source data files have been provided for Figures 1–6. Code and dataset have been uploaded to Dryad (https://doi.org/10.5061/dryad.rr4xgxd9h).

The following data sets were generated
    1. Qi G
    2. Fang W
    3. Li S
    4. Li J
    5. Wang L
    (2022) Dryad Digital Repository
    Code and dataset for neural dynamics of causal inference in the macaque frontoparietal circuit.
    https://doi.org/10.5061/dryad.rr4xgxd9h

References

    1. Graziano MSA
    2. Botvinick MM
    (2002)
    How the brain represents the body: insights from neurophysiology and psychology
    Common Mechanisms in Perception and Action 19:136–157.
    1. Pedregosa F
    2. Varoquaux G
    3. Gramfort A
    4. Michel V
    5. Thirion B
    6. Grisel O
    7. Blondel M
    8. Prettenhofer P
    9. Weiss R
    10. Dubourg V
    11. Vanderplas J
    12. Passos A
    13. Cournapeau D
    14. Brucher M
    15. Perrot M
    16. Duchesnay E
    (2011)
    Scikit-learn: machine learning in python
    Journal of Machine Learning Research 12:2825–2830.
  1. Book
    1. Saleem KS
    2. Logothetis NK
    (2012)
    A Combined MRI and Histology Atlas of the Rhesus Monkey Brain in Stereotaxic Coordinates
    Academic Press.

Decision letter

  1. Hugo Merchant
    Reviewing Editor; National Autonomous University of Mexico, Mexico
  2. Tirin Moore
    Senior Editor; Stanford University, United States
  3. Hugo Merchant
    Reviewer; National Autonomous University of Mexico, Mexico

Our editorial process produces two outputs: (i) public reviews designed to be posted alongside the preprint for the benefit of readers; (ii) feedback on the manuscript for the authors, including requests for revisions, shown below. We also include an acceptance summary that explains what the editors found interesting or important about the work.

Decision letter after peer review:

Thank you for submitting your article "Neural dynamics of causal inference in the macaque frontoparietal circuit" for consideration by eLife. Your article has been reviewed by 3 peer reviewers, including Hugo Merchant as the Reviewing Editor and Reviewer #1, and the evaluation has been overseen by Tirin Moore as the Senior Editor. The following individuals involved in review of your submission have agreed to reveal their identity: Hugo Merchant (Reviewer #1).

The reviewers have discussed their reviews with one another, and the Reviewing Editor has drafted this to help you prepare a revised submission.

Essential revisions:

The present results are novel and provide important notions. However, we have a series of important concerns.

1) First, the paper is difficult to follow. The authors should use a simpler and more intuitive framing of the paper to make it more accessible to a general audience. A reader needs to integrate a large part of the results to really have an idea of what is the paper about and how the authors tested their hypothesis. Indeed, the writing in part uses words or phrases that are 'unusual' or imprecise, which makes the text difficult to understand.

2) Another important concern is that the paper does not show the basic response properties of neurons on both areas across tasks. It is not clear how the neural activity changes between the VP, P, and VPC tasks. It is a change in preferred direction, in the width of the tuning function or on the gain modulation? We strongly suggest providing two 'layers' of information, one based on standards plotting and analysis, to be followed by a second layer of data investigation, addressed to more detailed computational aspects.

3) Statistics:

Results of model fitting: The methods mention that the authors compared three models: segregation, fusion and causal inference. However, in the results (Table S1) only two models are presented. The results for segregation should be reported for the sake of completeness. The Ppriori is high, e.g. 0.98 and 0.999. This makes me wonder in how far the R2 and the EPs' can be so much different between causal inference and fusion. Considering e.g. model averaging decision function for BIC, a prior of 0.999 would make fusion and BCI nearly identical. I feel that either the Methods section is lacking important details about the models (See above) or there is either a mistake in the analysis or the reported numbers in the table. The analysis of Bayesian models seems to lack major details, the statistical reporting is below standard (missing effect sizes, degrees of freedom, lack of individual data in figures), the study shows many unjustified parameter choices and key results seem to lack statistical support: not all statements about differences between parietal and premotor cortex seem supported by a direct statistical comparison. Further, while three monkeys contributed data, for only one does the study report data from both brain regions; this makes the claim of a difference between brain regions rather weak and this shortcoming needs to be clearly acknowledged. The actual underlying data (e.g. how single neuron responses are converted to tuning curves; how decoding accuracies vary across neurons) is not shown, which makes it difficult to interpret the robustness of the results. In particular, as the units of analyses vary tremendously between Figures (experimental blocks, neurons, pseudo-epochs, etc).

The updating of the believe about the sensory causal structure is a central component of this work. The authors present this as well established aspect of the BCI model (l. 170ff). However, most previous studies used a static model that was fit to the aggregate data of an entire experiment without taking the trial history into account (as in the cited Koerding et al. 2007 paper). AS some recent work has incorporated such trial dependencies, it would be important to acknowledge these studies and to explain the novelty of the present work (e.g. Rohe et al. NatComm 2019; Beierholm et al. eLife 2020; Badde et al. Cognition 2020 may also be relevant).

Analysis of recalibration: The analysis of P trials after the VP or VPC task effectively looks at what is known as trial-wise multisensory recalibration (see e.g. Bruns et al. Scientific Reports 2015; Park and Kayser eLife 2019; van der Burg et al. J Neurosci 2013; Wozny et al. JNeurosci 2011; Badde et al. Cognition 2020; there is extensive literature on this both in the spatial and the temporal domain!). It seems awkward to investigate this recalibration of uniusensory judgements without alluding to previous work.

If I understood Figure 3A and the Methods correctly, only in monkey N both brain regions were recorded? If this is correct, the statement that premotor and parietal regions differ in their representations is a result of a mixed within and between subjects analysis. This should be acknowledged explicitly, as it greatly reduces the statistical power of this statement, and the repeated statement about an N of 3 is misleading. CI neurons are defined based on a seemingly arbitrary criterion: exceeding a correlation threshold based on the arbitrary division of the data into 26 bins. Given that neural representations generally span a continuum, I would like to see the distribution of 'causal inference effects' for individual neurons, e.g. in form of a distribution of r2 values (obtained from the correlation; or a regression as I suggest below). The apparent difference between brain regions (Figure 3G) may simply result from the specific choice of statistical cutoff (the criterion of p<0.05 becomes meaningless in the presence of 475+238 tests in total). Seeing the individual-neuron data here seems vital.

The analysis of population timing suggests that premotor cortex leads (Figure 4F). Is it possible to extract by how much time? Also, the authors focus on the encoding of Pcom, which comprises both the a priori binding tendency and the discrepancy. Why did the authors not decode both the prior and the current multisensory discrepancy separately? This would seem important to differentiate neural signatures of priors from those of current sensory signals.

Showing the actual data: The key results (e.g. Figure 3C; 4F; 5B; 5F; 6C) would be much stronger rand more convincing if the actual units of analysis were shown in some ways. How does decoding accuracy vary across neurons?

Other details:

For most tests there are no measures of effect sizes reported, sometimes the respective test-statistics is missing, and the degrees of freedom remain very unclear. I understand that they wary between analysis, but given that some tests are based on the actually recorded units, some of pseudo-trials or binned data, it would be very important to report for each test the assumed independent units and their number. The false-discovery rate is mentioned frequently, but the precise method is not stated. Most analyses are based on Wilcoxon tests, but figures show mean and SDs. I encourage the authors to use the same nonparametric (or parametric) approaches for figures and stats (e.g. show boxplots and individual data). L. 892: what was precisely compared with the ANOVA? Cluster-based tests: the parameters and the procedures for this test are not reported (l.974)

To determine whether a neuron confirms to the expectation of causal inference, why is it necessary to bin the data (l 893ff)? Could one not simply derive a regression model for each neuron and visualize the R2 or F-value?

The authors seem to interpret differences in the significances (e.g. of cluster-based permutation tests) as significant differences between regions and as establishing differences in the relative timing of effects. These are statistical fallacies (e.g. Sassenhagen https://doi.org/10.1111/psyp.13335; and Makin https://doi.org/10.7554/eLife.48175).

For every statement claiming differences between parietal and premotor cortex it is necessary to directly impellent the respective contrast between neurons in each brain region to support such a difference.

Other methods:

Spike sorting: I could not find criteria used for spike sorting. Where the analyzed units single units or MUA? More details about spike thresholds, cluster separation etc. should be provided.

The total number of switches between blocks (e.g. P following VPC) should be reported, as this constitutes the effective degrees of analysis of the block switching analysis (Figure 2C).

Causal inference models and optimization: The methods leave it unclear how the two alternatives of common and separate sources were combined in the BCI model. Previous work has explored a number of decision functions (e.g. Rohe and Noppeney's work, or Wozny et al. PlosCompBiol 2010) but for the present study it remains unclear which decision function was used. Model fitting: how were likelihoods computed and the posteriors sampled for model fitting? I feel that the procedures are not described in sufficient detail to be reproduced. Over what range of disparities was the model optimized? This is important for the Null model mentioned later on. What is the precise number of data points that entered the BIC calculation?

Markov analysis: If I understood it correctly, SigmaA and SigmaV are fit to the entire block, and the Pprior derived from the entire block was used as starting value for this parameter? The authors conclude (l 256ff) that to 'maintain a consistency of causal inference, sensory uncertainty … is updated ' as well. However, the Markov model seems to focus only on the updating of the prior.

Processing of single unit data: In my view the paper would profit from showing actual single neuron PSTH's and how smoothing effected these. The methods (l. 857) mention a 400ms sliding window, but the periods of interest (e.g. target holding) are only minimally longer than this (500ms). This makes we worried that the analyzed data effectively blurs neural representations across epochs and is affected by movement artifacts. When computing the modality contributions to each response, what task epoch was analyzed to derive the tuning curves (l. 869ff)?

Overall there are many seemingly arbitrary choices in the methods. These include the thresholds to define neurons as 'causal inference', the number of trials required for neurons to be included in the population analysis (l. 964), the duration of smoothing kernels and temporal analysis windows (l. 848 ff), the binning of data for neuro-behavioral correlation (l. 893ff), in the generation of population patterns (l. 912ff), in the cluster-based test (not reported!). It would be good to see a justification for these choices or to learn whether the authors ensured that their main results do not depend on these precise choices.

4) The authors do not justify why they recorded in the transition between F4 (rostral ventral premotor) and F5 (caudal ventral premotor) with head visual/tactile optic flow signals and grasping signals respectively. The obvious target is F2 (dorsal premotor) since it has strong reaching signals and is highly connected with area 5 of the parietal lobe (Rizzolati, 1990; Mendoza and Merchant, 2014). The authors should provide a more detailed account on the areas studied and the criteria adopted to localize the recording sites. The specification that they were "determined by individual MRI atlas" does not warrant for areal identification, because on natural variability. On this regard, the Figure 3A' insets should be adjusted (for parietal recording sites in L and R hemispheres, the sulci orientation should be different, and for the premotor ones sulci should be reported).

5) For the discussion and comparison to previous work: The paradigm focuses on visuo-motor paradigm in which the sensory cues are both generated by the subject itself. In contrast, in many classical (e.g. audio-visual; or visual-vestibular)) paradigms both sensory cues are external in nature, and not linked to the subject's action. While in both types of paradigm sensory cues are integrated and can also induce perceptual recalibration, the visuo-motor paradigm still is conceptually distinct and this has implications for the interpretation of the results. The authors should discuss whether they believe that their findings generalize to other paradigms and whether the same or possibly distinct (e.g. parietal) brain regions should be investigated during such paradigms. Such a discussion seems important to place the present work in the context of the plethora of previous work. Indeed, the present study is completely lacking discussion of results with respect to current knowledge on the functional properties of premotor and parietal neurons subtending reaching. The literature on this topic is vast, but the following studies, as examples, could be relevant in this context:

1. Archambault et al. J Neurosci 2011 (comparison on premotor vs parietal, where premotor activity leads parietal one)

2. Caminiti et al. eNeuro, 2017 (overall picture of connectivity of fronto-parietal network with updated literature on functional properties of different areas)

3. Caminiti et al. J Neurosci 1991 (first paper on encoding of reaching in Premotor cortex)

4. Churchland MM, et al. Nature 2012

5. Cisek and Kalaska, Neuron 2005 (on premotor activity during reaching)

6. Gail and Andersen J Neurosci 2006 (on neural dynamics of sensorimotor transformations in parietal cortex)

7. Jerjian SJ, Sahani M, Kraskov A ELife 2020 (on movement representation in premotor cortex)

8. Jiang X et al. Cell Rep 2020 (onpremotor neural Activity during Observed and Executed Movements)

9. Mountcastle et al. J Neurophysiol 1975 (first pioneering study on the role of parietal cortex in visuomotor control)

10. Pezzulo et al. Progr Neurobiol 2022 (on the neural dynamics of premotor neurons during action execution and observation)

11. Santhanam et al. J Neurophysiol 2009

[Editors' note: further revisions were suggested prior to acceptance, as described below.]

Thank you for resubmitting your work entitled "Neural dynamics of causal inference in the macaque frontoparietal circuit" for further consideration by eLife. Your revised article has been evaluated by Tirin Moore (Senior Editor) and a Reviewing Editor.

The manuscript has been improved but there are some remaining issues that need to be addressed, as outlined below:

The authors did a good job at answering all the reviewers' comments in the rebuttal, particularly the once regarding analysis and statistically details. However, the consensus of the reviewers is that no real changes in the structure of the paper were carried out to simplify the framing of the manuscript and make it more accessible to a larger audience. In addition, all the reviewers are concerned with the lack of rational regarding the recording locations in the main text.

Reviewer #1 (Recommendations for the authors):

Although the authors did a good job at answering all the reviewers' comments in the rebuttal, particularly the once regarding analysis and statistically details. However, many of the framing and conceptual comments were not really incorporated in the actual reviewed manuscript. Specifically:

1) Please start the paper by giving an intuitive example of the key problem addressed in the manuscript.

2) There is no change in the introduction and the Results sections regarding a simpler and more intuitive framing of the paper (Figure 2A is the same). Again, the reader needs to go quite further into the manuscript to understand the main question and how the authors implemented the experiment.

3) The paper should refer to the classical notions of sensory motor integration in the parieto-premotor circuit in the discussion.

4) The authors did not mention why they recorded ventral premotor and a mix of area 5 and 7a in the main text.

Reviewer #2 (Recommendations for the authors):

In my previous review I pointed out that, although the experimental paradigm was overall well designed and the data analysis technically sophisticated, the manuscript was flawed in several aspects, particularly in relation to the way the paper was written, and the data reported and discussed.

Despite the extensive point-to-point reply, the revision of the paper remains disappointing, as no substantial changes have been made to consider the criticisms. As a matter of fact, the new version of it is essentially identical to the original one in all its sections (Abstract, Introduction, Results and Discussion). Surprisingly enough, despite the authors' attempt to reply with accuracy to the different issue raised in the reviewing process, no significant improvement of the resubmitted manuscript was achieved, as in most instances all new information was not fully integrated in the revised version.

The authors were invited to place the present work in the context of the extensive literature on the neurophysiology of the parieto-frontal network, with special attention to its role on reaching movements. In fact, the original manuscript did not adequately discuss the results within the conceptual frame offered by the knowledge accumulated over the last forty years on the dynamic properties of premotor and parietal neurons subtending arm movements. This suggestion was completely ignored, as both Discussion and Introduction remained virtually identical across versions and the authors just added a few references, among those suggested by the reviewers, in a rather superficial fashion, without any emphasis about how they were related to findings and conclusions of the present study.

Concerning point (1) the authors' action was limited to the mere insertion of new titles at the beginning of some paragraphs. In their response, it is reported that the logic of the manuscript is outlined in the unchanged Fig. 2A, which was already present in the previous version. Therefore, no significant change has been made to take into account this aspect.

Furthermore, the selected units shown in Fig. 3D to offer an example of neural activity in form of raster plots and mean firing rates (not histograms, as stated) are not indicative of clear response modulation.

The mentioned Table 1 is neither reported in the main text, nor in Supplementary Material.

When asked to evaluate the temporal difference between premotor and parietal activity, the authors just replied that "The population decoding of Pcom (Figure 4E) indicated that the premotor cortex leads the parietal by about 300 ms", but this observation refers to what already shown in the earlier version of the manuscript. Even in this case, in fact, no change was made to the analyses and to the text to take this point into consideration.

Also concerning the spike sorting technique adopted, despite the reviewer's request, no further details have been provided, relative to what was already reported in the first version of the manuscript.

Despite the explicit request (see point 4) to provide more details on which premotor area was considered in this study, the authors persist in referring loosely to "premotor" cortex, not specifying exactly which among the different premotor areas is being studied, apart from the details provided graphically in the brain figurine. Given the different functional properties and characteristics in connectivity among different premotor regions, it is inappropriate and misleading from a neurophysiological perspective to simply refer to premotor cortex. Provided that the region of recording mainly encompasses premotor area F2vr (medial to the spur of the arcuate sulcus), a small and medial part of premotor area F5, and part of F4, as evident from Fig. 3A, the main text did not report the rationale underlying the selection of the F2vr, F4/F5 for the present study. In addition, according to the new details provided in the current version of Fig. 3A on recording sites, some of the penetrations (corresponding to about 40-45 collected units) belong to prefrontal area 8 (FEF). These cells should be removed from the premotor database.

Furthermore, the lack of precise identification of the area/s of neural recording concerns parietal cortex as well. In fact, from a careful inspection of Fig. 3A, most of the penetrations in Monkey N belong to area 7 (which is part of the Inferior Parietal Lobule), and not to area 5 (which extends over the Superior Parietal Lobule instead), as stated throughout the manuscript. Finally, in all four insets of Fig. 3a it is not specified what the dashed grey lines refer to. This uncertainty would require a major revision of the parietal database.

Beyond the graphical (subjective) representation, no other information is provided about the criteria adopted to identify the recording areas and sites. First, as pointed out by the reviewers, the specification that they were "determined by individual MRI atlas" does not warrant for any areal identification; second is not even vaguely informative on how frontal and parietal areas were identified. The sentence that "The location of the recording chamber on each animal was determined by an individual MRI atlas" remained unchanged in the revised version of paper, without any further details, not even providing the reference on which Atlas has been used.

Finally, another reason of concern highlighted in the revision process refers to the lack of eye movements data, given that eye position and saccade direction exerts a well-known, although quantitatively different, influence on premotor and parietal neural activity. The authors did not refer to this critical aspect at all, nor did they discuss how eye-related signals might have influenced and eventually contaminated the reported findings.

Reviewer #3 (Recommendations for the authors):

The authors have addressed most reviewer comments to a sufficient degree. The work is still very dense given the large number of analyses implemented, but I have no specific suggestions for how to change this.

One remaining shortcoming is that I did not see a specific rationale for the choice of the precise recording locations in the manuscript. In reply to my previous comment, the authors have provided some rather generic text in the rebuttal, but ideally, a clear rationale for the choices should be in the manuscript.

https://doi.org/10.7554/eLife.76145.sa1

Author response

Essential revisions:

Main comments:

The present results are novel and provide important notions. However, we have a series of important concerns.

1) First, the paper is difficult to follow. The authors should use a simpler and more intuitive framing of the paper to make it more accessible to a general audience. A reader needs to integrate a large part of the results to really have an idea of what is the paper about and how the authors tested their hypothesis. Indeed, the writing in part uses words or phrases that are 'unusual' or imprecise, which makes the text difficult to understand.

In the revised manuscript, we have divided the behavior results into separate sections and summarized the conclusion of each part. In brief, the logic of the paper is outlined in Figure 2A, in which we decomposed the BCI model into sensory and hidden components, including the posterior probability of common source, the updating of prior belief, and sensory uncertainty, and then examined the behavioral evidence and corresponding neural representations respectively.

2) Another important concern is that the paper does not show the basic response properties of neurons on both areas across tasks. It is not clear how the neural activity changes between the VP, P, and VPC tasks. It is a change in preferred direction, in the width of the tuning function or on the gain modulation? We strongly suggest providing two 'layers' of information, one based on standards plotting and analysis, to be followed by a second layer of data investigation, addressed to more detailed computational aspects.

We have added the results with single-neuron analysis accordingly.

(i) We first examined whether neurons in the premotor and parietal cortex during the target-holding period were selective to basic task components, including condition (VP or P task), arm location, and visual disparity. For each neuron, we conducted a Two-ANOVA in two datasets. One dataset contains the VP and P tasks (condition (2 levels: VP and P tasks) × arm location (5 levels: [-30°, -20°, 0°, 20°, and 30°]); the response variable is the mean firing rate during the holding period of each neuron). If the main effect of condition (or arm location) in the Two-ANOVA was found (p < 0.05), this neuron was classified as a condition (or arm location) selective neuron. The other dataset is the VPC task (the visual disparity (9 levels: [-45°, -35°, -20°, -10°, 0°, 10°, 20°, 35°, 45°]) × target position (5 levels: [-30°, -20°, 0°, 20°, and 30°]); the response variable is the mean firing rate during the holding period of each neuron). If the main effect of visual disparity in the Two-ANOVA was found (p < 0.05), this neuron was classified as a disparity selective neuron. We found that in the premotor cortex, 39% (186/475) of neurons were selective to condition, 21% to arm location, and 34% to visual disparity (Figure 3—figure supplement 1A, upper panel). In the parietal cortex, 35% (83/238) of neurons were selective to Condition, 26% to arm location, and 31% to visual disparity (Figure 3—figure supplement 1 lower panel, ANOVA, main effect, p < 0.05). Over 25% of neurons in both regions were selective to multiple variables (premotor, 119/475; parietal, 64/238). Figure 3—figure supplement 1B represents example selective neurons in the premotor and parietal cortex, respectively.

(ii) For the neurons with arm-location selectivity (Figure 3—figure supplement 1A, premotor, n = 100; parietal, n = 63), we further investigated whether the tuning functions of these neurons were modulated in VPC (0°) task relative to the control condition (VP task). We fitted the tuning curve with a reduced von Mises function by using the neuron response in different arm locations (5 levels: [−30°, −20°, 0°, 20°, and 30°]) for these two tasks respectively. Here VPC (0°) task represents the trials in the VPC task where the disparity equals 0. And the fitting function was defined as:fr(x)=b+acos(xμ) where b is the spontaneous firing rate of the neuron, a is defined as the gain index, and μ is the preferred arm location. fr(x) represents the firing rate when the arm location is x. In the following, we analyzed the spontaneous firing rate, gain index, and preferred arm location of each neuron between the VP and VPC (0°) tasks in both premotor and parietal cortices. The results show that the preferred arm location was not significantly changed in both regions (Author response image 1A right, premotor, p = 0.62; parietal, p = 0.29, Wilcoxon signed-rank test). The spontaneous firing rates during the VP task were higher than that during the VPC (0°) task in the parietal cortex (Author response image 1A left, parietal, Wilcoxon signed-rank test, p = 0.012) but not in the premotor cortex (Author response image 1A left, premotor, Wilcoxon signed-rank test, p = 0.55). The gain index during the VP task was higher than that during the VPC (0°) task in the parietal cortex (Author response image 1A middle, parietal, Wilcoxon signed-rank test, p = 0.0012) but not in the premotor cortex (Author response image 1A middle, Wilcoxon signed-rank test, p = 0.081). The result is consistent with the conclusion that parietal, but not premotor, neurons update the neural information about arm locations between the VP and VPC (0°) tasks (Figure 6B).

Author response image 1

The comparisons of tuning curve parameters between the VP and VPC (0°) tasks. Left: The spontaneous firing rates during VP task were higher than that during VPC (0°) task in parietal cortex (parietal, W = 641.0, df = 62, rrb = 0.36, p = 0.012) but not in the premotor cortex (W = 2,350.0, df = 99, rrb = 0.069, p = 0.55). Middle: The gain index during VP task were higher than that during VPC (0°) task in parietal cortex (parietal, W = 535.0, df = 62, rrb = 0.47, p = 0.0012) but not premotor cortex (W = 2,017.0, df = 99, rrb = 0.20, p = 0.081). Right: The preferred arm location was not significantly changed in the premotor and parietal cortex (premotor, W = 2,334.0, df = 99, rrb = 0.056, p = 0.62; parietal, W = 852.0, df = 62, rrb = 0.15, p = 0.29). Pair-wise comparisons were performed using Wilcoxon signed rank test. Effect sizes (rrb) were performed using the rank-biserial correlation.

(iii) Furthermore, we reported raster plots and histograms of two example neurons (integration- and segregation- preferred) that varied their responses to the visual disparity during the target-holding periods (gray zones) in Figure 3D.

3) Statistics:

Results of model fitting: The methods mention that the authors compared three models: segregation, fusion and causal inference. However, in the results (Table S1) only two models are presented. The results for segregation should be reported for the sake of completeness. The Ppriori is high, e.g. 0.98 and 0.999. This makes me wonder in how far the R2 and the EPs' can be so much different between causal inference and fusion. Considering e.g. model averaging decision function for BIC, a prior of 0.999 would make fusion and BCI nearly identical. I feel that either the Methods section is lacking important details about the models (See above) or there is either a mistake in the analysis or the reported numbers in the table. The analysis of Bayesian models seems to lack major details, the statistical reporting is below standard (missing effect sizes, degrees of freedom, lack of individual data in figures), the study shows many unjustified parameter choices and key results seem to lack statistical support: not all statements about differences between parietal and premotor cortex seem supported by a direct statistical comparison. Further, while three monkeys contributed data, for only one does the study report data from both brain regions; this makes the claim of a difference between brain regions rather weak and this shortcoming needs to be clearly acknowledged. The actual underlying data (e.g. how single neuron responses are converted to tuning curves; how decoding accuracies vary across neurons) is not shown, which makes it difficult to interpret the robustness of the results. In particular, as the units of analyses vary tremendously between Figures (experimental blocks, neurons, pseudo-epochs, etc).

We thank the referee for the suggestions. The details of the model have been added to the revised Methods.

(i) The full segregation model fitting results are now included in Author response table 1.

Author response table 1
Model parameters and fitting evaluations of two models for monkeys.
subjectCausal inference (model averaging)Forced fusion
relBICgroupEPR2σPσVPriorrelBICgroupEPR2σPσV
Monkey H010.96±0.00177.72±0.145.83±0.0900.999±0.0004329.5200.93±0.00399.87±0.249.02±0.23
Monkey N010.93±0.00809.56±0.164.93±0.150.86±0.026812.3400.37±0.3611.34±0.1010.46±0.13
Monkey S010.96±0.00228.98±0.145.72±0.220.98±0.012290.0400.94±0.002710.10±0.148.34±0.17
subjectFull segregation (proprioceptive only)Full segregation (visual only)
relBICgroupEPR2σPrelBICgroupEPR2σV
Monkey H1876.1000.44±0.02614.43±0.151677.0400.58±0.01914.51±0.091
Monkey N1902.8200.53±0.01814.76±0.0701739.4000.51±0.07814.38±0.095
Monkey S1937.3200.39±0.04014.72±0.0661440.4700.71±0.01612.75±0.28

The model parameters and R2 were averaged across days for monkeys; data are presented as the means ± the standard errors. The relBICgroup was the summation of all days’ BIC for monkeys.

Abbreviations: σP, the standard deviation of the proprioception likelihood; σV, the standard deviation of the vision likelihood; Pprior, the prior probability of a common source; relBICgroup, Bayesian information criterion at the group level; EP, exceedance probability; R2, coefficient of determination.

(ii) We agree with the referee that the Pprior is high in the monkey H and S, and relatively low in the monkey N. Despite the high Pprior in the causal inference model, we would like to mention that there is an essential difference between these two models. In particular, the long-tailed or bimodal distribution of proprioceptive drift under large disparities can only be explained by the causal inference model (in red, Figure 1—figure supplement 2 ) but not by the forced-fusion model (unimodal distributions in green, Figure 1—figure supplement 2 ) (an example behavior session in Figure 1—figure supplement 2 ).

(iii) For the analysis of Bayesian models, the analysis of the Bayesian model was reported in revised Supplementary file 1-Table 1; see the relBICgroup (Bayesian information criterion at the group level) and EP (exceedance probability).

(iv) Rank-biserial correlation was used to estimate the effect size for the nonparametric test (Kerby, Comprehensive Psychology, 2014). The degrees of freedom and effect size were added to the revised figure legend.

(v) We used the premotor and parietal neurons from the same monkey (Monkey N) to decode Pcom. The results suggest the same tendency as the results using parietal and premotor neurons across monkeys (Figure 4—figure supplement 2 ).

(vi) To verify the robustness of the decoding results, we tested the decoding ability by sampling different numbers of neurons for Pcom, Pprior, disparity, and arm locations, respectively (Author response image 2 and 3). The conclusion holds.

Author response image 2
Population decoding of Pcom (upper panel), Pprior (middle panel), and Disparity (bottom panel) with different neuron numbers in the premotor (left panel) and parietal (right panel) cortex, respectively.

The horizontal dashed black line represents the chance level. The horizontal solid bars at the top represent the time of significant decoding accuracy (cluster-based permutation test, p < 0.05). Different color shades represent different numbers of neurons used for decoding.

Author response image 3
Population decoding of arm locations with different neuron numbers in the premotor and parietal cortex, respectively.

The horizontal dashed black line represents the chance level. The flat bar at the top represents the time bins in which the decoding accuracy for the VPC (0°) task was significantly lower than that for the VP task (cluster-based permutation test, p < 0.05). N: number of neurons used for decoding.

The updating of the believe about the sensory causal structure is a central component of this work. The authors present this as well established aspect of the BCI model (l. 170ff). However, most previous studies used a static model that was fit to the aggregate data of an entire experiment without taking the trial history into account (as in the cited Koerding et al. 2007 paper). AS some recent work has incorporated such trial dependencies, it would be important to acknowledge these studies and to explain the novelty of the present work (e.g. Rohe et al. NatComm 2019; Beierholm et al. eLife 2020; Badde et al. Cognition 2020 may also be relevant).

The discussion about these studies has been added and the references have been cited in the revised manuscript accordingly (L. 596).

Analysis of recalibration: The analysis of P trials after the VP or VPC task effectively looks at what is known as trial-wise multisensory recalibration (see e.g. Bruns et al. Scientific Reports 2015; Park and Kayser eLife 2019; van der Burg et al. J Neurosci 2013; Wozny et al. JNeurosci 2011; Badde et al. Cognition 2020; there is extensive literature on this both in the spatial and the temporal domain!). It seems awkward to investigate this recalibration of uniusensory judgements without alluding to previous work.

Indeed, the references suggested by the referee are relevant. We have cited them accordingly (L. 616). However, it is worth noting that, instead of the multisensory recalibration of the sensory errors [the spatial localization errors (Bruns et al., 2005; Park and Kayser, 2019; Badde et al. Cognition 2020) and temporal shifts (van der Burg et al. J Neurosci 2013)], we focused on sensory uncertainty and its updating.

If I understood Figure 3A and the Methods correctly, only in monkey N both brain regions were recorded? If this is correct, the statement that premotor and parietal regions differ in their representations is a result of a mixed within and between subjects analysis. This should be acknowledged explicitly, as it greatly reduces the statistical power of this statement, and the repeated statement about an N of 3 is misleading. CI neurons are defined based on a seemingly arbitrary criterion: exceeding a correlation threshold based on the arbitrary division of the data into 26 bins. Given that neural representations generally span a continuum, I would like to see the distribution of 'causal inference effects' for individual neurons, e.g. in form of a distribution of r2 values (obtained from the correlation; or a regression as I suggest below). The apparent difference between brain regions (Figure 3G) may simply result from the specific choice of statistical cutoff (the criterion of p<0.05 becomes meaningless in the presence of 475+238 tests in total). Seeing the individual-neuron data here seems vital.

(i) We thank the referee for reminding us of the missing information. We have acknowledged the limitations of our analysis as follows (L. 1130 in the revised manuscript):

“Note that, due to the limitations of the asynchronous recording (the premotor and parietal neurons were grouped from different individual animals, and only monkey N was recorded in both areas), further studies are required to clarify the dynamics and functional interactions between regions using a simultaneous recording.”

(ii) Here we show the histogram of the Pearson correlation coefficients between Pcom and VP weight (Author response image 4). The correlation coefficients of both regions are significantly larger than 0 (two-sided one-sample t-test, Premotor: t474 = 7.88, p < 0.001, Cohen’s d = 0.36; Parietal: t237 = 3.16, p < 0.01, Cohen’s d = 0.21). The Pearson correlation coefficients in premotor neurons are slightly higher than that in the parietal neurons (two-sided two-sample t-test, t474, 237 = 1.82, Cohen’s d = 0.10, p = 0.069).

Author response image 4
Histogram of Pearson correlation coefficients between VP weight and Pcom.

The correlation coefficients of both regions are significantly larger than 0 (two-sided one-sample t-test, Premotor: t474 = 7.88, Cohen’s d = 0.36, p < 0.001; Parietal: t237 = 3.16, Cohen’s d = 0.21, p < 0.01). ** p < 0.01; *** p < 0.001.

The analysis of population timing suggests that premotor cortex leads (Figure 4F). Is it possible to extract by how much time? Also, the authors focus on the encoding of Pcom, which comprises both the a priori binding tendency and the discrepancy. Why did the authors not decode both the prior and the current multisensory discrepancy separately? This would seem important to differentiate neural signatures of priors from those of current sensory signals.

(i) The jPECC analysis only suggested a leading tendency from premotor to parietal. The population decoding of Pcom (Figure 4E) indicated that the premotor cortex leads the parietal by about 300 ms.

(ii) We have presented the decoding result of the Pprior in Figure 5B and compared it with the Pcom decoding result in Figure 5F. We found that the premotor neurons encode the Pprior (previous trial’s Pcom) during the baseline period, where there is no sensory input.

(iii) We have now reported the disparity decoding results in Figure 5—figure supplement 2, and verified it with the number of units (Author response image 5). The result of population decoding (Author response image 5) showed a systematic sequential procession of causal inference in the frontoparietal circuit again. At first, the brain encodes the Pprior before the multisensory disparity onset (the Pprior can be decoded at the baseline period), after that the Pcom is estimated by combining the Pprior and multisensory disparity. Note that the Pcom and disparity can only be decoded after visual stimulus onset, and the disparity signal emerges earlier than the signal of Pcom.

Author response image 5
Population decoding of Pcom (upper panel), Pprior (middle panel), and Disparity (bottom panel) with different neuron numbers of premotor (left panel) and parietal (right panel).

The horizontal dashed black line represents the chance level. The horizontal solid bars at the top represent the time of significant decoding accuracy (cluster-based permutation test, p < 0.05). Different color shades represent different numbers of neurons used for decoding.

Showing the actual data: The key results (e.g. Figure 3C; 4F; 5B; 5F; 6C) would be much stronger rand more convincing if the actual units of analysis were shown in some ways. How does decoding accuracy vary across neurons?

We have shown the actual units of analysis for each result (see Author response image 1; Figure 3C; Author response images 2, 3 and 4). Furthermore, we have tested it with different numbers of units to decode Pcom, Pprior, disparity, and arm locations, respectively (Author response images 2 and 3). The same conclusion holds.

Other details:

For most tests there are no measures of effect sizes reported, sometimes the respective test-statistics is missing, and the degrees of freedom remain very unclear. I understand that they wary between analysis, but given that some tests are based on the actually recorded units, some of pseudo-trials or binned data, it would be very important to report for each test the assumed independent units and their number. The false-discovery rate is mentioned frequently, but the precise method is not stated. Most analyses are based on Wilcoxon tests, but figures show mean and SDs. I encourage the authors to use the same nonparametric (or parametric) approaches for figures and stats (e.g. show boxplots and individual data). L. 892: what was precisely compared with the ANOVA? Cluster-based tests: the parameters and the procedures for this test are not reported (l.974)

(i) Rank-biserial correlation was used to estimate nonparametric test effect size (Kerby, Comprehensive Psychology, 2014). The degrees of freedom and effect size were added to the revised figure legend, where a nonparametric test was performed.

(ii) Benjamini-Hochberg procedure was used to correct the false discovery rate (FDR) (see revised Methods, L. 914).

(iii) We have shown the individual data and used the boxplots (see revised Figures 2B, C, and D and 3C).

(iv) For L. 892 now L. 982, two-way ANOVA was performed with factors of Condition (2 levels: VP and P) and target location (5 levels).

(v) The detail of cluster-based permutation tests was added in Methods (L. 1059) as follows:

“The significant time duration was determined using a cluster-based permutation test for multiple comparisons across time intervals (permutations = 5000; cluster-level statistic: sum of the t values in a cluster; auxiliary cluster defining threshold t = 3) (Gramfort et al., 2013).”

To determine whether a neuron confirms to the expectation of causal inference, why is it necessary to bin the data (l 893ff)? Could one not simply derive a regression model for each neuron and visualize the R2 or F-value?

Since the Pcom is a hidden variable predicted by the BCI model, it cannot be directly obtained from the drift of one trial. To estimate the Pcom of each trial, we first simulated 5000 trials under each disparity using the fitted BCI model and grouped these trials into 29 bins. Thus, the Pcom of each simulated trial can be obtained by the BCI model. The Pcom of the simulated trials within each bin were averaged to obtain the Pcom of each bin. Finally, each behavior trial was assigned to one of these bins according to its proprioceptive drift and disparity, and the Pcom of each trial was assigned according to the bin it belongs to. Thus, the correlation between VP-weight and Pcom can be calculated across 29 bins.

The authors seem to interpret differences in the significances (e.g. of cluster-based permutation tests) as significant differences between regions and as establishing differences in the relative timing of effects. These are statistical fallacies (e.g. Sassenhagen https://doi.org/10.1111/psyp.13335; and Makin https://doi.org/10.7554/eLife.48175).

We do not rely upon the cluster-based permutation tests to determine differences in the significance as significant differences between regions. Rather, we adopted a randomization test (Panichello and Buschman, bioRxiv, 2020) ( also see Methods, L. 1063):

“To test whether premotor neurons encode Pcom earlier than parietal neurons, a randomization test was performed between them. Neurons with more than 50 trials in each Pcom group were included in this analysis. The corresponding numbers (here, 50 neurons per region) of neurons were randomly exchanged between the paired regions 1,000 times to generate a null distribution (chance level) of time lags. The significance was determined by a permutation test of the true time lag from the original data and the null distribution (Panichello and Buschman, 2020).”

For every statement claiming differences between parietal and premotor cortex it is necessary to directly impellent the respective contrast between neurons in each brain region to support such a difference.

We have added the direct comparison results in the revised manuscript and response letter. In brief, at the single-neuron level, we directly compared the neural information for the arm location (Figures 3C, Author response image 1, and Figure 6B) and Pcom (Figures 3H and Author response image 4) between the parietal and premotor cortex. At the population level, we conducted population decoding (Figures 4E, 5B, and 6C) and the joint peri-event canonical correlation (jPECC) analysis (Figure 4F) to demonstrate the difference between these regions.

Other methods:

Spike sorting: I could not find criteria used for spike sorting. Where the analyzed units single units or MUA? More details about spike thresholds, cluster separation etc. should be provided.

All spike data were re-sorted using off-line spike sorting clustering algorithms (Plexon, principal component analysis). With manual adjustments, only well-isolated units were considered for further analysis (signal-to-noise is more significant than 3). As shown in the revised Methods (L. 786):

“On-line raw neural signals were processed offline to obtain a single unit by Offline Sorter (Plexon Inc, Dallas, TX). All spike data were re-sorted using off-line spike sorting clustering algorithms (Plexon, principal component analysis). The auto sorted neurons were then refined manually, and only well-isolated units were considered for further analysis (signal-to-noise is larger than 3). The sorted files were then exported as MATLAB format for further analysis in MATLAB (Mathworks, Natick, MA, USA) and Python (The Python Software Foundation).”

The total number of switches between blocks (e.g. P following VPC) should be reported, as this constitutes the effective degrees of analysis of the block switching analysis (Figure 2C).

There are 282 P bocks following VPC blocks and 240 P bocks following VP blocks.

Causal inference models and optimization: The methods leave it unclear how the two alternatives of common and separate sources were combined in the BCI model. Previous work has explored a number of decision functions (e.g. Rohe and Noppeney's work, or Wozny et al. PlosCompBiol 2010) but for the present study it remains unclear which decision function was used. Model fitting: how were likelihoods computed and the posteriors sampled for model fitting? I feel that the procedures are not described in sufficient detail to be reproduced. Over what range of disparities was the model optimized? This is important for the Null model mentioned later on. What is the precise number of data points that entered the BIC calculation?

(i) We used the model average decision function to estimate the final arm location. We have revised the methods accordingly in the Methods (L. 836):

“Here, we used model average decision function to estimate final arm location (Fang et al., 2019):

S^P=p(C=1|xV,xP)S^VP,C=1+(1p(C=1|xV,xP))S^P,C=2.”

(ii) We used maximum likelihood estimation for model fitting, which is described in the Methods (L. 844):

“To estimate the best-fitting model parameters in the BCI model, for each recording session, an optimization search was implemented that maximized the log likelihood of each model given the monkey’s data under the VPC task. The prior probability of a common source (Pprior) and visual and proprioceptive standard deviations, σV and σP, respectively, were set as free parameters to be optimized. Five thousand trials per disparity were simulated for each optimization step to obtain the distribution, and the sum log likelihood of the observations given the model was calculated for each disparity. Then, the parameters were optimized by minimizing the sum log likelihood using a genetic algorithm (ga function in MATLAB).”

(iii) The total sessions of each monkey that entered the BIC calculation were 68 days (Monkey H), 90 days (Monkey N), and 85 days (Monkey S), respectively.

Markov analysis: If I understood it correctly, SigmaA and SigmaV are fit to the entire block, and the Pprior derived from the entire block was used as starting value for this parameter? The authors conclude (l 256ff) that to 'maintain a consistency of causal inference, sensory uncertainty … is updated ' as well. However, the Markov model seems to focus only on the updating of the prior.

(i) Yes, Σ P, Σ V, and Pprior derived from the entire block were used as starting values for the parameter, and only the Pprior was free in the Markov model fitting (see Methods L. 887):

“During the model fitting, we first used the Bayesian causal inference model (as mentioned before) to search the overall Pprior, σP, and σV for each session/day, which were used as initial parameter in the subsequent Markov model. The σP and σV were fixed during the model fitting.”

(ii) The referee raised an interesting question. However, as the sensory uncertainty cannot be estimated at a trial-by-trial level, the Markov model did not include it.

Processing of single unit data: In my view the paper would profit from showing actual single neuron PSTH's and how smoothing effected these. The methods (l. 857) mention a 400ms sliding window, but the periods of interest (e.g. target holding) are only minimally longer than this (500ms). This makes we worried that the analyzed data effectively blurs neural representations across epochs and is affected by movement artifacts. When computing the modality contributions to each response, what task epoch was analyzed to derive the tuning curves (l. 869ff)?

We have verified the decoding with different smoothing windows (200, 300, and 400 ms) (Author response image 6). The same conclusion holds.

Author response image 6
Population decoding of Pcom with various smoothing window sizes (200, 300, and 400 ms), respectively.

The horizontal dashed black line represents the chance level. The horizontal solid bars at the top represent the time of significant decoding accuracy (cluster-based permutation test, p < 0.05).

Overall there are many seemingly arbitrary choices in the methods. These include the thresholds to define neurons as 'causal inference', the number of trials required for neurons to be included in the population analysis (l. 964), the duration of smoothing kernels and temporal analysis windows (l. 848 ff), the binning of data for neuro-behavioral correlation (l. 893ff), in the generation of population patterns (l. 912ff), in the cluster-based test (not reported!). It would be good to see a justification for these choices or to learn whether the authors ensured that their main results do not depend on these precise choices.

We are grateful for the referee’s suggestion and have revised the Methods accordingly.

(i) The definition of ‘causal inference neuron’ follows our previous study (Fang et al., PNAS, 2019). The basic idea here is that the pattern of neuronal activity across 29 bins (see the above response) should correlate with the Pcom predicted by the BCI model. The significant level of p = 0.05 of this correlation was used as the threshold to define the ‘causal inference neuron’.

(ii) Only neurons with at least 50 trials of each condition were included in the analysis. We have added this information in the revised Methods (L. 1064).

(iii) The duration of smoothing kernels and temporal analysis windows were often used in previous studies (Fang et al., PNAS, 2019; Gu et al., Cerebral Cortex, 2016; Fried et al., Neuron, 2011).

(iv) As the neurons in our study did not record simultaneously, we generated a pseudo-simultaneous population by the bootstrap method.

(v) For the cluster-based test, the details of cluster-based tests were added in the revised Methods (L. 1059) as follows:

“The significant time duration was determined using a cluster-based permutation test for multiple comparisons across time intervals (permutations = 5000; cluster-level statistic: sum of the t values in a cluster; auxiliary cluster defining threshold t = 3) (Gramfort et al., 2013).”

4) The authors do not justify why they recorded in the transition between F4 (rostral ventral premotor) and F5 (caudal ventral premotor) with head visual/tactile optic flow signals and grasping signals respectively. The obvious target is F2 (dorsal premotor) since it has strong reaching signals and is highly connected with area 5 of the parietal lobe (Rizzolati, 1990; Mendoza and Merchant, 2014). The authors should provide a more detailed account on the areas studied and the criteria adopted to localize the recording sites. The specification that they were "determined by individual MRI atlas" does not warrant for areal identification, because on natural variability. On this regard, the Figure 3A' insets should be adjusted (for parietal recording sites in L and R hemispheres, the sulci orientation should be different, and for the premotor ones sulci should be reported).

(i) The reaching task we used in our study is an adapted task of rubber hand illusion focused on the multisensory representation of the arm. Previous studies have shown that the ventral premotor cortex and parietal area 5 are highly related to body representation and multisensory perception (see reviews (Blanke, Nat Rev Neurosci, 2012; Graziano and Botvinick, Common Mechanisms in Perception and Action, 2002)). More specifically, ventral premotor neurons have been shown to respond to visual stimuli in the space adjacent to the hand or arm (Graziano et al., Science, 1994), and parietal (area 5) neurons have been shown to respond to the seen position of a dummy arm (Graziano et al., Science, 2000). Furthermore, human fMRI studies have shown that the strength of illusion of the dummy arm was highly correlated to the neural response of ventral premotor (Ehrsson et al., Science, 2004).

(ii) We thank the referee for the suggestion and have adjusted the Figure 3A insets in the revised manuscript.

5) For the discussion and comparison to previous work: The paradigm focuses on visuo-motor paradigm in which the sensory cues are both generated by the subject itself. In contrast, in many classical (e.g. audio-visual; or visual-vestibular)) paradigms both sensory cues are external in nature, and not linked to the subject's action. While in both types of paradigm sensory cues are integrated and can also induce perceptual recalibration, the visuo-motor paradigm still is conceptually distinct and this has implications for the interpretation of the results. The authors should discuss whether they believe that their findings generalize to other paradigms and whether the same or possibly distinct (e.g. parietal) brain regions should be investigated during such paradigms. Such a discussion seems important to place the present work in the context of the plethora of previous work. Indeed, the present study is completely lacking discussion of results with respect to current knowledge on the functional properties of premotor and parietal neurons subtending reaching. The literature on this topic is vast, but the following studies, as examples, could be relevant in this context:

1. Archambault et al. J Neurosci 2011 (comparison on premotor vs parietal, where premotor activity leads parietal one)

2. Caminiti et al. eNeuro, 2017 (overall picture of connectivity of fronto-parietal network with updated literature on functional properties of different areas)

3. Caminiti et al. J Neurosci 1991 (first paper on encoding of reaching in Premotor cortex)

4. Churchland MM, et al. Nature 2012

5. Cisek and Kalaska, Neuron 2005 (on premotor activity during reaching)

6. Gail and Andersen J Neurosci 2006 (on neural dynamics of sensorimotor transformations in parietal cortex)

7. Jerjian SJ, Sahani M, Kraskov A ELife 2020 (on movement representation in premotor cortex)

8. Jiang X et al. Cell Rep 2020 (onpremotor neural Activity during Observed and Executed Movements)

9. Mountcastle et al. J Neurophysiol 1975 (first pioneering study on the role of parietal cortex in visuomotor control)

10. Pezzulo et al. Progr Neurobiol 2022 (on the neural dynamics of premotor neurons during action execution and observation)

11. Santhanam et al. J Neurophysiol 2009

We are grateful for the references that the referee suggested. The references now have been cited accordingly (L.596, 638).

Although a reaching task was used to require monkeys to report their arm locations, the neural signal about Pcom was observed before reaching. The current study focused on multisensory integration and causal inference, which merely relied on sensory inputs (V, P, and disparity). The reaching behavior in the task could be the action signal guided by the Pcom.

[Editors' note: further revisions were suggested prior to acceptance, as described below.]

The authors did a good job at answering all the reviewers' comments in the rebuttal, particularly the once regarding analysis and statistically details. However, the consensus of the reviewers is that no real changes in the structure of the paper were carried out to simplify the framing of the manuscript and make it more accessible to a larger audience. In addition, all the reviewers are concerned with the lack of rational regarding the recording locations in the main text.

Reviewer #1 (Recommendations for the authors):

Although the authors did a good job at answering all the reviewers' comments in the rebuttal, particularly the once regarding analysis and statistically details. However, many of the framing and conceptual comments were not really incorporated in the actual reviewed manuscript. Specifically:

1) Please start the paper by giving an intuitive example of the key problem addressed in the manuscript.

We have added an intuitive example in the introduction section to highlight the main questions we addressed in this study in the revised manuscript (L. 54-58).

2) There is no change in the introduction and the Results sections regarding a simpler and more intuitive framing of the paper (Figure 2A is the same). Again, the reader needs to go quite further into the manuscript to understand the main question and how the authors implemented the experiment.

As suggested by the referee, we have revised the results and added new paragraphs of the hypotheses in the present study to give a simpler framing of the paper (Line 123-144), which explicitly elaborates Figures 1C-F.

3) The paper should refer to the classical notions of sensory motor integration in the parieto-premotor circuit in the discussion.

As suggested by the referee, we have now added the discussion about the relationship between our study and previous findings of sensorimotor representations in the frontoparietal circuit as follows (L. 682-714; L. 737-745).

“The frontoparietal circuit, including the premotor and parietal cortices, has long been recognized as a central area in sensorimotor representations (Caminiti et al., 2017; Caminiti et al., 1991). Although the present experiments shared many movement features in the reaching task, the key findings of causal inference processing are unlikely to be explained by the kinematical components. First, previous studies have demonstrated that the neuronal activities in the premotor cortex are related to hand kinematics (e.g., hand position, speed, and direction) in the motor planning and execution (Caminiti et al., 1991; Churchland et al., 2006), which lead the neural activities in the parietal cortex (Archambault et al., 2011). However, in our study, the early activities of Pcom in the premotor cortex cannot be purely induced by the sequential activities of kinematics in the premotor and parietal cortices. Because the Pcom is abstract information, and its activity pattern is not correlated with any kinematical components. Expressly, under a given value of Pcom, the reaching kinematics can be varied (e.g., the hand position can be anywhere on the table according to the target position and disparity in a given trial). Moreover, the neural signals about Pcom in the premotor cortex were observed before the target onset, where no motor planning was possible during this period. Thus, our results are consistent with the idea that the high-level information, such as abstract and hidden structures, potential probability of multiple motor options, and visual-proprioceptive integration, are encoded in the frontoparietal circuit, which could later integrate with the low-level sensory representations to guide the desired movement (Cisek and Kalaska, 2005; Gail and Andersen, 2006; Limanowski and Blankenburg, 2016).

Second, the dynamic updating of prior and sensory representation proposed a putative mechanism for multisensory recalibration in sensorimotor tasks. At the behavioral level, our results are in accord with the observations that sensory perception is modulated by a multisensory context with sensory conflicts. The BCI theory thus provides a framework to explain how the multisensory context (e.g., the prior of common source) modulates the sensory representations, such as sensory uncertainty in our study and sensory estimation (e.g., spatial localizations) in previous sensorimotor studies (Badde et al., 2020; Bruns and Roder, 2015; Park and Kayser, 2019; Van der Burg et al., 2013). The results support the notion of dynamic representations of Pcom in the present study – the top-down signal of common source from the premotor cortex modulates the spatial tuning in the parietal cortex and then guides hand estimation.

[…]

Intriguingly, our results seem complementary to previous findings of mirror neuron systems in the premotor and parietal cortices in both humans and monkeys. Typically, a mirror neuron fires both when individual acts and when the individual observes the same action performed by another. That is, the mirror neuron is believed to mediate the understanding of others' behavior (Jerjian et al., 2020; Jiang et al., 2020; Pezzulo et al., 2022). By contrast, the role of causal inference neurons in our study was putatively participating in self-identification and self-other discrimination. Future studies are needed to examine how these two systems work together to identify both self and foreign agents”.

4) The authors did not mention why they recorded ventral premotor and a mix of area 5 and 7a in the main text.

The recording regions were chosen based on previous monkey electrophysiological and human studies. In brief, previous studies showed that the premotor (including dorsal and ventral premotor) and posterior parietal cortices (including areas 5 and 7) are related to visual-somatosensory integration and arm recognition (see the reference below). However, there is no direct evidence showing a specific region for the visual-proprioceptive integration and causal inference processes. As the first study of causal inference integration of visual and proprioceptive signals, we decided to record neurons in both dorsal and ventral premotor and posterior cortices, including areas 5 and 7. In fact, we did not find a significant difference between the sub-regions in the premotor cortex and the sub-regions in the parietal cortex (see Author response image 7, p > 0.05). We have added the reasons why we chose the recorded regions in the revised manuscript as follows (see L. 296-312 in the main text).

“Previous studies showed that the premotor and parietal cortices were highly involved in body representation and multisensory perception (see reviews (Blanke, 2012; Graziano and Botvinick, 2002)). In monkeys, bimodal neurons with visual and somatosensory receptive fields were found in both premotor (including F2vr in dorsal premotor and F4/F5 in ventral premotor) and posterior parietal cortices (including area 5 and area 7) (Fogassi et al., 1999; Graziano et al., 2000; Graziano and Gross, 1993, 1998; Graziano et al., 1994). Specifically, ventral premotor neurons responded to visual stimuli in the space adjacent to the arm (Graziano and Gross, 1998; Graziano et al., 1994). The bimodal neurons in the parietal cortex (area 5 and area 7) showed to respond to both the real arm position and the seen position of a dummy arm (Graziano et al., 2000), which have a significant projection of the premotor cortex (Graziano and Gross, 1998). Consistently, human fMRI studies found that the posterior parietal and premotor (dorsal and ventral) cortices selectively respond to visual stimulation near the hand (Brozzoli et al., 2011) or the dummy hand near one’s corresponding hand (Blanke et al., 2015; Ehrsson et al., 2004). A human MEG study also revealed that the activities in the prefrontal and intraparietal sulcus were related to the causal inference computation in visual-auditory integration (Cao et al., 2019; Rohe et al., 2019)”.

Author response image 7
(A) Left and middle: the fraction of selective neurons in the dorsal premotor (PMd) cortex and the ventral premotor (PMv) cortex (ANOVA, main effect, p < 0.05).

There was no significant difference between the fraction of selective neurons in PMd and PMv (Pearson's chi-square test, χ2 = 0.013, df = 2, p = 0.99). Right, the fraction of causal inference neurons in the dorsal premotor cortex and ventral premotor cortex. There was no significant difference between the fraction of causal inference neurons in PMd and PMv (χ2 = 0.15, df = 1, p = 0.70). (B) Left and middle: the fraction of selective neurons in the parietal Area 5 cortex and Area 7 cortices (ANOVA, main effect, p < 0.05). There was no significant difference between the fraction of selective neurons in parietal Area 5 and Area 7 cortices (Pearson's chi-square test, χ2 = 0.047, df = 2, p = 0.98). Right, the fraction of causal inference neurons in the parietal Area 5 cortex and the parietal Area 7 cortex. There was no significant difference between the fraction of causal inference neurons in parietal Area 5 and Area 7 (χ2 = 1.023×10-30, df = 1, p = 1).

Reviewer #2 (Recommendations for the authors):

In my previous review I pointed out that, although the experimental paradigm was overall well designed and the data analysis technically sophisticated, the manuscript was flawed in several aspects, particularly in relation to the way the paper was written, and the data reported and discussed.

Despite the extensive point-to-point reply, the revision of the paper remains disappointing, as no substantial changes have been made to consider the criticisms. As a matter of fact, the new version of it is essentially identical to the original one in all its sections (Abstract, Introduction, Results and Discussion). Surprisingly enough, despite the authors' attempt to reply with accuracy to the different issue raised in the reviewing process, no significant improvement of the resubmitted manuscript was achieved, as in most instances all new information was not fully integrated in the revised version.

The authors were invited to place the present work in the context of the extensive literature on the neurophysiology of the parieto-frontal network, with special attention to its role on reaching movements. In fact, the original manuscript did not adequately discuss the results within the conceptual frame offered by the knowledge accumulated over the last forty years on the dynamic properties of premotor and parietal neurons subtending arm movements. This suggestion was completely ignored, as both Discussion and Introduction remained virtually identical across versions and the authors just added a few references, among those suggested by the reviewers, in a rather superficial fashion, without any emphasis about how they were related to findings and conclusions of the present study.

We thank the referee for reminding us of the changes to the Discussion and Introduction. In the revised manuscript, we first added the hypotheses at the beginning of the results to give a simpler framing of the study, explaining the logic before we showed the results. We then added the reasons we chose to record the ventral and dorsal premotor and parietal (areas 5 and 7) cortices in the results before we showed the neural data. Finally, we modified the discussion to discuss that although our experiments shared many movement features in the reaching task, the key findings of causal inference are unlikely to be explained by the kinematical components during the arm movements. In the following, we list the responses point-by-point:

Concerning point (1) the authors' action was limited to the mere insertion of new titles at the beginning of some paragraphs. In their response, it is reported that the logic of the manuscript is outlined in the unchanged Fig. 2A, which was already present in the previous version. Therefore, no significant change has been made to take into account this aspect.

1) We have added an intuitive example in the introduction section to highlight the main questions we addressed in this study as follows (L. 54-58).

“For instance, in the ventriloquism illusion, when the audience is presented with a synchronous but spatially discrepant audiovisual stimulus (e.g., a speech sound from the speaker and a visibly moving mouth of the puppet), they usually infer these audiovisual stimuli are coming from a common source and illusive perceive the speech coming from the puppet.”

2) We have added an overview of the main predictions of this study at the beginning of the results section--“Behavioral paradigm and hierarchical Bayesian causal inference model” in the revised manuscript (L.123-144).

Furthermore, the selected units shown in Fig. 3D to offer an example of neural activity in form of raster plots and mean firing rates (not histograms, as stated) are not indicative of clear response modulation.

To better illustrate the response modulation by the disparity, we plotted the mean responses of the example neuron in Figure 3D during the target-holding period (Figure 3 and Figure 3-Figure supplement 2). Figure 3 shows a “segregation (P) neuron”, which is more active during the P task and exhibited increased activity under the large disparities in the VPC task. By contrast, Figure 3 shows an “integration (VP) neuron”, which is more active during the VP task and exhibited increased activity under the small disparities in the VPC task.

The mentioned Table 1 is neither reported in the main text, nor in Supplementary Material.

We indeed reported Table 1 in our last revision, but it was required to be submitted as an independent document. In this round of revision, we resubmit it as Supplementary file 1. Model parameters and fitting evaluations of two models for monkeys.docx.

When asked to evaluate the temporal difference between premotor and parietal activity, the authors just replied that "The population decoding of Pcom (Figure 4E) indicated that the premotor cortex leads the parietal by about 300 ms", but this observation refers to what already shown in the earlier version of the manuscript. Even in this case, in fact, no change was made to the analyses and to the text to take this point into consideration.

To further confirm the significance of the temporal difference between the premotor and parietal cortices, a randomization test was performed on the population decoding of Pcom between these two regions. Neurons with more than 50 trials in each Pcom group were included in this analysis. The corresponding numbers (200 neurons per region) of neurons were randomly exchanged between the paired regions 1,000 times to generate a null distribution (chance level) of time lags, and the significance was determined by a permutation test of the true time lag from the original data and the null distribution (Panichello & Buschman, 2021). The results again showed that the Pcom information appeared to occur significantly earlier in the premotor cortex than in the parietal cortex (Figure 4-Figure supplement 3, randomization test, p < 0.01). Furthermore, we also confirmed the result using the jPECC analysis in the earlier version of the manuscript (Figure 4F) and discussed the limitation of the asynchronous single-unit recording (L. 1293-1296).

Also concerning the spike sorting technique adopted, despite the reviewer's request, no further details have been provided, relative to what was already reported in the first version of the manuscript.

We have added the details of spike sorting and the reference in the revised Methods (L. 903-910), as follows:

“All isolated neurons were recorded regardless of their activity during the task, with the recording locations varying from session to session. At each location, the raw extracellular membrane potential was sampled at 40 kHz. Online raw neural signals were processed offline to obtain a single unit by Offline Sorter (Plexon Inc, Dallas, TX). All spike data were re-sorted using offline spike sorting clustering algorithms (Offline Sorter, principal component analysis) (Merchant et al., 2013). With manual adjustments, only well-isolated units were considered for further analysis (signal-to-noise is larger than 3).”

Despite the explicit request (see point 4) to provide more details on which premotor area was considered in this study, the authors persist in referring loosely to "premotor" cortex, not specifying exactly which among the different premotor areas is being studied, apart from the details provided graphically in the brain figurine. Given the different functional properties and characteristics in connectivity among different premotor regions, it is inappropriate and misleading from a neurophysiological perspective to simply refer to premotor cortex. Provided that the region of recording mainly encompasses premotor area F2vr (medial to the spur of the arcuate sulcus), a small and medial part of premotor area F5, and part of F4, as evident from Fig. 3A, the main text did not report the rationale underlying the selection of the F2vr, F4/F5 for the present study. In addition, according to the new details provided in the current version of Fig. 3A on recording sites, some of the penetrations (corresponding to about 40-45 collected units) belong to prefrontal area 8 (FEF). These cells should be removed from the premotor database.

We thank the referee for the acute observation. We have now added the reasons why we chose the recorded regions in the revised manuscript (L. 296-312). In the revised results, we removed the FEF neurons from the neuron database, conducted the same analysis, and updated all the relevant figures accordingly (Figures 3A, 3C, 3G, 3H, 4C, 4D, 4E, 4F, 5A, 5B, 5C, 5E, 5F, 6B, 6C, Figure 3-Figure supplement 1, Figure 4-Figure supplement 1, Figure 4-Figure supplement 2, Figure 5-Figure supplement 1, and Figure 5-Figure supplement 2). Note that the same conclusion holds.

The recording regions were chosen based on previous monkey electrophysiological and human studies. Please also see our response to Reviewer 1 (Question 5). The recording areas include (1) both ventral and dorsal premotor cortices; (2) both inferior (area 7) and superior (area 5) parietal lobes. Indeed, the previous studies in the parietal cortex also showed that area 7 neurons also responded to the multimodal sensory inputs. For example, Graziano and colleagues showed the visual receptive fields of the visual-somatosensory bimodal neurons in area 7 were fixed to the relevant body part, which was similar to the bimodal neurons in area 5 and premotor cortices (Graziano and Gross, 1993; Graziano et al., 2000). Our further analysis confirmed that there was no significant difference in the percentage of causal inference neurons between the subregions in the premotor and parietal cortices.

We thank the referee’s suggestions. In the revised manuscript, we have now claimed that the recording regions included both areas 5 and 7 in the parietal cortex (L. 312-315). In Figure 3A, the straight dash grey line roughly separated the dorsal and ventral parts of the premotor cortex in the middle panel. The straight dash grey line roughly indicates the middle between IPS and CS. The circular dash lines indicate the recording chambers. We have added this information in the revised manuscript (L. 338-340).

Beyond the graphical (subjective) representation, no other information is provided about the criteria adopted to identify the recording areas and sites. First, as pointed out by the reviewers, the specification that they were "determined by individual MRI atlas" does not warrant for any areal identification; second is not even vaguely informative on how frontal and parietal areas were identified. The sentence that "The location of the recording chamber on each animal was determined by an individual MRI atlas" remained unchanged in the revised version of paper, without any further details, not even providing the reference on which Atlas has been used.

We collected the structural magnetic resonance images (MRI) of three monkeys (3T, Center for Excellence in Brain Science and Intelligence Technology, Institute of Neuroscience, Chinese Academy of Sciences), while they were in an MRI-compatible Horsley-Clarke stereotaxic apparatus. The location of the recording chamber on each animal was determined by the atlas with the origin at the Ear Bar Zero (Saleem & Logothetis, 2012). The centers of implanting recording chambers were [right: 20.0 mm; forward: 10.0 mm] for the premotor cortex in Monkey N, [left: 21.9 mm; forward: 24.9 mm] for the premotor cortex in Monkey H, [right: 14.7 mm; forward: 1.1 mm] for the parietal cortex in Monkey N, and, [left: 17.0 mm; forward: 3.5 mm] for the parietal cortex in Monkey S (see Author response image 8). We have added this information in the revised manuscript (L. 891-901).

Author response image 8
The centers of implanting recording chambers in MRIs for individual monkeys.

CS, central sulcus; IPS, intraparietal sulcus. Note that the recording locations of monkey N’s premotor and parietal cortices are from different structural magnetic resonance images.

Finally, another reason of concern highlighted in the revision process refers to the lack of eye movements data, given that eye position and saccade direction exerts a well-known, although quantitatively different, influence on premotor and parietal neural activity. The authors did not refer to this critical aspect at all, nor did they discuss how eye-related signals might have influenced and eventually contaminated the reported findings.

We trained the monkeys to perform the task without their eye fixed, but the eye movement during the recording sessions was recorded. To examine whether the updating of sensory uncertainty was correlated with the uncertainty of eye position between VP and VPC (0°) tasks. We identified the eye fixation position at the target holding period. We examined the divergence of eye fixation position in VP and VPC (0°) tasks and found that there was no significant difference between these tasks (Figure 2-Figure supplement 2, Wilcoxon signed-rank test, p = 0.81).

Moreover, to examine whether the neural activity of Pcom was correlated with the eye position, we calculated the Pearson correlation coefficients between eye fixation position and VP weight. We found that there was no correlation between the VP weight and the eye fixation position at the population level for both regions (the premotor and parietal cortices) at both horizontal and vertical directions (Figure 3-Figure supplement 3, Wilcoxon signed-rank test, Premotor (horizontal): p = 0.11; Premotor (vertical): p = 0.86; Parietal (horizontal): p = 0.35; Parietal (vertical): p = 0.87). Note that the recorded eye movement data used in this analysis included 78 sessions for the premotor cortex and 45 sessions for the parietal cortex. And we added this information in the revised manuscript (L. 369-270, L. 410-415, L. 601-603, and L. 1032-1061).

Reviewer #3 (Recommendations for the authors):

The authors have addressed most reviewer comments to a sufficient degree. The work is still very dense given the large number of analyses implemented, but I have no specific suggestions for how to change this.

One remaining shortcoming is that I did not see a specific rationale for the choice of the precise recording locations in the manuscript. In reply to my previous comment, the authors have provided some rather generic text in the rebuttal, but ideally, a clear rationale for the choices should be in the manuscript.

We have added the reasons why we chose the recorded regions in the revised manuscript (L. 296-312). Please also see our response to Reviewer 1 (Question 5).

https://doi.org/10.7554/eLife.76145.sa2

Article and author information

Author details

  1. Guangyao Qi

    1. Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, China
    2. University of Chinese Academy of Sciences, Beijing, China
    Contribution
    Conceptualization, Data curation, Software, Formal analysis, Validation, Investigation, Visualization, Methodology, Writing – original draft, Writing – review and editing
    Contributed equally with
    Wen Fang
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-0479-7320
  2. Wen Fang

    Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, China
    Contribution
    Conceptualization, Data curation, Formal analysis, Validation, Investigation, Visualization, Methodology, Writing – original draft, Writing – review and editing, Funding acquisition
    Contributed equally with
    Guangyao Qi
    For correspondence
    wenfang@ion.ac.cn
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-7748-6772
  3. Shenghao Li

    1. Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, China
    2. University of Chinese Academy of Sciences, Beijing, China
    Contribution
    Data curation
    Competing interests
    No competing interests declared
  4. Junru Li

    Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, China
    Contribution
    Methodology
    Competing interests
    No competing interests declared
  5. Liping Wang

    Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, China
    Contribution
    Conceptualization, Supervision, Funding acquisition, Investigation, Writing – original draft, Project administration, Writing – review and editing
    For correspondence
    liping.wang@ion.ac.cn
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-2038-0234

Funding

National Science and Technology Innovation 2030 Major Program (2021ZD0204204)

  • Wen Fang

Shanghai Municipal Science and Technology Major Project (2021SHZDZX)

  • Liping Wang

Lingang Laboratory Grant (LG202105-02-01)

  • Liping Wang

Strategic Priority Research Programs (XDB32070201)

  • Liping Wang

National Natural Science Foundation of China (32100830)

  • Wen Fang

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Acknowledgements

We thank Florent Meyniel and Tianming Yang for their comments on the manuscript, and Xinjian Jiang, Jian Jiang, and Juntao Feng for experimental assistance. This work was supported by the National Science and Technology Innovation 2030 Major Program 2021ZD0204204 to WF, the Shanghai Municipal Science and Technology Major Project 2021SHZDZX to LW, the Lingang Laboratory Grant LG202105-02-01 to LW, the Strategic Priority Research Programs XDB32070201, the Strategic Priority Research Programs XDB32070201 to LW, and the National Natural Science Foundation of China 32100830 to WF.

Ethics

All animal procedures were approved by the Animal Care Committee of Center for Excellence in Brain Science and Intelligence Technology, Institute of Neuroscience, Chinese Academy of Sciences (Permit Number: CEBSIT-2020034).

Senior Editor

  1. Tirin Moore, Stanford University, United States

Reviewing Editor

  1. Hugo Merchant, National Autonomous University of Mexico, Mexico

Reviewer

  1. Hugo Merchant, National Autonomous University of Mexico, Mexico

Publication history

  1. Received: December 6, 2021
  2. Preprint posted: December 7, 2021 (view preprint)
  3. Accepted: October 23, 2022
  4. Accepted Manuscript published: October 24, 2022 (version 1)
  5. Version of Record published: November 8, 2022 (version 2)

Copyright

© 2022, Qi, Fang et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 774
    Page views
  • 194
    Downloads
  • 1
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Guangyao Qi
  2. Wen Fang
  3. Shenghao Li
  4. Junru Li
  5. Liping Wang
(2022)
Neural dynamics of causal inference in the macaque frontoparietal circuit
eLife 11:e76145.
https://doi.org/10.7554/eLife.76145

Further reading

    1. Neuroscience
    Kevin Vinberg, Jörgen Rosén ... Fredrik Ahs
    Research Article Updated

    Understanding the neural basis for individual differences in the skin conductance response (SCR) during discriminative fear conditioning may inform on our understanding of autonomic regulation in fear-related psychopathology. Previous region-of-interest (ROI) analyses have implicated the amygdala in regulating conditioned SCR, but whole brain analyses are lacking. This study examined correlations between individual differences in SCR during discriminative fear conditioning to social stimuli and neural activity throughout the brain, by using data from a large functional magnetic resonance imaging study of twins (N = 285 individuals). Results show that conditioned SCR correlates with activity in the dorsal anterior cingulate cortex/anterior midcingulate cortex, anterior insula, bilateral temporoparietal junction, right frontal operculum, bilateral dorsal premotor cortex, right superior parietal lobe, and midbrain. A ROI analysis additionally showed a positive correlation between amygdala activity and conditioned SCR in line with previous reports. We suggest that the observed whole brain correlates of SCR belong to a large-scale midcingulo-insular network related to salience detection and autonomic-interoceptive processing. Altered activity within this network may underlie individual differences in conditioned SCR and autonomic aspects of psychopathology.

    1. Neuroscience
    Rong Zhao, Stacy D Grunke ... Joanna L Jankowsky
    Research Article

    Neurodegenerative diseases are characterized by selective vulnerability of distinct cell populations; however, the cause for this specificity remains elusive. Here we show that entorhinal cortex layer 2 (EC2) neurons are unusually vulnerable to prolonged neuronal inactivity compared with neighboring regions of the temporal lobe, and that reelin+ stellate cells connecting EC with the hippocampus are preferentially susceptible within the EC2 population. We demonstrate that neuronal death after silencing can be elicited through multiple independent means of activity inhibition, and that preventing synaptic release, either alone or in combination with electrical shunting, is sufficient to elicit silencing-induced degeneration. Finally, we discovered that degeneration following synaptic silencing is governed by competition between active and inactive cells, which is a circuit refinement process traditionally thought to end early in postnatal life. Our data suggests that the developmental window for wholesale circuit plasticity may extend into adulthood for specific brain regions. We speculate that this sustained potential for remodeling by entorhinal neurons may support lifelong memory but renders them vulnerable to prolonged activity changes in disease.