Complementary congruent and opposite neurons achieve concurrent multisensory integration and segregation

  1. Wen-Hao Zhang
  2. He Wang
  3. Aihua Chen
  4. Yong Gu
  5. Tai Sing Lee
  6. KY Michael Wong  Is a corresponding author
  7. Si Wu  Is a corresponding author
  1. Hong Kong University of Science and Technology
  2. Carnegie Mellon University, United States
  3. East China Normal University, China
  4. Institute of Neuroscience, Chinese Academy of Sciences, China
  5. Peking University, China
8 figures and 1 additional file

Figures

Figure 1 with 1 supplement
Multisensory integration and segregation.

(A) Multisensory integration versus segregation. Two underlying stimulus features s1 and s2 independently generate two noisy cues x1 and x2, respectively. If the two cues are from the same stimulus, they should be integrated, and in the Bayesian framework, the stimulus estimation is obtained by computing the posterior p(s1|x1,x2) (or p(s2|x1,x2)) utilizing the prior knowledge p(s1,s2) (left). If two cues are from different stimuli, they should be segregated, and the stimulus estimation is obtained by computing the posterior p(s1|x1) (or p(s2|x2)) using the single cues (right). (B) Information of single cues is lost after integration. The same integrated result s^=0 is obtained after integrating two cues of opposite values (θ and -θ) with equal reliability. Therefore, from the integrated result, the values of single cues are unknown.

https://doi.org/10.7554/eLife.43753.002
Figure 1—figure supplement 1
Cue disparity information is lost after integration.

(A) Cue information is lost after integration. The information of cue 1 decreases with the extent of integration, which is controlled by κs, measuring the correlation between s1 and s2. (B) The percentage of cue information loss increases with the extent of integration.

https://doi.org/10.7554/eLife.43753.003
Congruent and opposite neurons in MSTd.

Similar results were found in VIP (Chen et al., 2011). (A–B) Tuning curves of a congruent neuron (A) and an opposite neuron (B). The preferred visual and vestibular directions are similar in (A) but are nearly opposite by 180° in (B). (C) The histogram of neurons according to their difference between preferred visual and vestibular directions. Congruent and opposite neurons are comparable in numbers. (A–B) are adapted from Gu et al. (2008), (C) from Gu et al. (2006).

https://doi.org/10.7554/eLife.43753.004
Geometric interpretation of multisensory processing of circular variables.

(A) Two von Mises distributions plotted in the polar coordinate (bottom-left) and their corresponding geometric representations (top-right). A von Mises distribution can be represented as a vector, with its mean and concentration corresponding to the angle and length of the vector, respectively. (B) Geometric interpretation of cue integration and the cue disparity information. The posteriors of s1 given single cues are represented by two vectors (green). Cue integration (blue) is the sum of the two vectors (green), and the cue disparity information (red) is the difference of the two vectors. (C–E) The mean and concentration of the integration (blue) and the cue disparity information (red) as a function of the cue reliability (C), cue disparity (D), and reliability of prior (E). In all plots, κs=50, κ1=κ2=50, x1=0 and x2=20, except that the variables are κ1=κ2 in C, x2 in D, and κs in E.

https://doi.org/10.7554/eLife.43753.005
The decentralized neural circuit model for multisensory processing.

(A) The network consists of two modules, which can be regarded as MSTd and VIP respectively. Each module has two groups of excitatory neurons, congruent (blue circles) and opposite neurons (red circles). Each group of excitatory neurons are connected recurrently with each other, and they are all connected to an inhibitory neuron pool (purple disk) to form a continuous attractor neural network. Each module receives a direct cue through feedforward inputs. Between modules, congruent neurons are connected in the congruent manner (blue arrows), while opposite neurons are connected in the opposite manner (brown lines). (B) Connection profiles between neurons. Black line is the recurrent connection pattern between neurons of the same type in the same module. Blue and red lines are the reciprocal connection patterns between congruent and opposite neurons across modules respectively. (C) The reliability of the network's estimate of a stimulus is encoded in the peak firing rate of the neuronal population. Typical parameters of network model: ω=3×10-4, Jint=0.5, Jrc=0.3Jc, Jrp=0.5Jrc, Ib and F in Equation 22 are 1 and 0.5 respectively.

https://doi.org/10.7554/eLife.43753.006
Tuning properties of congruent and opposite neurons in the network model.

(A–B) The tuning curves of an example congruent neuron (A) and an example opposite neuron (B) in module 1 under three cueing conditions. (C–D) The bimodal tuning properties of the example congruent (C) and the example opposite (D) neurons when cue 1 has relatively higher reliability than cue 2 in driving neurons in module 1, with α1=0.58α2, where αm is the amplitude of cue m given by Equation 22. The two marginal curves around each contour plot are the unimodal tuning curves. (E–F) Same as (C–D), but cue 1 has a reduced reliability with α1=0.12α2. (G–H) The histogram of the differences of neuronal preferred directions with respect to two cues in module 1 (G) and module 2 (H), when the reciprocal connections across network modules contain random components of roughly the same order as the connections. Parameters: (A–B) α1=0.35U0, and α2=0.8U0; (C–F) α2=1.5U0 in (C–D) while α1=0.1U0 in (E–F). Other parameters are the same as those in Figure 4.

https://doi.org/10.7554/eLife.43753.007
Figure 6 with 2 supplements
Optimal cue integration and segregation collectively emerge in the neural population activities in the network model.

(A) Illustration of the population response of congruent neurons in module 1 when both cues are presented. Color indicates firing rate. Right panel is the temporal average firing rates of the neural population during cue presentation, with shaded region indicating the standard deviation (SD). Note that the neuron index θ refers to the preferred direction with respect to the direct cue conveyed by feedforward inputs. (B) The position of the population activity bump at each instance is interpreted as the network’s estimate of the stimulus, referred to as z1, which is decoded by using population vector. Right panel is the distribution of the decoded network’s estimate during cue presentation. (C–E) The temporal average population activities of congruent (blue) and opposite (red) neurons in module 1 (top row) and module 2 (bottom row) under three cueing conditions: only cue 1 is presented (C), only cue 2 is presented (D), and both cues are simultaneously presented (E). (F–I) Comparing the estimates from congruent and opposite neurons in module 1 with the theoretical predictions, with varying cue intensity (F), with varying cue disparity (G), and with varying reciprocal connection strength between modules (H and I). Symbols: network results; lines: theoretical prediction. The theoretical predictions for the estimates of congruent and opposite neurons are obtained by Equations 4 and 7. Parameters: (A–E) α1=α2=0.35U0; (F) α2=0.7U0; (G–I) α1=α2=0.7U0, and others are the same as those in Figure 4. In (F–H), x1=0, x2=20 and in (I), x1=0, x2=160.

https://doi.org/10.7554/eLife.43753.008
Figure 6—figure supplement 1
Illustration of decoded joint distributions from congruent and opposite neurons.

Illustration of decoded joint distributions from congruent and opposite neurons respectively in two network modules under three cueing conditions, with the marginal distributions plotted in the margin plot. The joint distribution from congruent neurons in two network modules encode the posterior p(s1,s2|x1,x2), while the one from opposite neurons in two modules represent the cue disparity information pd(s1,s2|x1,x2). Color denotes the cueing condition and type of neurons (see the legend for details). Parameters are the same as those in Figure 6A–E in main text.

https://doi.org/10.7554/eLife.43753.009
Figure 6—figure supplement 2
Test of network’s performance.

Comparison of the mean and concentration of network’s estimate with theoretical prediction. (A-B) The mean (A) and the concentration (B) of the congruent neurons in two network modules versus the theoretical prediction. (C-D) The same as (A-B) but for opposite neurons. Parameters: Jrc[0.3,0.4]Jc, Jrp[0.1,0.7]Jrc, α1,α2[0.1,1.5]U0, Jint{0.05,0.25,0.5}, x1 and x2 are both uniformly distributed in (−180°, 180°], and others are the same as those in Figure 4 in main text.

https://doi.org/10.7554/eLife.43753.010
Concurrent multisensory processing with congruent and opposite neurons.

(A–B) Accessing integration versus segregation through the joint activity of congruent and opposite neurons. (A) The firing rate of congruent and opposite neurons exhibit complementary changes with cue disparity x1-x2. (B) The decision boundary of the competition between congruent and opposite neurons changes with read out weight from congruent Wcong and opposite neurons Woppo. It is given by the value of x1-x2 at which Wcongrmc=Woppormo. Dashed line is when Wcong=Woppo, the decision boundary is at 90°. (C–D) Recovering single cue information from two types of neurons. (C) Illustration of recovering through the joint activities of congruent (blue) and opposite (red) neurons under the combined cue condition. We decoded the estimate from congruent and opposite neurons respectively, and then vector sum the decoded results recovering the single cue information. (D) Comparing the recovered mean of the stimulus given the direct cue with the actual value. Parameters: those in (A–B) are the same as those in Figure 6A, and those in D are the same as those in Figure 6—figure supplement 2.

https://doi.org/10.7554/eLife.43753.011
Figure 8 with 1 supplement
Discrimination of cue disparity by single neurons.

(A) The tuning curve of an example congruent (blue) and opposite (red) neuron with respect to cue disparity x1-x2. In the tuning with respect to cue disparity, the mean of two cues was always at 0°, that is x1+x2=0, while their disparity x1-x2 was varied from −32° to 32° with a step of 4°. The two example neurons are in network module 1, and both prefer 90° with respect to cue 1. However, the congruent neuron prefers 90° of cue 2, while the opposite neuron prefers −90° with respect to cue 2. Error bar indicates the SD of firing rate across trials. (B) The neurometric function of the example congruent and opposite neuron in a discrimination task to determine whether the cue disparity x1-x2 is larger than 0° or not. Lines are the cumulative Gaussian fit of the neurometric function. (C) Averaged neuronal discrimination thresholds of the example congruent and opposite neurons. Parameters: α1=0.25U0, α2=0.8U0, and others are the same as those in Figure 4.

https://doi.org/10.7554/eLife.43753.012
Figure 8—figure supplement 1
Discrimination of heading direction by single neurons.

Discrimination of heading direction by single neurons. The directions of the two cues are always the same. (A and D) The tuning curve of the example congruent (A) and opposite (D) neurons with cue direction under three cueing conditions. The example neurons are the same as the ones shown in Figure 8 in main text. (B and E) The neurometric function of the example congruent (B) and opposite (E) neurons under three cueing conditions. Smooth lines show the cumulative Gaussian fit of the neurometric functions. (C and F) Average neuronal discrimination thresholds of the example neuron in three cueing conditions compared with the theoretical prediction. Parameters are the same as those in Figure 8 in the main text.

https://doi.org/10.7554/eLife.43753.013

Additional files

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Wen-Hao Zhang
  2. He Wang
  3. Aihua Chen
  4. Yong Gu
  5. Tai Sing Lee
  6. KY Michael Wong
  7. Si Wu
(2019)
Complementary congruent and opposite neurons achieve concurrent multisensory integration and segregation
eLife 8:e43753.
https://doi.org/10.7554/eLife.43753