Abstract
Summary
Although recent studies suggest that activity in the motor cortex, in addition to generating motor outputs, receives substantial information regarding sensory inputs, it is unclear how sensory context adjusts the motor commands. Here, we recorded population neural activity in the motor cortex via microelectrode arrays while monkeys performed flexible manual interceptions of moving targets. During this task, which requires predictive sensorimotor control, the activity of most neurons in the motor cortex encoding upcoming movements, was influenced by ongoing target motion. Single-trial neural states at movement onset formed staggered orbital geometries, suggesting that target speed modulates pre-movement activity in an orthogonal manner. This neural geometry was further evaluated with a representational model and a recurrent neural network (RNN) with task-specific input-output mapping. We propose that sensorimotor dynamics can be derived from neuronal mixed sensorimotor selectivity and dynamic interaction between modulations.
Introduction
The motor cortex, a brain region generating motor commands, is widely known for its relation to movement kinetics1,2 and kinematics3–5. Moreover, it has been reported to also carry substantial sensory information6–13. Activity in the motor cortex can be strongly influenced by reference frame14, hand trajectory15, and stimuli for target selection16; there even exists evoked activity during visual replay related to the tracking movements17. However, it is unclear how such sensory input contributes to pre-movement activity in the motor cortex.
From the dynamical systems perspective, preparatory population activity develops toward a movement-specific optimal subspace to set initial states seeding the motor generation18–20. However, sensory information may change the distribution of neural states during movement in an orthogonal fashion without affecting the motor output.
To investigate the neural dynamics shaped by concurrent sensorimotor signals, we recorded population activity in the primary motor cortex (M1) from monkeys performing a flexible manual interception task. Unlike previous studies constraining interception at a fixed location9,21, our task demands predictive spatiotemporal mappings to displace a body effector to a trial-varying location. We found that the activity of most neurons was jointly tuned to both reach direction and target motion via directional selectivity shifts, gain modulations, offset adjustments, or their combinations. Strikingly, such mixed sensorimotor selectivity exists throughout the entire trial, in contrast to the gradient of sensory-to-motor tuning from cue to movement epochs in posterior parietal cortex (PPC) that we recently reported 22,23. Principal component analysis (PCA) of the neural population revealed a clear orbital neural geometry in low-dimensional space at movement onset: The neural states were distributed in reach-direction order, and formed ringlike structures whose slopes were determined by target-motion conditions. Neuronal simulation indicates that these characteristics of neural population dynamics could be derived from the mixed sensorimotor selectivity of single neurons. A recurrent neural network (RNN) trained with proper input-output mappings offers insights into the relationship between neuronal modulation and neural geometry in a dynamical system. We propose that this sensory modulation occurs at both single-neuron and population levels as a general element of neural computations for predictive sensorimotor control.
Results
Mixed tuning of M1 single neurons during flexible manual interception task
Three monkeys (Macaca mulatta, C, G, and D, male, weight 7 to 10 kg) were trained to perform a delayed manual interception task (Figure 1A), which was modified from that utilized in our recent studies 22,23. To initiate a trial, the monkey needed to hold the center dot of a standing touch screen for 600 ms. Then, a peripheral target appeared, either stationary or rotating around the center dot at a constant angular speed. The monkey was required to wait during a random delay (400-800 ms) until the central dot went dark (GO) and then to immediately reach to the target. Once the monkey touched the screen (Touch), the target stopped and another dot showed the touching location, in red for a successful interception or in blue for a failure. The error tolerance between the target and the touch location (reach endpoint) was 3 cm. There were five targetmotion conditions, consisting of clockwise (CW) conditions - 240 °/s and -120 °/s, counterclockwise (CCW) conditions 120 °/s and 240 °/s, along with 0 °/s (static) condition. We define the target-speed magnitude as TSmag (0 °/s, 120 °/s, and 240 °/s) and the target-speed direction as TSdir (CCW and CW). These target-motion conditions were interleaved trial by trial, and initial location of the target was random. The reach endpoints of a well-trained monkey distributed uniformly around the circle (Figure 1B, Rayleigh’s test: p=0.36; data from monkey C, 772 correct trials in one session). Because the reach direction was defined as the angle of the reach endpoint, for simplicity, we divided the circular space equally into eight sectors (45° per each), and grouped trials according to the eight reach directions and five target-motion conditions.
We recorded neural data with Utah arrays from monkeys C, G, and D (implanted sites are shown in Figure 1C, and all datasets listed in Table S1) and hand trajectories from monkeys C and G (Figure S1), during the interception task. The hand trajectory was launched to the final interception position. The temporal profiles of hand velocity were unimodal bell-like curves and similar across target-motion conditions (correlation coefficients is 0.97±0.05 for monkey C and 0.99±0.02 for monkey G).
Notably, we found that neuronal directional tuning during motor execution (MO±100 ms, MO for movement onset) was modulated by target motion, according to our statistical criteria mainly in three ways: preferred direction (PD) shift, gain, and offset addition (Figure 1D and Figure S2-S4, Methods). PD-shift neurons had their PDs shifted in moving-target conditions compared to the static-target condition. As illustrated by an example neuron (Figure 1D, PD shift), whose PDs corresponding to CCW conditions (red and yellow) and CW conditions (blue and purple) exhibited obvious differences, TSdir rather than TSmag dominated this modulation. Gain-modulation neurons exhibited reach-direction tuning multiplied by target speed: while their directionality remained invariant, the neuronal responses at PD differed across target-motion conditions. This modulation was dominated by TSdir as well. The turning curves of the example neuron (Figure 1D, Gain) had higher responses at PD in CW conditions (blue and purple) than in the others (green, yellow, and red), indicating a varying tuning depth for reach direction. Neurons with addition modulation underwent changes of offset activity induced by target speed (both TSdir and TSmag). As shown by the example neuron (Figure 1D, Addition), this effect was almost equal for all reach directions.
The activity of these example neurons could be well explained by PD shift, gain, and additive models (Figure 1E, Methods), respectively. Nevertheless, it was difficult to classify all neurons with mixed sensorimotor selectivity into one of these three groups exclusively, because many of them experienced a mixture of two or three of above modulations. Moreover, the proportions of three groups were almost equal (Table 1). We found that the adjusted R2 of a full model (0.64 ± 0.21, mean ± sd.) was larger than that of the PD shift (0.59 ± 0.23), gain (0.56 ± 0.21), and additive (0.47 ± 0.24) models for monkey C (95 neurons), as well as for all three monkeys (rank-sum test, p<0.01, Figure S5). These target-motion modulations of neuronal directionality suggested the participation of sensory signals in shaping neural dynamics during interception execution.
Single neurons were classified by comparing firing rate patterns between target-speed conditions (more details in Methods). The first column is the mean±std. of recording unit number in one monkey across sessions. The other columns give the ratio of target-speed modulation unit across sessions.
Coding of sensory and motor information in neural populations
To quantify target motion information embodied in neural response in motor cortex, we performed a series of decoding analyses on the neural data from monkey C (n=95, 772 correct trials). To begin with, neural data were utilized to train a support vector machine (SVM) classifier for targetmotion conditions (one in five), and another for reach directions (one in eight; Methods). As Figure 2A shows, the decoding accuracy of the target-motion condition increased quickly and peaked at over 70% around GO, while the decoding accuracy of reach direction climbed and reached a plateau of about 80% before MO. These results are similar to those of the other two monkeys and the pseudo-populations (Figure S6), suggesting that target motion information existed in M1 throughout almost the entire trial. According to the demixed PCA results, while reach direction occupied top components, there was interaction components that should not be neglected and even explained more variance than target speed components (Figure S7)24.
Then, another decoding analysis was implemented to probe the potential interaction between reach-direction and target speed during execution period. We trained a reach-direction decoder (one in eight) to check if the decoder of one certain target-motion condition could be transferred to other conditions (Figure 2B left). It turned out that the performance of the transferred decoder deteriorated more significantly for TSdir. (CCW vs. CW, mean ± sd. of accuracy 0.26±0.06), compared with that for TSmag (120 vs. 240, 0.50±0.06, paired t-test, p<0.01) and for target’s state (static vs. motion 0.55±0.06, paired t-test, p<0.01). This result suggests that the coding of reach direction was rather sensitive to TSdir, but containing similarities for static and moving targets. As control, we also compared the neural coding rules across different reach-direction conditions. We trained a target-speed decoder (one in five), and similarly checked the transferred decoding accuracy (Figure 2B right). We observed that the target-speed decoder was locked with reach direction, as the transferred decoding accuracy diminished with increasing difference of reach direction. These results qualitatively imply the interaction as that target speeds affected the reach-directional tuning, especially by TSdir..
To explore how sensory information influences neural dynamics while preserving motor output, we performed PCA on the normalized population activity. We obtained the trial-averaged neural trajectories (5 target speeds x 8 reach directions 40 conditions) after TO or after GO (Figure S8A). The distance between neural trajectories grouped by reach-direction conditions was larger than the distance grouped by target-speed conditions especially after GO (Figure S8B), consistent with the demixed PCA results that M1 primarily encoded reach direction rather than target speed. In addition, the distance of neural trajectories between CCW and CW was much larger than the distance between 120°/s and 240 °/s conditions, indicating that TSdir dominated the target-speed effect, agreeing with the decoding results (Figure S8C).
However, these neural trajectories were not yet the ideal description, because they were shaped mostly by time. So, to highlight the proposed target-speed effect on reach direction, we focused on four key time windows and snapshotted the neural trajectory as neural state to extract the coding rule at single trial level and in a geometric view. Here, we define the “neural states” as the projection of single-trial data during a specific time bin on principal components25–27. At MO, the first two principal components (PCs) of the neural states explained the most variance ([24.8%, 13.8%]) and were most related to reach direction (the goodness of fitting reach direction, , Figure 2C). While reach direction was represented by the first two PCs at GO and during movement execution, target speed influenced the tuning pattern of the first three PCs (Figure 2D), suggesting that target motion affects M1 neural dynamics via a topologically invariant transformation.
Orbital neural geometry in latent space
We visualized the neural states in the low-dimensional space spanned by the above three PCs at MO. The projections of the single-trial neural state onto the PC1-PC2 subspace clustered, or rather distributed, in reach-direction order (Figure 3A top and Figure 3B left). Interestingly, the neural states formed ringlike structures under single target-motion conditions and the fitted ellipses exhibited concentric shapes (the fitting goodness of ellipses, R2 = 0.92 ± 0.01, mean ± sd., see Methods). Moreover, these ellipses tilted in condition-dependent angles, which is particularly evident in the PC2-PC3 subspace (Figure 3A bottom and Figure 3B right). Note that here we performed an linear transformation on all resulting neural state points to make the ellipse of the static condition orthogonal to the z-axis for better visualization.
Next, we quantified the spatial features of these ellipses by calculating the tilting angles, which were between the normal vectors of the moving-target and static-target conditions. Strikingly, this tilting angle was linearly correlated with target speed (both TSmag and TSdir), and the relationship was robust in nine datasets from three monkeys (Figure 3C and Figure S9).
Given these results, we propose that this orbital neural geometry epitomizes the sensorimotor dynamics of M1 at the population level. The sensory input can regularly modulate neural states in an orthogonal dimension (PC3), without interfering with motor generation (in PC1 and PC2).
Population neural geometry relies on neuronal tuning
We then asked whether a group of single neurons with a certain type of mixed sensorimotor selectivity could exhibit the orbit neural geometry.
For this purpose, three neuronal models were constructed for the three typical modulations described above (PD shift, gain, and addition, see Figure 1D and Figure S2-S4). We ran a simulation with these representational neuronal models (Figure 4A). Here, each group consisted of 300 model neurons with their PDs uniformly distributed, and was solely modulated as PD-shift, gain, or addition (Methods). The resulting neural geometry of the three simulation groups showed distinct features (Figure 4B): The single-condition ellipses were inclined with target-motion-dependent angles in the PD-shift and gain groups, similar to the real neural data, but the ellipses in the addition group were layered in parallel. The reach-direction clusters in the first two PCs were conservative in the gain and addition group, but not in the PD-shift group. These results indicate that the neural states of the real data mainly exhibit the geometry feature of the gain modulation group. Nonetheless, we found that a population with a uniform mixture of all three modulations was able to simulate the neural geometry as well (Figure 4C).
Comparing with the static-target condition, we calculated relative rotation angle and tilting angle between ellipses along with the vertical shift of neural states, in order to quantify the simulated structure (Figure 4D). The results show that the real data yielded a smaller rotation angle than the PD-shift group, a smaller vertical shift than the additive group, but larger tilting angles than all models. The mixed group had the most similar tilting angle, though with moderate performance in rotation angle and state shift.
These simulations suggest that the existence of PD-shift and additive modulation would not disrupt the neural geometry that is primarily driven by gain modulation; rather it is possible that these three modulations support each other in a mixed population.
Recurrent neural network provides dynamic insights
To infer how such modulated subpopulations would interact with each other in a dynamical system, we trained 100 RNN models with random weight initialization (Figure 5A; see Methods). The inputs included motor intention, target location, and a Go signal. Motor intention was defined as an abstract motor command predicted to compensate for sensorimotor delays28, and could be provided by the PPC29,30, here simplified as the interception location. The network was to generate hand velocity after MO. For a fixed validation set of 500 trials, these trained network models performed well (mean square error of the reach endpoint and the target was 0.0046 ± 0.0027, a. u., mean ± sd., while the radius of target motion was 0.15).
In these network models, we found comparable features of nodes. First, most activated nodes (for example, Figure 5B) could be classified into the above-mentioned three modulations (Table 1) by the same statistical standard as for the real neural data. Second, the states reduced from node population activity were arranged in a way resembling the actual neural geometry at MO (the fitting goodness of ellipses, R2 = 0.98 ± 0.05, mean ± sd.; Figure 5C). The tilting angles followed the same pattern as suggested by the actual results (Figure 5D).
With these RNN models, we perturbed the connection between different modulation groups. The connections within and between gain modulation and additive modulation were stronger than the others (Figure 5E), however, further strengthening the connection from gain to additive nodes or within additive nodes would disorder the orbital structure more significantly (Figure S10A and Table S2). We speculate that the influence, and perhaps the formation, of modulations both relate to the robust input and output weight pattern across models (Figure S10B, C). These results support strong and similar roles of gain and additive nodes, but what is even more important is that the three modulations interact each other, so the PD-shift nodes should not be neglected.
Discussion
To reveal how sensory context influence neural dynamics during movement, we recorded M1 population activity from monkeys performing a flexible manual interception that is highly dependent on predictive sensorimotor transformations. Single-neuron activity showed that the movement tuning of M1 neurons varied with target-motion conditions in complicated ways, including PD shift, gain modulation, addition, or their mixtures. Unsupervised dimensionality reduction (PCA) on population activity revealed an orbital neural geometry, where neural states of single trials spontaneously formed ellipses and were tilted according to target-motion conditions. Such a geometry, which could be simulated with a group of representational neurons for gain modulation alone, also emerged in the RNN with appropriate input-output mappings. As suggested by a dynamic and connected population as the RNN, the interactions between modulations were sophisticated in preserving the encoding of motor output. These results reveal sensory modulation of pre-movement neural dynamics at the single-neuron and population levels, and bridge the neuronal mixed sensorimotor selectivity and a low-dimensional neural geometry, compactly representing the sensorimotor co-function.
Previous studies have showed that M1 activity is seldom evoked by purely visual stimuli, with only weak tuning8,31,32, and has transient responses to behaviorally relevant visual cues7,10,33–35. In this study, the modulated neural dynamics, though distinguished according to target-motion conditions, is ultimately shaped for motor generation rather than sensory representation. It is a pity that we did not record sufficient muscle data during interception to rule out the representation of the motor cortex for kinetic variables such as force1,27,36 , but we believe that this would not invalidate our main conclusions.
Previous studies have revealed interactions between motor kinematic variables such as reach direction and hand velocity15,37,38. Although the effect of arm-dependent speed (hand velocity) on reach-direction tuning may exist, it cannot fully explain neural activity during interception. First, we found hand velocity to vary with TSmag, but the largest gap in neuronal activity was related to TSdir. Second, for a new dataset with trials resampled to ensure similar distributions of hand velocities in five target-motion conditions, the orbital neural geometry remained similar and the target-motion gain model provided a better explanation compared to the hand-velocity gain model (Figure S11).
In the present study, we focused on the orbital neural geometry by taking ‘snapshots’ of neural activities with PCA. In addition to unsupervised PCA, we tried supervised dimensionality reduction methods like dPCA24. However, the dPCA results (Figure S7) showed that the condition-independent temporal component explained the largest variance, followed by the component of reach direction, whereas the interactive and target-speed components only covered very small proportions. The dPCA did not distinguish a target-speed dimension, probably because reach directions were continually distributed around a circle and target speed was primarily mixed with reach directionality.
Our simulations with neuronal models and RNN demonstrate that differences in modulations and the interaction between them can be essential. In fact, modulations in the form of PD shift and gain have been widely found in the motor cortex and PPC regions contributing to sensorimotor transformation. For example, motor cortex neurons experience gain and PD shift modulation by arm posture36,39,40. Neurons in PPC have eye and hand gain fields for a visually-guided reach plan41–43, integrating target position related to gaze and hand position to form motor intention29,44,45. Therefore, we conclude that three types of modulations should all be involved in sensorimotor computation in predictive motor control. In addition, recent studies show that interactions of PPC-motor cortex circuity are involved in motor planning, spatial transformation and motor selection46–48. The PPC-M1 circuit, as a key part of cortico-subcortical networks for the predictive sensorimotor control49, is a topic for future studies.
STAR Methods
Key resources table
Resource availability Lead contact
Further information and requests for resources and reagents should be directed to and will be fulfilled by the Lead Contact, Dr. He Cui (hecui@cibr.ac.cn).
Materials availability
This study did not generate new unique reagents
Data and code availability
The example datasets and code have been deposited in Mendeley Preview, and will be made public as soon as the article is officially published.
Experimental model and study participant details
Monkey
Three adult male rhesus macaques (monkey C, D, and G, Macaca mulatta, 7-10 kg) were used in this study. The monkeys sat in a primate chair to perform the task. The stimuli were back projected onto a vertical touch screen (Elo Touch system; sampling at 100 Hz, spatial resolution <0.1 mm) about 30 cm in front of the monkeys. Hand trajectory was tracked by an optical camera (VICON Inc.) with an infrared marker on the fingertip. All animal maintenance and procedures were in accordance with NIH guidelines and approved by the Institutional Animal Care and Use Committee (IACUC) of Institute of Neuroscience, CAS.
Method details
Task and behavior
The monkeys were trained to perform a flexible manual interception task in a dark room. The task paradigm was modified based on the visually-guided reaching interception task in a previous study 22. At the beginning, the monkey held the green center dot of a touch screen for 600 ms to initiate a trial (Figure 1A). Then, a green target dot appeared at a random location, moving in a circular path centered on the center dot. The center dot turned dark as a GO cue after a random delay (400-800 ms); then the monkey could intercept the target at any moment within 150-800 ms after the GO cue. Once any peripheral location was touched, the target stopped. The tolerance range of the touch endpoint for correct trials was within 3 cm of the target. The monkey would be rewarded with juice after each correct trial. Targets moving clockwise (CW; -240 °/s, -120 °/s) and counterclockwise (CCW; 120 °/s, 240 °/s), as well as a stationary target (0°/s), were pseudo-randomly interleaved trial by trial. Additional target speeds (-360 °/s, -180 °/s, 180 °/s, 360 °/s added) were introduced in subsequent sessions (Table S1).
Data collection
After the monkeys were adequately trained for the task (successful rate > 90%), head-posts were implanted stereotaxically under anesthesia (induced by 10 mg/kg ketamine, then sustained by 2% Isoflurane). After few weeks of recovery and adaptation, the monkeys were implanted with Utah microelectrode arrays (Blackrock Microsystems, Salt Lake City, UT) in the motor cortex of the hemisphere contralateral to the handedness (Figure 1C, 128-channel array for monkey C, 96-channel array for monkey G and D). The location of recording areas were identified by Magnetic Resonance Imaging (MRI) and cortical sulcus features. Neuronal activity was recorded via a Blackrock Microsystem 256-channel recording system, sampled at 30 kHz. We collected 85 ± 16, 45 ± 8, 98 ± 18 well-isolated units from monkey C, G, and D across sessions (mean ± sd. more details in Table S1), respectively.
Peri-stimulus time histograms (PSTHs)
The PSTHs and spike rasters of single neurons are shown in Figure 1D and Figure S2-S4. All trials were classified into forty conditions, eight reach-endpoint sectors by five target-motion conditions. Condition-averaged firing rates were calculated with 50-ms bins and smoothed with a Gaussian kernel (standard deviation = 20 ms). The standard error of firing rates was estimated from the 10 bootstrap samples in the trials of corresponding condition.
Quantification and statistical analysis
Classification of neuronal tuning properties
To classify target-motion modulation for single neuron reach-direction tuning, we used the direction-averaged firing rate around movement onset (MO ± 100ms) to calculate three indexes for each target-motion conditions: the preferred direction (PD), the tuning depth, and the offset activity, and classified PD shift, gain, and addition groups in Table 1. The PD of each neuron for different target-motion conditions was calculated by the weighted sum of averaged neuronal firing rates in eight reach-direction sectors. The tuning depth of each neuron was determined by the range (max - min) of firing rates in corresponding target-motion conditions. The offset activity of each neuron was calculated by the mean firing rate (MO ± 100 ms) of all target-motion conditions.
Based on the PD, tuning depth and offset activity of single neurons, a neuron was classified as ‘PD shift’, if its PDs were significantly different between interception and static-target condition (Watson-Williams test in circular data, CircStat by50); a neuron was classified as ‘gain’ if its tuning depths were significantly different between interception and static-target condition (two-tailed Wilcoxon signed-rank test, p<0.05); a neuron was classified as ‘offset’ if its offset activities were significantly different between interception and static-target condition (two-tailed Wilcoxon signed-rank test, p<0.05).
Population decoding
The population activity of the motor cortex was used to decode target motion and reach direction by support vector machine (SVM). Neuronal firing rate was soft-normalized as
where raw firing rates was divided by the range of firing rates plus five 18. We trained two SVM classifiers (MATLAB function ‘fitcecoc’, 10-fold cross-validation) to decode reach direction (one in eight) and target motion (one in five) of single trials in 100-ms window sliding with 50-ms step (Figure 2A). The temporal decoding was repeated ten times to obtain the mean and standard deviation of decoding accuracy.
We tested the generalization of reach-direction and target-motion decoders (SVM, MATLAB function ‘fitcecoc’) in different conditions during execution period (MO ± 100 ms, Figure 2B). The decoder predicted single-trial reach direction (45°, 1/8 chance level) or target motion (1/5 chance level) across conditions based on normalized population activity. The reach-direction decoder (Figure 2B left), which was trained by trials in a set of target-motion conditions, was tested with trials from other target-motion conditions (CCW vs. CW, or 120 vs. 240, or static vs. motion, randomly selected without replacement 100 trials from corresponding training-test datasets), and the training-test decoding was repeated 1000 times to compare the distribution of accuracy with paired t-test. The target-speed decoder (Figure 2B right), which was trained by trials in a given reach-direction condition within 45°, was tested with trials from other reach-direction conditions.
Neural state
The population activity was stored in NKT datasets, where N, K, and T denote the number of neurons, trials, and time bins, respectively. We averaged T dimension of neural activity in a 100-ms bin (e.g., the two 50-ms bins around MO), and normalized neuronal dimension by Z-score (MATLAB function ‘zscore’) to get a K*N dataset. After preprocessing, we used PCA to reduce the dimension from K*N to K*C (C is the number of principal components, first two and first three PCs), and fitted the PCs with reach direction (PC = a1*cos(θ)+a2*sin(θ)+c) and target speed in Figure 2. Neural states of single trials were colored for target speed or reach direction and fitting ellipses with first two PCs (Figure 2D) and first three PCs (Figure 3) by MATLAB package ‘MatGeom’51. To show the relative position of ellipses in the best viewing angle, we used an isometric affine transformation to globally map all datapoints on new axes while preserving the proportional relation of distances between points. After this linear transformation, the azimuth and elevation of ellipses changed slightly, but the tilting angle between ellipses remained constant (Figure 3). The tilting angles, rotation angles, and state shift were calculated between ellipses of target-motion conditions and static-target condition in each session, defining CW as negative angles and CCW as positive angles. A set of tilting angles were obtained from corresponding conditions in one dataset, and the linear regression model was used to fit all ellipses angles (θ) and target speeds (sp.) in nine sessions.
Fitting and simulating single-neuron activity
We used PD shift, gain, offset, and full models to fit neuronal activity, basing on a cosine tuning model3. Neurons are fitted by single-trial data. We introduced a special sigmoid function to fit the nonlinear target-speed effects because target-speed direction (CCW vs. CW) has a stronger effect than target-speed magnitude (120 vs. 240)52,53. The gain and additive models refer to hand-velocity gain studies37,38,54.
In the gain model, the target-speed effects on the amplitude of cosine tuning are denoted as:
where FR is the firing rate at movement onset (MO ± 100 ms). θ and sp. are the reach direction and target speeds, respectively; θpd is the fitted preferred direction of the neuron; a1, a2, c1 are constants to be fitted.
In the additive model, the target speed adjusts the offset activity, as:
with similar symbols to the gain models, plus the new constant a3.
In the PD shift model, the target-speed effects on PDs are represented as:
with the similar symbols to the above model.
The full model integrates all the three of the above effects:
with constants a1, a2, a3, a4, a5.
We fitted neuronal activity with these four models (MATLAB ‘fit’ function), and compared the fitting goodness with adjusted R-squares , where n is the trial number, and p is the degree of the polynomial).
Simulations with model neurons were based on three models to investigate the relationship between neuronal tuning (Figure 4A) and population neural geometry (Figure 4B). We first built three model neuron groups (each n=300) and performed PCA to get the neural state. Then, we repeated this in a mixed group, with 100 each of PD shift, gain and additive model neurons (Figure 4C). Model neurons had three components: cosine-tuning for reach direction (θpd,n, PD uniform distribution around circle for each neuron n), Gaussian temporal profiles (t=1:200, σ=30, peak time tµ,n ∈ N(100, 10), the 100-th bin is MO, random distribution for tµ,n of neurons), and distinct target-motion modulation for each group. We designed five target speed values (sp. = [1,2,3,4,5]) and 64 reach directions , for 320 trials in total55.
The gain-model neuron has a nonlinear function with target-speed effects on the amplitude:
where gn is a random target-motion gain within [0:1] for different neurons, and modulates targetmotion gain in a sigmoidal function.
The PD-shift model neuron has target-speed change in PD:
where sn is a random value within [0:1.5] for different neurons, and contributes to a sigmoidal function with target-motion shift on neuronal PD (θpd,n).
The additive model neuron has target-speed effects in offset:
where gn is also a random value within [0:1] for different neurons, and adjusts the offset activity in a sigmoidal function with target-motion.
The firing rates of three groups model neuron (K*N*T, 320*300*200) were averaged at MO (mean T = [50:150]) to get the K*N (320*200) dataset for PCA. As with the real neural data, we selected the first three PCs (K*C, 320*3) to derive the simulated neural state shown in Figure 4.
RNN training
For the inputs of RNN, motor intention appeared from MO-50 ms to MO, and was represented as constant variables in the form of two-dimensional Cartesian coordinates; target location was designed as time varying two-dimensional Cartesian coordinates of the target throughout the entire trial; the Go-signal was a step function jumping from 0 to 1 at GO. The RNN consisted of 200 hidden units, and output hand velocity for accurate interception after MO.
The RNN nodes evolved according to a standard dynamic differential equation:
where τ is a time constant (here 50 ms), x is the activity, r is the firing rates, and u means the combined inputs. The connection matrix J of hidden layer is initiated as random in a normal distribution (mean = 0, , g = 1.5, N = 200), and the matrix B denotes the connection between inputs and hidden units, initiated as zero matrix. The output z is obtained by
where W is the read-out weight, and is expected to reproduce the desired hand velocity generated by bell-shaped physical equation56. W was also initiated as zero matrix. During training, the loss function was:
where e is the mean squared error of z and training target. r1 is regularity 57, denoting the magnitude of the nodes’ activity and is calculated as activity squared summed across time. For constants, α = 1e − 7. The optimization was realized with optim.Adam() based on PyTorch, the learning rate was 0.001.
The classification of nodes’ modulation type was finished with the same procedure for real data as above. When perturbating the connection between certain types of modulations, the selected connection was multiplied with 1.5. This perturbation was retained for the entire trial. The perturbed network was tested with the same test set of 500 trials as for the intact network.
Acknowledgements
We thank C. Li, J. Malpeli, C. Zheng, and R. Zheng for helpful comments and discussions; C. Guan for veterinary assistance; and P. Ding, L. Du, and Z. Xiao for administrative support. This work was supported by the National Key R&D Program (Grants 2020YFB1313400 and 2017YFA0701102), National Science Foundation of China (Grants 31871047 and 31671075), and CAS Strategic Priority Research Program (Grant XDB32040103).
Declaration of interests
The authors declare no conflict of interests.
References
- 1.Relation of pyramidal tract activity to force exerted during voluntary movementJ. Neurophysiol 31:14–27https://doi.org/10.1152/jn.1968.31.1.14
- 2.Modeling the motor cortex: Optimality, recurrent neural networks, and spatial dynamicsNeurosci. Res 104:64–71https://doi.org/10.1016/j.neures.2015.10.012
- 3.On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortexJ. Neurosci 2:1527–1537https://doi.org/10.1523/JNEUROSCI.02-11-01527.1982
- 4.Perspectives on classical controversies about the motor cortexJ. Neurophysiol 118:1828–1848https://doi.org/10.1152/jn.00795.2016
- 5.From Parametric Representation to Dynamical System: Shifting Views of the Motor Cortex in Motor ControlNeurosci. Bull https://doi.org/10.1007/s12264-022-00832-x
- 6.Afferent properties of periarcuate neurons in macaque monkeysII. Visual responses. Behav. Brain Res 2:147–163https://doi.org/10.1016/0166-4328(81)90053-X
- 7.Fast ballistic arm movements triggered by visual, auditory, and somesthetic stimuli in the monkey. I. Activity of precentral cortical neuronsJ. Neurophysiol 50:1343–1358https://doi.org/10.1152/jn.1983.50.6.1343
- 8.Effects of Optic Flow in Motor Cortex and Area 7aJ. Neurophysiol 86:1937–1954https://doi.org/10.1152/jn.2001.86.4.1937
- 9.Neural responses in motor cortex and area 7a to real and apparent motionExp. Brain Res 154:291–307https://doi.org/10.1007/s00221-003-1664-5
- 10.Cue to action processing in motor cortex populationsJ. Neurophysiol 111:441–453https://doi.org/10.1152/jn.00274.2013
- 11.Sensing with the Motor CortexNeuron 72:477–487https://doi.org/10.1016/j.neuron.2011.10.020
- 12.The neural mechanisms of manual dexterityNat. Rev. Neurosci 22:741–757https://doi.org/10.1038/s41583-021-00528-7
- 13.Activity in Primary Motor Cortex Related to Visual FeedbackCell Rep 29:3872–3884https://doi.org/10.1016/j.celrep.2019.11.069
- 14.Dorsal Premotor Neurons Encode the Relative Position of the Hand, Eye, and Goal during Reach PlanningNeuron 51:125–134https://doi.org/10.1016/j.neuron.2006.05.025
- 15.Spatiotemporal Tuning of Motor Cortical Neurons for Hand Position and VelocityJ. Neurophysiol 91:515–532https://doi.org/10.1152/jn.00587.2002
- 16.Neural Correlates of Reaching Decisions in Dorsal Premotor Cortex: Specification of Multiple Direction Choices and Final Selection of ActionNeuron 45:801–814https://doi.org/10.1016/j.neuron.2005.01.027
- 17.Congruent Activity during Action and Action Observation in Motor CortexJ. Neurosci 27:13241–13250https://doi.org/10.1523/JNEUROSCI.2895-07.2007
- 18.Neural population dynamics during reachingNature 487:51–56https://doi.org/10.1038/nature11129
- 19.Cortical Control of Arm Movements: A Dynamical Systems PerspectiveAnnu. Rev. Neurosci 36:337–359https://doi.org/10.1146/annurev-neuro-062111-150509
- 20.Computation Through Neural Population DynamicsAnnu. Rev. Neurosci 43:249–275https://doi.org/10.1146/annurev-neuro-092619-094115
- 21.Neural Responses during Interception of Real and Apparent Circularly Moving Stimuli in Motor Cortex and Area 7aCereb. Cortex 14:314–331https://doi.org/10.1093/cercor/bhg130
- 22.Eye-hand coordination during flexible manual interception of an abruptly appearing, moving targetJ. Neurophysiol 119:221–234https://doi.org/10.1152/jn.00476.2017
- 23.Posterior parietal cortex predicts upcoming movement in dynamic sensorimotor controlProc. Natl. Acad. Sci 119https://doi.org/10.1073/pnas.2118903119
- 24.Demixed principal component analysis of neural population dataElife 5:1–36https://doi.org/10.7554/eLife.10989
- 25.Context-dependent computation by recurrent dynamics in prefrontal cortexNature 503:78–84https://doi.org/10.1038/nature12742
- 26.Mixed selectivity morphs population codes in prefrontal cortexNat. Neurosci 20:1770–1779https://doi.org/10.1038/s41593-017-0003-2
- 27.Cortical preparatory activity indexes learned motor memoriesNature 602:274–279https://doi.org/10.1038/s41586-021-04329-x
- 28.Forward Prediction in the Posterior Parietal Cortex and Dynamic Brain-Machine InterfaceFront. Integr. Neurosci 10:1–6https://doi.org/10.3389/fnint.2016.00035
- 29.Intentional maps in posterior parietal cortexAnnu Rev Neurosci 25:189–220https://doi.org/10.1146/annurev.neuro.25.112701.142922
- 30.Intention, Action Planning, and Decision Making in Parietal-Frontal CircuitsNeuron 63:568–583https://doi.org/10.1016/j.neuron.2009.08.028
- 31.Properties of visual cue responses in primate precentral cortexBrain Res 343:24–35https://doi.org/10.1016/0006-8993(85)91154-0
- 32.Temporal Relation of Population Activity in Visual Areas MT/MST and in Primary Motor Cortex during Visually Guided Tracking MovementsCereb. Cortex 12:466–476https://doi.org/10.1093/cercor/12.5.466
- 33.Neural representations of the target (goal) of visually guided arm movements in three motor areas of the monkeyJ. Neurophysiol 64:164–178https://doi.org/10.1152/jn.1990.64.1.164
- 34.Cognitive spatial-motor processes7. The making of movements at an angle from a stimulus direction: studies of motor cortical activity at the single cell and population levels. Exp. Brain Res 87:562–580https://doi.org/10.1007/BF00227082
- 35.Motor Cortical Activity during Interception of Moving TargetsJ. Cogn. Neurosci 13:306–318https://doi.org/10.1162/08989290151137368
- 36.Reaching Movements With Similar Hand Paths But Different Arm Orientations. I. Activity of Individual Cells in Motor CortexJ. Neurophysiol 77:826–852https://doi.org/10.1152/jn.1997.77.2.826
- 37.Motor Cortical Representation of Speed and Direction During ReachingJ. Neurophysiol 82:2676–2692https://doi.org/10.1152/jn.1999.82.5.2676
- 38.Decoding arm speed during reachingNat. Commun 9:1–14https://doi.org/10.1038/s41467-018-07647-3
- 39.Muscle and Movement Representations in the Primary Motor CortexScience (80- 285:2136–2139https://doi.org/10.1126/science.285.5436.2136
- 40.Direction of action is represented in the ventral premotor cortexNat. Neurosci 4:1020–1025https://doi.org/10.1038/nn726
- 41.Reach Plans in Eye-Centered CoordinatesScience (80- 285:257–260https://doi.org/10.1126/science.285.5425.257
- 42.Direct visuomotor transformations for reachingNature 416:632–636https://doi.org/10.1038/416632a
- 43.Using a Compound Gain Field to Compute a Reach PlanNeuron 64:744–755https://doi.org/10.1016/j.neuron.2009.11.005
- 44.Multimodal Representation of Space in the Posterior Parietal Cortex and Its Use in Planning MovementsAnnu. Rev. Neurosci 20:303–330https://doi.org/10.1146/annurev.neuro.20.1.303
- 45.A common reference frame for movement plans in the posterior parietal cortexNat. Rev. Neurosci 3:553–562https://doi.org/10.1038/nrn873
- 46.Object vision to hand action in macaque parietal, premotor, and motor corticesElife 5:1–24https://doi.org/10.7554/eLife.15278
- 47.Uniting functional network topology and oscillations in the fronto-parietal single unit network of behaving primatesElife 5:1–27https://doi.org/10.7554/eLife.15719
- 48.Directed interaction between monkey premotor and posterior parietal cortex during motor-goal retrieval from working memoryCereb. Cortex 28:1866–1881https://doi.org/10.1093/cercor/bhy035
- 49.Gain mechanisms for contextually guided visuomotor transformationsJ. Neurosci 27:10588–10596https://doi.org/10.1523/JNEUROSCI.2685-07.2007
- 50.CircStat : A MATLAB Toolbox for Circular StatisticsJ. Stat. Softw 31:128–129https://doi.org/10.18637/jss.v031.i10
- 51.Legland, D. MatGeom library for geometric computing with MATLAB.
- 52.Spatial transformations in the parietal cortex using basis functionsJ. Cogn. Neurosci 9:222–237https://doi.org/10.1162/jocn.1997.9.2.222
- 53.Shifts in the population response in the middle temporal visual area parallel perceptual and motor illusions produced by apparent motionJ. Neurosci 21:9387–9402https://doi.org/10.1523/jneurosci.21-23-09387.2001
- 54.Directional tuning profiles of motor cortical cellsNeurosci. Res 36:73–79https://doi.org/10.1016/S0168-0102(99)00112-1
- 55.Neural Population Dynamics during Reaching Are Better Explained by a Dynamical System than Representational TuningPLoS Comput. Biol 12:1–22https://doi.org/10.1371/journal.pcbi.1005175
- 56.Optimal anticipatory control as a theory of motor preparation: A thalamo-cortical circuit modelNeuron 109:1567–1581https://doi.org/10.1016/j.neuron.2021.03.009
- 57.A neural network that finds a naturalistic solution for the production of muscle activityNat. Neurosci 18:1025–1033https://doi.org/10.1038/nn.4042
Article and author information
Author information
Version history
- Preprint posted:
- Sent for peer review:
- Reviewed Preprint version 1:
Copyright
© 2024, Zhang et al.
This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.
Metrics
- views
- 431
- downloads
- 33
- citations
- 0
Views, downloads and citations are aggregated across all versions of this paper published by eLife.