The flexible manual interception task and example neurons

A Task diagram. In each trial, the monkey first holds the center dot for 600 ms (Hold), then a target appears (Target on), after 400-800 ms delay the center dot turns dark (GO), immediately the monkey moves its hand to reach to the target (Touch). The movement time (from GO to Touch) is required to be within 800 ms, otherwise the trial will be aborted. The feedback dot, which is presented at the touched location of the screen, will be in red for success or in blue for failure.

B Distribution of touch endpoints. Left panel shows fifteen reaching-up example trials in five target-motion conditions, three trials in each condition. The squares mark the touch endpoints, while the circles and triangles are the target onset and stop location. The five target-motion conditions (-240°/s, - 120°/s, 0°/s, 120°/s, and 240°/s) are indicated in five colors (purple, blue, green, yellow, and red). Target onset location is randomly distributed. Right panel shows the touch endpoints of all trials, each point represents a trial, colored according to target-motion conditions. The distribution was uniform around the circle (monkey C 772 trials, Rayleigh’s test, p=0.36).

C Implanted locations of microelectrode array in the motor cortex of the three well-trained monkeys. Neural data were recorded from the cortical regions contralateral to the used hand. AS, arcuate sulcus; CS, central sulcus.

D Three example neurons with PD shift, gain modulation, and offset addition. The peri-stimulus time histograms (PSTH) show the activity of example neurons when monkeys reached to upper areas in five target-motion conditions. The solid lines represent the trial-averaged firing rates, the colored shadow represents the standard error. The gray shadow indicates the time window between MO-100 ms and MO+100 ms.

E The directional tuning curves of the three example neurons with PD shift, gain, and addition modulation around movement onset (MO ± 100 ms, adjusted R2: 0.70, 0.84, and 0.60). Dots and bars denote the average and standard error of firing rates, colored according to target-motion conditions.

Ratio of mixed selectivity neurons around movement onset

Features of the encoding pattern at population level

A The decoding accuracy (SVM with 10-fold cross-validation) of reach direction (black line) and target velocity (blue line) by population activity (monkey C, n=95, 772 trials), is aligned to target on (TO), GO, and movement onset (MO). The dash-dotted lines are chance level of decoding reach direction (black, one in eight) and target velocity (blue, one in five). The shaded area is the standard deviation of the decoding accuracy for 10 permutations.

B The left panel shows the performance of reach-direction decoder (chance level: one in eight) transferred between different target-motion conditions. The SVM decoder was built on randomly selected 100 trials in training dataset and tested in another 100 trials from a dataset of different conditions (CCW vs. CW, 120 vs. 240, static vs. motion). The distributions of decoding accuracy were from 1000 repetitions and compared with one tailed t-test (p<0.01, with three stars). The right panel shows the performance of target-velocity decoder (chance level: one in five) in different reach-direction conditions. The accuracy distribution was also obtained from 1000 repetitions.

C The explained variance and representation of the principal components. The first row shows the explained variance of each PC (cumulatively over 70%). The second row shows the PCs’ fitting goodness (R2PCs) of reach direction and target velocity in four epochs.

D Directional tuning curves of the principal components. Each row shows the directional tuning of one PC (the first three PCs in C) in four epochs. Each dot represents a trial, tuning curves are averaged in eight reach directions, and PDs of PCs are indicated by the short lines in the top of subplot by a weighted sum of response. The colors of the lines and dots mean the target-motion conditions, as the same as the legend on the left. The goodness of fitting reach direction (R2DirVel) for the single-trial PCs under single target-motion conditions is shown by mean ± sd. across conditions.

The orbital neural geometry in latent dynamics

A Three-dimensional neural state of M1 population activity obtained by PCA. Each point represents a single trial. The upper subplot is colored according to five target-motion conditions, while the bottom is in colors corresponding to eight reach directions. The explained variances of the first three PCs were 17.7%, 11.3%, and 5.9%. Neural data were from monkey C (N=480, merged six sessions), for each of the total 40 conditions, 15 trials were randomly sampled.

B Fitted ellipses of neural states. The ellipses fitted in A are projected onto three two-dimensional subspaces, colored by target velocities (left column) or reach directions (right column).

C The relation between the tiling angle and target velocity. The tilting angle is calculated between ellipses of the moving-target conditions and the static-target condition (0 °/s) in the range from - 90° to 90°, CCW is defined as positive. Circles, squares, and triangles correspond to monkeys C (7 sessions), G (4 sessions), and D (4 sessions), respectively. The lines indicate the linear fitting between the tilting angle (θ) and target velocity (vel.), with solid line for monkey C (θ = 0.23*vel.+4.2, R2=0.91), dashed line for monkey D (θ = 0.26*vel.+4.3, R2=0.81), and dotted line for monkey G (θ = 0.15*vel.-1.4, R2=0.89).

The shape of neural dynamics relies on neuronal mixed selectivity

A The tuning curves of three ideal neurons in five target-motion conditions. From up to down, they are PD shift, gain modulation, and addition.

B The simulated neural states from three groups of ideal neurons, colored according to target-motion conditions (first row) and reach directions (second row). For each simulation, the neural states were obtained from 300 model neurons by PCA. The neural state in 180° reach direction is marked with a red dot. The first two principal components can explain more than 95% of the variance in the data (the explained variance of the first three PCs, Gain: 49.5%, 46.6%, and 2.0%; PD shift: 50.1%, 47.1%, and 1.6%; Addition:50.8%, 47.9%, and 1.4%).

C The neural states of a mixed group of 100*3 model neurons, as in B. The explained variance of the first three principal components were 48.4%, 44.2%, and 3.1%.

D Quantification of the difference between neural-state ellipses in four simulated groups and a real dataset (monkey C, n=95). Rotational angle is the angular differences in the first two neural state. Tilting angle is the relative angle of the normal vector of ellipses. State shift is the root of mean squared distance between two ellipses.

The neural geometry in RNNs

A Network architecture. The network inputs consist of motor intention, target location, and GO signal. The motor intention is the two-dimensional Cartesian coordinate of the interception location, and exists fixed during MO-50ms to MO; the target location is the two-dimensional Cartesian coordinate of the moving target, and appears time-varying during the whole trial; the GO-signal is a step function, jumping from 0 to 1 at GO. The RNN with 200 hidden units is expected to output hand velocity in two-dimensional Cartesian coordinates for accurate interception.

B Three example nodes with PD-shift modulation, gain modulation, and additive modulation. Similar to Figure 1D.

C Three-dimensional neural state of node activity obtained by PCA, colored according to target-motion conditions (top) and reach-direction conditions (Bottom). Similar to Figure 3A.

D The tilting angle of ellipses. Similar to Figure 3D. The fitted line is θ = 0.15*vel.+0.11, R2=0.96, across five target velocities.

E The connectivity between different types of modulations. On the left is a boxplot representing the averaged absolute connection weight, across 100 models. S for PD-shift nodes, G for gain nodes, and A for additive nodes. On the right is a diagram of the connectivity, with linewidth representing the relative connection strength.