Conceptual approach. A. We hypothesized that the condition-dependent subspace of PM MN activity shifts progressively through the time course of behavioral trials both during execution (orange) and during observation (green). B. Trajectory segments (orange, green) were projected into time series of instantaneous subspaces (gray). C. Trajectory segments (latent dynamics) from execution and observation can be aligned even if they occupy distinct (orthogonal) subspaces.

The reach-grasp-manipulate (RGM) task. A. In separate blocks of trials monkeys reached to, grasped, and manipulated four different objects themselves (Exe), and then observed a human perform the same task (Obs). B. The times of eight behavioral events from Start-of-trial to End-of-trial divided each trial into seven epochs from Initial hold to Reward. For analyses the data were aligned separately on, and trajectories were sampled at, the times of four selected events—Instruction onset (I), Go cue (G), Movement onset (M), and Hold (H).

Numbers of trials in each session.

For each of the three sessions from each of the three monkeys, numbers of trials involving each of the four objects (sphere, button, coaxial cylinder, perpendicular cylinder are given in parentheses separately for execution and for observation.

Numbers of mirror neurons in each session.

For each of the three sessions from each of the three monkeys, numbers of PM MNs are given in the format of Total(PMv, PMd).

Alignment summary across sessions.

For each set of trajectory segments, across-session means ± standard deviations are given for the 1st and 2nd alignment correlation coefficients (CC1, CC2), as well as for the WSDs (d1, d2) between the Exe-Obs and Exe-Exe bootstrapped marginal distributions.

Trajectory segments projected into instantaneous subspaces. A. Using execution data from an example session (T_20220603), trajectory segments averaged across trials involving each of the four objects (sphere – purple, button – cyan, coaxial cylinder – magenta, perpendicular cylinder – yellow) were sampled immediately following each of four behavioral events (rows: Instruction onset, Go cue, Movement onset, Hold). Each set of these four segments then was projected into the instantaneous subspace present at four different times (columns: I, G, M, H). B. The same process was performed using observation data from the same session. The PC1 vs PC2 scales at lower left apply to all frames in both A and B. C and D. Cumulative separation values (CS, seem Methods) calculated for each of the frames in A and B, respectively, are shown as color matrices. E and F show CS values averaged across all 9 sessions for execution and observation, respectively.

Classification accuracy for execution trajectory segments projected into instantaneousexecution subspaces. A. Instruction trajectory segments. B. Go segments. C. Movement segments. D. Hold segments. In each frame, the short horizontal orange bars at the top of the vertical lines indicate the 100 ms during which each set of trajectory segments was sampled; the horizontal purple bar at lower left represents 500 ms. Results in 50 ms steps have been aligned separated at the times of the instruction onset (I), go cue (G), movement onset (M), and hold (H). Solid curves indicate mean classification accuracy of 10-fold cross-validation as a function of time, with the shaded areas indicating 1 standard deviation. Colors red, green, and blue represent sessions 1, 2, and 3 from each monkey, with black being their average. Horizontal black lines indicate the mean (solid) ± 3 standard deviations (dashed) classification accuracy obtained by projecting each set of trajectory segments into 500 randomly selected 3D spaces.

Classification accuracy for observation trajectory segments projected into instantaneous observation subspaces. A. Instruction trajectory segments. B. Go segments. C. Movement segments. D. Hold segments. The format of each frame is the same as that described for Figure 4.

Classification accuracy for execution trajectory segments projected into instantaneous observation subspaces. A. Instruction trajectory segments. B. Go segments. C. Movement segments. D. Hold segments. The format of each frame is the same as that described for Figure 4.

Classification accuracy for observation trajectory segments projected into instantaneous execution subspaces. A. Instruction trajectory segments. B. Go segments. C. Movement segments. D. Hold segments. The format of each frame is the same as that described for Figure 4.

Alignment of execution and observation trajectory segments. A. Hold trajectory segments from execution trials are shown in the original instantaneous execution subspace at time H (left), and from observation trials in the original instantaneous observation subspace also at time H (right). B. After alignment, both execution (left) and observation (right) segments have been projection into the original execution subspace. Colors indicate trajectory segments from trials involving the sphere – purple, button – cyan, coaxial cylinder – magenta, perpendicular cylinder – yellow.

Alignment of latent dynamics (trajectory segments). A. Instruction trajectory segments. B. Go segments. C. Movement segments. D. Hold segments. Two-dimensional and marginal distributions are shown of the 1st versus 2nd correlation coefficients from 500 repetitions of aligning each set of trajectory segments separately. Values for alignment of execution and observation segments from the same session (F_20200804) are shown in red; values for alignment of execution segments from two sessions two days apart (F_20200804, F_20200806) are shown in gray. Wasserstein distances between the marginal distributions in the 1st and 2nd dimensions are given as d1 and d2 at the upper left of each plot.

Recording array locations in A. Monkey F. B. Monkey R. C. Monkey T. PCD – precentral dimple; AS – arcuate sulcus; CS – central sulcus; r – rostral; m – medial. Scale bars apply to all three monkeys.