A synergy-based hand control is encoded in human motor cortical areas
- University of Pisa, Italy
- Istituto Italiano di Tecnologia, Italy
- Pisa University Hospital, Italy
- IMT School for Advanced Studies Lucca, Italy
- Arizona State University, United States
Figures
Figure 1
This probability map shows the voxels that were consistently engaged by the encoding procedure across subjects, i.e., those voxels whose activity was predictable on the basis of the kinematic synergies.
A hand-posture- related network comprising the left primary and supplementary motor areas, the superior parietal lobe and the anterior part of intraparietal sulcus (bilaterally) was recruited with high overlap across subjects. Despite additional regions (i.e., Brodmann Area 6) resulted from the encoding analyses, they are not evident in the map due to their deep location.
-
Figure 1—source data 1
This compressed NIfTI file in MNI152 space represents the voxels that were recruited by the encoding procedure in more than three subjects.
The value of each individual voxel corresponds to the number of subjects in which that voxel was recruited.
- https://doi.org/10.7554/eLife.13420.004
-
Figure 1—source data 2
his compressed NIfTI file in MNI152 space represents the Region of Interest chosen for encoding brain activity from the visual region, defined on the basis of a t-test of the overall brain activity (i.e., task versus rest condition) five seconds after the visual stimulus onset, corrected for multiple comparisons with False Discovery Rate (q<0.01).
- https://doi.org/10.7554/eLife.13420.005
Figure 2 with 1 supplement
Cortical flattened map depicting the topographical organization of the first three synergies across primary motor, somatosensory, and parietal regions.
The portion of cerebral cortex represented in the map corresponds to the area enclosed in the rectangle in the brain mesh (top, right). M1: Primary Motor Cortex. CS: Central Sulcus. S1: Primary Somatosensory cortex (postcentral gyrus). aIPS: anterior intraparietal sulcus. SPL: Superior Parietal lobule.
Figure 2—figure supplement 1
Topography assessment: map and feature spaces.
The two maps represent the map space (upper image), which depicts the pairwise physical distance (i.e., standardized Euclidean distance) between the voxels of the results map, and the feature space (lower image), which depicts the distance (i.e., standardized Euclidean distance) between the goodness-of-fit (R2) of the first three kinematic PCs in each voxel. For further details, see Materials and methods and Yarrow et al., 2014. There was a significant similarity between the two spaces, assessed with the permutation test described in the Methods (C=0.192; p-value=0.0383). Voxels were reordered accordingly to their physical distance to improve readability of the two maps.
Figure 3 with 1 supplement
This picture displays the mMultidimensional sScaling (MDS) results for kinematic synergies (left) and fMRI brain activity (right).
With the exception of few postures (e.g., dinner plate, frisbee and espresso cup) that were misplaced in the fMRI data with respect to the kinematic synergies representation, the other object-related postures almost preserved their relative distances.
Figure 3—figure supplement 1
Average correlations between behavioral models and fMRI data.
The histogram reports the correlation values (transformed to z-scores and averaged across subjects) between each behavioral model and the fMRI data. Error bars represent the SEM. The noise ceiling, estimated using the procedure described by Ejaz et al., 2015 is also reported. The two dashed lines describe the upper and lower bounds, respectively. The single-subject correlation values are reported in Supplementary file 1D.
Figure 4 with 2 supplements
This picture represents the postures obtained from the fMRI data and those originally recorded through optical tracking.
The figure shows three pairs of hand plots corresponding to three postures from a representative subject, and the goodness-of-fit between the original and decoded sets of joint angles. In these plots, the two wrist angles are not rendered.
-
Figure 4—source data 1
This compressed NIfTI file in MNI152 space represents the Region of Interest chosen for RSA and posture decoding, defined on the basis of a t-test of the overall brain activity (i.e., task versus rest condition), corrected for multiple comparisons with False Discovery Rate (q<0.05).
- https://doi.org/10.7554/eLife.13420.012
Figure 4—figure supplement 1
Marker placement for kinematic hand posture data acquisition: The picture depicts the hand of a subject with the complete set of optical markers used to define hand posture through optical tracking.
This set of markers corresponds to the joint and bones positions originally recorded; the rendering in Figure 4 was performed with reference to this acquisition protocol.
Figure 4—figure supplement 2
ROI used for performing RSA and posture decoding: This map represents the Region of Interest which contained all the voxels used for performing Representational Similarity Analysis and hand posture decoding.
The region was obtained with a t-test of the overall brain activity (i.e., task versus rest condition), corrected for multiple comparisons with False Discovery Rate (q<0.05). The population of voxels represented here was subsequently reduced with a PCA accounting for most of the variance as described in the Methods.
Figure 5 with 1 supplement
Discrimination accuracies for single postures as represented by kinematic synergies.
Two clusters of similar postures are easily identifiable (i.e., precision grip and power grasps). However, other postures were recognized without showing an evident clustering, suggesting that the encoding procedure was not biased by a coarse discrimination of motor acts.
Figure 5—figure supplement 1
Workflow of the encoding analysis.
This diagram depicts the workflow of the multiple linear regression procedure applied on fMRI data using the matrices obtained from the data acquired in the kinematic and EMG experiments as encoding models. The pairwise discrimination accuracy was estimated in the decoding phase, represented as the final step of this diagram.
Figure 6
The three graphs display the rank accuracy values as a function of the dimensionality (i.e., the number of retained PCs) of each behavioral model.
The two models derived from kinematic and EMG data (upper and middle graphs, respectively) have a number of synergies ranging from 1 to 10 while the individual digit model (lower) had 1 to 5 retained PCs. Darker bar colors indicate the dimensionality chosen for encoding brain functional data.
Appendix figure 1
Placement of the sixteen electrodes on the right arm.
Four configurations were tested, either with five (1–5, see Materials and methods), ten (1–4, 6–8, 14–16, from Bitzer and van der Smagt, 2006), or fourteen channels (from Ejaz et al., 2015).
Appendix figure 2
Results of the rank accuracy procedure as a function of the number of EMG channels.
The red line shows the accuracy values for random configurations of 5 to 16 electrodes, using the mean-based preprocessing adopted by Ejaz et al. (2015). The orange dots represent the accuracy values for three fixed configurations. The blue line shows the accuracy values for 5 to 16 channels using the feature-based preprocessing (see Materials and methods); the light blue dots show the accuracy for three fixed configurations. Values are reported as mean across subjects ± SEM (error bars and bands).
Videos
Video 1
This video shows the meaning of the kinematic synergies measured in this study, by presenting three movements from the minimum to the maximum values of kinematic synergies 1, 2, and 3, respectively, expressed as sets of twenty-four joint angles averaged across subjects.
It can be observed that the first synergy modulates abduction-adduction and flexion-extension of both the proximal and distal finger joints, while the second synergy reflects thumb opposition and flexion-extension of the distal joints only. Maximizing the first synergy leads, therefore, to a posture resembling a power grasp, while the second one is linked to pinch movements directed towards smaller objects, and the third one represents movements of flexion and thumb opposition (like in grasping a dish or a platter) (Santello et al., 1998; Gentner and Classen, 2006; Ingram et al., 2008; Thakur et al., 2008).
Additional files
-
Supplementary file 1
(A) Single subject rank accuracy values.
Values of rank accuracy, measured with the leave-one-stimulus-out procedure, for the nine subjects, with the p-value obtained from the permutation test (10000 iterations).
The comparison between the performance values indicate that the kinematic synergy model was significantly better than both the individual digit and muscle synergy models (Wilcoxon signed-rank test, p=0.0078), and the individual digit model was significantly more informative than the muscle synergy model (p=0.0156).
(B) Single subject encoding accuracy values. The accuracy of predicting brain activity from the behavioral models (kinematic synergy, individual digit and muscle synergy models), obtained with the cross-validation procedure, is reported here for each subject, along with the chance levels derived from the permutation tests, the threshold at p=0.05 and the actual p-value obtained from the tests against the null distributions of accuracies. The accuracy values reported in red are not significant. The comparisons between individual accuracy values, performed using Wilcoxon signed-rank tests, show that the kinematic synergy model outperformed both the individual digit (p=0.0234) and the muscle synergy (p=0.0391) models, whereas no significant difference was found between the individual digit and muscle synergy models (p=0.9453).
(C) Size and coordinates of the clusters of greatest overlap between subjects. This table reports the regions that were consistently recruited across subjects (p>0.33, 4 out of 9 subjects). The region names are reported alongside with their size and with the coordinates of the peak voxel in RAI orientation according to the MNI 152 atlas.
(D) RSA results: single-subject and group correlations between RSs. The table contains the results from Representational Similarity Analysis (RSA). The single-subject correlation values are reported, along with the group-level correlation (i.e. obtained from the averaged RSs across subjects) and with the p-values resulting from the Mantel test. Kinematic = kinematic synergy model; EMG = muscle synergy model; ID= Individual Digit model. The accuracy values reported in red are not significant according to the Mantel test (10,000 iterations).
(E) RSA results: single-subject and group correlations between behavioral and fMRI RSs. The table contains the results from Representational Similarity Analysis (RSA) between each behavioral model and fMRI data. The single-subject correlation values are reported, along with the group-level correlation (i.e. obtained from the averaged Representational Spaces – RSs – across subjects) and with the p-values resulting from the Mantel test. Kinematic = kinematic synergy model; EMG = muscle synergy model; ID= Individual Digit model. The accuracy values reported in red are not significant according to the Mantel test (10,000 iterations).
(F) Goodness of fit between original and decoded hand postures. Average goodness-of-fit (R2) values and Standard Deviations (STD) between original and reconstructed sets of joint angles related to specific hand postures across all subjects. The decoding procedure allowed us to obtain the set of synergies related to each grasping motor acts directly from fMRI activity, thus to reconstruct the different hand postures across participants.
(G) Rank accuracy values between original and decoded hand postures. The table reports the rank accuracy values for the discrimination between the original and decoded sets of joint angles related to specific hand postures across all subjects. The decoding procedure allowed us to obtain the set of synergies related to each grasping motor acts directly from fMRI activity, thus to reconstruct the different hand postures across participants.
(H) Encoding accuracy values for the picture-related brain activity. To assess to what extent the visual presentation of objects might have influenced the encoding of BOLD activity in motor regions, the encoding procedure was performed within the same ROI chosen for RSA and posture reconstruction and choosing BOLD activity at five seconds after the visual object presentation as an estimate of brain responses to the visual presentation of target objects. Only the kinematic synergy model was used. The chance levels derived from the permutation tests (1000 iterations) are reported, as well as the threshold at p=0.05 and the actual p-value obtained from the tests against the null distributions of accuracies. The accuracy values reported in red are not significant. The results show that the procedure is unsuccessful in all subjects and do not account for a confounding role of image-related activity on the posture encoding results.
(I) Encoding accuracy values for kinematic synergies in visual areas. To assess the impact of visual imagery on our results, the encoding procedure was performed within a Region of Interest selected based on the image-related activity (at 5 s after presentation) vs. rest (q<0.01, FDR corrected). The encoding of postures (using the kinematic synergy model only) was then tested in the voxels forming this ROI. The chance levels derived from the permutation tests (1000 iterations) are reported, as well as the threshold at p=0.05 and the actual p-value obtained from the tests against the null distributions of accuracies. The accuracy values reported in red are not significant. The results show that the procedure is unsuccessful in seven subjects and therefore it suggests a very limited impact of visual imagery on the posture encoding results.
(J) List of objects. Table displaying the twenty common-use objects (chosen from the 57 in Santello et al., 1998) that were used in this study.
(K) List of marked joints and bones. Complete list of hand joints and bones marked during the optical tracking experiment. Two additional markers were placed on the wrist, for a total of 26 optical markers. (L) EMG features The features that were extracted from the EMG signals are listed above. Muscle synergies were quantified through principal components analysis performed across features and EMG electrodes yielding a five-dimensional set of synergies.
(M) Rank accuracy values for 1 to 10 PCs. The table displays the rank accuracy values for the two models derived from kinematic and EMG data, with a number of retained PCs ranging from 1 to 10 (kinematic and EMG synergies) or 1 to 5 (individual digits). The reported values are the accuracy scores averaged across subjects and their SD. Notably, the individual digit model could explain only a moderate fraction of the total variance of the kinematic data (mean: 26.59%, range 14.46% to 34.97%). PCA dimensionality reduction was therefore successful as the first five synergies (later used for encoding fMRI activity) could explain a mean variance across subjects of 91.78% in the kinematic data and 72.64% in the EMG data. (N) Group synergies defined by constrained k-means The three core kinematic synergies from each participant were grouped across participants with a semi-supervised clustering algorithm (Bilenko et al., 2004). The procedure showed that the first three synergies were highly consistent and had the same rank across almost all subjects (i.e., PC 1 was in the first position in most of the subjects). Overall, 77.78% of the single subject synergies were consistently labeled across subjects. The table represents the three “group synergies” and lists the single-subject synergies that compose each of them.
- https://doi.org/10.7554/eLife.13420.018
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
A synergy-based hand control is encoded in human motor cortical areas
eLife 5:e13420.
https://doi.org/10.7554/eLife.13420
