Overview of the task and control strategy.

a. Wide initial workspace. Three hundred 7-DoF arm configurations (grey dots, only 3 angles displayed for convenience) within the joint operating range of a given participant (materialized by the parallelepiped) are transformed into 300 plausible target locations (grey arrows) using forward kinematics. b. Natural arm movements are recorded while participants equipped with movement trackers on arm and torso are involved in picking and placing a bottle at the 300 target locations in virtual reality. c. The ANN is trained on recorded natural arm movement to reconstruct distal DoFs (orange) from proximal ones (green) plus target information (position and orientation). d. Wide space covered during recorded natural arm movements. Two hundred nodes (red dots) that best represent the arm angular configurations actually produced (grey circles) by a participant during her/his recorded natural arm movements were identified using an unsupervised self-organizing neural network, and transformed into a set of 200 possible targets (red arrows) using forward kinematics. e. Movement-based prosthesis control. The participant performs the pick and place task at the 200 possible targets using a hybrid arm reproducing in real-time her/his own shoulder movements (green angles), and using the ANN predictions for the 5 remaining distal DoFs (orange angles).

Wide workspace covered in experiments.

a. All targets used for a representative participant of Exp1 are displayed, together with 5 arm postures (4 at extended positions and 1 flexed in the middle) to provide perspectives. Grey arrows represent plausible targets (n=300), and red arrows represent possible targets (n=200). b. Possible targets of all participants of Exp. 1 (n=2000, red arrows) and Exp. 3 (n=1400, in blue arrows), remapped for an average arm, and regrouped on the same graph. Note that for Exp3, possible targets corresponding to left sided amputees were mirrored to be represented in relation to a right arm. This figure illustrates the comparably large workspaces obtained for the 10 participants of Exp1 (used for the Generic ANN) and the 7 amputees of Exp3 (using the Generic ANN).

Protocols and results.

a. Protocols of the three experiments. Each box contains a phase name and the name of the control used. Fam. stand for Familiarization phase, and Initial Acq. for Initial Acquisition phase. The order of test phases conducted with the Own and the Generic ANNs were counterbalanced in Exp2. b-d. Results for success rate (b), movement time (c) and shoulder volume (d). Each grey line corresponds to a participant. In Exp2, dashed lines indicate participants who began by the control with the Generic ANN and plain lines those who began by the control with the Own ANN. Boxes limits show first and third quartiles whereas inside line shows the median value. Whiskers show min and max values. Own, Gen, Nat represent phases in which the control was performed with the Own ANN, the Generic ANN, and the Natural Virtual Arm, respectively. In Exp3, Gen1 and Gen2 refer to the first and second block performed with the Generic ANN. Stars represent significant differences, with * for p < 0.05, ** for p < 0.01, and *** for p < 0.001. The dashed red line represents a volume of 1 dm3 (= 1 L).

Physical Proof of Concept on a tele-operated robotic platform.

a. Task and setup. The participant stands still setback from the humanoid robotic platform that faces a board on which 5 sponges are placed at different positions and orientations. The participant tele-operates the robotic arm so as to reach and grasp each of the 5 sponges of a block, one trial after another, according to order indicated by numbers written on sponges. b. Three types of blocks define three spatial arrangements of sponges on the board. c Protocols of the Proof of Concept (POC) experiments. Each box contains a phase name and the name of the control used, either based on natural arm movements (TestNat) or on predictions from the Generic ANN (TestGeneric). Fam. stand for Familiarization phase. The order of test phases was counterbalanced in POCa and POCb. d-e. Results for success rate (d) and movement time (e). Each grey line corresponds to a participant. In POCa-b, dashed lines indicate participants who began by TestGeneric and plain lines those who began by TestNat. Boxes limits show first and third quartiles whereas inside line shows the median value. Whiskers show min and max values. Stars represent significant differences, with * for p < 0.05, ** for p < 0.01, and *** for p < 0.001. Triangles represent performances obtained for the Block 1 by the two transhumeral amputees whereas the square represents performances of the congenital participant on all 3 Blocks.

Exp3 participants’ amputation description.

Each line contains the time since amputation, the stump circumference and length, and the side of the amputation for a participant (R = right, L = left).

a. ANNs Inputs and Outputs, displayed together with movements done to get the joints range of motion (black arrows). ANNs Inputs include shoulder flexion-extension (θS-FE), shoulder abduction-adduction (θS-AA), target position in relation to the shoulder (PT-X, PT-Y and PT-Z), and target orientation (as angles of rotation) with respect to the frontal and sagittal plane (θT-F and θT-S). ANNs outputs include all distal angles from the humeral rotation included: humeral rotation (θH-R), elbow flexion-extension (θE-FE), forearm pronation-supination (θF-PS), wrist flexion-extension (θW-FE) and radial-ulnar deviation (θW-RU). b. Remapping target position for different arm morphologies. Two arms with the same angular configuration but different segments length lead to different positions of the target (orange arrows). Forward kinematics was used to remap target position for a subject with a different arm segments’ length (remapping for a shorter arm displayed). Note that the target orientation remains unaffected.