Elite tennis players are particularly skilled at striking a balance between the speed and the accuracy of their serves. If they only focus on placing the ball in a strategic location, they may inadvertently advantage their opponent by not paying close enough attention to the speed of their serve. On the other hand, serving the ball with as much speed as possible is not desirable as the ball may fail to clear the net or land in a location that is advantageous to their opponent. This relationship, commonly known as the ‘speed-accuracy tradeoff,’ is observed not only for purely motor, but also perceptual and cognitive tasks (e.g., Heitz, 2014; Wickelgren, 1977). In laboratory settings, the speed-accuracy tradeoff, quantified by Fitts’ law (Fitts, 1954), is an empirical finding that movements that require greater accuracy tend to be slower than those that do not. Fitts’ law relates the width of the target (W), the movement distance (A), and the movement duration (MD):

where a and b are subject-specific scalar parameters. The ${\displaystyle \mathrm{l}\mathrm{o}{\mathrm{g}}_2(2A/W)}$ term is known as the ‘index of difficulty’.

A widely held theory, initially proposed by Harris and Wolpert, posits that signal-dependent noise (i.e., ‘motor noise’ present at the neuromuscular junction) that scales with the size of the control signal is a sufficient theory to explain Fitts’ law (Harris and Wolpert, 1998; Lunardini et al., 2015; McCrea and Eng, 2005; Peternel et al., 2017). The theory proposes that if motor noise increases with the size of the control signal, then moving rapidly, which requires large control signals, increases the variability in the final movement position. Conversely, reaching to a smaller target, with low variability in the final position, requires smaller control signals that consequently produce slower movements. Thus, this theory provides an explanation for the speed-accuracy tradeoff. In the present work, we show that the speed-accuracy tradeoff, as described by Fitts’ law, emerges even without the presence of motor noise.

In support of this theory, a large body of prior work has employed simple one-dimensional linear systems or torque-driven two-joint models of the arm that are restricted to moving in a plane to study the speed-accuracy tradeoff (e.g., Flash and Hogan, 1985; Fagg et al., 2002; Harris and Wolpert, 1998; Alexander, 1997; DeWolf et al., 2016; Sketch et al., 2017). While it certainly is not the case that the mechanical properties of the plant being controlled explain phenomena such as Fitts’ law, the patterns of neural activity required to drive goal-directed movements are necessarily constrained by the dynamics of the plant. When musculoskeletal dynamics are modeled with sufficient fidelity, and these musculoskeletal dynamics are controlled in an optimal fashion, motor behavior that is consistent with Fitts’ law emerges and thus provides a parsimonious theory of the speed-accuracy tradeoff.

Thus, a primary innovation in this work is the development of a computational model of upper extremity movements with a more realistic model of the musculoskeletal system than previously described (three-dimensional, 47 muscles and five degrees-of-freedom [Saul et al., 2015]). The musculoskeletal model proposed by Saul and colleagues was previously used to track experimental kinematics, but not synthesize movements de novo. Our computational model synthesizes three-dimensional point-to-point reaching movements that reproduce features reported in motor control studies and in our experimental data. We focus on point-to-point movements because normal voluntary movements can be described as holding still after moving from one pose to another (Shadmehr, 2017). The movements are synthesized by minimizing a cost function based on end-point accuracy and squared muscle activation terms. We use the squared muscle activation term as an approximation to metabolic power consumption (Millard et al., 2013). We show that the asymmetry in the velocity profile with movement speed can be explained in terms of optimal control theory, as opposed to it either representing some unexplained feature, or resulting from constraints in the neural architecture (Bullock and Grossberg, 1988), or a learned strategy when approaching a target (Beggs and Howarth, 1972), or time-varying cost constraints (Nagasaki, 1989), or signal-dependent noise with a minimum variance model (Tanaka et al., 2004).

Guided by this computational model, we propose a new theory based on motor planning variability: Fitts’ law emerges as a consequence of the difficulty associated with finding good control solutions for challenging tasks. Concretely, our theory results from the observation that during learning of a new task, the search space for finding the control solutions is very high-dimensional and contains many local minima. In our model, we use a stochastic optimizer, which attempts to find appropriate motor plans (i.e., reach trajectories) in this search space. As a consequence of the high dimensionality of the search space and the presence of local minima, the optimizer is more likely to find locally optimal solutions (or solutions that are near locally optimal solutions) in lieu of globally optimal solutions. Our theory suggests that for more challenging tasks (e.g., reaching to smaller targets), the trajectory optimizer tends to find less effective solutions (i.e., slower reaches) than other undiscovered but possible solutions. For less challenging tasks (e.g., reaching to larger targets), the optimizer tends to find more effective solutions (i.e., faster reaches). With more challenging tasks, previously effective solutions are discarded as they do not produce trajectories that reach the target; the smaller set of effective solutions makes it more likely for the optimizer to converge to a less effective solution.

The crux of our theory is that some features of human movement are attributable to planning variability rather than execution noise (e.g., Figure 4 and Churchland et al., 2006a for related evidence). We analyze motor cortical neural activity from monkeys reaching to targets of different sizes. We hypothesized that if motor planning variability has a contribution to the speed-accuracy tradeoff, then we should observe a statistically significant difference in motor planning variability when reaching to large and small targets; this is not expected if the speed-accuracy tradeoff is only the result of execution noise. Furthermore, we hypothesized that the contribution of planning variability to MD variability should increase for smaller targets (i.e., as sensitivity to the motor plan increases for more challenging tasks). We test both hypotheses in this work and show that the neural analyses are consistent with our theory.

A realistic musculoskeletal model allows us to study human movement variability with greater resolution. Our theory highlights the role of motor planning rather than execution noise to explain features of human movement. That is not to say that signal-dependent noise does not exist or could not play a part in Fitts’ law. Rather, given the results here, we must re-examine the assumption that signal-dependent noise is the only factor that gives rise to Fitts’ law. Reaching to a smaller target may exhibit smaller velocities to minimize the effects of signal-dependent noise. Alternatively, slower reach trajectories for smaller targets might be ‘easier’ to find from an optimal control theory perspective. We believe our new theory provides a more complete account of the empirical observations, both behaviorally and neurally, surrounding the speed-accuracy tradeoff, without having to rely on assumptions surrounding improvements in the signal-to-noise ratio.

In this study, we investigate a fundamental phenomenon underlying goal-directed motor control. Fitts’ law formalizes the notion that movement time is a function of both the distance to a target and the size of that target. Taken together with a widely held theory initially proposed by Harris and Wolpert, 1998, Fitts’ law is thought to arise from the fact that signal-dependent noise governs a person’s ability to move with greater accuracy toward smaller targets by reducing movement speed. Signal-dependent noise or ‘motor noise’ has been shown across a variety of experiments, where noise scales in proportion to the size of the motor command being generated (e.g., Jones et al., 2002; Matthews, 1996). Thus, to a first-order approximation, signal-dependent noise is a sufficient theory to explain Fitts’ law. Here, we re-examine the assumption that Fitts’ law emerges primarily as a consequence of signal-dependent noise.

We develop a computational model of three-dimensional upper extremity movements that is more biomechanically realistic than previous work. Our model replicates kinematic features reported across several motor control studies. When solving the optimal control problem, we find that some of our simulations reproduce the empirical observation that arm movement paths are nearly straight and gently curved near the targets for fast reaches. The corresponding velocity profiles are bell-shaped and have a secondary, smaller peak near the target. These features are usually interpreted as discrete, open-loop corrections to acquire the target (Doeringer and Hogan, 1998). While our results are consistent with this interpretation, our analyses also show that these features are present in some local minima when solving the optimal control problem. Out of 20 trials during a reaching task, we consistently note that the second or third best solution (measured with respect to the cost; Equation 1) has these features, while the best solution consists of a nearly straight trajectory. The curve and the secondary velocity peak at the end of the movement are a consequence of the hand reaching the target with a high velocity; the final, gentle curve, brings the hand velocity toward zero, minimizes ‘effort’, and keeps the hand inside the target.

Bullock and Grossberg, 1988 proposed that the speed-dependent asymmetry in the velocity profile arises due to neural dynamics and that models based on optimal control always predict symmetric velocity profiles. Our work demonstrates that asymmetric velocity profiles in reaching movements arise from optimal control methods with realistic biomechanical models, without relying on other modeling assumptions such as time-varying jerk constraints (Nagasaki, 1989). This feature did not emerge in earlier work using simplified biomechanical models (Guigon et al., 2007; Harris and Wolpert, 1998). The upper extremity must decelerate when approaching the target to achieve zero velocity. The elbow flexors undergoing an eccentric contraction have a larger force generating capacity at higher speeds due to the force–velocity relationship of muscle (Millard et al., 2013). Our hypothesis is that this explains why faster movements have a later peak velocity. We have verified that this property does not arise in a torque-driven model (Figure 2—figure supplement 2), which supports the hypothesis.

Reaching involves feedback control, which we did not explicitly model as this is outside the scope of our study (we are not aware of any validated feedback models on realistic upper-extremity musculoskeletal models in the literature). We do not expect that explicitly modeling feedback would influence the results. Feedback would ensure that the hand successfully reaches the target in the presence of noise, but planning variability would still impact the MD.

During motor learning, there is a shift in performance that allows movements to be performed faster and more accurately. Previous studies hypothesize that this shift in performance is a result of improvements in the signal-to-noise ratio (i.e., the ability to use large control signals without large motor noise). This is perhaps possible due to expanded neural representations and/or the fine-tuning of individual neurons (Krakauer and Mazzoni, 2011; Shmuelof et al., 2012). Indeed, there is evidence of an improved signal-to-noise ratio during motor skill learning in rodents (Kargo and Nitz, 2004), but their mechanism proposes a reduction in signal-independent noise and pre-burst firing rates. If signal-dependent motor noise alone accounts for the speed-accuracy tradeoff, the mechanism that explains the shift in performance due to practice is not yet fully understood. Our analyses reveal that more stringent task constraints reduce the space of possible control solutions, thereby removing effective solutions from the search landscape (Figure 5). The controller (i.e., the central nervous system) is unable to find the global optimum solution for every task. Thus, for challenging tasks, the patterns of neural activity found during initial optimization (i.e., learning) correspond to ineffective solutions, leading to suboptimal behavior, e.g., reaching slower to smaller targets, consistent with Fitts’ law. Learning can force the motor system to explore different solutions, and if better ones are found, the internal models are updated accordingly resulting in behavioral improvements (Haith and Krakauer, 2018). Hence, our theory does not rely on assumptions regarding improvements in signal-to-noise ratio to explain the shift in performance with practice. A testable prediction of our theory is that the neural variability during motor planning should progressively decrease as the shift in performance occurs during practice (and as the central nervous system converges to better motor plans).

Figure 5

Download asset Open asset

While our theory does not account for signal-dependent noise, the broader mechanism for motor control likely engages both the mechanism proposed here and certain aspects of signal-dependent noise. We exclude signal-dependent noise in our model to demonstrate that it is not necessary to explain the speed-accuracy tradeoff. Adding signal-dependent noise with the same cost function and optimization procedure (that includes motor planning variability) would also yield Fitts’ law-like solutions, but the signal-to-noise ratio would limit how fast the reach to the target can be achieved. One difficulty with modeling signal-dependent noise is that there is very little experimental data on the signal-to-noise ratio and how it changes with practice – if it changes at all. Future work exploring tasks where subjects learn a task across sessions could be quite useful in making more progress toward answering this question. We posit that a realistic biomechanical model, as described here, allows for more accurate estimation of the optimization landscape (i.e., how the local optima are situated with respect to the control solutions). This explains why the realistic biomechanical model produces results that are strongly correlated with prior experimental findings (Goldberg et al., 2014), while all of our simplified models (which includes the torque-driven, fewer muscles, unrealistic force-length relationships, and unrealistic cost function models) are not statistically correlated with prior experimental findings since they are more likely to synthesize unrealistic movements (Table 1).

Our experiments show that Fitts’ law emerges both with a standard and a less effective optimizer. This follows from the fact that for an optimizer with a given performance (i.e., measured by the number of samples and trials that it can explore), imposing progressively more stringent accuracy constraints removes good solutions from the optimization landscape, resulting in performance degradation and thus yielding Fitts’ law. Interesting questions for future work include finding the number of samples and trials in our stochastic optimizer and exploring other forms of stochastic optimization (e.g., Lillicrap et al., 2015) that yields solutions that best match the experimental data on the speed-accuracy tradeoff. A theory based purely on signal-dependent motor noise does not predict that more severe accuracy constraints cause more MD variability, as seen in the experimental data. From this perspective, our theory provides a more complete account of the speed-accuracy tradeoff.

If our theory is correct, then the muscle excitations, activations, and joint kinematics predicted for an ineffective solution should be consistent with what is empirically observed in naïve or ‘non-expert’ human participants. At the same time, our theory predicts that an effective solution would result in muscle patterns that are consistent with empirical observations of highly trained or ‘experts’ performing the task. Future studies can perform this stringent test of our theory by utilizing our modeling framework to find the control solutions for some subset of challenging tasks and compare them against muscle activity measured in a laboratory setting.

In addition to planning variability, limb dynamics play a role in the speed-accuracy tradeoff: one reason why the movement to a smaller target is slower than a larger target with the same center is that a longer distance is required to reach the smaller target; hence, the larger target contains both a larger set of feasible solutions and more effective solutions. The differences in durations between the fastest trials to the smallest and largest targets provide an indicator of the role of limb dynamics in the speed-accuracy tradeoff. The fastest trial approaches the limit of what is dynamically feasible given enough trials. In our computational experiments, this difference amounts to 0.04 s. The average difference from the experimental data is also 0.04 ± 0.03 s (Supplementary file 1). The difference between the mean durations to the two targets minus the difference between the fastest reaches (65% of the total difference in Figure 3C) is attributable to motor planning variability (as we have excluded the roles of limb dynamics and execution noise), which is the dominant factor behind the speed-accuracy tradeoff in our results. The proportions would likely be reversed if the target sizes were drastically different (e.g., comparing targets that were an order of magnitude apart in size).

Our analysis does not preclude the possibility that signal-dependent noise is a factor in the speed-accuracy tradeoff in addition to limb dynamics and planning variability, but it shows that it is not a necessary condition and may not be the dominant factor as previously thought. The data in Supplementary file 1 also shows that the subjects that achieved the smallest difference in durations between the two targets on their best trials also had the largest variability (see subjects 1, 2, and 3). We hypothesize that this variability could be attributed to exploration, which allows for finding better control solutions, consistent with the experimental data. This is also consistent with recent findings that learning is associated with increases in task-relevant variability (Wu et al., 2014).

Turning our attention to neural mechanisms underlying motor cortical function, a current theory suggests that movement period neural activity evolves from an initial condition (termed the neural population preparatory state) set in premotor and primary motor cortex (Shenoy et al., 2013; Vyas et al., 2020a). Different movements have different initial conditions and generating initial conditions closer to the optimal initial condition results in behavioral benefits (Afshar et al., 2011). Furthermore, trial-by-trial behavior variability is strongly linked to neural variability during motor preparation, even for highly practiced movements (Churchland et al., 2006a), and motor learning is strongly linked with systematic changes to neural activity during motor preparation, where neural activity during motor preparation plays a causal role in error-driven learning (Vyas et al., 2018; Vyas et al., 2020b). Given these findings, we introduce a computational model with planning variability and demonstrate its ability to accurately recapitulate many previous results and phenomenon, including Fitts’ law, which suggests a potential role in the speed-accuracy tradeoff. If movement planning variability is a dominant factor underlying the speed-accuracy tradeoff, then the proportion of MD variability explained by preparatory variability should increase with reaches to smaller targets (otherwise motor planning does not contribute to the speed-accuracy tradeoff). We have experimentally verified this prediction (Figure 4E). Our results show that nearly 60% of the MD variability is attributed to planning variability when reaching to the smaller target (and that the MD standard deviation is large relative to the mean with a coefficient of variation of 0.4). When combined with the observation that MD variability increases for smaller targets, this indicates that motor planning variability impacts the scalar parameters found in Fitts’ law (see the section Introduction). Our experiments here further show that reaches to smaller targets are slower and less variable in preparatory activity than reaches to larger targets (Figure 4D), while the MDs are more variable in smaller targets than in larger targets. If the speed-accuracy tradeoff was solely caused by execution noise, then we would expect MD variability to decrease when planning variability decreases. These results, however, are consistent with our theory (Figure 5), which suggests that some of the effective control solutions (i.e., motor plans for fast reaches) have to be discarded when reaching to smaller targets (i.e., variability in preparatory activity decreases, while MD variability increases). We can estimate the relative contribution of planning variability to the speed-accuracy tradeoff as follows. In Figure 4C, the mean difference in the MD to the large and small target is 200 ms, and the standard deviation of the MD to the small target is 258 ms. Given that planning variability explains 60% of the MD variability (Figure 4E), and if we assume a normal distribution around the mean MD, then motor planning can cause the movement to the small target to be 155 ms faster (i.e., one standard deviation from the mean). This suggests that planning variability can account for more than 75% of the MD difference when reaching to the large and small target (i.e., 0.6 × 258/200). We note that this is an initial theory, and future studies will need to evaluate its range of applications under additional experimental conditions.

Our results here (Figure 4D), and those by Churchland and colleagues (Churchland et al., 2006a; Churchland et al., 2006b; Churchland et al., 2006c), point to preparatory activity as also being a central source of movement variability, rather than execution noise alone being the key factor. Our work presumes that the movement variability resulting from motor planning variability can be modeled with the solutions found from stochastic optimization (but we do not claim that a stochastic optimization is actually being literally performed by the brain during motor planning). In other words, during stochastic optimization, many possible candidate motor plans will be explored before converging to a particular motor plan. Our account of Fitts’ law does not depend on the optimization time or of the candidate motor plans explored before convergence. Rather, our results show that, at convergence (i.e., the solutions from stochastic optimization), when reaching to smaller targets, the motor plans tend to be less effective (i.e., slower reaches) because of task constraints (Figure 5). When comparing the reaches to a large and a small target, we have empirically verified that the reaches to the large target that also do not reach the small target are on average faster than the reaches to the small target. Our computational model implements motor planning stochasticity with covariance matrix adaptation evolution strategy (CMA-ES), which operates by applying Gaussian noise on an initial motor plan. It is unknown how motor planning stochasticity arises at the individual neural and neural population level, but it could be as simple as noise on an initial motor plan (e.g., due to variability in the inputs from other brain regions [Vyas et al., 2020b]). We have approximated the trial-to-trial variability in the preparatory state in our model via sampling from a Gaussian distribution. Although we cannot describe the exact neurophysiological mechanism during preparation that produces differential effects on variability when reaching to targets of different size, our contribution lies in discovering this relationship. In summary, we know from neurophysiological findings that a large proportion of the behavioral variability occurring in highly practiced tasks can be attributed to motor planning variability (Churchland et al., 2006a). When implementing planning variability in a computational model, we obtain MDs that are consistent with Fitts' law and experimental findings on MD variability (Figure 3C). This indicates a potential role for planning variability in giving rise to the speed-accuracy tradeoff, which is supported by further neurophysiological evidence introduced here (Figure 4D and Figure 4E).

It is important to note a few important caveats and considerations regarding our theory. First, we do not have causal evidence linking neural variability during preparation and Fitts’ law (although we have a causal link between variability during preparation and Fitts’ law in our computational model). Second, we cannot assert that stochasticity in the model’s motor plan is a perfect proxy for variance in the motor cortical preparatory state, but our working hypothesis is that there is enough alignment for our theory to hold given that our simulations match well with experimental data (see the section Speed-accuracy tradeoff and Fitts’ law); future work will need to be done to explore the extent of this alignment. Finally, while the preparatory state does not encode the whole reach trajectory (as the model’s motor plan does), it does encode important parameters of the upcoming reach, such as distance, direction, and speed (Messier and Kalaska, 2000; Churchland et al., 2006b; Even-Chen et al., 2019; Vyas et al., 2020a). One tantalizing interpretation of our results is that Fitts’ law is tightly linked to movement planning (which includes many of the aspects of the preparatory activity analyzed here) and not just movement execution. The difficulty associated with finding optimal solutions for challenging tasks could then be akin to finding the optimal preparatory neural states (as well as learning the appropriate movement period dynamics), where task constraints govern the optimization landscape, as suggested by our theory. This provides a concrete avenue for future studies to further investigate the relative contribution of signal-dependent noise and motor planning in giving rise to the speed-accuracy tradeoff.

The source code for the computer simulations and our data are available at https://simtk.org/projects/ue-reaching. Users must first create a free account (https://simtk.org/account/register.php) before they can download the datasets from the site.

1. 1. Afshar A
   2. Santhanam G
   3. Yu BM
   4. Ryu SI
   5. Sahani M
   6. Shenoy KV

   (2011) Single-Trial neural correlates of arm movement preparation

   Neuron 71:555–564.

   https://doi.org/10.1016/j.neuron.2011.05.047

   • Google Scholar
2. 1. Al Borno M
   2. Lasa MD

   (2012) Trajectory optimization for full-body movements with complex contacts

   IEEE Transactions on Visualization and Computer Graphics 8:1405–1414.

   https://doi.org/10.1109/TVCG.2012.325

   • Google Scholar
3. 1. Alexander RM

   (1997) A minimum energy cost hypothesis for human arm trajectories

   Biological Cybernetics 76:97–105.

   https://doi.org/10.1007/s004220050324

   • PubMed
   • Google Scholar
4. 1. Ames KC
   2. Ryu SI
   3. Shenoy KV

   (2014) Neural dynamics of reaching following incorrect or absent motor preparation

   Neuron 81:438–451.

   https://doi.org/10.1016/j.neuron.2013.11.003

   • PubMed
   • Google Scholar
5. 1. Beer RF
   2. Dewald JP
   3. Rymer WZ

   (2000) Deficits in the coordination of multijoint arm movements in patients with hemiparesis: evidence for disturbed control of limb dynamics

   Experimental Brain Research 131:305–319.

   https://doi.org/10.1007/s002219900275

   • PubMed
   • Google Scholar
6. 1. Beggs WDA
   2. Howarth CI

   (1972) The movement of the hand towards a target

   Quarterly Journal of Experimental Psychology 24:448–453.

   https://doi.org/10.1080/14640747208400304

   • Google Scholar
7. 1. Bullock D
   2. Grossberg S

   (1988) Neural dynamics of planned arm movements: emergent invariants and speed-accuracy properties during trajectory formation

   Psychological Review 95:49–90.

   https://doi.org/10.1037/0033-295X.95.1.49

   • PubMed
   • Google Scholar
8. 1. Churchland MM
   2. Afshar A
   3. Shenoy KV

   (2006a) A central source of movement variability

   Neuron 52:1085–1096.

   https://doi.org/10.1016/j.neuron.2006.10.034

   • PubMed
   • Google Scholar
9. 1. Churchland MM
   2. Santhanam G
   3. Shenoy KV

   (2006b) Preparatory activity in premotor and motor cortex reflects the speed of the upcoming reach

   Journal of Neurophysiology 96:3130–3146.

   https://doi.org/10.1152/jn.00307.2006

   • PubMed
   • Google Scholar
10. 1. Churchland MM
    2. Yu BM
    3. Ryu SI
    4. Santhanam G
    5. Shenoy KV

    (2006c) Neural variability in premotor cortex provides a signature of motor preparation

    Journal of Neuroscience 26:3697–3712.

    https://doi.org/10.1523/JNEUROSCI.3762-05.2006

    • PubMed
    • Google Scholar
11. 1. Churchland MM
    2. Cunningham JP
    3. Kaufman MT
    4. Foster JD
    5. Nuyujukian P
    6. Ryu SI
    7. Shenoy KV

    (2012) Neural population dynamics during reaching

    Nature 487:51–56.

    https://doi.org/10.1038/nature11129

    • PubMed
    • Google Scholar
12. 1. Delp SL
    2. Anderson FC

    (2007) OpenSim: open-source software to create and analyze dynamic simulations of movement

    IEEE Transactions on Biomedical Engineering 11:1940–1950.

    https://doi.org/10.1109/TBME.2007.901024

    • Google Scholar
13. 1. DeWolf T
    2. Stewart TC
    3. Slotine J-J
    4. Eliasmith C

    (2016) A spiking neural model of adaptive arm control

    Proceedings of the Royal Society B: Biological Sciences 283:20162134.

    https://doi.org/10.1098/rspb.2016.2134

    • Google Scholar
14. 1. Doeringer JA
    2. Hogan N

    (1998) Serial processing in human movement production

    Neural Networks 11:1345–1356.

    https://doi.org/10.1016/S0893-6080(98)00083-5

    • Google Scholar
15. 1. Even-Chen N
    2. Sheffer B
    3. Vyas S
    4. Ryu SI
    5. Shenoy KV

    (2019) Structure and variability of delay activity in premotor cortex

    PLOS Computational Biology 15:e1006808.

    https://doi.org/10.1371/journal.pcbi.1006808

    • PubMed
    • Google Scholar
16. 1. Fagg AH
    2. Shah A
    3. Barto AG

    (2002) A computational model of muscle recruitment for wrist movements

    Journal of Neurophysiology 88:3348–3358.

    https://doi.org/10.1152/jn.00621.2002

    • PubMed
    • Google Scholar
17. 1. Fitts PM

    (1954) The information capacity of the human motor system in controlling the amplitude of movement

    Journal of Experimental Psychology 47:381–391.

    https://doi.org/10.1037/h0055392

    • PubMed
    • Google Scholar
18. 1. Flash T
    2. Hogan N

    (1985) The coordination of arm movements: an experimentally confirmed mathematical model

    The Journal of Neuroscience 5:1688–1703.

    https://doi.org/10.1523/JNEUROSCI.05-07-01688.1985

    • PubMed
    • Google Scholar
19. 1. Goldberg K
    2. Faridani S
    3. Alterovitz R

    (2014) Two large Open-Access datasets for Fitts’ Law of Human Motion and a Succinct Derivation of the Square-Root Variant

    IEEE Transactions on Human-Machine Systems 45:62–73.

    https://doi.org/10.1109/THMS.2014.2360281

    • Google Scholar
20. Book

    1. Goodfellow I
    2. Bengio Y
    3. Courville A

    (2016)

    Deep Learning

    Cambridge: MIT press.

    • Google Scholar
21. 1. Guigon E
    2. Baraduc P
    3. Desmurget M

    (2007) Computational motor control: redundancy and invariance

    Journal of Neurophysiology 97:331–347.

    https://doi.org/10.1152/jn.00290.2006

    • PubMed
    • Google Scholar
22. 1. Haith AM
    2. Reppert TR
    3. Shadmehr R

    (2012) Evidence for hyperbolic temporal discounting of reward in control of movements

    Journal of Neuroscience 32:11727–11736.

    https://doi.org/10.1523/JNEUROSCI.0424-12.2012

    • Google Scholar
23. 1. Haith AM
    2. Krakauer JW

    (2018) The multiple effects of practice: skill, habit and reduced cognitive load

    Current Opinion in Behavioral Sciences 20:196–201.

    https://doi.org/10.1016/j.cobeha.2018.01.015

    • PubMed
    • Google Scholar
24. Book

    1. Hansen N

    (2006) The CMA evolution strategy: a comparing review

    In: Lozano J, editors. Towards a New Evolutionary Computation. Springer. pp. 75–102.

    https://doi.org/10.1007/3-540-32494-1_4

    • Google Scholar
25. 1. Harris CM
    2. Wolpert DM

    (1998) Signal-dependent noise determines motor planning

    Nature 394:780–784.

    https://doi.org/10.1038/29528

    • PubMed
    • Google Scholar
26. 1. Heitz RP

    (2014) The speed-accuracy tradeoff: history, physiology, methodology, and behavior

    Frontiers in Neuroscience 8:150.

    https://doi.org/10.3389/fnins.2014.00150

    • Google Scholar
27. 1. Jones KE
    2. Hamilton AF
    3. Wolpert DM

    (2002) Sources of signal-dependent noise during isometric force production

    Journal of Neurophysiology 88:1533–1544.

    https://doi.org/10.1152/jn.2002.88.3.1533

    • PubMed
    • Google Scholar
28. 1. Kargo WJ
    2. Nitz DA

    (2004) Improvements in the signal-to-noise ratio of motor cortex cells distinguish early versus late phases of motor skill learning

    Journal of Neuroscience 24:5560–5569.

    https://doi.org/10.1523/JNEUROSCI.0562-04.2004

    • PubMed
    • Google Scholar
29. 1. Krakauer JW
    2. Mazzoni P

    (2011) Human sensorimotor learning: adaptation, skill, and beyond

    Current Opinion in Neurobiology 21:636–644.

    https://doi.org/10.1016/j.conb.2011.06.012

    • PubMed
    • Google Scholar
30. Preprint

    1. Lillicrap TP
    2. Hunt JJ
    3. Pritzel A
    4. Heess N
    5. Erez T
    6. Tassa Y
    7. Silver D
    8. Wierstra D

    (2015) Continuous control with deep reinforcement learning

    arXiv.

    https://arxiv.org/abs/1509.02971

    • Google Scholar
31. 1. Lunardini F
    2. Bertucco M
    3. Casellato C
    4. Bhanpuri N
    5. Pedrocchi A
    6. Sanger TD

    (2015) Speed-Accuracy Trade-Off in a Trajectory-Constrained Self-Feeding task: a quantitative index of unsuppressed motor noise in children with dystonia

    Journal of Child Neurology 30:1676–1685.

    https://doi.org/10.1177/0883073815578526

    • PubMed
    • Google Scholar
32. 1. Matthews PB

    (1996) Relationship of firing intervals of human motor units to the trajectory of post-spike after-hyperpolarization and synaptic noise

    The Journal of Physiology 492 (Pt 2):597–628.

    https://doi.org/10.1113/jphysiol.1996.sp021332

    • PubMed
    • Google Scholar
33. 1. McCrea PH
    2. Eng JJ

    (2005) Consequences of increased neuromotor noise for reaching movements in persons with stroke

    Experimental Brain Research 162:70–77.

    https://doi.org/10.1007/s00221-004-2106-8

    • PubMed
    • Google Scholar
34. 1. Messier J
    2. Kalaska JF

    (2000) Covariation of primate dorsal premotor cell activity with direction and amplitude during a memorized-delay reaching task

    Journal of Neurophysiology 84:152–165.

    https://doi.org/10.1152/jn.2000.84.1.152

    • PubMed
    • Google Scholar
35. 1. Millard M
    2. Uchida T
    3. Seth A
    4. Delp SL

    (2013) Flexing computational muscle: modeling and simulation of musculotendon dynamics

    Journal of Biomechanical Engineering 135:021005.

    https://doi.org/10.1115/1.4023390

    • PubMed
    • Google Scholar
36. 1. Nagasaki H

    (1989) Asymmetric velocity and acceleration profiles of human arm movements

    Experimental Brain Research 74:319–326.

    https://doi.org/10.1007/BF00248865

    • PubMed
    • Google Scholar
37. 1. Ostry DJ
    2. Cooke JD
    3. Munhall KG

    (1987) Velocity curves of human arm and speech movements

    Experimental Brain Research 68:37–46.

    https://doi.org/10.1007/BF00255232

    • PubMed
    • Google Scholar
38. 1. Peternel L
    2. Sigaud O
    3. Babič J

    (2017) Unifying Speed-Accuracy Trade-Off and Cost-Benefit Trade-Off in human reaching movements

    Frontiers in Human Neuroscience 11:615.

    https://doi.org/10.3389/fnhum.2017.00615

    • PubMed
    • Google Scholar
39. Book

    1. Posa M
    2. Tedrake R

    (2013)

    Direct trajectory optimization of rigid body dynamical systems through contact

    In: Frazzoli E, Lozano-Perez T, Roy N, Rus D, editors. Algorithmic Foundations of Robotics X. Springer. pp. 527–542.

    • Google Scholar
40. Report

    1. Roetenberg D
    2. Luinge H
    3. Slycke P

    (2009)

    Xsens MVN: Full 6dof Human Motion Tracking Using Miniature Inertial Sensors

    Xsens Motion Technologies BV, Tech.

    • Google Scholar
41. 1. Saul KR
    2. Hu X
    3. Goehler CM
    4. Vidt ME
    5. Daly M
    6. Velisar A
    7. Murray WM

    (2015) Benchmarking of dynamic simulation predictions in two software platforms using an upper limb musculoskeletal model

    Computer Methods in Biomechanics and Biomedical Engineering 18:1445–1458.

    https://doi.org/10.1080/10255842.2014.916698

    • PubMed
    • Google Scholar
42. 1. Shadmehr R

    (2017) Distinct neural circuits for control of movement vs. holding still

    Journal of Neurophysiology 117:1431–1460.

    https://doi.org/10.1152/jn.00840.2016

    • PubMed
    • Google Scholar
43. 1. Shenoy KV
    2. Sahani M
    3. Churchland MM

    (2013) Cortical control of arm movements: a dynamical systems perspective

    Annual Review of Neuroscience 36:337–359.

    https://doi.org/10.1146/annurev-neuro-062111-150509

    • PubMed
    • Google Scholar
44. 1. Sherman MA
    2. Seth A
    3. Delp SL

    (2011) Simbody: multibody dynamics for biomedical research

    Procedia IUTAM 2:241–261.

    https://doi.org/10.1016/j.piutam.2011.04.023

    • Google Scholar
45. 1. Shmuelof L
    2. Krakauer JW
    3. Mazzoni P

    (2012) How is a motor skill learned? change and invariance at the levels of task success and trajectory control

    Journal of Neurophysiology 108:578–594.

    https://doi.org/10.1152/jn.00856.2011

    • PubMed
    • Google Scholar
46. Conference

    1. Sketch SM
    2. Simpson CS
    3. Crevecoeur F
    4. Okamura AM

    (2017) Simulating the impact of sensorimotor deficits on reaching performance

    International Conference on Rehabilitation Robotics (ICORR). pp. 31–37.

    https://doi.org/10.1109/ICORR.2017.8009217

    • Google Scholar
47. 1. Soechting JF

    (1984) Effect of target size on spatial and temporal characteristics of a pointing movement in man

    Experimental Brain Research 54:121–132.

    https://doi.org/10.1007/BF00235824

    • PubMed
    • Google Scholar
48. 1. Tanaka H
    2. Tai M
    3. Qian N

    (2004) Different predictions by the minimum variance and minimum torque-change models on the skewness of movement velocity profiles

    Neural Computation 16:2021–2040.

    https://doi.org/10.1162/0899766041732431

    • PubMed
    • Google Scholar
49. Conference

    1. Todorov E
    2. Li W

    (2005)

    A generalized iterative LQG method for locally-optimal feedback control of constrained nonlinear stochastic systems.

    Proceedings of the 2005, American Control Conference. pp. 300–306.

    • Google Scholar
50. 1. Trautmann EM
    2. Stavisky SD
    3. Lahiri S
    4. Ames KC
    5. Kaufman MT
    6. O'Shea DJ
    7. Vyas S
    8. Sun X
    9. Ryu SI
    10. Ganguli S
    11. Shenoy KV

    (2019) Accurate estimation of neural population dynamics without spike sorting

    Neuron 103:292–308.

    https://doi.org/10.1016/j.neuron.2019.05.003

    • PubMed
    • Google Scholar
51. 1. van der Wel RPRD
    2. Sternad D
    3. Rosenbaum DA

    (2009) Moving the arm at different rates: slow movements are avoided

    Journal of Motor Behavior 42:29–36.

    https://doi.org/10.1080/00222890903267116

    • Google Scholar
52. 1. Vyas S
    2. Even-Chen N
    3. Stavisky SD
    4. Ryu SI
    5. Nuyujukian P
    6. Shenoy KV

    (2018) Neural population dynamics underlying motor learning transfer

    Neuron 97:1177–1186.

    https://doi.org/10.1016/j.neuron.2018.01.040

    • PubMed
    • Google Scholar
53. 1. Vyas S
    2. Golub MD
    3. Sussillo D
    4. Shenoy KV

    (2020a) Computation through neural population dynamics

    Annual Review of Neuroscience 43:249–275.

    https://doi.org/10.1146/annurev-neuro-092619-094115

    • PubMed
    • Google Scholar
54. 1. Vyas S
    2. O'Shea DJ
    3. Ryu SI
    4. Shenoy KV

    (2020b) Causal role of motor preparation during Error-Driven learning

    Neuron 106:329–339.

    https://doi.org/10.1016/j.neuron.2020.01.019

    • PubMed
    • Google Scholar
55. 1. Wickelgren WA

    (1977) Speed-accuracy tradeoff and information processing dynamics

    Acta Psychologica 41:67–85.

    https://doi.org/10.1016/0001-6918(77)90012-9

    • Google Scholar
56. 1. Wood F
    2. Black MJ
    3. Vargas-Irwin C
    4. Fellows M
    5. Donoghue JP

    (2004) On the variability of manual spike sorting

    IEEE Transactions on Biomedical Engineering 51:912–918.

    https://doi.org/10.1109/TBME.2004.826677

    • PubMed
    • Google Scholar
57. 1. Wu HG
    2. Miyamoto YR
    3. Gonzalez Castro LN
    4. Ölveczky BP
    5. Smith MA

    (2014) Temporal structure of motor variability is dynamically regulated and predicts motor learning ability

    Nature Neuroscience 17:312.

    https://doi.org/10.1038/nn.3616

    • PubMed
    • Google Scholar
58. 1. Zajac E

    (1989)

    Muscle and tendon: properties, models, scaling, and application to biomechanics and motor control

    Critical Reviews in Biomedical Engineering 4:359–411.

    • Google Scholar

Acceptance summary:

The present modelling study and non-human primate experiments suggest a new theory for the origin of the speed-accuracy tradeoff, a key feature of motor behavior across effectors and contexts. The authors posit that, rather than signal-dependent noise, the speed-accuracy trade-off arises because of planning variability. That is, more difficult task demands (i.e. smaller targets) reduce the availability of good control solutions (i.e. fast reaches).

Decision letter after peer review:

Thank you for submitting your article "High-fidelity musculoskeletal modeling reveals a motor planning contribution to the speed-accuracy tradeoff" for consideration by eLife. Your article has been reviewed by three peer reviewers, including Andrew Pruszynski as the Reviewing Editor and Reviewer #1, and the evaluation has been overseen by Ronald Calabrese as the Senior Editor. The following individual involved in review of your submission has agreed to reveal their identity: Friedl de Groote (Reviewer #3).

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

As the editors have judged that your manuscript is of interest, but as described below that additional work is required before it is published, we would like to draw your attention to changes in our revision policy that we have made in response to COVID-19 (https://elifesciences.org/articles/57162). First, because many researchers have temporarily lost access to the labs, we will give authors as much time as they need to submit revised manuscripts. We are also offering, if you choose, to post the manuscript to bioRxiv (if it is not already there) along with this decision letter and a formal designation that the manuscript is "in revision at eLife". Please let us know if you would like to pursue this option. (If your work is more suitable for medRxiv, you will need to post the preprint yourself, as the mechanisms for us to do so are still in development.)

Summary:

The present paper presents a novel theory for the origin of Fitts' Law, that is, the speed-accuracy trade-off observed across movement tasks. The most prominent current theory suggests that the speed-accuracy tradeoff emerges due to the optimization of movements in the face of signal-dependent noise. The authors suggest that, under a physiologically realistic simulation of the arm, motor planning in the absence of motor noise can also give rise the speed-accuracy trade-off. The simulation predictions are coupled with behavioral data from a straightforward human reaching experiment and the simulations are linked to a potential underlying neural mechanism via single-units recordings obtained from a macaque monkey doing a similar reaching task.

All three reviewers thought that this is an interesting and provocative study that may provide an important new perspective on the origins of Fitts' Law, the implications of which would be very broad. However, all three reviewers converged on a set of common concerns about the current version of the paper that need to be addressed.

Essential revisions:

1) All the reviewers raised substantial concerns about how the authors handle physiological noise in their simulations. There were two related issues.

a) In the present version of the model, muscle activation terms are squared and the authors suggest that this serves as an approximation of metabolic power consumption. The reviewers raised the point that, in other contexts, minimizing squared muscle activations has been interpreted as minimizing motor noise (i.e. acts like the kind of signal-dependent noise term). It is important that the authors clarify how this term influences the ability to make a clean separation between the hypotheses being tested.

b) Physiological noise is an established phenomenon that is not included in the present model. The reviewers understand why the authors chose this route (both conceptually and practically) but thought it would be important to show what happens when physiological noise is included. In this case, would the same optimization procedure with the same cost function converge similarly on a Fitts law-like solution? How would the optimized control signals in this case differ from the ones they found presently?

2) All the reviewers raised the substantial concern that the present work does not establish a meaningful link between the optimal control model being proposed – and its reasons for showing Fitts' Law – and a sensible neurophysiological analog. That is, what neurally feasible computational mechanism occurs during movement preparation that produces greater variability for small targets. Given the present model algorithm does not seem feasible, what to the authors have in mind? At the very least, the authors need to provide more complete discussion about how their algorithm relates or does not relate to neurophysiological mechanisms.

3) The authors claim that a major factor that allows them to better investigate the origins of Fitt's Law is their physiologically realistic model of the upper limb. However the authors do not make any attempts to investigate which aspects of the model specifically are responsible, either alone, or in combination, for the observed effects. An analysis of the sort described by Lillicrap and Scott (2013, Neuron) would seem helpful here.

4) The description of the approach lacks important details. CMA-ES with a population size of 20 was used but it is not clear what parameter settings are used for sampling and how these settings influence the results. The authors refer to the solutions of the CMA-ES algorithm as local optima but this may not be mathematically correct as the number of iterations was predefined and optimality criteria were not evaluated. Also, it seems that the target was not successfully reached in all trials. For example in Figure 1B, the most rightward target is not reached. Finally, muscle excitations are prescribed every 100 ms only. This limits the set of feasible solutions. Would the results be different when increasing the number of controls (which might be more realistic physiologically)?

5) Methods related to the NHP work are lacking. In Figure 4, the authors talk about not using spike sorting and just using threshold crossings. Why? What are the implications of this approach. In general, methods associated with the monkey data are not described in the Materials and methods section. This needs substantial work to be acceptable.

[Editors' note: further revisions were suggested prior to acceptance, as described below.]

Thank you for resubmitting your article "High-fidelity musculoskeletal modeling reveals a motor planning contribution to the speed-accuracy tradeoff" for consideration by eLife. Your revised article has been reviewed by three peer reviewers, including Andrew Pruszynski as Reviewing Editor and Reviewer #1, and the evaluation has been overseen by Ronald Calabrese as the Senior Editor. The following individual involved in review of your submission has agreed to reveal their identity: Friedl de Groote (Reviewer #3).

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

Summary:

As you will see, the reviewers all thought that you did a very good job addressing their concerns and remain very enthusiastic about your work. There are however two important issues that remain outstanding, which the reviewers all believe need addressing to ensure the paper provides the advance it hopes to provide.

Revisions for this paper:

1) Previous concern #3 was not addressed and is seen by all reviewers as being very important for interpreting the findings. Specifically, the comparison is still limited to the torque-driven model and the fully complex model. The reviewers believe that it would be feasible to follow the approach of Lillicrap and Scott (2003) to assess which aspects of the model are responsible for the observed effect and that the insights gained from such assessment would be very valuable.

2) There remains a disconnect between the results of the modeling exercise and the neurophysiological data - a gap that one might expect to be bridged by a theory or model, which is still absent. In the simulations, the model behaves according to Fitt's Law because built into the model is a sort of explicit "search" within a subspace, a search that takes time. In the model this search happens pre-movement during what the authors call a movement planning epoch. The neural data are centered around a pre-movement time epoch as well, but clearly the brain is not literally executing an algorithmic "search" during this time. The reviewers all believe that, for this paper to have the desired impact, more discussion is needed to bridge the gap between the model and the nervous system.

Essential revisions:

1) All the reviewers raised substantial concerns about how the authors handle physiological noise in their simulations. There were two related issues.

a) In the present version of the model, muscle activation terms are squared and the authors suggest that this serves as an approximation of metabolic power consumption. The reviewers raised the point that, in other contexts, minimizing squared muscle activations has been interpreted as minimizing motor noise (i.e. acts like the kind of signal-dependent noise term). It is important that the authors clarify how this term influences the ability to make a clean separation between the hypotheses being tested.

This is a good point, and we thank the reviewers for raising it. Our cost function detailed in Equation 1, includes the sum of muscle activations squared. Including this term impacts the speed of a synthesized reaching movement to a target, but it does not explain the empirical observation that movements are slower to smaller targets. Simply adding the muscle activations term (without signal-dependent noise or motor planning variability) would predict the same movement duration for a large and a small target for the same reach distance (i.e., if the smaller target is placed at the edge of the larger target). Yet, Fitts’ law predicts slower movements to smaller targets. We verify these considerations by performing additional experiments to show that Fitts’ law emerges even without the muscle activations term (setting $w_2$ to 0 in Equation 1; see subsection “Speed-Accuracy Tradeoff and Fitts’ law”). We found that the model’s mean predictions are in close agreement with Fitts’ law (R2 = 0.955). We added the figure in the manuscript as (Figure 3—figure supplement 3).

“Our cost function (Equation 1) includes a term to minimize the sum of muscle activations squared, which impacts the speed of reaching movements (but unlike signal-dependent noise, it does not predict different speeds based on target size). We performed an additional experiment (Figure 3—figure supplement 3) to show that Fitts’ law emerges even without this term in the cost function.”

b) Physiological noise is an established phenomenon that is not included in the present model. The reviewers understand why the authors chose this route (both conceptually and practically) but thought it would be important to show what happens when physiological noise is included. In this case, would the same optimization procedure with the same cost function converge similarly on a Fitts law-like solution? How would the optimized control signals in this case differ from the ones they found presently?

We appreciate having the opportunity to discuss this issue in more detail. Adding signal-dependent noise with the same cost function and the same optimization procedure (that includes motor planning variability) would converge on a Fitts’ law-like solution. The signal-to-noise ratio in the signal-dependent noise would limit how fast the reach to the target can be achieved. Hence, the signal-dependent noise would simply increase the movement duration, but the resulting curves would still be in agreement with Fitts’ law. Motor planning variability would still cause the movement duration variability to increase for smaller targets. Improvements that occur with practice could be explained either by improvements in the signal-to-noise ratio (which is not yet demonstrated experimentally), improvements in the motor plan itself (which has been demonstrated experimentally), or a combination of the two.

Adding signal-dependent noise would cause the optimized control signals to be smaller when reaching to the smaller target, but not to the larger target. This readily illustrates the difference between signal-dependent noise and the muscle activations term discussed in 1.a (which impacts the movements to both the smaller and larger targets). Adding signal-dependent noise alone without motor planning variability would not explain the behavior shown in Figure 3C or the neural data shown in Figure 4D-E.

“We exclude signal-dependent noise in our model to demonstrate that it is not necessary to explain the speed-accuracy tradeoff. Adding signal-dependent noise with the same cost function and optimization procedure (that includes motor planning variability) would also yield Fitts’ law-like solutions, but the signal-to-noise ratio would limit how fast the reach to the target can be achieved. One difficulty with modeling signal-dependent noise is that there is very little experimental data on the signal-to-noise ratio and how it changes with practice — if it changes at all. Future work exploring tasks where subjects learn a task across sessions could be quite useful in making more progress towards answering this question.”

2) All the reviewers raised the substantial concern that the present work does not establish a meaningful link between the optimal control model being proposed – and its reasons for showing Fitts' Law – and a sensible neurophysiological analog. That is, what neurally feasible computational mechanism occurs during movement preparation that produces greater variability for small targets. Given the present model algorithm does not seem feasible, what to the authors have in mind? At the very least, the authors need to provide more complete discussion about how their algorithm relates or does not relate to neurophysiological mechanisms.

We thank the reviewers for highlighting that our manuscript did not address these questions. First, prior to this study, there was little evidence to suggest any relationship between Fitts’ law and the preparatory state in motor cortex. We have now presented, to our knowledge, the first empirical evidence that such a relationship exits. That is, reaching to smaller targets exhibit less variability in the distribution of preparatory states in premotor cortex (Figure 4D); at the same time, for smaller targets a larger fraction (and the majority) of the trial-by-trial variability is explained by variability in the preparatory state (compared to larger targets; Figure 4E). Thus Fitts’ law is not entirely a result of signal-dependent noise, but also a function of neural variability during motor preparation. This is a new neurophysiological finding, which forms the basis of our proposed theory (see next paragraph).

We attempt to further investigate variability during preparation by engaging a computational model that by virtue of stochasticity during motor planning yields movement durations consistent with Fitts’ law. In particular, the model implements motor planning stochasticity with CMA-ES, which operates by applying gaussian noise on an initial motor plan.

We recognize this is not the source of neural variability during preparation in (monkey) motor cortex, but it could be as simple as noise on an initial motor plan (e.g., due to variability in the inputs from other brain regions [Vyas et al., 2020b]). We have approximated the trial-to-trial variability in the preparatory state in our model via sampling from a gaussian distribution. Although we cannot describe the exact neurophysiological mechanism during preparation that produces differential variability when reaching to smaller vs larger targets, our contribution lies in discovering this relationship. Furthermore, the motor plan in the model is not exactly the same as the preparatory state in premotor and primary motor cortex; they are related, but the plan encodes the full trajectory, whereas the preparatory state encodes a subset of the parameters of the upcoming movement (e.g., Messier and Kalaska, 2000; Even-Chen et al., 2019).

We also cannot assert that stochasticity in the model’s motor plan is a perfect proxy for variance in the motor cortical preparatory state, but our working hypothesis is that there is enough alignment for our theory to hold given that our simulations match well with experimental data (see subsection “Speed-Accuracy Tradeoff and Fitts’ law”); future work will need to be done to explore the extent of this alignment.

With these caveats and model assumptions in mind, the key theory being put forward here is that variability in the motor plan yields multiple candidate control solutions (Figure 5). There is less variability (Figure 4D) and fewer effective solutions when reaching to smaller targets, which yields, on average, slower reaches (see Figure 5 caption for a more thorough account of why this holds). We do not have causal evidence linking neural variability during preparation and Fitts’ law (although we have a causal link between variability during preparation and Fitts’ law in our computational model). These are important caveats and considerations regarding our theory, and we apologize for not including them in the previous version of our manuscript; this has now been addressed in the main text (as shown below).

“Our results here (Figure 4D), and those by Churchland and colleagues (Churchland et al., 2006a, 2006b, 2006c), point to preparatory activity as also being a central source of movement variability, rather than execution noise alone being the key factor. […] This provides a concrete avenue for future studies to further investigate the relative contribution of signal-dependent noise and motor planning in giving rise to the speed-accuracy tradeoff.”

3) The authors claim that a major factor that allows them to better investigate the origins of Fitt's Law is their physiologically realistic model of the upper limb. However the authors do not make any attempts to investigate which aspects of the model specifically are responsible, either alone, or in combination, for the observed effects. An analysis of the sort described by Lillicrap and Scott (2013, Neuron) would seem helpful here.

We are grateful to the reviewers for highlighting how we can improve our manuscript. We have revised the discussion to describe how a realistic musculoskeletal model allows us to better investigate the speed-accuracy tradeoff. We have shown that both a realistic musculoskeletal model and a torque-driven model produce mean movement durations that are consistent with Fitts’ law. The realistic model synthesizes movement duration means and standard deviations that are statistically correlated with experimental data (Figure 3A). The torque-driven model, however, synthesizes predicted means and standard deviations that are not statistically correlated with the experimental data. The reason is that a realistic biomechanical model allows us to more accurately estimate the optimization landscape involved in human reaching. A torque-driven model, on the other hand, can synthesize unrealistic movements (i.e., produce torques that would not have been produced by the muscles, either because it would not have been possible physiologically, or because it would have been ineffective with respect to the cost function Equation 1). More accurately estimating the optimization landscape allows us to reproduce the increase in the movement duration variability that occurs when reaching for smaller targets (Figure 3C), which we did not reproduce in our torque-driven model (Figure 3—figure supplement 1). Lastly, a torque-driven or realistic musculoskeletal model with signal-dependent noise and no planning variability produces results consistent with Fitts’ law but does not make any prediction on how movement duration variability changes with the index of difficulty. We have included a table (Author response table 1) to summarize our results.

“We posit that a realistic biomechanical model, as described here, allows for more accurate estimation of the optimization landscape (i.e., how the local optima are situated with respect to the control solutions). This explains why the realistic biomechanical model produce results that are strongly correlated with prior experimental findings, while the simplified torque-driven model does not (since it is more likely to synthesize unrealistic movements).”

4) The description of the approach lacks important details. CMA-ES with a population size of 20 was used but it is not clear what parameter settings are used for sampling and how these settings influence the results. The authors refer to the solutions of the CMA-ES algorithm as local optima but this may not be mathematically correct as the number of iterations was predefined and optimality criteria were not evaluated. Also, it seems that the target was not successfully reached in all trials. For example in Figure 1B, the most rightward target is not reached. Finally, muscle excitations are prescribed every 100 ms only. This limits the set of feasible solutions. Would the results be different when increasing the number of controls (which might be more realistic physiologically)?

a) The reviewer is correct that the solutions of CMA-ES may not be local optima, and we revised the terminology in the paper to refer to effective/ineffective “control solutions”. We have also revised the manuscript to include the details requested by the reviewers, including all the parameters in CMA-ES. Our optimization does not require careful initialization: we set all the muscle excitations to 0.1 and all the standard deviations in the normal distribution to 0.3. We performed the following additional experiment to empirically verify that the optimization converges to similar solutions when setting the muscle excitations to 0.3 and the standard deviations to 0.5. In an average of ten trials, the average movement duration difference between the two CMA-ES parameters were 0.03 s and 0.0005 s for the smallest and largest targets (i.e., small values compared to the predicted mean movement durations in Figure 3C). We also empirically found that the movement would not successfully reach the target if the standard deviations are set too low (e.g., 0.025) or too high (e.g., 1.5). This indicates that our results are not sensitive to small changes to the reported CMA-ES parameters. It might be desirable to choose thousands of different initializations to further test the sensitivity of the results to these parameters; unfortunately, this would take months of computer time on a substantial computer cluster and was infeasible.

b) We updated Figure 1B to show that the rightward target can be reached (we had incorrectly specified the target size in the earlier figure).

c) We have performed the following additional experiment to show that doubling the number of controls (prescribing muscle excitations every 50 ms) does not affect the results. We performed ten reaches to the smallest (index of difficulty: 4.41) and largest targets (index of difficulty: 0.9) and compare the movement durations differences when prescribing the controls every 100 ms and every 50 ms. The average movement duration difference to the smallest target is 0.01 s and 0.0009 s for the largest target, which indicates that increasing the number of controls would not change the results in the paper (i.e., these are small values compared to the predicted mean movement durations in Figure 3C of 0.28 s and 0.08 s for the smallest and largest targets, respectively). We also added a target in between the smallest and largest targets (with an index of difficulty of 2.5) and again only noted a small average movement duration difference of 0.016 s when prescribing the controls every 100 ms and every 50 ms.

We have included the data and code, to be shared openly and freely, for all of our original and additional experiments described here on the following web site which we maintain: https://simtk.org/projects/ue-reaching/.

“We performed experiments to verify that prescribing the muscle excitations at every 0.05 s interval (instead of 0.1 s interval) do not have a significant impact on the computed movement durations.”

5) Methods related to the NHP work are lacking. In Figure 4, the authors talk about not using spike sorting and just using threshold crossings. Why? What are the implications of this approach. In general, methods associated with the monkey data are not described in the Materials and methods section. This needs substantial work to be acceptable.

We apologize for the lack of detailed methods for the NHP work. We relied too heavily on a previously published paper by one of the senior author’s group, but realize now that it is much better for this paper to be much more self-contained. Reproduced below are the new Methods for the NHP work. We have made clear when and where previously published techniques and experimental protocols were followed.

First, regarding spike sorting verses threshold crossings. In this work, we elected to use threshold crossings in lieu of spike sorting. That is, we do not assign individual spikes to individual neurons. Primarily, this allows us to consider all the data on the Utah electrode arrays, not just the well-isolated units. We (Trautmann et al., 2019) have recently established that there is a great deal of signal present on recording channels that do not contain well-isolated single units (i.e., so called “hash”). Thus, it is perhaps less biased to consider all the data that are available, rather than restrict the analysis to only well isolated single units. Nonetheless, we agree with the reviewers that this approach could have undesirable implications. In particular, it is not fully known what effect threshold crossing have on the raw estimation of the captured variance; perhaps threshold crossings yield a more biased estimate of the variance. However, it is worth noting that spike sorting itself is riddled with inaccuracies, especially when cells do not present with prototypical waveforms. Spike sorting also often results in large inter- and intra-operator variability (Wood et al., 2004). Regardless, any mis-estimation of the variance due to threshold crossing is not an issue for the primary claims of this paper for three key reasons. First, multi-units will not artificially affect the captured variance preferentially for any one condition; if anything, potentially lower SNR will make the two conditions appear more similar. Second, Figure 4 shows that the fraction of the trial-by-trial variability explained by preparatory state variability (for the large target) is ~0.5. This is essentially identical to the results found by Churchland et al., 2006a, where spike sorting was performed. Finally, and perhaps more importantly, we are not interested in the absolute values of the captured variance. We are primarily interested in assessing if there is a significant difference between the small and large target conditions; note in 4D-E, we used normalized variance instead of absolute variance.

“Recordings were made from the dorsal aspect of the premotor cortex (PMd) and the primary motor cortex (M1) of one male adult monkey (Macaca mulatta) who was 15 years old and weighed 16 kg at the time of these experiments. […] The analysis presented in Figure 4D was performed following the standard methods described in (Vyas et al., 2018; Even-Chen et al., 2019; Vyas et al., 2020b). The analysis presented in Figure 4E was performed following the standard methods described in Churchland et al., 2006a.”

[Editors' note: further revisions were suggested prior to acceptance, as described below.]

Revisions for this paper:

1) Previous concern #3 was not addressed and is seen by all reviewers as being very important for interpreting the findings. Specifically, the comparison is still limited to the torque-driven model and the fully complex model. The reviewers believe that it would be feasible to follow the approach of Lillicrap and Scott (2003) to assess which aspects of the model are responsible for the observed effect and that the insights gained from such assessment would be very valuable.

We are grateful to the reviewers for highlighting how we can improve our manuscript. We performed additional experiments and analyses to address this concern and identify which components of the model are responsible for the observed effects. Following the approach of Lillicrap and Scott (2003), we introduce six different models: a torque-driven model, a musculoskeletal model missing 14 important muscles (e.g., deltoid anterior, deltoid posterior, infraspinatus, teres minor, etc.), a musculoskeletal model with unrealistic muscle force-length relationships, a musculoskeletal model with an unrealistic cost function, a realistic musculoskeletal model with an effective optimizer, and a realistic musculoskeletal with a less effective optimizer. For each of these models, we tested whether the predicted movement durations are in agreement with Fitts’ law and experimental data. Our results indicate that all models produce movement durations in agreement with Fitts’ law, but only the realistic musculoskeletal models with the effective and less effective optimizers synthesized movement duration means and standard deviations that are statistically correlated with experimental data (Figure 3A).

Our hypothesis is that a realistic biomechanical model and a realistic cost function allows us to more accurately estimate the optimization landscape involved in human reaching. The other models tend to synthesize unrealistic movements (i.e., produce torques that would not have been produced by the muscles, either because it would not have been possible physiologically, or because it would have been ineffective with respect to the cost function Equation 1). More accurately estimating the optimization landscape allows us to reproduce the increase in the movement duration variability that occurs when reaching for smaller targets (Figure 3C), which we did not reproduce in our torque-driven model (Figure 3—figure supplement 1).

Thus, our work points to the importance of accurately modeling the optimization landscape, which includes at least the following components: the biomechanical model, the cost function and the optimization parameters. We note that our results on the speed-accuracy tradeoff are robust to the optimization parameters (both the effective and less effective optimizers are statistically correlated with experimental data). An open problem for future work would be investigate other optimization algorithms and parameters and determine which ones are most consistent with experimental data. Lastly, from the literature (e.g., Harris and Wolpert, 1998), we know that a torque-driven with signal-dependent noise and no planning variability produces results consistent with Fitts’ law but does not make any prediction on how movement duration variability changes with the index of difficulty. We have included a table (Author response table 2) to summarize our results.

We have made the following changes to address these points in the manuscript. We have added Table 1 and Figure 3—figure supplement 4 and Figure 3—figure supplement 5.

In subsection “Speed-Accuracy Tradeoff and Fitts’ law”:

“We also tested the sensitivity of our results to features of the biomechanical model: we performed the experiments with a model with muscles having unrealistic active force-length relationships (i.e., where the force is constant with respect to the muscle length; Figure 3—figure supplement 4) and a model missing important muscles (e.g., deltoid anterior, deltoid posterior, infraspinatus, teres minor; Figure 3—figure supplement 5). We found that these models produced movement durations that are consistent with Fitts’ law, but the predictions were not correlated with the experimental data in Goldberg et al., 2014 (Table 1).”

In the Discussion:

“We posit that a realistic biomechanical model, as described here, allows for more accurate estimation of the optimization landscape (i.e., how the local optima are situated with respect to the control solutions). This explains why the realistic biomechanical model produces results that are strongly correlated with prior experimental findings (Goldberg et al., 2014), while all of our simplified models (which includes the torque-driven, fewer muscles, unrealistic force-length relationships, and unrealistic cost function models) are not statistically correlated with prior experimental findings since they are more likely to synthesize unrealistic movements (Table 1).”

2) There remains a disconnect between the results of the modeling exercise and the neurophysiological data – a gap that one might expect to be bridged by a theory or model, which is still absent. In the simulations, the model behaves according to Fitt's Law because built into the model is a sort of explicit "search" within a subspace, a search that takes time. In the model this search happens pre-movement during what the authors call a movement planning epoch. The neural data are centered around a pre-movement time epoch as well, but clearly the brain is not literally executing an algorithmic "search" during this time. The reviewers all believe that, for this paper to have the desired impact, more discussion is needed to bridge the gap between the model and the nervous system.

We are grateful to the reviewers for highlighting how we can improve the presentation and discussion of our theory. Our modeling exercise and the subsequent non-human primate experiments are suggestive of a new theory, one that hopefully will inspire much more future work on the speed-accuracy tradeoff. Our new proposal can be better understood if placed side-by-side with the prevailing theory (Harris and Wolpert, 1998).

Prevailing theory:

Harris and Wolpert suggest that on a trial where you reach to a small target, you need to modify your control signal to be smaller if you wish to minimize your variability in the endpoint position. This naturally slows down your reaches, thus yielding (some of) the empirical aspects of Fitts’ law. In this theory, if the neural control signal is corrupted by noise that scales with the size of the control signal, then in order to minimize endpoint variability, motor cortex issues commands with smaller overall mean to accomplish the task.

Our proposal:

At its core essence, our theory relies on motor planning variability and questions whether signal-dependent noise is the only factor that ultimately gives rise to Fitts’ law.

1) We start with previous findings by Churchland and colleagues that demonstrate that trial by-trial movement duration variability, even during highly practiced tasks, can be accounted for by motor planning (but more specifically preparatory) variability.

2) Given these findings, we introduce a computational model with planning variability and demonstrate its ability to accurately recapitulate many previous (and sometimes puzzling) results and phenomenon, including Fitts’ law. In our opinion, this alone is a non-trivial contribution.

a) We focus on the modeling assumption that the movement variability resulting from motor planning variability can be modeled with the solutions found from stochastic optimization (though note, we do not think that the brain literally performs a stochastic optimization during motor planning). Our model’s account of Fitts’ law does not rely on the optimization time or the explicit search before finding the motor plans. The optimization search will explore many possible motor plans before converging to a set of possible solutions. Our theory only relates to the final (converged) solutions. Our model needs to perform a search to converge to this set of possible solutions, but the central nervous system does not.

b) Our model shows that for more stringent task constraints, effective solutions are rendered inaccessible (Figure 5). When comparing the reaches to a large and a small target, we have empirically verified that the reaches to the large target that also do not reach the small target are on average faster than the reaches to the small target. This yields on average slower reaches to smaller targets. We hypothesize that this also occurs in the brain when reaching to targets of different sizes.

3) We test this hypothesis by analyzing motor cortical activity of monkey reaching movements. First, we replicate the findings by Churchland and colleagues 2006, and both extend them for harder tasks (i.e., reaching to smaller targets) and show that variability in the preparatory state in motor cortex is associated with some of the empirical aspects of Fitts’ law.

4) Having established that there is a relationship between movement duration variability and planning variability, let’s consider what happens when making reaches to a small target. Empirically, more (and the majority) of the trial-by-trial movement duration variability is accounted for by variability in the preparatory state (compared to a larger target). At the same time, there is less variability in the preparatory state in motor cortex and more movement duration variability. For reaches to a larger target, there is more variability in the preparatory state in motor cortex and less movement duration variability.

5) This new empirical evidence indicates that motor planning variability has a significant contribution to the speed-accuracy tradeoff and is consistent with our theory, which predicts that reaches to larger targets are on average faster because the brain has access to a larger set of effective motor plans compared to smaller targets.

When viewed in this manner, it is the combination of previous neurophysiology, new behavior and neurophysiology, and computational modeling that gives rise to this theory. Our proposal does not require signal-dependent noise. Our proposal explains many of the confusing looking responses seen in previous studies that signal-dependent noise cannot explain. Perhaps the reviewers would agree that this meets the bar for publication at this time. Indeed, significant future work will be needed to pin down an exact neurophysiological mechanism. To put it succinctly, we have proposed a new hypothesis, one that is very different from the prevailing theory, and our succinct hope is that future studies will either deepen or disprove our proposal.

We also added the following testable prediction of our theory. With practice, there’s a shift in in performance that allows movements to be performed faster and more accurately. Our theory predicts that the neural variability during motor planning should progressively decrease as the shift in performance occurs (and as the central nervous system converges to better motor plans).

We have revised the Discussion to clarify the presentation of our theory:

“Turning our attention to neural mechanisms underlying motor cortical function, a current theory suggests that movement period neural activity evolves from an initial condition (termed the neural population preparatory state) set in premotor and primary motor cortex (Shenoy et al., 2013, Vyas et al., 2020a). […] This indicates a potential role for planning variability in giving rise to the speed-accuracy tradeoff, which is supported by further neurophysiological evidence introduced here (Figure 4D and Figure 4E).”

We included our testable prediction here:

“During motor learning, there is a shift in performance that allows movements to be performed faster and more accurately. […] A testable prediction of our theory is that the neural variability during motor planning should progressively decrease as the shift in performance occurs during practice (and as the central nervous system converges to better motor plans).”

Author details

1. Mazen Al Borno

   1. Department of Bioengineering, Stanford University, Stanford, United States
   2. Department of Computer Science and Engineering, University of Colorado Denver, Denver, United States

   Contribution

   Conceptualization, Software, Formal analysis, Investigation, Writing - original draft

   For correspondence

   mazen.alborno@ucdenver.edu

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0003-2208-9934

2. Saurabh Vyas

   Department of Bioengineering, Stanford University, Stanford, United States

   Contribution

   Formal analysis, Investigation, Writing - original draft

   Competing interests

   No competing interests declared

3. Krishna V Shenoy

   1. Department of Bioengineering, Stanford University, Stanford, United States
   2. Neurosciences Program, Stanford University, Stanford, United States
   3. Department of Electrical Engineering, Stanford University, Stanford, United States
   4. Wu Tsai Neuroscience Institute, Stanford University, Stanford, United States
   5. Department of Neurobiology, Stanford University, Stanford, United States
   6. Howard Hughes Medical Institute, Stanford University, Stanford, United States

   Contribution

   Supervision, Funding acquisition, Writing - review and editing

   Competing interests

   No competing interests declared

4. Scott L Delp

   1. Department of Bioengineering, Stanford University, Stanford, United States
   2. Department of Mechanical Engineering, Stanford University, Stanford, United States

   Contribution

   Supervision, Funding acquisition, Writing - review and editing

   Competing interests

   No competing interests declared

• 1,504

  Page views

• 185

  Downloads

• 5

  Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.