Multiple learning processes contribute to successful goal-directed actions in the face of changing physiological states, body structures, and environments (Taylor et al., 2014; Huberdeau et al., 2015; McDougle et al., 2016). Among these processes, implicit sensorimotor adaptation is of primary importance for maintaining appropriate calibration of sensorimotor maps over both short and long timescales. A large body of work has focused on how sensory prediction error (SPE), the difference between predicted and actual sensory feedback, drives sensorimotor adaptation (Shadmehr et al., 2010). In addition, there is growing appreciation of the contribution of other processes to sensorimotor learning, including strategic aiming and reward-based learning (Taylor et al., 2014; Wu et al., 2014; Bond and Taylor, 2015; Galea et al., 2015; Nikooyan and Ahmed, 2015; Summerside et al., 2018). In terms of the latter, several recent studies have shown that rewarding successful actions alone is sufficient to learn a new sensorimotor mapping (Izawa and Shadmehr, 2011; Therrien et al., 2016; Therrien et al., 2018).

Little is known about how feedback about task outcome impacts adaptation from SPE; indeed, the literature presents an inconsistent picture of how reward impacts performance in sensorimotor adaptation tasks. For example, two recent visuomotor rotation studies using similar tasks and reward structures led to divergent conclusions: One reported that reward enhanced retention of the adapted state, but had no effect on the rate of adaptation (Galea et al., 2015), whereas the other reported a beneficial effect of rewards specifically on adaptation rate (Nikooyan and Ahmed, 2015). More recently, Leow and colleagues (Leow et al., 2018) created a situation in which task outcome was experimentally manipulated by shifting the target on-line to either intersect a rotated cursor or move away from the cursor. Task success, artificially imposed by allowing the displaced cursor to intersect the target, led to attenuated adaptation.

One factor that may contribute to these inconsistencies is highlighted by studies showing that, even in relatively simple sensorimotor adaptation tasks, overall behavior reflects a combination of explicit and implicit processes (Taylor and Ivry, 2011; Taylor et al., 2014). That is, while SPE is thought to drive adaptation (Tseng et al., 2007), participants are often consciously aware of the perturbation and strategically aim as one means to counteract the perturbation. It may be that reward promotes the activation of such explicit processes (Bond and Taylor, 2015). Consistent with this hypothesis, Codol and colleagues (Codol et al., 2017), showed that at least one of the putative effects of reward, the strengthening of motor memories (Shmuelof et al., 2012), is primarily the result of re-instantiating an explicit aiming strategy rather than via the direct modulation of adaptation. As explicit processes are more flexible than implicit processes (Bond and Taylor, 2015), differential demands on strategies may contribute toward the inconsistent effects reported across previous studies manipulating reward (Holland et al., 2018).

We recently introduced a new method, referred to as clamped visual feedback, designed to isolate implicit adaptation (Morehead et al., 2017; Kim et al., 2018). During the clamp, the angular trajectory of the feedback cursor is invariant with respect to the target location and thus spatially independent of hand position (Shmuelof et al., 2012; Vaswani et al., 2015; Morehead et al., 2017; Kim et al., 2018; Vandevoorde and Orban de Xivry, 2018). Participants are informed of the invariant nature of the visual feedback and instructed to ignore it. In this way, explicit aiming should be eliminated and, thus, allow for a clean probe of implicit learning (Morehead et al., 2017).

Here, we employ the clamp method to revisit how task outcome, even when divorced from actual performance, influences implicit adaptation. In a series of three experiments, the clamp angle was held constant and only the target size was manipulated. We assume that the clamp angle, defined with respect to the centers of the target and feedback cursor, specifies the SPE. In contrast, by varying the target size, we independently manipulate the information regarding task outcome, comparing conditions in which the feedback cursor signals the presence or absence of a target error (TE), defined in a binary manner by whether the cursor misses or hits the target. Given that the participants are aware that they have no control over the feedback cursor, the effect of this task outcome information would presumably operate in an implicit, automatic manner, similar to how we assume the clamped feedback provides an invariant SPE signal.

Our experiments show that hitting the target has a strong effect on performance, attenuating the rate and magnitude of learning. Through computational modeling, we explore hypotheses that might account for this effect, considering three models in which implicit learning is driven by both SPE and task outcome information. In the first two models, hitting the target serves as an intrinsic reward signal that either reinforces associated movements or directly modulates adaptation. In the third model, hitting or missing the target serves as a task-outcome feedback signal that drives a second implicit learning process, one that operates in parallel with implicit adaptation.

In all experiments, we used clamped visual feedback, in which the angular trajectory of a feedback cursor is invariant with respect to the target location and thus spatially independent of hand position (Morehead et al., 2017). The instructions (see Supplementary file 1-Target Size Experiment Instructions) emphasized that the participant’s behavior would not influence the cursor trajectory: They were to ignore this stimulus and always aim directly for the target. This method allows us to isolate implicit learning from an invariant error, eliminating potential contributions from explicit aiming that might be used to reduce task performance error.

Experiment 1

In Experiment 1, we asked if the task outcome, defined in terms of whether or not the cursor hit the target, would modulate learning under conditions in which the feedback is not contingent on behavior. We tested three groups of participants (n = 16/group) with a 3.5° clamp for 80 cycles (eight targets per cycle). The purpose of this experiment was to examine the effects of three different relationships between the clamp and target while holding the visual error (defined as the center-to-center distance between the cursor and target) constant (Figure 1b): Hit Target (when the terminal position of the clamped cursor is fully embedded within a 16 mm diameter target), Straddle Target (when roughly half of the cursor falls within a 9.8 mm target, with the remaining part outside the target), Miss Target (when the cursor is fully outside a 6 mm target).

Figure 1

Download asset Open asset

Figure 1—source data 1

This file contains hand angle data for each trial and participant in Experiment 1, and was used to generate Figure 1d–g.

Reaction times (RTs) and movement times (MTs) are also included. Note that hand angles were flipped for participants who experienced a counter-clockwise clamp.

https://doi.org/10.7554/eLife.39882.003

Download elife-39882-fig1-data1-v1.txt

Hitting the target reduced the overall change in behavior (Figure 1d). Statistically, there was a marginal difference on the rate of initial adaptation (one-way ANOVA: F(2,45)=2.67, p=0.08, η² = 0.11; permutation test: p=0.08; Figure 1e) and a significant effect on late learning (F(2,45)=4.44, p=0.016, η² = 0.17; Figure 1f). For the latter measure, the value for the Hit Target group was approximately 35% lower than for the Straddle and Miss Target groups, with post-hoc comparisons confirming the substantial differences in late learning between the Hit Target and both the Straddle Target (95% CI [−16.13°, −2.34°], t(30)=-2.73, p=0.010, d = 0.97) and Miss Target (95% CI [−16.76°, −2.79°], t(30)=-2.86, p=0.008, d = 1.01) groups. These differences were also evident in the aftereffect measure, taken from the first cycle of the no feedback block (see Materials and methods). The learning functions for the Straddle and Miss Target groups were remarkably similar throughout the entire clamp block and reached similar magnitudes of late learning (95% CI [−7.90°, 8.97°], t(30)=.13, p=0.898, d = 0.05).

As seen in Figure 1d, the change in hand angle from the final cycle of the clamp block to the final cycle of the no feedback block was less for the Hit than the Straddle and Miss groups (one-way ANOVA: F(2,45)=4.42, p=0.018, η² = 0.16; Hit vs Miss: 95% CI [1.47°, 8.00°], t(30)=2.96, p=0.006, d = 1.05; Hit vs Straddle: 95% CI [1.06°, 8.74°], t(30)=2.61, p=0.014, d = 0.92). This result indicates that retention was strongest in the Hit group. However, retention is generally analyzed as a relative, rather than absolute measure, especially when the amount of learning differs between groups. We thus re-analyzed the change in hand angle across the no feedback block, but now as the ratio of the last no-feedback cycle relative to the last clamp cycle. In this analysis, there was no difference between the three groups (Figure 1g; F(2,45)=2.06, p=0.139, η² = 0.08; permutation test: p=0.138).

Interestingly, the results from this experiment are qualitatively different to those observed when manipulating the angular deviation of the clamp. Our previous study using clamped visual feedback demonstrated that adaptation in response to errors of varying size, which was assessed by manipulating the clamp angle, results in different early learning rates, but produces the same magnitude of late learning (Kim et al., 2018). In contrast, the results in Experiment 1 show that hitting the target attenuates learning, with the effect becoming pronounced after prolonged exposure to the perturbation. Furthermore, the effect of task outcome appears to be categorical, as it was only observed for the condition in which the cursor was fully embedded within the target (Hit Target), and not when the terminal position of the cursor fell partially outside the target (Straddle Target).

Experiment 2

Experiment 2 was designed to extend the results of Experiment 1 in two ways: First, to verify that the effect of hitting a target generalized to other contexts, we changed the size of the clamp angle. We tested two groups of participants (n = 16/group) with a small 1.75° clamp. For the Hit Target group (Figure 2a), we used the large 16 mm target, and thus, the cursor was fully embedded. For the Straddle Target group, we used the small 6 mm diameter target, resulting in an endpoint configuration in which the cursor was approximately half within the target and half outside the target. We did not test a Miss Target condition because having the clamped cursor land fully outside the target would have necessitated an impractically small target (~1.4 mm). Moreover, the results of Experiment 1 indicate that this condition is functionally equivalent to the Straddle Target group. The second methodological change was made to better assess asymptotic learning. We increased the number of clamped reaches to each location to 220 (reducing the number of target locations to four to keep the experiment within a 1.5 hr session). This resulted in a nearly three-fold increase in the number of clamped reaches per location.

Figure 2

Download asset Open asset

Figure 2—source data 1

This file contains hand angle data for each trial and participant in Experiment 2, and was used to generate Figure 2b–e.

Reaction times (RTs) and movement times (MTs) are also included. Note that hand angles were flipped for participants who experienced a counter-clockwise clamp.

https://doi.org/10.7554/eLife.39882.005

Download elife-39882-fig2-data1-v1.txt

Consistent with the results of Experiment 1, the Hit Target group showed an attenuated learning function compared to the Straddle Target group (Figure 2b). Statistically, there was again only a marginal difference in the rate of early adaptation (95% CI [−0.52°/cycle, .01°/cycle], t(30)=-1.96, p=0.06, d = 0.69; Figure 2c), whereas the difference in late learning was more pronounced (95% CI [−11.38°, −1.25°], t(30)=-2.54, p=0.016, d = 0.90; permutation test: p=0.007; Figure 2d). Indeed, the 35% attenuation in asymptote for the Hit Target group compared to the Straddle Target group is approximately equal to that observed in Experiment 1.

We used a different approach to examine retention in Experiment 2, having participants complete 10 cycles with a 0° clamp following the extended 1.75° clamp block (Shmuelof et al., 2012). We opted to use this alternative method since the presence of the 0° clamp would create less contextual change when switching from the clamp to the retention block, compared to the no feedback block of Experiment 1. In terms of absolute change across the 0° clamp block, there was a trend for greater retention in the Hit group compared to the Straddle group (95% CI [−0.27°, 3.53°], t(30)=1.75, p=0.090, d = 0.62). However, when analyzed as a proportional change, the difference was not reliable (95% CI [−0.06,. 27], t(30)=1.27, p=0.21, d = 0.45).

The results of these first two experiments converge in showing that learning from an invariant error is attenuated when the cursor hits the target, relative to conditions in which at least part of the cursor falls outside the target. This effect replicated across two experiments that used different clamp sizes.

Attenuated behavioral changes are not due to differences in motor planning

Although we hypothesized that manipulating target size in Experiments 1 and 2 would influence learning mechanisms that respond to the differential task outcomes (i.e., hit or miss), it is also important to consider alternative explanations for the effect of target size on learning. Figure 3 provides a schematic of the core components of sensorimotor adaptation. The figure highlights that changes in adaptation might arise because target size alters the inputs on which learning operates, rather than from a change in the operation of the learning process itself. For example, increasing the target size may increase perceptual uncertainty, creating a weaker error signal. We test this hypothesis in a control condition in Experiment 3.

Figure 3

Download asset Open asset

Another hypothesis centers on how variation in target size might alter motor planning. Assuming target size influences response preparation, participants in the Hit Target groups had reduced accuracy demands relative to the other groups, given that they were reaching to a larger target (Soechting, 1984). If the accuracy demands were reduced for these large targets, then the motor command could be more variable, resulting in more variable sensory predictions from a forward model, and thus a weaker SPE (Körding and Wolpert, 2004). While we do not have direct measures of planning noise, a reasonable proxy can be obtained by examining movement variability during the unperturbed baseline trials (data from clamped trials would be problematic given the induced change in behavior). If there is substantially more noise in the plan for the larger target, then the variability of hand angles should be higher in this group (Churchland et al., 2006). In addition, one may expect faster movement times (or peak velocities) and/or reaction times for reaches to the larger target, assuming a speed-accuracy tradeoff (Fitts, 1992).

Examination of kinematic and temporal variables (see Appendix 1) did not support the noisy motor plan hypothesis. In Experiment 1, average movement variability across the eight targets during cycles 2–10 of the veridical feedback baseline block were not reliably different between groups (variability: F(2,45)=2.32, p=0.110, η² = 0.093). Movement times across groups were not different (F(2,45)=2.19, p=0.123, η² = 0.089). However, we did observe a difference in baseline RTs (F(2,45)=4.48, p=0.017, η² = 0.166), with post hoc t-tests confirming that the large target (Hit) group had faster RTs than the small target (Miss) group (95% CI [−108 ms, −16 ms], t(30)=-2.74, p=0.010, d = 0.97) and medium target (Straddle) group (95% CI [−66 ms, −10 ms], t(30)=-2.76, p=0.010, d = 0.97). The medium target (Straddle) and small target groups’ RTs were not reliably different (95% CI [−74 ms, 26 ms], t(30)=-.984, p=0.333, d = 0.348). This baseline difference in RTs was only observed in this experiment (see Appendix 1), and there was no correlation between baseline RT and late learning for the large target group (r = 0.09, p=0.73), suggesting that RTs are not associated with the magnitude of learning.

During baseline trials with veridical feedback in Experiment 2, mean spatial variability, measured in terms of hand angle, was actually lower for the group reaching to the larger target (Hit Target group: 3.09° ±. 18°; Straddle Target group: 3.56° ±. 16°; t(30)=-1.99 p=0.056, d = 0.70). Further supporting the argument that planning was no different across conditions, neither reaction times (Hit Target: 378 ± 22 ms; Straddle Target: 373 ± 12 ms) nor movement times (Hit Target: 149 ± 8 ms; Straddle Target: 157 ± 8 ms) differed between the groups (t(30)=-0.183, p=0.856, d = 0.06 and t(30)=0.71, p=0.484, d = 0.25, respectively).

One reason for not observing consistent effects of target size on accuracy or temporal measures could be due to the constraints of the task. Studies showing an effect of target size on motor planning typically utilize point-to-point movements (Soechting, 1984; Knill et al., 2011) in which accuracy requires planning of both movement direction and extent. In our experiments, we utilized shooting movements, thus minimizing demands on the control of movement extent. Endpoint variability is generally larger for movement extent compared to movement direction (Gordon et al., 1994). It is also possible that participants are near ceiling-level performance in terms of hand angle variability.

Theoretical analysis of the effect of task outcome on implicit learning

Having ruled out a motor planning account of the differences in performance in Experiments 1 and 2, we next considered different ways in which target error could affect the rate and asymptotic level of learning. Adaptation from SPE can be thought of as recalibrating an internal model that learns to predict the sensory outcome of a motor command (Figure 3). Here, we model adaptation with a single rate state-space equation of the of the following form:

(1) ${\displaystyle \mathrm{x}(\mathrm{n}+1)={\mathrm{A}}^{\ast }\mathrm{x}(\mathrm{n})+\mathrm{U}(\mathrm{e})}$

where x represents the motor output on trial n, A is a retention factor, and U represents the update/correction size (or, learning rate) as a function of the error (clamp) size, e. This model is mathematically equivalent to a standard single rate state-space model (Thoroughman and Shadmehr, 2000), with the only modification being the replacement of the error sensitivity term, B, with a correction size function, U (Kim et al., 2018). Unlike standard adaptation studies where error size changes over the course of learning, e is a constant with clamped visual feedback and thus, U(e) can be estimated as a single parameter. We refer to this model as the motor correction variant of the standard state space model. The first two experiments make clear that a successful model must account for the differences between hitting and missing the target, even while holding the error term in Equation. (1) (clamp angle) constant.

We consider three variants to the basic model that might account for how task outcome influences learning. The first model is motivated by previous studies that have considered how reinforcement processes might operate in sensorimotor adaptation tasks, and in particular, the idea that task outcome information impacts a model-free operant reinforcement process (Huang et al., 2011; Shmuelof et al., 2012). We can extend this idea to the clamp paradigm, considering how the manipulation of target size affects reward signals: When the clamp hits the target, the feedback generates a positive reinforcement signal; when the clamp misses (or straddles) the target, this reinforcement signal is absent. We refer to the positive outcome as an intrinsic reward given that it is not contingent on the participant’s behavior. This signal could strengthen the representation of its associated movement (Gonzalez Castro et al., 2011; Shmuelof et al., 2012), and thus increase the likelihood that future movements will be biased in a similar direction.

We combine this idea with the state space model to create a Movement Reinforcement model (Figure 4a). Here, a model-free reinforcement-based process is combined with a model-based adaptation process. Intuitively, this model accounts for the attenuated learning functions for the Hit conditions in Experiments 1 and 2 because the effect of movement reinforcement resists the directional change in hand angle induced by SPEs. In this model, intrinsic reward has no direct effect on SPE-driven adaptation. That is, reward and error-based learning are assumed to operate independently of each other, with the final movement being the sum of these two processes.

Figure 4

Download asset Open asset

Here, we formalize a Movement Reinforcement model, taking this as illustrative of a broad class of operant reinforcement models in which the reinforcement process acts in parallel to a traditional state space model of sensorimotor adaptation. The motor output, y, is a weighted sum of a model-free reinforcement process and an adaptation process, x:

(2) ${\displaystyle \mathrm{y}(\mathrm{n})=(1-{\mathrm{V}}_1(\mathrm{n}){)}^{\ast }\mathrm{x}(\mathrm{n})+{\mathrm{V}}_1(\mathrm{n}{)}^{\ast }{\mathrm{V}}_{\mathrm{d}}(\mathrm{n})}$

where a population vector (Georgopoulos et al., 1986), V, indicates the current bias of motor representations within the reinforcement system (see Materials and methods). The direction of this vector (V_d) corresponds to the mean preferred direction resulting from the reinforcement history, with the length (V_l) corresponding to the strength of this biasing signal. This vector can be viewed as a weight on the movement reinforcement process (0 = no wt, 1 = full wt), relative to the adaptation process.

In sum, the Movement Reinforcement model entails four parameters, composed of separate update and retention parameters for the reinforcement learning process and the adaptation process (see Materials and methods). The former is model-free, dependent on an operant conditioning process by which a task outcome signal modifies movement biases, whereas the latter is model-based, using SPE to recalibrate an internal model of the sensorimotor map. Importantly, the predictions of this model are not dependent on whether we model the effect of reinforcement as operating on discrete units, as we have done here, or as basis functions (Donchin et al., 2003; Tanaka et al., 2012; Taylor et al., 2013).

The second model entails a single process whereby the task outcome directly modulates the adaptation process. For example, an intrinsic reward signal associated with hitting the target could modulate adaptation, attenuating the trial-to-trial change induced by the SPE (Figure 4b). In this Adaptation Modulation model, the reward signal can be interpreted as a gain controller, similar to previous efforts to model the effect of explicit rewards and punishments on adaptation (Galea et al., 2015). In Experiments 1 and 2, hitting the target presumably reduces the gain on adaptation, thus leading to attenuated learning.

We formalize the Adaptation Modulation model as follows:

(3) ${\displaystyle \mathrm{x}(\mathrm{n}+1)={{\gamma }_{\mathrm{A}}}^{\ast }{\mathrm{A}}^{\ast }\mathrm{x}(\mathrm{n})+{\gamma }_{\mathrm{u}}{}^{\ast }\mathrm{U}(\mathrm{e})}$

where γ_A and γ_u are gains on the retention and update parameters, respectively. In the current implementation, we set γ_A and γ_u to one on miss trials and estimate the values of γ_A and γ_u for the hit trials. Although this could be reversed (e.g. set gains to one on hit trials and estimate values on miss trials), our convention seems more consistent with previous modeling studies of adaption where the movements generally miss the target. We impose no additional constraint on the gain parameters; the effect of retention or updating can be larger or smaller on hit trials compared to miss trials. As with the Movement Reinforcement model, the Adaptation Modulation model has four free parameters.

The third model we consider here, the Dual Error model, postulates that learning is the composite of two implicit learning processes that operate on different error signals. The first is an adaptation process driven by SPE (as in Equation (1)). The second process operates in the same manner as adaptation, but here the error signal is sensitive to the task outcome. This idea of a TE-sensitive process stems from previous studies in which an error is produced, not by perturbing the visual feedback of hand position, but rather by displacing the visual feedback of the target position (Magescas and Prablanc, 2006; Cameron et al., 2010a; Cameron et al., 2010a; Schmitz et al., 2010). The resulting mismatch between the hand position and displaced target position can be viewed as a TE rather than SPE, under the assumption that the veridical feedback of hand position roughly matches the predicted hand position (see Discussion). When this error signal is consistent (e.g. target is displaced in the same direction on every trial), a gradual change in heading angle is observed, similar to that seen in studies of visuomotor adaptation. Moreover, this form of learning is implicit: By shifting the target position during a saccade, just prior to the reach, the participants are unaware of the target displacement.

In the Dual Error model, the motor output is the sum of two processes:

(4) ${\displaystyle {\mathrm{x}}_{\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}}(\mathrm{n})={\mathrm{x}}_{\mathrm{s}\mathrm{p}\mathrm{e}}(\mathrm{n})+{\mathrm{x}}_{\mathrm{t}\mathrm{e}}(\mathrm{n})}$

where

(5) ${\displaystyle {\mathrm{x}}_{\mathrm{s}\mathrm{p}\mathrm{e}}(\mathrm{n}+1)={\mathrm{A}}_{\mathrm{s}\mathrm{p}\mathrm{e}}{}^{\ast }{\mathrm{x}}_{\mathrm{s}\mathrm{p}\mathrm{e}}(\mathrm{n})+{\mathrm{U}}_{\mathrm{s}\mathrm{p}\mathrm{e}}{}^{\ast }(\mathrm{S}\mathrm{P}\mathrm{E})}$

(6) ${\displaystyle {\mathrm{x}}_{\mathrm{t}\mathrm{e}}(\mathrm{n}+1)={\mathrm{A}}_{\mathrm{t}\mathrm{e}}{}^{\ast }{\mathrm{x}}_{\mathrm{t}\mathrm{e}}(\mathrm{n})+{\mathrm{U}}_{\mathrm{t}\mathrm{e}}{}^{\ast }(\mathrm{T}\mathrm{E})}$

Equation (5) is the same as in the other two models, describing adaptation from a sensory prediction error, but with the notation modified here to explicitly contrast with the second process. Equation (6) describes a second implicit learning process, but one that is driven by the target error.

The SPE-sensitive process updates from the error term on every trial given that the SPE is always present, even on hit trials. In contrast, the TE-sensitive process only updates from the error term on miss trials. The error component of Equation (6) is absent on hit trials. This would account for the attenuated learning observed in the large target (Hit) conditions in Experiments 1 and 2. In the context of our clamp experiments, TE is modeled as a step function (Figure 4c), set to 0 when the cursor hits the target and one when the cursor misses or straddles the target. However, if the cursor position varied (as in studies with contingent feedback), TE might take on continuous, signed values, similar to SPE.

We note that the Dual Error model is similar to the influential two-process state space model of adaption introduced by Smith and colleagues (Smith et al., 2006). In their model, dual-adaptation processes have different learning rates and retention factors, resulting in changes that occur over different time scales. Here, the different learning rates and retention factors are related to the different error signals, TE and SPE. Whereas the dual-rate model imposes a constraint on the parameters (i.e. process with faster learning must also have faster forgetting), the four parameters in the Dual Error model are unconstrained relative to each other.

Experiment 3

The experimental design employed in Experiments 1 and 2 cannot distinguish between these three models because all make qualitatively similar predictions. In the Movement Reinforcement model, the attenuated asymptote in response to Hit conditions arises because movements are rewarded throughout, including during early learning, biasing future movements toward baseline. The Adaptation Modulation model predicts a lower asymptote during the Hit condition because the adaptation system is directly attenuated by reward. The Dual Error model similarly predicts a lower asymptote because only one of two learning processes is active when there is no target error.

In contrast to the single perturbation blocks used in Experiments 1 and 2, a transfer design in which the target size changes after an initial adaptation phase affords an opportunity to contrast the three models. In Experiment 3, we tested two groups of participants (n = 12/group) with a 1.75° clamp, varying the target size between the first and second halves of the experiment (Figure 5a). The key manipulation centered on the order of when the target was large (hit condition) or small (straddle condition).

Figure 5

Download asset Open asset

Figure 5—source data 1

This file contains hand angle data for each trial and participant in Experiment 3, and was used to generate Figure 5c–e and Figure 7.

Reaction times (RTs) and movement times (MTs) are also included. Note that hand angles were flipped for participants who experienced a counter-clockwise clamp.

https://doi.org/10.7554/eLife.39882.009

Download elife-39882-fig5-data1-v1.txt

For the Straddle-to-Hit group, a small target was used in an initial acquisition phase (first 120 clamp cycles). Based on the results of Experiments 1 and 2, we expect to observe a relatively large change in hand angle at the end of this phase since the outcome is always an effective ‘miss’. The key test comes during the transfer phase (final 80 clamp cycles), in which the target size is increased such that the invariant clamp now results in a target hit. For the Movement Reinforcement model, hitting the target will produce an intrinsic reward signal, reinforcing the associated movement. Therefore, there should be no change in performance (hand angle) following transfer since the SPE remains the same and the current movements are now reinforced (Figure 5b). In contrast, both the Adaptation Modulation and Dual Error models predict that, following transfer to the large target, there will be a drop in hand angle, relative to the initial asymptote. For the former, hitting the target will attenuate the adaptation system; for the latter, hitting the target will shut down learning from the process that is sensitive to target error.

We also tested a second group in which the large target (hit) was used in the acquisition phase and the small target (effective ‘miss’) in the transfer phase (Hit-to-Straddle group). All three models make the same qualitative predictions for this group. At the end of the acquisition phase, there should be a smaller change in hand angle compared to the Straddle-to-Hit group, due to the persistent target hits. Following transfer, all three models predict an increase in hand angle, relative to the initial asymptote. For the Movement Reinforcement model, the reduction in target size removes the intrinsic reward signal, which over time, lessens the contribution of the reinforcement process as the learned movement biases decay in strength. The Adaptation Modulation model predicts that hand angle will increase due to the removal of the attenuating effect on adaptation following transfer. The Dual Error model also predicts an increase in hand angle, but here the effect occurs because the introduction of a target error activates the second implicit learning process. Although the Hit-to-Straddle group does not provide a discriminative test between the three models, the inclusion of this group does provide a second test of each model, as well as an opportunity to rule out alternative hypotheses for the behavioral effects at transfer. For example, the absence of a change at transfer might be due to reduced sensitivity to the clamp following a long initial acquisition phase.

Experiment 3 – modeling results

We evaluated the three models by simultaneously fitting group-averaged data for both groups. As depicted in Figure 6, all three models capture the initial plateau followed by increased learning of the Hit-to-Straddle group. However, the quality of the fits diverges for the Straddle-to-Hit group, where the Movement Reinforcement model cannot produce a decrease in hand angle once the large target is introduced. Instead, the best-fit parameters for this model result in an asymptote that falls between the hand angle values observed during the latter part of each phase. In contrast, the Adaptation Modulation and Dual Error models both predict the drop in hand angle during the second phase of the experiment for the Straddle-to-Hit group.

Figure 6 with 1 supplement see all

Download asset Open asset

Consistent with the preceding qualitative observations, the Movement Reinforcement model yielded a lower R² value and higher Akaike Information Criterion (AIC) score (higher AIC indicates relatively worse fit) than the Adaptation Modulation and Dual Error models (Table 1). A comparison of the latter two shows that the Dual Error model provides the best account of the results. This model yielded a lower AIC score and accounted for 90% of the variance in the group-averaged data compared to 86% for the Adaptation Modulation model.

Table 1

Basic models	# of free parameters	R-squared	AIC
Movement Reinforcement	4	0.824	363
Adaptation Modulation	4	0.861	269
Dual Error	4	0.895	156
Hybrid Models
Movement Reinforcement + Adaptation Modulation	6	0.945	−100
Movement Reinforcement + Dual Error	6	0.945	−97

To better understand the effects of target size on learning and retention, we examined the parameter estimates for the Adaptation Modulation and Dual Error models. We first generated 1000 bootstrapped samples of group-averaged behavior by resampling with replacement from each group. We then fit each of the bootstrapped samples simultaneously and report the results here in terms of 95% confidence intervals. For the Adaptation Modulation model, the estimates of γ_u*U were larger during miss than hit conditions, with no overlap of the confidence intervals ([.693, 1.302] vs [.182,. 573], respectively); thus, the error-driven adjustment in the state of the internal model was much larger after a miss than a hit. For the Dual Error model, the estimates of U_spe were larger than for U_te, again with no overlap of the confidence intervals ([.414, 1.08], vs [.157,. 398]), indicating that the state change was more strongly driven by SPE than TE. For each model, the process that produced a larger error-based update also had the lower retention factor, although here there was overlap in the 95% confidence intervals for the latter (γ_r*A for Miss: [.939,.969] vs Hit: [. 961,.989]; A_spe: [.900,.972] vs A_te: [.938,.993]). In sum, our model fits suggest the impact of task outcome (hit or miss) was primarily manifest in the estimates of the learning rate parameters. However, this interpretation is tempered by the correlations observed between certain parameters (Cheng and Sabes, 2006) (see Figure 6—figure supplement 1).

The behavioral pattern observed in Experiment 3, complemented by the modeling results, are problematic for the Movement Reinforcement model, challenging the idea that the effect of task outcome arises solely from a model-free learning process that operates independent of model-based adaptation. However, this does not exclude the possibility that task outcome information influences both model-free and model-based processes. For example, hitting the target might not only reinforce an executed movement, but might also modulate adaptation. Formally, this hypothesis would correspond to a hybrid model that combines the Adaptation Modulation and Movement Reinforcement models. Indeed, hybrids that combine the Movement Reinforcement model with either the Adaptation Modulation or Dual Error models (see Materials and methods) yield improved model fits and lower AIC values, with the two hybrids producing comparable values (see Table 1).

Control group for testing perceptual uncertainty hypothesis

Across the three experiments, the amount of learning induced by clamped visual feedback was attenuated when participants reached to the large target. We considered if this effect could be due, in part, to the differences between the Hit and Straddle/Miss conditions in terms of perceptual uncertainty. For example, the reliability of the visual error signal might be weaker if the cursor is fully embedded within the target; in the extreme, failure to detect the angular offset might lead to the absence of an error signal on some percentage of the trials.

To evaluate this perceptual uncertainty hypothesis, we tested an additional group in Experiment 3 with a large target, but modified the display such that a bright line, aligned with the target direction, bisected the target (Figure 7). With this display, the feedback cursor remained fully embedded in the target, but was clearly off-center. If the attenuation associated with the large target is due to perceptual uncertainty, then the inclusion of the bisecting line should produce an adaptation effect similar to that observed with small targets. Alternatively, if perceptual uncertainty does not play a prominent role in the target size effect, then the adaptation effects would be similar to that observed with large targets.

Figure 7

Download asset Open asset

Consistent with the second hypothesis, performance during the acquisition phase for the group reaching to a bisected target was similar to that of the group reaching to the standard large target (Hit-to-Straddle, Figure 7). To provide support for this observation, we first performed an omnibus one-way ANOVA on the late learning data at the end of the acquisition phase, given our analysis plan entailed multiple planned pair-wise comparisons. There was a significant effect of group (F(2,33)=9.33, p=0.0006, η² = 0.36). Subsequent planned pair-wise comparisons showed no significant differences between the bisected target and standard large target (Hit-to-Straddle) groups (early adapt: 95% CI [−0.34°/cycle,. 22°/cycle], t(22)=-.47; p=0.64,; d = 0.19; late learning: 95% CI [−7.80° 1.19°], t(22)=-1.52; p=0.14; d = 0.62). In contrast, the group reaching to bisected targets showed slower early adaptation rates (95% CI [−0.81°/cycle, −0.07°/cycle], t(22)=-2.49, p=0.02, d = 1.02) and lower magnitudes of late learning (95% CI [−12.58°, −1.35°], t = −2.57, p=0.017, d = 1.05) when compared with the group reaching to small targets (Straddle-to-Hit).

During the transfer phase, the target size for the perceptual uncertainty group remained large, but the bisection line was removed. If perceptual uncertainty underlies the effect we have attributed to hitting the target, we would expect to observe a decrease in hand angle following transfer, since uncertainty would increase. However, following transfer to the non-bisected large target, there was no change in asymptote (95% CI [−0.87°, 2.32°], t(11)=1.0, p=0.341, d_z = 0.29). In sum, the results from this control group indicate that the attenuated adaptation observed when the cursor is fully embedded within the target is not due to perceptual uncertainty.

Models of sensorimotor adaptation have emphasized that this form of learning is driven by sensory prediction errors, the difference between the observed and predicted sensory consequences of a movement. In this formulation, task outcome, defined as hitting or missing the target, is not part of the equation (although in most adaptation tasks, the sensory prediction is at the target, thus conflating SPE and task outcome). While a number of recent studies have demonstrated that task outcome signals can influence overall performance in these tasks (Galea et al., 2015; Reichenthal et al., 2016; Leow et al., 2018; van der Kooij et al., 2018), it is unclear whether these reinforcement signals impact sensorimotor adaptation (Shmuelof et al., 2012; Galea et al., 2015), or whether they are exploited by other learning systems, distinct from SPE-driven implicit adaptation (Codol et al., 2018; Holland et al., 2018).

The interpretation of the results from these studies is complicated by the fact that the experimental tasks may conflate different learning processes. In the present study, we sought to avoid this complication by employing a new method to study implicit learning, one in which participants are specifically instructed to ignore an invariant visual error signal, thus eliminating explicit processes (Morehead et al., 2017). Using this clamp method, we observed a striking difference between conditions in which the final position of the cursor was fully embedded in the target compared to conditions in which the cursor either terminated outside or straddled the target: When the cursor was fully embedded, the rate of learning was reduced and the asymptotic level of learning was markedly attenuated.

All data generated or analysed during this study are included in the manuscript and supporting files. Source data files have been provided for Figures 1, 2, and 5.

1. 1. Bond KM
   2. Taylor JA

   (2015) Flexible explicit but rigid implicit learning in a visuomotor adaptation task

   Journal of Neurophysiology 113:3836–3849.

   https://doi.org/10.1152/jn.00009.2015

   • PubMed
   • Google Scholar
2. 1. Butcher PA
   2. Ivry RB
   3. Kuo SH
   4. Rydz D
   5. Krakauer JW
   6. Taylor JA

   (2017) The cerebellum does more than sensory prediction error-based learning in sensorimotor adaptation tasks

   Journal of Neurophysiology 118:1622–1636.

   https://doi.org/10.1152/jn.00451.2017

   • PubMed
   • Google Scholar
3. 1. Cameron BD
   2. Franks IM
   3. Inglis JT
   4. Chua R

   (2010a) Implicit motor learning from target error during explicit reach control

   Experimental Brain Research 206:99–104.

   https://doi.org/10.1007/s00221-010-2402-4

   • PubMed
   • Google Scholar
4. 1. Cameron BD
   2. Franks IM
   3. Inglis JT
   4. Chua R

   (2010b) Reach adaptation to explicit vs. implicit target error

   Experimental Brain Research 203:367–380.

   https://doi.org/10.1007/s00221-010-2239-x

   • PubMed
   • Google Scholar
5. 1. Cashaback JGA
   2. McGregor HR
   3. Mohatarem A
   4. Gribble PL

   (2017) Dissociating error-based and reinforcement-based loss functions during sensorimotor learning

   PLOS Computational Biology 13:e1005623.

   https://doi.org/10.1371/journal.pcbi.1005623

   • PubMed
   • Google Scholar
6. 1. Cheng S
   2. Sabes PN

   (2006) Modeling sensorimotor learning with linear dynamical systems

   Neural Computation 18:760–793.

   https://doi.org/10.1162/neco.2006.18.4.760

   • PubMed
   • Google Scholar
7. 1. Churchland MM
   2. Afshar A
   3. Shenoy KV

   (2006) A central source of movement variability

   Neuron 52:1085–1096.

   https://doi.org/10.1016/j.neuron.2006.10.034

   • PubMed
   • Google Scholar
8. Preprint

   1. Codol O
   2. Holland PJ
   3. Galea JM

   (2017) The relationship between reinforcement and explicit strategies during visuomotor adaptation

   bioRxiv.

   https://doi.org/10.1101/206284

   • Google Scholar
9. 1. Codol O
   2. Holland PJ
   3. Galea JM

   (2018) The relationship between reinforcement and explicit control during visuomotor adaptation

   Scientific Reports 8:9121.

   https://doi.org/10.1038/s41598-018-27378-1

   • PubMed
   • Google Scholar
10. 1. Daw ND
    2. Gershman SJ
    3. Seymour B
    4. Dayan P
    5. Dolan RJ

    (2011) Model-based influences on humans' choices and striatal prediction errors

    Neuron 69:1204–1215.

    https://doi.org/10.1016/j.neuron.2011.02.027

    • PubMed
    • Google Scholar
11. 1. Day KA
    2. Roemmich RT
    3. Taylor JA
    4. Bastian AJ

    (2016) Visuomotor learning generalizes around the intended movement

    eNeuro 3:ENEURO.0005-16.2016.

    https://doi.org/10.1523/ENEURO.0005-16.2016

    • PubMed
    • Google Scholar
12. 1. Donchin O
    2. Francis JT
    3. Shadmehr R

    (2003) Quantifying generalization from trial-by-trial behavior of adaptive systems that learn with basis functions: theory and experiments in human motor control

    The Journal of Neuroscience 23:9032–9045.

    https://doi.org/10.1523/JNEUROSCI.23-27-09032.2003

    • PubMed
    • Google Scholar
13. 1. Fitts PM

    (1992) The information capacity of the human motor system in controlling the amplitude of movement

    Journal of Experimental Psychology: General 121:262–269.

    https://doi.org/10.1037/0096-3445.121.3.262

    • Google Scholar
14. 1. Galea JM
    2. Mallia E
    3. Rothwell J
    4. Diedrichsen J

    (2015) The dissociable effects of punishment and reward on motor learning

    Nature Neuroscience 18:597–602.

    https://doi.org/10.1038/nn.3956

    • PubMed
    • Google Scholar
15. 1. Gallivan JP
    2. Logan L
    3. Wolpert DM
    4. Flanagan JR

    (2016) Parallel specification of competing sensorimotor control policies for alternative action options

    Nature Neuroscience 19:320–326.

    https://doi.org/10.1038/nn.4214

    • PubMed
    • Google Scholar
16. 1. Georgopoulos AP
    2. Schwartz AB
    3. Kettner RE

    (1986) Neuronal population coding of movement direction

    Science 233:1416–1419.

    https://doi.org/10.1126/science.3749885

    • PubMed
    • Google Scholar
17. 1. Gonzalez Castro LN
    2. Monsen CB
    3. Smith MA

    (2011) The binding of learning to action in motor adaptation

    PLOS Computational Biology 7:e1002052.

    https://doi.org/10.1371/journal.pcbi.1002052

    • PubMed
    • Google Scholar
18. 1. Gordon J
    2. Ghilardi MF
    3. Ghez C

    (1994) Accuracy of planar reaching movements. I. independence of direction and extent variability

    Experimental Brain Research 99:97–111.

    https://doi.org/10.1007/BF00241415

    • PubMed
    • Google Scholar
19. Conference

    1. Hadjiosif A
    2. Smith M

    (2013)

    Savings is restricted to the temporally labile component of motor adaptation

    Translational and Computational Motor Control (TCMC).

    • Google Scholar
20. 1. Haith AM
    2. Krakauer JW

    (2018) The multiple effects of practice: skill, habit and reduced cognitive load

    Current Opinion in Behavioral Sciences 20:196–201.

    https://doi.org/10.1016/j.cobeha.2018.01.015

    • PubMed
    • Google Scholar
21. 1. Holland P
    2. Codol O
    3. Galea JM

    (2018) Contribution of explicit processes to reinforcement-based motor learning

    Journal of Neurophysiology 119:2241–2255.

    https://doi.org/10.1152/jn.00901.2017

    • PubMed
    • Google Scholar
22. 1. Huang VS
    2. Haith A
    3. Mazzoni P
    4. Krakauer JW

    (2011) Rethinking motor learning and savings in adaptation paradigms: model-free memory for successful actions combines with internal models

    Neuron 70:787–801.

    https://doi.org/10.1016/j.neuron.2011.04.012

    • PubMed
    • Google Scholar
23. 1. Huberdeau DM
    2. Krakauer JW
    3. Haith AM

    (2015) Dual-process decomposition in human sensorimotor adaptation

    Current Opinion in Neurobiology 33:71–77.

    https://doi.org/10.1016/j.conb.2015.03.003

    • PubMed
    • Google Scholar
24. Preprint

    1. Huberdeau DM
    2. Krakauer JW
    3. Haith AM

    (2017) Practice induces a qualitative change in the memory representation for visuomotor learning

    bioRxiv.

    https://doi.org/10.1101/226415

    • Google Scholar
25. Preprint

    1. Hutter SA
    2. Taylor JA

    (2018) Relative sensitivity of explicit re-aiming and implicit motor adaptation

    bioRxiv.

    https://doi.org/10.1101/308510

    • Google Scholar
26. 1. Izawa J
    2. Criscimagna-Hemminger SE
    3. Shadmehr R

    (2012) Cerebellar contributions to reach adaptation and learning sensory consequences of action

    Journal of Neuroscience 32:4230–4239.

    https://doi.org/10.1523/JNEUROSCI.6353-11.2012

    • PubMed
    • Google Scholar
27. 1. Izawa J
    2. Shadmehr R

    (2011) Learning from sensory and reward prediction errors during motor adaptation

    PLOS Computational Biology 7:e1002012.

    https://doi.org/10.1371/journal.pcbi.1002012

    • PubMed
    • Google Scholar
28. 1. Kim HE
    2. Morehead JR
    3. Parvin DE
    4. Moazzezi R
    5. Ivry RB

    (2018) Invariant errors reveal limitations in motor correction rather than constraints on error sensitivity

    Communications Biology 1:19.

    https://doi.org/10.1038/s42003-018-0021-y

    • PubMed
    • Google Scholar
29. Software

    1. Kim HE

    (2019) Influence-of-task-outcome-on-implicit-motor-learning, version a94fed0

    Github.

    https://github.com/hyosubkim/Influence-of-task-outcome-on-implicit-motor-learning
30. 1. Knill DC
    2. Bondada A
    3. Chhabra M

    (2011) Flexible, task-dependent use of sensory feedback to control hand movements

    Journal of Neuroscience 31:1219–1237.

    https://doi.org/10.1523/JNEUROSCI.3522-09.2011

    • PubMed
    • Google Scholar
31. 1. Körding KP
    2. Wolpert DM

    (2004) Bayesian integration in sensorimotor learning

    Nature 427:244–247.

    https://doi.org/10.1038/nature02169

    • PubMed
    • Google Scholar
32. 1. Krakauer JW
    2. Ghez C
    3. Ghilardi MF

    (2005) Adaptation to visuomotor transformations: consolidation, interference, and forgetting

    Journal of Neuroscience 25:473–478.

    https://doi.org/10.1523/JNEUROSCI.4218-04.2005

    • PubMed
    • Google Scholar
33. 1. Lakens D

    (2013) Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs

    Frontiers in Psychology 4:863.

    https://doi.org/10.3389/fpsyg.2013.00863

    • PubMed
    • Google Scholar
34. 1. Leow LA
    2. Marinovic W
    3. de Rugy A
    4. Carroll TJ

    (2018) Task errors contribute to implicit aftereffects in sensorimotor adaptation

    European Journal of Neuroscience 48:3397–3409.

    https://doi.org/10.1111/ejn.14213

    • PubMed
    • Google Scholar
35. 1. Magescas F
    2. Prablanc C

    (2006) Automatic drive of limb motor plasticity

    Journal of Cognitive Neuroscience 18:75–83.

    https://doi.org/10.1162/089892906775250058

    • PubMed
    • Google Scholar
36. 1. Manohar SG
    2. Finzi RD
    3. Drew D
    4. Husain M

    (2017) Distinct motivational effects of contingent and noncontingent rewards

    Psychological Science 28:1016–1026.

    https://doi.org/10.1177/0956797617693326

    • PubMed
    • Google Scholar
37. 1. Mazzoni P
    2. Krakauer JW

    (2006) An implicit plan overrides an explicit strategy during visuomotor adaptation

    Journal of Neuroscience 26:3642–3645.

    https://doi.org/10.1523/JNEUROSCI.5317-05.2006

    • PubMed
    • Google Scholar
38. 1. McDougle SD
    2. Bond KM
    3. Taylor JA

    (2015) Explicit and implicit processes constitute the fast and slow processes of sensorimotor learning

    Journal of Neuroscience 35:9568–9579.

    https://doi.org/10.1523/JNEUROSCI.5061-14.2015

    • PubMed
    • Google Scholar
39. 1. McDougle SD
    2. Ivry RB
    3. Taylor JA

    (2016) Taking aim at the cognitive side of learning in sensorimotor adaptation tasks

    Trends in Cognitive Sciences 20:535–544.

    https://doi.org/10.1016/j.tics.2016.05.002

    • PubMed
    • Google Scholar
40. 1. Morehead JR
    2. Taylor JA
    3. Parvin DE
    4. Ivry RB

    (2017) Characteristics of implicit sensorimotor adaptation revealed by Task-irrelevant clamped feedback

    Journal of Cognitive Neuroscience 29:1061–1074.

    https://doi.org/10.1162/jocn_a_01108

    • PubMed
    • Google Scholar
41. 1. Nikooyan AA
    2. Ahmed AA

    (2015) Reward feedback accelerates motor learning

    Journal of Neurophysiology 113:633–646.

    https://doi.org/10.1152/jn.00032.2014

    • PubMed
    • Google Scholar
42. 1. Ohmae S
    2. Medina JF

    (2015) Climbing fibers encode a temporal-difference prediction error during cerebellar learning in mice

    Nature Neuroscience 18:1798–1803.

    https://doi.org/10.1038/nn.4167

    • PubMed
    • Google Scholar
43. 1. Oldfield RC

    (1971) The assessment and analysis of handedness: the edinburgh inventory

    Neuropsychologia 9:97–113.

    https://doi.org/10.1016/0028-3932(71)90067-4

    • PubMed
    • Google Scholar
44. 1. Pelli DG

    (1997) The VideoToolbox software for visual psychophysics: transforming numbers into movies

    Spatial Vision 10:437–442.

    https://doi.org/10.1163/156856897X00366

    • PubMed
    • Google Scholar
45. 1. Reichenthal M
    2. Avraham G
    3. Karniel A
    4. Shmuelof L

    (2016) Target size matters: target errors contribute to the generalization of implicit visuomotor learning

    Journal of Neurophysiology 116:411–424.

    https://doi.org/10.1152/jn.00830.2015

    • PubMed
    • Google Scholar
46. 1. Schlerf J
    2. Ivry RB
    3. Diedrichsen J

    (2012) Encoding of sensory prediction errors in the human cerebellum

    Journal of Neuroscience 32:4913–4922.

    https://doi.org/10.1523/JNEUROSCI.4504-11.2012

    • PubMed
    • Google Scholar
47. 1. Schmitz G
    2. Bock O
    3. Grigorova V
    4. Ilieva M

    (2010) Adaptation of eye and hand movements to target displacements of different size

    Experimental Brain Research 203:479–484.

    https://doi.org/10.1007/s00221-010-2245-z

    • PubMed
    • Google Scholar
48. 1. Shadmehr R
    2. Smith MA
    3. Krakauer JW

    (2010) Error correction, sensory prediction, and adaptation in motor control

    Annual Review of Neuroscience 33:89–108.

    https://doi.org/10.1146/annurev-neuro-060909-153135

    • PubMed
    • Google Scholar
49. 1. Shmuelof L
    2. Huang VS
    3. Haith AM
    4. Delnicki RJ
    5. Mazzoni P
    6. Krakauer JW

    (2012) Overcoming motor "forgetting" through reinforcement of learned actions

    Journal of Neuroscience 32:14617–14621.

    https://doi.org/10.1523/JNEUROSCI.2184-12.2012

    • PubMed
    • Google Scholar
50. 1. Smith MA
    2. Ghazizadeh A
    3. Shadmehr R

    (2006) Interacting adaptive processes with different timescales underlie short-term motor learning

    PLOS Biology 4:e179.

    https://doi.org/10.1371/journal.pbio.0040179

    • PubMed
    • Google Scholar
51. 1. Soechting JF

    (1984) Effect of target size on spatial and temporal characteristics of a pointing movement in man

    Experimental Brain Research 54:121–132.

    https://doi.org/10.1007/BF00235824

    • PubMed
    • Google Scholar
52. 1. Summerside EM
    2. Shadmehr R
    3. Ahmed AA

    (2018) Vigor of reaching movements: reward discounts the cost of effort

    Journal of Neurophysiology 119:2347–2357.

    https://doi.org/10.1152/jn.00872.2017

    • PubMed
    • Google Scholar
53. 1. Tanaka H
    2. Krakauer JW
    3. Sejnowski TJ

    (2012) Generalization and multirate models of motor adaptation

    Neural Computation 24:939–966.

    https://doi.org/10.1162/NECO_a_00262

    • PubMed
    • Google Scholar
54. 1. Taylor JA
    2. Klemfuss NM
    3. Ivry RB

    (2010) An explicit strategy prevails when the cerebellum fails to compute movement errors

    The Cerebellum 9:580–586.

    https://doi.org/10.1007/s12311-010-0201-x

    • PubMed
    • Google Scholar
55. 1. Taylor JA
    2. Hieber LL
    3. Ivry RB

    (2013) Feedback-dependent generalization

    Journal of Neurophysiology 109:202–215.

    https://doi.org/10.1152/jn.00247.2012

    • PubMed
    • Google Scholar
56. 1. Taylor JA
    2. Krakauer JW
    3. Ivry RB

    (2014) Explicit and implicit contributions to learning in a sensorimotor adaptation task

    Journal of Neuroscience 34:3023–3032.

    https://doi.org/10.1523/JNEUROSCI.3619-13.2014

    • PubMed
    • Google Scholar
57. 1. Taylor JA
    2. Ivry RB

    (2011) Flexible cognitive strategies during motor learning

    PLOS Computational Biology 7:e1001096.

    https://doi.org/10.1371/journal.pcbi.1001096

    • Google Scholar
58. 1. Therrien AS
    2. Wolpert DM
    3. Bastian AJ

    (2016) Effective reinforcement learning following cerebellar damage requires a balance between exploration and motor noise

    Brain 139:101–114.

    https://doi.org/10.1093/brain/awv329

    • PubMed
    • Google Scholar
59. 1. Therrien AS
    2. Wolpert DM
    3. Bastian AJ

    (2018) Increasing motor noise impairs reinforcement learning in healthy individuals

    Eneuro 5:ENEURO.0050-18.2018.

    https://doi.org/10.1523/ENEURO.0050-18.2018

    • PubMed
    • Google Scholar
60. 1. Thoroughman KA
    2. Shadmehr R

    (2000) Learning of action through adaptive combination of motor primitives

    Nature 407:742–747.

    https://doi.org/10.1038/35037588

    • PubMed
    • Google Scholar
61. 1. Tseng YW
    2. Diedrichsen J
    3. Krakauer JW
    4. Shadmehr R
    5. Bastian AJ

    (2007) Sensory prediction errors drive cerebellum-dependent adaptation of reaching

    Journal of Neurophysiology 98:54–62.

    https://doi.org/10.1152/jn.00266.2007

    • PubMed
    • Google Scholar
62. 1. van der Kooij K
    2. Oostwoud Wijdenes L
    3. Rigterink T
    4. Overvliet KE
    5. Smeets JBJ

    (2018) Reward abundance interferes with error-based learning in a visuomotor adaptation task

    PLOS ONE 13:e0193002.

    https://doi.org/10.1371/journal.pone.0193002

    • PubMed
    • Google Scholar
63. Preprint

    1. Vandevoorde K
    2. Orban de Xivry J-J

    (2018) Motor adaptation but not internal model recalibration declines with aging

    bioRxiv.

    https://doi.org/10.1101/292250

    • Google Scholar
64. 1. Vaswani PA
    2. Shmuelof L
    3. Haith AM
    4. Delnicki RJ
    5. Huang VS
    6. Mazzoni P
    7. Shadmehr R
    8. Krakauer JW

    (2015) Persistent residual errors in motor adaptation tasks: reversion to baseline and exploratory escape

    Journal of Neuroscience 35:6969–6977.

    https://doi.org/10.1523/JNEUROSCI.2656-14.2015

    • PubMed
    • Google Scholar
65. 1. Wagner MJ
    2. Kim TH
    3. Savall J
    4. Schnitzer MJ
    5. Luo L

    (2017) Cerebellar granule cells encode the expectation of reward

    Nature 544:96–100.

    https://doi.org/10.1038/nature21726

    • PubMed
    • Google Scholar
66. 1. Wu HG
    2. Miyamoto YR
    3. Gonzalez Castro LN
    4. Ölveczky BP
    5. Smith MA

    (2014) Temporal structure of motor variability is dynamically regulated and predicts motor learning ability

    Nature Neuroscience 17:312–321.

    https://doi.org/10.1038/nn.3616

    • PubMed
    • Google Scholar

[Editorial note: This article has been through an editorial process in which the authors decide how to respond to the issues raised during peer review. The Reviewing Editor's assessment is that all the issues have been addressed.]

Thank you for submitting your article "Intrinsic rewards modulate sensorimotor adaptation" for consideration by eLife. Your article has been reviewed by three peer reviewers, including Tamar R Makin as the Reviewing Editor and Reviewer #1, and the evaluation has been overseen by Timothy Behrens as the Senior Editor.

The Reviewing Editor has highlighted the concerns that require revision and/or responses, and we have included the separate reviews below for your consideration. If you have any questions, please do not hesitate to contact us.

In this paper the authors demonstrate how visual cues, uninformative of participants' performance, attenuates intrinsic sensorimotor learning (i.e. adaptation). For this purpose, the authors employ an “error clamp” paradigm, which they previously showed to induced implicit adaptation of reaching movements. By varying the size of the target with respect to the curser's final position, they demonstrate substantial shrinkage of the implicit adaptation effect, which they suggest occurs due to a reward which is experienced when the curser hits the target. Since previous research suggested that sensorimotor adaptation is insensitive to reward, the proposed study offers an interesting new piece of evidence. In particular, the authors suggest that their findings demonstrate that reward directly acts on the adaptation process, and run an experiment and a computational model to rule out an alternative account, by which reward impacts the movement itself.

While all reviewers agreed that the paper elegantly portrayed an interesting phenomenon, they were unsatisfied with how this phenomenon has been interpreted. In particular, the authors make, but don't substantiate, a series of assumptions that require further validation. Unless these assumptions could be substantiated with more data, alternative interpretations should be considered more thoroughly. The reviewers therefore suggest that the authors produce stronger evidence to pin-point the learning process being affected by the visual feedback, give more consideration to the alternative mechanism and provide quantitative evidence for the superiority of the adaptation modulation account. Unless these suggestions have been implemented, there was a consensus that the authors will need to consider substantially moderating some of their most central interpretations and conclusions, as currently reflected in the title and Abstract.

Major comments:

1) The authors focus their design and interpretation on SPE-driven learning, under the assumption that the error clamp isolates a single learning process, implicit recalibration driven by SPE. The possible alternatives to the "Adaptation Modulation" model are collapsed onto a single model that represents only one very specific alternative proposal about how reward might influence learning under the clamp. Therefore, rejecting this one particular model is not grounds to reject every possible architecture in which reward does not act by modulating SPE-driven adaptation. There reviewers point at several possible interpretations of what is happening and the paper needs to be thorough in considering these. Couldn't there be an implicit contribution that is not SPE-driven but is instead driven by reward/task error? How can the authors be certain that the participants were not engaging in some kind of re-aiming that may have been implicit or involuntary? The “hit” condition might increase salience, rather than impact SPE-related mechanisms, etc. Alternatively, there could be unexplored processes underlying the “miss” group, and the target size effect may be driven by alternative processes.

2) While focusing on the SPE-driven learning, an equally valid alternative to the authors account is that not hitting the target could be conceived as punishment, or in other words, failure to earn a reward amplifies sensitivity to sensory prediction error. Based on this logic, punishment (or lack of reward) will increase cerebellar learning, as the key mechanism underlying the results [see more specific comments in the individual reviews]. Relatedly, the "reward reduces learning" pitch requires some further conceptual qualification, e.g. from other related literature.

3) The reviewers were particularly dissatisfied with how the results of Experiment 3 were interpreted, based on the “adaptation” and “reinforcement” models. As this is perhaps the most crucial result in the paper, this requires more consideration. The current model assumes that reward has an immediate effect on learning rate parameters, when it seems likely it would have a more incremental onset. The model also needs to explain the differences in no vision/error clamp retention observed in Experiment 1 and 2. Furthermore, the model put forward for “reinforcement” was in no way exhaustive, and therefore doesn’t adequately represent the alternative process [see individual reviews for more specific suggestions]. Crucially, the comparison across the two competing "models" needs to be formally quantified. Currently, the data seems to sit in the middle of the two model predictions, rather than providing robust evidence for either.

4) The reviewers felt that more data/analysis/discussion of no visual feedback and zero error clamp is needed, to quality some of the assumptions of SPE-driven learning. This will help clarify to what extent the known properties of reward in adaptation tasks (often assumed to be explicit) might be attributed to an implicit process. This should also allow the authors to deal with the different rates of washout observed (but not discussed) in Experiment 1 and 2, indicating a role of reward in retention, with important potential implications on different models for learning. [see some specific suggestions in the individual reviews]

5) Inherent to the study is the key assumption that hitting the target would be intrinsically rewarding to the participants, despite explicit information that this is irrelevant to the task (and presumably the monetary reward?). The assumption is unsubstantiated and needs better evidence (e.g. pupilometry, control conditions with monetary reward)

Separate reviews (please respond to each point):

Reviewer #1:

This paper elegantly dissects a curious phenomenon, where small variations in task-irrelevant visual information dramatically and systematically modulate participants' performance on a basic motor reaching task. The study design is elegant, the results seem robust, the statistical analysis and reporting is appropriate, and the manuscript is clearly written. While I take no issue with the design, analysis and results, I have to question the interpretation the authors provide for their own findings. I wish to raise a few alternative accounts that I don't think have been sufficiently addressed in the manuscript, and I also ask for some clarification on the key interpretation the authors are offering.

Major comments:

1) If the visual feedback determines the SPE, then in the Hit condition there's no error, you correctly hit the target, and therefore there is no "need" for corrective movement (i.e. adaptation). According to this account, the target size ("reward") impacts the SPE by modulating the cursor feedback (Figure 4), not the adaptation or the movement. In other words, the subjects interpreted the visual feedback as correct task performance. This should not be translated to kinematic changes during baseline, because the visual cursor feedback is missing from the baseline. The authors suggest a control in their third experiment (the condition with a bisected target), but I didn't really understand how this eliminates the SPE interpretation.

2) The kinematics analysis could be potentially useful for ruling out that the large target was more salient. But the fact that participants tended to show lower variability (Experiment 2) and faster RT (Experiment 1) in the Hit target condition is concerning [*please note that upon discussion it has been agreed by the reviewers that this response profile is also compatible with reward, consistent with the authors' main interpretation]. They need to perhaps put the groups from Experiments 1 and 2 together to compare these baseline differences? Regardless, they should definitely report the results for Experiment 1 in the main results.

3) I don't really get the authors' conceptual pitch (that reward reduces learning), assuming that adaptation (perturbed hand position) is learning, why would a reword for a perturbed trial reduce that learning? Are there any precedents to this concept from other research? Surely, animal work on reinforcement learning would have noticed such an effect before, if it exists? Their discussion didn't help me understand their idea any better.

4) Inherent to the study is the key assumption that hitting the target would be intrinsically rewarding to the participants, despite explicit information that this is irrelevant to the task (and presumably the monetary reward?). I'm not sure what they mean here by reward, and how this could be generalised to other fields of research where reward is a key concept (e.g. decision making, reinforcement learning).

5) How much of the late effect shown across studies is actual adaptation/learning (i.e. due to the visual feedback), as opposed to drift? What happens when you ask participants to just repeatedly perform a movement to a target with no visual feedback? It seems necessary to quantify these drifts, in order to accurately account for the "learning".

Minor Comments:

1) "Interestingly, these results appear qualitatively different to those observed when manipulating the clamp offset. Our previous study using clamped visual feedback demonstrated that varying clamp offset alone results in different early learning rates, but produces the same magnitude of late learning (Kim et al., 2018)." please spare us having to read this paper, what do you mean by clamp offset? please provide better description of the study (perhaps in the discussion?).

2) It would be very beneficial if the authors could provide the exact instructions given to the participants (i.e. the study script).

3) "This was supported by our control analyses, perceptual control experiment, and our finding that the Straddle group in Experiment 1 was similar to the Hit group, suggesting that the effect of target size was categorical." In that experiment Hit was different from Straddle, which was similar to miss.

Reviewer #2:

The work by Kim et al., investigates a very interesting and well investigated topic; the effect of reward on sensorimotor adaptation. Through 3 well designed experiments, the authors show that task-based reward attenuates sensorimotor adaptation. The authors suggest that these results can only be explained by a model in which reward directly acts on the adaptation system (learning parameter in a state-space model). This is a well written manuscript, with clear results and interesting conclusions. My biggest issue is the lack of analysis and discussion of their after-effect (no vision/zero error clamp) data from Experiment 1 and 2. I cannot explain the differences between groups during these blocks (see below) by reward only acting on the learning component of the state-space model. To me, this data indicates that reward (Hit group) is having an effect on retention. I also have issues with the implementation of the model in Experiment 3 and the possibility that the task could also involve implicit punishment. As a result, I feel the conclusions made in this manuscript create a clear story in contrast to the data which suggests a more complicated narrative.

Specifics:

No visual feedback vs. 0⁰ error clamp following adaptation: Why did the authors switch from “no feedback” to “0⁰ error clamp” from Experiment 1 to Experiment 2? No explanation is given for this switch.

After-effects: Following this, I find it odd how little the “after-effect” is analysed/discussed considering this was a major component of the analysis in the seminal work by Morehead et al., 2017, and is also analysed in Kim et al., 2018. The authors describe in the Materials and methods that the after-effect was defined by the first no-feedback cycle but then it is never further discussed… Although it is difficult to see in Figure 1, the data provided by the authors reveals that performance is substantially different across groups during no feedback. Whereas, Miss and Straddle decline across trials, Hit maintains a plateau (by the end of No FB, groups are near the same level of performance). This plateau is strikingly different from the behaviour observed in Kim et al., 2018. This is also seen in Experiment 2, where the Straddle group decline across trials and the Hit group maintain a plateau (annoyingly I am unable to submit figures in my review, but this is clear in the data provided by the authors). If the authors believe that reward is having a specific effect on learning, and not retention, how would a state-space model explain these results? During both these trials types, error has to be fixed to zero (there is no error to adjust behaviour with). Therefore, the only parameter which influences behaviour is the retention parameter. Thus, Straddle and Miss show the normal decay seen with no-vision/error clamp trials with a retention parameter of approx. 0.8-0.9. But the Hit group must have a retention parameter of 1. To me, this would suggest reward is having a direct effect on retention causing enhanced retention (relative to where they started at). Can the author's explain this behaviour in the context of the adaptation modulation model? In other words, can you model the plateau performance in no vision/zero clamp trials of the Hit group when reward only affects the learning parameter?

Interpretation of results: Throughout the paper there is an assumption (first stated in paragraph five of the Introduction) that hitting the target is intrinsically rewarding, is it not an equally valid assumption that missing the target is intrinsically punishing? Or in fact both options may be true and have differing effects. How do we know what the “null” cerebellar behaviour is (this also relevant for the Morehead et al., and Kim et al., papers)? As the participants move from a baseline situation where they are being successful to a situation where they are being unsuccessful, couldn't the error clamp behaviour be regarded as intrinsically punishing? Therefore, across all these studies we could be observing a cerebellar process which is augmented by intrinsic punishment (aversive signals in the cerebellum are well known). Within this context, could the results not be viewed as punishment (moving from hitting the target to missing) increasing cerebellar learning (U) and reward increasing retention (A)? This is relevant for paragraph six of the Discussion, modification of the A parameter alone would indeed lead to higher asymptote in opposition to the current results, however the retention findings in Experiments 1 and 2 suggest that A is affected by reward (see above). If both A is increased and U decreased (by a lack of intrinsic punishment) when hitting the target then lower asymptotes can be observed and would explain the findings (including the retention results which are not possible with a learning-only effect).

Model comparison: Another issue I have is the use of the model in the transfer phase. Figure 5B clearly shows the predictions of the transfer phase for the straddle-hit group based on the two opposing theories. Considering this is the key result to the entire paper, surely the authors need to compare which model predicts the data better? To me the data seems to sit in the middle of these model predictions, rather than providing robust evidence for either. The authors could compare their initial adaptation modulation prediction as outlined in Figure 5F with a model that reflects what would happen with a movement reinforcement model (no change/flat line). They could use BIC or AIC model selection to determine which model explains the individual participant data (or the bootstrapped data) better. At the moment, there is no comparison between the opposing models. The authors have simply provided a prediction for the adaptation modulation model and said it approximately looks similar to the data (without even providing any form of model fit i.e. r-squared for transfer phase).

Similar conclusions to Leow et al., 2018: I am unsure what the rules are regarding a preprint but the conclusions for the current manuscript are very similar to Leow et al., 2018; implicit sensorimotor adaptation is reduced by task-based reward (modulating the amount of task error/reward). So how novel are these conclusions?

No normality or homogeneity of variance tests: As ANOVAs are used, tests for normality (Shapiro-Wilk) and homogeneity are required (Levene). If these have been performed, then this needs to be made clear.

Reaction and movement time across experiments: the authors state in the Materials and methods that there was a 300ms movement time cutoff. Did this include reaction time? Did the authors examine reaction time during adaptation and no-vision/0 error clamp blocks, rather than at baseline? This analysis should be included (i.e. reaction and movement time for the trials used in Figure 1E, F and similar for Experiment 2).

Early adaptation rate: I take issue with calling this a measurement of “rate” (subsection “Data Analysis”)? This value is a measurement of average early performance across cycles, not the change in performance which would be measured by calculating the difference between cycles. Or is this what the authors did and the methods are incorrect?

Experiment 1 post hoc t-tests: Are these corrected for multiple comparisons (p=0.16). Although both would survive, this is simply good practise.

Experiment 3 statistics: The order of statistical analysis is odd. Why not start with the omnibus ANOVA and then describe the comparisons between the 3 groups? In addition, I do not like the use of planned comparisons unless these are pre-registered somewhere. I am guessing the t-test between the control group and straddle-to-hit does not survive a correction for multiple comparisons? However, as this is a control experiment, planned comparisons are begrudgingly acceptable.

Wrong group described (Discussion paragraph three): The straddle group is similar to the Miss group.

Reviewer #3:

Implicit recalibration is a specific learning process that is known to be implicit, cerebellum-dependent, and driven by sensory prediction errors. Previous work has suggested that this process is insensitive to reward. The core claim in this paper is that, in fact, implicit recalibration can be modulated by reward in that failure to earn a reward amplifies sensitivity to sensory prediction error. If true, this is certainly an interesting and important finding. I do not believe, however, that the experiments presented here are capable of supporting this conclusion.

The experiments employ a clever “error clamp” paradigm which the authors have previously shown induces implicit adaptation of reaching movements. The data in the current paper demonstrate clearly that the extent of adaptation under this clamp depends on the size of the target. If the target is big enough that the cursor lands inside it, less is learned from the error. I think it is fair to conclude from these results that there is an implicit learning process at play which is sensitive to reward. My major concern, however, is with the attempt to distinguish between the "Adaptation modulation" theory and the "Movement reinforcement" theory, which really is the heart of the paper.

The authors make the very strong assumption that their clamp manipulation isolates a single learning process – implicit recalibration driven by SPE. The results state quite clearly; "This [clamp] method allows us to isolate implicit learning from an invariant SPE, eliminating potential contributions that might be used to reduce task performance error." I agree that the clamp isolates implicit learning, but couldn't there be an implicit contribution that is not SPE-driven but is instead driven by reward/task error? How can the authors be certain that the participants were not engaging in some kind of re-aiming that may have been implicit or involuntary?

The authors do seem to entertain the possibility of distinct SPE-based and reward-based processes, alluding in the Abstract to a model "in which reward and adaptation systems operate in parallel." Figure 4 even quite clearly illustrates such a parallel architecture. But by later in the paper the myriad possible alternatives to the "Adaptation Modulation" model are collapsed onto a single idea in which reward reinforces particular movements, causing them to act as an “attractor”. Ultimately, the conclusion rests on rejecting this “attractor” / "Movement Reinforcement" model because it incorrectly predicts that asymptote will be maintained in Experiment 3 when the target becomes larger. The Adaptation Modulation model is the only model left and so must be correct.

Of course the “reinforcement” model represents only one very specific alternative proposal about how reward might influence learning under the clamp (other than by modulating SPE-driven recalibration). Rejecting this one particular model is not grounds to reject EVERY possible architecture in which reward does not act by modulating SPE-driven adaptation. For this logic to work, it would have to be the case that ANY model in which reward does not modulate SPE-driven learning would necessarily predict a consistent asymptote after the straddle->hit transition.

It is not too difficult to formulate a model in which reward does NOT modulate SPE-driven learning, but which doesn't predict a consistent asymptote after the straddle->hit transition. Suppose behavior in the clamp is the sum of two distinct processes acting in parallel, one feeding on SPE, and one feeding on task error, both with SSM-like learning curves. The latter processes might correspond to a kind of implicit re-aiming and might plausibly only be updated when a movement fails to earn reward. Then disengaging this system by suddenly providing reward (at the transition from straddle to hit) might lead to a gradual decline in that component of compensation. This would lead to behavior quite similar to what the authors observe. I have appended MATLAB code at the end this review which implements a version of this model.

Finally, I would add that I'm not even convinced that it is reasonable to assume that the "Movement Reinforcement" “attractor” model, in which the authors suggest that reinforcing movement attenuates decay, would necessarily lead to a persistent asymptote. What if the reinforcement only partially attenuates decay, or requires repetition to do so (during which time it may decay slightly)?

For these reasons, I don't believe that the primary conclusion is tenable.

Putting aside the question of adaptation modulation, I do like the experiments in this paper and I think phenomena that have been shown are interesting. I think the results convincingly establish the existence of an implicit learning process that is sensitive to reward, and the paradigm provides a platform for exploring the properties of this process. I would suggest the authors try to develop the manuscript along these lines, though it is not particularly satisfying if the results cannot distinguish between modulation of SPE-driven learning by reward and a parallel, reward-driven process as I feel is the case at present.

As a constructive suggestion, it may be enlightening to examine behavior in the 0deg clamp and no-feedback conditions more closely. The authors draw no attention to it in the paper (nor really explain why these blocks were included) but the data seem to suggest a difference in rate of decay between the “hit” and “miss” conditions in Experiments 1 and 2. Different theories about the nature of the implicit reward-based learning would predict quite different behavior in these conditions, but it's impossible to judge from such short periods whether the decay curves might converge, stay separate, or even cross over. I wonder if an additional experiment exploring this phenomenology in more detail, with a decay block that is longer than 5-10 cycles, might be enlightening, though I admit I'm unsure if this would necessarily help to disambiguate the SPE-modulation model from the alternative “parallel” model. Perhaps comparing a NoFB condition to a 0-deg clamp condition might be useful. A more concrete (and exhaustive) set of theories/models would help to formulate clearer predictions about what to expect here.

Matlab Code:

clear all

Ntrials = 220;% number of trials

x_reward = zeros(2,Ntrials);% reward component of compensation (one row for each group)

x_spe = zeros(2,Ntrials);% SPE-driven component of compensation (one row for each group)

U_reward(1) = 0.3;% initial sensitivity of reward component to task error for first group (straddle->hit)

U_reward(2) = 0;% initial sensitivity to reward component to task error for second group (hit->straddle)

U_spe = 0.25;% sensitivity to sensory prediction error throughout

A_reward = 0.9;% retention of reward component

A_spe = 0.98;% retention of SPE-driven component

for i=2:Ntrials

if(i==131)% swap U_rwd values for the two groups on trial 131

U_reward = fliplr(U_reward);

end

% update components

for j=1:2% iterate through two groups

x_reward(j,i) = A_reward*x_reward(j,i-1) + U_reward(j);% reward component update

x_spe(j,i) = A_spe*x_spe(j,i-1) + U_spe;% spe component update

end

end

% sum components to get total compensation

x_total = x_reward+x_spe;

% plot results

figure(2); clf; hold on

plot(x_total(1,:),'b','linewidth',2)

plot(x_total(2,:),'g','linewidth',2)

xlabel('Trial Number')

ylabel('Reach Angle')

legend('Straddle->Hit','Hit->Straddle')

Minor Comments:

Results third paragraph: “angle” or “deviation” may be a better word here than “offset” which could easily be confused with switching off the clamp.

The "models" are never formulated in concrete terms. I would be fine with a verbal argument as to why the Movement Reinforcement theory should predict a consistent asymptote in Experiment 3, but it feels a stretch to call them “models” when they are never concretely fleshed out. Fleshing out the models mathematically may also help to bring to the surface some of the tacit assumptions that have led to the headline conclusion of the paper.

The "model" that generates the "model prediction" in Figure 5F is rather ad-hoc. It’s odd to suggest that the parameters of the underlying process would instantly flip. Also why are the predictions discontinuous across the dashed “transfer” line?

Results subsection “Experiment 3”: "the predictions of the model slightly underestimated the observed rates of change for both groups". Do you mean OVERestimated?

[Editors' note: further revisions were suggested, as described below.]

Thank you for submitting your article "The influence of task outcome on implicit motor learning" for consideration by eLife. Your article has been reviewed by three peer reviewers, including Tamar R Makin as the Reviewing Editor and Reviewer #1, and the evaluation has been overseen by Timothy Behrens as the Senior Editor. The following individual involved in review of your submission has agreed to reveal their identity: Adrian M Haith (Reviewer #3).

All three reviewers were fully satisfied with the amendments made to the manuscript, and agreed the manuscript was much improved. Reviewers 2 and 3 provided some valuable follow-up suggestions, which I expect you would be happy to consider. I'm pasting these below for your benefit.

Please incorporate your changes in a final version of your manuscript (no need for a detailed rebuttal).

Reviewer #1:

I find the revised manuscript much improved. The analysis and discussion are more comprehensive and provide a more nuanced, and as such more transparent and balanced, account of the very interesting behavioural effects observed in the study. I'm also satisfied with the responses provided to the reviewers comments. On a personal level, I found the manuscript easier to read (though this might be due to a repetition effect!). I have no further comments.

Reviewer #2:

The authors have done a great job at addressing the issues highlighted by all reviewers, and the manuscript is much improved. Although the overall story is less eye-catching than “reward modulates SPE learning”, I believe it is now a far more informative piece of work which is still novel. I have a few fairly small outstanding issues:

Collinearity of parameters: Could the authors test the collinearity of the parameters in each model? In other words, is the data rich enough to have 4/6 independent parameters (I know this is an issue when using the 2-state Smith model with certain paradigms)? For example, when you use the bootstrapping procedure how correlated are the parameters? A heat map maybe for each model would be a good way to show this. This will ensure the results from paragraph three of subsection “Experiment 3 – Modeling Results” are valid. Also parameter recovery (Palminteri et al., Plos CB, 2017) might be of interest.

Control group figure: I would add Figure 5—figure supplement 1 to the main document.

Discussion (paragraph nine of subsection “Modeling the Influence of Task Outcome on Implicit Changes in Performance”): Aren't the target jump predictions similar to what Leow et al., 2018, have already shown to be true?

Reviewer #3:

The authors have revised the paper very thoroughly. For the most part, I find that the paper hits the mark well in terms of articulating theories that can versus can't be disambiguated based on the current experiments. I do, however, have a few further suggestions to help clarify the paper further.

Use of the term “target error” is a bit ambiguous. At times it is characterized as a binary hit/miss signal (e.g. Introduction paragraph five; Discussion paragraph five). At other times it seems to refer to a vector error (i.e. having magnitude and direction) (e.g. Introduction paragraph six distinguishes it from “hitting the target”; penultimate paragraph of subsection “Experiment 2”). This is liable to cause considerable confusion. It confused me until reading the description of the dual adaptation model, in practical terms it is tantamount to a binary signal, given the constant error size and direction. But I think a little more conceptual clarity is needed. e.g. does TE already take reward into account? Or are TE and SPE equivalent (in this task) but learning from TE is subject to modulation by reward, while learning from SPE is not?

The term “model-based” is used throughout, sometimes (e.g. paragraph four of subsection “Modeling the Influence of Task Outcome on Implicit Changes in Performance”) in the sense of a model-based analysis of the effects of reward, i.e. examining how presence/absence of reward affects fitted parameters of a model (this usage is fine). At other times (e.g. Conclusions section) it's used essentially as a synonym for error-based. I'm a bit skeptical as to whether this latter usage adds much value or clarity. I appreciate it derives from Huang et al. and Haith and Krakauer, who made a case that implicit adaptation reflected updating an internal forward model. However, error-driven learning need not necessarily be model-based. For instance, if learning is driven by target error, it's not clear that this has anything to do with updating any kind of model of the environment (and "inverse model" doesn't really count as a proper model in this sense). So I would caution against using the term “model-based” when the idea doesn't inherently involve a forward model. Particularly given the two distinct usages of “model-based” in this paper.

The Movement Reinforcement model is reasonable, although it is fairly ad hoc. The Discussion argues convincingly that this model is largely illustrative, and that its behavior can be taken as representative of a broad class of “operant reinforcement” models. I think articulating something to this effect earlier, when the model is first introduced, would be helpful. At the moment, it comes from nowhere and it's a little perplexing to follow exactly why this is the model and not, say, a more principled value-function-approximation approach. With that in mind, some of the finer details of the model might be better placed in the Materials and methods section so as not to bog down the results with specifics that aren't all that pertinent to the overall argument.

Minor Comments:

I appreciate the discussion of implicit TE-driven learning in motivating the Dual Error model (subsection “Theoretical analysis of the effect of task outcome on implicit learning”). But I was surprised the authors didn't mention this again in the discussion, instead only speculating that TE-based learning might be re-aiming that has become implicit through automatization/caching, and consequent making the dual error model seem implausible. But it seems perfectly plausible that TE-based learning is just another implicit, error-based learning system, separate from SPE-driven implicit learning, that never has anything to do with re-aiming.

Subsection “Modeling the Influence of Task Outcome on Implicit Changes in Performance”, it doesn't seem necessary to invoke “SPE-driven” here. Could in principle be error-based learning driven by something like "target error" (i.e. just the distance between the center of the cursor and the center of the target). Ditto in the Conclusion section.

Introduction section: "We recently introduced a new method.… designed to isolate learning from implicit adaptation" slightly ambiguous sentence, I first read it as though learning and implicit adaptation are separate things being dissociated. Maybe just drop "learning from"?

Introduction section: "Given that participants have no control over the feedback cursor, the effect of this task outcome would presumably operate in an implicit, automatic manner." It's not having no control that makes it implicit… Might be better rephrased to something like "Given that participants are aware that they have no control over the feedback cursor…"?

Second paragraph of subsection “Theoretical analysis of the effect of task outcome on implicit learning”: this paragraph misses a key detail, that “reinforcing” or “strengthening the representation of” rewarded actions really means that it makes those actions more likely to be selected in the future.

Third paragraph of subsection “Theoretical analysis of the effect of task outcome on implicit learning”: “composite” is somewhat vague. Would “sum” or “average” be accurate?

Third paragraph of subsection “Experiment 3 – Modeling Results”: something is up with the brackets here.

Major comments:

1) The authors focus their design and interpretation on SPE-driven learning, under the assumption that the error clamp isolates a single learning process, implicit recalibration driven by SPE. The possible alternatives to the "Adaptation Modulation" model are collapsed onto a single model that represents only one very specific alternative proposal about how reward might influence learning under the clamp. Therefore, rejecting this one particular model is not grounds to reject every possible architecture in which reward does not act by modulating SPE-driven adaptation. There reviewers point at several possible interpretations of what is happening and the paper needs to be thorough in considering these. Couldn't there be an implicit contribution that is not SPE-driven but is instead driven by reward/task error? How can the authors be certain that the participants were not engaging in some kind of re-aiming that may have been implicit or involuntary? The “hit” condition might increase salience, rather than impact SPE-related mechanisms, etc. Alternatively, there could be unexplored processes underlying the “miss” group, and the target size effect may be driven by alternative processes.

This comment was at the center of our thinking throughout the revision process and has inspired major changes in the manuscript. The reviewers are correct, rejecting one model is not “grounds to reject every possible architecture…”. The revision now includes a substantially expanded section on the modeling front, including the addition of an alternative model in which we consider how a second error signal, based on the task outcome, could be used to recalibrate an internal model. While obviously not exhaustive, we now formally present three distinct models: Movement Reinforcement, Adaptation Modulation, and Dual Error models, and briefly consider possible combinations of these models. The Dual Error model has been adapted from reviewer 3’s comments and postulates that implicit motor learning could result from target error as well as from SPE. To preview, this model provides the best fit to the data and figures prominently in Experiment 3 and the Discussion.

We gave extensive consideration to whether we should consider the effect of target error as a form of re-aiming. Indeed, this led us to submit a query to the reviewers on this point given our concerns that an aiming-based account didn’t seem congruent with traditional views on the role of aiming in sensorimotor adaptation. This query led to an extended Skype session with reviewer 2, Adrian Haith, and this discussion helped sharpen our thinking on this front, as well as inspired some ideas for future experiments. The Discussion section includes a few paragraphs on this issue, both in terms of explicit and implicit forms of an aiming hypothesis. There we highlight how an implicit process sensitive to target error does not share key features of explicit aiming (error size-dependency, flexibility, fast learning), or the idea of a cached (and thus implicit) aiming process. In the end, we think it suffices at this stage to highlight that the results can be accounted for by the operation of two internal models, one operating on SPE and the other on TE. We sketch out ways in which future work can directly test this model and also explore whether these different learning systems engage similar or dissimilar neural systems.

With regard to the issue of unexplored processes underlying the Miss group, we take that up in our response to the following comment.

2) While focusing on the SPE-driven learning, an equally valid alternative to the authors account is that not hitting the target could be conceived as punishment, or in other words, failure to earn a reward amplifies sensitivity to sensory prediction error. Based on this logic, punishment (or lack of reward) will increase cerebellar learning, as the key mechanism underlying the results [see more specific comments in the individual reviews]. Relatedly, the "reward reduces learning" pitch requires some further conceptual qualification, e.g. from other related literature.

The point here is well-taken: Rather than think of hitting the target as providing a “reward” that attenuates the operation of this process, it may well be that missing the target provides a “punishment” that amplifies the operation of this process. We now make this point explicitly in our formalization of the Adaptation Modulation model and the new Dual Error model, as well as in the Discussion.

In the Dual Error model, misses can be considered to add to learning by providing a second error-based learning process, one sensitive to target error. In contrast, the Movement Reinforcement model incorporates a process that only learns a bias when hitting the target (i.e., when rewarded), and thus also contrasts with the other two models in that it cannot be interpreted from a “failure sensitizes learning” perspective.

We have expanded our discussion of previous studies on the impact of task outcome on adaptation in a number of places. Here, too, we are grateful to reviewer 3, for bringing to our attention the relevance of some of the earlier work in which the target is displaced, sometimes bringing it into alignment with a perturbed cursor (and thus manipulating target error, independent of SPE.

We recognize that, in a number of studies using more traditional adaptation tasks (e.g., standard visuomotor rotation) reward is viewed as improving learning (Galea et al., 2015, Nikooyan and Ahmed, 2015), and indeed, a boost would seem more intuitive. The clamp task differs in that “learning” is arguably detrimental to performance, since the instructions to the participants are to ignore the cursor and reach straight toward the target. More generally, the revision includes many changes to clarify the assumptions of the models (including their formalization), as well as note limitations with our particular implementations.

3) The reviewers were particularly dissatisfied with how the results of Experiment 3 were interpreted, based on the “adaptation” and “reinforcement” models. As this is perhaps the most crucial result in the paper, this requires more consideration. The current model assumes that reward has an immediate effect on learning rate parameters, when it seems likely it would have a more incremental onset. The model also needs to explain the differences in no vision/error clamp retention observed in Experiment 1 and 2. Furthermore, the model put forward for “reinforcement” was in no way exhaustive, and therefore doesn’t adequately represent the alternative process [see individual reviews for more specific suggestions]. Crucially, the comparison across the two competing "models" needs to be formally quantified. Currently, the data seems to sit in the middle of the two model predictions, rather than providing robust evidence for either.

We have overhauled the presentation of Experiment 3, formalizing all three of the models including the new Dual Error model, opting for a different method to fit the data, and quantifying the model comparisons using goodness-of-fit measures and AIC scores for model selection. We emphasize that the results do not support the Movement Reinforcement model in its current implementation, and more generally, any model that incorporates independent activity of model-based adaptation and model-free reinforcement learning processes. The new fitting method, by which we model the data for both groups simultaneously, shows that the Adaptation Modulation and Dual Error models do a good job of fitting the data and accounting for the significant change in asymptote for the Straddle-to-Hit group (see new Figure 6).

4) The reviewers felt that more data/analysis/discussion of no visual feedback and zero error clamp is needed, to quality some of the assumptions of SPE-driven learning. This will help clarify to what extent the known properties of reward in adaptation tasks (often assumed to be explicit) might be attributed to an implicit process. This should also allow the authors to deal with the different rates of washout observed (but not discussed) in Experiment 1 and 2, indicating a role of reward in retention, with important potential implications on different models for learning. [see some specific suggestions in the individual reviews]

We now include an analysis and discussion of the no feedback and 0° clamp data. The Hit Target groups do show less decay of the adapted state when analyzed in terms of the absolute change in hand angle. However, retention is normally viewed in proportional terms (and formalized this way in all of our models). When the analysis is performed in terms of proportional changes, there were no reliable differences between the conditions in both Experiments 1 and 2. Similarly, although the bootstrapped parameter estimates of the retention parameter were higher in the Hit condition relative to the Miss (or Straddle) condition, the confidence intervals indicate that the modulation of retention due to task outcome was much less robust than the changes in learning rate. We provide all of this information in the revision, and also acknowledge that hitting the target may enhance retention.

5) Inherent to the study is the key assumption that hitting the target would be intrinsically rewarding to the participants, despite explicit information that this is irrelevant to the task (and presumably the monetary reward?). The assumption is unsubstantiated and needs better evidence (e.g. pupilometry, control conditions with monetary reward)

Inspired by the reviewers’ comments, our interpretations no longer rely on the assumption that hitting the target is intrinsically rewarding. Rather, we focus more generally on how task outcome can affect overall implicit learning. We note that there is precedence for the intrinsic reward idea (Leow et al., 2018, Huang et al., 2011). However, as developed in the revision, this is just one of the directions we take in the modeling. We now also consider the possibility that missing the target creates a separate target error signal for learning (as in the Dual Error model), or that the task outcome might act as a gain controller on adaptation (as in the Adaptation Modulation model).

Separate reviews (please respond to each point):

Reviewer #1:

This paper elegantly dissects a curious phenomenon, where small variations in task-irrelevant visual information dramatically and systematically modulate participants' performance on a basic motor reaching task. The study design is elegant, the results seem robust, the statistical analysis and reporting is appropriate, and the manuscript is clearly written. While I take no issue with the design, analysis and results, I have to question the interpretation the authors provide for their own findings. I wish to raise a few alternative accounts that I don't think have been sufficiently addressed in the manuscript, and I also ask for some clarification on the key interpretation the authors are offering.

Major comments:

1) If the visual feedback determines the SPE, then in the Hit condition there's no error, you correctly hit the target, and therefore there is no "need" for corrective movement (i.e. adaptation). According to this account, the target size ("reward") impacts the SPE by modulating the cursor feedback (Figure 4), not the adaptation or the movement. In other words, the subjects interpreted the visual feedback as correct task performance.

We have revised the manuscript to clarify our definitions of sensory prediction error and target error. The SPE is defined in terms of the angle between the centers of the cursor and target. As such, one can still have an SPE, even when hitting the target. So, even if the participant thinks they are performing the task as instructed (“reach directly to the target and ignore the feedback”), the motor system is modified. In the current work, target error is defined by whether the cursor hits or misses the target. We do not know if this need be binary; the fact that the Miss and Straddle groups lead to similar behavior suggests there may be some categorical aspect to this. Nonetheless, future work is required to determine if the learning from TE scales with the size of the target error.

This should not be translated to kinematic changes during baseline, because the visual cursor feedback is missing from the baseline.

In our alternative hypotheses section (subsection “Attenuated behavioral changes are not due to differences in motor planning”), we consider other ways in which the manipulation of target size might affect behavior. The kinematic analyses were performed to assess the extent to which differences in planning reaches to small or large targets might influence learning. On the comment about the lack of cursor feedback during baseline, we note that in all experiments there are is a baseline period with veridical cursor feedback and that these are the data used in our kinematic and temporal analyses.

The authors suggest a control in their third experiment (the condition with a bisected target), but I didn't really understand how this eliminates the SPE interpretation.

The bisected target group in Experiment 3 was to test a perceptual uncertainty hypothesis: Perhaps there is less certainty about the position of the cursor in the large target context and this weakens the SPE signal, resulting in attenuated adaptation. We added the bisecting line to provide a salient reference point, thus making it clear that the clamped cursor was off center. As our analyses show, the target size effect was not influenced by the bisecting line, arguing against the hypothesis that the effect of the large target is due to perceptual uncertainty.

2) The kinematics analysis could be potentially useful for ruling out that the large target was more salient. But the fact that participants tended to show lower variability (experiment 2) and faster RT (Experiment 1) in the Hit target condition is concerning [*please note that upon discussion it has been agreed by the reviewers that this response profile is also compatible with reward, consistent with the authors' main interpretation]. They need to perhaps put the groups from Experiments 1 and 2 together to compare these baseline differences? Regardless, they should definitely report the results for Experiment 1 in the main results.

We now include the analyses of temporal and kinematic variables from Experiment 1 in the main text, as well as a table of RTs, MTs, and movement variability from all three experiments in the Appendix. Baseline RTs were different in Experiment 1, with faster RTs observed in the Hit group compared to the other two conditions. However, there were no differences between the Hit and Straddle groups in Experiment 2, and in fact, numerically, RTs were slower for the Hit group. There were no differences nor trends in the movement variability measures between groups in Experiments 1 and 2. Given the absence of systematic differences in the temporal and kinematic measures (see Tables 1-3 in the Appendix), we infer that there were no substantial differences in motor planning between the target size conditions.

3) I don't really get the authors' conceptual pitch (that reward reduces learning), assuming that adaptation (perturbed hand position) is learning, why would a reword for a perturbed trial reduce that learning? Are there any precedents to this concept from other research? Surely, animal work on reinforcement learning would have noticed such an effect before, if it exists? Their discussion didn't help me understand their idea any better.

Please refer to our response to the latter part of Summary Comment 2.

4) Inherent to the study is the key assumption that hitting the target would be intrinsically rewarding to the participants, despite explicit information that this is irrelevant to the task (and presumably the monetary reward?). I'm not sure what they mean here by reward, and how this could be generalised to other fields of research where reward is a key concept (e.g. decision making, reinforcement learning).

Please see our response to Summary Comment 5.

5) How much of the late effect shown across studies is actual adaptation/learning (i.e. due to the visual feedback), as opposed to drift? What happens when you ask participants to just repeatedly perform a movement to a target with no visual feedback? It seems necessary to quantify these drifts, in order to accurately account for the "learning".

All hand angles were baseline-corrected to account for intrinsic biases. We counterbalanced clockwise and counter-clockwise clamp directions within each group; the data for the clockwise group were flipped for the analyses and visualizations of the data. The counterbalancing across subjects would cancel out any systematic biases which were not due to the error clamp. Hand angles going in the positive direction are indicative of a sign-dependent correction in response to error—that is, in the opposite direction of the clamp. Finally, we note that we have included a 0° visual clamp in two previous studies as a baseline condition, and found no evidence of systematic drift (Morehead et al., 2017; Kim et al., 2018).

Minor Comments:

1) "Interestingly, these results appear qualitatively different to those observed when manipulating the clamp offset. Our previous study using clamped visual feedback demonstrated that varying clamp offset alone results in different early learning rates, but produces the same magnitude of late learning (Kim et al., 2018)." please spare us having to read this paper, what do you mean by clamp offset? please provide better description of the study (perhaps in the Discussion?).

What we meant by “clamp offset” was the angular deviation of the clamp relative to the center of the target. We have removed all usage of the phrase “clamp offset” to avoid confusion. We have now provided a clearer description of our previous work (Results paragraph five).

2) It would be very beneficial if the authors could provide the exact instructions given to the participants (i.e. the study script).

We now provide the study script in the Supplementary file 1.

3) "This was supported by our control analyses, perceptual control experiment, and our finding that the Straddle group in Experiment 1 was similar to the Hit group, suggesting that the effect of target size was categorical." In that experiment Hit was different from Straddle, which was similar to miss.

This statement has been corrected.

Reviewer #2:

The work by Kim et al., investigates a very interesting and well investigated topic; the effect of reward on sensorimotor adaptation. Through 3 well designed experiments, the authors show that task-based reward attenuates sensorimotor adaptation. The authors suggest that these results can only be explained by a model in which reward directly acts on the adaptation system (learning parameter in a state-space model). This is a well written manuscript, with clear results and interesting conclusions. My biggest issue is the lack of analysis and discussion of their after-effect (no vision/zero error clamp) data from Experiment 1 and 2. I cannot explain the differences between groups during these blocks (see below) by reward only acting on the learning component of the state-space model. To me, this data indicates that reward (Hit group) is having an effect on retention. I also have issues with the implementation of the model in Experiment 3 and the possibility that the task could also involve implicit punishment. As a result, I feel the conclusions made in this manuscript create a clear story in contrast to the data which suggests a more complicated narrative.

We now provide two analyses of the no feedback and 0° clamp data: comparing absolute change as well as the proportional change during these trials. In terms of the statistical analyses, there is actually no difference between the groups during the washout phase when analyzed as proportional change, and we report this while noting that, numerically, the data do indicate stronger retention in the Hit groups in Experiments 1 and 2. We also now provide the parameter values for the different models (subsection “Experiment 3 - Modeling Results”) and here we note that, although the biggest (and only reliable) effect is on the learning rates, the estimate of the retention parameter is larger for the Hit groups.

For a more extensive discussion of the changes in the modeling work, please see our response to Summary Comment 1. For interpretation of whether the task involves punishment, please see our response to Summary Comment 2.

Specifics:

No visual feedback vs. 0⁰ error clamp following adaptation: Why did the authors switch from “no feedback” to “0⁰ error clamp” from Experiment 1 to Experiment 2? No explanation is given for this switch.

We now provide an explanation in Results section, Experiment 2. In brief, we opted to use the 0° clamp in Experiment 2 as an alternative way to look at retention differences, thinking that retaining the clamp (but changing its size) might be a better way to reduce contextual effects from a change in the stimulus conditions. We followed the lead of Shmuelof and colleagues (2012) here.

After-effects: Following this, I find it odd how little the “after-effect” is analysed/discussed considering this was a major component of the analysis in the seminal work by Morehead et al., 2017, and is also analysed in Kim et al., 2018. The authors describe in the Materials and methods that the after-effect was defined by the first no-feedback cycle but then it is never further discussed.

We now point out in the Results and Materials and methods that the results here, in terms of the statistical comparison between groups, mirrors what was observed in the late learning analyses. We chose to focus on the late learning analysis in Experiment 1 since this allowed us to be consistent with the analyses performed in Experiments 2 and 3 where neither had no-feedback aftereffect trials. In addition, as noted in our response to the reviewer’s major comment, we now provide a more extensive analysis of the no-feedback trials.

Although it is difficult to see in Figure 1, the data provided by the authors reveals that performance is substantially different across groups during no feedback. Whereas, Miss and Straddle decline across trials, Hit maintains a plateau (by the end of No FB, groups are near the same level of performance). This plateau is strikingly different from the behaviour observed in Kim et al., 2018. This is also seen in Experiment 2, where the Straddle group decline across trials and the Hit group maintain a plateau (annoyingly I am unable to submit figures in my review, but this is clear in the data provided by the authors).

Please see our response to Summary Comment 4.

If the authors believe that reward is having a specific effect on learning, and not retention, how would a state-space model explain these results? During both these trials types, error has to be fixed to zero (there is no error to adjust behaviour with). Therefore, the only parameter which influences behaviour is the retention parameter. Thus, Straddle and Miss show the normal decay seen with no-vision/error clamp trials with a retention parameter of approx. 0.8-0.9. But the Hit group must have a retention parameter of 1. To me, this would suggest reward is having a direct effect on retention causing enhanced retention (relative to where they started at). Can the author's explain this behaviour in the context of the adaptation modulation model? In other words, can you model the plateau performance in no vision/zero clamp trials of the Hit group when reward only affects the learning parameter?

The revision provides a much more extensive discussion of the modeling work, including a presentation of the parameter estimates. These estimates show that, for the two viable models (Adaptation Modulation and Dual Error), there is a large difference in learning rates between the Hit and Straddle conditions. However, there is also a difference in retention rates (although there is considerable overlap in the confidence intervals). Thus, the modeling work does indicate that reward, or the lack of target error, may boost retention, in addition to the substantial effect on learning rate.

Interpretation of results: Throughout the paper there is an assumption (first stated ion paragraph five of the Introduction) that hitting the target is intrinsically rewarding, is it not an equally valid assumption that missing the target is intrinsically punishing? Or in fact both options may be true and have differing effects. How do we know what the “null” cerebellar behaviour is (this also relevant for the Morehead et al., and Kim et al., papers)? As the participants move from a baseline situation where they are being successful to a situation where they are being unsuccessful, couldn't the error clamp behaviour be regarded as intrinsically punishing? Therefore, across all these studies we could be observing a cerebellar process which is augmented by intrinsic punishment (aversive signals in the cerebellum are well known). Within this context, could the results not be viewed as punishment (moving from hitting the target to missing) increasing cerebellar learning (U) and reward increasing retention (A)?

Please see our response to Summary Comment 2.

This is relevant for paragraph six of the Discussion, modification of the A parameter alone would indeed lead to higher asymptote in opposition to the current results, however the retention findings in Experiments 1 and 2 suggest that A is affected by reward (see above). If both A is increased and U decreased (by a lack of intrinsic punishment) when hitting the target then lower asymptotes can be observed and would explain the findings (including the retention results which are not possible with a learning-only effect).

As noted above, the modeling work is substantially overhauled in the revision. As part of this revision, we have shifted the focus on what can be gleaned from the modeling work (e.g., that the Movement Reinforcement model is not viable). As the reviewer notes, in the two models we consider viable, the task outcome manipulation appears to have an effect on both the learning rate and retention parameters (albeit with a stronger effect on learning rate, based on bootstrapped parameter estimates).

Model comparison: Another issue I have is the use of the model in the transfer phase. Figure 5B clearly shows the predictions of the transfer phase for the straddle-hit group based on the two opposing theories. Considering this is the key result to the entire paper, surely the authors need to compare which model predicts the data better? To me the data seems to sit in the middle of these model predictions, rather than providing robust evidence for either. The authors could compare their initial adaptation modulation prediction as outlined in Figure 5F with a model that reflects what would happen with a movement reinforcement model (no change/flat line). They could use BIC or AIC model selection to determine which model explains the individual participant data (or the bootstrapped data) better. At the moment, there is no comparison between the opposing models. The authors have simply provided a prediction for the adaptation modulation model and said it approximately looks similar to the data (without even providing any form of model fit i.e. r-squared for transfer phase).

We now present three models that we see as qualitatively distinct to capture different ways in which task outcome might influence performance. We formalize our presentation of all three models, provide a new way to fit the models that analyzes all of the data simultaneously, and then report R-squared and AIC values as a way to evaluate and compare the models.

Similar conclusions to Leow et al., 2018: I am unsure what the rules are regarding a preprint but the conclusions for the current manuscript are very similar to Leow et al., 2018; implicit sensorimotor adaptation is reduced by task-based reward (modulating the amount of task error/reward). So how novel are these conclusions?

We certainly see the Leow et al. paper as relevant to the present work and discuss that paper at a few places in the manuscript. In some respects, the results of our study are consistent with the claims of Leow—we see this as a good thing, providing convergence. We do believe there are marked differences between the studies and our work allows us to reach some novel conclusions. We believe that our non-contingent feedback method, coupled with the explicit instructions about this manipulation, puts us in a strong position to argue that all of the learning effects observed here are implicit. Moreover, we provide a much more extensive computational analysis of the impact of task outcome and Experiment 3 provides a way to directly assess key predictions of these models.

No normality or homogeneity of variance tests: As ANOVAs are used, tests for normality (Shapiro-Wilk) and homogeneity are required (Levene). If these have been performed, then this needs to be made clear.

We now make explicit in the Materials and methods (Data Analysis) that we do test for both normality and homogeneity of variance. In cases where either assumption is violated, we supplemented the main analysis with non-parametric permutation tests and report the results of both the parametric and non-parametric tests. In terms of significance, the statistical outcome results were consistent in all of these cases.

Reaction and movement time across experiments: the authors state in the Materials and methods that there was a 300ms movement time cutoff. Did this include reaction time? Did the authors examine reaction time during adaptation and no-vision/0 error clamp blocks, rather than at baseline? This analysis should be included (i.e. reaction and movement time for the trials used in Figure 1E, F and similar for Experiment 2).

We now state that the 300 ms criterion applies only to movement time and does not include RT (L954). We have added tables to the Appendix that present the RT and MT data for four different phases of the experiment (baseline, early adaptation, late learning, no feedback/0° clamp).

Early adaptation rate: I take issue with calling this a measurement of “rate” (subsection “Data Analysis”)? This value is a measurement of average early performance across cycles, not the change in performance which would be measured by calculating the difference between cycles. Or is this what the authors did and the methods are incorrect?

We have edited the Materials and methods to make clear that this measure represents the mean change in hand angle per cycle over the first five cycles. To provide a more stable estimate of hand angle at cycle 5, we averaged over cycles 3-7 of the clamp block.

Experiment 1 post hoc t-tests: Are these corrected for multiple comparisons (p=0.16). Although both would survive, this is simply good practise.

These are corrected for multiple comparisons. We report the corrected α value (.0167) in the Materials and methods.

Experiment 3 statistics: The order of statistical analysis is odd. Why not start with the omnibus ANOVA and then describe the comparisons between the 3 groups? In addition, I do not like the use of planned comparisons unless these are pre-registered somewhere. I am guessing the t-test between the control group and straddle-to-hit does not survive a correction for multiple comparisons? However, as this is a control experiment, planned comparisons are begrudgingly acceptable.

We have revised the order of the statistical analyses, following the reviewer’s guidelines here. We do think the planned comparisons are appropriate here, given the specific hypothesis we were testing in this control group. Nonetheless, as the reviewer suggests, the p-values for comparisons between the control group and the Straddle-to-Hit group are just above the corrected p-value for multiple comparisons of.0167 (asymptote p-value was.017). However, if we had not used the planned comparisons, we would have performed post-hoc Scheffe contrasts, as this contrast most accurately reflects our hypothesis that the control group should behave similar to the Hit-to-Straddle group in the first phase, but different than Straddle-to-Hit. In this case, the contrast would have been significant.

Wrong group described (Discussion paragraph three): The straddle group is similar to the Miss group.

This has been corrected.

Reviewer #3:

Implicit recalibration is a specific learning process that is known to be implicit, cerebellum-dependent, and driven by sensory prediction errors. Previous work has suggested that this process is insensitive to reward. The core claim in this paper is that, in fact, implicit recalibration can be modulated by reward in that failure to earn a reward amplifies sensitivity to sensory prediction error. If true, this is certainly an interesting and important finding. I do not believe, however, that the experiments presented here are capable of supporting this conclusion.

The experiments employ a clever “error clamp” paradigm which the authors have previously shown induces implicit adaptation of reaching movements. The data in the current paper demonstrate clearly that the extent of adaptation under this clamp depends on the size of the target. If the target is big enough that the cursor lands inside it, less is learned from the error. I think it is fair to conclude from these results that there is an implicit learning process at play which is sensitive to reward. My major concern, however, is with the attempt to distinguish between the "Adaptation modulation" theory and the "Movement reinforcement" theory, which really is the heart of the paper.

We have overhauled the modeling work in this paper, with this part of the revision summarized in our response to Summary Comment 1.

The authors make the very strong assumption that their clamp manipulation isolates a single learning process – implicit recalibration driven by SPE. The results state quite clearly; "This [clamp] method allows us to isolate implicit learning from an invariant SPE, eliminating potential contributions that might be used to reduce task performance error." I agree that the clamp isolates implicit learning, but couldn't there be an implicit contribution that is not SPE-driven but is instead driven by reward/task error? How can the authors be certain that the participants were not engaging in some kind of re-aiming that may have been implicit or involuntary?

The reviewer brings up an excellent point, namely that we cannot be sure that the clamp manipulation isolates a single learning process, implicit recalibration driven by SPE. The Movement Reinforcement model did postulate a second implicit learning process, namely a model-free bias reflecting reward history. But we had not considered that target error might also drive implicit recalibration. This led to the development of the Dual Error model (the idea of which was initially presented to us by the reviewer), a two-process model along the lines of that introduced by Smith et al., 2006, but in the present context, one in which the two state estimates are sensitive to different error signals rather than different time scales. Further explanation of changes related to this comment can be found in our response to Summary Comment 1, including our thoughts on why we are hesitant to psychologically characterize the target-error sensitive process as one of implicit aiming.

The authors do seem to entertain the possibility of distinct SPE-based and reward-based processes, alluding in the Abstract to a model "in which reward and adaptation systems operate in parallel." Figure 4 even quite clearly illustrates such a parallel architecture. But by later in the paper the myriad possible alternatives to the "Adaptation Modulation" model are collapsed onto a single idea in which reward reinforces particular movements, causing them to act as an “attractor”. Ultimately, the conclusion rests on rejecting this “attractor” / "Movement Reinforcement" model because it incorrectly predicts that asymptote will be maintained in Experiment 3 when the target becomes larger. The Adaptation Modulation model is the only model left and so must be correct.

Of course the “reinforcement” model represents only one very specific alternative proposal about how reward might influence learning under the clamp (other than by modulating SPE-driven recalibration). Rejecting this one particular model is not grounds to reject EVERY possible architecture in which reward does not act by modulating SPE-driven adaptation. For this logic to work, it would have to be the case that ANY model in which reward does not modulate SPE-driven learning would necessarily predict a consistent asymptote after the straddle->hit transition.

We agree with the reviewer—we were guilty in the original submission of advocating for the Adaptation Modulation model based on the shortcomings of the Movement Reinforcement model. As noted above, we now introduce the Dual Error model as another account for how task outcome information might influence implicit learning (and this model ends up providing the best account of the results). While we recognize that these three models are not exhaustive, we do see them as representative of qualitatively different conceptualizations (e.g., model-free vs model-based; single vs multiple state estimates; independent vs interactive processes) and expect the expanded formalizations will help make this point. For example, the formalizations of the Movement Reinforcement model should make clear why any model in entailing independent operation of model-based adaptation (insensitive to task outcome) and model-free reinforcement learning processes will not be able to account for the Miss (Straddle) -> Hit conditions.

It is not too difficult to formulate a model in which reward does NOT modulate SPE-driven learning, but which doesn't predict a consistent asymptote after the straddle->hit transition. Suppose behavior in the clamp is the sum of two distinct processes acting in parallel, one feeding on SPE, and one feeding on task error, both with SSM-like learning curves. The latter processes might correspond to a kind of implicit re-aiming and might plausibly only be updated when a movement fails to earn reward. Then disengaging this system by suddenly providing reward (at the transition from straddle to hit) might lead to a gradual decline in that component of compensation. This would lead to behavior quite similar to what the authors observe. I have appended MATLAB code at the end this review which implements a version of this model.

We thank the reviewer for suggesting this model and providing the code to reinforce the idea. In the revision, we present this as the Dual Error model, with SPE and TE being used to update two independent state estimates. We address the interpretation of this process, and in particular, whether it constitutes a form of implicit aiming in the Discussion. We also outline some future experiments for further evaluating this model.

Finally, I would add that I'm not even convinced that it is reasonable to assume that the "Movement Reinforcement" “attractor” model, in which the authors suggest that reinforcing movement attenuates decay, would necessarily lead to a persistent asymptote. What if the reinforcement only partially attenuates decay, or requires repetition to do so (during which time it may decay slightly)?

We have revised our presentation of the Movement Reinforcement model, and the formalization makes clear that the independent operation of model-based adaptation and model-free reinforcement learning processes will predict a persistent asymptote in the Straddle-to-Hit condition. We recognize that a key assumption here is that the parameters in the model-based process remain the same in the Hit and Straddle conditions (which seems justified in a model in which the two processes are assumed to be independent). As such, the only change that comes about at transfer is that which arises from the model-free reinforcement learning process. In the Straddle-to-Hit condition, adaptation will stay at asymptote since the SPE does not change, and reinforcement will only reward movements about the asymptotic hand angle. There is no process that would result in a decay of hand angle. The rate at which the relative weighting between adaptation and reinforcement changes will not affect the hand angle in this condition (but will in the Hit-to-Straddle condition).

For these reasons, I don't believe that the primary conclusion is tenable.

Putting aside the question of adaptation modulation, I do like the experiments in this paper and I think phenomena that have been shown are interesting. I think the results convincingly establish the existence of an implicit learning process that is sensitive to reward, and the paradigm provides a platform for exploring the properties of this process. I would suggest the authors try to develop the manuscript along these lines, though it is not particularly satisfying if the results cannot distinguish between modulation of SPE-driven learning by reward and a parallel, reward-driven process as I feel is the case at present.

As noted in our response to the previous comment, we do think the results can rule out the Movement Reinforcement model. But we do not think they allow us to distinguish between the Adaptation Modulation and Dual Error models. We certainly discuss that the Dual Error model outperforms the Adaptation Modulation model, but give consideration to both in the Discussion as well as describe future work that can provide stronger comparisons between the two models. We do think there is considerable value in the paper, providing new insight into the impact of task outcome on implicit learning in sensorimotor adaptation tasks.

As a constructive suggestion, it may be enlightening to examine behavior in the 0deg clamp and no-feedback conditions more closely. The authors draw no attention to it in the paper (nor really explain why these blocks were included) but the data seem to suggest a difference in rate of decay between the “hit” and “miss” conditions in Experiments 1 and 2. Different theories about the nature of the implicit reward-based learning would predict quite different behavior in these conditions, but it's impossible to judge from such short periods whether the decay curves might converge, stay separate, or even cross over. I wonder if an additional experiment exploring this phenomenology in more detail, with a decay block that is longer than 5-10 cycles, might be enlightening, though I admit I'm unsure if this would necessarily help to disambiguate the SPE-modulation model from the alternative “parallel” model. Perhaps comparing a NoFB condition to a 0-deg clamp condition might be useful. A more concrete (and exhaustive) set of theories/models would help to formulate clearer predictions about what to expect here.

We have added analyses of the no-feedback condition (Experiment 1) and 0 deg clamp (Experiment 2) to look at the retention issue (and supplement the modeling analyses). Please see our response to Summary Comment 4. An experiment in which we extend the washout period seems like a way to test retention directly, but is problematic when the level of learning differs between groups prior to the washout period (see comments above on absolute vs relative changes). Moreover, given that, even with a long washout period, hand angle does not return to zero (Brennan and Smith, MLMC, 2015). This would make it difficult to compare forgetting rates with a one-parameter model (i.e., A term in state space model). This latter problem is not as acute in the initial washout cycles, although the absolute vs relative issue still holds.

Minor Comments:

Results third paragraph: “angle” or “deviation” may be a better word here than “offset” which could easily be confused with switching off the clamp.

We have made the recommended changes.

The "models" are never formulated in concrete terms. I would be fine with a verbal argument as to why the Movement Reinforcement theory should predict a consistent asymptote in Experiment 3, but it feels a stretch to call them “models” when they are never concretely fleshed out. Fleshing out the models mathematically may also help to bring to the surface some of the tacit assumptions that have led to the headline conclusion of the paper.

The revision includes formalizations of all three models, including the Movement Reinforcement model. We do believe this sharpens the presentation of the ideas, and allows us to objectively compare the models.

The "model" that generates the "model prediction" in Figure 5f is rather ad-hoc. It’s odd to suggest that the parameters of the underlying process would instantly flip. Also why are the predictions discontinuous across the dashed “transfer” line?

This comment, coupled with our overhaul of the modeling section, led us to take a different tack in modeling the data. In the initial submission, we fit the first half of the data and used the parameter estimates to predict transfer performance, evaluating this against the group-averaged data from participants in the other group. We now simultaneously fit the data from both halves of the experiment for both groups (again using group-averaged data). This was essential to get stable parameter estimates to generate the best-fitting function for the Movement Reinforcement model since, with the Straddle-to-Hit group, any values for the reinforcement parameters during the Hit phase will predict a persistent asymptote.

Results subsection “Experiment 3”: "the predictions of the model slightly underestimated the observed rates of change for both groups". Do you mean OVERestimated?

The reviewer is correct—we mean the model overestimated the observed rate of change (we were thinking about the behavior in the transfer phase falling short of the predicted rate of change). However, given our new strategy to use the acquisition and transfer data in fitting the models, this point is now moot.

[Editors' note: further revisions were suggested, as described below.]

All three reviewers were fully satisfied with the amendments made to the manuscript, and agreed the manuscript was much improved. Reviewers #2 and 3 provided some valuable follow-up suggestions, which I expect you would be happy to consider. I'm pasting these below for your benefit.

Please incorporate your changes in a final version of your manuscript (no need for a detailed rebuttal).

We are very pleased that the reviewers were satisfied with the revised manuscript. Their input has certainly strengthened the paper. We also appreciate the follow-up suggestions and have addressed them in this new revision. Please see our notes below for how we have addressed each comment.

Reviewer #1:

I find the revised manuscript much improved. The analysis and discussion are more comprehensive and provide a more nuanced, and as such more transparent and balanced, account of the very interesting behavioural effects observed in the study. I'm also satisfied with the responses provided to the reviewers comments. On a personal level, I found the manuscript easier to read (though this might be due to a repetition effect!). I have no further comments.

Thank you! We do hope the improved clarity of the manuscript is not a repetition effect, but reflects the changes made over the course of revision.

Reviewer #2:

The authors have done a great job at addressing the issues highlighted by all reviewers, and the manuscript is much improved. Although the overall story is less eye-catching than “reward modulates SPE learning”, I believe it is now a far more informative piece of work which is still novel. I have a few fairly small outstanding issues:

Collinearity of parameters: Could the authors test the collinearity of the parameters in each model? In other words, is the data rich enough to have 4/6 independent parameters (I know this is an issue when using the 2-state Smith model with certain paradigms)? For example, when you use the bootstrapping procedure how correlated are the parameters? A heat map maybe for each model would be a good way to show this. This will ensure the results from paragraph three of subsection “Experiment 3 – Modeling Results” are valid. Also parameter recovery (Palminteri et al., Plos CB, 2017) might be of interest.

We appreciate the reviewer’s comments. We now include a supplemental figure (Figure 6—figure supplement 1) with heat maps of the parameters. As can be seen in this new figure, there is a trade-off between the ‘A’ and ‘U’ terms, something that is frequently observed with these models when the perturbation is relatively simple. We address this issue in the third paragraph. Note that the tests of collinearity are only performed for the models in which we used a bootstrapping approach to estimate the parameter values.

Control group figure: I would add Figure 5—figure supplement 1 to the main document.

We have made this change. The control group data are now shown in Figure 7 of the main text.

Discussion (paragraph nine of subsection “Modeling the Influence of Task Outcome on Implicit Changes in Performance”): Aren't the target jump predictions similar to what Leow et al.,. 2018, have already shown to be true?

The Leow study is certainly relevant to our predictions here. However, we propose to use the target jump method to manipulate TE in the absence of SPE (by using a 0^o clamp). This is what leads to the differential predictions of the Dual Error and Adaptation Modulation models. We have revised the text to clarify this issue and how our proposed experiment differs from the Leow study.

Reviewer #3:

The authors have revised the paper very thoroughly. For the most part, I find that the paper hits the mark well in terms of articulating theories that can versus can't be disambiguated based on the current experiments. I do, however, have a few further suggestions to help clarify the paper further.

Use of the term “target error” is a bit ambiguous. At times it is characterized as a binary hit/miss signal (e.g. Introduction paragraph five; Discussion paragraph five). At other times it seems to refer to a vector error (i.e. having magnitude and direction) (e.g. Introduction paragraph six distinguishes it from “hitting the target”; penultimate paragraph of subsection “Experiment 2”). This is liable to cause considerable confusion. It confused me until reading the description of the dual adaptation model, in practical terms it is tantamount to a binary signal, given the constant error size and direction. But I think a little more conceptual clarity is needed. e.g. does TE already take reward into account? Or are TE and SPE equivalent (in this task) but learning from TE is subject to modulation by reward, while learning from SPE is not?

We now make clear early on that we operationalize TE as a binary signal. These changes are in the fifth paragraph and reinforced in the final paragraph of the Introduction section.

The term “model-based” is used throughout, sometimes (e.g. paragraph four of subsection “Modeling the Influence of Task Outcome on Implicit Changes in Performance”) in the sense of a model-based analysis of the effects of reward, i.e. examining how presence/absence of reward affects fitted parameters of a model (this usage is fine). At other times (e.g. Conclusions section) it's used essentially as a synonym for error-based. I'm a bit skeptical as to whether this latter usage adds much value or clarity. I appreciate it derives from Huang et al. and Haith and Krakauer, who made a case that implicit adaptation reflected updating an internal forward model. However, error-driven learning need not necessarily be model-based. For instance, if learning is driven by target error, it's not clear that this has anything to do with updating any kind of model of the environment (and "inverse model" doesn't really count as a proper model in this sense). So I would caution against using the term “model-based” when the idea doesn't inherently involve a forward model. Particularly given the two distinct usages of “model-based” in this paper.

We agree with the reviewer that the multiple senses of “model-based” can be confusing. In the revision, we now limit our use of the term model-based to places in which the term is used to refer to learning processes driven by SPE. We do not use this term when referring to learning processes driven/modulated by TE.

The Movement Reinforcement model is reasonable, although it is fairly ad hoc. The Discussion argues convincingly that this model is largely illustrative, and that its behavior can be taken as representative of a broad class of “operant reinforcement” models. I think articulating something to this effect earlier, when the model is first introduced, would be helpful. At the moment, it comes from nowhere and it's a little perplexing to follow exactly why this is the model and not, say, a more principled value-function-approximation approach. With that in mind, some of the finer details of the model might be better placed in the Materials and methods section so as not to bog down the results with specifics that aren't all that pertinent to the overall argument.

Based on the reviewer’s suggestions, the discussion of the MR model has been revised. We have simplified our motivation for the model in the section where we introduce the different models, and as the reviewer has suggested, we make clear early on that this model’s behavior is representative of a broad class of operant reinforcement models. We have also moved some of the details into the Materials and methods section.

Minor Comments:

I appreciate the discussion of implicit TE-driven learning in motivating the Dual Error model (subsection “Theoretical analysis of the effect of task outcome on implicit learning”). But I was surprised the authors didn't mention this again in the discussion, instead only speculating that TE-based learning might be re-aiming that has become implicit through automatization/caching, and consequent making the dual error model seem implausible. But it seems perfectly plausible that TE-based learning is just another implicit, error-based learning system, separate from SPE-driven implicit learning, that never has anything to do with re-aiming.

We have modified the Discussion, now having a line to make clear that it remains possible that there may be a TE- sensitive implicit learning process, distinct from SPE-driven adaptation. This point is also reinforced in our discussion of future experiments that might distinguish between the Dual Error and Adaptation Modulation models.

Subsection “Modeling the Influence of Task Outcome on Implicit Changes in Performance”: it doesn't seem necessary to invoke “SPE-driven” here. Could in principle be error-based learning driven by something like "target error" (i.e. just the distance between the center of the cursor and the center of the target). Ditto in the Conclusion section.

We have modified the text in both places in recognition of this point.

Introduction section: "We recently introduced a new method.… designed to isolate learning from implicit adaptation" slightly ambiguous sentence, I first read it as though learning and implicit adaptation are separate things being dissociated. Maybe just drop "learning from"?

We have made this change.

Introduction section: "Given that participants have no control over the feedback cursor, the effect of this task outcome would presumably operate in an implicit, automatic manner." It's not having no control that makes it implicit… Might be better rephrased to something like "Given that participants are aware that they have no control over the feedback cursor…"?

We have made this change.

Second paragraph of subsection “Theoretical analysis of the effect of task outcome on implicit learning”: this paragraph misses a key detail, that “reinforcing” or “strengthening the representation of” rewarded actions really means that it makes those actions more likely to be selected in the future.

We have now added: “…and increase the likelihood that future movements will be biased in a similar direction.”

Third paragraph of subsection “Theoretical analysis of the effect of task outcome on implicit learning”: “composite” is somewhat vague. Would “'sum”' or “'average”' be accurate?

As reviewer 3 has suggested, we have changed the wording to “sum”.

Third paragraph of subsection “Experiment 3 – Modeling Results”: something is up with the brackets here.

We have now fixed things.

Author details

1. Hyosub E Kim

   1. Department of Psychology, University of California, Berkeley, Berkeley, United States
   2. Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, United States
   3. Department of Physical Therapy, University of Delaware, Newark, United States
   4. Department of Psychological and Brain Sciences, University of Delaware, Newark, United States

   Contribution

   Conceptualization, Data curation, Software, Formal analysis, Visualization, Writing—original draft, Writing—review and editing

   For correspondence

   hyosub@udel.edu

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0003-0109-593X

2. Darius E Parvin

   1. Department of Psychology, University of California, Berkeley, Berkeley, United States
   2. Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, United States

   Contribution

   Conceptualization, Formal analysis, Writing—review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0001-5278-2970

3. Richard B Ivry

   1. Department of Psychology, University of California, Berkeley, Berkeley, United States
   2. Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, United States

   Contribution

   Conceptualization, Resources, Formal analysis, Supervision, Funding acquisition, Writing—review and editing

   Competing interests

   Senior editor, eLife

   "This ORCID iD identifies the author of this article:" 0000-0003-4728-5130

• 3,872

  Page views

• 424

  Downloads

• 44

  Citations

Article citation count generated by polling the highest count across the following sources: Scopus, Crossref, PubMed Central.