Dopamine promotes instrumental motivation, but reduces reward-related vigour
Abstract
We can be motivated when reward depends on performance, or merely by the prospect of a guaranteed reward. Performance-dependent (contingent) reward is instrumental, relying on an internal action-outcome model, whereas motivation by guaranteed reward may minimise opportunity cost in reward-rich environments. Competing theories propose that each type of motivation should be dependent on dopaminergic activity. We contrasted these two types of motivation with a rewarded saccade task, in patients with Parkinson’s disease (PD). When PD patients were ON dopamine, they had greater response vigour (peak saccadic velocity residuals) for contingent rewards, whereas when PD patients were OFF medication, they had greater vigour for guaranteed rewards. These results support the view that reward expectation and contingency drive distinct motivational processes, and can be dissociated by manipulating dopaminergic activity. We posit that dopamine promotes goal-directed motivation, but dampens reward-driven vigour, contradictory to the prediction that increased tonic dopamine amplifies reward expectation.
Introduction
Organisms expend more effort when their actions can lead to rewards, as the value of the reward offsets the extra effort expended to attain them (Kool and Botvinick, 2018; Manohar et al., 2015; Niv et al., 2006; Shenhav et al., 2017). They will even do so if the extra effort does not increase the reward they receive (Glaser et al., 2016; Milstein and Dorris, 2007; Xu-Wilson et al., 2009), indicating that mere expectation of reward is enough to justify the effort cost. Motivation, which promotes this effort expenditure, has two facets: it allows actions to be directed towards goals, and it energises our actions when rewards are expected (Niv et al., 2006). These two aspects are not always coupled. For example, employees might be salaried, where a fixed reward is guaranteed irrespective of achievements, or they might receive merit-based pay that is contingent on meeting performance targets (Lazear, 2000).
Contingent rewards motivate us because we understand the causal relation between successful actions and reward. This is instrumental, in that we apply knowledge of action-outcome associations. For instance, people must realise that merit-based pay depends on their performance for it to incentivise them. In animals, dopaminergic input to dorsal striatum is necessary for instrumental motivation (Lex and Hauber, 2010b).
In contrast, reward that is independent of what an agent does might motivate us because in a variable environment, we capitalise on rewards while they are available (Niv et al., 2007). One proposed mechanism for this is that tonic dopamine encodes expected reward rate, such that in a rich environment agents are motivated to respond faster to maximise the rewards they receive (Niv et al., 2007). Equally, dopamine can be viewed as signalling an opportunity cost– time is more costly when reward is available, and so organisms act faster (Otto and Daw, 2019; Shadmehr et al., 2010). The dopaminergic drive has not only generalised motivating effects, termed vigour (Beierholm et al., 2013; Guitart-Masip et al., 2011; Niv et al., 2007), but also context-specific effects. For example, a stimulus that predicts rewards drives conditioned responses that are uncoupled with reward (Lovibond, 1981) – similar to how salary increases might improve job performance. This phenomenon, known as Pavlovian-to-Instrumental transfer, requires dopamine projections to nucleus accumbens (Hall et al., 2001; Kelley and Delfs, 1991; Talmi et al., 2008; Wassum et al., 2013; Wyvell and Berridge, 2000). Similarly, animals tend to approach stimuli associated with rewards, even in the absence of action-contingency, a behaviour called autoshaping or sign-tracking, which also relies on nucleus accumbens dopamine (Day et al., 2006; Di Ciano et al., 2001).
The dopaminergic basis of instrumental and Pavlovian motivation could potentially explain the impaired motivation seen in PD patients and the rescue of such deficits by rewards (Ang et al., 2018; Chong et al., 2015; de Wit et al., 2011; Kojovic et al., 2014). However in certain situations, motivation by reward can paradoxically be stronger in patients with low dopamine (Aarts et al., 2012; Timmer et al., 2018), making dopamine’s exact role in motivation unclear.
These two effects of contingent and expected rewards frequently overlap in real life and in previous experiments – higher stakes raise reward expectation, but also mean that actions carry more weight. However, experimental control of expectation and contingency allows them to be dissociated (Manohar et al., 2017), which reveals that both contingency and expectation can separately motivate behaviour, and that these effects are independent rather than correlated or antagonistic, suggesting distinct mechanisms.
We used this incentivised saccade task (Manohar et al., 2017) here to test PD patients ON and OFF their dopaminergic medication, along with healthy age-matched controls. We tested the two predictions that dopamine is involved in motivation by expected rewards, and by contingent rewards.
Results
Dopamine promotes contingent motivation and attenuates reward-expectation motivation
Participants made saccades to a target after hearing cues indicating how reward would be determined (Figure 1b). To measure motivation by contingent rewards, we compared trials where rewards were delivered depending on participants’ response times (Performance), to trials where rewards were given with 50% probability (Random). We matched the average reward rate so that both these conditions had identical reward expectation and uncertainty, and only differed in their contingency. To measure motivation by reward expectation, we compared trials with a guaranteed reward (10 p) to those with a guaranteed no-reward (0 p). In both these conditions rewards were delivered unconditionally, and only differed in terms of expected reward. We tested 26 PD patients ON and OFF dopaminergic medication (PD ON and PD OFF) and 29 healthy age-matched controls (HC) on a rewarded eye-movement task that separated effects of contingent and non-contingent motivation (see Figure 1a for task, see Table 1 for participant details).In all trials, feedback was given about whether the response was fast or slow, in addition to the reward received, to control for intrinsic motivation. A saccade’s velocity is tightly governed by its amplitude, a relation known as the ‘main sequence’ (Bahill et al., 1975). To account for this, we regressed out the effect of amplitude on peak velocity leaving peak saccade velocity residuals as our main measure of response vigour (see Figure 1e), as in previous work (Blundell et al., 2018; Manohar et al., 2017; Muhammed et al., 2020; Muhammed et al., 2016; Van Opstal et al., 1990). This measures how much faster each saccade is than the speed predicted from its amplitude. Thus, positive (negative) residuals mean a particular saccade was faster (slower) than predicted by the main sequence, and makes response vigour independent of any changes to saccade amplitude also caused by our manipulations or by group differences between PD patients and HC. We did this regression for each participant and session separately, but across conditions. A three-way repeated-measures ANOVA tested whether dopamine differentially affected contingent and guaranteed motivation – manifest by a three-way (contingency*motivation*drug) interaction.
Dopaminergic medication significantly modulated how contingent and guaranteed motivation affected motor vigour (Figure 2a, three-way interaction on peak velocity residuals, p=0.0023; see Table 2 for statistics). This was because, when ON medication, patients were motivated by contingency but not reward expectation (separate two-way ANOVA in PD ON: contingency*motivation, p=0.0170; see Supplementary file 1A), whereas after overnight withdrawal of medication there was a borderline significant interaction in the opposite direction, as PD OFF were motivated by reward expectation but not contingency (PD OFF ANOVA: p=0.0501; Supplementary file 1A). This indicates that when PD patients were ON medication, motivation was strongest when reward was contingent on performance, but when they were OFF medication, patients were motivated by guaranteed rewards.
To confirm that the effects on peak velocity residuals were not driven by changes in other aspects of saccades, the same 3-way ANOVA was run on each of the other saccade measures. There were no significant effects on saccadic amplitude (see Table 2 and Figure 2c). Saccadic RT had an effect of contingency as saccades started faster for Performance and Random conditions than 10 p or 0 p conditions (Figure 2d, p=0.0396). Endpoint variability had a contingency*motivation interaction (Figure 2e, p=0.0482) as variability was higher for 0 p condition. Raw peak velocity had an effect of motivation, as both types of motivation increased speed (Figure 2f, p=0.0110), although this will include effects of changes in amplitude (via the main sequence) which showed a borderline significant effect of motivation (Figure 2c, p=0.0607).
The HC peak velocity residuals were not affected by contingency, motivation or the interaction (p>0.05; see Table 3), suggesting that healthy older adults do not adjust their response vigour for contingent or guaranteed rewards. There were also no significant effects on amplitude, saccadic RT, or raw peak velocity in HC, although endpoint variability did have a significant contingency*motivation interaction (p=0.0048; see Table 3). Post-hoc pairwise comparisons showed this was due to guaranteed rewards significantly reducing variability (p=0.0316), while contingent rewards did not (p=0.1219).
We also compared both PD ON and OFF separately against the HC with three-way mixed ANOVA, to see under which conditions patients deviated from healthy behaviour. As expected, HC had overall larger amplitudes, quicker saccadic RTs and lower endpoint variability than both PD ON or OFF (Figure 2, see Supplementary file 1B-C for statistics). The use of peak velocity residuals rather than raw velocity factors out the effects of PD on movement amplitude, allowing comparison of the motivational changes in velocity while controlling for differences in the main sequence (Bahill et al., 1975; Manohar et al., 2017). HC did not significantly differ from PD ON or OFF in peak velocity residuals, although their pattern was numerically closest to PD ON with greater contingent motivation.
We additionally checked whether there were practice effects in the PD patients, in case patients behaved differently on their second session due to different expectations. We found no effects or interactions of session on any measure in PD patients (p>0.05).
Velocity profiles
The effects above demonstrate peak velocity shows strong effects of reward and dopamine, so next we examined the time-course of how velocity was modulated during a saccade. We computed the velocity across time within the movements, and compared the reward effects for PD ON and OFF using cluster-wise permutation tests. Contingent rewards (Performance – Random) did not significantly affect velocity or acceleration for PD ON or OFF, as permutation tests for each condition and the difference between conditions found no significant clusters (cluster-wise permutation tests: p>0.05; Figure 3a&b). However, guaranteed rewards (10 p – 0 p) lead to greater velocity early in the saccade for PD OFF (p<0.05; Figure 3c), which was significantly different from PD ON (p<0.05). Acceleration traces showed this was due to PD OFF having greater acceleration early in the movement (Figure 3d, p<0.05). HC showed no effects of contingent or guaranteed rewards on velocity or acceleration, perhaps unsurprising as there were no differences in overall velocity as reported above. Permutation testing revealed no differences between HC and PD ON or OFF for velocity or acceleration (p>0.05).
Faster movements are known to be more error-prone (Harris and Wolpert, 1998; Harris and Wolpert, 2006), but motivation can attenuate this effect, making movements more accurate (Manohar et al., 2019). Autocorrelation of eye position over time within saccades provides an indicator of corrective motor signals during movements: noise accumulates during movements, so that variability early in a movement causes endpoint error. This is manifest in autocorrelation, where across trials the eye position at early time-points predicts late time-points. Negative feedback signals correct movement errors during the saccade, and manifest as reductions in this autocorrelation (Codol et al., 2020; Manohar et al., 2019). This feedback, provided by corrective motor signals, can be increased by incentives (Codol et al., 2019; Manohar et al., 2019). In the current study, guaranteed rewards led to greater autocorrelation early in the saccades for PD OFF than ON (Figure 4e & g). This coincides with the greater acceleration PD OFF patients had at the beginning of saccades to guaranteed rewards (Figure 3d), as faster movements have greater motor noise (Harris and Wolpert, 1998; Harris and Wolpert, 2006). Notably, this reward-related autocorrelation did not persist until the end of the saccade, suggesting that negative feedback corrected it. However, as we did not find decreased autocorrelation around the end of the saccades, this represents only indirect evidence of negative feedback.
No correlation of the velocity effects for the distinct motivational processes
Previous work had shown that motivation by contingent and guaranteed reward did not correlate across participants (Manohar et al., 2017), so we asked whether dopamine’s effects upon these two types of motivation was also uncorrelated. We found no correlation between effects of contingent and guaranteed rewards on peak saccade velocity residuals in PD ON, PD OFF or HC separately, nor a correlation between medication states, nor between the drug-induced changes in the effects (p>0.05; see Figure 5 legend for statistics). This suggests that the two effects are separate and independent, and not antagonistic within the same person. In particular, the degree to which dopamine improved performance-contingent motivation did not predict the degree to which it reduced motivation by guaranteed rewards.
Source data are available in Figure 5—source data 1.
Pupil dilatation
We examined pupil dilatation after the cue onset and before the target appeared (after 1400 ms). Previous research has shown a greater effect of contingent than guaranteed reward on pupil dilatation, maximal around 1200 ms after the cue (Manohar et al., 2017), so we used a window-of-interest analysis on the mean pupil dilatation 1000–1400 ms after the cue. There were no significant effects or interactions (p>0.05; Figure 6, see Supplementary file 2A-C for statistics), suggesting that dopamine and reward did not affect pupil responses in PD patients.
We also used a hypothesis-free analysis, using cluster-wise permutation testing across the whole time-course to look for significant differences between conditions and groups, which also found no significant effects (p>0.05).
We found no correlations between pupil dilatation and motivation effects in any group, or overall (p>0.05; Figure 6—figure supplement 1). Thus, the vigour effects were not related to pupillary dilatation before the movement.
PD severity
We looked to see whether the dopaminergic effects on velocity residuals could be tied to PD symptom expression. The UPDRS (Martínez-Martín et al., 2015) is a measure of PD symptom severity and was performed in each session; part III measures motor symptom severity. We found no correlations between UPDRS-III scores and reward effects on peak velocity residuals in PD ON (Guaranteed: ρ = −0.1256, p=0.5410; Contingent: ρ = −0.2327, p=0.2527) or OFF (Guaranteed: ρ = −0.2067, p=0.3110; Contingent: ρ = 0.1553, p=0.4487). Thus, the reward effects were unrelated to PD symptom severity.
Depression and apathy
We gave participants questionnaires measuring apathy, the AMI (Ang et al., 2017) and depression, BDI-II and HADS (Beck et al., 1996; Zigmond and Snaith, 1983). We found no significant correlations between these questionnaires and contingent or guaranteed motivational effects on peak velocity residuals in PD ON or OFF (p>0.05, see Supplementary file 4 for statistics).
Fixation period
We looked at whether motivation was affecting behaviour during the fixation period (1400 ms between condition cue onset and target onset) differently, which could potentially lead to differences during the movements. We excluded trials with saccades, blinks, deviations greater than 1.8° and segments with velocities greater than 30°s−1.
PD OFF had more microsaccades (<1°) during the 1400ms fixation period than PD ON (F (1, 201) = 5.0451, p = .0258, = . 0245), but there were no other effects or interactions (p >.05, Supplementary File 3A for statistics). Conversely, ocular drift speed was higher in PD ON than OFF (F (1, 216) = 5.4327, p = .0207, = .0245), but there were no other significant effects or interactions (p >.05, see Supplementary File 3B). Importantly, the lack of interactions means that while patients may have differed in their fixation activity, this was unaffected by motivation conditions, and thus a different pattern to the main effects shown above.
To quantify ocular tremor, we performed Fourier transforms on the eye position in the early (200–700 ms) and late (700–1200 ms) fixation periods, and compared these between conditions with cluster-wise permutation tests to look for clusters of frequencies where patients differed. We found no significant clusters (p>0.05).
Discussion
In this study, we tested two competing theories of dopaminergic motivation – that dopamine improves instrumental, contingent motivation, and that dopamine improves guaranteed reward motivation via reward expectation. Patients with PD made more vigorous responses, measured by peak saccade velocity residuals (Figure 2a), when rewards were either contingent on performance or guaranteed, but these two effects were differentially affected by dopaminergic medication. When ON medication, PD patients were motivated by rewards contingent on performance, but not by guaranteed rewards. In contrast, when patients were OFF their dopaminergic medication, the opposite pattern was observed; they were motivated by guaranteed rewards, but not by rewards contingent on performance. In this study, older healthy controls were not significantly invigorated by either guaranteed or contingent rewards, although they showed a numerically similar pattern to PD ON. Guaranteed rewards led to PD OFF having earlier increases in velocity and acceleration (Figure 3c & d), which was not seen in PD ON or when rewards were contingent, and this was accompanied by increased autocorrelation of eye position (Figure 4), suggesting increased motor noise early in the saccade. The two motivational effects were uncorrelated across people and between medication states (Figure 5) indicating that dopamine does not promote one type of motivation over another in a competitive fashion, and were not associated with changes in pupil dilatation (Figure 6). Rather, reward expectation and contingency provide distinct motivational drives (Figure 7), which can be dissociated by dopaminergic medication.
The results suggest that dopamine is necessary for contingent motivation. Contingent motivation requires the use of stimulus-action-outcome associations for goal-directed behaviour (Daw and Dayan, 2014; Dickinson, 1985), while reward expectation can occur via stimulus-outcome associations (Niv et al., 2007) that do not require understanding the causal role of action. Our results align with rodent work demonstrating that dorsomedial striatum dopaminergic lesions impair action-outcome associations, such that animals continue to respond to previously rewarding cues even when action-contingency is removed (Lex and Hauber, 2010b). At a more general level, our result is also consistent with dopamine being necessary for behaviours involving a causal state-action-state model (Sharpe et al., 2017), but not simple value-guided actions (Sharp et al., 2016).
Our finding reveals that dopaminergic medication attenuates the cue-driven reward expectation effect on vigour can be contrasted with previous work suggesting that tonic dopamine couples vigour to average reward rate (Beierholm et al., 2013; Niv et al., 2007). Our adaptive reward schedule held the average reward rate constant over time, while manipulating the average reward rate within each condition, such that the guaranteed 10 p and 0 p trials had different expected rewards. Dopamine might reduce these expectation effects through a different mechanism; the guaranteed cues elicit Pavlovian signals that track expected rewards across states and cues rather than time. Our result implicates dopamine in this signalling, but the direction of effect contrasts with naïve predictions. Dopamine is necessary for Pavlovian-to-Instrumental transfer (Hall et al., 2001; Kelley and Delfs, 1991; Wyvell and Berridge, 2000) via the nucleus accumbens. In contrast, we show that reward expectation influences vigour when dopaminergic tone is low, yet does not when dopaminergic tone is high. This aligns with the finding that slow, tonic dopaminergic activity is not related to Pavlovian-to-Instrumental transfer (Wassum et al., 2013). A possible explanation is that being ON dopamine led to a saturation in tonic dopamine leaving little room for phasic cue-related reward expectation signals. But if this were the case, one might expect generally higher velocities when ON, compared to PD OFF, which was not seen. Because our contingent and random conditions were matched for average reward rate, and thus opportunity cost, invigoration by contingent reward indicates a truly instrumental effect.
An alternative explanation for the discrepancy with previous research showing dopamine encodes reward rate, is that the previous studies did not fully decouple contingent and non-contingent motivation. In many studies, expected rewards were only given for successful performance (Beierholm et al., 2013; Niv et al., 2007), meaning the rewards were still contingent on performance. However, when separated, contingent motivation has larger effects on vigour than reward expectation (Manohar et al., 2017), and so it is possible that some previously reported effects of average reward rate on vigour were due to the greater contingency, separate from or in addition to, reward expectation. Indeed, vigour may be reduced by dopamine in PD, though reward sensitivity is increased (Muhammed et al., 2016). An additional challenge to the tonic dopamine theory of reward expectation comes from the finding that fast phasic dopaminergic responses in the nucleus accumbens encode average reward rate, but slow tonic responses do not (Mohebi et al., 2019). That study suggests that reward expectation signals are independent of ventral tegmental area dopaminergic neuron firing, and may instead be due to ‘local’ control over nucleus accumbens core dopamine release. As dopamine is depleted in PD via dopamine-neuron death in the substantia nigra and ventral tegmental area, local dopamine release in other areas may be relatively preserved, and thus still able to influence vigour when PD patients are without dopamine.
The effect of reward-expectation on peak velocity was accompanied by greater velocity, acceleration, and autocorrelation early in the saccade for PD OFF than ON. Greater autocorrelation at this point is expected, as greater velocity increases noise (Harris and Wolpert, 2006; Fitts, 1954). However, this noise increase did not persist until the end of the saccade, as there was no increase in autocorrelation at the end of the saccade (Figure 4) and no greater endpoint variability (Figure 2e) – indeed, guaranteed rewards actually decreased endpoint variability, although this was not affected by dopamine. This offers some indirect evidence that the increased noise in this condition was attenuated via negative feedback (c.f. Manohar et al., 2019).
PD patients had slower saccadic RTs, and slower, smaller and more variable saccades compared to age-matched controls. The pattern of invigoration also differed from controls, who did not show significant effects of either contingent motivation or reward-expectation on speed. Instead, controls had lower motor variability when rewards were guaranteed, but no other significant motivation effects. This leads to a pattern where PD ON show contingent motivation, PD OFF show reward-expectation effects, and HC show neither. As these effects themselves are not statistically different between groups, we are limited in the conclusions that we can draw about them. Numerically, controls show a similar pattern to PD ON (Figure 2a), with faster velocity residuals for contingent rewards, which could suggest that dopaminergic medication is restoring healthy function, but care must be taken with this interpretation. The lack of either type of motivation in the older HC is surprising given that in healthy young adults, both contingent and guaranteed rewards increase saccade velocity (Manohar et al., 2017). This could suggest ageing decreases both contingent-motivation and reward-expectation, although a study directly comparing ages would be needed to conclude this.
The motivational effects reported here were not related to any pupillary responses, unlike our previous findings in young people, which may be due to both ageing and PD decreasing the influence of rewards on pupil size (Manohar and Husain, 2015; Muhammed et al., 2016). Additionally, while the two distinct motivational effects on velocity were uncorrelated within PD patients, it is possible that subgroups of patients showed different effects. For example, whether patients were on D2 agonists (Bryce and Floresco, 2019) or had tremor-dominant disease (Wojtala et al., 2019) might be relevant. However, this study was not powered to detect such differences as only six patients were taking agonists in addition to levodopa.
Considering the neuroanatomical differences between contingent motivation and reward expectation may help to explain our results. The nucleus accumbens and ventral pallidum modulate their activity by reward expectation (Mohebi et al., 2019; Tachibana and Hikosaka, 2012), while the caudate nucleus is active when rewards are contingent on behaviour (Lex and Hauber, 2010a; Tricomi et al., 2004). Both the caudate and accumbens/pallidum project to the output nuclei of the basal ganglia, allowing saccade initiation via the superior colliculus, which controls not only the direction of saccades, but also their instantaneous velocity during the movement (Smalianchuk et al., 2018). We propose contingent motivation and reward expectation both lead to motivational signals affecting the superior colliculus’ activity controlling the velocity and acceleration of saccades, and these are differentially affected by dopamine (Figure 7), although we remain agnostic as to the mechanism for this difference. Possibilities include the two systems receiving input from separate regions of the dopaminergic system which are differentially depleted in PD (e.g. dopamine overdose hypothesis [Cools, 2006]), differences in ‘global’ and ‘local’ dopamine signals (Mohebi et al., 2019), or differences in D1-like and D2-like receptor expression within these systems (Surmeier et al., 2007; Yetnikoff et al., 2014). Further studies should address this question of the underlying mechanism.
We have shown that in PD, dopaminergic medication boosts motivation by contingent rewards, but reduces motivation by expected reward. Nonspecific invigoration by reward may thus be generated by a different neural system than goal-directed motivation. This suggests that dopaminergic medication may be a potential treatment for impairments in contingent motivation, but not for deficits related to reward expectation.
Materials and methods
Participants
Thirty PD patients were recruited from volunteer databases in the University of Oxford. They were all taking levodopa medication, and some were also taking monoamine oxidase inhibitors and/or dopamine agonists (Table 3). They were randomly assigned to be tested ON or OFF medication first, and withdrawn from standard release medication for 16+ hours and controlled-release medication for 24+ hours. Two patients did not complete both sessions, and two did not have enough trials that passed all the criteria (see Analysis section) so were excluded, leaving 26 patients. Thirty healthy controls (HC) were recruited from volunteer databases also, and tested once, and one HC was excluded for insufficient trials passing the criteria. We recruited 30 participants in each group as this was the sample size used in previous experiments with this task and yielded robust effects (Manohar et al., 2017). Sensitivity power calculations showed this would detect effect sizes above 0.46 (Faul et al., 2009) (α = 0.05, power = 0.8, sample size = 30), although as we only included 26 PD in the analysis, this effect size rose to 0.5.
All participants gave written informed consent, and ethical approval was granted by the South Central Oxford A REC (18/SC/0448).
Procedure
Request a detailed protocolThe task was run in Matlab (www.mathworks.com, version 7) using the Psychophysics toolbox (Kleiner et al., 2007), on a Windows XP computer with a CRT monitor (1024 × 768 pixels, 40 × 30 cm, 100 Hz refresh rate) at 70 cm viewing distance. Eye movements and pupil size were recorded with Eyelink1000 at 1000 Hz.
On each trial of the task a fixation dot (0.3° radius) was presented at the centre of the screen, with two empty circles (1.1° radius) shown 9.3° to the left and right of the fixation dot. After 500 ms of fixation, a cue was given by a voice over the speaker, indicating the type of trial the participant was in:
‘Performance’ indicated that fast response times would win 10 p, while slow response times would win 0 p
‘Random’ indicated a 50% probability of 10 p or 0 p, regardless of response time
‘Ten pence’ indicated a guaranteed 10 p, regardless of response time
‘Zero pence’ indicated guaranteed 0 p, regardless of response time
A delay of 1400, 1500 or 1600 ms was given (with equal probability), after which one of the two circles turned white (50% probability of left or right) and participants had to saccade to this circle to complete the trial and receive the outcome.
Participants could only affect the outcome in the Performance condition (by moving faster); all others were independent of their speed. In the Performance condition, rewards were based upon response time (i.e. total time between the target appearing and gaze arriving at the target), which is only minimally influenced by saccade velocity. Participants were rewarded when response time was quicker than their recent median response time for the last 20 Performance trials, which thus yielded a 50% reward rate overall. The Random condition acts as a control to these trials, with a random 50% of trials rewarded, and thus equal expected value but with no performance-contingency. Rewards in the guaranteed conditions also had zero contingency on performance, but yielded different expected rewards (10 p vs 0 p), thus comparing them indexes the pure effect of expecting reward.
When rewards are contingent, people get feedback about how they performed. This itself is known to increase motivation, independent of reward – a phenomenon termed intrinsic motivation. To control for this, we ensured participants always received feedback on their speed (fast/slow, using median split over 20 previous trials in that condition – i.e. the same criteria as for contingent rewards), regardless of reward. This should equate the level of intrinsic motivation across conditions, providing that the feedback is as noticeable as the reward. In order to ensure the speed feedback and reward were matched in physical salience, the feedback modalities were counterbalanced. Two blocks gave auditory feedback for speed and visual feedback for reward, and vice versa for the other two blocks, with order randomised across participants. This counterbalancing accords with our previous study (Manohar et al., 2017) which found no modality effects, suggesting participants were attending to audio and visual feedback equally. We also found no effects of modality on any of the measures of interest (p>0.05), so collapsed across the two modalities for all the analyses.
There were 12 types of each trial in a block, in a random order, and participants completed four blocks.
Analysis
The Performance and 10 p conditions are high motivation conditions. The difference between Performance and Random conditions gives the effect of contingent motivation, while the difference between 10 p and 0 p conditions gives the effect of reward expectation.
As in previous studies (Manohar et al., 2017), our primary measure of interest was saccadic vigour. We measured peak saccade velocity on each trial. We took the first saccade after target onset which was greater than 1° in amplitude, and used a sliding window of 4 ms width to calculate velocity, excluding segments faster than 3000°s−1 or where eye tracking was lost. Saccades with peak velocities outside 80–2500°s−1 were excluded, as were trials where participants reached the target before 180 ms or after 580 ms. Two PD patients and one HC had fewer than 10 trials that passed these criteria for one condition, so were excluded from the analysis.
To remove the main sequence effect of amplitude on velocity (Bahill et al., 1975; Harris and Wolpert, 2006), we regressed velocity against amplitude and took the peak velocity residuals as our measure of interest. This measures the difference between the velocity predicted by the main sequence, and the velocity actually recorded, with positive (negative) values meaning faster (slower) velocity. This was done for each participant’s separate session. This approach has been used before, by us and others (Blundell et al., 2018; Manohar et al., 2017; Muhammed et al., 2020; Muhammed et al., 2016; Van Opstal et al., 1990), and it is similar to simply including amplitude as a covariate when analysing raw peak velocity, but it does not reduce the degrees of freedom and yields simpler to interpret results. Moreover since motivation increases amplitude (Manohar et al., 2019), including amplitude as a covariate would mean that amplitude would compete with motivation to explain variance in velocity, potentially resulting in overestimation of motivation effects. Our findings did not qualitatively change when we used the covariate approach instead.
We also measured amplitude, saccadic reaction time (RT), and endpoint variability of these saccades. Saccadic RT is the time between the target onset and the start of the saccade.
To analyse velocity and acceleration traces, and autocorrelation and covariance of the eye movements we linearly interpolated 50 points along each saccade to move them into the same units. Instantaneous velocity was smoothed across three time-points, while acceleration was smoothed across 5. We also calculated velocity and acceleration traces on the raw (non-interpolated) traces and then interpolated them afterwards, which gave very similar results.
Pupil dilatation was measured in arbitrary units (a.u.) relative to the baseline pupil size at the cue onset. Blinks under 500 ms were linearly interpolated, steps in pupil size above 2.5 a.u./ms were removed, and data were averaged in 20 ms bins for plotting.
We used rmanova from the matlib toolbox (https://github.com/sgmanohar/matlib; Manohar, 2020) to perform analyses – this uses fitglme to perform the repeated-measures test and anova to perform hypothesis tests on the GLME. We used three-way repeated-measures ANOVA to compare effects of motivation, contingency and dopaminergic medication in PD patients, and followed this up with two-way ANOVA when a three-way interaction was found. These analyses were also performed using a full linear mixed effects model including each trial, which produced qualitatively identical results. To compare each PD condition against HC we used mixed ANOVA. We also used cluster-wise permutation tests for the time-course data (velocity, acceleration, pupil dilatation, autocorrelation and covariance), to control the family-wise error rate at. 05.
Data and code availability
Request a detailed protocolAnalyses were performed in Matlab using custom scripts, which are available on GitHub (https://doi.org/10.5281/zenodo.4032711). Anonymous data are available on OSF (https://osf.io/2k6x3), as is the experiment file (osf.io/y9xhp) https://doi.org/10.5281/zenodo.4032711.
Data availability
Anonymised data are available on OSF (https://osf.io/2k6x3).
References
-
Dopamine modulates option generation for behaviorCurrent Biology 28:1561–1569.https://doi.org/10.1016/j.cub.2018.03.069
-
The main sequence, a tool for studying human eye movementsMathematical Biosciences 24:191–204.https://doi.org/10.1016/0025-5564(75)90075-9
-
ConferenceBeck depression Inventory–IIIn the Psychological Corporation.https://doi.org/10.1037/t00742-000
-
Dopamine modulates reward-related vigorNeuropsychopharmacology 38:1495–1503.https://doi.org/10.1038/npp.2013.48
-
Oculomotor abnormalities in children with Niemann-Pick type CMolecular Genetics and Metabolism 123:159–168.https://doi.org/10.1016/j.ymgme.2017.11.004
-
Reward-Based improvements in motor control are driven by multiple Error-Reducing mechanismsThe Journal of Neuroscience 40:3604–3620.https://doi.org/10.1523/JNEUROSCI.2646-19.2020
-
Dopaminergic modulation of cognitive function-implications for L-DOPA treatment in Parkinson's diseaseNeuroscience & Biobehavioral Reviews 30:1–23.https://doi.org/10.1016/j.neubiorev.2005.03.024
-
The algorithmic anatomy of model-based evaluationPhilosophical Transactions of the Royal Society B: Biological Sciences 369:20130478.https://doi.org/10.1098/rstb.2013.0478
-
Nucleus accumbens neurons encode pavlovian approach behaviors: evidence from an autoshaping paradigmEuropean Journal of Neuroscience 23:1341–1351.https://doi.org/10.1111/j.1460-9568.2006.04654.x
-
Habitual versus goal-directed action control in parkinson diseaseJournal of Cognitive Neuroscience 23:1218–1229.https://doi.org/10.1162/jocn.2010.21514
-
Actions and habits : the development of behavioural autonomyPhilosophical Transactions of the Royal Society B 308:67–78.https://doi.org/10.1098/rstb.1985.0010
-
Statistical power analyses using G*power 3.1: tests for correlation and regression analysesBehavior Research Methods 41:1149–1160.https://doi.org/10.3758/BRM.41.4.1149
-
The information capacity of the human motor system in controlling the amplitude of movementJournal of Experimental Psychology 47:381–391.https://doi.org/10.1037/h0055392
-
Role of expected reward in frontal eye field during natural scene searchJournal of Neurophysiology 116:645–657.https://doi.org/10.1152/jn.00119.2016
-
Vigor in the face of fluctuating rates of reward: an experimental examinationJournal of Cognitive Neuroscience 23:3933–3938.https://doi.org/10.1162/jocn_a_00090
-
The main sequence of saccades optimizes Speed-accuracy Trade-offBiological Cybernetics 95:21–29.https://doi.org/10.1007/s00422-006-0064-x
-
What’s new in Psychtoolbox-3? A free cross-platform toolkit for Psychophysics with Matlab & GNU/OctavePerception 36:1.
-
Performance pay and productivityAmerican Economic Review 90:1346–1361.https://doi.org/10.1257/aer.90.5.1346
-
The role of nucleus accumbens dopamine in outcome encoding in instrumental and pavlovian conditioningNeurobiology of Learning and Memory 93:283–290.https://doi.org/10.1016/j.nlm.2009.11.002
-
Appetitive Pavlovian-Instrumental interactions: effects of Inter-Stimulus interval and baseline reinforcement conditionsThe Quarterly Journal of Experimental Psychology Section B 33:257–269.https://doi.org/10.1080/14640748108400811
-
Reward pays the cost of noise reduction in motor and cognitive controlCurrent Biology 25:1707–1716.https://doi.org/10.1016/j.cub.2015.05.038
-
Distinct motivational effects of contingent and noncontingent rewardsPsychological Science 28:1016–1026.https://doi.org/10.1177/0956797617693326
-
Reduced pupillary reward sensitivity in Parkinson’s diseaseNpj Parkinson’s Disease 1:20–23.https://doi.org/10.1038/npjparkd.2015.26
-
parkinson's disease severity levels and MDS-Unified Parkinson's Disease Rating ScaleParkinsonism & Related Disorders 21:50–54.https://doi.org/10.1016/j.parkreldis.2014.10.026
-
The influence of expected value on saccadic preparationJournal of Neuroscience 27:4810–4818.https://doi.org/10.1523/JNEUROSCI.0577-07.2007
-
A normative perspective on motivationTrends in Cognitive Sciences 10:375–381.https://doi.org/10.1016/j.tics.2006.06.010
-
Tonic dopamine: opportunity costs and the control of response vigorPsychopharmacology 191:507–520.https://doi.org/10.1007/s00213-006-0502-4
-
The opportunity cost of time modulates cognitive effortNeuropsychologia 123:92–105.https://doi.org/10.1016/j.neuropsychologia.2018.05.006
-
Temporal discounting of reward and the cost of time in motor controlJournal of Neuroscience 30:10507–10516.https://doi.org/10.1523/JNEUROSCI.1343-10.2010
-
Toward a rational and mechanistic account of mental effortAnnual Review of Neuroscience 40:99–124.https://doi.org/10.1146/annurev-neuro-072116-031526
-
Instantaneous midbrain control of saccade velocityThe Journal of Neuroscience 38:10156–10167.https://doi.org/10.1523/JNEUROSCI.0962-18.2018
-
Human pavlovian-instrumental transferJournal of Neuroscience 28:360–368.https://doi.org/10.1523/JNEUROSCI.4028-07.2008
-
Enhanced motivation of cognitive control in Parkinson's diseaseEuropean Journal of Neuroscience 48:2374–2384.https://doi.org/10.1111/ejn.14137
-
Comparison of saccades evoked by visual stimulation and collicular electrical stimulation in the alert monkeyExperimental Brain Research 79:299–312.https://doi.org/10.1007/BF00608239
-
Cognitive decline in Parkinson’s disease: the impact of the motor phenotype on cognitionJournal of Neurology, Neurosurgery & Psychiatry 90:171–179.https://doi.org/10.1136/jnnp-2018-319008
-
The intrinsic value of visual information affects saccade velocitiesExperimental Brain Research 196:475–481.https://doi.org/10.1007/s00221-009-1879-1
-
The hospital anxiety and depression scaleActa Psychiatrica Scandinavica 67:361–370.https://doi.org/10.1111/j.1600-0447.1983.tb09716.x
Article and author information
Author details
Funding
MRC (MR/P00878X)
- Sanjay G Manohar
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Acknowledgements
We would like to thank the patients and participants for their time in taking part in this study, and the funders for their support (MRC Clinician Scientist Fellowship to SGM, MR/P00878X).
Ethics
Human subjects: Ethical approval was granted by the South Central Oxford A REC (18/SC/0448). All participants gave written informed consent.
Copyright
© 2020, Grogan et al.
This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.
Metrics
-
- 5,799
- views
-
- 404
- downloads
-
- 26
- citations
Views, downloads and citations are aggregated across all versions of this paper published by eLife.
Download links
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Further reading
-
- Neuroscience
Sour taste, which is elicited by low pH, may serve to help animals distinguish appetitive from potentially harmful food sources. In all species studied to date, the attractiveness of oral acids is contingent on concentration. Many carboxylic acids are attractive at ecologically relevant concentrations but become aversive beyond some maximal concentration. Recent work found that Drosophila ionotropic receptors IR25a and IR76b expressed by sweet-responsive gustatory receptor neurons (GRNs) in the labellum, a peripheral gustatory organ, mediate appetitive feeding behaviors toward dilute carboxylic acids. Here, we disclose the existence of pharyngeal sensors in Drosophila melanogaster that detect ingested carboxylic acids and are also involved in the appetitive responses to carboxylic acids. These pharyngeal sensors rely on IR51b, IR94a, and IR94h, together with IR25a and IR76b, to drive responses to carboxylic acids. We then demonstrate that optogenetic activation of either Ir94a+ or Ir94h+ GRNs promotes an appetitive feeding response, confirming their contributions to appetitive feeding behavior. Our discovery of internal pharyngeal sour taste receptors opens up new avenues for investigating the internal sensation of tastants in insects.
-
- Neuroscience
Granule cells of the cerebellum make up to 175,000 excitatory synapses on a single Purkinje cell, encoding the wide variety of information from the mossy fibre inputs into the cerebellar cortex. The granule cell axon is made of an ascending portion and a long parallel fibre extending at right angles, an architecture suggesting that synapses formed by the two segments of the axon could encode different information. There are controversial indications that ascending axon (AA) and parallel fibre (PF) synapse properties and modalities of plasticity are different. We tested the hypothesis that AA and PF synapses encode different information, and that the association of these distinct inputs to Purkinje cells might be relevant to the circuit and trigger plasticity, similar to the coincident activation of PF and climbing fibre inputs. Here, by recording synaptic currents in Purkinje cells from either proximal or distal granule cells (mostly AA and PF synapses, respectively), we describe a new form of associative plasticity between these two distinct granule cell inputs. We show for the first time that synchronous AA and PF repetitive train stimulation, with inhibition intact, triggers long-term potentiation (LTP) at AA synapses specifically. Furthermore, the timing of the presentation of the two inputs controls the outcome of plasticity and induction requires NMDAR and mGluR1 activation. The long length of the PFs allows us to preferentially activate the two inputs independently, and despite a lack of morphological reconstruction of the connections, these observations reinforce the suggestion that AA and PF synapses have different coding capabilities and plasticity that is associative, enabling effective association of information transmitted via granule cells.