1 Background

How are you feeling right now? Research across economics, psychology, and health sciences suggests the answer to this question—your subjective well-being—is closely tied to objective quality of life^1,2 and health across the lifespan³. But feelings are far from static, momentarily fluctuating in response to recent events^4–6, and even individual choices. Frequently asking participants to rate their feelings enables a read-out of moment-to-moment changes in subjective well-being, or their affective dynamics.

In influential work, Rutledge et al. (2014)⁵ demonstrated momentary happiness ratings during a gambling task could be accurately predicted by a computational model incorporating the average reward for a gamble (expected value) and the outcome of the gamble minus this average (prediction error). Using a task where reward magnitude and probability were uncorrelated, Blain & Rutledge (2020)⁷ subsequently showed that momentary happiness is particularly sensitive to changes in learning-related variables—specifically, prediction errors for reward probability—as compared to as compared to reward information that was relevant to behaviour but not learning. Links between happiness ratings and learning-related quantities may extend to subjective assessments of other affective states. Theoretical accounts posit that momentary confidence is the approximate probability a choice is correct^8,9 (though see^10,11), while effort costs decrease the value of choices independently of reward probability^12,13, in turn influencing momentary engagement. Together, these results suggest that affective dynamics are closely coupled with objective quantities that drive choices during learning.

Biases in subjective affect are a core feature of mental ill-health. Symptoms of depression have been consistently linked to lower^7,14 and less stable^15,16 momentary happiness, while transdiagnostic features of mental ill-health have been linked to biased confidence judgements at different timescales^17–20 and impairments in motivation and engagement^13,21–23. Affective biases maintain symptoms of mental ill-health by inducing changes in emotion processing and perception^24,25. For example, negatively biased perception—a common feature of depression²⁶—may cause low mood by making outcomes appear less rewarding; low mood in turn further negatively biases perception, causing a positive feedback loop which spirals toward a depressive episode²⁷. Successful psychiatric treatment may act to perturb these maladaptive cycles. Short-term selective serotonin reuptake inhibitor (SSRI) administration induces positive perceptual biases in healthy participants²⁸, suggesting that affective biases may be an early target of antidepressant drugs, acting to shift choices away from those that maintain low mood²⁹. Crucially, given that affective biases are precipitated and maintained by negative thinking patterns—a core target of cognitive psychotherapy^30,31— they may represent a transdiagnostic treatment target of psychological and pharmacological interventions for symptoms of mental ill-health.

Here, we aimed to link choice behaviour to affective dynamics throughout a reinforcement learning (RL) task^32–34 (Figure 1A) and to relate this to mental ill-health symptoms and treatments. We asked online participants (N =935) to rate their feelings (from 0–100) on one of three different affect scales—happiness, confidence, and engagement—after receiving feedback on each choice they made. Half (49.1%) of the participants were randomised to an acute psychological intervention known as “cognitive distancing”, a common³⁵ and effective³¹ component of psychological therapy which alters learning in this task³⁴. We also collected information on current antidepressant medication use in a demographic questionnaire (reported by 13.9% of participants), and derived transdiagnostic mental health symptom factor scores from a psychiatric questionnaire battery^36,37. We then assessed how participants’ affect ratings across each of the three scales covaried with learning-related outcomes throughout the task, accounting for underlying affective biases, using computational modelling. By quantifying the associations between model-derived measures of affective dynamics and transdiagnostic features of mental ill-health, the cognitive distancing intervention, and self-reported antidepressant medication use, we asked whether affective dynamics may be a common target of treatments for symptoms of mental ill-health.

Figure 1

2 Methods

2.1 Online experiment and sample

A total of 995 participants were recruited via Prolific³⁸ over three weeks in April–May 2021. Participants were recruited in batches with fixed pre-screeners for age range, gender, and history of any mental health diagnosis, which resulted in a sample broadly representative of the UK population in terms of these characteristics (see³⁴ for details). After completing a demographic questionnaire, which included questions on current medication usage and mental health diagnoses, participants completed the RL task described below. They then took a short test of working memory (visual digit span), and answered questions from several psychiatric questionnaires, answers from which were used to derive three validated transdiagnostic features of mental health (anxiety/depression, compulsive behaviour, and social withdrawal)³⁶, using methods for computational factor modelling³⁷ described in detail elsewhere^34,39. Sixty participants were excluded for meeting pre-registered criteria³⁴. The study was approved by the University of Cambridge Human Biology Research Ethics Committee (HBREC) (HBREC.2020.40) and jointly sponsored by the University of Cambridge and Cambridge University Hospitals National Health Service (NHS) Foundation Trust (IRAS ID 289980). All participants provided written informed consent through an online form, in line with University of Cambridge HBREC procedures for online studies.

2.2 Reinforcement learning task

The reinforcement learning task in the present study—the probabilistic selection task^32,33— involved learning which symbol in each of three pairs was more likely correct. Consistent choice of the “better” symbol in each pair enabled a participant to accumulate more points (and maximise their chances of winning a monetary bonus). One symbol in each pair was always more likely correct, but the contingencies varied across the pairs, from 0.8/0.2 (‘AB’) to 0.7/0.3 (‘CD’), to 0.6/0.4 (‘EF’). All participants saw the same six symbols, but the pairs were randomised across individuals and counterbalanced across trials. After making a choice, participants received feedback (“Correct!” or “Incorrect.”), and then rated their subjective happiness, confidence, or engagement (Figure 1A).

One of the three questions was asked after each trial outcome, with each question asked twenty times per block of sixty trials, and never more than twice in a row. Participants were also asked to rate (again from 0–100) how fatigued they felt compared to the beginning of the block after the end of each of the sixty-trial training blocks. After six training blocks, participants were tested on all fifteen unique pairs without feedback. We previously reported the effects of cognitive distancing on task performance and learning, including results of the test phase, in the same sample³⁴.

2.3 Acute psychological intervention and antidepressant use

Half of the participants (n=459; 49.1%) were randomised to be taught, and then practice throughout the task, a psychotherapeutic technique termed “cognitive distancing”, which encouraged them to “take a step back” from their emotional reactions to feedback throughout the task (see here³⁴ for further details). Apart from an additional instructional video before the task started and a small prompt to “Distance yourself...” which appeared with each fixation cross (Figure 1A), the task was identical for distanced and non-distanced participants. To explore similarities between the effects of this psychological intervention, and a pharmacological treatment for mental ill-health, antidepressant medication, we also asked participants to report their current medication use: 130 participants (13.9%) reported current antidepressant use, with the majority (n=94; 72.3%) taking an SSRI.

2.4 Computational modelling: joint RL-affect models

For consistency with previous literature, we used the well-characterised model of momentary happiness first described by Rutledge et al. (2014)⁵ as a baseline model. This model assumes fluctuations around a baseline (i.e., longer-term mean) can be captured by a weighted sum of recent expected values and prediction error and, importantly, does not condition on previous ratings.

The Rutledge et al. (2014)⁵ model has been primarily validated in (e.g., gambling) tasks where expected values and prediction errors are explicitly available to participants^5,40. As such, we extended it to account for learning in this task. Specifically, our model—which we term a joint RL-affect model—comprised two components: (1) a Q-learning model to infer expected values and prediction errors from participants’ choices (which has been shown to accurately capture choice behaviour in this task^33,34); and (2) a model for momentary affect which assumes fluctuations around a baseline can be captured by a recency-weighted sum of Q-learning model-derived expected values and prediction errors^5,41. Hierarchical models were simultaneously fitted to task choices plus happiness, confidence, and engagement selfreports, assuming different parameter weights across all scales and participants.

2.4.1 RL models

A Q-learning model infers expected values (termed Q-values) from participant choices—the action (a_t) of choosing one symbol over the other in each of the three pairs (denoted as states, a_t)—by assuming they update at each trial t based on prediction errors δ_t, with the update magnitude controlled by a learning rate α ∈ [0,1]:

We additionally considered a dual learning rate Q-learning model for choices, in which the learning rate is assumed to differ depending on whether the outcome was rewarding (α_reward) or not (α_loss)³³:

In both cases, the difference in Q-values between the chosen (a_t) and avoided action (ā_t) is transformed to a choice probability using a softmax function, weighted by an inverse temperature β:

2.4.2 Affect models

Baseline model

Affect ratings scaled to [0, 1] for each participant and rating type (i.e., happiness, confidence or engagement) were assumed to be drawn from independent Beta distributions with a mean-variance reparameterization which models the shape parameters as functions of a (conditional) mean and precision⁴². Extreme ratings (0 or 1) were allowed in the task, so we transformed ratings to the (0,1) interval using a simple transformation⁴³ ; here N is the total number of participants but could be any large number). A logit link function was used, so the Beta regression weights (excluding the intercept ) can be interpreted as the log odds ratio per unit change in the covariate for an increase in affect rating, with other terms held constant (note that our approach differs slightly from that of Rutledge et al. (2014)⁵, who assume momentary happiness ratings follow a Gaussian distribution; see Forbes & Bennett (2024)⁴¹ for a validation of this Beta regression approach). Following^5,41 we write the ith participant’s affect rating Y⁽ⁱ⁾ for rating type p at trial t as,

where t is the overall trial number at rating number i for rating type p, γ is the discount or ‘forgetting’ factor which imposes a strict weighting on recent trials, and the weights on each of the k quantities of interest for the pth rating type; and Q_t and r_t, are the Q-learning model-derived expected values for the chosen symbol a_t, in the state s_t, (i.e., the pair of symbols presented on trial t^′) and the feedback valence (±1 for correct/incorrect to allow for negative Q-values) respectively, both at trial t^′. Note that both sums are over all previous trials, not just those of rating type p.

Accounting for drift over time

We also fit models including an extra weight to account for potential “drift over time” in affect⁴⁴, by modifying (5) as follows,

where τ_t is some measure of time elapsed up to trial t: either trial number, block number, overall time elapsed, or time elapsed since the start of that block (see Model comparison).

2.4.3 Fits to data

Models were fitted in a hierarchical Bayesian manner, with approximate posteriors derived via automatic differentiation variational inference (ADVI)⁴⁵ implemented in CmdStan⁴⁶. All models were fit to choices and ratings on all three affect scales simultaneously across both distancing and non-distancing participants, with separate weights and decay factors assumed for each person and question, and separate group-level (hyper)priors on each parameter. In other words, participant-level parameter distributions are assumed to be conditionally independent given the group-level distribution over that parameter. Individuallevel predictive accuracy was assessed by comparing responses predicted from each participant’s approximate posterior to their observed affect ratings via pseudo-R², defined following previous work⁴² as the squared correlation between observed and mean posterior predictions.

2.4.4 Model comparison

In the affect model, we tested for “drift over time“⁴⁴; and in the Q-learning model we tested for separate learning rates for rewarding and non-rewarding outcomes. We assumed the Rutledge et al. (2014)⁵ model (equation 5) to be the baseline model, and so the parameters in this model were included in all models.

Drift over time in affect may be particularly relevant to our task, as participants were able to take as long as they wished to rate their subjective feelings, and time between blocks was additionally unconstrained. As such, we compared four models with different measures of time elapsed (i.e., variants of equation 6) to the baseline model (equation 5), and either single or dual learning rates. The extra parameter added linear weights on either trial number, block number, or total time elapsed. We also tested a final model with two extra parameters: weights on both total time elapsed and time elapsed since the beginning of that block. We then compared all ten models in terms of their approximate leave one out (LOO) expected log pointwise predictive density (ELPD), a metric of estimated out-of-sample predictive accuracy⁴⁷, corrected for the use of variational approximations to the true posterior⁴⁸.

2.5 Statistical analysis

For consistency with computational modelling, we adopt a fully Bayesian approach for statistical analyses where possible. As such, results are given as estimates with a highest density interval (HDI), which (unlike a confidence interval (CI)⁴⁹) can be interpreted as the probability that the true value falls within a given range. We report 95% HDIs to align with convention but interpret results in terms of strength of evidence throughout; an overlap with the null value should not be seen as evidence for lack of an effect, but rather weakened evidence for it.

2.5.1 Associations between model parameters and mental health symptoms & treatments

We tested the effect of differences in transdiagnostic mental health symptoms, current selfreported antidepressant use, or cognitive distancing on model parameters using generalised linear models (GLMs), adjusted for age, gender, and working memory capacity (measured with visual digit span), separately for each rating type. GLMs relating model parameters to antidepressant use were run with and without adjustment for concurrent anxiety/depression symptoms (i.e., factor score), as medication use was not randomised. We also considered whether the effect of cognitive distancing and current antidepressant use on affect model parameters may differ in relation to their association with transdiagnostic mental health, by including factor score as an interaction term in GLMs.

Posterior samples for GLM coefficients were obtained via Markov chain Monte Carlo (MCMC) implemented in CmdStan⁴⁶, with 2,000 warm-up and 10,000 sampling iterations for each of four chains, using models and priors from the rstanarm R package⁵⁰. Response distributions were assumed Gaussian for all parameters except for learning and decay rates, which were modelled via Gamma family GLMs (with log link functions). See Interpretation of model-derived parameters in the Supplementary Methods for intuition as to how these regression coefficients are interpreted.

2.5.2 Differences in associations with baseline affect between transdiagnostic factors

To account for potential collinearities in the three transdiagnostic mental health symptom factor scores obtained via computational factor modelling, we used partial least squares (PLS) regression to test which of the transdiagnostic factor scores was most strongly associated with baseline affect (). PLS regression is a data-driven method which identifies latent components linking multivariate responses to predictors based on shared covariance, and so is well-suited to the problem of multicollinearity⁵¹.

In line with best-practices³⁹, we first identified the number of components that best described our data in a training set (80% of participants) in terms of mean squared error (MSE) and R² via ten-fold cross-validation. We then validated the predictive accuracy of this number of components in held-out test data (20% of participants), and formally tested this using a permutation test, where the PLS regression model was re-trained on 10,000 training datasets with shuffled outcome labels (providing a null distribution), and the fraction of these datasets where the MSE between the test data and the predictions from the permuted datasets was lower than the true train-test MSE taken as the p-value. The PLS regression with the chosen number of components was then refitted in the whole dataset, to obtain component loadings on each of the responses and predictors. Lastly, we computed bias-corrected and accelerated (BCa) CIs for each of the loadings, plus the differences in loadings between transdiagnostic factors, from 10,000 bootstrap samples.

3 Results

3.1 A computational model of subjective happiness accounting for learning and affective drift also captures momentary confidence and engagement

Following model comparison, we found that the best-fitting RL-affect model included separate learning rates for rewarding and non-rewarding outcomes and a linear effect of time elapsed since the beginning of the task (equation 6; Figure 1D). This model, when fitted to all affect ratings and participants simultaneously, explained participants’ variance in happiness, confidence, and engagement assessments with similar accuracy (mean [standard deviation (SD)] pseudo-R² = 0.40–0.42 [0.23–0.26]; see Figure 1C for example individuals).

3.2 Baseline affect is negatively associated with transdiagnostic mental health

We first assessed whether individuals’ estimated model parameters from the best-fitting joint RL-affect model were associated with transdiagnostic features of mental health.

In line with previous work^7,14, we found strong evidence for a negative association between baseline happiness () and anxiety/depression symptoms from a linear model adjusting for age, gender, digit span, and distancing (mean 4.44-point lower baseline happiness rating per SD increase in factor score; 95% HDI = [−5.46, −3.41]). Higher anxiety/ depression factor scores were additionally associated with lower baseline confidence (mean 3.75-point lower per SD increase in score; 95% HDIs = [−5.01, −2.46]) and lower baseline engagement (mean 2.90-point lower per SD increase in score; 95% HDIs = [−4.12, −1.66]; Figure 2Ai). We also found that higher anxiety/depression factor scores were associated with lower odds of increases over time in happiness (mean 11.2% lower per SD increase in anxiety/depression score per hour; 95% HDI for multiplier = [0.808, 0.971]) and engagement (17.0% lower for a unit increase in anxiety/depression score per hour; 95% HDI for multiplier = [0.726, 0.943]; Figure 2Aii), suggesting a higher rate of decline (i.e., more drift) in happiness and engagement in participants with higher anxiety/depression scores.

Figure 2

Higher compulsive behaviour and social withdrawal factor scores were also each associated with lower baseline happiness (e.g., mean 2.20-point lower [95% HDI = (−3.31, −1.06)] per SD increase in compulsive behaviour score; Figure Figure 2Bi) and confidence (e.g., mean 3.59-point lower [95% HDI = (−3.98, −1.38)] per SD increase in social withdrawal score; Figure 2Ci), but were not associated with greater drift in affect over time (Figure 2Bii & Figure 2Cii). Associations between transdiagnostic symptom scores and choice-related affect measures (i.e., and were also observed (Figure S3), as were strong associations between post-block fatigue ratings and both baseline and drift in affect over time (Figure S5; see Supplementary Results for more details).

3.3 Baseline affect is most strongly associated with anxiety/depression symptoms

The three transdiagnostic mental health symptom factor scores obtained via computational factor modelling were highly correlated (r = 0.47 [95% CI: (0.42, 0.52)] between anxiety/ depression and compulsive behaviour; r = 0.61 [95% CI: (0.57, 0.65)] between anxiety/ depression and social withdrawal; r = 0.42 [95% CI: (0.37, 0.47)] between compulsive behaviour and social withdrawal). As such, to compare the strength of associations between baseline affect and transdiagnostic symptom factors, we used a partial least squares regression model to relate baseline affect to the three scores plus age, gender, digit span, and distancing.

We found that a three-component model represented the best compromise between predictive accuracy (in training data) and parsimony (Figure 2D), and there was strong statistical evidence that this model could accurately predict responses in held-out test data (permutation test p<0.0001). The first component of the model negatively loaded on baseline happiness (loading = −0.185, BCa bootstrapped 95% CI = [−0.217, −0.135]), confidence (loading = −0.112, BCa bootstrapped 95% CI = [−0.157, −0.049]), and engagement (loading = −0.131, BCa bootstrapped 95% CI = [−0.171, −0.068]) (Figure 2F). It also positively loaded on each of the transdiagnostic symptom factors: anxiety/depression (loading = 0.660, BCa bootstrapped 95% CI = [0.608, 0.733]), compulsive behaviour (loading = 0.491, BCa bootstrapped 95% CI = [0.415, 0.545]), and social withdrawal (loading = 0.577, BCa bootstrapped 95% CI = [0.529, 0.623]). The other two components did not show this pattern (Figure 2E). There was strong evidence that the first component’s loading was higher for anxiety/depression than both compulsive behaviour and social withdrawal (component 1 loading difference [BCa bootstrapped 95% CI] = 0.169 [0.083, 0.299] and 0.083 [0.037, 0.152] respectively; Figure 2G).

3.4 Cognitive distancing slows affective drift and antidepressant use positively modulates baseline affect

We then assessed the evidence for effects of cognitive distancing and antidepressant use on choice-independent affective dynamics: baseline affect and its drift.

Previously, in the same sample, we reported evidence from linear mixed models that participants practising cognitive distancing declined slightly less in happiness and engagement, but not confidence, over the course of the task³⁴. Evidence from our RL-affect model was consistent with a decrease in affect drift over time: distancing individuals on average drifted less across the task in happiness (estimated mean 21.3% higher odds of increase in happiness over per hour; 95% HDI for multiplier = [1.01, 1.46]; Figure 3Aii), in spite of lower baseline engagement (estimated mean = −2.89 points; 95% HDI = [−5.42, −0.464]; Figure 3Ai). There was also some weak evidence of less drift in engagement in participants randomised to the intervention (17.4% higher in distancing individuals; 95% HDI for multiplier = [0.91, 1.52]; Figure 3Aii).

Figure 3

There was limited evidence for any difference in affective drift in participants taking antidepressants (Figure 3Bii). However, there was evidence that participants self-reporting current antidepressant use had 3.50-point higher baseline happiness (95% HDI = [0.311, 6.60]) and 2.69-point higher baseline confidence (with much weaker evidence: 95% HDI = [−1.16, 6.54]), after adjusting for anxiety/depression symptom scores (Figure 3Bi).

3.5 Cognitive distancing and antidepressant use have opposite effects on the weighting of choices in subjective happiness

Next, we quantified the associations between treatments—cognitive distancing and antidepressant use—and choice-dependent parameters (i.e., and ) which control the extent of trial-to-trial fluctuations in affect.

There was limited evidence for an association between either treatment and weights on recent prediction errors (; Figure 3Aiv & Figure 3Biv). However, there was some evidence that cognitive distancing lowered weighting of recent expected (choice) values in happiness ratings (Figure 3Aiii), with 1.83% (95% HDI for multiplier=[0.961, 1.003]) lower odds of increased happiness ratings for the same weighted sum of recent expected values in distanced participants. Meanwhile, exploratory analyses with an extended between-rating model showed a specific effect of antidepressant use on the weighting of recent expected values in engagement and confidence ratings (see Supplementary Results & Figure S4Biii–iv).

Current antidepressant use was associated with less forgetting of choices and outcomes in happiness ratings (12.5% higher γ_happy; 95% HDI for multiplier = [1.04,1.21]) and engagement (6.52% higher γ_engaged; 95% HDI for multiplier = [1.01,1.13]); Figure 3Biii), suggesting higher weighting of earlier trials’ expected values and prediction errors in subsequent affect ratings. Evidence for this association remained, albeit slightly weakened, after additionally adjusting for current anxiety/depression symptoms (Figure 3Biii), which were themselves positively associated with γ_happy and (to a lesser extent) γ_engaged (Figure S3Aiii). Notably, the converse effect was seen in distancing participants, with the psychological intervention associated with lower happiness forgetting factors, albeit with weak evidence (4.69% lower γ_happy; 95% HDI for multiplier = [0.906, 1.004]; Figure 3Aiii).

3.6 Cognitive distancing and antidepressant use dampen negative associations between baseline affect and anxiety/depression symptoms

Lastly, we explored whether the negative associations between choice-independent affective dynamics and transdiagnostic anxiety/depression symptoms were altered by cognitive distancing or current antidepressant use, by including treatment by symptom interactions in outcome GLMs.

We found that both the distancing intervention and antidepressant use weakened the negative associations between baseline happiness and confidence, but not engagement. Specifically, distancing individuals with higher anxiety/depression scores had higher baseline happiness and confidence (mean 1.37-point higher and 1.79-point higher per SD increase in anxiety/depression score respectively) relative to non-distancing participants with the same symptom scores, though with weak evidence (95% HDIs for this distancing by symptom interaction = [−0.63, 3.46] for baseline happiness and [−1.11, 6.12] for confidence (Figure 3Ci). These effects were mirrored in participants taking antidepressants. The evidence for an antidepressant by anxiety/depression symptom interaction with respect to baseline happiness was fairly weak (1.86-point higher per SD increase in anxiety/depression score; 95% HDI = [−1.52, 5.27]), but the evidence for the corresponding interaction effect on baseline confidence was stronger (mean 4.59-point higher per SD increase in anxiety/depression score; 95% HDI = [0.559, 12.53]; Figure 3Cii).

4 Discussion

Here, we applied a computational model of momentary happiness which assumes fluctuations in affect ratings depend simply on baseline affect, its drift over time, and recency-decayed expected and received outcomes. By extending this model to also capture learning, we were able to link objective behaviour to subjective feelings across distinct affective states—happiness, confidence, and engagement ratings—and show that a a core component of psychological therapy, cognitive distancing, and antidepressant medication use have different effects on affective dynamics, but converge to alter affective biases associated with symptoms of mental ill-health.

There were distinct effects of both treatments on affective dynamics. Randomisation to a psychotherapeutic intervention, cognitive distancing, attenuated declines in happiness and engagement over time, adding to our previously reported findings that this psychotherapeutic technique alters aspects of reward learning³⁴. Self-reported antidepressant use, meanwhile, was associated with higher baseline happiness and confidence after adjustment for current anxiety/depression symptoms (as antidepressant use was not randomised), which is consistent with evidence that antidepressants exert positive affective biases²⁸. Subsequent exploration of changes in affect revealed further mechanistic divergence: current antidepressant use was associated with lower recency biases across all scales (i.e., forgetting factors closer to one), and cognitive distancing reduced the weighting of expectations and higher recency bias in happiness ratings. Together, these results suggest that psychiatric treatments act to alter the contribution of objective learning-related quantities to subjective value judgements.

Consistent with extensive evidence^14,18,19,26, we found negative associations between model-derived baseline affect (across all scales) and transdiagnostic psychiatric symptom measures derived from computational factor modelling^36,37. This effect, which was strongest for anxiety/depression symptom scores, indicates a consistent, time-invariant negative affective bias which scales with mental ill-health symptom load. Critically, we found evidence for a convergent treatment by symptom interaction with baseline affect across both cognitive distancing and antidepressant use: negative associations between anxiety/depression symptoms and baseline happiness and confidence were attenuated in participants with higher anxiety/depression symptom scores. These results support the cognitive neuropsychological model of antidepressant action, whereby antidepressants are proposed to act acutely to revert negative or maladaptive affective biases^29,52, and suggest that cognitive distancing³¹ and other components of psychotherapy may also act clinically to alter affective biases contributing to symptoms of mental ill-health. We propose that changes in affective dynamics should be investigated further in longitudinal studies as a computational predictor of subsequent symptom change.

Our methodological approach also extends previous work in two ways. First, we applied a theory-driven computational model which allows for fluctuations in momentary happiness as a function of the history of expected values and prediction errors resulting from those expectations, which has been primarily validated in tasks where learning is not required⁵. We not only show that this model can capture happiness ratings in a task where expected values are never explicitly available and have to be learned from experience, but can also accurately capture variation in subjective ratings of confidence and engagement. Second, we found evidence for drift in happiness over time, replicating recent work which characterised ‘mood drift over time‘⁴⁴, and extended this to both confidence and engagement. We also found that this drift was strongly associated with self-reported fatigue (Figure S5; see Supplementary Results for more details). We note that we did not find evidence of a previously reported effect—reduced mood drift over time with increased depressive symptoms⁴⁴—instead finding evidence to the contrary (Figure 1Aii). However, this work primarily reported evidence from short gambling tasks⁴⁴; our results indicate that associations between affective drift and mental ill-health symptoms are not task-invariant.

We note several limitations. Firstly, there were limitations in our outcome measures. Transdiagnostic measures of mental health psychopathology were estimated using questionnaire subsets^34,39, precluding investigation of associations between parameters and individual diagnostic scales. Antidepressant use, meanwhile, was self-reported, non-randomised, and we did not collect information concerning length of treatment. Secondly, our use of a single computational model for all three affect scales is a powerful approach, but limited in its ability to truly contrast trial-to-trial fluctuations in each individual rating scale, as we are only comparing the contribution of a small number of computational components (i.e., affective weights) to a fraction of their variation; the residual (scale-specific) variation is likely also important in explaining how these ratings overlap and differ. Third, as model complexity meant only approximate (mean-field variational, rather than samplingbased) inference was viable, we were unable to account for uncertainty in estimates of individual-level posterior mean parameters in associations with quantities of interest (e.g., by using precision-weighted GLMs), as the posterior covariance matrix cannot accurately capture local interdependencies, meaning that parameter precisions are not reliable enough for uncertainty-weighted outcome models⁴⁵.

To conclude, we integrated objective choice behaviour in a learning task with trial-to-trial affect ratings across three distinct states—happiness, confidence, and engagement— within a unified computational model. This enabled us to uncover associations between model parameters and treatments for mental health conditions, offering new insights into their underlying mechanisms-of-action. Our results demonstrate the critical importance of affective biases in the maintenance and updating of affective states in mental ill-health, and indicate that existing, effective treatments can be understood at least in part as acting to shift these biases towards the healthy range.

Supplementary Material

Supplementary Methods

Interpretation of model-derived parameters

In the Results, we report intercept (i.e., baseline affect) parameters () following an inverse logit transformation to allow interpretation of the GLM coefficient on the original (point) scale (from 0 to 1) as the difference in baseline affect rating between individuals differing only in the covariate of interest (by one unit). Other GLM-estimated weight parameter differences are interpreted as the difference in the (log) odds of an increase in affect rating between individuals differing only in the covariate that the parameter weights.

For intuition, consider an estimated GLM coefficient of 0.1 for the distancing group in relation to baseline happiness () inverse logit transformed to a 0 to 1 scale. This result indicates an estimated 10-point (on the 0–100 scale) higher baseline happiness rating in distanced participants after covariate adjustment. Meanwhile, a GLM coefficient for the distancing group of 0.1 in relation to for the time elapsed (overall) model (i.e., where is the adjusted log odds ratio for an increase in happiness for a one hour increase in elapsed time) suggests that the distanced group were 10.5% more likely than non-distanced participants to have an increase in happiness after an hour (i.e., the estimated multiplier is exp(0.1) = 1.105) (note the exact interpretation (i.e., greater increase vs. slower decline in odds) depends on the GLM intercept term, which in this case would be the estimated in non-distanced participants after covariate adjustment). Similarly, a GLM coefficient for the distancing group of 0.1 in relation to suggests that a unit increase in Q-value is 10.5% more likely to produce an increase in happiness rating, suggesting a higher contribution of (recent) expected values in the affect rating computation.

Between-rating RL-affect model

Model definition and fit

In an exploratory analysis, we modified (6) to partition the weights on expected values and prediction errors ( and ) into individual weights on the outcomes of each previous trial since the previous rating (in our task, this could be up to four intervening outcomes plus the current trial). As such, the lagged trial weight parameters and can be seen as capturing the between-rating change in affective dynamics. We hence modify (6) as follows,

where I denotes the number of ratings between the current rating and the last rating of the same type; in this task, I ≤ 4. are the weights on the outcomes t^′ trials back, so is the weight on the outcome of the current trial, is the weight on the outcome of the previous trial, and so on. This model was fit to choices and ratings on all three affect scales simultaneously in a hierarchical Bayesian manner via ADVI⁴⁵ as explained in Methods. The only difference was that lagged trial weights were assumed drawn from and priors specific to each individual which in turn were drawn from overall group-level hyperpriors (i.e., a three-level hierarchy).

Associations between between-rating RL-affect model parameters and treatments

The associations between the weights on expected values and prediction errors for individual trials and cognitive distancing and antidepressant use were quantified using multilevel Bayesian linear regression models implemented in the brms R package⁵³ and CmdStan⁴⁶. All models controlled for the same covariates (i.e., age, gender, digit span), but also included a participant-level random intercept and slopes (on trial lag) to account for the fact there were five parameters per person for each affect rating, as well as the main effect of trial lag and its interaction with each treatment. Separate models were fit for each affect rating and parameter (i.e., w_2(–t′) and w_3(–t′).

Parameter recovery

Joint RL-affect model with mood drift over time

To test whether we could recover known parameter values from the best-fitting model (i.e., dual learning rate, time elapsed over time), we simulated one hundred datasets (including choices and affect ratings) with parameters drawn from the following distributions:

We then fit the model to these simulated data with approximate inference, and compared the posterior mean parameter estimated to those known to have generated the data. We found that all parameters could be recovered with high accuracy (r >0.87; Figure S1A).

Table S1

Figure S1

Between-rating RL-affect model

Parameter recovery for the between-rating RL-affect model was tested similarly, with one hundred simulated datasets. The parameter settings were identical to the above except for the time-dependent parameters (and the absence of γ_p in the model). Specifically, and were sampled from Beta(1, j) distributions, where t^′ is the trial lag, so that weights for earlier trials were weighted (on average) lower, as we see in the real dataset. Again, even with the high complexity of the model, we found that all parameters could be recovered with reasonably high accuracy (r >0.77; Figure S1B).

Comparison between results from models fit to choices alone (with sampling-based inference) vs. in the joint RL-affect model (with variational inference)

We previously reported results in this dataset where we fit Q-learning models to choices alone and compared parameters in distanced and non-distanced participants³⁴.

Besides the obvious difference—that the models were fit to choices alone as opposed to both choices and affect ratings—the models in our previous work³⁴ were fit to data via sampling-based inference (MCMC), as opposed to variational inference (ADVI⁴⁷). However, despite these differences, we find that individuals’ posterior mean Q-learning parameters from (i) dual learning rate models fit to choices alone with MCMC, and (ii) the best-fitting joint RL-affect model fit choices and affect ratings simultaneously with ADVI are highly correlated with those we previously reported from Q-learning models fit to choices alone³⁴ in the same sample (α_reward: r=0.61 [95% CI = (0.57, 0.65)]; α_loss: r=0.67 [95% CI: 0.63, 0.70]; β: r = 0.74 [95% CI = (0.70, 0.76)]; Figure S2A). We also replicate a key result from our earlier work: higher inverse temperatures (β) in the distancing group (Figure S2B-C).

Figure S2

Supplementary Results

Group-level parameter estimates for the best-fitting joint RL-affect model

At the group-level, model-predicted baseline affect () was highest for engagement (group-level mean = 65.9 points; 95% HDI = [65.9, 66.0]), followed by happiness (group-level mean = 56.5, 95% HDI = [56.4, 56.6]), and lowest for confidence (group-level mean = 32.2, 95% HDI = [32.1, 32.2]). Affect drift over time () was steepest for engagement: an estimated 81.2% lower odds of increased engagement per hour (all other terms being equal; 95% HDI for multiplier = [0.187, 0.189]), compared to 54.1% lower confidence (95% HDI for multiplier = [0.457, 0.462]) and 37.7% lower happiness (95% HDI for multiplier = [0.620, 0.625]) per hour. Group-level means for weights on recent expected values () and prediction errors () were positive for all three ratings, showing higher affect with increased recent rewards, and were lowest for engagement (posterior mean [95% HDI] for multiplier: = 1.200 [1.198, 1.201], = 1.216 [1.216, 1.218], = 1.116 [1.114, 1.116]; = 1.136 [1.134, 1.136], = 1.208 [1.207, 1.210], = 1.109 [1.108, 1.109]). Lastly, the average decay factor was highest for confidence (group-level mean = 0.563; 95% HDI = [0.561, 0.565]), and lowest for happiness (group-level mean=0.407; 95% HDI = [0.405, 0.409]), suggesting higher weighting of outcomes of earlier trials (i.e., prior to the most recent trial) in confidence judgements compared to happiness ratings.

Compulsive behaviour and social withdrawal factor scores are also associated with altered affective dynamics

In addition to differences in baseline affect and its drift over time detailed in the main text, there was evidence that participants with higher compulsive behaviour scores placed more weight on recent expected values (higher ; 95% HDI excluding zero for only) and prediction errors (higher ) in their subjective affect judgements (Figure S3i–iii). The consequences of this were evident in the observed data. For example, there was evidence of a weak positive correlation between happiness rating variability and compulsive behaviour factor scores (correlation coefficient [95% CI] r = 0.15 [0.086, 0.212], p < 0.0001; Figure S3D), which can also be qualitatively observed by comparing mean-centred happiness ratings for participants in the bottom quartile versus the upper quartile of compulsive behaviour factor scores (Figure S3E). That said, the actual effect of the estimated differences in and on ratings is small, as shown by simulating happiness ratings for individuals who differ only in having the estimated and for those with the 25^th percentile versus the 75^th percentile compulsive behaviour factor score (Figure S3E).

Figure S3

We additionally found some weak evidence that increases in anxiety/depression factor were associated with slightly higher weighting of previous trials’ expected values and prediction errors for happiness (γ_happy; e.g., trial-before-last weighted an estimated 4.04% higher; 95% HDI for multiplier = [1.003, 1.079]; Figure S3Aiii). There was also some stronger evidence for a positive association between social withdrawal factor score and decay factors for happiness and engagement (Figure S3Ciii), suggesting marginally higher weighting of previous trials’ expected values and prediction errors in the computation of affect ratings in those with higher levels social withdrawal symptoms.

Antidepressant use is associated with increased weighting of previous choices’ expected values in affect ratings

To further unpick the effects of treatments on the weighting of previous outcomes, we fit a more flexible between-rating RL-affect model (equation 7). This model, which allowed for different weights on previous expected values () and prediction errors () since the previous rating (up to five trials back), was able to capture the ratings well, albeit with marginally worse accuracy than the winning drift over time model (mean [SD] pseudo-R² for between-rating RL-affect model = 0.39 [0.22–0.25] across all three ratings). We then related and from each rating type separately to both treatments via multilevel GLMs with participant-level random intercepts and slopes (on trial lag), adjusting for age, gender, and digit span as before.

Parameters from this between-rating model suggested limited evidence for a difference between distancing and non-distancing participants in the weighting of the most recent or intervening outcomes in their affective judgements (Figure S4A). There was also no evidence of an effect of either treatment on weightings of prediction errors from previous trials (Figure S4Aiii–iv & Figure S4Biii–iv). There was, however, some evidence of a small effect of antidepressant use on between-rating changes in affect: higher weighting of the most recent expected value in subjective affect ratings (Figure S4Bi). The evidence for this was strongest for engagement, with a unit increase in the most recent Q-value associated with 4.33% higher odds of an increase in engagement rating (95% HDI for multiplier = [1.001, 1.089]). Furthermore, there was limited evidence of an accompanying interaction effect (Figure S4Bii), suggesting the contribution of less recent expected values to engagement ratings was also marginally higher in participants taking antidepressants, which may in turn explain the higher forgetting factor γ_engaged (Figure 3Bv).

Figure S4

Affective drift over time is associated with self-reported fatigue

Previous work on ‘mood drift over time’ has suggested it is mostly distinct from boredom and mind wandering⁴⁴. Here, were able to test an additional aspect of this phenomenon, namely its relation to fatigue, as we asked participants the following question after the end of each of the six blocks: “How fatigued do you feel compared to the beginning of the block?”. We hence examined the association between and both participants’ overall mean post-block fatigue and their change in post-block fatigue ratings over the course of the six training blocks. To quantify change in post-block fatigue, the six ratings were regressed on block number for each participant, with the regression coefficient (β_Δfatigue) on block number taken as the quantity of interest (i.e., higher values suggest increases in post-block fatigue ratings over time).

We found strong evidence, after adjusting for age, gender, digit span, and distancing group, that higher mean post-block fatigue ratings were associated with lower baseline affect (lower ; Figure S5Ai) and decreased odds of higher affect ratings across the task (lower ; Figure S5Aii), across all three affect rating types (e.g., estimated mean 18.9% lower odds of increased happiness across the task with a ten-point increase in mean post-block fatigue; 95% HDI for multiplier = [0.767, 0.856]). In addition, regression coefficients capturing change in post-block fatigue were strongly negatively associated with across all rating types after adjusting for mean post-block fatigue, most strongly for engagement (estimated mean 5.74% lower odds of increase in engagement for a one-point per block in fatigue rating rate-of-change, 95% HDI for multiplier = [0.944, 0.968]; Figure S5Bii). Notably, associations in the opposite (positive) direction were observed between β_Δfatigue and baseline affect, again most strongly for engagement (estimated 0.714-point increase in engagement rating for a one-point per block increase in fatigue rating rate-of-change; 95% HDI = [0.452, 0.975]; Figure S5Bi). Speculatively, this may represent an effect of motivation, where participants who were more engaged towards the beginning of the task also exerted more effort, resulting in larger overall increases in fatigue and decreases in engagement.

Figure S5

Data and code availability

All code to replicate the analyses here can be found in accompanying Jupyter notebooks, alongside cleaned, anonymised data. See the GitHub repository for more details.

Acknowledgements

This study was funded by an AXA Research Fund Fellowship awarded to C.L.N. (G102329) and the Medical Research Council (MC_UU_00030/12). C.L.N. is funded by a Wellcome Career Development Award (226490/Z/22/Z) and acknowledges support by the NIHR Cambridge NIHR Biomedical Research Centre (BRC-1215-20014). Q.D. is funded by a Wellcome Trust PhD studentship. Q.D. and Q.J.M.H acknowledge support by the NIHR UCLH BRC. Q.J.M.H. acknowledges grant funding from the NIHR, Wellcome Trust, Carigest S.A. and Koa Health. R.B.R. is supported by the National Institute of Mental Health (R01MH124110). R.B.R. holds equity in Maia.

Additional information

Rights retention

For the purpose of open access, the author has applied a Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising from this submission.

Acronyms

• ADVI: Automatic Differentiation Variational Inference

• BCa: Bias-Corrected and accelerated

• CI: Confidence Interval

• ELPD: Expected Log Pointwise Predictive Density

• GLM: Generalised Linear Model

• HDI: Highest Density Interval

• HBREC: Human Biology Research Ethics Committee

• LOO: Leave One Out

• MSE: Mean Squared Error

• MCMC: Markov Chain Monte Carlo

• NHS: National Health Service

• PLS: Partial Least Squares

• RL: Reinforcement Learning

• SD: Standard Deviation

• SSRI: Selective Serotonin Reuptake Inhibitor