Characterizing a psychiatric symptom dimension related to deficits in goal-directed control
Figures

Two-step reinforcement learning task used to assess goal-directed (model-based) learning.
(a) Subjects chose between two fractals, which probabilistically determined whether they would transition to the orange or blue second stage state. For example, the fractal on the left had a 70% chance of leading to the blue second stage state (‘common’ transition) and a 30% chance of leading to the orange state (‘rare’ transition). These transition probabilities were fixed and could be learned over time. In the second stage state, subjects chose between two fractals, each of which was associated with a distinct probability of being rewarded with a 25 cents coin. The probability of receiving a reward associated with each second stage fractal could also be learned, but (unlike the transition structure) these drifted slowly over time (0.25 < P <0.75, panel b). This meant that in order to earn the most rewards possible, subjects had to track which second stage fractals were currently best as they changed over time. Reward probabilities depicted (34%, 68%, 72%, 67%) refer to example trial 50, denoted by the vertical dashed line in (b). (b) Drifting reward probabilities determined by Gaussian Random Walks for 200 trials with grey horizontal lines indicating boundaries at 0.25 and 0.75. (c) Schematic representing the performance of a purely ‘model-free’ learner, who only exhibits sensitivity to whether or not the previous trial was rewarded vs. unrewarded, and does not modify their behavior in light of the transition that preceded reward. (d) Schematic representing the performance of a purely ‘model-based’ learner, who is more likely to repeat an action (i.e. ‘stay’) following a rewarded trial, only if the transition was common. If the transition to that rewarded state was rare, they are more likely to switch on the next trial.

Associations between Goal-directed (model-based) deficits and self-reported psychopathology.
The y-axes indicate the% change in model-based learning for each change of 1 standard deviation (SD) of clinical symptoms. Error bars denote standard error. (a) In Experiment 1, total scores on a self-report questionnaire assessing OCD symptoms in a general population sample were associated with deficits in goal-directed (model-based) learning. Specifically, for each increase of 1 SD in OCD symptoms reported, model-based learning was 14% lower than the group mean. No effects were observed in depression or trait anxiety. (b) In Experiment 2, the results from Experiment 1 were replicated: OCD symptoms were associated with deficits in goal-directed learning, while total scores on questionnaires assessing depression and trait anxiety were not. We found that the association between compulsive behavior and goal-directed deficits generalized to symptoms associated with other disorders that are similarly characterized by a loss of control over behavior, alcohol addiction, eating disorders and impulsivity. No significant effects were observed for scores on questionnaires assessing schizotypy, depression, trait anxiety, apathy or social anxiety.

Trans-diagnostic factors.
(a) Factor analysis on the correlation matrix of 209 questionnaire items suggested that 3-factor solution best explained these data. Factors were ‘Anxious-Depression’, ‘Compulsive Behavior and Intrusive Thought’ and ‘Social Withdrawal’. Item loadings for each factor are presented on the top, left and bottom sides of the correlation matrix, color-codes indicate the questionnaire from which each item was drawn. (b) These factors were entered into mixed-effects models, revealing that only the Factor 2 ‘Compulsive Behavior and Intrusive Thought’ was associated with goal-directed deficits, the effect size (17% reduction in model-based learning for every 1 SD increase in ‘Compulsive Behavior and Intrusive Thought’) was larger than for any individual questionnaire, and pairwise contrasts revealed that these deficits were specific to this factor, compared to Factor 1 ‘Anxious-Depression’ and Factor 3 ‘Social Withdrawal’. The y-axes indicate the% change in model-based learning for each change of 1 standard deviation (SD) of clinical symptomatology. Error bars denote standard error.

Behavioral data from experiments 1 (N=548) and 2 (N=1413).
Error bars denote standard error. Data illustrate that consistent with previous studies (Daw et al., 2011), participants use a mixture of model-based and model-free learning to guide choice. Associated statistics are presented in Supplementary file 1B. *p<0.05 **p<0.01 ***p<0.001.

Scree plot of eigenvalues.
The outer frame shows the eigenvalues for every possible factor solution, N=209. Inset is data for the first 20 potential factor solutions only. An empirically defined elbow, where Eigenvalues begin to level out, was identified at factor 4 using the nFactors package in R, provideing evidence for a 3-factor solution (Cattell, 1966), indicated in orange.*p<0.05 **p<0.01 ***p<0.001.

Plotted are the regression lines for 18 different analyses, where the population was split into the top 25% and bottom 75% for each of the nine clinical questionnaires.
https://doi.org/10.7554/eLife.11305.016Tables
Self-report questionnaire total scores and model-based learning.
Questionnaire | β (SE) | z-value | p-value |
---|---|---|---|
Experiment 1 (N=548) | |||
OCD | -0.040 (0.02) | -1.97 | 0.049 * |
Depression | -0.016 (0.02) | -0.77 | 0.439 |
Trait Anxiety | -0.006 (0.02) | -0.28 | 0.778 |
Experiment 2 (N=1413) | |||
Eating Disorders | -0.037 (0.01) | -3.30 | <0.001 *** |
Impulsivity | -0.034 (0.01) | -3.01 | 0.007 ** |
OCD | -0.026 (0.01) | -2.34 | 0.020 * |
Alcohol Addiction | -0.025 (0.01) | -2.18 | 0.029 * |
Schizotypy | -0.017 (0.01) | -1.48 | 0.14 |
Depression | -0.010 (0.01) | -0.87 | 0.385 |
Trait Anxiety | -0.008 (0.01) | -0.68 | 0.498 |
Apathy | -0.001 (0.01) | -0.06 | 0.953 |
Social Anxiety | 0.008 (0.01) | 0.68 | 0.496 |
-
*p<0.05; **p<0.01; ***p<0.001.
-
SE=standard error.
-
Each row reflects the results from an independent analysis where each questionnaire total score (z-transformed) was entered as SymptomScorez in the following model: glmer(Stay ~ Reward * Transition * SymptomScorez + Reward * Transition * (IQz + Agez + Gender) + (Reward * Transition + 1 | Subject)). Model-based statistics refer to the following interaction: SymptomScorez x Reward x Transition. For each, positive β values indicate that the symptom score is associated with greater model-based learning, while negative β values indicate that the symptom score is associated with reduced model-based learning.
Means and standard deviations (in parentheses) of loadings onto Factor 1 ‘Anxious-Depression’, Factor 2 ‘Compulsive Behavior and Intrusive Thought’ and Factor 3 ‘Social Withdrawal’ factors for each questionnaire.
Anxious-depression | Compulsive behavior and intrusive thought | Social withdrawal | |
---|---|---|---|
(Factor 1) | (Factor 2) | (Factor 3) | |
Alcohol addiction | 0.15 (0.05) | 0.31 (0.07) | -0.23 (0.06) |
Apathy | 0.44 (0.16) | -0.05 (0.13) | 0.04 (0.13) |
Depression | 0.38 (0.23) | 0.14 (0.14) | 0.04 (0.06) |
Eating disorders | -0.05 (0.10) | 0.36 (0.15) | 0.06 (0.06) |
Impulsivity | 0.24 (0.22) | 0.15 (0.15) | -0.11 (0.11) |
OCD | -0.05 (0.14) | 0.50 (0.06) | 0.09 (0.07) |
Schizotypy | 0.16 (0.11) | 0.18 (0.13) | 0.08 (0.14) |
Social anxiety | 0.04 (0.05) | 0.08 (0.09) | 0.57 (0.14) |
Trait anxiety | 0.52 (0.17) | 0.15 (0.16) | 0.13 (0.08) |
-
Scores greater than 0.25 are emboldened to highlight the dominant constructs.
Trans-diagnostic factors and model-based learning.
Construct | β (SE) | z-value | p-value |
---|---|---|---|
Independent Models | |||
‘Anxious-Depression’ (Factor 1) | -0.001(0.01) | 0.10 | 0.920 |
‘Compulsive Behavior and Intrusive Thought’ (Factor 2) | -0.046(0.01) | -4.06 | <0.001 *** |
‘Social Withdrawal’ (Factor 3) | 0.013(0.01) | 1.18 | 0.238 |
Covariate Model | |||
‘Anxious-Depression’ (Factor 1) | 0.003(0.01) | 0.28 | 0.781 |
‘Compulsive Behavior and Intrusive Thought’ (Factor 2) | -0.058(0.01) | -4.71 | <0.001 *** |
‘Social Withdrawal’ (Factor 3) | 0.031(0.01) | 2.45 | 0.014* |
-
*p<0.05; **p<0.01; ***p<0.001.
-
SE=standard error.
-
Top panel shows results from Independent Models. Bottom panel shows results from Covariate Model, where trans-diagnostic factors were entered together into the same model: glmer(Stay ~ Reward * Transition * (Factor1z + Factor2z + Factor3z + IQz + Agez + Gender) + (Reward * Transition + 1 | Subject)). Statistics refer to the interaction between scores on each factor and Reward x Transition, i.e. the extent to which that score is associated with changes in model-based learning. Positive β values indicate that the symptom score is associated with greater model-based learning, while negative β values indicate that the symptom score is associated with reduced model-based learning.
Each row reflects the results from an independent analysis where each questionnaire total score (z-transformed) was entered as SymptomScorez in the following model: lm(SymptomScorez ~ Agez + Genderz+ IQz + ModelBasedScore). ModelBasedScore was derived from from the one-trial back regression (first three columns) or the computational model (last three columns results). For each, positive β values indicate that the ModelBasedScore is associated with fewer symptoms, whereas negative β values indicate that the symptom score is associated with increased symptoms.
One-Trial Back Regression | Computational model | |||||
---|---|---|---|---|---|---|
Clinical Scores | β (SE) | p-value | R2 | β (SE) | p-value | R2 |
Eating Disorders | -0.09(0.03) | <.001*** | .042 | -0.09(0.03) | <.001*** | .043 |
Impulsivity | -0.09(0.03) | .002** | .028 | -0.10(0.03) | <.001*** | .032 |
OCD | -0.07(0.03) | .012* | .050 | -0.07(0.03) | 0.005** | .051 |
Alcohol Addiction | -0.06(0.03) | .029* | .052 | -0.06(0.03) | .028* | .052 |
Schizotypy | -0.04(0.03) | .101 | .028 | -0.05(0.03) | .044* | .029 |
Depression | -0.03(0.03) | .351 | .031 | -0.05 (0.03) | .09 | .033 |
Trait Anxiety | -0.02(0.03) | .552 | .038 | -0.03(0.03) | .221 | .038 |
Apathy | -0.00(0.03) | .897 | .015 | -0.02(0.03) | .584 | .015 |
Social Anxiety | 0.01(0.03) | .593 | .028 | 0.01(0.03) | .775 | .028 |
Anxious-Depression | -0.00(0.03) | .967 | .018 | -0.02 (0.03) | .474 | .018 |
Compulsive Behavior and Thought | -0.11(0.03) | <.001*** | .088 | -0.12(0.03) | <.001*** | .089 |
Social Withdrawal | 0.03(0.03) | .282 | .036 | 0.02(0.03) | .440 | .036 |
Additional files
-
Supplementary file 1
(A) Questionnaire Total Scores in Experiments 1 and 2.
a For the purposes of comparison across studies using the EAT, responses on the 6-point scale are converted as follows: (1:0, 2:0, 3:0, 4:1, 5:2, 6:3) and totaled to produce the mean reported above (Everitt and Robbins, 2005). For analysis purposes (including correlations reported above), however, the continuous (i.e. 1,2,3,4,5,6) values were used (M=33.31, SD=17.36). Likewise, for LSAS, the mean reported above corresponds to the summed total of the full questionnaire - i.e. including answers to both avoidance and fear/anxiety probes. However, for the purposes of analyses, we used the average of the avoidance and fear/anxiety answers for each item. b Positive t-values indicate higher scores among males, while negative t-values indicate higher scores in females. (B) Results from Basic Logistic Regression Model in Experiment 1 (N=548) and Experiment 2 (N=1413) with Age, Gender and IQ as fixed effects predictors. *p<0.05 ** p<0.01 ***p<0.001SE=standard error. (C) Questionnaire Total Scores and Model-Free Learning*p<0.05 ** p<0.01 ***p<0.001SE=standard error. Each row reflects the results from an independent analysis where each questionnaire total score (z-transformed) was entered as SymptomScorez in the following model: glmer(Stay ~ Reward * Transition * SymptomScorez + Reward * Transition * (IQz + Agez + Gender) + (Reward * Transition + 1 | Subject)). Model-free statistics refer to the following interaction: SymptomScorez x Reward. For each, positive β values indicate that the symptom score is associated with greater model-free learning, while negative β values indicate that the symptom score is associated with reduced model-free learning.
- https://doi.org/10.7554/eLife.11305.011
-
Supplementary file 2
(A) ‘Top loading items on ‘Anxious-Depression’ factor
Summary of item loadings onto ‘Anxious-Depression’ factor. The top loading items from each questionnaire are displayed in descending order, provided they are above a threshold loading of +/- 0.25. Words in parentheses, e.g. “(do not)” are added here (but were not presented to participants) to facilitate interpretation of the direction of effects for items that are reverse-coded. (B) ‘Top loading items on ‘Compulsive Behavior and Intrusive Thought’. Summary of item loadings onto ‘Compulsive Behavior and Intrusive Thought’ factor. The top loading items from each questionnaire are displayed in descending order, provided they are above a threshold loading of +/- 0.25. (C) ‘Top loading items on ‘Social Withdrawal’ factorSummary of item loadings onto ‘Social Withdrawal’ factor. The top loading items from each questionnaire are displayed in descending order, provided they are above a threshold loading of +/- 0.25. Words in parentheses, e.g. '(do not)' are added here (but were not presented to participants) to facilitate interpretation of the direction of effects for items that are reverse-coded.
- https://doi.org/10.7554/eLife.11305.012
-
Supplementary file 3
The relationship between model-based learning and Factor 2 is broadly consistent across ‘putative patients’ (top 25%) and subjects scoring in the normal range (bottom 75%).
Plotted here are regression lines indicating the strength of the relationship between model-based deficits and Factor 2 (‘Compulsive Behavior and Intrusive Thought’) in putative patients (top 25%, blue) and in subjects in the normal range (bottom 75%, red). Each subplot reflects a different subset of the population, based on that clinical questionnaire. Eighteen independent analyses were subsequently carried out (i.e. two per subplot).
- https://doi.org/10.7554/eLife.11305.013
-
Supplementary file 4
Significant Predictors of Goal-Directed (Model-Based) Learning from Supervised Analysis
Features that are significantly associated with model-based learning identified using elastic net regularization with tenfold cross-validation, observed in >95% of 100 iterations tested. Index refers to the item number from the questionnaire of origin. Beta refers to the coefficient from the regularized regression model. Words in parentheses, e.g. '(do not)' are added here (but were not presented to participants) to facilitate interpretation of the direction of effects for items that are reverse coded. The last column 'FA Loadings' indicates the significant overlap in terms of loading on Factors F1 (‘Anxious-Depression’), F2 (‘Compulsive Behavior and Intrusive Thought’) or F3 (‘Social Withdrawal), in the positive (+) or negative (-) direction using a cut-off at loadings >= 0.25.
- https://doi.org/10.7554/eLife.11305.014
-
Supplementary file 5
(A) Association between questionnaire total scores and model-based learning defined using full computational model.
*p<0.05 ** p<0.01 ***p<0.001. Each row reflects the results from an independent analysis where each questionnaire total score (z-transformed) was entered as SymptomScorez in the following model: lm(Model-Based-Learning ~ IQz + Agez + Gender + SymptomScorez). Statistics refer to the main effect of SymptomScorez on Model-Based-Learning, i.e. the extent to which that questionnaire total score is associated with changes in model-based learning (which was defined for each participant using the full computational model). (B) Comparing the predictive power of model-based learning defined using the computational model versus one-trial back regression analysis. Each row reflects the results from an independent analysis where each questionnaire total score (z-transformed) was entered as SymptomScorez in the following model: lm(SymptomScorez ~ ModelBasedScore). Prior to conducting these analyses, we regressed out the effects of age, gender and IQ so that we could directly compare the r2 of the models. ModelBasedScore was derived from the one-trial back regression (first three columns) or the computational model (last three columns results). For each, positive β values indicate that the ModelBasedScore is associated with fewer symptoms, whereas negative β values indicate that the symptom score is associated with increased symptoms. The difference between the two approaches is negligible. However the computational model did produce nominally higher r2 and lower p-values for the relationship between clinical scores and model-based learning.
- https://doi.org/10.7554/eLife.11305.015