1. Human Biology and Medicine
  2. Neuroscience
Download icon

Characterizing a psychiatric symptom dimension related to deficits in goal-directed control

  1. Claire M Gillan  Is a corresponding author
  2. Michal Kosinski
  3. Robert Whelan
  4. Elizabeth A Phelps
  5. Nathaniel D Daw
  1. New York University, United States
  2. University of Cambridge, United Kingdom
  3. Stanford University, United States
  4. University College Dublin, Ireland
  5. Nathan Kline Institute, United States
  6. Princeton University, United States
Research Article
Cite this article as: eLife 2016;5:e11305 doi: 10.7554/eLife.11305
6 figures, 4 tables and 5 additional files

Figures

Two-step reinforcement learning task used to assess goal-directed (model-based) learning.

(a) Subjects chose between two fractals, which probabilistically determined whether they would transition to the orange or blue second stage state. For example, the fractal on the left had a 70% chance of leading to the blue second stage state (‘common’ transition) and a 30% chance of leading to the orange state (‘rare’ transition). These transition probabilities were fixed and could be learned over time. In the second stage state, subjects chose between two fractals, each of which was associated with a distinct probability of being rewarded with a 25 cents coin. The probability of receiving a reward associated with each second stage fractal could also be learned, but (unlike the transition structure) these drifted slowly over time (0.25 < P <0.75, panel b). This meant that in order to earn the most rewards possible, subjects had to track which second stage fractals were currently best as they changed over time. Reward probabilities depicted (34%, 68%, 72%, 67%) refer to example trial 50, denoted by the vertical dashed line in (b). (b) Drifting reward probabilities determined by Gaussian Random Walks for 200 trials with grey horizontal lines indicating boundaries at 0.25 and 0.75. (c) Schematic representing the performance of a purely ‘model-free’ learner, who only exhibits sensitivity to whether or not the previous trial was rewarded vs. unrewarded, and does not modify their behavior in light of the transition that preceded reward. (d) Schematic representing the performance of a purely ‘model-based’ learner, who is more likely to repeat an action (i.e. ‘stay’) following a rewarded trial, only if the transition was common. If the transition to that rewarded state was rare, they are more likely to switch on the next trial.

https://doi.org/10.7554/eLife.11305.003
Associations between Goal-directed (model-based) deficits and self-reported psychopathology.

The y-axes indicate the% change in model-based learning for each change of 1 standard deviation (SD) of clinical symptoms. Error bars denote standard error. (a) In Experiment 1, total scores on a self-report questionnaire assessing OCD symptoms in a general population sample were associated with deficits in goal-directed (model-based) learning. Specifically, for each increase of 1 SD in OCD symptoms reported, model-based learning was 14% lower than the group mean. No effects were observed in depression or trait anxiety. (b) In Experiment 2, the results from Experiment 1 were replicated: OCD symptoms were associated with deficits in goal-directed learning, while total scores on questionnaires assessing depression and trait anxiety were not. We found that the association between compulsive behavior and goal-directed deficits generalized to symptoms associated with other disorders that are similarly characterized by a loss of control over behavior, alcohol addiction, eating disorders and impulsivity. No significant effects were observed for scores on questionnaires assessing schizotypy, depression, trait anxiety, apathy or social anxiety.

https://doi.org/10.7554/eLife.11305.005
Trans-diagnostic factors.

(a) Factor analysis on the correlation matrix of 209 questionnaire items suggested that 3-factor solution best explained these data. Factors were ‘Anxious-Depression’, ‘Compulsive Behavior and Intrusive Thought’ and ‘Social Withdrawal’. Item loadings for each factor are presented on the top, left and bottom sides of the correlation matrix, color-codes indicate the questionnaire from which each item was drawn. (b) These factors were entered into mixed-effects models, revealing that only the Factor 2 ‘Compulsive Behavior and Intrusive Thought’ was associated with goal-directed deficits, the effect size (17% reduction in model-based learning for every 1 SD increase in ‘Compulsive Behavior and Intrusive Thought’) was larger than for any individual questionnaire, and pairwise contrasts revealed that these deficits were specific to this factor, compared to Factor 1 ‘Anxious-Depression’ and Factor 3 ‘Social Withdrawal’. The y-axes indicate the% change in model-based learning for each change of 1 standard deviation (SD) of clinical symptomatology. Error bars denote standard error.

https://doi.org/10.7554/eLife.11305.007
Behavioral data from experiments 1 (N=548) and 2 (N=1413).

Error bars denote standard error. Data illustrate that consistent with previous studies (Daw et al., 2011), participants use a mixture of model-based and model-free learning to guide choice. Associated statistics are presented in Supplementary file 1B. *p<0.05 **p<0.01 ***p<0.001.

https://doi.org/10.7554/eLife.11305.009
Scree plot of eigenvalues.

The outer frame shows the eigenvalues for every possible factor solution, N=209. Inset is data for the first 20 potential factor solutions only. An empirically defined elbow, where Eigenvalues begin to level out, was identified at factor 4 using the nFactors package in R, provideing evidence for a 3-factor solution (Cattell, 1966), indicated in orange.*p<0.05 **p<0.01 ***p<0.001.

https://doi.org/10.7554/eLife.11305.010
Author response image 1
Plotted are the regression lines for 18 different analyses, where the population was split into the top 25% and bottom 75% for each of the nine clinical questionnaires.
https://doi.org/10.7554/eLife.11305.016

Tables

Table 1

Self-report questionnaire total scores and model-based learning.

https://doi.org/10.7554/eLife.11305.004
Questionnaireβ (SE)

z-value

p-value
Experiment 1 (N=548)
OCD-0.040 (0.02)-1.970.049 *
Depression-0.016 (0.02)-0.770.439
Trait Anxiety-0.006 (0.02)-0.280.778
Experiment 2 (N=1413)
Eating Disorders-0.037 (0.01)-3.30<0.001 ***
Impulsivity-0.034 (0.01)-3.010.007 **
OCD-0.026 (0.01)-2.340.020 *
Alcohol Addiction-0.025 (0.01)-2.180.029 *
Schizotypy-0.017 (0.01)-1.480.14
Depression-0.010 (0.01)-0.870.385
Trait Anxiety-0.008 (0.01)-0.680.498
Apathy-0.001 (0.01)-0.060.953
Social Anxiety0.008 (0.01)0.680.496
  1. *p<0.05; **p<0.01; ***p<0.001.

  2. SE=standard error.

  3. Each row reflects the results from an independent analysis where each questionnaire total score (z-transformed) was entered as SymptomScorez in the following model: glmer(Stay ~ Reward * Transition * SymptomScorez + Reward * Transition * (IQz + Agez + Gender) + (Reward * Transition + 1 | Subject)). Model-based statistics refer to the following interaction: SymptomScorez x Reward x Transition. For each, positive β values indicate that the symptom score is associated with greater model-based learning, while negative β values indicate that the symptom score is associated with reduced model-based learning.

Table 2

Means and standard deviations (in parentheses) of loadings onto Factor 1 ‘Anxious-Depression’, Factor 2 ‘Compulsive Behavior and Intrusive Thought’ and Factor 3 ‘Social Withdrawal’ factors for each questionnaire.

https://doi.org/10.7554/eLife.11305.006
Anxious-depressionCompulsive behavior and intrusive thoughtSocial withdrawal
(Factor 1)(Factor 2)(Factor 3)
Alcohol addiction0.15 (0.05)0.31 (0.07)-0.23 (0.06)
Apathy0.44 (0.16)-0.05 (0.13)0.04 (0.13)
Depression0.38 (0.23)0.14 (0.14)0.04 (0.06)
Eating disorders-0.05 (0.10)0.36 (0.15)0.06 (0.06)
Impulsivity0.24 (0.22)0.15 (0.15)-0.11 (0.11)
OCD-0.05 (0.14)0.50 (0.06)0.09 (0.07)
Schizotypy0.16 (0.11)0.18 (0.13)0.08 (0.14)
Social anxiety0.04 (0.05)0.08 (0.09)0.57 (0.14)
Trait anxiety0.52 (0.17)0.15 (0.16)0.13 (0.08)
  1. Scores greater than 0.25 are emboldened to highlight the dominant constructs.

Table 3

Trans-diagnostic factors and model-based learning.

https://doi.org/10.7554/eLife.11305.008
Constructβ (SE)z-valuep-value
Independent Models
‘Anxious-Depression’ (Factor 1)-0.001(0.01)0.100.920
‘Compulsive Behavior and Intrusive Thought’ (Factor 2)-0.046(0.01)-4.06<0.001 ***
‘Social Withdrawal’ (Factor 3)0.013(0.01)1.180.238
Covariate Model
‘Anxious-Depression’ (Factor 1)0.003(0.01)0.280.781
‘Compulsive Behavior and Intrusive Thought’ (Factor 2)-0.058(0.01)-4.71<0.001 ***
‘Social Withdrawal’ (Factor 3)0.031(0.01)2.450.014*
  1. *p<0.05; **p<0.01; ***p<0.001.

  2. SE=standard error.

  3. Top panel shows results from Independent Models. Bottom panel shows results from Covariate Model, where trans-diagnostic factors were entered together into the same model: glmer(Stay ~ Reward * Transition * (Factor1z + Factor2z + Factor3z + IQz + Agez + Gender) + (Reward * Transition + 1 | Subject)). Statistics refer to the interaction between scores on each factor and Reward x Transition, i.e. the extent to which that score is associated with changes in model-based learning. Positive β values indicate that the symptom score is associated with greater model-based learning, while negative β values indicate that the symptom score is associated with reduced model-based learning.

Author response table 1

Each row reflects the results from an independent analysis where each questionnaire total score (z-transformed) was entered as SymptomScorez in the following model: lm(SymptomScorez ~ Agez + Genderz+ IQz + ModelBasedScore). ModelBasedScore was derived from from the one-trial back regression (first three columns) or the computational model (last three columns results). For each, positive β values indicate that the ModelBasedScore is associated with fewer symptoms, whereas negative β values indicate that the symptom score is associated with increased symptoms.

https://doi.org/10.7554/eLife.11305.017

One-Trial Back Regression

Computational model

Clinical Scores

β (SE)p-valueR2β (SE)p-valueR2

Eating Disorders

-0.09(0.03)

<.001***

.042

-0.09(0.03)

<.001***

.043

Impulsivity

-0.09(0.03)

.002**

.028

-0.10(0.03)

<.001***

.032

OCD

-0.07(0.03)

.012*

.050

-0.07(0.03)

0.005**

.051

Alcohol Addiction

-0.06(0.03)

.029*

.052

-0.06(0.03)

.028*

.052

Schizotypy

-0.04(0.03)

.101

.028

-0.05(0.03)

.044*

.029

Depression

-0.03(0.03)

.351

.031

-0.05 (0.03)

.09

.033

Trait Anxiety

-0.02(0.03)

.552

.038

-0.03(0.03)

.221

.038

Apathy

-0.00(0.03)

.897

.015

-0.02(0.03)

.584

.015

Social Anxiety

0.01(0.03)

.593

.028

0.01(0.03)

.775

.028

Anxious-Depression

-0.00(0.03)

.967

.018

-0.02 (0.03)

.474

.018

Compulsive Behavior and Thought

-0.11(0.03)

<.001***

.088

-0.12(0.03)

<.001***

.089

Social Withdrawal

0.03(0.03)

.282

.036

0.02(0.03)

.440

.036

Additional files

Supplementary file 1

(A) Questionnaire Total Scores in Experiments 1 and 2.

a For the purposes of comparison across studies using the EAT, responses on the 6-point scale are converted as follows: (1:0, 2:0, 3:0, 4:1, 5:2, 6:3) and totaled to produce the mean reported above (Everitt and Robbins, 2005). For analysis purposes (including correlations reported above), however, the continuous (i.e. 1,2,3,4,5,6) values were used (M=33.31, SD=17.36). Likewise, for LSAS, the mean reported above corresponds to the summed total of the full questionnaire - i.e. including answers to both avoidance and fear/anxiety probes. However, for the purposes of analyses, we used the average of the avoidance and fear/anxiety answers for each item. b Positive t-values indicate higher scores among males, while negative t-values indicate higher scores in females. (B) Results from Basic Logistic Regression Model in Experiment 1 (N=548) and Experiment 2 (N=1413) with Age, Gender and IQ as fixed effects predictors. *p<0.05 ** p<0.01 ***p<0.001SE=standard error. (C) Questionnaire Total Scores and Model-Free Learning*p<0.05 ** p<0.01 ***p<0.001SE=standard error. Each row reflects the results from an independent analysis where each questionnaire total score (z-transformed) was entered as SymptomScorez in the following model: glmer(Stay ~ Reward * Transition * SymptomScorez + Reward * Transition * (IQz + Agez + Gender) + (Reward * Transition + 1 | Subject)). Model-free statistics refer to the following interaction: SymptomScorez x Reward. For each, positive β values indicate that the symptom score is associated with greater model-free learning, while negative β values indicate that the symptom score is associated with reduced model-free learning.

https://doi.org/10.7554/eLife.11305.011
Supplementary file 2

(A) ‘Top loading items on ‘Anxious-Depression’ factor

Summary of item loadings onto ‘Anxious-Depression’ factor. The top loading items from each questionnaire are displayed in descending order, provided they are above a threshold loading of +/- 0.25. Words in parentheses, e.g. “(do not)” are added here (but were not presented to participants) to facilitate interpretation of the direction of effects for items that are reverse-coded. (B) ‘Top loading items on ‘Compulsive Behavior and Intrusive Thought’. Summary of item loadings onto ‘Compulsive Behavior and Intrusive Thought’ factor. The top loading items from each questionnaire are displayed in descending order, provided they are above a threshold loading of +/- 0.25. (C) ‘Top loading items on ‘Social Withdrawal’ factorSummary of item loadings onto ‘Social Withdrawal’ factor. The top loading items from each questionnaire are displayed in descending order, provided they are above a threshold loading of +/- 0.25. Words in parentheses, e.g. '(do not)' are added here (but were not presented to participants) to facilitate interpretation of the direction of effects for items that are reverse-coded.

https://doi.org/10.7554/eLife.11305.012
Supplementary file 3

The relationship between model-based learning and Factor 2 is broadly consistent across ‘putative patients’ (top 25%) and subjects scoring in the normal range (bottom 75%).

Plotted here are regression lines indicating the strength of the relationship between model-based deficits and Factor 2 (‘Compulsive Behavior and Intrusive Thought’) in putative patients (top 25%, blue) and in subjects in the normal range (bottom 75%, red). Each subplot reflects a different subset of the population, based on that clinical questionnaire. Eighteen independent analyses were subsequently carried out (i.e. two per subplot).

https://doi.org/10.7554/eLife.11305.013
Supplementary file 4

Significant Predictors of Goal-Directed (Model-Based) Learning from Supervised Analysis

Features that are significantly associated with model-based learning identified using elastic net regularization with tenfold cross-validation, observed in >95% of 100 iterations tested. Index refers to the item number from the questionnaire of origin. Beta refers to the coefficient from the regularized regression model. Words in parentheses, e.g. '(do not)' are added here (but were not presented to participants) to facilitate interpretation of the direction of effects for items that are reverse coded. The last column 'FA Loadings' indicates the significant overlap in terms of loading on Factors F1 (‘Anxious-Depression’), F2 (‘Compulsive Behavior and Intrusive Thought’) or F3 (‘Social Withdrawal), in the positive (+) or negative (-) direction using a cut-off at loadings >= 0.25.

https://doi.org/10.7554/eLife.11305.014
Supplementary file 5

(A) Association between questionnaire total scores and model-based learning defined using full computational model.

*p<0.05 ** p<0.01 ***p<0.001. Each row reflects the results from an independent analysis where each questionnaire total score (z-transformed) was entered as SymptomScorez in the following model: lm(Model-Based-Learning ~ IQz + Agez + Gender + SymptomScorez). Statistics refer to the main effect of SymptomScorez on Model-Based-Learning, i.e. the extent to which that questionnaire total score is associated with changes in model-based learning (which was defined for each participant using the full computational model). (B) Comparing the predictive power of model-based learning defined using the computational model versus one-trial back regression analysis. Each row reflects the results from an independent analysis where each questionnaire total score (z-transformed) was entered as SymptomScorez in the following model: lm(SymptomScorez ~ ModelBasedScore). Prior to conducting these analyses, we regressed out the effects of age, gender and IQ so that we could directly compare the r2 of the models. ModelBasedScore was derived from the one-trial back regression (first three columns) or the computational model (last three columns results). For each, positive β values indicate that the ModelBasedScore is associated with fewer symptoms, whereas negative β values indicate that the symptom score is associated with increased symptoms. The difference between the two approaches is negligible. However the computational model did produce nominally higher r2 and lower p-values for the relationship between clinical scores and model-based learning.

https://doi.org/10.7554/eLife.11305.015

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Download citations (links to download the citations from this article in formats compatible with various reference manager tools)

Open citations (links to open the citations from this article in various online reference manager services)