Biased matching pennies (BMP) task and ketamine-induced behavioral modulation. A. Temporal sequence of trial events. Gray cross indicates the target that the animal was required to fixate during each epoch. Each trial began with the animal’s gaze on the fixation target displayed on the center of a computer monitor for 0.5-s. Solid red disks around the fixation target indicated the tokens owned by the animal, namely asset, with empty disks serving as placeholders for the tokens to be acquired for exchange with juice reward. After two green disks were displayed for 0.5-s at the diametrically opposed positions along the horizontal meridian, the fixation target was extinguished signaling the animal to indicate its choice by shifting gaze to one of the two green disks. After 0.5-s of fixation, a feedback ring appeared around the chosen target with its color indicating the choice outcome, followed by the corresponding change in tokens. Once the animal collected 6 tokens, they were automatically exchanged with 6 drops of apple juice. After juice reward, the animal began the subsequent trial with 2-4 free tokens. Tokens and placeholders stayed on the screen throughout the trial and inter-trial interval. B. Payoff matrix of BMP. C. Coefficients from logistic regression models applied separately to the data from saline (sal) and ketamine (ket) sessions with intramuscular (IM) and intranasal (IN) administration. Lines are exponential functions best fit to the coefficients from saline (solid) and ketamine (dotted) sessions, and those from IM (thick) and IN (thin) sessions. Bars at the top of each panel indicate that the difference between saline and ketamine sessions is statistically significant with the horizontal position and color of each bar coding trial lag and outcome (same color scheme as that of saline sessions), respectively.

Reinforcement learning models for normal behavior during saline sessions.

Reinforcement learning models for ketamine-induced modulation of choice behavior.

Model comparison for saline sessions. Each row shows free (filled circles), unused (empty circles) and fixed (specific values) parameters of a variant of reinforcement learning (RL) model. For the details of RL models and parameters, see Table 1. Heat map on the right represents natural logarithm of differential BIC (Bayesian Information Criterion) of each variant from the best model. Best model for each animal is indicated by check (√) mark. Q(standard): standard Q-learning model; Q-SE: Q-learning with subjective outcome evaluation; DF: differential forgetting model; DF-R: differential forgetting with neutral outcome being the reference point; NDF: non-differential forgetting model; NDF-R: non-differential forgetting model with neutral outcome being the reference point; NDF-A: non-differential forgetting model with asset-gated outcome evaluation.

Model comparison for the effects of ketamine. Each row shows fixed (gray-filled circles), free (colored circles) and unused (empty circles) parameters of a variant of differential forgetting (DF) model. For the details of each model, see Table 2. θ represents a set of parameters added to a particular variant of DF model, and the number inside colored circles indicate the number of added parameters (>1). For the details of each model, see Table 2. Heat map on the right represents natural logarithm of differential BIC (Bayesian Information Criterion) of each variant from the best model. Best model for each animal is indicated by check (√) mark. DF(saline): differential forgetting model fit to the data from saline session; Common mod. non-gain: common modulation of non-gain outcome evaluation; Separate mod. neutral vs. loss: differential modulation of neutral and loss outcome evaluation; Separate mod. gain vs. non-gain: differential modulation of gain and non-gain outcome evaluation; Separate mod. all: outcome-dependent modulation of outcome evaluation; Separate mod. forgetting rate: differential modulation of forgetting rate for chosen and unchosen target, Inc. perseveration: increased perseveration; Mod. value for all & inc. perseveration: outcome-dependent modulation of outcome evaluation and increased perseveration; Inc. misassign: increased backward credit misassignment; Inc. spread: increased forward spread in credit assignment; Inc. statistical learning: increased statistical learning of reward rate; Mod. asset gating: modulation of asset-gated outcome evaluation.

Ketamine-induced behavioral modulation simulated with differential forgetting model (for saline session) and best-fitting K-model (for ketamine session). Simulated data was generated with the maximum likelihood parameters of best-fitting models for saline (differential forgetting model) and ketamine (K-model 4: differential modulation of value for all the outcomes) sessions. Simulated choice was analyzed with logistic regression model (Eq. 1).

Maximum likelihood parameter estimates of the best models for saline and ketamine sessions.

Time course of ketamine-induced ocular nystagmus. A. Ocular position and velocity during fixation on the peripheral target in example trials during saline (left panel) and ketamine (right panel) sessions. B. Time course of mean ocular velocity aligned at the time of saline or ketamine injection. Shades indicate standard error. IM, IN indicates intramuscular and intranasal administration, respectively.

Time course of ketamine-induced modulation of outcome-dependent choice behavior. Time course of attenuation in loss evaluation induced by 0.5mg/kg of intramuscularly (A) and 1mg/kg of intranasally (B) administered ketamine. Regression coefficient reflecting ketamine’s modulation of the effect of each outcome from the previous trial (trial lag 1) is plotted as a function of time relative to the injection. Dotted lines represent standard error obtained from shuffled data between saline and ketamine sessions separately for each outcome.

Effect of motivation on countering ketamine-induced fixation errors. A. Difference in the rate of fixation break/trial between ketamine and saline sessions is plotted as a function of cumulative number of tokens (as a proxy for time with the satiation effect being controlled). Number of tokens owned by the animal at a given trial (asset) is color-coded. Vertical dotted lines demarcate the latest data point that was included in the linear regression analysis. B. Difference in the probability of choice switch after loss from the previous trial (trial lag 1) between ketamine and saline sessions is plotted as a function of asset at the time of decision. Due to limited number of loss trials, analysis was performed after dividing trials into 4 groups according to asset (0∼1, 2, 3, 4).

Behavioral effect of 0.25mg/kg intramuscularly (IM) administered (top) and 0.5mg/kg intranasally (IN) administered (bottom) ketamine. Regression coefficients reflecting the effects of gain, neutral (zero-token) and loss outcomes obtained in the past trials are plotted. Solid and dotted lines represent data from saline and ketamine sessions respectively. Solid (empty) symbols indicate that the corresponding coefficients are (not) significantly different from zero. * indicates that the coefficients from ketamine and saline sessions are significantly different for corresponding trial lag of outcome.

Time course of plasma concentration of ketamine following intramuscular (IM) and intranasal (IN) administration. Dotted lines indicate the data from individual sessions, and dotted lines for average across individual sessions. Blood sample was taken every 20 minute after injection.

ketamine-induced behavioral modulation simulated with best-fitting K-model of each class of reinforcement learning (RL) models. Format is same as in Figure 4. Each column represents simulated data with best-fitting parameters for individual monkey (P, Y and B from left to right).

Behavioral effects of ketamine do not spread over the sessions subsequent to the injection. Regression coefficients reflecting the effects of gain, neutral (zero-token) and loss outcomes obtained in the past trials are plotted. Solid (dotted) lines represent data from saline sessions > 1 day (1 day) after a ketamine session. Solid (empty) symbols indicate that the corresponding coefficients are (not) significantly different from zero.