We consider that value synthesis is operated by neural networks composed of two layers (see Methods): (i) an attribute-specific layer further divided into two sets of units that differ in terms of their inputs (either trial-by-trial gains or losses), and (ii) an integration layer receiving outputs from both attribute-specific units (see Figure 1A below).

Figure 1 with 4 supplements see all

Download asset Open asset

By assumption, the gamble’s value $V_t$ at trial $t$ is read out from the response pattern in the integration layer using a linear population code (Pessiglione and Daunizeau, 2021):

where $z_t^{\left(k\right)}$ (respectively, $x_t^{\left(i,j\right)}$) is the output response of the $k$^th integration unit (respectively, $j$^th unit within the $i$^th attribute sublayer), $w^{\left(k\right)}$ is the corresponding value readout weight in the population code, $f_z^{\left(k\right)}$ is its input-output activation function, ${\upsilon }_t^{\left(k\right)}$ is the input signal to the $k$^th integration unit (which is made of a weighted mixture of attribute units’ outputs), $C^{\left(i,j,k\right)}$ is its connection strength with the $j$^th unit of the $i$^th attribute sublayer, and $u_t^{\left(i\right)}$ is the input signal to the $i$^th attribute sublayer (i.e. either gain or loss information) at trial $t$. Importantly, we assume that units have a bounded response range, i.e., their firing rate cannot exceed some predefined physiological limit. Recall that the main mechanistic constraint that acts on a neuron’s firing rate is its action potential’s refractory period, which depends upon how long it takes ion channels to complete a whole voltage activate-deactivate cycle (about a few milliseconds). Typically, even fast-spiking neocortical neurons cannot fire at a frequency higher than about 500 or 600 Hz (Wang et al., 2016). In our context, this can be simply modeled using saturating (more precisely: sigmoidal) input-output activation functions. In turn, the receptive field of integration units over the bidimensional domain of prospective gains and losses can exhibit arbitrary idiosyncratic shapes (see e.g. Figure 1—figure supplement 1).

In principle, artificial neural networks (ANNs) described by Equation 1 can be trained to output almost any subjective value landscape over the bidimensional domain spanned by prospective gains and losses. In particular, they can be trained to output actions’ expected value (EV), which would yield a neural implementation of normative decision-making. For example, consider a gamble decision that entails a 50/50 chance of either earning an amount G of money or losing an amount L. In this case, the gamble’s EV is simply the sum of prospective gains and losses, weighted by their occurrence probability, i.e., $EV=\frac{1}{2}G-\frac{1}{2}L$. Training the network to operate this kind of (linear) value synthesis is trivial. If the units’ responses were perfectly reliable (i.e. noiseless), then this would eventually yield rational behavioral responses. But, due to the limited response range of neural units, even small amounts of noise on the integration layer may induce a strong loss of information on value. This happens when inputs to integration units fall outside their responsive range, because it saturates the output contrast. By analogy with efficient coding in the brain’s perceptual systems, efficient value synthesis would rely upon some form of unsupervised adaptation mechanism to mitigate such information loss. For ANNs that obey Equation 1, one can show that the following self-organized plasticity rule operates efficient value synthesis (see Methods section):

where $\mathrm{\Delta }{C}_t^{(i,j,k)}$ is the trial-by-trial change in connection strength between attribute and integration units and ${\sigma }_z^{\left(k\right)}$ is the slope parameter of the $k$^th integration unit’s activation function. Here, $\alpha$ and $\beta$ determine both the magnitude and temporal scale at which self-organized plasticity unfolds (see Methods section). Equation 2 modifies the network connections such that inputs to integration units (i.e. $v_t^{\left(k\right)}$ in Equation 1) fall within the responsive range of their activation function, which maximizes the output variability induced by attribute inputs (see Figure 1B). This eventually improves the behavioral resilience of the system to neural noise (see Figure 1—figure supplement 2). The plasticity rule in Equation 2 is ‘self-organized’ in the sense that it does not require any teaching or feedback signal. It is also ‘local,’ in that a connection only changes in response to (the recent history of) the outputs of the corresponding pair of connected units. Finally, it is ‘anti-Hebbian,’ in that connections tend to weaken when units co-activate. The exact form of self-organized plasticity that operates efficient value synthesis depends upon the units’ activation function, but these properties generalize to any nonlinear activation function that is continuous and monotonically increasing.

Setting $\alpha =0$ or $\beta =0$ yields non-adaptive value synthesis, whereby the value readout is independent of the history of spanned prospective gains and losses. In what follows, we refer to this as static value synthesis. Otherwise (i.e. when ${\displaystyle \alpha >0}$ and ${\displaystyle \beta >0}$), the self-organized plasticity mechanism in Equation 2 progressively modifies the shape of integration units’ receptive fields over the spanned gain/loss domain (see Figure 1—figure supplement 3). This has three main notable consequences.

First, this eventually induces apparent value range adaptation, i.e., the average response of integration units settles between two (almost) invariant bounds which systematically map onto the gambles’ EV extrema over the spanned gain/loss domain. In addition, despite potentially strong nonlinearities in the receptive fields of integration units, the average relationship between their activity and gambles’ EV is almost linear. Interestingly, this apparent value range adaptation effect comes in two variants, depending upon the sign of the correlation between units’ activity and gambles’ EV.

Figure 2 exemplifies the apparent value range adaptation effect. This is an average of over 1000 Monte-Carlo simulations, where we randomized the trained connections of ANNs that operate (either efficient or static) value synthesis prior to exposing them to four different series of 256 decision trials made of prospective gains and losses with a predefined range. More precisely, we considered two ranges (either narrow or wide) for both prospective gains and losses, and exposed the ANNs to each of the 2×2 range combinations. By chance, some units become less sensitive to prospective gains than to prospective losses: those will tend to show a negative correlation with EV (‘-EV-units’). Interestingly, the responses of ‘+EV-units’ and ‘-EV-units’ shown on panels A and B are reminiscent of value range adaptation effects evidenced using electrophysiological recordings of OFC neuron activity (Conen and Padoa-Schioppa, 2019; Padoa-Schioppa, 2009). In particular, one can see that the slope of the relationship between EV and integration units’ activity only depends upon the EV range, and not upon the actual bounds of the spanned domain of EVs. However, this value range adaptation effect is only apparent, in the sense that integration units do not respond to value: they respond to prospective gains and losses. This is important, because the underlying plasticity mechanism reacts to the relative range of spanned gains and losses, which is partially orthogonal to the induced range of EVs. On thus needs to consider the shape of the spanned domain of decision-relevant attributes to properly understand the neural and behavioral consequences of efficient value synthesis.

Figure 2

Download asset Open asset

Second, self-organized plasticity changes the information content within the integration layer. To show this, we measure the neural dissimilarity between response patterns within the integration layer for any pair of decision trials, and then quantify its change when pairwise differences in either prospective gains, losses or EV vary. We define the ‘neural encoding strength’ of gains, losses, or EV in terms of the gradient of neural dissimilarity per unit of absolute difference in gains, losses, or EV, respectively (see Figure 1—figure supplement 4). In brief, the neural encoding strength of gains (resp. losses) decreases when the spanned range of gains (resp. losses) increases. This is also true for EV, whose neural encoding strength in the integration layer decreases when either the range of gains or losses increases.

Third, this modifies the relative sensitivity of readout value to prospective gains and losses. We quantify the sensitivity of readout value w.r.t. to its constituent attributes in terms of its gradient per unit of prospective gain and losses. In brief, the sensitivity of the ANN’s readout value signal to prospective gains (resp. losses) decreases when the spanned range of gains (resp. losses) increases. In turn, behavioral loss aversion - as defined by the ratio between loss and gain sensitivities- increases (resp. decreases) when the range of gains (resp. losses) increases. This is important, because this suggests how peoples’ behavior will deviate from classical decision theory and exhibit irrational context-dependency effects.

Figure 3 below summarizes the impact of the shape of the spanned bidimensional gain/loss domain. This is an average of over 1000 Monte-Carlo simulations, where the trained ANN’s connections are randomized prior to self-organizing according to Equation 2 in response to a series of 256 decision trials made of prospective gains and losses with a predefined range, which is systematically varied.

Figure 3

Download asset Open asset

Strikingly, the behavior will exhibit positive (respectively, negative) loss aversion when the context is favorable (respectively, unfavorable), i.e., when the spanned range of gains is greater (respectively, smaller) than that of losses (Figure 3F). Also, behavioral loss aversion is expected to be neutral when the spanned ranges of gains and losses are comparable (symmetrical gain/loss domain).

In addition to the main effects described above, one can see that there is a cross-attribute spillover effect, such that the behavioral and neural sensitivities to prospective gains (respectively, losses) also decrease when the spanned range of losses (respectively, gains) increases (Figure 3A, B, D and E). The magnitude of these spillover effects is comparatively weaker and may thus be more difficult to detect in an empirical setting. In fact, the only quantity that is similarly impacted by the ranges of gains and losses is the neural encoding strength of EV (Figure 3C). This effect is partly driven by changes in the neural sensitivity to gains and losses (Figure 3A and B), which constrains the availability of information on EV within the integration layer. But it also derives from the distortion of the readout value profile (Figure 3D and E), which weakens the statistical relationship between EV and integration units’ activity patterns. Both are consequences of the changes in integration units’ receptive fields that are induced by self-organized plasticity (see Figure 1—figure supplements 1 and 3). This eventually translates into an apparent value range adaptation phenomenon that generalizes the univariate effect reported in Figure 2.

Note that all these range adaptation effects actually unfold over time, as the network progressively self-organizes in response to prospective gains and losses. Importantly, however, numerical simulations show that the ensuing dynamics of neural and behavioral sensitivities converge, i.e., they eventually reach a steady-state. The convergence rate is governed by the parameter $\beta$, whereas the overall magnitude of these context-dependency effects is determined by the parameter $\alpha$. Although the exact setting of the plasticity magnitude and rate parameters do modify the global magnitude of neural and behavioral sensitivity changes, the results shown in Figure 3 are representative of the impact of the spanned gain/loss domain’s shape.

We now present our re-analysis of the NARPS dataset (Botvinik-Nezer et al., 2020b). This dataset includes two studies, each of which is composed of a group of 54 participants who make a series of risky decisions. On each trial, a gamble was presented, entailing a 50/50 chance of gaining an amount G of money or losing an amount L. As in Tom et al., 2007, participants were asked to evaluate whether or not they would like to accept or reject the gambles presented to them. In the first study (hereafter referred to as the ‘narrow range’ group), participants decided on gambles made of gain and loss levels that were sampled from the same range (G and L independently varied between 5$ and 20$). In the second study (hereafter: the ‘wide range’ group), gain levels scaled to double the loss levels (L varied between 5$ and 20$, and G independently varied between 10$ and 40$). Importantly, both groups experience the exact same range of losses. In both studies, all 256 possible combinations of gains and losses were presented across trials (see Methods section). Importantly, the gambles’ outcomes were not revealed until the end of the experiment.

To begin with, we ask whether peoples’ loss aversion exhibits range adaptation, as predicted by efficient value synthesis. In our context, this implies that (i) peoples’ gambling rate should depend upon the gain range context (even within the EV range common to both groups), (ii) peoples’ behavioral sensitivity to gains should be higher in the narrow gain range group than in the wide gain range group, (iii) within-group averages of loss aversion should be initially similar and then progressively diverge as time unfolds, and (iv) participants from the wide gain range group should eventually exhibit strong loss aversion while participants from the narrow gain range group should be loss-neutral (cf. symmetrical gain/loss domain). Figure 4 below summarizes the results of behavioral data analyses that aim at testing these predictions.

Figure 4 with 1 supplement see all

Download asset Open asset

Overall, the average gambling rate of people from the wide range group (65±2%) is much higher than that of people from the narrow range group (44±2%), and the group difference is significant (p<10^–4, F=44.8, dof=101). This is of course expected, given that people from the wide range group are exposed to gambles with higher value on average. However, and most importantly, within the range of EV that is common to both groups, people from the wide range group are less likely to gamble than people from the narrow range group (Figure 4A). Here, the average gambling rate of people from the wide range group is 41±3%, whereas it is 54±3% for people in the narrow range group, and this group difference is significant (p=0.003, F=9.2). This difference is likely due to the context in which people made these decisions, which is more favorable (higher gain prospects on average) in the wide-range group. This is the hallmark of context-dependency effects.

From Figure 4A, it seems that the variation in peoples’ tendency to accept risky gambles approximately spans the range of gambles’ values that they are exposed to. Under the efficient value synthesis scenario, this apparent value range adaptation effect is due to context-dependent changes in loss aversion. To investigate this effect, we first performed a within-subject logistic regression of trial-by-trial choice data onto gains and losses (including an intercept, see Methods section). In terms of balanced accuracy, this regression accurately explains 91±0.7% (respectively, 87±0.8%) of individual choices in the narrow (respectively, wide) range group (cf. plain lines in Figure 4A). A random effect analysis on regression weight estimates shows that all regression weights are significant at the group level (all p<10^–3), except for the intercept parameters (narrow range: p=0.26, wide range: p=0.14). This implies that peoples’ gambling behavior exhibits no systematic bias above and beyond the effects of prospective gains and losses. Regarding group differences, this analysis also failed to identify a group difference in the constant gambling bias (w₀: p=0.82, F=0.05). However, peoples’ sensitivity to gains is significantly higher in the narrow range group than in the wide range group (w_G: p=0.022, F=17.5), and this difference is almost significant for loss sensitivity (w_L: p=0.068, F=3.4). This means that increasing the range of gain prospects decreases peoples’ sensitivity to gains (and maybe to losses as well, though to a lesser extent; see Figure 4B).

We then derived indices of individual loss aversion, which we define as the log-transformed ratio of loss sensitivity to gain sensitivity, i.e., log(w_L/w_G) (Tom et al., 2007). This definition is not confounded by possible behavioral temperature differences between groups of participants. Mean loss aversion indices are shown in Figure 4C. We found that people from the wide range group exhibit significant loss aversion (mean loss aversion index=0.41, sem=0.05, p<10^–4) whereas people from the narrow range group do not (mean loss aversion index=0.037, sem = 0.05, p=0.46), and the ensuing group difference is significant (p=0.0031, F=9.2). Importantly, inter-individual differences in loss aversion explain the observed inter-individual differences in peoples’ gambling rate within the common EV range across all participants (p<10^–4, F=39.9, see Figure 4—figure supplement 1).

But is this loss aversion difference due to inter-individual trait differences, or did it grow over time as people are exposed to more gambles? To address this question, we repeated the within-subject logistic regression, this time on consecutive chunks of 16 trials (see Methods section). The resulting temporal dynamics of loss aversion are shown in Figure 3D. We found no significant time-by-group interaction (p=0.43, F=0.61), which is why we report separate (instantaneous) group comparisons. At the start of the experiment (first 16 trials), loss aversion is significant in both groups (wide range: mean loss aversion index=0.26, sem = 0.1, p=0.027, narrow range: mean loss aversion=0.25, sem=0.1, p=0.039), and there is no significant difference between groups (p=0.96, F=0.003). However, as time unfolds, loss aversion in both groups tends to spread apart: the difference between groups starts becoming significant after 32 trials (p=0.005, F=13.0) and stays significant thereafter (all p<0.05) except for two chunks of trials (p=0.12 and p=0.054). At the end of the experiment (last 16 trials), loss aversion is significant in participants of the wide range group (mean loss aversion=0.41, sem=0.07, p<10^–4) but not in the narrow range group (mean loss aversion=0.07, sem=0.06, p=0.24), and the group difference is significant (p=0.00042, F=13.3).

Those results validate the behavioral predictions of the efficient value synthesis scenario. We now wish to test its neural predictions, namely: (i) EV, as well as prospective gains and losses, should be encoded in neural activity patterns within the OFC, (ii) the neural encoding strength of prospective gains should be higher in the narrow gain range group than in the wide gain range group, (iii) the neural encoding strength of prospective losses should be equivalent in both groups (up to cross-attribute spillover effects), and (iv) the neural encoding strength of EV should be higher in the narrow gain range group than in the wide gain range group.

We thus extracted the multivariate trial-by-trial BOLD response in five OFC subregions: the lateral and medial parts of Brodmann area 11, Brodmann area 13, Brodmann area 14, and Brodmann area 32 (see Figure 11 in the Methods section). After correcting for between-session and temporal autocorrelation confounding effects (see Methods), we derived the ROI-specific representational dissimilarity matrices and measured the neural encoding strengths of gains (Figure 5), losses (Figure 6), and EV (Figure 7). The corresponding RDMs are shown in Figure 5—figure supplement 1, Figure 6—figure supplement 1, and Figure 7—figure supplement 1, respectively. For the sake of completeness, the results of standard univariate fMRI data analyses can also be eyeballed in Figure 7—figure supplement 2.

Figure 5 with 1 supplement see all

Download asset Open asset

Figure 6 with 1 supplement see all

Download asset Open asset

Figure 7 with 2 supplements see all

Download asset Open asset

Of course, prospective gain distances for the wide range-group extend beyond those of the narrow-range group. One can see that, in all subregions of the OFC, neural dissimilarity tends to increase when the absolute difference in prospective gains increases, though this gradient typically attenuates for extreme gain differences. The lateral part of Brodmann area 11 is the only OFC subregion that exhibits a significant encoding of prospective gains in both groups of participants (wide range: p0.0051, narrow range: p<10^–3), as well as a significantly higher encoding strength of gains in the narrow range group than in the wide range group (p=0.032).

Overall, it seems that prospective losses are encoded less strongly in OFC neural activity than prospective gains (though typically about four times stronger in magnitude). Nevertheless, the lateral part of Brodmann area 11 still exhibits a significant encoding of prospective losses in both groups of participants wide range: p=0.0049, narrow range: p=0.0056, without a significant group difference (p=0.59).

The pattern of neural encoding of EV is globally similar to that of prospective gains. Importantly, the lateral part of Brodmann area 11 is the only OFC subregion that exhibits a significant encoding of EV in both groups of participants (wide range: p=0.0061, narrow range: p<10^–3), as well as a significantly higher encoding strength of EV in the narrow range group than in the wide range group (p=0.0012).

Note that ignoring the fMRI confounding effects does not alter qualitatively the results, though it tends to bury the signal within structured noise, which dampens statistical significance.

In brief, qualitative predictions of the efficient value synthesis scenario at both the behavioral and neural levels have been confirmed (at least in the lateral part of Brodmann area 11). We will now provide further quantitative evidence that efficient value synthesis in the OFC can explain range adaptation of loss aversion.

As can be seen from Equations 1; 2, quantitative predictions from the efficient value synthesis scenario actually depend upon model parameters that may vary across individuals. For example, differences in the connectivity matrix $C^{(i,j,k)}$ (at the start of the experiment) and/or value readout weights $w^{\left(k\right)}$ can, in principle, account for a broad range of inter-individual differences in gambling behavior (and, possibly, in the neural encoding strength of prospective losses and gains). This raises the question: can the observed neural and behavioral differences between groups be explained by inter-individual differences in static value synthesis, without caring about self-organized plasticity?

To address this question, we fit the ANN model of value synthesis, with and without self-organized plasticity, to each participant’s series of gamble decisions (given the corresponding prospective gains and losses). In what follows, we refer to the models’ predictions about fitted behavioral data as models’ postdiction. We then perform counterfactual model simulations: for each subject-specific fitted model, we simulate the trial-by-trial gamble decisions that would have been observed, had this subject/model been exposed to the sequence of prospective gains and losses that each subject of the other group was exposed to (see Methods). That is, we ask what an ANN trained on the gambling decisions of a participant in the narrow gain range group would predict when exposed to the trial-by-trial series of gains and losses of participants from the wide gain range group (and reciprocally). These out-of-sample predictions provide a strong test of the model’s generalization ability. Figure 8 below shows both postdiction and out-of-sample predictions of the two ANN model variants (static versus efficient value synthesis).

Figure 8 with 2 supplements see all

Download asset Open asset

Unsurprisingly, both model postdictions accurately describe the qualitative group difference in gambling behavior. In addition, both candidate ANNs perform similarly to the logistic model, in terms of both percentage of explained variance (wide range group: 65.8±2.5%, narrow range group: 75.1±1.9%) and balanced fit accuracy (wide range group: 87.3±0.9%, narrow range group: 92.1±0.7%). We note that both ANN models yield postdiction error rates that are similar to the logistic model, in that they are maximal for hard decisions, i.e., when EV lies around zero (see Figure 8—figure supplement 1). In addition, under the efficient value synthesis scenario, empirical distributions of self-organized plasticity parameter estimates are comparable across both groups of participants (see Figure 8—figure supplement 2). But did ANN models capture a mechanism that faithfully generalizes to different gain range contexts (i.e. across groups)? First, static ANNs do not yield accurate out-of-sample predictions. This is expected, because static ANNs cannot exhibit range adaptation. Thus, they leave gambling behavior unchanged within the common range of expected values, and simply extrapolate postdicted behavior outside that range (as is the case for the logistic model, see Figure 4—figure supplement 1). In other terms, within the range of expected values that is common to both groups, static ANNs wrongly predict that the gambling rate should be higher in the wide range group than in the narrow range group (mean gambling rate group difference=9.7%). The situation is quite different for plastic ANNs, which yield more accurate out-of-sample predictions of peoples’ risk attitudes within the common range of expected values. In particular, plastic ANNs correctly predict that gambling rate should be lower in the wide-range group than in the narrow-range group (mean gambling rate group difference=−6.2%, p=0.038, F=1.8). We then measured the absolute out-of-sample prediction error of both plastic and static ANN models, for both participant groups. We found that it was significantly greater for static than for plastic ANN models (wide range group: p=0.013, F=9.45, narrow range group: p=0.032, F=6.43).

We also quantified postdicted loss aversion dynamics under both types of models (Figure 8C and D). One can see that ANNs that operate efficient value synthesis do exhibit realistic loss aversion dynamics, whereby both groups are initially comparable and then progressively spread apart as time unfolds (and the impact of the range of prospective gains accumulates). Note that this systematic dynamical change in peoples’ behavior is the information that plastic ANNs exploit to calibrate both the magnitude and the rate of self-organized plasticity, which reacts to the past history of prospective gains and losses. This does not hold, however, for ANNs that operate static value synthesis, which overlook dynamical changes and attempt to explain gambling choices in terms of an idiosyncratic value landscape. Note that, under the efficient value synthesis scenario, the dynamics of self-organized plasticity are determined by magnitude ($\alpha$ in Equation 2) and rate ($\beta$ in Equation 2) parameters. Accordingly, inter-individual differences in fitted plasticity magnitudes -but nor plasticity rates- significantly correlate with inter-individual differences in behavioral loss aversion indices (narrow range: p=0.024, wide range: p<10^–3, see Figure 8—figure supplement 2). Taken together, these results suggest that the self-organized plasticity mechanism in Equation 2 is necessary to capture the context-dependency of peoples’ loss aversion.

We now aim to evaluating the neurophysiological validity of fitted ANN models of value synthesis. To address this question, we ask whether the activity patterns in ANN models that were fitted to each participant’s gambling choices resemble the corresponding within-subject fMRI activity patterns in the OFC. We approach this problem using representational similarity analysis (RSA) within each subregion of the OFC. This allows us to compare the trial-by-trial multivariate activity patterns of candidate ANNs with those of fMRI signals in the OFC, without any additional ANN parameter adjustment. In brief, we compute four types of within-subject Representational Dissimilarity Matrices or RDMs (see Methods): (i) full trial-by-trial RDMs, (ii) gain-RDMs, where trials have been binned according to prospective gains, (iii) loss-RDMs, and (iv) EV-RDMs. We then measure the correlation between ANN-based and fMRI-based RDMs, for each OFC subregion and each participant. We then test for the statistical significance of this correlation within each group of participants (one-sample t-tests on Fisher-transformed within-subject correlations). Figure 9 below summarizes the RSA results in terms of the group-average RDM correlations, for both plastic and static ANNs.

Figure 9 with 1 supplement see all

Download asset Open asset

Intriguingly, both plastic and static ANN variants yield trial-by-trial activity patterns that significantly correlate with fMRI activity patterns in almost all OFC subregions (upper panels in Figure 9). This suggests that raw fMRI estimates of trial-by-trial dynamics of neural activity are not reliable enough to reveal the functional segregation of OFC subregions. This is not the case, however, when considering gain/loss/EV-RDMs. In brief, irrespective of the ANN model variant, no OFC subregion reaches statistical significance in both groups, for all types of RDM. Nevertheless, the lateral part of Brodmann area 11 almost meets this criterion, in that all RSA analyses are significant except for loss-RDMs in the narrow range group of participants, for both ANN variants (plastic ANN: p=0.22, static ANN: p=0.20). Given the anatomical specificity of this result, this is strong evidence that ANNs that operate value synthesis (whether plastic or static) provide a reasonably realistic prediction of the representational geometry of OFC neurons within the lateral part of Brodmann area 11. We note that, irrespective of the type of RDM considered, nowhere in the OFC is the comparison between the two model variants statistically significant.

Interestingly, ANNs that operate efficient value synthesis also reproduce other known features of value-coding neurons in the OFC. Recall that OFC neurons are notoriously diverse in their response profile, but a consistent finding is that, in the context of value-based decision-making, they can be classified in terms of so-called ‘choice cells,’ ‘chosen value cells,’ and ‘offer value cells’ (Padoa-Schioppa and Assad, 2006; Padoa-Schioppa and Assad, 2008). Given that this can be considered a pre-requisite for any computational model of value coding in the OFC, we asked whether plastic ANNs reproduce this known property of OFC neurons. For each subject, we thus tested whether the response of integration units correlates (across trials) with choice, chosen value, and/or gamble value, where value is defined as the weighted sum of gains and losses (according to the static logistic model parameter estimates). The results of this analysis are shown on Figure 9—figure supplement 1: in brief, plastic ANNs do exhibit this type of apparent coding variability, and predicted category proportions are qualitatively comparable to those reported in the existing literature. This provides additional neurobiological validity to ANN models of efficient value synthesis in the OFC.

Finally, we show that other adaptation models (in particular: efficient coding at the level of decision attributes) cannot explain neural data on value range adaptation. This is summarized in Figure 10 below. Although they predict qualitatively similar behavioral range adaptation effects (see Figure 10—figure supplement 1), they do not predict value range adaptation in the ANN’s integration layer (Figure 10AB). They also predict that increasing the range of prospective gains should increase the neural encoding strengths of gains within the integration layer (Figure 10C), which is at odds with the empirical data that we report here (Figures 5 and 6). Finally, when fitted on participants’ behavioral choices, they do not generalize well across gain range contexts (Figure 10—figure supplement 2), and their RSA results are less convincing (even in the lateral part of Brodman area 11, see Figure 10—figure supplement 3). The mathematical derivation of such models, as well as the analysis of their predictions, are summarized in the Supplementary material.

Figure 10 with 3 supplements see all

Download asset Open asset

Taken together these behavioral and neural analyses provide converging evidence that self-organized plasticity that operates efficient value synthesis in the lateral part of BA11 is a likely explanation for range adaptation of loss aversion.

Artificial neural networks or ANNs decompose a possibly complex form of information processing in terms of a combination of very simple computations performed by connected ‘units,’ which are mathematical abstractions of neurons. Here, we take inspiration from a growing number of studies that use ANNs as mechanistic models of neural information processing (Güçlü and van Gerven, 2015; Kietzmann et al., 2017; Kietzmann et al., 2019; Kriegeskorte and Golan, 2019), with the added requirement that they eventually explain (possibly irrational) behavioural data.

In abstract terms, any decision can be thought of as a cognitive process that transforms some input information $u=\left\{u^{\left(1\right)},u^{\left(2\right)},...,u^{\left(n_u\right)}\right\}$ into a behavioral output response $r$. Here, participants have to accept or reject a risky gamble composed of a 50% chance of obtaining a gain G and a 50% chance of experiencing a loss L, i.e., $u$ is composed of $n_u=2$ input attributes: $u=\left\{G,L\right\}$. Under an ANN model of such decisions, people’s behavioral response is the output of a neural network that processes the attributes $u$, i.e.,: $r\approx g_{ANN}\left(u,\vartheta \right)$, where $\vartheta$ are unknown ANN parameters and ${\displaystyle g_{ANN}()}$ is the ANN’s input-output transformation function. So-called ‘shallow’ ANNs effectively reduce ${\displaystyle g_{ANN}()}$ to a combination of neural units organized in a single hidden layer. In what follows, we rather rely on (moderately) deep ANNs with two hidden layers: namely, an attribute-specific layer (which is itself decomposed into gain-specific and loss-specific layers) and an integration layer (which receives inputs from both attribute-specific layers). The units that compose the latter then collectively determine gamble decisions by integrating prospective gains and losses.

We assume that each attribute $u_t^{\left(i\right)}$ is encoded into the activity of neurons $\left[x_t^{(i,1)},x_t^{(i,2)},...,x_t^{(i,j)},...,x_t^{(i,n_x)}\right]$ of its dedicated ‘attribute-specific layer,’ where $n_x$ is the number of attribute-specific neurons per attribute. What we mean here is that the neuron $j$ in the attribute-specific layer $i$ responds to $u_t^{\left(i\right)}$ as follows:

where $f\left(.\right)$ is the activation function of neural units that compose the ANN’s attribute-specific layer:

which yields a sigmoidal transform of inputs. Critically, such activation functions are bounded, i.e., we assume that neural units cannot fire beyond a certain rate. In neocortical neurons, the main mechanistic constraint that acts on firing rate is the action potential’s refractory period, which depends upon how long it takes ion channels to complete a whole voltage activate-deactivate cycle (about a few milliseconds). Typically, even fast-spiking neocortical neurons cannot fire at a frequency higher than about 500 or 600 Hz (Wang et al., 2016). As we will see, this type of response saturation is a critical component of range adaptation.

The parameters ${\displaystyle {\theta }^{(i,j)}=\left\{{\mu }^{(i,j)},{\sigma }^{(i,j)}\right\}}$ in Equation 3 captures idiosyncratic properties of the neuron $j$ in the input layer $i$ (e.g. its firing rate threshold ${\displaystyle {\mu }^{(i,j)}}$ and the slope parameter ${\displaystyle {\sigma }^{(i,j)}}$).

Collectively, the activity vector ${\left[x_t^{(i,j)}\right]}_{j=1,...,n_x}$ forms a multivariate representation of attribute ${\displaystyle u_t^{\left(i\right)}}$ in the form of a population code (Ebitz and Hayden, 2021). Then the output of the attribute-integration layers is passed to the ‘integration layer’ ${\displaystyle \left[z_t^{\left(1\right)},z_t^{\left(2\right)},...,z_t^{\left(k\right)},...,z_t^{\left(n_z\right)}\right]}$, i.e., the neuron $k$ of the integration layer responds to ${\left[x_t^{(i,j)}\right]}_{j=1,...,n_x}^{i=1,...,n_u}$ as follows:

where ${\displaystyle C^{\left(i,j,k\right)}}$ is the connection weight from the neuron $j$ in the attribute-specific layer $i$ to the neuron $k$ of the integration layer, ${\displaystyle {\varphi }^{\left(k\right)}}$ capture idiosyncratic properties of the integration neuron $k$ (i.e. its firing rate threshold and slope), and ${\displaystyle {\upsilon }_t^{\left(k\right)}}$ are the inputs of integration units.

Collectively, integration neurons form a representation of decision value in the form of a population code, i.e., the gambles’ subjective value $V_t$ at the time or trial $t$ is read out from the integration layer as follows:

where ${\displaystyle W^{\left(k\right)}}$ are the population readout weights.

Taken together, Equations 3–6 define the end-to-end ANN’s transformation of prospective gains and losses into decision value $V_t=V\left(u_t,\vartheta \right)$:

where $\vartheta$ lumps all ANN parameters together, i.e.,: ${\displaystyle \vartheta \triangleq \left\{W,C,\theta ,\varphi ,\upsilon \right\}}$. This is what we coin value synthesis. A schematic summary of the ANN’s double-layer structure is shown on panel A of Figure 1.

We start with the premise that the brain system that integrates value-relevant option features (here: prospective gains and losses) to construct value signals may be doing this under neural noise, which degrades the information about value. In particular, the limited range of physiological responses of neural units that perform this integration induces some information loss on value signals. This is because, when inputs to integration units fall too far away from their firing threshold (say outside a $\pm 2\sqrt{2}\sigma$ range), activation functions saturate, i.e., they produce non-discriminable outputs (close to 0 or 1). In this context, efficient value synthesis refers to the idea that neural networks that perform the integration of prospective gains and losses to construct value may adapt their response properties to mitigate information loss, hence the ‘efficiency’ of value synthesis. We now sketch how efficient value synthesis can be achieved within ANNs whose 2-layer structure is described in Equations 3-6.

In the presence of neural noise, the ANN’s readout value ${\tilde{V}}_t$ of a gamble made of a pair ${\displaystyle \left(G_t,L_t\right)}$ of prospective gain and loss is given by (in lieu of Equation 6):

where ${\eta }_t^{\left(k\right)}$ is some (uncontrollable) neural noise that competes with the ‘utile’ component ${\displaystyle z^{\left(k\right)}\left(G_t,L_t\right)}$ that is given in Equation 5.

Equation 8 can serve to measure the information loss $IL$ that is induced by neural noise under units’ limited response range:

Equation 9 states that the information loss increases when the mutual information between the noisy responses of integration units and their “utile” (i.e. noiseless) component decreases. Here, $MI\left(\cdot ,\cdot \right)$ is Shannon’s mutual information, $K$ is a constant, $H\left[\cdot \right]$ is Shannon’s entropy, and $E\left[\cdot \right]$ is the standard expectation operator. The right-hand term in Equation 9 arises at the small noise limit (Nadal, 1994), and the expectation is taken under the distribution of integration units’ inputs $\upsilon$. The last term in Equation 9 is simply the average steepness (in log space) of units’ activation functions. Importantly, Equation 9 holds irrespective of the type of nonlinearity of ANN units’ activation functions.

The entropy $H\left[\upsilon \right]$ has no closed-form expression, but can be given a multivariate gaussian approximation, i.e.,: $H\left[\upsilon \right]\approx ln\left|CS{C}^T\right|/2+K`$, where $S=E\left[x{x}^T\right]$ is the covariance matrix of the output of the ANN’s first layer and $K`$ is a constant. In principle, this approximation works because, when the size of the network grows, the central limit theorem implies that the distribution of integration units’ inputs $\upsilon$ will tend towards normality. The robustness of this approximation has been established in the context of undercomplete ICA (Porrill and Stone, 1998).

Efficient value synthesis can then be simply achieved by modifying the connectivity matrix $C$ to decrease the information loss $IL$, i.e., along the direction of the information loss gradient:

where $\alpha$ controls the magnitude of the gradient-following step and the first term in the right-hand side of Equation 10 is given by:

The matrix multiplier in the right-hand side of Equation 11 is non-local, i.e., the gradient ${\displaystyle \mathrm{\partial }H\left[\upsilon \right]/\mathrm{\partial }{C}^{\left(i,j,k\right)}}$ depends upon all connection weights in the network. This is unrealistic for biological systems, and we thus drop this term in the remainder of this manuscript. In turn, Equation 11 can be approximated as a collection of local changes to the connectivity matrix:

We will see that Equation 12 only involves the output response ${\displaystyle z^{\left(k\right)}}$ and ${\displaystyle x^{\left(i,j\right)}}$ of the pair of attribute and integration units that are connected through ${\displaystyle C^{\left(i,j,k\right)}}$. Equation 12 implies that efficient integration will tend to change the distribution of inputs ${\displaystyle {\upsilon }^{\left(k\right)}}$ to each integration unit such that they span the range where the steepness of its activation function is maximal. Focusing inputs to the responsive range of integration units’ activation functions then maximizes the output variability induced by attribute inputs. This makes sense, since this is expected to yield maximal contrast over the response outputs of integration units.

But Equation 12 still requires a last modification to derive a realistic self-organized plasticity rule for efficient value synthesis. This is because self-organized plasticity is a dynamical process, which reacts to recent network activity, as trials and/or time unfolds.

Note that the expectation in Equation 12 is taken under the distribution of prospective gains and losses, and can, therefore, be defined as a sample average over trial-by-trial gamble attributes. If the underlying distribution is non-stationary, then $E\left[\bullet \right]$ can be estimated at trial or time $t$ using a simple weighted moving average operator ${\displaystyle {\hat{E}}_t[\cdot ]}$:

where $\beta$ (note: ${\displaystyle 0<\beta <1}$) controls the exponential decay of past samples’ weights in the moving average operator.

Let $\mathrm{\Delta }{C}_t^{(i,j,k)}$ be the change of connectivity at trial or time $t$. Replacing the expectation in Equation 12 with the moving average operator ${\displaystyle {\hat{E}}_t[\cdot ]}$ in Equation 13 now yields:

where the local gradient can be written as:

Under sigmoidal activation functions, then:

Replacing Equation 16 into Equations 14-15 then yields:

which only depends upon the output response of connected pairs of attribute-specific and attribute-integration units.

Note that accounting for the nonlocal component of Equations 10-11 would require inserting the correction term ${\displaystyle \alpha \beta {\left[({\upsilon }_t{\upsilon }_t^T{)}^{-1}{\upsilon }_t{x}_t^T\right]}_{ij}}$ in the right-hand side of Equation 17. In our experience, its magnitude is typically small when compared to the Hebbian term in Equation 17. In turn, this term can be neglected without altering the main properties of efficient value synthesis.

Equation 17 states that efficient value synthesis can be operated by local, history-dependent, self-organized plasticity within the network. The plasticity in Equation 17 is ‘self-organized’ in the sense that it does not require any teaching or feedback signal. In this context, $\beta$ determines the adaptation rate of the network’s connectivity to changes in the distribution of prospective gains and losses. Importantly, the anti-Hebbian component of self-organized plasticity generalizes to any nonlinear activation function that is continuous and monotonically increasing. This is not the case, however, for non-monotonic activation functions (e.g. pseudo-gaussian activation functions).

In summary, as long as ANN units have monotonically increasing activation functions, efficient value synthesis can be implemented through some self-organized plasticity rule of the form given in Equation 17. It turns out that the self-organized plasticity rule in Equation 17 essentially modifies the integration units’ receptive fields, i.e., their pattern of response to a given pair of prospective gain and loss. This has two main consequences: it changes the information content of the network, and it distorts the readout value. We unpack these two phenomena using numerical simulations, which we report in the two first section of the Supplementary materials. At this point, we simply note that the effect of self-organized plasticity on both the readout value profile and the information content within the integration layer depends upon the ranges of prospective gains and losses that the ANN is exposed to. The neural and behavioral impacts of the shape of the spanned domain of gains and losses are summarized in Figures 2 and 3 of the Results section.

In this work, we perform a re-analysis of the NARPS dataset (Botvinik-Nezer et al., 2019; Botvinik-Nezer et al., 2020a), openly available on https://openneuro.org/; Poldrack et al., 2013. This dataset includes two studies, each of which is composed of a group of 54 participants who make a series of decisions made of 256 risky gambles. On each trial, a gamble was presented, entailing a 50/50 chance of gaining an amount G of money or losing an amount L. As in Tom et al., 2007, participants were asked to evaluate whether or not they would like to play each of the gambles presented to them (strongly accept, weakly accept, weakly reject, or strongly reject). They were told that, at the end of the experiment, four trials would be selected at random: for those trials in which they had accepted the corresponding gamble, the outcome would be decided with a coin toss and for the other ones -if any- the gamble would not be played. In the first study (hereafter: ‘narrow range’ group), participants decided on gambles made of gain and loss levels that were sampled from within the same range (G and L varied between 5 and 20 $). In the second study (hereafter: the ‘wide range’ group), gain levels scaled to double the loss levels (L varied between 5 and 20$, and G varied between 10 and 40$). In both studies, all 16×16=256 possible combinations of gains and losses were presented across trials, which were separated by 7 s on average with some random jitter (min 6, max 10).

MRI scanning was performed on a 3T Siemens Prisma scanner. High-resolution T1-weighted structural images were acquired using a magnetization-prepared rapid gradient echo (MPRAGE) pulse sequence with the following parameters: TR=2530ms, TE=2.99ms, FA=7, FOV=224 × 224 mm, resolution=1 × 1 × 1 mm. Whole-brain fMRI data were acquired using echo-planar imaging with a multi-band acceleration factor of 4 and parallel imaging factor (iPAT) of 2, TR=1000 ms, TE=30 ms, flip angle=68 degrees, in plane resolution of 2 × 2 mm 30 degrees of the anterior commissure-posterior commissure line to reduce the frontal signal dropout, with a slice thickness of 2 mm and a gap of 0.4 mm between slices to cover the entire brain. See https://www.narps.info/analysis.html#protocol for more details.

In the results Section, we focus on five subregions of the OFC: namely, the lateral and medial parts of Brodmann area 11, Brodmann area 13, Brodmann area 14, and the subgenual part of Brodmann area 32. This parcellation is based on anatomical masks in standard MNI coordinates obtained from the BRAINNETOME atlas (https://atlas.brainnetome.org/, Fan et al., 2016). As can be seen in Figure 11 below, these areas tile the entire OFC, except its most rostro-lateral part (which is Brodmann area 12).

Figure 11

Download asset Open asset

The standard MNI coordinates of each subregion barycenter are given in Table 1 below.

To balance the statistical power across OFC subregions, we then removed the voxels that fall outside a 200-voxel sphere centered on the barycenter of the masks. This procedure yielded spherical ROIs with similar sizes across all ROIs.

FMRI data were preprocessed using SPM (https://www.fil.ion.ucl.ac.uk/spm/), following standard realignment and movement correction guidelines. Note that we excluded five participants from the narrow range group because the misalignment between functional and anatomical scans could not be corrected. In each ROI, we regressed trial-by-trial activations with SPM through a GLM that included one stick regressor for each trial (at the time of the gamble presentation onset), which was convolved with the canonical HRF. To account for variations in hemodynamic delays, we added the basis function set induced by the HRF temporal derivative. To correct for movement artifacts, we also included the six head movement regressors and their squared values as covariates of no interest. We then extracted the 256 trial-wise regression coefficients in each voxel of each ROI. Next, we removed potential between-session confounding effects by projecting the ensuing trial series onto the null space of a categorical session-encoding design matrix. This effectively provided a BOLD trial series $Y_{fMRI}$ that are deconvolved from the hemodynamic response function (Dale, 1999) and corrected for standard confounding effects. No spatial smoothing was applied to preserve information buried in spatial fMRI activity patterns. Finally, we concatenated the corrected multivariate fMRI activity patterns of left and right analogous ROIs, eventually yielding five OFC subregions.

First, we describe peoples’ behavior in terms of the probability of gambling given the gamble’s expected value EV=0.5*(G-L), where G and L are the gamble’s prospective gain and loss, respectively. For each participant, we binned trials according to deciles of EV, and measured the rate of gamble acceptance (Figure 2, upper-left panel).

Second, we regressed peoples’ decision to gamble onto gains and losses. Within each participant, we fit the following logistic regression model: ${\displaystyle \mathrm{p}({\mathrm{g}}_{\mathrm{i}})=\mathrm{s}({\mathrm{w}}_0+{\mathrm{w}}_{\mathrm{G}}\ast {\mathrm{G}}_{\mathrm{i}}-{\mathrm{w}}_{\mathrm{L}}\ast {\mathrm{L}}_{\mathrm{i}})}$, where g_i is the binary gamble decision at trial i, G_i and L_i are the prospective gain and loss of trial i, w₀ is the intercept or gambling bias, w_G and w_L are the sensitivity to gains and losses, respectively, and s(.) is the standard sigmoid mapping. Note that logistic model parameter estimates can be recombined to measure peoples’ loss aversion (log(w_L/w_G)). We then report within-subject parameter estimates at the group-level for random effect analyses (see Figure 3 of the Results section). The logistic model can also be used to perform counterfactual model simulations. For each subject, we use the corresponding fitted parameters to evaluate the trial-by-trial probability of gamble acceptance that would have been observed, had this subject/model been exposed to the sequence of prospective gains and losses that each subject of the other group was exposed to. It turns out that such out-of-sample predictions of peoples’ behaviour are (expectedly) inaccurate. More precisely, such logistic regression cannot predict the observed group-difference in peoples’ gambling rate (see Figure 4—figure supplement 1).

Third, we performed a sliding window analysis: decisions were first partitioned into chunks of 16 consecutive trials each, which were then regressed against corresponding gains and losses using the same logistic model as above. From this, we obtain a set of logistic parameter estimates (intercept and sensitivity to gains and losses) per temporal window, per subject. Temporal changes in the ensuing loss aversion index can thus be followed as time unfolds (Figure 2D).

Each ANN model of value synthesis makes specific trial-by-trial predictions of activity patterns within the integration layer that can be compared to multivariate fMRI signals in each OFC subregion. This enables us to evaluate the neurophysiological validity of candidate models. Here, we have chosen to rely on representational similarity analysis or RSA (Diedrichsen and Kriegeskorte, 2017; Friston et al., 2019; Kriegeskorte et al., 2008). In brief, RSA consists of evaluating the statistical resemblance between model-based and data-based representational dissimilarity matrices or RDMs, which we derive as follows. Let $Y$ be the $n_y\times n_t$ multivariate time series of (modeled or empirical) neural activity, where $n_y$ and $n_t$ are the number of units and trials, respectively. Note that, for model predictions, ‘units’ mean artificial elementary units in ANNs, whereas they mean voxels in a given ROI for fMRI data. First, we derive the $n_t\times n_t$ raw RDM $D_Y=\left\{D_Y^{t,t`}\right\}$, where the matrix element $D_Y^{t,t`}$ measures the dissimilarity of neural patterns of activity between trial $t$ and trial $t`$: $D_Y^{t,t`}=1-corr\left(Y_t,Y_{t`}\right)$. By construction, these RDMs are invariant to affine transformations of activity patterns. In particular, this implies that the ensuing RSA are orthogonal to univariate analyses that rely on the mean activity within OFC subregions.

Second, we correct the raw RDM for autocorrelation confounds. To do this, we remove the average neural dissimilarity for each possible delay between trial pairs from the raw RDM. Note that this correction does not confound the existing relationship between neural dissimilarity and prospective gains and losses or EV, because these are randomized across trials.

When quantifying the neural encoding strength of prospective gains and losses, we simply regress the vectorized lower-left triangular part of $D_Y$ against Euclidean distances in either gains or losses concurrently (having included a constant term). This measures the gradient of neural dissimilarity per unit of gains and losses. We quantify the neural encoding strength of EV similarly (using a distinct regression analysis, to prevent regressor collinearities).

Finally, we measure the statistical similarity of $D_{Y_{ANN}}$ and $D_{Y_{fMRI}}$, where $D_{Y_{ANN}}$ is derived from activity patterns of the ANNs’ integration layer and $D_{Y_{fMRI}}$ is derived from HRF-deconvolved multi-voxel fMRI trial series in each ROI, in terms of the Pearson correlation coefficient $\rho$ between the vectorized lower-left triangular part of $D_Y$. We then assess the group-level statistical significance of RDMs' correlations using one-sample t-tests on the group mean of Fisher-transformed RDM correlation coefficients $\rho$. Note that ANN-RSA summary statistics (such as RDM correlation coefficients) do not favor more complex ANNs (i.e. ANNs with more parameters, such as plastic ANNs). This is because, once fitted to behavioural data, ANNs produce activity patterns that have no degree of freedom whatsoever when they enter RDM derivations. In particular, this means that static ANNs can a priori show a greater RDM correlation than plastic ANNs. In turn, this enables a simple yet unbiased statistical procedure for comparing candidate ANN models. Importantly, this procedure is immune to arbitrary modeling choices such as the total number of units in ANN models.

To address this question, we first reproduce the analysis of Figure 1—figure supplement 1, this time after self-organized plasticity has modified the network connectivity, while being exposed to the ‘wide.’ domain above (see Figure 1—figure supplement 3: below).

Comparing Figure 1—figure supplement 1 and Figure 1—figure supplement 3: shows that the receptive fields of integration units have been altered, even beyond the domain of prospective gains and losses that were spanned during efficient integration (except, maybe for unit #3, which underwent very small changes in its connectivity to the feature layer of the network). In particular, units #1 and #2 are now mostly sensitive to losses, whereas unit #4 has become mostly sensitive to gains. In turn, this eventually distorted the readout value profile. Importantly, the readout value profile now shows, within the spanned domain of prospective gains and losses, distorted sensitivities to gains and losses. More precisely, the sensitivity to losses ${\displaystyle {\omega }_L}$ is now about twice that of gains ${\displaystyle {\omega }_G}$. Self-organized plasticity over this domain of prospective gains and losses would thus eventually yield loss aversion.

One can also see that self-organized plasticity has changed the statistical relationship between integration units’ responses and gamble EVs. In brief, all integration units now show stronger response variations across the spanned range of EVs, i.e., inputs to integration units now tend to span the non-saturating range of their activation function (this is what is measured in the left panel of Figure 1—figure supplement 2). In addition, this relationship also tends to become more ambiguous, i.e., there are stronger variations of response outputs within EV bins. This is due to the induced distortion of the units’ receptive fields. Nevertheless, self-organized plasticity results in an apparent phenomenon of (partial) adaptation to the range of EVs. Practically speaking, should one attempt to detect those integration units that show a significant relationship with EV, one would conclude that self-organized plasticity seems to have ‘recruited’ integration units (that would otherwise show no strong covariation with EV). But here again, this phenomenon is only apparent, because EV is not an input to the network.

Now, we ask whether and how self-organized plasticity changed the information content within the integration layer. To do this, we rely upon RSA, which can be reproduced in an empirical setting where neural populations are sampled over an arbitrary domain of prospective gains and losses. We first measure the neural dissimilarity ${\displaystyle d(i,j,k,l)}$ of each possible pairwise combination of gains and losses, in terms of the correlation (across integration units) between the corresponding response patterns, i.e.,: ${\displaystyle d(i,j,k,l)=1-corr[z(G_i,L_j),z(G_k,L_l)]}$. By construction, this measure of dissimilarity is bounded between 0 (when the response patterns are colinear) and 2 (when they are anti-correlated). We then construct the RDM for EV as follows. Recall that each pair of prospective gain and loss belongs to a given EV bin. The RDM element that corresponds to a pair ${\displaystyle \left\{E{V}_m,E{V}_n\right\}}$ of EV bins is estimated as the average dissimilarity ${\displaystyle ⟨d(i,j,k,l)⟩}$ over all combinations for which ${\displaystyle \left(G_i-L_j\right)/2\in E{V}_m}$ and ${\displaystyle \left(G_k-L_l\right)/2\in E{V}_n}$. We also construct RDMs for prospective gains and losses similarly. If the network contains information about a given variable, then the corresponding RDM should show a diagonal pattern, such that the neural dissimilarity increases with the absolute difference between elements of the pair. Accordingly, we define the neural encoding strength of EV (resp., prospective gains or losses) in terms of the gradient of neural dissimilarity per unit of absolute difference of EV (resp., prospective gains or losses). This analysis can be reproduced after self-organized plasticity has modified the integration layer’s receptive fields, while exposing the ANN to the wide (asymmetrical) domain above. Figure 1—figure supplement 4 below summarizes the results of these analyses.

In brief, self-organized plasticity strongly strengthened the encoding of all variables, i.e., the ANN’s integration layer contains more information about EV and prospective gains/losses than before self-organized plasticity. Interestingly, although this network has been trained to perform value synthesis, the encoding of EVs after self-organized plasticity is relatively weaker than before (smaller variations of dissimilarity along directions orthogonal to iso-EV lines), at least when compared to prospective gains and losses. This is because changes in integration units’ receptive fields eventually reduced the redundancy between integration units, which now tend to decompose the array of feature units’ outputs into their independent sources of variations (here: prospective gains and losses). Nevertheless, as one can check in Figure 1—figure supplement 4, applying the readout weights on the integration units’ response still yields a reasonable readout value profile (that exhibits loss aversion).

The logistic regression analysis of peoples’ sensitivity to gains and losses provides a trial-by-trial prediction of gamble acceptance, for each pair of prospective gain and loss. But are hard decisions (i.e. when EV is close to zero) as well explained as easy decisions (e.g., when there is a strong incentive to gamble)? To address this question, we binned trials according to EV deciles (see Methods), and measured the rate of the logistic model’s postdiction error (see Figure 4—figure supplement 1A).

Note: hereafter, we refer to ‘postdiction”’ as model predictions on data that was used to fit the model’s parameters. In contrast, ‘out-of-sample predictions’ are model-proper predictions on yet unseen data.

One can see that the logistic regression model achieves relatively similar postdiction error profiles in both groups. In particular, easy decisions (either low or high EV) exhibit much lower postdiction errors than difficult decisions (EV around zero).

We also performed counterfactual model simulations: for each subject, we simulated the trial-by-trial gamble acceptances that would have been observed, under the logistic model, had this subject/model been exposed to the sequence of prospective gains and losses that each subject of the other group was exposed to (see Figure 4—figure supplement 1B). Those out-of-sample predictions are inaccurate: within the EV range that are common to both groups, it wrongly predicts that the gambling rate should be higher in the wide-range group than in the narrow-range group. This is not surprising, because predictions of the logistic model are entirely determined by the pair of prospective gain and loss (and cannot exhibit contextual effects). In other words, for a given gamble (defined in terms of its constituent gain and loss), the logistic model makes an out-of-sample prediction that corresponds to its equivalent postdiction (or is an extrapolation of it, if performed outside the EV range it was originally trained with).

Finally, we asked whether inter-individual differences in gambling rate (within the common EV range) were better explained in terms of inter-individual differences in loss aversion (i.e. the ratio $w_L/w_G$, where $w_L$ and $w_G$ are estimates of behavioral sensitivity to gains and losses, respectively) or in terms of gambling bias ($w_0$). This is summarized in Figure 4—figure supplement 1C, D. In brief, loss aversion negatively correlates with gambling rate within the common EV range (wide gain range group: r=0.59, p<10^–3, narrow gain range group: r=0.26, p=0.067, both groups together: r=0.53, p<10^–3).

In the main text, we summarize the information content within OFC subregions in terms of the gradients of neural dissimilarity per unit of prospective gains, losses, or EV. For the sake of completeness, we report below the underlying representational dissimilarity matrices, where trials have been binned according to prospective gains (gain-RDMs, Figure 5—figure supplement 1), losses (loss-RDMs, Figure 6—figure supplement 1), or EV (EV-RDMs, Figure 7—figure supplement 1).

Note that a strong neural encoding of gains would correspond to strong variations in neural dissimilarity along directions orthogonal to iso-distance lines. Such RDMs would show a clear bandwise-diagonal structure, where neural dissimilarity would increase when moving away from the main diagonal. One can see that gain-RDMs within the lateral part of BA11 are the closest to this ideal situation.

Here again, loss-RDMs within the lateral part of BA11 show the strongest encoding of prospective losses. Interestingly, the loss-RDM of BA11 for the narrow gain range group seems to exhibit a block structure, such that trials seem to be partitioned into two classes according to whether they lie either below or above the median loss. This suggests that the encoding of losses is not, strictly speaking, linear. Nevertheless, this does not confound the analysis of encoding strength which we report in the main text.

Except in the lateral part of BA11, all EV-RDMs exhibit antidiagonal elements with weak neural dissimilarity. This means that multivariate fMRI patterns of trials that correspond to either very low or very high EV are similar to each other. This is the reason why the average neural dissimilarity tends to eventually decrease for extreme absolute EV distances (Figure 7 of the main text).

In the main text, we report the results of multivariate (RSA) analyses of fMRI data, in five distinct subregions of the orbitofrontal cortex. For completeness, we also performed univariate analyses, having summarized the activity within OFC subregions in terms of the average trial-by-trial BOLD response over voxels (see Methods section for parametric estimation of trial-by-trial BOLD responses), and corrected the ensuing univariate responses for between-session confounding effects.

Here, we rely on two distinct general linear models or GLMs (Friston et al., 1995). All models incorporated only one event per trial, which was the onset of the gamble presentation, convolved with a canonical hemodynamic response function, as well as with its temporal derivative (Hopfinger et al., 2000). In the first GLM, we used two parametric modulations of the trial epoch regressor: namely prospective gains (G) and losses (L). We regressed the average trial-by-trial BOLD response against prospective gains and losses concurrently, using a GLM including an offset (constant term). The ensuing GLM parameter estimates measure the within-subjects gradient of BOLD responses per unit of prospective gains and losses. In the second GLM, we used only one parametric modulation, i.e. the gamble’s expected value EV=0.5*(G-L). Here, the ensuing GLM parameter estimates measure the within-subject’s gradient of BOLD response per unit of EV. Note that gain, loss and EV regressors were mean-centered but not rescaled to allow for a proper between-group comparison of neural sensitivity to these factors. For both regression analyses, we reported the ensuing parameter estimates at the group level and performed within-group and between-group statistical significance tests using standard one-sample and two-sample F-tests, respectively. The results of these analyses are summarized in Figure 7—figure supplement 2 below.

One can see that EV is significantly encoded in both groups of subjects in subregions BA14 and BA32. At the very least, the latter result survives the correction for multiple comparisons across OFC subregions. This reproduces established univariate results regarding the encoding of value in the fMRI literature. However, although some regions also show a significant encoding of prospective losses, nowhere in the OFC is there a significant encoding of both gains and losses, in both groups of subjects. This is strikingly different from the multivariate fMRI analyses results summarized in Figures 4—6 of the main text, which clearly exhibit stronger statistical power. We note that, by construction, these two analyses are orthogonal to each other. This is because we chose to measure neural dissimilarity in terms of Pearson correlations across voxels, which are invariant under isotropic (within OFC subregions) affine transformations of voxel-wise trial series.

We fit each candidate ANNs to observed trial-by-trial gamble decision sequences using a dedicated Bayesian approach, which requires setting specific prior distributions on model parameters. These priors distributions are summarized on Appendix 1—table 1 below.

All parameter notations are defined in the Methods section of the main text. Note that we performed all the analyses using the VBA academic freeware (Daunizeau et al., 2014). Although this toolbox only handles Gaussian prior distributions, native (Gaussian) VBA parameters can be passed through arbitrary mappings prior to entering model computations. This enables VBA to enforce any required constraint (see e.g. Daunizeau, 2017). This is the case here for the slopes of activation functions, as well as for the self-organized plasticity magnitude and rate parameters. In Appendix 1—table 1, θ denotes VBA native parameters: they are given Gaussian prior distributions, and then passed through the appropriate nonlinear mapping to enforce positivity or bounding constraints.

In addition to the prior distributions for ANN parameters given in Appendix 1—table 1 above, we set the number of attribute-specific and attribute-integration units to ${\displaystyle n_x=4}$ (per sublayer) and ${\displaystyle n_z=8}$, respectively. We also rescaled the ANN’s inputs (i.e. prospective gains and losses) such that they lie within the unit interval. To ensure an unbiased comparison between decision attributes and/or between groups of participants, this rescaling is invariant across decision attributes and groups, and such that the upper bound of the unit interval corresponds to 150% of the maximum prospective gain or loss (here: 40$). This also prevents artifactual ceiling effects in the population code of the attribute layer (whose units with the highest firing rate thresholds always remain weakly activated).

We fit the static and plastic ANN models to each subject’s sequence of gamble decisions. In what follows, we provide summary statistics of our ANN-based behavioral data analyses.

In addition to the balanced accuracy scores given in the main text, Appendix 1—table 2 below gives the group average percentage of explained behavioral variance (R²) and its standard deviation (across participants) for each model (including the logistic regression model, for comparison purposes), and each group.

In brief, all models achieve similar fit accuracies, in both groups, and no pairwise comparison between models reaches statistical significance.

But do the models yield similar types of postdiction errors? To address this question, we bin trials according to EV deciles, prior to averaging the rate of postdiction error across subjects. Figure 8—figure supplement 1 below summarizes the accuracy of behavioral postdiction as a function of gambles’ expected value, for all ANNs.

As was the case for the logistic regression model, ANNs exhibit high fit accuracy for easy decisions (extreme EVs) and lower fit accuracy for hard decisions (EV around 0).

Of particular interest in our ANN-based analysis of behavioral data are the parameters that control the self-organized plasticity rule of efficient value synthesis (plasticity magnitude α and rate β parameters). Figure 8—figure supplement 2 below shows the empirical histograms of parameter estimates, for both groups of participants.

Overall, empirical distributions of parameter estimates are qualitatively similar in both groups of participants. When comparing groups of participants with respect to either α or β parameter estimates, nothing reaches statistical significance.

We also asked whether fitted plasticity parameters explained inter-individual differences in observed loss aversion, as measured using the logistic model (see Figure 8—figure supplement 2C, D). Nothing really stands out for plasticity rates, but the situation is quite different for plasticity magnitudes. Within the wide gain range group, there is a significant correlation between plasticity magnitudes and loss aversion across subjects (r=0.47, p<10^–3), while this correlation is negative within the narrow gain range group (r=−0.32, p=0.024). That the sign of this correlation is different in both groups makes sense, given that the effect of self-organized plasticity is to decrease loss aversion within the narrow gain range group, whereas, if anything, it tends to increase it within the wide range group (see Figure 4D in the main text).

OFC neurons are notoriously diverse in their response profile, but a consistent finding is that, in the context of value-based decision-making, they can be classified in terms of so-called ‘choice cells,’ ‘chosen value cells’, and ‘offer value cells’ (Padoa-Schioppa and Assad, 2006; Padoa-Schioppa and Assad, 2008). Given that this can be considered a pre-requisite for any computational model of value integration in the OFC, we asked whether ANNs reproduce this known property of OFC neurons.

For each subject, we thus tested whether the response of integration units correlates (across trials) with choice, chosen value, and/or gamble value, where value is defined as the weighted sum of gains and losses (according to the static logistic model parameter estimates). Integration units are then classified in terms of which variable it correlate most with (but none of these labels is assigned if the best correlation is not significant). In brief, ‘choice units’ show response outputs that vary in a quasi-categorical manner with value (i.e. that discriminate trials in which the value of gambling is either positive or negative), ‘chosen value units’ exhibit a ReLU-like relationship with value (null when the value of gambling is negative, and increasing with value when it is positive), and ‘offer value units’ show a quasi-linear relationship with value. The results of this analysis are summarized in Figure 9—figure supplement 1 below.

In previous electrophysiological experiments, the detection rate of ‘offer value,’ ‘chosen value,’ and ‘choice’ cells within OFC neurons depends upon the delay to stimulus onset. This is because the detection analysis is typically performed at each time point within a given peristimulus time window. We thus extracted the frequency of ‘offer value,’ ‘chosen value,’ and ‘choice’ cells detected at OFC neurons’ response peak, i.e., about 300 msec after stimulus onset (see Figure 4 in Padoa-Schioppa and Assad, 2006). For comparison purposes, we then normalized these estimated frequencies to remove non-responsive OFC neurons (see horizontal grey bars in Figure 9—figure supplement 1). We find that, although HP-ANN’s integration units were not at all designed to encode these quantities, they eventually reproduce the known response variability observed in OFC neurons (although few units are eventually classified as ‘choice cells’ in the narrow gain range group). This means that our ANN models reproduce the known diversity of response profiles within OFC neurons. Importantly, the response profiles are almost identical for both groups. In particular, we find that about 34% of integration units are classified as ‘chosen value’ cells. This is interesting, because the computational role of these units is in fact exactly the same as that of units that are classified as ‘offer value’ cells: together, they form a population code for the subjective value of gambling. In other words, this classification is not directly relevant for guessing the underlying computational role of integration units.

In the main text, we assumed that neural noise would be acting on the output of integration units. But it may also be acting on its inputs, or equivalently on the outputs of the attribute units:

where ${\eta }_t^{(i,j)}$ is some (uncontrollable) neural noise that competes with the ‘utile’ component $x_t^{(i,j)}\left(u_t^{\left(i\right)}\right)$ of the responses of attributes units.

Similarly to Equation 9, this also induces an information loss $IL$:

In our framework, we do not consider how brain systems upstream of the attribute layers extract gain and loss information from visual stimuli and project it onto attribute layers. Rather, we assume that prospective gains and losses are encoded into population codes within attribute layers. Nevertheless, we can still model efficient coding at the level of attribute units. More precisely, efficient coding of gains and losses can then be achieved by modifying the response properties of attribute units to decrease the information loss $IL$ in Equation A2 i.e.,:

where ${\theta }^{(i,j)}$ is the 2 × 1 vector of location and scale parameters of attribute units’ activation functions.

This is but a proxy for the impact of self-organized plasticity upstream attribute layers. Note that there is no nonlocal term in Equation A3, because the entropy term $H\left[u\right]$ is beyond the network’s control.

Let $\mathrm{\Delta }{\theta }_t^{(i,j)}$ be the change of location and scale parameters at trial or time $t$. Similarly to efficient value synthesis, an online implementation of Equation A3 is operated as follows:

where $\beta$ (note: $0\S amp;lt;\beta \S amp;lt;1$) controls the exponential decay of past samples’ weights in the moving average operator (Equations 13-14 in the main text).

If the ANN is equipped with sigmoid activation functions, then simple analytical derivations show that Equation A4 reduces to the following update rule for location and scale parameters of attribute units:

Equation A5 is the equivalent of Equation 2 of the main text, i.e., it describes how the properties of attribute units within the network should modify their response properties to operate efficient coding of attribute inputs. In what follows, we simply refer to Equation A5 as efficient coding of attributes. Note that Equation A5 would also modify the receptive fields of integration units within the ANN. In principle, it could thus yield effects that are qualitatively similar to those of efficient value integration.

But what are the neural and behavioral impacts of this efficient coding mechanism? To address this question, we reproduced the same analyses as for efficient value synthesis.

First, we asked whether Equation A5 would produce apparent value range adaptation (Figure 2 in the main text) in integration units. We randomized the trained connections of ANNs that operate value synthesis prior to exposing them to four different series of 256 decision trials made of prospective gains and losses with a predefined range. As before, we considered two ranges (either narrow or wide) for both prospective gains and losses, and exposed the ANNs to each of the 2 × 2 range combinations. We then averaged the activity of integration units, after having binned trials into EV deciles. We repeated this procedure 1000 times, and Figure 10 in the main text summarizes the results of this analysis. One can see that the lower and upper limits of units’ mean responses are tied to the bounds of the spanned EV range, as is the case for static value synthesis (see Figure 2C and D in the main text). In turn, the slope of the relationship between EV and integration units’ mean responses does not seem to vary strongly with the range of spanned EVs. In other words, the efficient coding mechanism in Equation A5 does not induce apparent value range adaptation in the univariate response of integration units.

Second, we asked whether and how the shape of spanned the gain/loss domain modifies the neural and behavioral sensitivities to prospective gains and losses. We thus performed the same series of Monte-Carlo simulations as for efficient value synthesis. We simulated the response of the ANN, modified according to Equation A5 to operate efficient coding of attributes, to a series of 256 gambles, after having randomized the trained connectivity within the network. We systematically varied the spanned range of prospective gains and losses, and measured the behavioral sensitivities to gains and losses (in terms of the gradient of the readout value per unit of gain or loss) and the neural sensitivity to EV (in terms of the gradient of the integration layer’s neural dissimilarity per unit of EV). We repeated this procedure 1000 times, and Figure 10—figure supplement 1 below summarizes the results of this analysis.

In brief, the behavioral impact of efficient coding is qualitatively similar to that of efficient value synthesis. More precisely, the behavioral sensitivity to gains (resp., losses) decreases as the spanned range of gains (resp., losses) increases (Figure 10—figure supplement 1D, E). Note that cross-attribute spillover effects are also present here, and to a greater extent than under efficient value synthesis. In addition, efficient coding of attributes induces a similar effect on behavioral loss aversion, which follows the ratio of the spanned range of gains relative to the spanned range of losses (Figure 10—figure supplement 1F). This implies that behavioral observations alone would not disambiguate efficient value synthesis (as described in Equation 2 in the main text) from efficient coding of attributes (as described in Equation A5).

Also, the neural encoding strength of EV in the integration layer is similarly impacted by the shape of the spanned gain/loss domain. That is, the neural encoding strength of EV decreases when the spanned range of either gains or losses increases. Thus, this effect does not disambiguate efficient value synthesis from efficient value coding of attributes. However, the neural sensitivity to gains and losses does not react to the shape of the spanned gain/loss domain as they do under the efficient value synthesis scenario (see Figure 10C, D in the main text). The difference here is twofold. First, the cross-attribute spillover effect dominates, i.e., the most salient effect is that the neural sensitivity to gains (resp. losses) increases when the spanned range of losses (resp., gains) increases. Second, if anything, the within-attribute effect seems to be opposite to that of efficient value synthesis. That is, the neural sensitivity to gains (resp. losses) increases when the spanned range of gains (resp., losses) increases. At the very least, this holds for small to intermediate ranges of gains and losses; this effect reverses for extreme ranges of gains and losses (most likely because of artefactual ceiling effects). This is interesting, because this is clearly at odds with the neural predictions of the efficient value synthesis scenario. Importantly, this directly contradicts the fMRI data sampled in the lateral part of Brodman area 11 (see Figure 5 in the main text).

We then reproduced the same model-based analyses as with the efficient value synthesis scenario.

First, we fit the ANN model equipped with efficient coding of attributes to each participant’s trial-by-trial gamble decisions, and extract counterfactual out-of-sample behavioral predictions when exposing the fitted ANNs to the gamble series of the other group. We also performed the sliding window analysis to investigate the temporal dynamics of loss aversion, as captured by the scenario of efficient coding of attributes. The results of these analyses are summarized in Figure 10—figure supplement 2 below.

In brief, behavioral postdictions under the scenario of efficient coding of attributes are as accurate as under the efficient value synthesis scenario. In particular, the postdicted dynamics of loss aversion exhibit the same qualitative properties (compared with Figure 7C). However, out-of-sample behavioral predictions are not as convincing: although the model does predict a behavioral change, predicted gamble rates are still higher for the wide gain range group (mean gambling rate=0.58±0.02) than for the narrow gain range group (mean gambling rate=0.48±0.02). This observation is partially confirmed when comparing the absolute out-of-sample prediction error of both plastic ANN models. In brief, we found that out-of-sample behavioral predictions under efficient coding of attributes were significantly less accurate than under efficient value synthesis for the wide range group (p=0.006, F=12.8), but not for the narrow range group (p=0.31, F=1.15).

Second, we reproduced the same RSA analyses as before, i.e., we evaluated the similarity between the ANN model and fMRI data sampled in the same five OFC subregions. The results of these analyses are summarized in Figure 10—figure supplement 3 below.

No OFC subregion reaches statistical significance in both groups, for all types of RDMs. Importantly, this also holds for the lateral part of Brodman area 11 (only 5 out of 8 tests are significant). We note that, irrespective of the type of RDM considered, nowhere in the OFC is the comparison between both plastic models statistically significant.

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

© 2024, Brochard and Daunizeau

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.