Cognitive neuroscience studies the links between the physical brain and cognition. Computational models that attempt to describe the relationships between the brain and specific behaviours quantitatively is becoming increasingly popular in this field. This approach may help determine the causes of certain behaviours and make predictions about what behaviours will be triggered by specific changes in the brain.

Many of the computational models used in cognitive neuroscience are built based on experimental data. A good model can predict the results of new experiments given a specific set of conditions with few parameters. Candidate models are often called ‘generative’: models that can simulate data. However, typically, cognitive neuroscience studies require going the other way around: they need to infer models and their parameters from experimental data. Ideally, it should also be possible to properly assess the remaining uncertainty over the parameters after having access to the experimental data. To facilitate this, the Bayesian approach to statistical analysis has become popular in cognitive neuroscience.

Common software tools for Bayesian estimation require a ‘likelihood function’, which measures how well a model fits experimental data for given values of the unknown parameters. A major obstacle is that for all but the most common models, obtaining precise likelihood functions is computationally costly. In practice, this requirement limits researchers to evaluating and comparing a small subset of neurocognitive models for which a likelihood function is known. As a result, it is convenience, rather than theoretical interest, that guides this process.

In order to provide one solution for this problem, Fengler et al. developed a method that allows users to perform Bayesian inference on a larger number of models without high simulation costs. This method uses likelihood approximation networks (LANs), a computational tool that can estimate likelihood functions for a broad class of models of decision making, allowing researchers to estimate parameters and to measure how well models fit the data. Additionally, Fengler et al. provide both the code needed to build networks using their approach and a tutorial for users.

The new method, along with the user-friendly tool, may help to discover more realistic brain dynamics underlying cognition and behaviour as well as alterations in brain function.

Computational modeling has gained traction in cognitive neuroscience in part because it can guide principled interpretations of functional demands of cognitive systems while maintaining a level of tractability in the production of quantitative fits of brain-behavior relationships. For example, models of reinforcement learning (RL) are frequently used to estimate the neural correlates of the exploration/exploitation tradeoff, of asymmetric learning from positive versus negative outcomes, or of model-based vs. model-free control (Schönberg et al., 2007; Niv et al., 2012; Frank et al., 2007; Zajkowski et al., 2017; Badre et al., 2012; Daw et al., 2011b). Similarly, models of dynamic decision-making processes are commonly used to disentangle the strength of the evidence for a given choice from the amount of that evidence needed to commit to any choice, and how such parameters are impacted by reward, attention, and neural variability across species (Rangel et al., 2008; Forstmann et al., 2010; Krajbich and Rangel, 2011; Frank et al., 2015; Yartsev et al., 2018; Doi et al., 2020). Parameter estimates might also be used as a theoretically driven method to reduce the dimensionality of brain/behavioral data that can be used for prediction of, for example, clinical status in computational psychiatry (Huys et al., 2016).

Interpreting such parameter estimates requires robust methods that can estimate their generative values, ideally including their uncertainty. For this purpose, Bayesian statistical methods have gained traction. The basic conceptual idea in Bayesian statistics is to treat parameters θ and data $\mathbf{𝐱}$ as stemming from a joint probability model $p(\theta ,\mathbf{𝐱})$. Statistical inference proceeds by using Bayes’ rule,

to get at $p(\theta |\mathbf{𝐱})$, the posterior distribution over parameters' given data. $p(\theta )$ is known as the prior distribution over the parameters θ, and $p(\mathbf{𝐱})$ is known as the evidence or just probability of the data (a quantity of importance for model comparison). The term $p(\mathbf{𝐱}|\theta )$, the probability (density) of the dataset $\mathbf{𝐱}$ given parameters θ, is known as the likelihood (in accordance with usual notation, we will also write $\mathrm{\ell }(\theta |\mathbf{𝐱})$ in the following). It is common in cognitive science to represent likelihoods as $p(\mathbf{𝐱}|\theta ,s)$, where $s$ specifies a particular stimulus. We suppress $s$ in our notation, but note that our approach easily generalizes when explicitly conditioning on trial-wise stimuli. Bayesian parameter estimation is a natural way to characterize uncertainty over parameter values. In turn, it provides a way to identify and probe parameter tradeoffs. While we often do not have access to $p(\theta |\mathbf{𝐱})$ directly, we can draw samples from it instead, for example, via Markov Chain Monte Carlo (MCMC) (Robert and Casella, 2013; Diaconis, 2009; Robert and Casella, 2011).

Bayesian estimation of the full posterior distributions over model parameters contrasts with maximum likelihood estimation (MLE) methods that are often used to extract single best parameter values, without considering their uncertainty or whether other parameter estimates might give similar fits. Bayesian methods naturally extend to settings that assume an implicit hierarchy in the generative model in which parameter estimates at the individual level are informed by the distribution across a group, or even to assess within an individual how trial-by-trial variation in (for example) neural activity can impact parameter estimates (commonly known simply as hierarchical inference). Several toolboxes exist for estimating the parameters of popular models like the drift diffusion model (DDM) of decision-making and are widely used by the community for this purpose (Wiecki et al., 2013; Heathcote et al., 2019; Turner et al., 2015; Ahn et al., 2017). Various studies have used these methods to characterize how variability in neural activity, and manipulations thereof, alters learning and decision parameters that can quantitatively explain variability in choice and response time distributions (Cavanagh et al., 2011; Frank et al., 2015; Herz et al., 2016; Pedersen and Frank, 2020).

Traditionally, however, posterior sampling or MLE for such models required analytical likelihood functions: a closed-form mathematical expression providing the likelihood of observing specific data (reaction times and choices) for a given model and parameter setting. This requirement limits the application of any likelihood-based method to a relatively small subset of cognitive models chosen for so-defined convenience instead of theoretical interest. Consequently, model comparison and estimation exercises are constrained as many important but likelihood-free models were effectively untestable or required weeks to process a single model formulation. Testing any slight adjustment to the generative model (e.g., different hierarchical grouping or splitting conditions) requires a repeated time investment of the same order. For illustration, we focus on the class of sequential sampling models (SSMs) applied to decision-making scenarios, with the most common variants of the DDM. The approach is, however, applicable to any arbitrary domain.

In the standard DDM, a two-alternative choice decision is modeled as a noisy accumulation of evidence toward one of two decision boundaries (Ratcliff and McKoon, 2008). This model is widely used as it can flexibly capture variations in choice, error rates, and response time distributions across a range of cognitive domains and its parameters have both psychological and neural implications. While the likelihood function is available for the standard DDM and some variants including inter-trial variability of its drift parameter, even seemingly small changes to the model form, such as dynamically varying decision bounds (Cisek et al., 2009; Hawkins et al., 2015) or multiple-choice alternatives (Krajbich and Rangel, 2011), are prohibitive for likelihood-based estimation, and instead require expensive Monte Carlo (MC) simulations, often without providing estimates of uncertainty across parameters.

In the last decade and a half, approximate Bayesian computation (ABC) algorithms have grown to prominence (Sisson et al., 2018). These algorithms enable one to sample from posterior distributions over model parameters, where models are defined only as simulators, which can be used to construct empirical likelihood distributions (Sisson et al., 2018). ABC approaches have enjoyed successful application across life and physical sciences (e.g., Akeret et al., 2015), and notably, in cognitive science (Turner and Van Zandt, 2018), enabling researchers to estimate theoretically interesting models that were heretofore intractable. However, while there have been many advances without sacrificing information loss in the posterior distributions (Turner and Sederberg, 2014; Holmes, 2015), such ABC methods typically require many simulations to generate synthetic or empirical likelihood distributions, and hence can be computationally expensive (in some cases prohibitive – it can take weeks to estimate parameters for a single model). This issue is further exacerbated when embedded within a sequential MCMC sampling scheme, which is needed for unbiased estimates of posterior distributions. For example, one typically needs to simulate between 10,000 and 100,000 times (the exact number varies depending on the model) for each proposed combination of parameters (i.e., for each sample along a Markov chain [MC], which may itself contain tens of thousands of samples), after which they are discarded. This situation is illustrated in Figure 1, where the red arrows point at the computations involved in the approach suggested by Turner et al., 2015.

Figure 1

Download asset Open asset

To address this type of issue, the statistics and machine learning communities have increasingly focused on strategies for the amortization of simulation-based computations (Gutmann et al., 2018; Papamakarios and Murray, 2016; Papamakarios et al., 2019a; Lueckmann et al., 2019; Radev et al., 2020b; Radev et al., 2020a; Gonçalves et al., 2020; Järvenpää et al., 2021). The aim is generally to use model simulations upfront and learn a reusable approximation of the function of interest (targets can be the likelihood, the evidence, or the posterior directly).

In this paper, we develop a general ABC method (and toolbox) that allows users to perform inference on a significant number of neurocognitive models without repeatedly incurring substantial simulation costs. To motivate our particular approach and situate it in the context of other methods, we outline the following key desiderata:

1. The method needs to be easily and rapidly deployable to end users for Bayesian inference on various models hitherto deemed computationally intractable. This desideratum naturally leads us to an amortization approach, where end users can benefit from costs incurred upfront.

2. Our method should be sufficiently flexible to support arbitrary inference scenarios, including hierarchical inference, estimation of latent covariate (e.g., neural) processes on the model parameters, arbitrary specification of parameters across experimental conditions, and without limitations on dataset sizes. This desideratum leads us to amortize the likelihood functions, which (unlike other amortization strategies) can be immediately applied to arbitrary inference scenarios without further cost.

3. We desire approximations that do not a priori sacrifice covariance structure in the parameter posteriors, a limitation often induced for tractability in variational approaches to approximate inference (Blei et al., 2017).

4. End users should have access to a convenient interface that integrates the new methods seamlessly into preexisting workflows. The aim is to allow users to get access to a growing database of amortized models through this toolbox and enable increasingly complex models to be fit to experimental data routinely, with minimal adjustments to the user’s working code. For this purpose, we will provide an extension to the widely used HDDM toolbox (Wiecki et al., 2013; Pedersen and Frank, 2020) for parameter estimation of DDM and RL models.

5. Our approach should exploit modern computing architectures, specifically parallel computation. This leads us to focus on the encapsulation of likelihoods into neural network (NN) architectures, which will allow batch processing for posterior inference.

Guided by these desiderata, we developed two amortization strategies based on NNs and empirical likelihood functions. Rather than simulate during inference, we instead train NNs as parametric function approximators to learn the likelihood function from an initial set of a priori simulations across a wide range of parameters. By learning (log) likelihood functions directly, we avoid posterior distortions that result from inappropriately chosen (user-defined) summary statistics and corresponding distance measures as applied in traditional ABC methods (Sisson et al., 2018). Once trained, likelihood evaluation only requires a forward pass through the NN (as if it were an analytical likelihood) instead of necessitating simulations. Moreover, any algorithm can be used to facilitate posterior sampling, MLE or maximum a posteriori estimation (MAP).

For generality, and because they each have their advantages, we use two classes of architectures, multilayered perceptrons (MLPs) and convolutional neural networks (CNNs), and two different posterior sampling methods (MCMC and importance sampling). We show proofs of concepts using posterior sampling and parameter recovery studies for a range of cognitive process models of interest. The trained NNs provide the community with a (continually expandable) bank of encapsulated likelihood functions that facilitate consideration of a larger (previously computationally inaccessible) set of cognitive models, with orders of magnitude speedup relative to simulation-based methods. This speedup is possible because costly simulations only have to be run once per model upfront and henceforth be avoided during inference: previously executed computations are amortized and then shared with the community.

Moreover, we develop standardized amortization pipelines that allow the user to apply this method to arbitrary models, requiring them to provide only a functioning simulator of their model of choice.

In the 'Approximate Bayesian Computation' section, we situate our approach in the greater context of ABC, with a brief review of online and amortization algorithms. The subsequent sections describe our amortization pipelines, including two distinct strategies, as well as their suggested use cases. The 'Test beds' section provides an overview of the (neuro)cognitive process models that comprise our benchmarks. The 'Results' section shows proof-of-concept parameter recovery studies for the two proposed algorithms, demonstrating that the method accurately recovers both the posterior mean and variance (uncertainty) of generative model parameters, and that it does so at a runtime speed of orders of magnitude faster than traditional ABC approaches without further training. We further demonstrate an application to hierarchical inference, in which our trained networks can be imported into widely used toolboxes for arbitrary inference scenarios. In the'Discussion' and the last section, we further situate our work in the context of other ABC amortization strategies and discuss the limitations and future work.

Learning the likelihood with simple NN architectures

In this section, we outline our approach to amortization of computational costs of large numbers of simulations required by traditional ABC methods. Amortization approaches incur a one-off (potentially quite large) simulation cost to enable cheap, repeated inference for any dataset. Recent research has led to substantial developments in this area (Cranmer et al., 2020). The most straightforward approach is to simply simulate large amounts of data and compile a database of how model parameters are related to observed summary statistics of the data (Mestdagh et al., 2019). This database can then be used during parameter estimation in empirical datasets using a combination of nearest-neighbor search and local interpolation methods. However, this approach suffers from the curse of dimensionality with respect to storage demands (a problem that is magnified with increasing model parameters). Moreover, its reliance on summary statistics (Sisson et al., 2018) does not naturally facilitate flexible reuse across inference scenarios (e.g., hierarchical models, multiple conditions while fixing some parameters across conditions, etc.).

To fulfill all desiderata outlined in the introduction, we focus on directly encapsulating the likelihood function over empirical observations of a simulation model so that likelihood evaluation is substantially cheaper than constructing (empirical) likelihoods via model simulation online during inference. Such empirical observations could be, for example, trial-wise choices and reaction times (RTs). This strategy then allows for flexible reuse of such approximate likelihood functions $\widehat{\mathrm{\ell }}(\theta |\mathbf{𝐱})$ in a large variety of inference scenarios applicable to common experimental design paradigms. Specifically, we encapsulate $\widehat{\mathrm{\ell }}(\theta |\mathbf{𝐱})$ as a feed-forward NN, which allows for parallel evaluation by design. We refer to these networks as likelihood approximation networks (LANs).

Figure 1 spells out the setting (A) and usage (B) of such a method. The LAN architectures used in this paper are simple, small in size, and are made available for download for local usage. While this approach does not allow for instantaneous posterior inference, it does considerably reduce computation time (by up to three orders of magnitude; see ‘Run time’ section) when compared to approaches that demand simulations at inference. Notably, this approach also substantially speeds up inference even for models that are not entirely likelihood free but nevertheless require costly numerical methods to obtain likelihoods. Examples are the full-DDM with inter-trial variability in parameters (for which likelihoods can be obtained via numerical integration, as is commonly done in software packages such as HDDM but with substantial cost), but also other numerical methods for generalized DDMs (Shinn et al., 2020; Drugowitsch, 2016). At the same time, we maintain the high degree of flexibility with regards to deployment across arbitrary inference scenarios. As such, LANs can be treated as a highly flexible plug-in to existing inference algorithms and remain conceptually simple and lightweight.

Before elaborating our LAN approach, we briefly situate it in the context of some related work.

One branch of literature that interfaces ABC with deep learning attempts to amortize posterior inference directly in end-to-end NNs (Radev et al., 2020b; Radev et al., 2020a; Papamakarios and Murray, 2016; Papamakarios et al., 2019a; Gonçalves et al., 2020). Here, NN architectures are trained with a large number of simulated datasets to produce posterior distributions over parameters, and once trained, such networks can be applied to directly estimate parameters from new datasets without the need for further simulations. However, the goal to directly estimate posterior parameters from data requires the user to first train a NN for the very specific inference scenario in which it is applied empirically. Such approaches are not easily deployable if a user wants to test multiple inference scenarios (e.g., parameters may vary as a function of task condition or brain activity, or in hierarchical frameworks), and consequently, they do not achieve our second desideratum needed for a user-friendly toolbox, making inference cheap and simple.

We return to discuss the relative merits and limitations of these and further alternative approaches in the 'Discussion' section.

Formally, we use model simulations to learn a function $f_{\mathrm{\Phi }}(x,\theta )$, where $f_{\mathrm{\Phi }}(.)$ is the output of a NN with parameter vector $\mathrm{\Phi }$ (weights, biases). The function $f_{\mathrm{\Phi }}(\mathbf{𝐱},\theta )$ is used as an approximation $\widehat{\mathrm{\ell }}(\theta |\mathbf{𝐱})$ of the likelihood function $\mathrm{\ell }(\theta |\mathbf{𝐱})$. Once learned, we can use such $f$ as a plug-in to Bayes' rule:

${\displaystyle \hat{p}(\theta |\mathbf{x})\propto \prod _{i=1}^N\hat{\ell }(\theta |x_i)p(\theta )}$

To perform posterior inference, we can now use a broad range of existing MC or MCMC algorithms. We note that, fulfilling our second desideratum, an extension to hierarchical inference is as simple as plugging in our NN into a probabilistic model of the form

$\widehat{p}(\theta |𝐱)\propto \prod _{j=1}^J\prod _{i=1}^N\widehat{\mathrm{\ell }}({\theta }_j|x_{ji})p({\theta }_j|\alpha ,\beta )p(\alpha ,\beta |\gamma )$

where α, β are generic group-level global parameters, and γ serves as a generic fixed hyperparameter vector.

We provide proofs of concepts for two types of LANs. While we use MLPs and convolutional neural networks (CNNs), of conceptual importance is the distinction between the two problem representations they tackle, rather than the network architectures per se.

The first problem representation, which we call the pointwise approach, considers the functions $f_{\mathrm{\Phi }}(x|\theta )$, where θ is the parameter vector of a given stochastic simulator model, and $x$ is a single datapoint (trial outcome). The pointwise approach is a mapping from the input dimension $|\mathrm{\Theta }|+|x|$, where $|.|$ refers to the cardinality, to the one-dimensional output. The output is simply the log-likelihood of the single datapoint $x$ given the parameter vector θ. As explained in the next section, for this mapping we chose simple MLPs.

The second problem representation, which we will refer to as the histogram approach, instead aims to learn a function $f_{\mathrm{\Phi }}(.|\theta )$, which maps a parameter vector θ to the likelihood over the full (discretized) dataspace (i.e., the likelihood of the entire RT distributions at once). We represent the output space as an outcome histogram with dimensions $n\times m$ (where in our applications $n$ is the number of bins for a discretization of reaction times, and $m$ refers to the number of distinct choices). Thus, our mapping has input dimension $|\mathrm{\Theta }|$ and output dimension $|m\times n|$. Representing the problem this way, we chose CNNs as the network architecture.

In each of the above cases, we pursued the architectures that seemed to follow naturally, without any commitment to their optimality. The pointwise approach operates on low-dimensional inputs and outputs. With network evaluation speed being of primary importance to us, we chose a relatively shallow MLP to learn this mapping, given that it was expressive enough. However, when learning a mapping from a parameter vector θ to the likelihood over the full dataspace, as in the histogram approach, we map a low-dimensional data manifold to a much higher dimensional one. Using an MLP for this purpose would imply that the number of NN parameters needed to learn this function would be orders of magnitude larger than using a CNN. Not only would this mean that forward passes through the network would take longer, but also an increased propensity to overfit on an identical data budget.

The next two sections will give some detail regarding the networks chosen for the pointwise and histogram LANs. Figure 1 illustrates the general idea of our approach while Figure 2 gives a conceptual overview of the exact training and inference procedure proposed. These details are thoroughly discussed in the last section of the paper.

Figure 2

Download asset Open asset

Test beds

We chose variations of SSMs common in the cognitive neurosciences as our test bed (Figure 3). The range of models we consider permits great flexibility in allowable data distributions (choices and response times). We believe that initial applications are most promising for such SSMs because (1) analytical likelihoods are available for the most common variants (and thus provide an upper-bound benchmark for parameter recovery) and (2) there exist many other interesting variants for which no analytic solution exists.

Figure 3

Download asset Open asset

We note that there is an intermediate case in which numerical methods can be applied to obtain likelihoods for a broader class of models (e.g., Shinn et al., 2020). These methods are nevertheless computationally expensive and do not necessarily afford rapid posterior inference. Therefore, amortization via LANs is attractive even for these models. Figure 3 further outlines this distinction.

We emphasize that our methods are quite general, and any model that generates discrete choices and response times from which simulations can be drawn within a reasonable amount of time can be suitable to the amortization techniques discussed in this paper (given that the model has parameter vectors of dimension roughly lt₁₅). In fact, LANs are not restricted to models of reaction time and choice to begin with, even though we focus on these as test beds.

As a general principle, all models tested below are based on stochastic differential equations (SDEs) of the following form:

$d{𝐗}_t=a(t,x)dt+b(t,x)d{𝐁}_t,{𝐗}_0=w$

where we are concerned with the probabilistic behavior of the particle (or vector of particles) $\mathbf{𝐗}$. The behavior of this particle is driven by $a(t,x)$, an underlying drift function, $b(t,x)$, an underlying noise transformation function, $B_t$, an incremental noise process, and $X_0=w$, a starting point.

Of interest to us are specifically the properties of the first-passage-time-distributions (FPTD) for such processes, which are needed to compute the likelihood of a given response time/choice pair $\{rt,c\}$. In these models, the exit region of the particle (i.e., the specific boundary it reaches) determines the choice, and the time point of that exit determines the response time. The joint distribution of choices and reaction times is referred to as a FPTD.

Given some exit region $ℰ$, such FPTDs are formally defined as

$f_{ℰ}(t)=p(\underset{\tau }{inf}\{(X_{\tau }\in ℰ)\}=t)$

In words, a first passage time, for a given exit region, is defined as the first time point of entry into the exit region, and the FPTD is the probability, respectively, of such an exit happening at any specified time $t$ (Ratcliff, 1978; Feller, 1968). Partitioning the exit region into subsets ${ℰ}_1,\mathrm{\dots }.,{ℰ}_n$ (e.g., representing $n$ choices), we can now define the set of defective distributions

$\{f_{{ℰ}_1}(t;\theta ),\mathrm{\dots },f_{{ℰ}_N}(t;\theta )\}$

where $\theta \in \mathrm{\Theta }$ describes the collection of parameters driving the process. For every ${ℰ}_i$,

${\int }_{[0,+\mathrm{\infty }]}{f}_{{ℰ}_i}(t;\theta )𝑑t=P(𝐗\text{exits into}{ℰ}_i)=P(i\text{gets chosen})$

$\{f_{{ℰ}_{𝒾}},\mathrm{\dots },f_{{ℰ}_{𝓃}}\}$ jointly define the FPTD such that

$\sum _{i=1}^n{\int }_{[0,+\mathrm{\infty }]}{f}_{{ℰ}_i}(t;\theta )𝑑t=1$

These functions $f_{{ℰ}_i}$, jointly serve as the likelihood function s.t.

$\mathrm{\ell }(\theta ;\{rt,c\})=f_{{ℰ}_c}(t;\theta )$

For illustration, we focus the general model formulation above to the standard DDM. Details regarding the other models in our test bed are relegated to the 'Materials and methods' section.

To obtain the DDM from the general equation above, we set $a(t,x)=v$ (a fixed drift across time), $b(t,x)=1$ (a fixed noise variance across time), and $\mathrm{\Delta }\mathbf{𝐁}\sim 𝒩(0,\mathrm{\Delta }t)$. The DDM applies to the two alternative decision cases, where decision corresponds to the particle crossings of an upper or lower fixed boundary. Hence, ${ℰ}_1=\{ℝ\ge a\}$ and ${ℰ}_2=\{ℝ\le -a\}$, where $a$ is a parameter of the model. The DDM also includes a normalized starting point $w$ (capturing potential response biases or priors), and finally a nondecision time τ (capturing the time for perceptual encoding and motor output). Hence, the parameter vector for the DDM is then $\theta =(v,a,w,\tau )$. The SDE is defined as

$d{𝐗}_{\tau +t}=vdt+d𝐖,{𝐗}_{\tau }=w$

The DDM serves principally as a basic proof of concept for us, in that it is a model for which we can compute the exact likelihoods analytically (Feller, 1968; Navarro and Fuss, 2009).

The other models chosen for our test bed systematically relax some of the fixed assumptions of the basic DDM, as illustrated in Figure 3.

We note that many new models can be constructed from the components tested here. As an example of this modularity, we introduce inhibition/excitation to the race model, which gives us the leaky competing accumulator (LCA) (Usher and McClelland, 2001). We could then further extend this model by introducing parameterized bounds. We could introduce reinforcement learning parameters to a DDM (Pedersen and Frank, 2020) or in combination with any of the other decision models. Again we emphasize that while these diffusion-based models provide a large test bed for our proposed methods, applications are in no way restricted to this class of models.

Our results demonstrate the promise and potential of amortized LANs for Bayesian parameter estimation of neurocognitive process models. Learned manifolds and parameter recovery experiments showed successful inference using a range of network architectures and posterior sampling algorithms, demonstrating the robustness of the approach.

Although these methods are extendable to any model of similar complexity, we focused here on a class of SSMs, primarily because the most popular of them – the DDM – has an analytic solution, and is often applied to neural and cognitive data. Even slight departures from the standard DDM framework (e.g., dynamic bounds or changes in the noise distribution) are often not considered for full Bayesian inference due to the computational complexity associated with traditional ABC methods. We provide access to the learned likelihood functions (in the form of network weights) and code to enable users to fit a variety of such models with orders of magnitude speedup (minutes instead of days). In particular, we provided an extension to the commonly used HDDM toolbox (Wiecki et al., 2013) that allows users to apply these models to their own datasets immediately. We also provide access to code that would allow users to train their own likelihood networks and perform recovery experiments, which can then be made available to the community.

We offered two separate approaches with their own relative advantages and weaknesses. The MLP is suited for evaluating likelihoods of individual observations (choices, response times) given model parameters, and as such can be easily extended to hierarchical inference settings and trial-by-trial regression of neural activity onto model parameters. We showed that importing the MLP likelihood functions into the HDDM toolbox affords fast inference over a variety of models without tractable likelihood functions. Moreover, these experiments demonstrated that use of the NN likelihoods even confers a performance speedup over the analytical likelihood function – particularly for the full-DDM, which otherwise required numerical methods on top of the analytical likelihood function for the simple DDM.

Conversely, the CNN approach is well suited for estimating likelihoods across parameters for entire datasets in parallel, as implemented with importance sampling. More generally and implying potential further improvements, any sequential MC method may be applied instead. These methods offer a more robust path to sampling from multimodal posteriors compared to MCMC, at the cost of the curse of dimensionality, rendering them potentially less useful for highly parameterized problems, such as those that require hierarchical inference. Moreover, representing the problem directly as one of learning probability distributions and enforcing the appropriate constraints by design endows the CNN approach with a certain conceptual advantage. Finally, we note that in principle (with further improvements) trial-level inference is possible with the CNN approach, and vice versa, importance sampling can be applied to the MLP approach.

In this work, we employed sampling methods (MCMC and importance sampling) for posterior inference because in the limit they are well known to allow for accurate estimation of posterior distributions on model parameters, including not only mean estimates but their variances and covariances. Accurate estimation of posterior variances is critical for any hypothesis testing scenario because it allows one to be confident about the degree of uncertainty in parameter estimates. Indeed, we showed that for the simple DDM we found that posterior inference using our networks yielded nearly perfect estimation of the variances of model parameters (which are available due to the analytic solution). Of course, our networks can also be deployed for other estimation methods even more rapidly: they can be immediately used for MLE via gradient descent or within other approximate inference methods, such as variational inference (see Acerbi, 2020 for a related approach).

Other approaches exist for estimating generalized diffusion models. A recent example, not discussed thus far, is the pyDDM Python toolbox (Shinn et al., 2020), which allows MLE of generalized drift diffusion models. The underlying solver is based on the Fokker–Planck equations, which allow access to approximate likelihoods (where the degree of approximation is traded off with computation time/discretization granularity) for a flexible class of diffusion-based models, notably allowing arbitrary evidence trajectories, starting point, and nondecision time distributions. However, to incorporate trial-by-trial effects would severely inflate computation time (on the order of the number of trials) since the solver would have to operate on a trial-by-trial level. Moreover, any model that is not driven by Gaussian diffusion, such as the Levy model we considered here or the linear ballistic accumulator, is out of scope with this method. In contrast, LANs can be trained to estimate any such model, limited only by the identifiability of the generative model itself. Finally, pyDDM does not afford full Bayesian estimation and thus quantification of parameter uncertainty and covariance.

We moreover note that our LAN approach can be useful even if the underlying simulation model admits other likelihood approximations, regardless of trial-by-trial effect considerations, since a forward pass through a LAN may be speedier. Indeed, we observed substantial speedups in HDDM for using our LAN method to the full-DDM, for which numerical methods were previously needed to integrate over inter-trial variability.

We emphasize that our test bed application to SSMs does not delimit the scope of application of LANs. Neither are reasonable architectures restricted to MLPs and CNNs (see Lueckmann et al., 2019; Papamakarios et al., 2019b for related approaches that use completely different architectures). Models with high-dimensional (roughly gt₁₅) parameter spaces may present a challenge for our global amortization approach due to the curse of dimensionality. Further, models with discrete parameters of high cardinality may equally present a given network with training difficulties. In such cases, other methods may be preferred over likelihood amortization generally (e.g., Acerbi, 2020); given that this is an open and active area of research, we can expect surprising developments that may in fact turn the tide again in the near future.

Despite some constraints, this still leaves a vast array of models in reach for LANs, of which our test bed can be considered only a small beginning.

By focusing on LANs, our approach affords the flexibility of networks serving as plug-ins for hierarchical or arbitrarily complex model extensions. In particular, the networks can be immediately transferred, without further training, to arbitrary inference scenarios in which researchers may be interested in evaluating links between neural measures and model parameters, and to compare various assumptions about whether parameters are pooled and split across experimental manipulations. This flexibility in turn sets our methods apart from other amortization and NN-based ABC approaches offered in the statistics, machine learning, and computational neuroscience literature (Papamakarios and Murray, 2016; Papamakarios et al., 2019a; Gonçalves et al., 2020; Lueckmann et al., 2019; Radev et al., 2020b), while staying conceptually extremely simple. Instead of focusing on extremely fast inference for very specialized inference scenarios, our approach focuses on achieving speedy inference while not implicitly compromising modeling flexibility through amortization step.

Closest to our approach is the work of Lueckmann et al., 2019, and Papamakarios et al., 2019b, both of which attempt to target the likelihood with a neural density estimator. While flexible, both approaches imply the usage of summary statistics, instead of a focus on trial-wise likelihood functions. Our work can be considered a simple alternative with explicit focus on trial-wise likelihoods.

Besides deep learning-based approaches, another major machine learning-inspired branch of the ABC literature concerns log-likelihood and posterior approximations via Gaussian process surrogates (GPSs) (Meeds and Welling, 2014; Järvenpää et al., 2018; Acerbi, 2020). A major benefit of GPSs lies in the ability for clever training data selection via active learning since such GPSs allow uncertainty quantification out of the box, which in turn can be utilized for the purpose of targeting high-uncertainty regions in parameter space. GPS-based computations scale with the number of training examples, however, which make them much more suitable for minimizing the computational cost for a given inference scenario than facilitating global amortization as we suggest in this paper (for which one usually need larger sets of training data than can traditionally be handled efficiently by GPS). Again when our approach is applicable, it will offer vastly greater flexibility once a LAN is trained.

All code is provided freely and is available at the following links: https://github.com/lnccbrown/lans/tree/master/hddmnn_tutorial, https://github.com/lnccbrown/lans/tree/master/al-mlp and https://github.com/lnccbrown/lans/tree/master/al-cnn (copy archived at https://archive.softwareheritage.org/swh:1:rev:e3369b9df138c75d0e490be0c48c53ded3e3a1d6).

1. Conference

   1. Abadi M
   2. Barham P
   3. Chen J
   4. Chen Z
   5. Davis A
   6. Dean J
   7. Devin M
   8. Ghemawat S
   9. Irving G
   10. Isard M

   (2016)

   Tensorflow: a system for large-scale machine learning

   12th USENIX Symposium on Operating SystemsDesign and Implementation (OSDI 16). pp. 265–283.

   • Google Scholar
2. Conference

   1. Acerbi L

   (2020)

   Variational bayesian monte carlo with noisy likelihoods

   Advances in Neural Information Processing Systems.

   • Google Scholar
3. 1. Ahn WY
   2. Haines N
   3. Zhang L

   (2017) Revealing neurocomputational mechanisms of reinforcement learning and Decision-Making with the hBayesDM package

   Computational Psychiatry 1:24–57.

   https://doi.org/10.1162/CPSY_a_00002

   • PubMed
   • Google Scholar
4. 1. Akeret J
   2. Refregier A
   3. Amara A
   4. Seehars S
   5. Hasner C

   (2015) Approximate bayesian computation for forward modeling in cosmology

   Journal of Cosmology and Astroparticle Physics 2015:043.

   https://doi.org/10.1088/1475-7516/2015/08/043

   • Google Scholar
5. 1. Badre D
   2. Doll BB
   3. Long NM
   4. Frank MJ

   (2012) Rostrolateral prefrontal cortex and individual differences in uncertainty-driven exploration

   Neuron 73:595–607.

   https://doi.org/10.1016/j.neuron.2011.12.025

   • PubMed
   • Google Scholar
6. 1. Behnel S
   2. Bradshaw R
   3. Citro C
   4. Dalcin L
   5. Seljebotn DS
   6. Smith K

   (2010) Cython: the best of both worlds

   Computing in Science & Engineering 13:31–39.

   https://doi.org/10.1109/MCSE.2010.118

   • Google Scholar
7. Report

   1. Bishop CM

   (1994)

   Mixture Density Networks

   Technical report.

   • Google Scholar
8. 1. Blei DM
   2. Kucukelbir A
   3. McAuliffe JD

   (2017) Variational inference: a review for statisticians

   Journal of the American Statistical Association 112:859–877.

   https://doi.org/10.1080/01621459.2017.1285773

   • Google Scholar
9. 1. Boehm U
   2. Annis J
   3. Frank MJ
   4. Hawkins GE
   5. Heathcote A
   6. Kellen D
   7. Krypotos A-M
   8. Lerche V
   9. Logan GD
   10. Palmeri TJ
   11. van Ravenzwaaij D
   12. Servant M
   13. Singmann H
   14. Starns JJ
   15. Voss A
   16. Wiecki TV
   17. Matzke D
   18. Wagenmakers E-J

   (2018) Estimating across-trial variability parameters of the diffusion decision model: expert advice and recommendations

   Journal of Mathematical Psychology 87:46–75.

   https://doi.org/10.1016/j.jmp.2018.09.004

   • Google Scholar
10. 1. Braak C

    (2006) A markov chain monte carlo version of the genetic algorithm differential evolution: easy bayesian computing for real parameter spaces

    Statistics and Computing 16:239–249.

    https://doi.org/10.1007/s11222-006-8769-1

    • Google Scholar
11. 1. Cappé O
    2. Douc R
    3. Guillin A
    4. Marin J-M
    5. Robert CP

    (2008) Adaptive importance sampling in general mixture classes

    Statistics and Computing 18:447–459.

    https://doi.org/10.1007/s11222-008-9059-x

    • Google Scholar
12. 1. Cavanagh JF
    2. Wiecki TV
    3. Cohen MX
    4. Figueroa CM
    5. Samanta J
    6. Sherman SJ
    7. Frank MJ

    (2011) Subthalamic nucleus stimulation reverses mediofrontal influence over decision threshold

    Nature Neuroscience 14:1462–1467.

    https://doi.org/10.1038/nn.2925

    • PubMed
    • Google Scholar
13. 1. Charpentier A
    2. Flachaire E

    (2015)

    Log-transform kernel density estimation of income distribution

    L’actualité Économique 91:141–159.

    • Google Scholar
14. 1. Cisek P
    2. Puskas GA
    3. El-Murr S

    (2009) Decisions in changing conditions: the urgency-gating model

    Journal of Neuroscience 29:11560–11571.

    https://doi.org/10.1523/JNEUROSCI.1844-09.2009

    • PubMed
    • Google Scholar
15. 1. Cranmer K
    2. Brehmer J
    3. Louppe G

    (2020) The frontier of simulation-based inference

    PNAS 117:30055–30062.

    https://doi.org/10.1073/pnas.1912789117

    • PubMed
    • Google Scholar
16. Conference

    1. Daw ND

    (2011a)

    Trial-by-trial data analysis using computational models

    Decision Making, Affect, and Learning: Attention and Performance XXIII.

    • Google Scholar
17. 1. Daw ND
    2. Gershman SJ
    3. Seymour B
    4. Dayan P
    5. Dolan RJ

    (2011b) Model-based influences on humans' choices and striatal prediction errors

    Neuron 69:1204–1215.

    https://doi.org/10.1016/j.neuron.2011.02.027

    • PubMed
    • Google Scholar
18. 1. Diaconis P

    (2009) The markov chain monte carlo revolution

    Bulletin of the American Mathematical Society 46:179–205.

    https://doi.org/10.1090/S0273-0979-08-01238-X

    • Google Scholar
19. 1. Doi T
    2. Fan Y
    3. Gold JI
    4. Ding L

    (2020) The caudate nucleus contributes causally to decisions that balance reward and uncertain visual information

    eLife 9:e56694.

    https://doi.org/10.7554/eLife.56694

    • PubMed
    • Google Scholar
20. 1. Drugowitsch J

    (2016) Fast and accurate monte carlo sampling of first-passage times from Wiener Diffusion models

    Scientific Reports 6:20490.

    https://doi.org/10.1038/srep20490

    • PubMed
    • Google Scholar
21. Book

    1. Feller W

    (1968)

    An Introduction to Probability Theory and Its Applications

    Wiley.

    • Google Scholar
22. Software

    1. Fengler A

    (2021) projectABC, version swh:1:rev:e3369b9df138c75d0e490be0c48c53ded3e3a1d6

    Software Heritage.

    https://archive.softwareheritage.org/swh:1:rev:e3369b9df138c75d0e490be0c48c53ded3e3a1d6
23. 1. Forstmann BU
    2. Anwander A
    3. Schäfer A
    4. Neumann J
    5. Brown S
    6. Wagenmakers EJ
    7. Bogacz R
    8. Turner R

    (2010) Cortico-striatal connections predict control over speed and accuracy in perceptual decision making

    PNAS 107:15916–15920.

    https://doi.org/10.1073/pnas.1004932107

    • PubMed
    • Google Scholar
24. 1. Frank MJ
    2. Samanta J
    3. Moustafa AA
    4. Sherman SJ

    (2007) Hold your horses: impulsivity, deep brain stimulation, and medication in parkinsonism

    Science 318:1309–1312.

    https://doi.org/10.1126/science.1146157

    • PubMed
    • Google Scholar
25. 1. Frank MJ
    2. Gagne C
    3. Nyhus E
    4. Masters S
    5. Wiecki TV
    6. Cavanagh JF
    7. Badre D

    (2015) fMRI and EEG predictors of dynamic decision parameters during human reinforcement learning

    The Journal of Neuroscience 35:485–494.

    https://doi.org/10.1523/JNEUROSCI.2036-14.2015

    • PubMed
    • Google Scholar
26. Conference

    1. Frazier PI
    2. Angela JY

    (2008)

    Sequential hypothesis testing under stochastic deadlines

    Advances in Neural Information Processing Systems. pp. 465–472.

    • Google Scholar
27. 1. Gelman A
    2. Rubin DB

    (1992) Inference from iterative simulation using multiple sequences

    Statistical Science 7:457–472.

    https://doi.org/10.1214/ss/1177011136

    • Google Scholar
28. 1. Geweke J

    (1992)

    Evaluating the accuracy of sampling-based approaches to the calculations of posterior moments

    Bayesian Statistics 4:641–649.

    • Google Scholar
29. 1. Gonçalves PJ
    2. Lueckmann JM
    3. Deistler M
    4. Nonnenmacher M
    5. Öcal K
    6. Bassetto G
    7. Chintaluri C
    8. Podlaski WF
    9. Haddad SA
    10. Vogels TP
    11. Greenberg DS
    12. Macke JH

    (2020) Training deep neural density estimators to identify mechanistic models of neural dynamics

    eLife 9:e56261.

    https://doi.org/10.7554/eLife.56261

    • PubMed
    • Google Scholar
30. Conference

    1. Greenberg D
    2. Nonnenmacher M
    3. Macke J

    (2019)

    Automatic posterior transformation for Likelihood-Free inference

    International Conference on Machine Learning, PMLR. pp. 2404–2414.

    • Google Scholar
31. Thesis

    1. Guillaumes AB

    (2017)

    Mixture density networks for distribution and uncertainty estimation

    Universitat Politècnica de Catalunya. Facultat d’Informàtica de Barcelona, PhD thesis.

    • Google Scholar
32. 1. Gutenkunst RN
    2. Waterfall JJ
    3. Fergal PC
    4. Brown KS
    5. Myers CR
    6. Sethna JP

    (2007) Sloppy models and parameter indeterminancy in systems biology

    PLOS Computational Biology 3:30189.

    https://doi.org/10.1371/journal.pcbi.0030189

    • Google Scholar
33. 1. Gutmann MU
    2. Dutta R
    3. Kaski S
    4. Corander J

    (2018) Likelihood-free inference via classification

    Statistics and Computing 28:411–425.

    https://doi.org/10.1007/s11222-017-9738-6

    • PubMed
    • Google Scholar
34. 1. Hawkins GE
    2. Forstmann BU
    3. Wagenmakers EJ
    4. Ratcliff R
    5. Brown SD

    (2015) Revisiting the evidence for collapsing boundaries and urgency signals in perceptual decision-making

    Journal of Neuroscience 35:2476–2484.

    https://doi.org/10.1523/JNEUROSCI.2410-14.2015

    • PubMed
    • Google Scholar
35. 1. Heathcote A
    2. Lin YS
    3. Reynolds A
    4. Strickland L
    5. Gretton M
    6. Matzke D

    (2019) Dynamic models of choice

    Behavior Research Methods 51:961–985.

    https://doi.org/10.3758/s13428-018-1067-y

    • PubMed
    • Google Scholar
36. 1. Herz DM
    2. Zavala BA
    3. Bogacz R
    4. Brown P

    (2016) Neural correlates of decision thresholds in the human subthalamic nucleus

    Current Biology 26:916–920.

    https://doi.org/10.1016/j.cub.2016.01.051

    • PubMed
    • Google Scholar
37. 1. Hoffman MD
    2. Gelman A

    (2014)

    The no-u-turn sampler: adaptively setting path lengths in hamiltonian monte carlo

    Journal of Machine Learning Research : JMLR 15:1593–1623.

    • Google Scholar
38. 1. Holmes WR

    (2015) A practical guide to the probability density approximation (PDA) with improved implementation and error characterization

    Journal of Mathematical Psychology 68-69:13–24.

    https://doi.org/10.1016/j.jmp.2015.08.006

    • Google Scholar
39. Book

    1. Huber PJ

    (1992) Robust estimation of a location parameter

    In: Kotz S, Johnson N. L, editors. Breakthroughs in Statistics. Springer. pp. 492–518.

    https://doi.org/10.1007/978-1-4612-4380-9_35

    • Google Scholar
40. 1. Huys QJ
    2. Maia TV
    3. Frank MJ

    (2016) Computational psychiatry as a bridge from neuroscience to clinical applications

    Nature Neuroscience 19:404–413.

    https://doi.org/10.1038/nn.4238

    • PubMed
    • Google Scholar
41. 1. Järvenpää M
    2. Gutmann MU
    3. Vehtari A
    4. Marttinen P

    (2018) Gaussian process modelling in approximate bayesian computation to estimate horizontal gene transfer in Bacteria

    The Annals of Applied Statistics 12:2228–2251.

    https://doi.org/10.1214/18-AOAS1150

    • Google Scholar
42. 1. Järvenpää M
    2. Gutmann MU
    3. Vehtari A
    4. Marttinen P

    (2021) Parallel gaussian process surrogate bayesian inference with noisy likelihood evaluations

    Bayesian Analysis 16:147–178.

    https://doi.org/10.1214/20-BA1200

    • Google Scholar
43. Preprint

    1. Kingma DP
    2. Ba J

    (2014) Adam: a method for stochastic optimization

    arXiv.

    https://arxiv.org/abs/1412.6980

    • Google Scholar
44. 1. Krajbich I
    2. Rangel A

    (2011) Multialternative drift-diffusion model predicts the relationship between visual fixations and choice in value-based decisions

    PNAS 108:13852–13857.

    https://doi.org/10.1073/pnas.1101328108

    • PubMed
    • Google Scholar
45. Preprint

    1. Lipton A
    2. Kaushansky V

    (2018) On the first hitting time density of an ornstein-uhlenbeck process

    arXiv.

    https://arxiv.org/abs/1810.02390

    • Google Scholar
46. Book

    1. Liu JS

    (2008) Monte Carlo Strategies in Scientific Computing

    Springer Science & Business Media.

    https://doi.org/10.1007/978-0-387-76371-2

    • Google Scholar
47. Conference

    1. Lueckmann J-M
    2. Bassetto G
    3. Karaletsos T
    4. Macke JH

    (2019)

    Likelihood-free inference with emulator networks

    Symposium on Advances in Approximate Bayesian Inference, PMLR. pp. 32–53.

    • Google Scholar
48. 1. Malhotra G
    2. Leslie DS
    3. Ludwig CJH
    4. Bogacz R

    (2018) Time-varying decision boundaries: insights from optimality analysis

    Psychonomic Bulletin & Review 25:971–996.

    https://doi.org/10.3758/s13423-017-1340-6

    • PubMed
    • Google Scholar
49. Preprint

    1. Meeds E
    2. Welling M

    (2014) Gps-abc: gaussian process surrogate approximate bayesian computation

    arXiv.

    https://arxiv.org/abs/1401.2838

    • Google Scholar
50. 1. Mestdagh M
    2. Verdonck S
    3. Meers K
    4. Loossens T
    5. Tuerlinckx F

    (2019) Prepaid parameter estimation without likelihoods

    PLOS Computational Biology 15:e1007181.

    https://doi.org/10.1371/journal.pcbi.1007181

    • PubMed
    • Google Scholar
51. Preprint

    1. Minka TP

    (2013) Expectation propagation for approximate bayesian inference

    arXiv.

    https://arxiv.org/abs/1301.2294

    • Google Scholar
52. Report

    1. Mullowney P
    2. Iyengar S

    (2006)

    Maximum Likelihood Estimation and Computation for the Ornstein-Uhlenbeck Process

    stat.pitt.

    • Google Scholar
53. 1. Navarro DJ
    2. Fuss IG

    (2009) Fast and accurate calculations for first-passage times in Wiener Diffusion models

    Journal of Mathematical Psychology 53:222–230.

    https://doi.org/10.1016/j.jmp.2009.02.003

    • Google Scholar
54. Thesis

    1. Neal RM

    (1995)

    Bayesian learning for neural networks

    University of Toronto, PhD thesis.

    • Google Scholar
55. 1. Neal RM

    (2003) Slice sampling

    The Annals of Statistics 31:705–741.

    https://doi.org/10.1214/aos/1056562461

    • Google Scholar
56. 1. Nilsson H
    2. Rieskamp J
    3. Wagenmakers E-J

    (2011) Hierarchical bayesian parameter estimation for cumulative prospect theory

    Journal of Mathematical Psychology 55:84–93.

    https://doi.org/10.1016/j.jmp.2010.08.006

    • Google Scholar
57. 1. Niv Y
    2. Edlund JA
    3. Dayan P
    4. O'Doherty JP

    (2012) Neural prediction errors reveal a risk-sensitive reinforcement-learning process in the human brain

    Journal of Neuroscience 32:551–562.

    https://doi.org/10.1523/JNEUROSCI.5498-10.2012

    • PubMed
    • Google Scholar
58. 1. O'Reilly RC
    2. Frank MJ

    (2006) Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia

    Neural Computation 18:283–328.

    https://doi.org/10.1162/089976606775093909

    • PubMed
    • Google Scholar
59. 1. Palestro JJ
    2. Weichart E
    3. Sederberg PB
    4. Turner BM

    (2018) Some task demands induce collapsing bounds: evidence from a behavioral analysis

    Psychonomic Bulletin & Review 25:1225–1248.

    https://doi.org/10.3758/s13423-018-1479-9

    • PubMed
    • Google Scholar
60. Preprint

    1. Papamakarios G
    2. Nalisnick E
    3. Rezende DJ
    4. Mohamed S
    5. Lakshminarayanan B

    (2019a) Normalizing flows for probabilistic modeling and inference

    arXiv.

    https://arxiv.org/abs/1912.02762

    • Google Scholar
61. Conference

    1. Papamakarios G
    2. Sterratt D
    3. Murray I

    (2019b)

    Sequential neural likelihood: fast likelihood-free inference with autoregressive flows PMLR

    The 22nd International Conference on Artificial Intelligence and Statistics. pp. 837–848.

    • Google Scholar
62. Conference

    1. Papamakarios G
    2. Murray I

    (2016)

    Fast ε-free inference of simulation models with bayesian conditional density estimation

    Advances in Neural Information Processing Systems. pp. 1028–1036.

    • Google Scholar
63. 1. Pedersen ML
    2. Frank MJ

    (2020) Simultaneous hierarchical bayesian parameter estimation for reinforcement learning and drift diffusion models: a tutorial and links to neural data

    Computational Brain & Behavior 3:458–471.

    https://doi.org/10.1007/s42113-020-00084-w

    • Google Scholar
64. Conference

    1. Radev ST
    2. Mertens UK
    3. Voss A
    4. Ardizzone L
    5. Kothe U

    (2020a) BayesFlow: learning complex stochastic models with invertible neural networks

    IEEE Transactions on Neural Networks and Learning Systems. pp. 1–15.

    https://doi.org/10.1109/TNNLS.2020.3042395

    • Google Scholar
65. 1. Radev ST
    2. Mertens UK
    3. Voss A
    4. Köthe U

    (2020b) Towards end-to-end likelihood-free inference with convolutional neural networks

    British Journal of Mathematical and Statistical Psychology 73:23–43.

    https://doi.org/10.1111/bmsp.12159

    • PubMed
    • Google Scholar
66. 1. Rangel A
    2. Camerer C
    3. Montague PR

    (2008) A framework for studying the neurobiology of value-based decision making

    Nature Reviews Neuroscience 9:545–556.

    https://doi.org/10.1038/nrn2357

    • PubMed
    • Google Scholar
67. 1. Ratcliff R

    (1978) A theory of memory retrieval

    Psychological Review 85:59–108.

    https://doi.org/10.1037/0033-295X.85.2.59

    • Google Scholar
68. 1. Ratcliff R
    2. Childers R

    (2015) Individual differences and fitting methods for the two-choice diffusion model of decision making

    Decision 2:237–279.

    https://doi.org/10.1037/dec0000030

    • Google Scholar
69. 1. Ratcliff R
    2. Frank MJ

    (2012) Reinforcement-based decision making in corticostriatal circuits: mutual constraints by neurocomputational and diffusion models

    Neural Computation 24:1186–1229.

    https://doi.org/10.1162/NECO_a_00270

    • PubMed
    • Google Scholar
70. 1. Ratcliff R
    2. McKoon G

    (2008) The diffusion decision model: theory and data for two-choice decision tasks

    Neural Computation 20:873–922.

    https://doi.org/10.1162/neco.2008.12-06-420

    • PubMed
    • Google Scholar
71. 1. Reynolds AM
    2. Rhodes CJ

    (2009) The lévy flight paradigm: random search patterns and mechanisms

    Ecology 90:877–887.

    https://doi.org/10.1890/08-0153.1

    • PubMed
    • Google Scholar
72. Conference

    1. Rezende D
    2. Mohamed S

    (2015)

    Variational inference with normalizing flows

    International Conference on Machine Learning, PMLR. pp. 1530–1538.

    • Google Scholar
73. 1. Robert C
    2. Casella G

    (2011) A short history of markov chain monte carlo: subjective recollections from incomplete data

    Statistical Science 26:102–115.

    https://doi.org/10.1214/10-STS351

    • Google Scholar
74. Book

    1. Robert C
    2. Casella G

    (2013) Monte Carlo Statistical Methods

    Springer Science & Business Media.

    https://doi.org/10.1007/978-1-4757-4145-2

    • Google Scholar
75. 1. Schönberg T
    2. Daw ND
    3. Joel D
    4. O'Doherty JP

    (2007) Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making

    Journal of Neuroscience 27:12860–12867.

    https://doi.org/10.1523/JNEUROSCI.2496-07.2007

    • PubMed
    • Google Scholar
76. 1. Shinn M
    2. Lam NH
    3. Murray JD

    (2020) A flexible framework for simulating and fitting generalized drift-diffusion models

    eLife 9:e56938.

    https://doi.org/10.7554/eLife.56938

    • PubMed
    • Google Scholar
77. Book

    1. Silverman BW

    (1986) Density Estimation for Statistics and Data Analysis, 26

    CRC press.

    https://doi.org/10.1201/9781315140919

    • Google Scholar
78. Book

    1. Sisson SA
    2. Fan Y
    3. Beaumont M

    (2018) Handbook of Approximate Bayesian Computation

    CRC Press.

    https://doi.org/10.1201/9781315117195

    • Google Scholar
79. 1. Storn R
    2. Price K

    (1997) Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces

    Journal of Global Optimization 11:341–359.

    https://doi.org/10.1023/A:1008202821328

    • Google Scholar
80. 1. Turner BM
    2. van Maanen L
    3. Forstmann BU

    (2015) Informing cognitive abstractions through neuroimaging: the neural drift diffusion model

    Psychological Review 122:312–336.

    https://doi.org/10.1037/a0038894

    • PubMed
    • Google Scholar
81. 1. Turner BM
    2. Sederberg PB

    (2014) A generalized, likelihood-free method for posterior estimation

    Psychonomic Bulletin & Review 21:227–250.

    https://doi.org/10.3758/s13423-013-0530-0

    • PubMed
    • Google Scholar
82. 1. Turner BM
    2. Van Zandt T

    (2018) Approximating bayesian inference through model simulation

    Trends in Cognitive Sciences 22:826–840.

    https://doi.org/10.1016/j.tics.2018.06.003

    • PubMed
    • Google Scholar
83. 1. Usher M
    2. McClelland JL

    (2001) The time course of perceptual choice: the leaky, competing accumulator model

    Psychological Review 108:550–592.

    https://doi.org/10.1037/0033-295X.108.3.550

    • PubMed
    • Google Scholar
84. 1. van Opheusden B
    2. Acerbi L
    3. Ma WJ

    (2020) Unbiased and efficient log-likelihood estimation with inverse binomial sampling

    PLOS Computational Biology 16:e1008483.

    https://doi.org/10.1371/journal.pcbi.1008483

    • PubMed
    • Google Scholar
85. 1. Vandekerckhove J
    2. Tuerlinckx F
    3. Lee MD

    (2011) Hierarchical diffusion models for two-choice response times

    Psychological Methods 16:44–62.

    https://doi.org/10.1037/a0021765

    • PubMed
    • Google Scholar
86. 1. Virtanen P
    2. Gommers R
    3. Oliphant TE
    4. Haberland M
    5. Reddy T
    6. Cournapeau D
    7. Burovski E
    8. Peterson P
    9. Weckesser W
    10. Bright J
    11. van der Walt SJ
    12. Brett M
    13. Wilson J
    14. Millman KJ
    15. Mayorov N
    16. Nelson ARJ
    17. Jones E
    18. Kern R
    19. Larson E
    20. Carey CJ
    21. Polat İ
    22. Feng Y
    23. Moore EW
    24. VanderPlas J
    25. Laxalde D
    26. Perktold J
    27. Cimrman R
    28. Henriksen I
    29. Quintero EA
    30. Harris CR
    31. Archibald AM
    32. Ribeiro AH
    33. Pedregosa F
    34. van Mulbregt P
    35. SciPy 1.0 Contributors

    (2020) SciPy 1.0: fundamental algorithms for scientific computing in Python

    Nature Methods 17:261–272.

    https://doi.org/10.1038/s41592-019-0686-2

    • PubMed
    • Google Scholar
87. 1. Wiecki TV
    2. Sofer I
    3. Frank MJ

    (2013) HDDM: hierarchical bayesian estimation of the Drift-Diffusion model in Python

    Frontiers in Neuroinformatics 7:14.

    https://doi.org/10.3389/fninf.2013.00014

    • PubMed
    • Google Scholar
88. 1. Wiecki TV
    2. Frank MJ

    (2013) A computational model of inhibitory control in frontal cortex and basal ganglia

    Psychological Review 120:329–355.

    https://doi.org/10.1037/a0031542

    • PubMed
    • Google Scholar
89. 1. Wieschen EM
    2. Voss A
    3. Radev S

    (2020) Jumping to conclusion? A lévy flight model of decision making

    The Quantitative Methods for Psychology 16:120–132.

    https://doi.org/10.20982/tqmp.16.2.p120

    • Google Scholar
90. 1. Wilson RC
    2. Collins AG

    (2019) Ten simple rules for the computational modeling of behavioral data

    eLife 8:e49547.

    https://doi.org/10.7554/eLife.49547

    • PubMed
    • Google Scholar
91. 1. Wosniack ME
    2. Santos MC
    3. Raposo EP
    4. Viswanathan GM
    5. da Luz MGE

    (2017) The evolutionary origins of lévy walk foraging

    PLOS Computational Biology 13:e1005774.

    https://doi.org/10.1371/journal.pcbi.1005774

    • PubMed
    • Google Scholar
92. 1. Wraith D
    2. Kilbinger M
    3. Benabed K
    4. Cappé O
    5. Cardoso J-F
    6. Fort G
    7. Prunet S
    8. Robert CP

    (2009) Estimation of cosmological parameters using adaptive importance sampling

    Physical Review D 80:023507.

    https://doi.org/10.1103/PhysRevD.80.023507

    • Google Scholar
93. 1. Yartsev MM
    2. Hanks TD
    3. Yoon AM
    4. Brody CD

    (2018) Causal contribution and dynamical encoding in the striatum during evidence accumulation

    eLife 7:e34929.

    https://doi.org/10.7554/eLife.34929

    • PubMed
    • Google Scholar
94. 1. Zajkowski WK
    2. Kossut M
    3. Wilson RC

    (2017) A causal role for right frontopolar cortex in directed, but not random, exploration

    eLife 6:e27430.

    https://doi.org/10.7554/eLife.27430

    • PubMed
    • Google Scholar

Author details

1. Alexander Fengler

   1. Department of Cognitive, Linguistic and Psychological Sciences, Brown University, Providence, United States
   2. Carney Institute for Brain Science, Brown University, Providence, United States

   Contribution

   Conceptualization, Data curation, Software, Formal analysis, Validation, Investigation, Visualization, Methodology, Writing - original draft, Writing - review and editing

   For correspondence

   alexander_fengler@brown.edu

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-0104-3905

2. Lakshmi N Govindarajan

   1. Department of Cognitive, Linguistic and Psychological Sciences, Brown University, Providence, United States
   2. Carney Institute for Brain Science, Brown University, Providence, United States

   Contribution

   Conceptualization, Data curation, Software, Formal analysis, Investigation, Visualization, Methodology, Writing - review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-0936-2919

3. Tony Chen

   Psychology and Neuroscience Department, Boston College, Chestnut Hill, United States

   Contribution

   Software, Investigation

   Competing interests

   No competing interests declared

4. Michael J Frank

   1. Department of Cognitive, Linguistic and Psychological Sciences, Brown University, Providence, United States
   2. Carney Institute for Brain Science, Brown University, Providence, United States

   Contribution

   Conceptualization, Resources, Software, Supervision, Funding acquisition, Validation, Writing - original draft, Writing - review and editing

   For correspondence

   Michael_Frank@brown.edu

   Competing interests

   Senior editor, eLife

   "This ORCID iD identifies the author of this article:" 0000-0001-8451-0523

• 4,268

  views

• 622

  downloads

• 38

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.