HSSM: A Widely Applicable Toolbox for Hierarchical Bayesian Neurocognitive Modeling

Reviewer #1 (Public review):

Summary:

This article describes a new software package, HSSM, for simulating and fitting sequential sampling models. The package consists of three modules - one for simulating the models, one to train neural networks on mappings from behavior to parameter values, and one for combining these tools to fit particular models to users' data.

Strengths:

This is a very detailed description of a new package that is building on an already highly successful package. It promises to become a go-to software package for cognitive modelers, experimentalists, and practitioners.

Weaknesses:

I have only a few critiques of the article, and some are a matter of taste:

(1) I think it would be helpful for the authors to take the reader through one complete example at the end of the article, including loading in a dataset, fitting it with a regression model, checking model convergence, reading out parameter values, and doing posterior predictive checks (etc). It would be helpful to see it all in one place to get a sense of how much code is required to go through the whole process. One or two actual examples would help readers who are not already familiar with HDDM.

(2) The article assumes a certain level of computing and modeling expertise, which somewhat limits its reach. There are many abbreviations and references to other software tools that a reader might not be familiar with. Such readers might feel like HSSM is beyond their reach. Below is a non-exhaustive list of such undefined or unexplained terms:
DDM, API, LBA, fMRI, EEG, JAX, PyTorch, ONNX, MCMC, VI, MAP, PyMC, PyTensor, NUTS, ArviZ, WAIC, LOO, KDE, CLI, YAML, GUI, LBA, RDM, QP.

(3) I found some of the figures/listings to be unpolished, unhelpful, and/or unnecessary. For instance, Figure 3 and Listing 3 seem to just be zoomed-in pieces of Figure 2. In Figure 4, what is the meaning of the little globe traveling between the within-trial and across-trial rows? Why is Figure 5 a figure and not a listing? Does HSSM not produce these plots directly? I have the same question for Figure 7. Figures 8 and 9 seem unnecessary to me, but perhaps they serve a function that I'm missing. Perhaps they could be turned into supplements for Figure 2.

Reviewer #2 (Public review):

Summary:

This manuscript introduces HSSM, a Python-based toolbox for fitting cognitive models, with a specific focus on sequential sampling models. The toolbox brings together several components: a model construction interface, surrogate likelihoods, sampling tools, an inference backend, formula-based regressions, and tools for validation and visualization.

One of the key advantages is that HSSM relies on well-established open-source packages. This ensures both a robust foundation and also opens a potential for future (community-driven) development. While HSSM is not the first publicly available toolbox for fitting sequential sampling models, it introduces several novel features that will be very valuable to researchers in the field.

Strengths:

The biggest strength of HSSM is its flexibility and ease of use for hierarchical modeling. Many existing toolboxes work as closed systems that are hard to modify. In contrast, HSSM's modular design allows it to be used either as a stand-alone tool or to pick out specific components to integrate into existing pipelines. In addition, the toolbox combines simulation-based inference, surrogate likelihoods, and formula-based regression. This opens up a lot of new modeling possibilities and makes it easy to incorporate trial-by-trial neural or physiological covariates alongside standard RT and choice data.

Weaknesses:

The paper provides a high-level overview of the toolbox, rather than a didactic walk-through that shows how to use it in practice. Additionally, despite being framed as a broad toolbox for "neurocognitive modeling", HSSM currently focuses on sequential sampling models. While these models are widely used, they represent only a small slice of neurocognitive modeling as a whole. Additionally, the toolbox is currently in beta phase, and lots of planned extensions are not implemented yet. Finally, there is currently no information on general performance benchmarks.