Figures and data
Effects of different coupling types within the generative model of the HGF.
Upper row: Plots illustrate the effects of a state’s total (constant) volatility (left), drift (middle) and mean (right) on the state’s evolution over time steps. Each plot shows 30 simulated state trajectories per value of volatility/drift/mean for a continuous state performing a Gaussian random walk over 50 time steps. Lower row: Plots illustrate the effects of phasic changes in a state’s volatility (left), drift (middle), and mean (right). Each plot shows the trajectory of the parent node and one simulated trajectory of the child node coupled to the parent via volatility/drift/mean coupling.
An example of a generative model of sensory inputs with 11 hidden states and two observable outcomes.
In this example, the volatility parents xǎ and 
Comparing the flow of information in the generative model of the HGF with the implied belief network.
A In the generative model, higher-level states influence the evolution of lower-level states (top-down information flow), either by affecting their mean (value coupling, left) or by changing their evolution rate (volatility coupling, right). B Representation of the message-passing within and between belief nodes as implied by the HGF’s belief update equations. New observations cause a cascade of message-passing between nodes that includes bottom-up and top-down information flow. Higher-level beliefs send down their posteriors to inform lower-level predictions. Lower-level belief nodes send prediction errors and the precision of their own prediction bottom-up to drive higher-level belief updating. Within a node, we have placed separate units for the three computational steps that each node has to perform at a given time: the prediction step (green), the update step which results in a new posterior belief (blue), and the prediction error step (red). This message passing scheme generalizes across value and volatility coupling, although the specific messages passed along the connections as well as the computations within the nodes will depend on coupling type (see main text and Figures 4 and 5 for details).
Message-passing for value coupling.
Interactions of two nodes, node a and its value parent node b, are shown during the three steps of a trial (Prediction step, left; Update step, middle; Prediction error step, right). The quantities that are being computed in each step are highlighted in white. Note that each step, we only show the computations for either the parent or the child node. Connections with arrowheads indicate positive (excitatory) influences, connections with circular heads indicate negative (inhibitory) influences. Arrows ending on units indicate additive influences, those ending on other arrows indicate multiplicative influences. Each HGF quantity that changes across trials is assigned its own unit. Parameters (α, κ, λ, ω and ρ) determine connection strengths. For clarity, the volatility and drift nodes Ω and P are only shown during the prediction step.
Message-passing for volatility coupling.
Interactions of two nodes, node ǎ and its volatility parent node a, are shown during the three steps of a trial (Prediction step, left; Update step, middle; Prediction error step, right). The quantities that are being computed in each step are highlighted in white. Logic of display as in figure 4.
Example simulation 1: Local versus global volatility.
A Generative model. Global volatility state xc is a drift value parent to two local volatility states xǎ and 
Example simulation 2: Multisensory cue combination with dynamic noise.
A Generative model. State xc generates two observations on each trial, ua and ub. These could correspond to cues in different modalities, for example a visual and an auditory cue. Both observations are corrupted by noise, the level of which can change from trial to trial according to the hidden noise states xǎ and 
Parameter values Θ used for the example simulations, for the generative process in the environment as well as the HGF’s generative model.
Starting states x(0) in the HGF’s generative model axe Gaussian beliefs with mean and precision (μ0, π0). In all simulations, all drifts ρ were set to zero, and all autoconnection strengths λ were set to 1 (no autoregression). Values are indicated as n/a when state trajectories have been pre-specified instead of simulated, making the parameter value irrelevant. This includes the global volatility drift xc in example 1, the two noise trajectories xǎ and 
Example simulation 3: Multimodel observations.
A Generative model. State xa produces two observations on every trial: continuous observations ua and, through binary hidden state xb, binary observations ub. This could reflect an experiment where the agent has access to binary observations, but also a continuous readout of the probability with which these observations are generated - at least on some trials. The timecourse of this probability can additionally be influenced by a volatility parent xǎ. B Simulated state trajectories (generative model) and observable outcomes. The trajectory for state xa was hand-crafted to reflect a typical experimental protocol in decision-making studies (’bandit’ tasks, where state ub corresponds to a reward outcome, and the probability of being rewarded reverses at some points during the task). Between trials 50 and 150 the participant is only presented the binary outcomes. C Simulated inference. Belief trajectories result from running the belief update equations on the sequence of observations ua and ub. The simulated agent can infer on the hidden state xa. even in the absence of continuous observations, as long as the jumps/reversals are large. Picking up on the more subtle jump around trial 100 is much harder only based on binary observations. Moreover, when the true probability is around 0.5 (from trial 100 onwards), the agent tends to infer a fluctuating probability as opposed to the stable p = 0.5, which improves once they also receive continuous observations (from trial 175), leading to a drop in estimated volatility (top panel). Parameters used for this simulations are given in table 1.
List of variables in the HGF and their notation.
This includes the free parameters χ, as well as various changing belief states.
