Proposed neural population dynamics for encoding a single orientation into VWM and maintaining it over a delay. Top: Stimulus onset is followed by a ramping increase in activity (indicated by color) of sensory neurons whose tuning (indicated on y axis) matches the stimulus orientation. Following stimulus offset, this sensory signal rapidly decays. The sensory signal, including its decaying post-stimulus component, provides input into VWM. Bottom: At stimulus onset, the VWM population begins to accumulate activity from the sensory population. This accumulation saturates at a maximum amplitude determined by global normalization. As the sensory activity decays, the activity in the VWM population is maintained at a constant amplitude, but accumulation of random errors causes the activity bump to diffuse along the feature dimension (y axis) over time, changing the orientation represented by the population. At recall, when the VWM population activity is decoded, accuracy of the recall estimate depends on both the orientation represented (centre of the activity bump) and the fidelity with which it can be retrieved (determined by activity amplitude).

Schematic of signal amplitudes in the DyNR model during a cued recall trial. (A) Observers are presented with a memory array (left), followed after a blank delay (not shown) by an arrow cue (center) indicating the location of one item (the target) whose remembered orientation should immediately be reported (right). (B) The amplitude of the visual input associated with each item is modelled as a step function (left). The sensory response (D) is modelled as a low-pass filtering of the stimulus input, with different time constants for rise and decay (C). (F) Amplitude of the working memory signal reflects a saturating accumulation of activity from the sensory population (illustrated in E). Beginning with stimulus onset, activity associated with each item is accumulated from the sensory population into the VWM population, approaching an upper bound (green dashed line) that reflects a total activity limit shared between the N items in memory. Once the cue has been presented (solid orange line) and processed (dashed orange line), uncued items can be dropped from VWM, raising the ceiling on activity available to represent the cued item (green arrow). This allows more information about the cued item to be accumulated from the decaying sensory trace (equivalent to the red shaded area in D). Response variability depends on the asymptotic VWM signal amplitude available for decoding (red circle) combined with the accumulated effects of diffusion (see text).

DyNR model parameters (1–9) and other variables (10–24) used in model description.

Experimental procedure. (A) Experiment 1. On each trial, a memory array was presented consisting of 1, 4, or 10 randomly oriented Gabor stimuli. In 50% of all trials, the stimuli underwent a change of phase and contrast towards the end of the exposure period intended to minimize retinal aftereffects. After a variable delay, an arrow cue was shown pointing towards the location of one stimulus from the preceding array. Observers reported the remembered orientation of the cued stimulus by swiping their index finger on the touchpad. The response was followed by feedback showing the true orientation. (B) In a proportion of trials, the cue was presented simultaneously with the stimuli. (C) Experiment 2. On each trial a memory array consisting of 1, 4, or 10 randomly oriented Gabors was presented for a variable duration, and followed by a white noise flickering mask. The mask was replaced by an arrow cue pointing towards the location of one stimulus from the preceding array. Observers reported its remembered its orientation and received feedback as in Experiment 1. Stimuli are not drawn to scale.

Experiment 1 data and model fits show the consequences of varying set size and delay duration on WM reproduction error. (A) Empirical recall error distributions (black circles) and the DyNR model fits (colored curves). Different panels correspond to different set sizes (rows) and delays (columns). (B) Corresponding RMS errors from experimental data (circles and errorbars) and the DyNR model fits (curves and error patches). Error bars and patches indicate ± 1 SEM.

Time course of sensory and WM gain with variable exposure duration. (A, B) The signal amplitude in the sensory population increases from stimulus onset, exponentially approaching the maximum sensory activity (). For shorter presentation durations (A) the attained amplitude at stimulus offset is only a fraction of the maximum (compare B, late offset). Following offset, sensory areas produce a decaying neural response, that is curtailed (faster decay) but not abolished by a backward mask. (C, D) Information about the stimulus is accumulated in WM from sensory activity. A shorter presentation (C) provides less sensory evidence for the initial accumulation of all items into VWM (compare D, late offset), and subsequently less decaying sensory activity that can supplement VWM activity for the target item following the cue.

Experiment 2 results and modelling data show the consequences of varying set size and stimulus exposure time on VWM reproduction error. (A) Empirical recall error distributions (black circles) and the DyNR model fits (colored curves). Different panels correspond to different set sizes (rows) and exposure durations (columns). (B) Corresponding RMS errors from experimental data (circles and errorbars) and the DyNR model fits (curves and error patches). Error bars and patches indicate ± 1 SEM.

(A) Experiment 1 RMSE for trials without phase shift. (B) Differences in RMSE between trials with and without phase shift across set size and delay conditions. Negative values indicate better performance in the condition without phase shift.

Swap error estimates. (A&B) Probability of swap errors estimated from empirical data using the three-component mixture model [108] in Experiment 1 (A) and Experiment 2 (B). (C&D) Probability of swap errors in best-fitting DyNR model in Experiment 1 (C) and Experiment 2 (D).

Experiment 1 behavioral data and model fit for the DyNR model without sensory persistence after stimulus offset. (A) A version of the DyNR model with equal diffusion across set sizes. (B) A version of the DyNR model with diffusion that scales with set size.

Experiment 2 behavioral data and model fit for the neural model without sensory persistence after stimulus offset. (A) A version of the DyNR model without sensory persistence. (B) Separate fits of the simplified neural model to each exposure time.

Behavioral data and model fit for a neural model with the direct read-out of information from sensory memory for (A) Experiment 1 and (B) Experiment 2.

Behavioral data and model fit for the DyNR model without the cue processing time for (A) Experiment 1 and (B) Experiment 2.

Behavioral data and model fit for a neural model with constant accumulation of information into WM for (A) Experiment 1 and (B) Experiment 2.

Experimental procedure. Stimuli are not drawn to scale.

Behavioral data and model fit for Experiment 1a

Empirical recall error distributions (black circles) from Experiment 1 in Bays et al. [38] and the DyNR model fits to the data (colored curves).

Summary statistics (black circles) from Experiment 1 in Bays et al. [38] and the DyNR model fits to the data (colored curves). The DyNR model was fit to the distributions of recall errors shown in Fig. S10.