 # A resource-rational theory of set size effects in human visual working memory

1. Ronald van den Berg 2. Wei Ji Ma 1. University of Uppsala, Sweden
2. New York University, United States
Research Article
9 figures, 4 tables, 1 data set and 1 additional file

## Figures

Figure 1 Effects of probing probability and set size on J¯optimal in the resource-rational model for local tasks. (A) The model has three different optimal solutions depending on probing probability pi: invest no resource when pi is smaller than some threshold value p0, invest infinite resource when pi is larger than p∞, and invest a finite amount of resource when p0 1, then only the first regime exists; if p0 <1 < p∞ then only the first two regimes exist. (B) If, in addition, pi = 1/N, then the domain of N partitions in a similar manner. https://doi.org/10.7554/eLife.34963.003
Figure 2 with 1 supplement A resource-rational model for delayed-estimation tasks. (A) Example of a trial in a delayed-estimation experiment. The subject is briefly presented with a set of stimuli and, after a short delay, reports the value of the item at a randomly chosen location (here indicated with thick circle). (B) The distribution of estimation errors in delayed-estimation experiments typically widens with set size (data from Experiment E5 in Table 1). (C) This suggests that the amount of resource per encoded item decreases with set size. The estimated amount of resource per item was computed using the same non-parametric model as the one underlying Figure 3C. (D) Expected cost per item as a function of the amount of invested resource (model parameters: λ = 0.01, β = 2, τ↓0). Left: The expected behavioral cost per item (colored curves) decreases with the amount of invested resource, while the expected neural cost per item increases (black line). Center: The sum of these two costs has a unique minimum, whose location (arrows) depends on probing probability pi. Right: The optimal amount of resource per item increases with the probability that the item will be probed. (E) Expected cost across all items, when each item is probed with a probability pi = 1/N; the model parameters are the same as in D and the set sizes correspond with the values of pi in D. The predicted set size effect (right panel) is qualitatively similar to set size effects observed in empirical data (cf. panel C). (D) and (E) are alternative illustrations of the same optimization problem; the right panel of (E) could also be obtained by replotting the right panel of (D) as a function of N = 1/pi. https://doi.org/10.7554/eLife.34963.004
Figure 2—figure supplement 1 Fits to the three delayed-estimation benchmark data sets that were excluded from the main analyses. Circular variance (top) and circular kurtosis (bottom) of the estimation error distributions as a function of set size, split by experiment. Error bars and shaded areas represent 1 s.e.m. of the mean across subjects. The first three data sets were excluded from the main analyses on the grounds that they were published in papers that were later retracted (Anderson and Awh, 2012; Anderson et al., 2011). The data set from the study by Rademaker et al. (2012) was excluded from the main analyses because it contains only two set sizes, which makes it less suitable for a fine-grained study of the relationship between encoding precision and set size. https://doi.org/10.7554/eLife.34963.006
Figure 3 Model fits to data from six delayed-estimation experiments with equal probing probabilities. (A) Maximum-likelihood fits to raw data of the worst-fitting and best-fitting subjects (subjects S10 in E6 and S4 in E4, respectively). Goodness of fit was measured as R2, computed for each subject by concatenating histograms across set sizes. (B) Subject-averaged circular variance and kurtosis of the estimation error, as a function of set size and split by experiment. The maximum-likelihood fits of the model account well for the trends in these statistics. (C) Estimated amounts of resource per item in the resource-rational model scattered against the estimates in the non-parametric model. Each dot represents estimates from a single subject. (D) Estimated amount of resource per item (red) and total resource (black) plotted against set size. Here and in subsequent figures, error bars and shaded areas represent 1 s.e.m. of the mean across subjects. https://doi.org/10.7554/eLife.34963.007
Figure 4 Model fits to data from three delayed-estimation experiments with unequal probing probabilities. (A) Fits of the resource-rational model (curves) to the data (black circles) of experiments E7-E9. (B) Estimated amount of resource per item as a function of probing probability (red) and the corresponding estimated total amount of resource that the subject would spend on encoding a display filled with items with equal probing probabilities (black). (C) Error histograms and a plot of J¯optimal as a function of pi for a single subject (S4 in E9). The estimated value of p0 was 0.18 for this subject, which was larger than the smallest probing probability in the experiment. The error histograms for items with the four lowest probing probabilities appear to be uniform for this subject, which is indicative of guessing (p>0.23 in Kolgomorov-Smirnov tests for uniformity on these four distributions). https://doi.org/10.7554/eLife.34963.010
Figure 5 A resource-rational model for change detection tasks. (A) Example of a trial in a change detection task with a set size of 2. The subject is sequentially presented with two sets of stimuli and reports whether there was a change at any of the item locations. (B) Simulated expected total cost in the resource-rational cost function applied to a task with a set size of 2 and a reward of 0.05 (left), 0.20 (center), or 0.35 (right) units per correct trial. The red dot indicates the location of minimum cost, that is the resource-optimal combination of J¯1 and J¯2 (note that the expected cost function in the central panel has a minimum at two distinct locations). When reward is low (left), the optimal strategy is to encode neither of the two stimuli. When reward is high (right), the optimal strategy is to encode both stimuli with equal amounts of resource. For intermediate reward (center), the optimal strategy is to encode one of the two items, but not the other one. (C) Model predictions as a function of trial rewards at N = 2. Left: The amount of resource assigned to the two items for a range of reward values. Right: the corresponding optimal number of encoded items (top) and optimal amount of resource per encoded item (bottom) as a function of reward. (D) Model predictions as a function of set size (trial reward = 1.5). The model predicts set size effects in both the number of encoded items (left, top) and the amount of resource with which these items are encoded (left, bottom). Moreover, the model produces response data (right) that are qualitatively similar to human data (see, for example, Figure 2C in Keshvari et al., 2013). The parameter values used in all simulations were λ = 0.01 and τ↓0. https://doi.org/10.7554/eLife.34963.011
Figure 6 Model predictions for a delayed-estimation task with binary feedback (N = 5). In this experiment, the observer receives positive feedback (e.g. "correct") when their estimation error is smaller than the positive feedback threshold and negative feedback (e.g. "error") otherwise. We modelled this using a behavioral cost function that maps errors below the feedback threshold to a cost of 0 and errors larger than this threshold to a cost equal to 1. The model predicts that subjects do not invest any resource when the feedback threshold is very small (extremely difficult tasks) or very large (extremely easy tasks), such that the expected absolute estimation error is π/2 (guessing). In an intermediate regime, the prediction is U-shaped and contains a region in which the predicted estimation error barely changes as a function of feedback threshold. In this region, any performance benefit from increasing the amount of invested resource is almost exactly outdone by the added neural cost. The dashed lines show the feedback thresholds corresponding to the "high precision" and "low precision" conditions in the experiment by Nassar et al. (2018). Under the chosen parameter settings (λ = 0.08, τ = 30), the model predicts that the average absolute estimation errors in these two conditions (black circles) are very similar to each other. https://doi.org/10.7554/eLife.34963.012
Appendix 1—figure 1 Examples of the expected behavioral cost function and its first and second derivative under a behavioral cost function cbehavioral(ε)=|ε|β. Different colors represent different choices of parameters β and τ (randomly drawn). https://doi.org/10.7554/eLife.34963.016
Appendix 1–figure 2 Graphical illustration of the solution to the cost-minimization problem that determines the value of J¯optimal. The cost-minimizing value of solution of J¯ lies at the intersection between the derivative of the expected behavioral cost function (black curve) and a flat line at a value –λ/pi (colored lines). This value (indicated with arrows) necessarily increases with pi. The parameter values used in this simulation were the same as those used to generate Figure 2D and E (λ = 0.01, β = 2, τ↓0). https://doi.org/10.7554/eLife.34963.017
Author response image 1

Table 1
Table 2
Table 3
Table 4

## Data availability

Data from experiments E1-E7 (Table 1) and Matlab code for model fitting and simulations are available at http://dx.doi.org/10.5061/dryad.nf5dr6c.

The following data sets were generated
1. 1
Data from: A resource-rational theory of set size effects in human visual working memory (2018)
Available at Dryad Digital Repository under a CC0 Public Domain Dedication.