Our experiences in the world are perceived and remembered both as individual items, events, and episodes, and also as aggregated collections or sets of related items with common properties. For example, one can remember seeing a brown bear at the zoo, a polar bear at an aquarium, an animated bear in a Winnie the Pooh movie, and on and on; but one also can readily understand the phrase ‘smarter than your average bear’ by aggregating over those individual experiences. A fundamental question in cognitive science is how we extract summary statistics from individual instances, both during perception and working memory (where aggregated information is often referred to as an ‘ensemble’) and during episodic encoding and retrieval (where aggregated information is often referred to as a ‘schema’ or as the ‘gist’ of an experience) [The ‘gist’ in this paper means ‘generalities across individual instances’, instead of ‘lack of details’, ‘less precise’, or ‘abstract’ as in fuzzy trace theory (Brainerd and Reyna, 2002)]. In addition, researchers have attempted to characterize how memory for these types of information changes over time. For example, studies of long-term memory in both humans and animals have demonstrated that gist memory persists or even improves over time, whereas memory for the individual items from which the gist is built fades (Posner and Keele, 1970; Richards et al., 2014).

What do these observations of temporal dissociations tell us about the relation between item memory and gist memory? On the one hand, a persisting gist memory with less accurate item memory is often taken as evidence that a gist representation becomes independent of individual item representations as it is abstracted during encoding (Posner and Keele, 1970) or through consolidation (Richards et al., 2014). On the other hand, a persisting gist memory with less accurate item memory is not sufficient evidence for the independence of a gist representation: Even when item memories become noisy and less accurate, they still can retain enough information to support a relatively intact memory of gist at retrieval (Alvarez, 2011; Squire et al., 2015).

Disentangling these two possibilities based on existing evidence is difficult, because previous studies do not have a direct measurement of the gist information retained in item memories. In this study, we developed a paradigm to test item memory, gist memory, and ‘estimated’ gist memory, which is an estimate of gist memory given the assumption that it is assembled from individual memories of constituent items. Ensemble perception research was a source of inspiration in developing such a paradigm. Studies of rapid perception of complex visual arrays reveal precise representations of gist (i.e. ensemble statistics) with less accurate item memory retrieval in working memory (Ariely, 2001). In order to investigate the relation between item and gist, ensemble perception paradigms often operationalize the gist as the average representation across instances.

Following this reasoning, we operationalize item memory as a set of landmarks (e.g. restaurant and university) whose locations are clustered together on a screen, and gist memory as the center for these landmark locations. In our paradigm, participants learn this set of landmarks, and are then asked to report the spatial center of these landmarks and recall the location of each landmark. Importantly, an ‘estimated center’ can be computed based on their retrieval of individual items, and its accuracy can thus reveal the amount of gist information available in item memories. Thus, we can investigate the relation between gist memories and item memories over the course of long-term memory consolidation by measuring the relationship between the accuracy of the reported center and the estimated center. A positive correlation between estimated and reported center accuracy could mean that participants’ gist memory was still supported by individual item memories, or that the gist was influencing the retrieval of items.

To probe the direction of this relationship, we developed a gist-based bias measurement, an approach borrowed from research on hierarchical clustering models and from semantic memory, which both reveal how gist memory influences memory for specific items (Brady and Alvarez, 2011; Hemmer and Steyvers, 2009; Huttenlocher et al., 1991; Tompary and Thompson-schill, 2019). This measure of bias indicates the magnitude of the particular direction in which the items were attracted. Theories suggest that this influence reveals a reconstructive memory retrieval process (Brady et al., 2015; Hemmer and Steyvers, 2009; Schacter et al., 2011) that depends on the relative strength of item and gist memory (Tompary et al., 2020). Consistent with this theory, prior work in long-term memory consolidation, which examines gist memory that is newly acquired, has shown that over time, as the strength of gist memory is preserved or improves and/or that of item memory decreases, items that are consistent with the gist are recalled more precisely (Richter et al., 2019; Sweegers and Talamini, 2014; Tompary et al., 2020). However, these results did not demonstrate a gist-based bias, a distortion of item memory from such a newly acquired gist. An increasing bias of item memories toward the remembered gist — in this paradigm, the reported center — would be strong evidence for the increasing influence of gist memory over item memories. Our interest in examining the influence of gist memory on item memory at long delays stems from a desire to bridge the literature reviewed above with a potentially related literature reporting the effects of prior knowledge on memory retrieval (e.g. Huttenlocher et al., 2000; Tompary and Thompson-schill, 2019).

The current study aimed to understand the relation between item and gist memory over the course of a month. In Experiment 1, we trained three groups of participants on spatial locations of six landmarks, and we measured the change of error in memory ffor these items as well as the memory for the gist (i.e. the center participants reported) at one of three delay periods: 24 hr, 1 week, or 1 month. We predicted that the accuracy of the reported center would persist or improve despite the accuracy of retrieved items decreasing over a month, as seen in prior work. We extended prior observations by including two new measures—estimated center and gist-based bias—in order to explore how the relation between memories for items and the gist changes over the course of 1 month. In order to understand how influence of a gist memory on item memory changes over time, we compared observed bias from participants with bias generated under two simulations, one assuming that participants only had item memory, the other assuming that participants had item memory and a separate gist representation.

In Experiment 2, we explored the influence of an ‘outlier’ item in spatial location both on the gist, and on the relation between item and gist memories demonstrated in Experiment 1. Research in long-term memory (Richards et al., 2014) and working memory (Whitney and Yamanashi Leib, 2018) shows that outliers that are inconsistent with the pattern across all items differently influence the memory for the pattern, compared to items that are more consistent with the pattern. Outliers greatly disrupted or shifted the overall pattern (Richards et al., 2014), or were discounted in estimating the pattern (Haberman and Whitney, 2010) compared to other items. In Experiment 2, we examined the extent to which the gist representation was influenced more or less by an outlier item over time. We also investigated whether bias in item memory was to the center location including or excluding the outlier item with the same simulation approach as in Experiment 1. Taken together, these experiments provide new information about how item and gist memory are consolidated over time.

We examined how human learners extract the ‘gist’ (generalities, common properties, or summary statistics) across individual instances, and how memory for these instances and for the gist evolve and influence each other over time through two behavioral experiments spanning one to two months. We demonstrated that the accuracy of item memory (memory for spatial locations on a screen) decreased more compared to the accuracy of gist memory (center of the locations) over time, although there was a persistent positive correlation between them. Critically, item memories were increasingly biased toward the gist over time. Participants’ biases grew less similar over time to a simulation relying only on memory for individual items and more similar to a simulation assuming a separate gist representation. In the presence of an outlier item, the local gist, excluding the outlier, became the source of bias, instead of the gist participants directly reported, which consistently overweighted the outlier across time. We think that gist memories, initially built from item memory, gradually developed to guide item memory as their relative strength changed over time.

Consistent with prior research (Antony and Stiver, 2020; Berens et al., 2020; Lutz et al., 2017; Posner and Keele, 1970), item memory became less accurate over time while gist memory remained relatively intact over time. Our findings converge with this prior research even when using an explicit instruction to retrieve gist memory, rather than inferring gist from another measure as in prior research. A shortcoming of prior research is that the relation between item and gist memory over time is rarely assessed. We showed that the relationship between item memory and gist memory persisted across delay periods despite decreased accuracy in item memory. This relationship could have resulted from the influence of gist memory on the retrieval of item memory, from the influence of item memory on gist memory, or both. Our gist-based bias results shed light on the direction of this relationship: Item memory retrieval was biased toward the gist only after 1 month, which suggested that the correlation at 1 month was likely to be due to the influence of gist memory on the retrieval of item memory.

Our findings that items are increasingly biased toward the gist as the accuracy of item memories decreases over time provide new evidence for the memory reconstruction framework, which proposes that memory retrieval is a combination of different sources with varying strength (Brady et al., 2015; Hemmer and Steyvers, 2009; Huttenlocher et al., 2000; Tompary et al., 2020). Our work extends prior evidence of increased schematization in memory consolidation (Richards et al., 2014; Richter et al., 2019; Tompary et al., 2020) by demonstrating a new form of influence from the gist on item memory: gist-based bias. In contrast to prior memory consolidation research that showed increased schematization earlier than 1 month (Graves et al., 2020; Richter et al., 2019; Tompary et al., 2020), the gist-based bias in our current work did not increase by 24 hr or 1 week. This discrepancy could be because the intensive training participants experienced in our paradigm increased the strength of item memories relative to gist memory during learning, and only after a long retention interval did the strength of item memory decrease to an extent that allowed bias to manifest.

The increased bias may reflect a slow systems consolidation process that results in a qualitatively different memory representation after longer retention intervals (McClelland et al., 1995; Richards et al., 2014). An increased reliance on neocortical areas over time would be expected to strengthen gist memory, to the extent that neocortex tends to represent information in a ‘semanticized’ form (Sekeres et al., 2017). The results are also consistent, however, with a change in reliance on different forms of memory within the same memory systems. The current results are not diagnostic on this point — they are consistent with a range of theories on the interplay between episodic and semantic memories over time (Renoult et al., 2019; Richards et al., 2014; Robin and Moscovitch, 2017; Winocur and Moscovitch, 2011; Sekeres et al., 2018).

Our results of increasing gist-based bias over time parallel visual working memory work, which shows evidence of a hierarchical organization of memory: items are more biased toward their center as uncertainty increases in order to increase the overall precision of retrieval (Brady and Alvarez, 2011; Lew and Vul, 2015; Orhan and Jacobs, 2013). Our results detected a similar gist-based bias in long-term memory consolidation. Moreover, in Experiment 2, after a long retention interval, the reported gist overweighted the outlier, whereas the item memories were biased toward the local gist which discounted the outlier. This finding also mirrors prior ensemble perception results that outliers are discounted or excluded in estimating summary statistics (de Gardelle and Summerfield, 2011) and suggests that the gist influencing item retrieval is not a simple average of the items. The results might reveal two different sampling strategies for gist extraction. Because participants had explicit knowledge about the outlier, they might have given more weight to the outlier in explicitly recalling and reporting the gist, similar to the change in the pattern by inconsistent items observed in long-term memory work (Richards et al., 2014). In contrast, the local center that influenced the items might reflect an implicit representation with a sampling strategy discounting the outlier, consistent with findings in perception work (de Gardelle and Summerfield, 2011; Haberman and Whitney, 2010). Our results suggest that visual working memory and long-term memory might be underpinned by a similar reconstructive mechanism and open up new directions to bridge the two fields.

One limitation of the current experiments is that the testing order (i.e. gist memory before item memory) might have encouraged the retrieval of the items to be consistent with the gist (Tversky and Kahneman, 2019; Mutluturk and Boduroglu, 2014). We initially chose this order because we were most interested in the change in gist representation and wanted to minimize the influence of item memories on gist estimation in later recall. We also were concerned that the extent that these two tests influenced the other was not symmetric; in other words, the influence of item memory on gist might be more pronounced than the influence of gist on item memory. Because the testing order is the same for the three different delay intervals, we reasoned that the changes in item memory and bias across delay groups could not simply be a result of the order. In addition, in Experiment 2, the items were not biased toward the center participants reported, suggesting that even if the gist test occurring before item test influences the recall for the items, the influence may be minor. However, multiplicative effects, such as floor or ceiling effects present only at one time point, could still influence the results and the influence from the testing order may still exist. Although the testing order is likely not to influence the change over time, the bias for all delay groups may be lower overall under the reverse testing order. More studies with counterbalanced testing order will be helpful to evaluate this possibility.

Future research can be done to test the generality of our findings to other domains of human cognition. It would be interesting to explore whether our findings, which considered gist memory as a spatial average, would generalize to a broader definition of gist, such as gist-like memory for events (Moscovitch et al., 2016). For example, when first learning what a birthday party is from attending a few, the ‘gist’ representation of a birthday party may be dominated by memory for a few parties, but over time the gist becomes a more stable representation that can influence retrieval of those specific birthday party events. In addition, the dissociation of gist-based bias in Experiment 2 also mirrors the dissociable implicit and explicit attitude in social categories (Gawronski and Bodenhausen, 2006). More work could further disentangle these processes in long-term memory consolidation, which could enlighten our understanding of the cognitive mechanism underlying the formation of gist in social categories.

We began by posing the question of how one extracts a summary statistic from individual instances, but without a doubt, the summary statistic we have used here to answer this question—the arithmetic average of x,y coordinates—is overly simplistic. Firstly, the item-only simulations in our work are oversimplified compared to the item-only models in the categorization literature (Nosofsky, 1988). There surely could be other more sophisticated item-only models that can fit our data. However, our results put a new constraint (i.e. a gist-based bias) for item-only models in long-term memory. Secondly, our implementation of a gist representation in our bias simulations was very simplistic. For example, we assumed an arbitrary amount of gist influence, implementing a more qualitative than quantitative assessment of the presence of a gist representation. Additional experiments with more within-subject statistical power could be used to constrain models that quantify the precise amount of gist influence (as a parameter in individual model fits). Thirdly, this gist influence may be influenced by many other factors, such as the variability in item locations, the accuracy and the confidence of item memories, the distance from the items to the screen center or to the boundary (Intraub and Richardson, 1989), individual differences in cognitive functions (e.g. executive control), and the demand characteristics of explicitly recalling the center. The current design did not allow for enough variability to tease apart these possibilities, but future research systematically manipulating these factors will be helpful in addressing these issues. Finally, and perhaps most importantly, the principles that govern aggregation of individual spatial locations do not in any obvious way translate to the nature of summary statistics for other episodic memories (like that average bear!). Although there is much work to be done to understand the ways in which we aggregate information across multiple experiences, the current experiments should provide a useful launching off pad for future explorations of this question.

In summary, we have shown that memory for individual items and memory for the gist of a set of items changed over the course of long-term memory consolidation. We propose that the gist that was initially extracted from item memories gradually started to guide item memory retrieval over longer durations as their relative memory strength changed. These findings bridge research in areas of cognitive science ranging from perception and working memory to episodic and semantic memory, providing important new insights into our ability both to learn about distinct events and to generalize across similar experiences.

All data generated or analysed during this study are included in the manuscript and supporting files.

1. 1. Zeng T

   (2020) Open Science Framework

   ID jxme8. Tracking the relation between gist and item memory over the course of long-term memory consolidation.

   https://osf.io/jxme8/

1. 1. Alvarez GA

   (2011) Representing multiple objects as an ensemble enhances visual cognition

   Trends in Cognitive Sciences 15:122–131.

   https://doi.org/10.1016/j.tics.2011.01.003

   • Google Scholar
2. Preprint

   1. Antony J
   2. Stiver C

   (2020) Spatial gist extraction during human memory consolidation

   PsyArXiv.

   https://doi.org/10.31234/osf.io/dbxh8

   • Google Scholar
3. 1. Ariely D

   (2001) Seeing sets: representation by statistical properties

   Psychological Science 12:157–162.

   https://doi.org/10.1111/1467-9280.00327

   • PubMed
   • Google Scholar
4. Website

   1. Ariely D
   2. Carmon Z

   (2003) Summary assessment of experiences: the whole is different from the sum of its parts

   Accessed January 26, 2021.

   https://www.ebsco.com/terms-of-use
5. 1. Berens SC
   2. Richards BA
   3. Horner AJ

   (2020) Dissociating memory accessibility and precision in forgetting

   Nature Human Behaviour 4:866–877.

   https://doi.org/10.1038/s41562-020-0888-8

   • Google Scholar
6. 1. Brady T
   2. Schacter D
   3. Alvarez G

   (2015) The adaptive nature of false memories is revealed by Gist-based distortion of true memories

   Journal of Vision 15:948.

   https://doi.org/10.1167/15.12.948

   • Google Scholar
7. 1. Brady TF
   2. Alvarez GA

   (2011) Hierarchical encoding in visual working memory: ensemble statistics Bias memory for individual items

   Psychological Science 22:384–392.

   https://doi.org/10.1177/0956797610397956

   • PubMed
   • Google Scholar
8. 1. Brainerd CJ
   2. Reyna VF

   (2002) Fuzzy-Trace theory and false memory

   Current Directions in Psychological Science 11:164–169.

   https://doi.org/10.1111/1467-8721.00192

   • Google Scholar
9. 1. de Gardelle V
   2. Summerfield C

   (2011) Robust averaging during perceptual judgment

   PNAS 108:13341–13346.

   https://doi.org/10.1073/pnas.1104517108

   • Google Scholar
10. 1. Gawronski B
    2. Bodenhausen GV

    (2006) Associative and propositional processes in evaluation: an integrative review of implicit and explicit attitude change

    Psychological Bulletin 132:692–731.

    https://doi.org/10.1037/0033-2909.132.5.692

    • PubMed
    • Google Scholar
11. 1. Graves KN
    2. Antony JW
    3. Turk-Browne NB

    (2020) Finding the pattern: on-line extraction of spatial structure during virtual navigation

    Psychological Science 31:1183–1190.

    https://doi.org/10.1177/0956797620948828

    • PubMed
    • Google Scholar
12. 1. Haberman J
    2. Whitney D

    (2010) The visual system discounts emotional deviants when extracting average expression

    Attention, Perception & Psychophysics 72:1825–1838.

    https://doi.org/10.3758/APP.72.7.1825

    • Google Scholar
13. 1. Hemmer P
    2. Steyvers M

    (2009) A bayesian account of reconstructive memory

    Topics in Cognitive Science 1:189–202.

    https://doi.org/10.1111/j.1756-8765.2008.01010.x

    • PubMed
    • Google Scholar
14. 1. Huttenlocher J
    2. Hedges LV
    3. Duncan S

    (1991) Categories and particulars: prototype effects in estimating spatial location

    Psychological Review 98:352–376.

    https://doi.org/10.1037/0033-295X.98.3.352

    • PubMed
    • Google Scholar
15. 1. Huttenlocher J
    2. Hedges LV
    3. Vevea JL

    (2000) Why do categories affect stimulus judgment?

    Journal of Experimental Psychology: General 129:220–241.

    https://doi.org/10.1037/0096-3445.129.2.220

    • Google Scholar
16. 1. Intraub H
    2. Richardson M

    (1989) Wide-angle memories of close-up scenes

    Journal of Experimental Psychology: Learning, Memory, and Cognition 15:179–187.

    https://doi.org/10.1037/0278-7393.15.2.179

    • Google Scholar
17. Software

    1. Kay M
    2. Wobbrock JO

    (2020) Aligned Rank Transform, version 0.11.0

    Package ‘ARTool’.

    https://github.com/mjskay/ARTool/
18. 1. Lew TF
    2. Vul E

    (2015) Ensemble clustering in visual working memory biases location memories and reduces the weber noise of relative positions

    Journal of Vision 15:10–14.

    https://doi.org/10.1167/15.4.10

    • Google Scholar
19. 1. Lutz ND
    2. Diekelmann S
    3. Hinse-Stern P
    4. Born J
    5. Rauss K

    (2017) Sleep supports the slow abstraction of gist from visual perceptual memories

    Scientific Reports 7:1–9.

    https://doi.org/10.1038/srep42950

    • Google Scholar
20. 1. McClelland JL
    2. McNaughton BL
    3. O'Reilly RC

    (1995) Why there are complementary learning systems in the Hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory

    Psychological Review 102:419–457.

    https://doi.org/10.1037/0033-295X.102.3.419

    • Google Scholar
21. 1. Moscovitch M
    2. Cabeza R
    3. Winocur G
    4. Nadel L

    (2016) Episodic memory and beyond: the Hippocampus and neocortex in transformation

    Annual Review of Psychology 67:105–134.

    https://doi.org/10.1146/annurev-psych-113011-143733

    • PubMed
    • Google Scholar
22. 1. Mutluturk A
    2. Boduroglu A

    (2014) Effects of spatial configurations on the resolution of spatial working memory

    Attention, Perception, & Psychophysics 76:2276–2285.

    https://doi.org/10.3758/s13414-014-0713-4

    • Google Scholar
23. 1. Nosofsky RM

    (1988) Exemplar-based accounts of relations between classification, recognition, and typicality

    Journal of Experimental Psychology: Learning, Memory, and Cognition 14:700–708.

    https://doi.org/10.1037/0278-7393.14.4.700

    • Google Scholar
24. 1. Orhan AE
    2. Jacobs RA

    (2013) A probabilistic clustering theory of the organization of visual short-term memory

    Psychological Review 120:297–328.

    https://doi.org/10.1037/a0031541

    • PubMed
    • Google Scholar
25. 1. Posner MI
    2. Keele SW

    (1970) Retention of abstract ideas

    Journal of Experimental Psychology 83:304–308.

    https://doi.org/10.1037/h0028558

    • Google Scholar
26. 1. Renoult L
    2. Irish M
    3. Moscovitch M
    4. Rugg MD

    (2019) From knowing to remembering: the Semantic-Episodic distinction

    Trends in Cognitive Sciences 23:1041–1057.

    https://doi.org/10.1016/j.tics.2019.09.008

    • PubMed
    • Google Scholar
27. 1. Richards BA
    2. Xia F
    3. Santoro A
    4. Husse J
    5. Woodin MA
    6. Josselyn SA
    7. Frankland PW

    (2014) Patterns across multiple memories are identified over time

    Nature Neuroscience 17:981–986.

    https://doi.org/10.1038/nn.3736

    • PubMed
    • Google Scholar
28. 1. Richter FR
    2. Bays PM
    3. Jeyarathnarajah P
    4. Simons JS

    (2019) Flexible updating of dynamic knowledge structures

    Scientific Reports 9:1–15.

    https://doi.org/10.1038/s41598-019-39468-9

    • Google Scholar
29. 1. Robin J
    2. Moscovitch M

    (2017) Details, gist and schema: hippocampal–neocortical interactions underlying recent and remote episodic and spatial memory

    Current Opinion in Behavioral Sciences 17:114–123.

    https://doi.org/10.1016/j.cobeha.2017.07.016

    • Google Scholar
30. 1. Schacter DL
    2. Guerin SA
    3. St Jacques PL

    (2011) Memory distortion: an adaptive perspective

    Trends in Cognitive Sciences 15:467–474.

    https://doi.org/10.1016/j.tics.2011.08.004

    • PubMed
    • Google Scholar
31. 1. Schapiro AC
    2. McDevitt EA
    3. Chen L
    4. Norman KA
    5. Mednick SC
    6. Rogers TT

    (2017) Sleep benefits memory for semantic category structure while preserving Exemplar-Specific information

    Scientific Reports 7:1–13.

    https://doi.org/10.1038/s41598-017-12884-5

    • PubMed
    • Google Scholar
32. Book

    1. Sekeres MJ
    2. Moscovitch M
    3. Winocur G

    (2017) Mechanisms of memory consolidation and transformation

    In: Axmacher N, Rasch B, editors. Cognitive Neuroscience of Memory Consolidation. Cham: Springer. pp. 17–44.

    https://doi.org/10.1007/978-3-319-45066-7

    • Google Scholar
33. 1. Sekeres MJ
    2. Winocur G
    3. Moscovitch M

    (2018) The Hippocampus and related neocortical structures in memory transformation

    Neuroscience Letters 680:39–53.

    https://doi.org/10.1016/j.neulet.2018.05.006

    • PubMed
    • Google Scholar
34. 1. Squire LR
    2. Genzel L
    3. Wixted JT
    4. Morris RG

    (2015) Memory consolidation

    Cold Spring Harbor Perspectives in Biology 7:a021766.

    https://doi.org/10.1101/cshperspect.a021766

    • PubMed
    • Google Scholar
35. 1. Sweegers CC
    2. Talamini LM

    (2014) Generalization from episodic memories across time: a route for semantic knowledge acquisition

    Cortex 59:49–61.

    https://doi.org/10.1016/j.cortex.2014.07.006

    • PubMed
    • Google Scholar
36. 1. Tompary A
    2. Zhou W
    3. Davachi L

    (2020) Schematic memories develop quickly, but are not expressed unless necessary

    Scientific Reports 10:16968.

    https://doi.org/10.1038/s41598-020-73952-x

    • PubMed
    • Google Scholar
37. Preprint

    1. Tompary A
    2. Thompson-schill SL

    (2019) Semantic influences on episodic memory distortions

    PsyArXiv.

    https://doi.org/10.31234/osf.io/u754s

    • Google Scholar
38. 1. Tversky A
    2. Kahneman D

    (2019) Judgment under uncertainty

    Foundations of Cognitive Psychology 185:1124–1131.

    https://doi.org/10.1126/science.185.4157.1124

    • Google Scholar
39. 1. Whitney D
    2. Yamanashi Leib A

    (2018) Ensemble perception

    Annual Review of Psychology 69:105–129.

    https://doi.org/10.1146/annurev-psych-010416-044232

    • PubMed
    • Google Scholar
40. 1. Winocur G
    2. Moscovitch M

    (2011) Memory transformation and systems consolidation

    Journal of the International Neuropsychological Society 17:766–780.

    https://doi.org/10.1017/S1355617711000683

    • PubMed
    • Google Scholar

Acceptance summary:

This paper addresses an important and as yet unresolved question in the cognitive science of memory, namely how do we extract gist or summary representations across related experiences, and how does this relationship change over time? Using innovative behavioural paradigms, the authors demonstrate that gist memory remains robust after 1 month, while item memory becomes increasingly biased towards the gist representation. This provides important new clues as to how we consolidate memories of experiences and their gist.

Decision letter after peer review:

Thank you for submitting your article "Tracking the relation between gist and item memory over the course of long-term memory consolidation" for consideration by eLife. Your article has been reviewed by 3 peer reviewers, one of whom is a member of our Board of Reviewing Editors, and the evaluation has been overseen by Chris Baker as the Senior Editor. The following individuals involved in review of your submission have agreed to reveal their identity: Karsten Rauss (Reviewer #2); Richard Henson (Reviewer #3).

The reviewers have discussed their reviews with one another, and the Reviewing Editor has drafted this to help you prepare a revised submission.

Reviewer #1:

Zeng and colleagues present an interesting and timely exposition of how memory for items and generalities across related experiences are formed and influence each other. Across two carefully designed longitudinal studies spanning 1-2 months, the authors integrate insights from ensemble perception research to develop a novel landmark learning paradigm in which a set of landmarks are clustered together. Participants were required to learn the specific location of each landmark (item memory) as well as the spatial centre of the locations (reported gist). Leveraging hierarchical clustering models, the authors computed a gist-based bias measurement, enabling them to comment on the extent to which gist memory influences memory for specific items, as well as an index of estimated centre assembled from the retrieval of individual items. This enabled the authors to establish the amount of gist information available in item memories, and to tease apart the direction of the relationship between item and gist memory. I particularly appreciated Study 2's exploration of how the presence of an "outlier" item in spatial location impacts the gist and the relationship between item and gist representations from Study 1. This innovative approach allowed them to determine the extent to which the gist is robust against outlier items over time.

Overall, I enjoyed reading this manuscript. It elegantly addresses an important and as yet unresolved question in cognitive science, namely the extent to which gist representations become independent of individual item memories as they are extracted during encoding and how this relationship potentially changes over time. The inclusion of a longitudinal dimension further enables us to understand something of the temporality of consolidation over short versus longer time periods. The manuscript is very well written, the experimental studies have been meticulously conducted using a novel interdisciplinary approach, limitations are appropriately acknowledged, and the conclusions drawn are highly appropriate and measured. I believe this study will make a very nice contribution to the memory literature and will certainly spur new lines of enquiry in this field.

Reviewer #2:

It is often said that the amount of information humans can commit to long-term memory is essentially unlimited. However, this large storage capacity partly rests on our ability to compress new information before and during memory consolidation. One way to achieve such compression is by storing the common denominator of related experiences, which is often referred to as gist memory. During later recall, the brain is thought to deduce individual memories from the stored gist representation, rather than reading them out one by one. This not only reduces the memory footprint associated with learning, it also supports adaptive generalization of learning to novel situations.

Zeng and colleagues address two important questions regarding gist memory: First, how do gist memory traces develop over time in comparison to memory traces for individual items? And second, how do the two classes of memory traces influence each other?

The authors developed an experimental protocol in which healthy participants learn the association between landmark labels (e.g. university, park, restaurant) and abstract locations on a computer screen. Memory is subsequently measured in terms of the precision with which participants are able to indicate the learned locations, as well as their geometric center (i.e., the gist location).

A major strength of this approach is that it allows for precise, quantitative assessments of item and gist memory, and how they influence each other. On the other hand, the abstract nature of the task makes it difficult to generalize the results to other forms of gist memory. Moreover, gist memory is explicitly assessed directly after initial learning, which does not allow for clear conclusions as to whether similar memory traces would have developed spontaneously. Nevertheless, the protocol represents a valuable tool for studying the interaction of related memory traces over time.

The authors performed two experiments using this task. In the first experiment, they show that gist memory is more stable over one month compared to item memory; and that gist and item memory are positively related across all measured time-points (i.e., after 24 hours, one week, and one month). However, the nature of this positive correlation changes over time, with gist memory biasing item memory only after one month. The results of a second experiment corroborate these conclusions. They additionally indicate that outlier items (i.e., landmarks that are spatially distant from the cluster of all other landmarks) affect explicit, but not implicit gist memory, with only the latter biasing memories for individual items. Taken together, these results corroborate the notion that the importance of gist memories in guiding recall increases over time. Importantly, the data also provide an estimate of the relevant time window, i.e. between one week and one month after learning.

My only major concern is that the experimental task provides a rather restricted view of gist memory. Thus, it remains unclear what the results mean for other kinds of gist memory, both visual and otherwise. The authors discuss this point on p.16 and offer some interesting speculations. However, the main problem I see is that memory for gist locations may reflect a combination of basic perceptual strategies (as acknowledged by the authors in their references to ensemble perception) and the demand characteristics of the task. This issue is most prominently seen in the difference between the effects for global and local centers observed in Experiment 2. Here, the instruction to explicitly recall the center is what may have created (or at least emphasized) the memory for the global center in the first place. More generally, the fact that gist memory was tested first and early on after learning may have changed the associated memory trace, making it difficult to generalize these findings to cases in which gist memories emerge spontaneously. While this may be the price to pay for precise, quantitative error and bias measures of errors, the authors should discuss these limitations in more detail.

Reviewer #3:

A fundamental puzzle in human memory has been whether we retain individual experiences, and extract their gist by pooling over those experiences only when the gist is needed, or whether we also store a separate representation of the gist (regularities). This study uses locations of landmarks on a screen to argue that such a gist representation (the central tendency of the 2D locations of landmarks) is extracted gradually over time during memory consolidation, i.e., over a period between one day and a month after initial encoding of the landmark locations. Furthermore, the study suggests that this gist representation is not influenced by atypical locations, suggesting it is more than the simple average of all item memories, and that atypical stimuli may be encoded and/or retrieved separately.

I think it would really improve the paper if the authors could compare a number of simple computational models, and show the model corresponding to their favoured conclusion (i.e., that a separate gist representation increases its influence on memory over time) is the only one that is consistent with their data. Ideally, this would entail quantitative fitting of parametrised models, but failing that, it might be sufficient to demonstrate qualitatively that certain models can never explain critical results (like the bias measure above, or the effect of an outlier). For example, I presume that an item-only model in which random noise (in both x,y directions) increased with delay could explain their first result of a greater effect of delay on reported item error (Ir) than Gr? But is this model consistent with their regression results, and most importantly, could it never reproduce their bias results?

Another concern is whether the same results would hold if participants' representation of gist (Gr) were more than the simple average of their reported item locations (Ge). For example, would there be any consequences for the authors' conclusions if some 2D locations were represented more accurately than others – e.g., one might expect the location of landmarks close to the edge of the screen to be stored more accurately than ones closer to the centre of the screen (which could be tested by whether edge locations need fewer training trials?). I appreciate that locations are trained to the same criterion, but it is nonetheless possible that some representations are still more precisely encoded than others even after such criterion-training. Then if participants' reported centre were the weighted average of the reported item locations, weighted by each location's remembered precision, could that affect the authors' bias measure? I appreciate that this would not explain why the difference between reported and true centre changes over retention interval, but if one allowed location and precision information to decay at different rates, could this cause such an interaction with retention interval? This could be another model to simulate?

It is unfortunate that the authors did not counterbalance the order of item and gist memory tests. They do consider this limitation in the Discussion, and note that any additive effect of test order would not explain the interaction between memory type and delay, but of course the presence of nonlinear/multiplicative effects (e.g., floor/ceiling effects) means it is not sufficient to conclude that their results could not depend on test order, i.e., it is always better to empirically test generalisation by running the other counterbalancing. This is particularly important here, where there are good a priori, theoretical reasons why test order might matter, e.g., shift people from strategies based on retrieving items or retrieving gist, depending on what type of memory is probed first. So while the authors state that this could be tested in future studies, the paper would be stronger if the authors could run this counterbalancing themselves and show the same results, so that future researchers do not waste time trying to replicate effects that turn out to be conditional on test order.

Essential revisions:

1. As the authors note, a central debate in many fields of cognitive psychology is whether gist is extracted and stored separately from items, or whether only items are stored, and gist is estimated on-the-fly by retrieving multiple items. For example, can memory performance be explained by an exemplar-only model (like Nosofsky's), or is a separate prototype representation needed? I thought that this paradigm offered a neat way to address this debate by comparing the gist reported by participants with the gist estimated from all their item reports. While the two might differ due to random noise, if this difference systematically increased with the retention interval, this would seem to support a separate representation of gist (prototype), whose influence increases with delay/consolidation. However, the authors do not appear to test this simple effect of delay on the difference score between reported gist error (Gr) and estimated gist error (Ge). Is there a reason they didn't do this? They do regress Gr against Ge and Delay, but I'm not sure why regression is better to test the above hypothesis. In any case, the positive relationship between Gr and Ge, but lack of a significant interaction between Ge and Delay, would seem more compatible with an exemplar-only model, i.e., no need for a separate representation of gist. This is where the authors' bias measure may be critical, but I could not intuit whether their finding of increased bias with delay towards Gr (relative to true gist, Gt) rules out an exemplar-only model.

2. I think it would really improve the paper if the authors could compare a number of simple computational models, and show the model corresponding to their favoured conclusion (i.e., that a separate gist representation increases its influence on memory over time) is the only one that is consistent with their data. Ideally, this would entail quantitative fitting of parametrised models, but failing that, it might be sufficient to demonstrate qualitatively that certain models can never explain critical results (like the bias measure above, or the effect of an outlier). For example, I presume that an item-only model in which random noise (in both x,y directions) increased with delay could explain their first result of a greater effect of delay on reported item error (Ir) than Gr? But is this model consistent with their regression results, and most importantly, could it never reproduce their bias results?

3. Another concern is whether the same results would hold if participants' representation of gist (Gr) were more than the simple average of their reported item locations (Ge). For example, would there be any consequences for the authors' conclusions if some 2D locations were represented more accurately than others – e.g., one might expect the location of landmarks close to the edge of the screen to be stored more accurately than ones closer to the centre of the screen (which could be tested by whether edge locations need fewer training trials?). I appreciate that locations are trained to the same criterion, but it is nonetheless possible that some representations are still more precisely encoded than others even after such criterion-training. Then if participants' reported centre were the weighted average of the reported item locations, weighted by each location's remembered precision, could that affect the authors' bias measure? I appreciate that this would not explain why the difference between reported and true centre changes over retention interval, but if one allowed location and precision information to decay at different rates, could this cause such an interaction with retention interval? This could be another model to simulate?

4. A related concern is the more general problem that the mental representations of item locations may not be a simple linear transform of the true locations. If the mental space of locations is a nonlinearly warped version of the physical space (even if still diffeomorphic, e.g., expanded towards the edges of the physical space, rendering edge items more distinct), then not only will the reported item locations (Ir) be systematically biased with respect to their true locations, but the simple average of the reported item locations would not match the reported gist location. Take the simple case of a 1D space with two items encoded at location x=1 and x=3, such that the gist is x=2, but presume the mental representation of those locations were warped by x²: then the reported item locations would be 1 and 9 (before downscaling to the original min/max of the space), resulting in an estimated gist (Ge) of 5, whereas the gist in participants' memory (as reported in Gr) would be 2² = 4. I think this potential nonlinear warping of the stimulus space can be captured to some extent by generalising the Euclidean error measure used by the authors to Minkowski's measure (for 2D vectors) = ((x1-x2)^g + (y1-y2)^g)^1/g (where g=gamma=2 for Euclidean, but could be fit to allow non-Euclidean distances). See for example Tversky and Gati, Psych Review (1982) for further discussion about differences between physical and psychological spaces. When implemented in an item-only model (and the presence of noise that increases with delay), could such nonlinearities/non-Euclidean measures allow such a model to fit the results, particularly the authors' bias measure? Perhaps not, or perhaps the authors can explain such considerations are irrelevant to their results, but I think some thought and ideally modelling about this possibility might be worthwhile.

5. The results from the outlier in Experiment 2 lead the authors to say themselves that "the gist influencing item retrieval is not a simple average of the items" – so they clearly need a more complex model in which outliers are down-weighted according to the type of memory question asked, and I wasn't 100% clear what that model was, i.e how exactly the authors explain all the results of both Experiments. I think this is where some simple computational models extended to handle outliers, would be very helpful, in terms of mechanistic understanding.

6. Regarding the authors' critical bias measure, they write it on page 20 as:

"(Encoded Item – Reported Gist) – (Retrieved Item – Reported Gist) / (Encoded Item – Retrieved Item)". Firstly, parentheses around the numerator are missing, i.e. it should be: ((Encoded Item – Reported Gist) – (Retrieved Item – Reported Gist)) / (Encoded Item – Retrieved Item) at least, compared to the code in their bias_func.m, and compared to what would be necessary for the bias to range between 0 and 1. Secondly, if evaluated literally, this measure would always be 1, ((It – Gr) – (Ir – Gr)) = (It – Ir), which is identical to the denominator (where It = true or encoded item locations). I realise this "equation" was meant to give a conceptual idea rather than literal definition, but I think it would be safer to give the precise definition, which is ( d(It,Gr) – d(Ir,Gr) ) / d(It,Ir) where d = some distance function (and It, Ir and Gr are 2D vectors), and in the authors case, they use a Euclidean distance (as in Point 4 above), i.e. $\mathrm{\text{d}}(a,b)=\sqrt{(a_1-b_1{)}^2+(a_2-b_2{)}^2}$.

7. More importantly, please could the authors help me understand the rationale for this specific bias measure, and why it is so critical for establishing the presence of a gist representation? I appreciate that Gr can be replaced with Ge, but why is this combined measure of item memory better than the simple Gr – Ge gist measure in terms of distinguishing theories? Apologies if I have missed something fundamental to the paper, in which case, perhaps explain a bit more in the Introduction why this particular bias measure is so critical?

8. It is unfortunate that the authors did not counterbalance the order of item and gist memory tests. They do consider this limitation in the Discussion and note that any additive effect of test order would not explain the interaction between memory type and delay, but of course the presence of nonlinear/multiplicative effects (e.g., floor/ceiling effects) means it is not sufficient to conclude that their results could not depend on test order, i.e., it is always better to empirically test generalisation by running the other counterbalancing. This is particularly important here, where there are good a priori, theoretical reasons why test order might matter, e.g., shift people from strategies based on retrieving items or retrieving gist, depending on what type of memory is probed first.

9. The experimental task provides a somewhat restricted view of gist memory. Thus, it remains unclear what the results mean for other kinds of gist memory, both visual and otherwise. While the authors discuss this point on p.16, memory for gist locations may reflect a combination of basic perceptual strategies (as acknowledged by the authors in their references to ensemble perception) and the demand characteristics of the task. This issue is most prominently seen in the difference between the effects for global and local centres observed in Experiment 2. Here, the instruction to explicitly recall the centre is what may have created (or at least emphasized) the memory for the global centre in the first place. The fact that gist memory was tested first and early on after learning may have changed the associated memory trace, making it difficult to generalize these findings to cases in which gist memories emerge spontaneously. While this may be the price to pay for precise, quantitative error and bias measures of errors, the authors should discuss these limitations in more detail.

10. The authors touch on the subject of neural substrates in the Discussion however, I would like to have seen slightly more consideration of this topic. For example, I was surprised by the statement on page: 15 that "neocortex tends to represent information in a 'semanticized' form" – this strikes me as somewhat at odds with more recent theoretical positions on the interplay between episodic and semantic memory (e.g., Renoult et al. TICS 2019) as well the processes by which memory transformation from item to gist memory occurs (Robin and Moscovitch, 2017). While not the focus of this manuscript, it would be helpful for the authors to consider these contemporary positions in relation to their findings.

11. As a point of curiosity, have the authors considered the potential role of individual differences in some of these processes and whether some individuals may be more adept at extracting gist versus item representations?

12. Is there any relationship of the present results to Brainerd and Reyna's "Fuzzy Trace Theory" (https://en.wikipedia.org/wiki/Fuzzy-trace_theory), which is based on the role of verbatim and gist representations in memory?

[Editors' note: further revisions were suggested prior to acceptance, as described below.]

Thank you for submitting your article "Tracking the relation between gist and item memory over the course of long-term memory consolidation" for consideration by eLife. Your article has been re-reviewed by a Reviewing Editor and Chris Baker as the Senior Editor.

The reviewers have discussed their reviews with one another, and the Reviewing Editor has drafted this to help you prepare a revised submission.

Essential revisions:

There was some concern regarding the extent to which some of the comments raised by the reviewers had been addressed in the previous revision. The below issues have been raised by the reviewers as warranting serious consideration.

(i) The failure to counterbalance the order of gist and item tests was not felt to have been satisfactorily discussed in the Discussion. The reviewers all agree that the paradigm is interesting and relevant, but the lack of counterbalanced replication based on an explicit power analysis limits the generalisability of these findings. The authors need to temper their claims and more deeply acknowledge and address the limitations of this study.

(ii) More emphasis needs to be placed on the modelling, moving the models to the main text and discussing these more fully. The reviewers had suggested simulating a range of models, yet only the simplest of these models was simulated (exemplar model with noise), and this was placed in Supplementary Material. While a weighted version was explored and an alternative Minkowski value, it would be more elegant to simulate a range of models up front in the paper. In particular, the reviewers have requested to see a simulation of the model favoured by the authors (with a separate gist representation) to see how it explains the results of Experiment 2 (e.g., how an outlier is determined by people and how its influence changes with delay).

Essential revisions:

1. As the authors note, a central debate in many fields of cognitive psychology is whether gist is extracted and stored separately from items, or whether only items are stored, and gist is estimated on-the-fly by retrieving multiple items. For example, can memory performance be explained by an exemplar-only model (like Nosofsky's), or is a separate prototype representation needed? I thought that this paradigm offered a neat way to address this debate by comparing the gist reported by participants with the gist estimated from all their item reports. While the two might differ due to random noise, if this difference systematically increased with the retention interval, this would seem to support a separate representation of gist (prototype), whose influence increases with delay/consolidation. However, the authors do not appear to test this simple effect of delay on the difference score between reported gist error (Gr) and estimated gist error (Ge). Is there a reason they didn't do this? They do regress Gr against Ge and Delay, but I'm not sure why regression is better to test the above hypothesis. In any case, the positive relationship between Gr and Ge, but lack of a significant interaction between Ge and Delay, would seem more compatible with an exemplar-only model, i.e., no need for a separate representation of gist. This is where the authors' bias measure may be critical, but I could not intuit whether their finding of increased bias with delay towards Gr (relative to true gist, Gt) rules out an exemplar-only model.

In this comment, and one that follows (comment 7), there is a question about why we did not calculate the error difference or the distance between the reported gist and the estimated gist in any of our analyses. This in some ways appears to be the most obvious way to examine changes in the relation between the two over time, but there is an important problem with that measure that we now explain in the manuscript: Because estimated gist was calculated from an aggregation of item memories which might already be influenced by the reported center after a delay, the estimated gist error (Ge) might have been influenced by the reported center too. Therefore, this analysis would not provide evidence for an item-only model, which assumes participants only remembered the items and the item memories would not be influenced by the gist. If this is the case, the difference between Ge and Gr may remain the same across time because of a persisting dependence of the two (possibly, initially the Gr depends on Ge and over time the Ge starts to depend on Gr). Therefore, we did not think testing the difference between Ge and Gr over time would be helpful to distinguish these hypotheses.

However, your comment inspired us to include a follow-up analysis to contrast a simple form of item-only model with the error data. Critically, as with our baseline bias measure, we assumed that the magnitude of error for each item memory would remain the same, but the direction of error would not be systematically influenced by the gist. In brief, we generated 1000 simulations for each participant. Each simulation consisted of all simulated retrieved items, corresponding to all the landmark locations. For each item location, we randomly generated a retrieved location based on the participant’s true error for this specific item location, allowing angle to vary randomly across the simulations. Then, we computed the center for these locations to get the simulated estimated gist for each simulated participant. The error for such simulated estimated gist was the Euclidean distance between the true center and the simulated estimated gist. The simulated estimated gist error for each real participant was the average value of simulated estimated gist error for their corresponding 1000 simulated participants.

In addition to the results from the analyses described above, we also added Ge as a comparison in the results, as you asked.

There is a significant group interaction for three gist error types F(4, 381) = 3.83, p <.01, suggesting the increase of three types of gist errors is different over time. For reported gist error and simulated estimated gist error, we found a significant interaction between delay group and gist memory error type, F(2, 254) = 6.68, p = .001. For estimated gist error and simulated estimated gist error, we also found a significant interaction between delay group and gist memory error type, F(2, 254) = 3.28, p = .039. Gist error and estimated gist error both increase less over time compared to the simulated estimated gist error under this simple item-only model over time, suggesting that participants’ data are not compatible with this simple item-only model. We did not find a significant interaction between Gr and Ge across time, F(2, 254) = 1.18, p = .31. This is consistent with the idea that Ge was calculated from item memories influenced by the center after delay.

There may be other more sophisticated exemplar-based models that can fit our data. However, our results put a constraint (i.e., a center bias) for item-only models, whereas models with a separate gist can readily explain the center bias. We have added a paragraph at the end of Experiment 1 results (page 9, line 195) and included these new results in the supplementary material (page 33, Figure 2-supplement 2). We have also attached the code for these analyses in our most updated code.

2. I think it would really improve the paper if the authors could compare a number of simple computational models, and show the model corresponding to their favoured conclusion (i.e., that a separate gist representation increases its influence on memory over time) is the only one that is consistent with their data. Ideally, this would entail quantitative fitting of parametrised models, but failing that, it might be sufficient to demonstrate qualitatively that certain models can never explain critical results (like the bias measure above, or the effect of an outlier). For example, I presume that an item-only model in which random noise (in both x,y directions) increased with delay could explain their first result of a greater effect of delay on reported item error (Ir) than Gr? But is this model consistent with their regression results, and most importantly, could it never reproduce their bias results?

As discussed in our response to comment 1, we conducted a simulation for the simple item-only model in which random noise increased with delay, and we computed a simulated estimated gist error based on this item-only model (see our response above). Following comment 2, we conducted a regression analysis to evaluate the effect of such simulated estimated gist error on reported gist error. Then, we conducted model comparisons to understand whether reported gist error was best explained by estimated gist error or the simulated gist error based on the item-only model, with delay as an interaction term.

We found that the item-only simulated gist error was associated with gist error (SSE = 14248, F(1, 124) = 4.81, p = .03). However, when the estimated gist error and item-only simulated gist error were both included as predictors, the model did not provide a better fit for the data compared to the model which only has estimated gist error as a predictor (SSE = 500.28, F(3, 121) = 0.062, p = .98). On the other hand, the model with both the estimated gist error and item-only simulated gist error as predictors had a significantly better fit compared to the model which only has item-only simulated gist error as a predictor (SSE = 41842, F(3, 121) = 5.18, p <.01). Taken together, these results suggest that although the item-only model with increased error alone may be associated with an increased gist error, it could not add any explanatory power for the gist error in addition to the estimated gist error.

As for the bias results, the bias in our original submission is a difference score between the absolute bias and baseline bias. The baseline bias is an item-only model in which random noise increased with delay. Our bias results indicate that this simple item-only model could not explain the bias results over time (Figure 3).

3. Another concern is whether the same results would hold if participants' representation of gist (Gr) were more than the simple average of their reported item locations (Ge). For example, would there be any consequences for the authors' conclusions if some 2D locations were represented more accurately than others – e.g., one might expect the location of landmarks close to the edge of the screen to be stored more accurately than ones closer to the centre of the screen (which could be tested by whether edge locations need fewer training trials?). I appreciate that locations are trained to the same criterion, but it is nonetheless possible that some representations are still more precisely encoded than others even after such criterion-training. Then if participants' reported centre were the weighted average of the reported item locations, weighted by each location's remembered precision, could that affect the authors' bias measure? I appreciate that this would not explain why the difference between reported and true centre changes over retention interval, but if one allowed location and precision information to decay at different rates, could this cause such an interaction with retention interval? This could be another model to simulate?

Based on this comment, we first tested whether an estimated center weighting each item by their accuracy will be closer to the center participants reported, compared to the original estimated gist by simple average, using the data from Experiment 1. Each item was given a weight based on its accuracy (1- error/error of all items)/(number of items – 1) such that items with higher accuracy would be weighted more in the estimated gist. We divided each of the weights by the number of items – 1 in order to make the sum of the weight of items equal to 1. We then compared this weighted estimated gist with the estimated gist in their error and the change of their difference in error across time. We found that the two types of error do not differ from each other at 24 hours, 1 week, and 1 month (ps >.1). The difference in these two types of error does not change reliably over time (F(2, 131) = 1.26, p = .28). In addition, we calculated the bias change over time using such weighted estimated gist as center. The bias using the estimated center weighing items by accuracy as bias center increased over time (SSE = 2.452, F(2, 127) = 11.77, p <.001).

Author response image 1

Download asset Open asset

Overall, an estimated center that is weighted based on item memory accuracy does not perform differently than an estimated center that weights all items equally. Again, it is possible that other kinds of models weighing items by their accuracy or their remembered precision (i.e., the confidence for the items) could perform differently, or that a study in which item locations were not learned to criterion would result in more variability in item memory accuracy, which could give rise to a gist memory that is meaningfully weighted by item precision. Our study offers a paradigm as a start for future modeling work to explore how the weights of the items can vary according to various kinds of factors, and whether the weights change over time. We have added this point to the discussion (page 17, line 418).

4. A related concern is the more general problem that the mental representations of item locations may not be a simple linear transform of the true locations. If the mental space of locations is a nonlinearly warped version of the physical space (even if still diffeomorphic, e.g., expanded towards the edges of the physical space, rendering edge items more distinct), then not only will the reported item locations (Ir) be systematically biased with respect to their true locations, but the simple average of the reported item locations would not match the reported gist location. Take the simple case of a 1D space with two items encoded at location x=1 and x=3, such that the gist is x=2, but presume the mental representation of those locations were warped by x²: then the reported item locations would be 1 and 9 (before downscaling to the original min/max of the space), resulting in an estimated gist (Ge) of 5, whereas the gist in participants' memory (as reported in Gr) would be 2² = 4. I think this potential nonlinear warping of the stimulus space can be captured to some extent by generalising the Euclidean error measure used by the authors to Minkowski's measure (for 2D vectors) = ( (x1-x2) ^g + (y1-y2)^g ) ^1/g (where g=gamma=2 for Euclidean, but could be fit to allow non-Euclidean distances). See for example Tversky and Gati, Psych Review (1982) for further discussion about differences between physical and psychological spaces. When implemented in an item-only model (and the presence of noise that increases with delay), could such nonlinearities/non-Euclidean measures allow such a model to fit the results, particularly the authors' bias measure? Perhaps not, or perhaps the authors can explain such considerations are irrelevant to their results, but I think some thought and ideally modelling about this possibility might be worthwhile.

Thank you for pointing this out. Indeed, the mental representations can be nonlinear. We add this point as well as the paper reference in a discussion paragraph (page 17, line 422). Following your suggestion, we have conducted the same bias analyses with g = 1.5 (as Minkowski’s distance usually has values of g between 1 and 2) and found that this bias change increased over time (SSE = 0.86, F(2, 131) = 5.419, p = .005). This suggests that our result of bias increased over time is not limited to Euclidean distance. However, the result is non exhaustive and other forms of nonlinearities/non-Euclidean measures may allow an item-only model to fit, which may be helpful in explaining the change over time. In addition, even though there may be other item-only models with non-linearly warped space that could explain our change in bias over time, it is not clear whether and how this particular distortion of physical distances would change over time.

Author response image 2

Download asset Open asset

5. The results from the outlier in Experiment 2 lead the authors to say themselves that "the gist influencing item retrieval is not a simple average of the items" – so they clearly need a more complex model in which outliers are down-weighted according to the type of memory question asked, and I wasn't 100% clear what that model was, i.e how exactly the authors explain all the results of both Experiments. I think this is where some simple computational models extended to handle outliers, would be very helpful, in terms of mechanistic understanding.

We conducted a simulation in Experiment 2 comparing a series of models which weight outliers from 0 (not weighted at all) to 1 (over-weighted). We found that the model that generates the estimated center that is closest to the reported center is a model that over-weighted outliers (Results: page 13, line 299; Methods: page 26, line 651). This is a result in comparison to models that down-weighted outliers.

This principle of weighting items that are more distant from the center may apply to Experiment 1 as well. This is a valuable point but unfortunately there is not enough variance across the items on their distance to the center (Mean: 266.9, SD: 46) to allow for such analysis and so an aggregated center based on the distance is not much different from a simple average. Future work manipulating the distance of the items towards the center can be done to investigate this principle. We have incorporated this point in the new paragraph illustrating various factors that learners could be weighting items by (page 17, line 423).

6. Regarding the authors' critical bias measure, they write it on page 20 as:

"(Encoded Item – Reported Gist) – (Retrieved Item – Reported Gist) / (Encoded Item – Retrieved Item)". Firstly, parentheses around the numerator are missing, i.e. it should be: ((Encoded Item – Reported Gist) – (Retrieved Item – Reported Gist)) / (Encoded Item – Retrieved Item) at least, compared to the code in their bias_func.m, and compared to what would be necessary for the bias to range between 0 and 1. Secondly, if evaluated literally, this measure would always be 1, ((It – Gr) – (Ir – Gr)) = (It – Ir), which is identical to the denominator (where It = true or encoded item locations). I realise this "equation" was meant to give a conceptual idea rather than literal definition, but I think it would be safer to give the precise definition, which is ( d(It,Gr) – d(Ir,Gr) ) / d(It,Ir) where d = some distance function (and It, Ir and Gr are 2D vectors), and in the authors case, they use a Euclidean distance (as in Point 4 above), i.e. $\mathrm{\text{d}}(a,b)=\sqrt{(a_1-b_1{)}^2+(a_2-b_2{)}^2}$.

Thank you for pointing this out. We have edited the methods section in the manuscript (page 21, line 537) to give a precise definition.

7. More importantly, please could the authors help me understand the rationale for this specific bias measure, and why it is so critical for establishing the presence of a gist representation? I appreciate that Gr can be replaced with Ge, but why is this combined measure of item memory better than the simple Gr – Ge gist measure in terms of distinguishing theories? Apologies if I have missed something fundamental to the paper, in which case, perhaps explain a bit more in the Introduction why this particular bias measure is so critical?

This is an important comment that overlaps with comment 1. The Gr – Ge gist measurement will not be able to distinguish the theories for the same reason raised in comment 1. In brief, because Ge was an aggregation of item memories which might already be influenced by the reported center, this analysis would not provide evidence to isolate the influence from the gist on the items.

On the other hand, the specific bias analysis is critical because, by comparing the data with an item-only model baseline, it disentangles error from the bias and shows the magnitude of the particular direction the items were attracted to. We have added a paragraph in the paper in order to clarify this point (page 9, line 195).

8. It is unfortunate that the authors did not counterbalance the order of item and gist memory tests. They do consider this limitation in the Discussion and note that any additive effect of test order would not explain the interaction between memory type and delay, but of course the presence of nonlinear/multiplicative effects (e.g., floor/ceiling effects) means it is not sufficient to conclude that their results could not depend on test order, i.e., it is always better to empirically test generalisation by running the other counterbalancing. This is particularly important here, where there are good a priori, theoretical reasons why test order might matter, e.g., shift people from strategies based on retrieving items or retrieving gist, depending on what type of memory is probed first.

We revised our claim to acknowledge the possibility of nonlinear/multiplicative effects (page 18, line 433).

9. The experimental task provides a somewhat restricted view of gist memory. Thus, it remains unclear what the results mean for other kinds of gist memory, both visual and otherwise. While the authors discuss this point on p.16, memory for gist locations may reflect a combination of basic perceptual strategies (as acknowledged by the authors in their references to ensemble perception) and the demand characteristics of the task. This issue is most prominently seen in the difference between the effects for global and local centres observed in Experiment 2. Here, the instruction to explicitly recall the centre is what may have created (or at least emphasized) the memory for the global centre in the first place. The fact that gist memory was tested first and early on after learning may have changed the associated memory trace, making it difficult to generalize these findings to cases in which gist memories emerge spontaneously. While this may be the price to pay for precise, quantitative error and bias measures of errors, the authors should discuss these limitations in more detail.

We agree with this comment. A sentence reflecting this point has been added to the Discussion section in the discussion paragraph (page 17, line 430).

10. The authors touch on the subject of neural substrates in the Discussion however, I would like to have seen slightly more consideration of this topic. For example, I was surprised by the statement on page: 15 that "neocortex tends to represent information in a 'semanticized' form" – this strikes me as somewhat at odds with more recent theoretical positions on the interplay between episodic and semantic memory (e.g., Renoult et al. TICS 2019) as well the processes by which memory transformation from item to gist memory occurs (Robin and Moscovitch, 2017). While not the focus of this manuscript, it would be helpful for the authors to consider these contemporary positions in relation to their findings.

This is a very good point. We completely agree that our results are not diagnostic of a change in memory system and are indeed consistent with various views on the interplay between episodic and semantic memory. We have rewritten the relevant portion of the discussion accordingly (page 16, line 367).

11. As a point of curiosity, have the authors considered the potential role of individual differences in some of these processes and whether some individuals may be more adept at extracting gist versus item representations?

Very valuable point. We do not have measures of various kinds of cognitive functions that we could use to relate to the variance in learners’ ability to extract the gist versus item representations. We added this point as a future direction in the discussion (page 17, line 430).

12. Is there any relationship of the present results to Brainerd and Reyna's "Fuzzy Trace Theory" (https://en.wikipedia.org/wiki/Fuzzy-trace_theory), which is based on the role of verbatim and gist representations in memory?

According to this theory, memory of a past event has a verbatim copy which is detailed and a fuzzy copy, which is “gisty,” and people rely on the fuzzy copy in making decisions. The meaning of the word “gist” in the present results and fuzzy trace theory differs in the sense that in fuzzy trace theory the word means “imprecise” but in our study it refers to the generalities across items, in particular, a spatial average. This is an important point and we clarified it in the introduction (page 1, footnote 1).

[Editors' note: further revisions were suggested prior to acceptance, as described below.]

Essential revisions:

There was some concern regarding the extent to which some of the comments raised by the reviewers had been addressed in the previous revision. The below issues have been raised by the reviewers as warranting serious consideration.

(i) The failure to counterbalance the order of gist and item tests was not felt to have been satisfactorily discussed in the Discussion. The reviewers all agree that the paradigm is interesting and relevant, but the lack of counterbalanced replication based on an explicit power analysis limits the generalisability of these findings. The authors need to temper their claims and more deeply acknowledge and address the limitations of this study.

A full paragraph acknowledging and discussing this limitation has been added to the text in the Discussion section (page 20, lines 464-479):

“One limitation of the current experiments is that the testing order (i.e., gist memory before item memory) might have encouraged the retrieval of the items to be consistent with the gist (Tversky and Kahneman, 1974; Mutluturk and Boduroglu, 2014). […] More studies with counterbalanced testing order will be helpful to evaluate this possibility.”

(ii) More emphasis needs to be placed on the modelling, moving the models to the main text and discussing these more fully. The reviewers had suggested simulating a range of models, yet only the simplest of these models was simulated (exemplar model with noise), and this was placed in Supplementary Material. While a weighted version was explored and an alternative Minkowski value, it would be more elegant to simulate a range of models up front in the paper. In particular, the reviewers have requested to see a simulation of the model favoured by the authors (with a separate gist representation) to see how it explains the results of Experiment 2 (e.g., how an outlier is determined by people and how its influence changes with delay).

In our previous submission, we compared our observed data to simulations of bias in a model with noisy item-location representations but no separate gist representation. The reviewer’s suggestion to compare our observed data (and the simulations from an “item only” model) to simulations from an “item plus gist” model is a valuable one, and we have taken some time to consider how best to implement such a model. The complication, of course, is that unlike the selection of an “item only” model, there are an infinite number of “item plus gist” models that we could evaluate, where the degree of influence and the form of the influence of the gist varies. We considered two forms of influence, one in which the gist location influences the probability of a particular retrieved location (instead of the uniform probability distribution around the item location in the “item only model”); and alternatively one in which the gist location shifts the item location (and the surrounding item locations). We ultimately selected the former because it would not change the error of the item memories, making it a closer comparison to the item-only model, and so a version of that “item plus gist” model is now reported in the paper. Turning to the degree of influence, this could be modelled as a parameter in the “item plus gist” model, where increasing influence of gist shifts the probability distribution further from the uniform distribution. The challenge is that, because we did not design the experiment with this sort of modelling exercise in mind, we do not have enough statistical power for a quantitative fitting of parametrized models to determine the best parameter for gist influence per participant (due to the limited number of item retrievals per participant). For the same reason and with no variability in the outlier item, we could not quantitatively examine how the outlier is determined by participants and how its influence changed over time.

We now include a comparison of the observed data to the (original) “item only” model and a new “item plus gist” model (henceforth “gist simulation” for simplicity) as follows: In the gist simulation, the magnitude of error for each item memory is the same as the observed data from participants, but the probability of each location being simulated was inversely related to the distance between that location and the gist (the center) (p25-27, lines 625-664). That is, locations that are closer to the gist were more likely to be retrieved in the simulation compared to locations that are farther from the gist. We computed bias under gist simulation (abbreviated as gist simulated bias) and compared it with the observed bias from participants for both Experiment 1 (methods: p28, lines 675-693) and 2 (methods: p31, lines 764-766). We found that over time, the observed bias became increasingly similar to the gist simulated bias and dissimilar to the item only simulated bias over time in Experiment 1 (as discussed, we could not fit the most accurate parameter of gist influence and therefore the amount of gist influence under the gist simulation is arbitrary and likely not accurately reflecting the amount of gist influence in observed data. Therefore, we did not predict that over time the observed bias would become indistinguishable from the gist simulated bias and focused on testing the change between the difference between the two over time.). In Experiment 2, we observed the same results for the same bias analysis using the local center excluding the outlier (the same analysis is not significant for the global reported center). These results suggest that participants’ bias grew increasingly similar to the gist simulated bias and dissimilar to item only simulated bias over time, suggesting an increasing gist influence on item memories over time. We have added these corresponding results (Experiment 1 results: p8-p10, lines 169-224; Experiment 2 results: p14-16, lines 312-322, 332-343, 352-361) to the main text.

Second, we have added a range of simulations to the main text of Experiment 1, which includes: a bias analysis using the true center of encoded items (rather than the center participants reported), a bias analysis where the center is weighted by item accuracy, and a bias analysis assuming non-Euclidean space (results: “Follow-up bias analyses”, p11, lines 233-261; methods: p28-29, lines 695-708). We found that the results do not differ from our main findings under these different simulations, potentially because the stimuli do not allow for enough variability in item accuray to influence the results.

We know that the efforts we have made to compare our data to different models are not perfect. We hope that the reviewers agree that our results provide a starting point for future endeavours to quantify the gist influence on item memories in long-term memory consolidation and to examine the factors that may affect this influence, such as the variability in item locations and participants’ accuracy and confidence of item memories. We have added these points to the discussion (p21, lines 493-503):

“Our implementation of a gist representation in our bias simulations was very simplistic. […] The current design did not allow for enough variability to tease apart these possibilities, but future research systematically manipulating these factors will be helpful in addressing these issues.”

Author details

1. Tima Zeng

   Department of Psychology, University of Pennsylvania, Philadelphia, United States

   Contribution

   Conceptualization, Data curation, Software, Formal analysis, Investigation, Visualization, Methodology, Writing - original draft, Project administration, Writing - review and editing

   For correspondence

   zengtima@gmail.com

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0003-3805-4701

2. Alexa Tompary

   Department of Psychology, University of Pennsylvania, Philadelphia, United States

   Contribution

   Conceptualization, Formal analysis, Methodology, Writing - review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0001-7735-3849

3. Anna C Schapiro

   Department of Psychology, University of Pennsylvania, Philadelphia, United States

   Contribution

   Conceptualization, Methodology, Writing - review and editing

   Competing interests

   Reviewing editor, eLife

   "This ORCID iD identifies the author of this article:" 0000-0001-8086-0331

4. Sharon L Thompson-Schill

   Department of Psychology, University of Pennsylvania, Philadelphia, United States

   Contribution

   Conceptualization, Resources, Supervision, Funding acquisition, Methodology, Writing - review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-9750-4306

• 1,316

  Page views

• 171

  Downloads

• 10

  Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.