The appearance of objects within a single category can vary greatly. For instance, despite the shared category of dog, the exemplars (i.e. different dogs) may have different snouts, tails, and/or torsos. Despite this variability, humans readily categorize never-before-seen dog breeds as members of the same basic-level category. How do we do this? Although there is widespread agreement that shape information is crucial for object categorization (Biederman, 1995; Mervis and Rosch, 1981), it remains unclear how humans come to form global representations of shape that are tolerant to variations among exemplars.

It has been suggested that global shape information becomes crucial for object categorization in early childhood because linguistic experience, particularly the learning of object labels (e.g. ‘dog’), draws children’s attention to global shape as a diagnostic cue—inducing a so-called ‘shape bias’ (Landau et al., 1998; Smith et al., 1996; Smith et al., 2002). Indeed, labels may bootstrap object recognition abilities more generally, such that even prelinguistic children are better at individuating and categorizing objects when verbal labels are provided (Ferry et al., 2010; Xu et al., 2005). The advantage of labeled object experience is particularly evident in supervised artificial neural networks (ANNs), which have begun to match the object recognition abilities and neural representations of human adults (Krizhevsky et al., 2017; Rajalingham et al., 2018; Schrimpf et al., 2018). These models learn the diagnostic properties of objects following training with millions of labeled naturalistic images. With appropriate experience, these models may even develop a shape bias that supports object categorization, at least when generalizing across variations in color or texture (Ritter et al., 2017; Tartaglini et al., 2022). Thus, global representations of shape may develop with labeled object experience that highlights the diagnostic properties of objects.

An alternative possibility, however, is that rather than labeled experience, humans develop global shape representations by relying on (non-linguistic) invariant perceptual properties inherent to the object (Biederman, 1987; Feldman, 1997; Rakison and Butterworth, 1998; Sloutsky, 2003). One such property is known as the shape skeleton—a quantitative model that describes an object’s global shape via a series of internal symmetry axes (Blum, 1967; Feldman and Singh, 2006). These axes define the topological arrangement of object parts, making models of skeletal structure tolerant to local variations in shape typical of basic-level exemplars (Ayzenberg et al., 2019a; Wilder et al., 2011). From this perspective, extensive experience with objects and linguistic labels may not be necessary to form global shape representations, and, instead, one might rely on the shape skeleton (Feldman, 1997). However, the contributions of labeled experience and the shape skeleton are difficult to examine because, by adulthood, humans have had massive amounts of labeled object experience, making the source of their shape representations ambiguous. Here we tested whether human infants (who have little linguistic or object experience) represent global object shape according to a shape skeleton.

Object representations in infancy have most often been tested using visual attention procedures. In these experiments, infants are typically habituated to stimuli that share a common visual dimension (e.g. shape), but vary according to other dimensions (e.g. color). Infants are then tested with objects that are either familiar (e.g. similar in shape to the habituated object) or novel (i.e. different in shape to the habituated object). If infants learn the relevant dimension during the habituation phase, then their looking times are longer for the novel object compared to the familiar one. Habituation paradigms provide an informative window into infants’ object representations because they reveal what properties infants learned during the habituation phase and subsequently generalized to the test phase. Using this approach, researchers have shown that newborns can already discriminate between simple 2D shapes (Slater et al., 1983) and display shape constancy, such that they recognize a shape from a novel orientation (Slater and Morison, 1985). By 6 months of age, infants’ shape representations are also robust to variations among category exemplars, such that they can categorize objects using only the stimulus’ shape silhouette (Quinn et al., 1993; Quinn et al., 2001a), as well as extend category membership to objects with varying local contours, but the same global shape (Quinn et al., 2002; Quinn et al., 2001b; Turati et al., 2003). However, the mechanisms underlying global shape representation remain unclear. Indeed, because infants in these studies were habituated to multiple (often familiar) objects, it is unclear whether shape representations in these studies were learned from the statistics of the habituation period (Oakes and Spalding, 1997; Younger, 1990) or, rather, are an invariant perceptual property infants extract from objects more generally.

In the current study, we used a habituation paradigm to examine how 6–12 months old infants represent the global shape of objects. In particular, we tested whether infants classified never-before-seen objects by comparing the similarity between their shape skeletons. Importantly, we habituated infants to only a single object so as to measure their pre-existing shape representations, rather than ones they may have learned over the course of habituation. We chose to test 6–12-month-olds because the visual experience at this age is dominated by just a few common objects (~10 objects; Clerkin et al., 2017) and they have relatively little linguistic understanding (~7 words; Bergelson and Swingley, 2012). Moreover, we elucidate the mechanisms that support object perception in infancy by comparing infants to a range of computational models. Of particular interest was a flux-based medial axis algorithm that computes a shape skeleton from images (Rezanejad and Siddiqi, 2013). If infants represent objects using the shape skeleton, then their classification judgments should best match that of a Skeletal model.

In addition, we included four ANNs, known for their success on object recognition tasks but which do not represent the shape skeleton. These ANNs included two ResNet models, one trained on ImageNet (ResNet-IN), a convolutional ANN frequently used in object recognition tasks, and a variation of ResNet trained on Stylized-ImageNet (ResNet-SIN), an image set that leads models to develop a shape bias, at least in relation to color and textures cues (Geirhos et al., 2018). Two other ANNs were CorNet-S, a top-performing model of object recognition behavior and neural processing in primates, as measured by the brain-score benchmark (Schrimpf et al., 2018), and ResNext-SAY, a model trained with an unsupervised learning algorithm on first-person videos from infants (Orhan et al., 2020). All of these ANNs were included because they exhibit varying degrees of biological plausibility in terms of neural organization or visual experience (Russakovsky et al., 2015; Schrimpf et al., 2018). If infant performance is best matched by ANNs, then this would suggest that global shape representations might be learned as a diagnostic cue following extensive object experience. Importantly, because none of these models represent the shape skeleton, they make for an excellent contrast to the Skeletal model. Finally, we also included a model of pixel similarity, and FlowNet, a model of optic flow (Ilg et al., 2017) in order to assess the extent to which shape representations may be supported by lower-level visual properties like image similarity (Kiat et al., 2022; Xie et al., 2021) or motion trajectory (Kellman, 1984; Kellman and Short, 1987; Wood and Wood, 2018). Altogether, these comparisons provided a novel approach to understanding object perception in human infants.

Because the strength of any object classification task depends on the degree of dissimilarity between training (i.e. habituation) and test objects, infants were tested with objects that mimicked the variability of basic-level category exemplars (Figure 1). Within-category objects comprised objects with the same skeletons, but visually distinct component parts. Variation in the component parts was generated by manipulating the objects’ surface forms, thereby changing both the image-level features and the non-accidental properties (NAPs; Biederman, 1987), without altering the skeleton. Between-category objects comprised different skeletons and surface forms. If infants are capable of classifying objects vis-à-vis a shape skeleton, then they should look longer at objects with different skeletons compared to those with the same skeleton, even though both objects differ from the habituated object in surface form. Importantly, we tested whether within-category and between-category objects were equally discriminable by infants to ensure that any differences in looking time were not related to infants’ ability to differentiate surface forms. Moreover, if infants rely on objects’ shape skeletons to determine similarity, then their classification performance should be best matched by the Skeletal model, rather than ANNs, or other models of vision. Thus, the current study provides critical insight regarding the nature of shape representations at an age when experience with labeled objects is minimal, and, crucially, provides a novel benchmark by which to evaluate the biological plausibility of computational models of vision.

Figure 1

Download asset Open asset

Infants’ looking times were analyzed using two-sided paired sample t-tests (α=0.05) and standard measures of effect size (Cohen’s d). Furthermore, to ensure that sample size decisions did not unduly influence the results, we also conducted non-parametric (Binomial tests) and Bayesian analyses. A Bayes factor (BF₁₀) was computed for each analysis using a standard Cauchy prior (d=0.707). A BF₁₀ greater than 1 is evidence that two distributions are different from one another, whereas a BF₁₀ less than 1 is evidence that two distributions are similar to one another (Rouder et al., 2009).

How do infants represent shape?

A comparison of the two types of test trials in Experiment 1 (Figure 2A) revealed that, across the test phase, infants looked longer at the object with the different skeleton compared to the one with the matching skeleton (t[33]=3.04, p=0.005, d=0.52, 95% CI [0.16, 0.88], BF₁₀=8.42; Figure 2B), with the majority of infants showing this effect (25/34 infants, p=0.009). In addition, a comparison between the end of habituation (mean of last 4 trials) and looking times across the test phase revealed that dishabituation only occurred for the object with the different skeleton (t[33]=3.36, p=0.002, d=0.58, 95% CI [0.21, 0.94], BF₁₀=17.47; 25/34 infants, p=0.009), not the object with the matching skeleton (t[33]=1.00, p=0.325, d=0.17, 95% CI [–0.16, 0.51], BF₁₀=0.29; Figure 2B). Thus, infants treated objects with matching skeletons as more similar to one another than objects with different skeletons.

Figure 2

Download asset Open asset

However, one might ask whether infants were simply unable to differentiate between surface forms, leading to greater looking at the object with a different skeleton. To test this possibility, we compared infants’ looking times on the first test trial following habituation to the last trial during habituation. Because the first test trial immediately follows habituation, this comparison allows for a direct measure of perceptual discriminability between habituation and test objects when the memory demands are minimal and in the absence of carry-over effects between test objects. This analysis revealed that infants were indeed capable of discriminating objects on the basis of surface form alone. That is, infants dishabituated to the object with the same skeleton but different surface form, t(15) = 3.76, p=0.002, d=0.94, 95% CI [0.34, 1.52], BF₁₀=22.23, with the majority of infants showing this effect (14/16, p=0.004). Moreover, and crucially, infants' looking times on the first test trial did not differ for either test object (8.31 s vs. 9.18 s; t[32]=0.50, p=0.624, d=.17, 95% CI [–0.51, 0.84], BF₁₀=0.36). These findings demonstrate that not only could infants differentiate between surface forms, but also, that the two types of test objects were matched for discriminability relative to the habituation object. These findings argue against a pure discrimination account and, instead, support the interpretation that infants classified objects on the basis of skeletal similarity.

But did infants actually rely on the object’s shape skeleton to judge similarity or some other visual representation? To explore this possibility, we compared infants’ classification behavior to a flux-based Skeletal model, a range of ANNs (ResNet-IN, ResNet-SIN, and CorNet-S, ResNext-Say), and models of image similarity (Pixel) and motion (FlowNet). If infants relied on the shape skeleton to classify objects, then their performance would be best matched by the Skeletal model, rather than the others.

Models were tested with the same stimuli presented to infants using a procedure comparable to the habituation paradigm. More specifically, because habituation/dishabituation can be conceived as a measure of alignment between the stimulus and the infant’s internal representation (Mareschal et al., 2000; Westermann and Mareschal, 2004), we tested models by feeding their outputs into an autoencoder and measuring the error signal across habituation and test phases (see Methods). Like habituation paradigms, the error signal of an autoencoder reflects the degree of alignment between the internal representation of the model and the input stimulus (for review, see Yermolayeva and Rakison, 2014). Unlike conventional classifiers, which often require multiple labeled contrasting examples (e.g. Support Vector Machines), autoencoders allow for measuring a model’s performance following exposure to just one exemplar, as with infants in the current study. Moreover, like infant learning during habituation, the learned representation of an autoencoder reflects the entire habituation video, rather than the representation of individual frames. Most importantly, unlike other techniques, autoencoders can be tested using the same habituation and test criteria as infants (see Methods). For comparison, performance for both infants and models was converted into a classification score (see Methods) and significance was assessed using bootstrapped confidence intervals (5000 iterations).

These analyses revealed that all models performed above chance (0.50; see Figure 2C). However, and importantly, infant performance was best matched to the Skeletal model. Both infants and the Skeletal model performed significantly better than all other models, except FlowNet. That infants outperformed the ANNs suggests that extensive object experience may not be necessary to develop robust shape representations. Likewise, that infants outperformed the Pixel model suggests that infant performance is not explained by the low-level visual similarity between objects. However, the success of FlowNet does suggest that motion information may contribute to representations of shape. Nevertheless, FlowNet’s performance did not differ from that of any ANN or the Pixel model, leaving its exact role in object classification unclear. Altogether, these results suggest that infants’ performance is most closely aligned with the Skeletal model and, thus, that infants classified objects, at least in part, by relying on the similarity between objects’ shape skeletons.

Model classification using local shape properties

Given the competitive performance of ANNs on other object recognition tasks, and their match to adult performance in other studies (Schrimpf et al., 2018), one might wonder why ANNs performed so poorly on our task. One possibility is that ANNs rely on local shape properties (e.g. surface form), which were irrelevant to our task (Baker et al., 2018; Baker et al., 2020). To test this possibility, we examined whether ANNs were capable of classifying objects using local shape properties, namely, the surface forms of the objects. For comparison, we also tested the other models’ performance on this task (Skeleton, Pixel, and FlowNet).

Using a comparable procedure to the previous experiments, all models were tested with objects in which the surface forms either matched or differed from the habituated object; both test objects differed in their shape skeletons (Figure 3A). For objects from Experiment 1 (Figure 1), the analyses revealed that ResNet-IN, CorNet-S, and ResNext-SAY performed above chance (Ms = 0.62–0.66, 95% CIs [0.53–0.59, 0.70–0.76]; Figure 3B). By contrast, none of the other models (Skeleton, ResNet-SIN, Pixel, FlowNet) performed differently from chance (Ms = 0.52–0.61, 95% CIs [0.40–0.48, 0.60–0.72]; Figure 3B) when classifying objects by surface form (i.e. across shape skeletons). For objects from Experiment 2 (Figure 1), all models performed above chance (Ms = 0.67–0.74, 95% CIs [0.51–0.67, 0.72–0.80]), except Skeletal and Pixel models (Ms = 0.54–0.59, 95% CIs [0.40–0.41, 0.68–0.73]; Figure 3C). Altogether, these findings demonstrate that most ANNs are capable of classifying objects using local shape properties. These findings are consistent with other research showing sensitivity to local shape properties in ANNs (Baker et al., 2018).

Figure 3

Download asset Open asset

All stimuli and data are available at: https://osf.io/4vswu/.

1. 1. Ayzenberg V
   2. Lourenco SF

   (2022) Open Science Framework

   Perception of an object's global shape is best described by a model of skeletal structure in human infants.

   https://doi.org/10.17605/OSF.IO/4VSWU

1. 1. Amir O
   2. Biederman I
   3. Hayworth KJ

   (2012) Sensitivity to nonaccidental properties across various shape dimensions

   Vision Research 62:35–43.

   https://doi.org/10.1016/j.visres.2012.03.020

   • PubMed
   • Google Scholar
2. 1. Arcaro MJ
   2. Livingstone MS

   (2017) A hierarchical, retinotopic proto-organization of the primate visual system at birth

   eLife 6:e26196.

   https://doi.org/10.7554/eLife.26196

   • PubMed
   • Google Scholar
3. 1. Ardila D
   2. Mihalas S
   3. von der Heydt R
   4. Niebur E

   (2012) 46th Annual Conference on Information Sciences and Systems (CISS

   Princeton 1:1–4.

   https://doi.org/10.1109/CISS.2012.6310946

   • Google Scholar
4. 1. Ayzenberg V
   2. Chen Y
   3. Yousif SR
   4. Lourenco SF

   (2019a) Skeletal representations of shape in human vision: Evidence for a pruned medial axis model

   Journal of Vision 19:1–21.

   https://doi.org/10.1167/19.6.6

   • PubMed
   • Google Scholar
5. 1. Ayzenberg V
   2. Lourenco SF

   (2019b) Skeletal descriptions of shape provide unique perceptual information for object recognition

   Scientific Reports 9:1–13.

   https://doi.org/10.1038/s41598-019-45268-y

   • PubMed
   • Google Scholar
6. 1. Ayzenberg V
   2. Kamps FS
   3. Dilks DD
   4. Lourenco SF

   (2022) Skeletal representations of shape in the human visual cortex

   Neuropsychologia 164:108092.

   https://doi.org/10.1016/j.neuropsychologia.2021.108092

   • PubMed
   • Google Scholar
7. 1. Baker N
   2. Lu H
   3. Erlikhman G
   4. Kellman PJ

   (2018) Deep convolutional networks do not classify based on global object shape

   PLOS Computational Biology 14:e1006613.

   https://doi.org/10.1371/journal.pcbi.1006613

   • PubMed
   • Google Scholar
8. 1. Baker N
   2. Lu H
   3. Erlikhman G
   4. Kellman PJ

   (2020) Local features and global shape information in object classification by deep convolutional neural networks

   Vision Research 172:46–61.

   https://doi.org/10.1016/j.visres.2020.04.003

   • PubMed
   • Google Scholar
9. 1. Bergelson E
   2. Swingley D

   (2012) At 6-9 months, human infants know the meanings of many common nouns

   PNAS 109:3253–3258.

   https://doi.org/10.1073/pnas.1113380109

   • PubMed
   • Google Scholar
10. 1. Biederman I

    (1987) Recognition-by-components: a theory of human image understanding

    Psychological Review 94:115–147.

    https://doi.org/10.1037/0033-295X.94.2.115

    • PubMed
    • Google Scholar
11. 1. Biederman I
    2. Gerhardstein PC

    (1993) Recognizing depth-rotated objects: evidence and conditions for three-dimensional viewpoint invariance

    Journal of Experimental Psychology. Human Perception and Performance 19:1162–1182.

    https://doi.org/10.1037//0096-1523.19.6.1162

    • PubMed
    • Google Scholar
12. Book

    1. Biederman I.

    (1995)

    Visual object recognition, 2

    Cambridge, MA, USA: MIT press.

    • Google Scholar
13. 1. Biederman I
    2. Bar M

    (1999) One-shot viewpoint invariance in matching novel objects

    Vision Research 39:2885–2899.

    https://doi.org/10.1016/s0042-6989(98)00309-5

    • PubMed
    • Google Scholar
14. Book

    1. Blum H

    (1967)

    A transformation for extracting descriptors of shape

    In: Wathen-Dunn W, editors. Models for the Perception of Speech and Visual Form. Cambridge, MA: MIT Press. pp. 362–380.

    • Google Scholar
15. 1. Butler DJ
    2. Wulff J
    3. Stanley GB
    4. Black MJ

    (2012) A Naturalistic Open Source Movie for Optical Flow Evaluation

    Berlin, Heidelberg 7577:611–625.

    https://doi.org/10.1007/978-3-642-33783-3_44

    • Google Scholar
16. 1. Cassia VM
    2. Simion F
    3. Milani I
    4. Umiltà C

    (2002) Dominance of global visual properties at birth

    Journal of Experimental Psychology. General 131:398–411.

    https://doi.org/10.1037/0096-3445.131.3.398

    • PubMed
    • Google Scholar
17. 1. Clerkin EM
    2. Hart E
    3. Rehg JM
    4. Yu C
    5. Smith LB

    (2017) Real-world visual statistics and infants’ first-learned object names

    Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences 372:20160055.

    https://doi.org/10.1098/rstb.2016.0055

    • PubMed
    • Google Scholar
18. 1. Coutanche MN
    2. Thompson-Schill SL

    (2014) Fast mapping rapidly integrates information into existing memory networks

    Journal of Experimental Psychology. General 143:2296–2303.

    https://doi.org/10.1037/xge0000020

    • PubMed
    • Google Scholar
19. 1. Craft E
    2. Schütze H
    3. Niebur E
    4. von der Heydt R

    (2007) A neural model of figure-ground organization

    Journal of Neurophysiology 97:4310–4326.

    https://doi.org/10.1152/jn.00203.2007

    • PubMed
    • Google Scholar
20. 1. Davitt LI
    2. Cristino F
    3. Wong ACN
    4. Leek EC

    (2014) Shape information mediating basic- and subordinate-level object recognition revealed by analyses of eye movements

    Journal of Experimental Psychology. Human Perception and Performance 40:451–456.

    https://doi.org/10.1037/a0034983

    • PubMed
    • Google Scholar
21. Conference

    1. Dimitrov P
    2. Damon JN
    3. Siddiqi K

    (2003)

    Flux invariants for shape

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003.

    • Google Scholar
22. Conference

    1. Dosovitskiy A
    2. Fischer P
    3. Ilg E
    4. Hausser P
    5. Hazirbas C
    6. Golkov V
    7. Smagt PVD
    8. Cremers D
    9. Brox T

    (2015) FlowNet: Learning Optical Flow with Convolutional Networks

    IEEE International Conference on Computer Vision.

    https://doi.org/10.1109/ICCV.2015.316

    • Google Scholar
23. 1. Ellis CT
    2. Yates TS
    3. Skalaban LJ
    4. Bejjanki VR
    5. Arcaro MJ
    6. Turk-Browne NB

    (2020) Retinotopic Organization of Visual Cortex in Human Infants

    Neuroscience 1:437.

    https://doi.org/10.1101/2020.12.01.407437

    • Google Scholar
24. 1. Feldman J

    (1997) The Structure of Perceptual Categories

    Journal of Mathematical Psychology 41:145–170.

    https://doi.org/10.1006/jmps.1997.1154

    • PubMed
    • Google Scholar
25. 1. Feldman J
    2. Singh M

    (2006) Bayesian estimation of the shape skeleton

    PNAS 103:18014–18019.

    https://doi.org/10.1073/pnas.0608811103

    • PubMed
    • Google Scholar
26. Book

    1. Feldman J
    2. Singh M
    3. Briscoe E
    4. Froyen V
    5. Kim S
    6. Wilder J

    (2013)

    An Integrated Bayesian Approach to Shape Representation and Perceptual Organization

    In: Dickinson SJ, Pizlo Z, editors. Shape Perception in Human and Computer Vision: An Interdisciplinary Perspective. London: Springer. pp. 55–70.

    • Google Scholar
27. 1. Ferry AL
    2. Hespos SJ
    3. Waxman SR

    (2010) Categorization in 3- and 4-month-old infants: an advantage of words over tones

    Child Development 81:472–479.

    https://doi.org/10.1111/j.1467-8624.2009.01408.x

    • PubMed
    • Google Scholar
28. 1. Fleming RW
    2. Schmidt F

    (2019) Getting “fumpered”: Classifying objects by what has been done to them

    Journal of Vision 15:15.

    https://doi.org/10.1167/19.4.15

    • Google Scholar
29. Conference

    1. Gant JM
    2. Banburski A
    3. Deza A

    (2021)

    Evaluating the Adversarial Robustness of a Foveated Texture Transform Module in a CNN

    NeurIPS 2021 Workshop SVRHM. pp. 1–12.

    • Google Scholar
30. Preprint

    1. Geirhos R
    2. Rubisch P
    3. Michaelis C
    4. Bethge M
    5. Wichmann FA
    6. Brendel W

    (2018) ImageNet-Trained CNNs Are Biased towards Texture; Increasing Shape Bias Improves Accuracy and Robustness

    arXiv.

    https://doi.org/10.48550/arXiv.1811.12231

    • Google Scholar
31. Conference

    1. He K
    2. Zhang X
    3. Ren S
    4. Sun J

    (2016) Deep Residual Learning for Image Recognition

    2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR.

    https://doi.org/10.1109/CVPR.2016.90

    • Google Scholar
32. Conference

    1. Huang X
    2. Belongie S

    (2017) Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization

    2017 IEEE International Conference on Computer Vision (ICCV.

    https://doi.org/10.1109/ICCV.2017.167

    • Google Scholar
33. 1. Hummel JE
    2. Stankiewicz BJ

    (1996) Categorical relations in shape perception

    Spatial Vision 10:201–236.

    https://doi.org/10.1163/156856896x00141

    • PubMed
    • Google Scholar
34. Conference

    1. Ilg E
    2. Mayer N
    3. Saikia T
    4. Keuper M
    5. Dosovitskiy A
    6. Brox T

    (2017) FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks

    2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR.

    https://doi.org/10.1109/CVPR.2017.179

    • Google Scholar
35. 1. Jagadeesh AV
    2. Gardner JL

    (2022) Texture-like Representation of Objects in Human Visual Cortex

    Neuroscience 1:849.

    https://doi.org/10.1101/2022.01.04.474849

    • Google Scholar
36. 1. James KH
    2. Jones SS
    3. Smith LB
    4. Swain SN

    (2014) Young Children’s Self-Generated Object Views and Object Recognition

    Journal of Cognition and Development 15:393–401.

    https://doi.org/10.1080/15248372.2012.749481

    • PubMed
    • Google Scholar
37. 1. Jang H
    2. Tong F

    (2021) Convolutional neural networks trained with a developmental sequence of blurry to clear images reveal core differences between face and object processing

    Journal of Vision 21:e6.

    https://doi.org/10.1167/jov.21.12.6

    • PubMed
    • Google Scholar
38. 1. Kellman PJ

    (1984) Perception of three-dimensional form by human infants

    Perception & Psychophysics 36:353–358.

    https://doi.org/10.3758/BF03202789

    • Google Scholar
39. 1. Kellman PJ
    2. Short KR

    (1987) Development of three-dimensional form perception

    Journal of Experimental Psychology. Human Perception and Performance 13:545–557.

    https://doi.org/10.1037//0096-1523.13.4.545

    • PubMed
    • Google Scholar
40. 1. Kellman PJ
    2. Shipley TF

    (1991) A theory of visual interpolation in object perception

    Cognitive Psychology 23:141–221.

    https://doi.org/10.1016/0010-0285(91)90009-d

    • PubMed
    • Google Scholar
41. Book

    1. Kellman PJ
    2. Arterberry ME

    (2006)

    Infant Visual Perception

    In: Kuhn D, Siegler RS, Damon W, Lerner RM, editors. Handbook of Child Psychology: Cognition, Perception, and Language. John Wiley & Sons Inc. pp. 109–160.

    • Google Scholar
42. 1. Kiat JE
    2. Luck SJ
    3. Beckner AG
    4. Hayes TR
    5. Pomaranski KI
    6. Henderson JM
    7. Oakes LM

    (2022) Linking patterns of infant eye movements to a neural network model of the ventral stream using representational similarity analysis

    Developmental Science 25:e13155.

    https://doi.org/10.1111/desc.13155

    • PubMed
    • Google Scholar
43. 1. Krizhevsky A
    2. Sutskever I
    3. Hinton GE

    (2017) ImageNet classification with deep convolutional neural networks

    Communications of the ACM 60:84–90.

    https://doi.org/10.1145/3065386

    • Google Scholar
44. Conference

    1. Kubilius J
    2. Schrimpf M
    3. Kar K
    4. Rajalingham R
    5. Hong H
    6. Majaj N
    7. Schmidt K

    (2019)

    Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNs

    Advances in Neural Information Processing Systems 32 (NeurIPS 2019.

    • Google Scholar
45. Conference

    1. Lake B
    2. Salakhutdinov R
    3. Gross J
    4. Tenenbaum J

    (2011)

    One shot learning of simple visual concepts

    Proceedings of the annual meeting of the cognitive science society.

    • Google Scholar
46. 1. Lake BM
    2. Salakhutdinov R
    3. Tenenbaum JB

    (2015) Human-level concept learning through probabilistic program induction

    Science (New York, N.Y.) 350:1332–1338.

    https://doi.org/10.1126/science.aab3050

    • PubMed
    • Google Scholar
47. 1. Lake BM
    2. Piantadosi ST

    (2019) People Infer Recursive Visual Concepts from Just a Few Examples

    Computational Brain & Behavior 3:54–65.

    https://doi.org/10.1007/s42113-019-00053-y

    • Google Scholar
48. 1. Landau B
    2. Smith LB
    3. Jones SS

    (1988) The importance of shape in early lexical learning

    Cognitive Development 3:299–321.

    https://doi.org/10.1016/0885-2014(88)90014-7

    • Google Scholar
49. 1. Landau B
    2. Smith L
    3. Jones S

    (1998) Object perception and object naming in early development

    Trends in Cognitive Sciences 2:19–24.

    https://doi.org/10.1016/s1364-6613(97)01111-x

    • PubMed
    • Google Scholar
50. Preprint

    1. Lee D
    2. Gujarathi P
    3. Wood JN

    (2021) Controlled-Rearing Studies of Newborn Chicks and Deep Neural Networks

    arXiv.

    https://doi.org/10.48550/arXiv.2112.06106

    • Google Scholar
51. 1. Lescroart MD
    2. Biederman I

    (2013) Cortical representation of medial axis structure

    Cerebral Cortex (New York, N.Y 23:629–637.

    https://doi.org/10.1093/cercor/bhs046

    • PubMed
    • Google Scholar
52. 1. Long B
    2. Yu CP
    3. Konkle T

    (2018) Mid-level visual features underlie the high-level categorical organization of the ventral stream

    PNAS 115:E9015–E9024.

    https://doi.org/10.1073/pnas.1719616115

    • PubMed
    • Google Scholar
53. 1. Mareschal D
    2. French RM
    3. Quinn PC

    (2000) A connectionist account of asymmetric category learning in early infancy

    Developmental Psychology 36:635–645.

    https://doi.org/10.1037/0012-1649.36.5.635

    • PubMed
    • Google Scholar
54. 1. Margalit E
    2. Biederman I
    3. Herald SB
    4. Yue X
    5. von der Malsburg C

    (2016) An applet for the Gabor similarity scaling of the differences between complex stimuli

    Attention, Perception & Psychophysics 78:2298–2306.

    https://doi.org/10.3758/s13414-016-1191-7

    • PubMed
    • Google Scholar
55. Conference

    1. Mayer N
    2. Ilg E
    3. Hausser P
    4. Fischer P
    5. Cremers D
    6. Dosovitskiy A
    7. Brox T

    (2016) A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation

    2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR.

    https://doi.org/10.1109/CVPR.2016.438

    • Google Scholar
56. 1. Mervis CB
    2. Rosch E

    (1981) Categorization of Natural Objects

    Annual Review of Psychology 32:89–115.

    https://doi.org/10.1146/annurev.ps.32.020181.000513

    • Google Scholar
57. 1. Morgenstern Y
    2. Schmidt F
    3. Fleming RW

    (2019) One-shot categorization of novel object classes in humans

    Vision Research 165:98–108.

    https://doi.org/10.1016/j.visres.2019.09.005

    • PubMed
    • Google Scholar
58. 1. Oakes LM
    2. Spalding TL

    (1997) The role of exemplar distribution in infants’ differentiation of categories

    Infant Behavior and Development 20:457–475.

    https://doi.org/10.1016/S0163-6383(97)90036-9

    • Google Scholar
59. 1. Ons B
    2. Wagemans J

    (2012) Generalization of Visual Shapes by Flexible and Simple Rules

    Seeing and Perceiving 25:237–261.

    https://doi.org/10.1163/187847511X571519

    • PubMed
    • Google Scholar
60. Preprint

    1. Orhan EA
    2. Gupta PV
    3. Lake BM

    (2020) Self-Supervised Learning through the Eyes of a Child

    arXiv.

    https://doi.org/10.48550/arXiv.2007.16189

    • Google Scholar
61. 1. Ostrovsky Y
    2. Meyers E
    3. Ganesh S
    4. Mathur U
    5. Sinha P

    (2009) Visual parsing after recovery from blindness

    Psychological Science 20:1484–1491.

    https://doi.org/10.1111/j.1467-9280.2009.02471.x

    • PubMed
    • Google Scholar
62. 1. Quinn PC
    2. Eimas PD
    3. Rosenkrantz SL

    (1993) Evidence for representations of perceptually similar natural categories by 3-month-old and 4-month-old infants

    Perception 22:463–475.

    https://doi.org/10.1068/p220463

    • PubMed
    • Google Scholar
63. 1. Quinn PC
    2. Eimas PD
    3. Tarr MJ

    (2001a) Perceptual categorization of cat and dog silhouettes by 3- to 4-month-old infants

    Journal of Experimental Child Psychology 79:78–94.

    https://doi.org/10.1006/jecp.2000.2609

    • PubMed
    • Google Scholar
64. 1. Quinn PC
    2. Slater AM
    3. Brown E
    4. Hayes RA

    (2001b) Developmental change in form categorization in early infancy

    British Journal of Developmental Psychology 19:207–218.

    https://doi.org/10.1348/026151001166038

    • Google Scholar
65. 1. Quinn PC
    2. Bhatt RS
    3. Brush D
    4. Grimes A
    5. Sharpnack H

    (2002) Development of form similarity as a Gestalt grouping principle in infancy

    Psychological Science 13:320–328.

    https://doi.org/10.1111/1467-9280.00459

    • PubMed
    • Google Scholar
66. 1. Rajalingham R
    2. Issa EB
    3. Bashivan P
    4. Kar K
    5. Schmidt K
    6. DiCarlo JJ

    (2018) Large-Scale, High-Resolution Comparison of the Core Visual Object Recognition Behavior of Humans, Monkeys, and State-of-the-Art Deep Artificial Neural Networks

    The Journal of Neuroscience 38:7255–7269.

    https://doi.org/10.1523/JNEUROSCI.0388-18.2018

    • PubMed
    • Google Scholar
67. 1. Rakison DH
    2. Butterworth GE

    (1998) Infants’ attention to object structure in early categorization

    Developmental Psychology 34:1310–1325.

    https://doi.org/10.1037//0012-1649.34.6.1310

    • PubMed
    • Google Scholar
68. 1. Rezanejad M
    2. Siddiqi K

    (2013)

    Advances in Computer Vision and Pattern Recognition

    41–54, Flux graphs for 2D shape analysis, Advances in Computer Vision and Pattern Recognition, Springer, 10.1007/978-1-4471-5195-1_3.

    • Google Scholar
69. Conference

    1. Ritter S
    2. Barrett DG
    3. Santoro A
    4. Botvinick MM

    (2017)

    Cognitive psychology for deep neural networks: A shape bias case study

    Proceedings of the 34 th International Conference on Machine Learning.

    • Google Scholar
70. 1. Rosch E
    2. Mervis CB
    3. Gray WD
    4. Johnson DM
    5. Boyes-Braem P

    (1976) Basic objects in natural categories

    Cognitive Psychology 8:382–439.

    https://doi.org/10.1016/0010-0285(76)90013-X

    • Google Scholar
71. 1. Rouder JN
    2. Speckman PL
    3. Sun D
    4. Morey RD
    5. Iverson G

    (2009) Bayesian t tests for accepting and rejecting the null hypothesis

    Psychonomic Bulletin & Review 16:225–237.

    https://doi.org/10.3758/PBR.16.2.225

    • PubMed
    • Google Scholar
72. 1. Rouder JN
    2. Ratcliff R

    (2016) Comparing Exemplar- and Rule-Based Theories of Categorization

    Current Directions in Psychological Science 15:9–13.

    https://doi.org/10.1111/j.0963-7214.2006.00397.x

    • Google Scholar
73. 1. Rule JS
    2. Riesenhuber M

    (2020) Leveraging Prior Concept Learning Improves Generalization From Few Examples in Computational Models of Human Object Recognition

    Frontiers in Computational Neuroscience 14:586671.

    https://doi.org/10.3389/fncom.2020.586671

    • PubMed
    • Google Scholar
74. 1. Russakovsky O
    2. Deng J
    3. Su H
    4. Krause J
    5. Satheesh S
    6. Ma S
    7. Huang Z
    8. Karpathy A
    9. Khosla A
    10. Bernstein M
    11. Berg AC
    12. Fei-Fei L

    (2015) ImageNet Large Scale Visual Recognition Challenge

    International Journal of Computer Vision 115:211–252.

    https://doi.org/10.1007/s11263-015-0816-y

    • Google Scholar
75. 1. Schrimpf M
    2. Kubilius J
    3. Hong H
    4. Majaj NJ
    5. Rajalingham R
    6. Issa EB
    7. Kar K
    8. Bashivan P
    9. Prescott-Roy J
    10. Geiger F
    11. Schmidt K
    12. Yamins DLK
    13. DiCarlo JJ

    (2018) Brain-Score: Which Artificial Neural Network for Object Recognition Is Most Brain-Like?

    Neuroscience 1:e7.

    https://doi.org/10.1101/407007

    • Google Scholar
76. 1. Shepard RN

    (1987) Toward a universal law of generalization for psychological science

    Science (New York, N.Y.) 237:1317–1323.

    https://doi.org/10.1126/science.3629243

    • PubMed
    • Google Scholar
77. 1. Slater A
    2. Morison V
    3. Rose D

    (1983) Perception of shape by the new-born baby

    British Journal of Developmental Psychology 1:135–142.

    https://doi.org/10.1111/j.2044-835X.1983.tb00551.x

    • Google Scholar
78. 1. Slater A
    2. Morison V

    (1985) Shape constancy and slant perception at birth

    Perception 14:337–344.

    https://doi.org/10.1068/p140337

    • PubMed
    • Google Scholar
79. 1. Slone LK
    2. Smith LB
    3. Yu C

    (2019) Self-generated variability in object images predicts vocabulary growth

    Developmental Science 22:e12816.

    https://doi.org/10.1111/desc.12816

    • PubMed
    • Google Scholar
80. 1. Sloutsky VM

    (2003) The role of similarity in the development of categorization

    Trends in Cognitive Sciences 7:246–251.

    https://doi.org/10.1016/s1364-6613(03)00109-8

    • PubMed
    • Google Scholar
81. 1. Smith LB
    2. Jones SS
    3. Landau B

    (1996) Naming in young children: A dumb attentional mechanism?

    Cognition 60:143–171.

    https://doi.org/10.1016/0010-0277(96)00709-3

    • PubMed
    • Google Scholar
82. 1. Smith LB
    2. Jones SS
    3. Landau B
    4. Gershkoff-Stowe L
    5. Samuelson L

    (2002) Object name learning provides on-the-job training for attention

    Psychological Science 13:13–19.

    https://doi.org/10.1111/1467-9280.00403

    • PubMed
    • Google Scholar
83. 1. Spriet C
    2. Abassi E
    3. Hochmann JR
    4. Papeo L

    (2022) Visual object categorization in infancy

    PNAS 119:e2105866119.

    https://doi.org/10.1073/pnas.2105866119

    • PubMed
    • Google Scholar
84. 1. Spröte P
    2. Schmidt F
    3. Fleming RW

    (2016) Visual perception of shape altered by inferred causal history

    Scientific Reports 6:1–11.

    https://doi.org/10.1038/srep36245

    • PubMed
    • Google Scholar
85. Preprint

    1. Sullivan J
    2. Mei M
    3. Perfors A
    4. Wojcik EH
    5. Frank MC

    (2020) SAYCam: A Large, Longitudinal Audiovisual Dataset Recorded from the Infant’s Perspective

    PsyArXiv.

    https://doi.org/10.31234/osf.io/fy8zx

    • Google Scholar
86. 1. Tarr MJ
    2. Bülthoff HH

    (1998) Image-based object recognition in man, monkey and machine

    Cognition 67:1–20.

    https://doi.org/10.1016/s0010-0277(98)00026-2

    • PubMed
    • Google Scholar
87. Preprint

    1. Tartaglini AR
    2. Vong WK
    3. Lake BM

    (2022) A Developmentally-Inspired Examination of Shape versus Texture Bias in Machines

    arXiv.

    https://doi.org/10.48550/arXiv.2202.08340

    • Google Scholar
88. 1. Turati C
    2. Simion F
    3. Zanon L

    (2003) Newborns’ Perceptual Categorization for Closed and Open Geometric Forms

    Infancy 4:309–325.

    https://doi.org/10.1207/S15327078IN0403_01

    • Google Scholar
89. 1. Vogelsang L
    2. Gilad-Gutnick S
    3. Ehrenberg E
    4. Yonas A
    5. Diamond S
    6. Held R
    7. Sinha P

    (2018) Potential downside of high initial visual acuity

    PNAS 115:11333–11338.

    https://doi.org/10.1073/pnas.1800901115

    • PubMed
    • Google Scholar
90. 1. Westermann G
    2. Mareschal D

    (2004) From Parts to Wholes: Mechanisms of Development in Infant Visual Object Processing

    Infancy 5:131–151.

    https://doi.org/10.1207/s15327078in0502_2

    • PubMed
    • Google Scholar
91. 1. Wiesel TN
    2. Hubel DH

    (1974) Ordered arrangement of orientation columns in monkeys lacking visual experience

    The Journal of Comparative Neurology 158:307–318.

    https://doi.org/10.1002/cne.901580306

    • PubMed
    • Google Scholar
92. 1. Wilder J
    2. Feldman J
    3. Singh M

    (2011) Superordinate shape classification using natural shape statistics

    Cognition 119:325–340.

    https://doi.org/10.1016/j.cognition.2011.01.009

    • PubMed
    • Google Scholar
93. 1. Wilder J
    2. Rezanejad M
    3. Dickinson S
    4. Siddiqi K
    5. Jepson A
    6. Walther DB

    (2019) Local contour symmetry facilitates scene categorization

    Cognition 182:307–317.

    https://doi.org/10.1016/j.cognition.2018.09.014

    • PubMed
    • Google Scholar
94. 1. Wood JN
    2. Wood SMW

    (2018) The Development of Invariant Object Recognition Requires Visual Experience With Temporally Smooth Objects

    Cognitive Science 42:1391–1406.

    https://doi.org/10.1111/cogs.12595

    • PubMed
    • Google Scholar
95. Preprint

    1. Xie S
    2. Hoehl S
    3. Moeskops M
    4. Kayhan E
    5. Kliesch C
    6. Turtleton B
    7. Köster M
    8. Cichy RM

    (2021) Visual Category Representations in the Infant Brain

    bioRxiv.

    https://doi.org/10.1101/2021.11.03.466293

    • Google Scholar
96. 1. Xu F
    2. Cote M
    3. Baker A

    (2005) Labeling guides object individuation in 12-month-old infants

    Psychological Science 16:372–377.

    https://doi.org/10.1111/j.0956-7976.2005.01543.x

    • PubMed
    • Google Scholar
97. 1. Xu F
    2. Kushnir T

    (2013) Infants Are Rational Constructivist Learners

    Current Directions in Psychological Science 22:28–32.

    https://doi.org/10.1177/0963721412469396

    • Google Scholar
98. 1. Yermolayeva Y
    2. Rakison DH

    (2014) Connectionist modeling of developmental changes in infancy: approaches, challenges, and contributions

    Psychological Bulletin 140:224–255.

    https://doi.org/10.1037/a0032150

    • PubMed
    • Google Scholar
99. 1. Younger B

    (1990)

    Infants’ detection of correlations among feature categories

    Child Development 61:614–620.

    • PubMed
    • Google Scholar

Decision letter after peer review:

[Editors’ note: the authors submitted for reconsideration following the decision after peer review. What follows is the decision letter after the first round of review.]

Thank you for submitting the paper "The shape skeleton supports one-shot categorization in human infants: Behavioral and computational evidence" for consideration by eLife. Your article has been reviewed by 3 peer reviewers, one of whom is a member of our Board of Reviewing Editors, and the evaluation has been overseen by a Senior Editor. The reviewers have opted to remain anonymous.

We are sorry to say that, after consultation with the reviewers, we have decided that this work will not be considered further for publication by eLife.

Specifically, the reviewers thought that results may be alternatively explained by motion similarity and/or low-level visual similarity between habituation and test stimuli, as explained in more detail below.

Reviewer #1:

This is a very lucidly written article on a fascinating and important topic: how humans are able to learn novel visual categories based on few or even just one single example. This ability can be contrasted with conventional machine learning models which typically require massive data sets with thousands of training examples to learn the ability to categorize novel examples accurately. How humans generalize so well from such sparse training data remains poorly understood and working out how we achieve this is important not only for the psychological sciences, but also has implications for artificial intelligence and machine learning too.

The authors start with a strong premise, which is also likely to be true: a crucial aspect of human one-shot visual learning likely involves the perceptual processes that segment observed shapes into parts and represent them as a hierarchy of limbs, in a representation known in the field as 'shape skeletons'. There is a long history of evidence suggesting the human visual system analyses shape in this way, and such representations naturally lend themselves to abstractions and generalizations that are robust against significant variations in appearance, such as the surface structure and even pose of objects. While this is an excellent starting point for investigating one-shot learning, the idea that such representations might play a role in human visual categorization more generally is rather well accepted in the field, and thus perhaps not so original on its own. The potential novelty and importance of the contribution therefore rests on demonstrating that such representations are central to one-shot learning in particular.

Unfortunately, due to their choice of stimuli, I believe that the experiments the authors perform do not yet provide compelling evidence that the infants' looking times are driven by skeletal representations, or indeed whether one-shot learning is necessarily playing a role in their habituation.

The problem is that the different Surface Forms of each Skeleton are more similar to one another than those of the other Skeletons, in terms of the raw pixel similarity. I took screenshots of the stimuli from the MS and compared them in MATLAB. For 7 out of 10 possible comparisons, the corresponding Surface Forms from the same 'category' are closer in raw pixel terms than to any of the rivals. This leads me to believe that the pattern of results is based on the raw physical (and thus perceptual) similarity between the habituation and test stimuli, rather than based on one-shot generalization per se, or on skeletal representations of the shapes in particular. In my opinion, the fact that sophisticated artificial neural network models don't predict this is not in itself strong evidence against my suggested explanation of the findings.

In future studies, this issue could be addressed by using stimuli for which this confound is not a problem. This could be achieved, for example, by using additional transformations of the objects, such as rotations, that preserve part structure but radically alter the projected image of the objects. This way skeletal structure could be decoupled from straightforward image similarity, and a stronger case for generalization beyond the 'one-shot' training exemplar would be provided.

Reviewer #2:

This is an important paper. The issue of "one-shot" learning---how people can learn categories and concepts from a single example or set of examples---lies at the heart of the current controversies over "deep learning" models. These models are ubiquitous these days and are often promoted as descriptive models of human learning, but they inherently require many trials to learn. But human learners can often learn from single trials, presenting a fundamental challenge to the entire deep learning paradigm. This has been pointed out in broad terms before, but to really advance this debate, a study needs to establish that humans can indeed learn from single examples, simultaneously demonstrate that appropriate deep learning models cannot, and explain something about exactly what why---all of which this paper accomplishes with impressive rigor.

In that context, the paper specifically studies shape category learning by human infants, a particularly interesting case, and establishes a specific pattern: that infants generalize shape categories based on the "shape skeleton," a structural description of the part structure of the shape. That is, infants exposed to a single shape with a particular shape skeleton tend to be "unsurprised" by other shapes with the same skeleton, but more surprised by shapes with different skeletons, indicating that the single example was enough to establish in their minds an apparent shape category.

This generalization is in a sense unsurprising, because the shape skeleton is supposed to define characteristic "invariants" within shape categories, but it is nevertheless novel. Although the tendency for infants to generalize lexical categories based on shape is well established, the precise nature of one-shot shape generalizations has not previously been studied. For researchers in shape this is a very important result, and for anybody interested in how humans learn categories I think is is both fundamental and thought-provoking.

I found the methodology and statistical analysis in the paper comprehensive and rigorous, and the writing clear, so and have only relatively minor comments on the manuscript, which follow. I don't see page numbers, so give quotations to indicate the relevant location.

– "a phenomenon known as 'one-shot categorization'" – The issue of one-shot category learning, and the computational question of what makes it possible for human learners to instantly generalize from some examples but not others, has been studied somewhat more extensively than this very brief introduction lets on. I would point to for example Feldman (1997, J. Math Psych) which explicitly takes it up and argues for a mechanism related to the current paper (namely, that one-shot categorization is possible when the one example implies a highly specific structural model).

– "However, one might ask whether these findings are truly indicative of categorization, or simply better discrimination of objects with the different skeletons." I'm not sure these are really different explanations. Learners are better at discriminating objects that appear to be from different categories (categorical perception, etc.).

– "infants' looking times on the first test trial did not differ for within- and between-category test objects" – This claim is followed by a non-significant NHST test, which does not allow an affirmative conclusion of no difference here, and a Bayes Factor, which does. The NHST test really doesn't add anything. I personally think these tests could be omitted throughout the paper in favor of the more informative BFs – but particularly when null results are discussed.

– members from different -> members of different.

– "we tested models by feeding their outputs into an autoencoder and measuring the error signal across habituation and test phases (see Methods)." I didn't quite get this. To evaluate similarity within the network models, can't one use a Euclidean norm or a cosine? Please clarify.

– "but did not differ from one another" – Meaning what? Low BF? If so, please give BF.

– "it has remained unknown whether one-shot categorization is possible within the first year of human life." Has this really never been established, even for simple categories? Anecdotally, infants seem to do one-shot learning all the time.

– "Moreover, V2 and V3 are evolutionarily preserved in primate and non-primate animals" I think the entire discussion of neural analogs here is misleading. I am not a bird expert but I didn't think birds have homologous visual cortical areas to primates. But that doesn't matter, because functional organization is analogous when computational problems are analogous. In other words, birds don't generalize the same way we do because they have V2 areas like us, but because they are solving problems like us.

– "set was comprised of two…" -> "set comprised two…" Use of "comprised of" to mean "composed of" is colloquial. The US comprises states, not the other way around.

Reviewer #3:

This study relates habituation in infants to categorization in neural networks, showing that infants learn (as evidenced by looking times) shape skeletons across surface form, while neural networks that lack an explicit skeletal representation do not show this generalization. A key aspect of the study is that infants are only exposed to one exemplar, suggesting that infants learn shape skeletons using "one-shot categorization". The study is well-conducted, using relatively large sample sizes, comprehensive statistical testing, and careful modelling. The manuscript is well-written and easy to follow. However, results may be alternatively explained by motion similarity and/or low-level visual similarity between habituation and test stimuli.

– The shapes are shown as videos during both habituation and test phases. While I understand that this was preferred for drawing the infants' attention, it complicates the interpretation of the results. First, it becomes hard to disentangle the learning of the shape skeleton from the learning of the motion trajectory. Second, the comparison with neural networks becomes more difficult as these neural networks are not sensitive to motion.

– Because surface form differed for both test objects, the same-skeleton object will be visually more similar to the habituated object than the different-skeleton object. Therefore, it cannot be ruled out that results reflect habituation to lower-level stimulus properties rather than shape skeleton. This is also suggested by recent fMRI results using these stimuli (Ayzenberg et al., Neuropsychologia 2022), showing that the skeletal model correlates with activity patterns throughout the visual system, including V1 (though the cross-surface form results are only shown for V3 and LO, as far as I could tell).

I would recommend the authors to include a discussion of the possible contributions of motion and non-skeletal visual properties to the habituation results.

[Editors’ note: further revisions were suggested prior to acceptance, as described below.]

Thank you for choosing to send your work entitled "The shape skeleton supports one-shot categorization in human infants: Behavioral and computational evidence" for consideration at eLife. Your letter of appeal has been considered by a Senior Editor and a Reviewing Editor, and we are prepared to consider a revised submission with no guarantees of acceptance.

In addition to the comments of the previous reviews, during the consultation of the appeal, the following questions came up:

1. What new insight do we gain from the infant study above and beyond what we already know from adults?

2. Why do you think that (dis)habitation is evidence of learning per se, rather than comparisons of perceptual similarity between items?

[Editors’ note: further revisions were suggested prior to acceptance, as described below.]

Thank you for resubmitting your work entitled "The shape skeleton supports one-shot categorization in human infants" for further consideration by eLife. Your revised article has been evaluated by Floris de Lange (Senior Editor) and a Reviewing Editor, in consultation with reviewers.

The manuscript has been improved but there are some remaining issues that need to be addressed, as outlined below:

The reviewers were not convinced that the experiments convincingly demonstrate “one-shot” categorization. This concern can be addressed by adjusting the headline claim throughout (title, abstract, intro, results and discussion would need some modifications). A suggested alternative title could be: "The shape skeleton supports shape similarity judgments in human infants".

The design involves making a judgment about which objects appear to be more similar. This requires comparing the distances between presented items (in some feature space describing the stimuli). Making such a comparison doesn't involve learning anything on the basis of the experience. It doesn't involve generalization and there is no sense in which it is a 'one-shot' task, except that a given trial presents only a few items. But this is true of practically ANY experiment involving comparing a small number of items.

Here is an analogy: suppose the infants were shown three different (i.e., easily discriminable) patches of gray shades: two patches are relatively light shades of gray and one is a significantly darker shade. A habituation experiment like the one the authors performed would reveal that the infants see the two light grays as more similar than the darker one. But in what sense is this 'one shot categorization'? The infants wouldn't have learned a new category. There is no meaningful generalization. The experiment simply reflects the fact that similar grays look more similar than more different ones.

The same is true in the authors' experiments. The experiments demonstrate that objects with more similar skeletons appear more similar to infants. This is not a trivial result, and is worthy of publication in its own right (although similar findings have already been shown in adults). However not under the title of 'one-shot categorization'. Instead, it should be pitched (correctly) as the shape skeleton contributing substantially to judgments of shape similarity.

To reiterate: I think the authors have performed an elegant study with some interesting findings. I think their interpretation can certainly be discussed in the MS. However, the headline claim about one-shot categorization should be toned down. Otherwise, the term would 'one-shot' would not mean anything anymore.

[Editors’ note: The authors appealed the original decision. What follows is the authors’ response to the first round of review.]

Reviewer #1:

This is a very lucidly written article on a fascinating and important topic: how humans are able to learn novel visual categories based on few or even just one single example. This ability can be contrasted with conventional machine learning models which typically require massive data sets with thousands of training examples to learn the ability to categorize novel examples accurately. How humans generalize so well from such sparse training data remains poorly understood and working out how we achieve this is important not only for the psychological sciences, but also has implications for artificial intelligence and machine learning too.

The authors start with a strong premise, which is also likely to be true: a crucial aspect of human one-shot visual learning likely involves the perceptual processes that segment observed shapes into parts and represent them as a hierarchy of limbs, in a representation known in the field as 'shape skeletons'. There is a long history of evidence suggesting the human visual system analyses shape in this way, and such representations naturally lend themselves to abstractions and generalizations that are robust against significant variations in appearance, such as the surface structure and even pose of objects. While this is an excellent starting point for investigating one-shot learning, the idea that such representations might play a role in human visual categorization more generally is rather well accepted in the field, and thus perhaps not so original on its own. The potential novelty and importance of the contribution therefore rests on demonstrating that such representations are central to one-shot learning in particular.

Unfortunately, due to their choice of stimuli, I believe that the experiments the authors perform do not yet provide compelling evidence that the infants' looking times are driven by skeletal representations, or indeed whether one-shot learning is necessarily playing a role in their habituation.

The problem is that the different Surface Forms of each Skeleton are more similar to one another than those of the other Skeletons, in terms of the raw pixel similarity. I took screenshots of the stimuli from the MS and compared them in MATLAB. For 7 out of 10 possible comparisons, the corresponding Surface Forms from the same 'category' are closer in raw pixel terms than to any of the rivals. This leads me to believe that the pattern of results is based on the raw physical (and thus perceptual) similarity between the habituation and test stimuli, rather than based on one-shot generalization per se, or on skeletal representations of the shapes in particular. In my opinion, the fact that sophisticated artificial neural network models don't predict this is not in itself strong evidence against my suggested explanation of the findings.

We thank the reviewer for raising this concern. It is indeed the case that the test objects with the familiar skeleton exhibited greater image-level similarity to the habituation object than the test objects with the novel skeleton. Importantly, however, greater numerical similarity does not guarantee greater perceptual similarity, as raw pixel differences could be indiscriminable to humans. Thus, as in adult psychophysics, we ensured that both test objects were equally discriminable from the habituation object by infants. As discussed on pages 6 and 8, infants discriminated both test objects from the habituation object. To address this concern further, we also compared infants to a model based on pixel similarity (see pages 6-9, 18-20). Although the Pixel model performed above chance in both experiments, we found that infants (and the Skeletal model) outperformed the pixel model in both cases (see pages 7-9). Thus, despite some evidence that pixel similarity may allow for categorization, it is unlikely that this alone accounts for infants’ performance. Nevertheless, we now discuss the potential role of image-level similarity (and motion trajectory) to object categorization in the General Discussion (see page 11-12).

In future studies, this issue could be addressed by using stimuli for which this confound is not a problem. This could be achieved, for example, by using additional transformations of the objects, such as rotations, that preserve part structure but radically alter the projected image of the objects. This way skeletal structure could be decoupled from straightforward image similarity, and a stronger case for generalization beyond the 'one-shot' training exemplar would be provided.

We have provided greater justification for the claim that infants’ performance was based on the shape skeleton, rather than image similarity alone. Nevertheless, we agree with the reviewer that future research would benefit from the suggested manipulations.

Reviewer #2:

This is an important paper. The issue of "one-shot" learning---how people can learn categories and concepts from a single example or set of examples---lies at the heart of the current controversies over "deep learning" models. These models are ubiquitous these days and are often promoted as descriptive models of human learning, but they inherently require many trials to learn. But human learners can often learn from single trials, presenting a fundamental challenge to the entire deep learning paradigm. This has been pointed out in broad terms before, but to really advance this debate, a study needs to establish that humans can indeed learn from single examples, simultaneously demonstrate that appropriate deep learning models cannot, and explain something about exactly what why---all of which this paper accomplishes with impressive rigor.

In that context, the paper specifically studies shape category learning by human infants, a particularly interesting case, and establishes a specific pattern: that infants generalize shape categories based on the "shape skeleton," a structural description of the part structure of the shape. That is, infants exposed to a single shape with a particular shape skeleton tend to be "unsurprised" by other shapes with the same skeleton, but more surprised by shapes with different skeletons, indicating that the single example was enough to establish in their minds an apparent shape category.

This generalization is in a sense unsurprising, because the shape skeleton is supposed to define characteristic "invariants" within shape categories, but it is nevertheless novel. Although the tendency for infants to generalize lexical categories based on shape is well established, the precise nature of one-shot shape generalizations has not previously been studied. For researchers in shape this is a very important result, and for anybody interested in how humans learn categories I think is is both fundamental and thought-provoking.

I found the methodology and statistical analysis in the paper comprehensive and rigorous, and the writing clear, so and have only relatively minor comments on the manuscript, which follow. I don't see page numbers, so give quotations to indicate the relevant location.

– "a phenomenon known as 'one-shot categorization'" – The issue of one-shot category learning, and the computational question of what makes it possible for human learners to instantly generalize from some examples but not others, has been studied somewhat more extensively than this very brief introduction lets on. I would point to for example Feldman (1997, J. Math Psych) which explicitly takes it up and argues for a mechanism related to the current paper (namely, that one-shot categorization is possible when the one example implies a highly specific structural model).

We thank the reviewer for this suggestion. In the general discussion, we now situate our results within the broader one-shot categorization literature and have included the suggested paper (see pages 12-13).

– "However, one might ask whether these findings are truly indicative of categorization, or simply better discrimination of objects with the different skeletons." I'm not sure these are really different explanations. Learners are better at discriminating objects that appear to be from different categories (categorical perception, etc.).

We agree with the reviewer that visual similarity plays an important role in categorization. However, an important prerequisite of categorization is that exemplars within a category are distinct from one another, yet grouped into the same category. Thus, and in response to concerns about image similarity, we provide evidence that the test objects were comparably discriminable from the habituation object (see pages 6 and 8).

– "infants' looking times on the first test trial did not differ for within- and between-category test objects" – This claim is followed by a non-significant NHST test, which does not allow an affirmative conclusion of no difference here, and a Bayes Factor, which does. The NHST test really doesn't add anything. I personally think these tests could be omitted throughout the paper in favor of the more informative BFs – but particularly when null results are discussed.

Though we appreciate the reviewer’s suggestion, we have opted to continue to include NHST to ensure accessibility to a larger community, who may be less familiar with Bayes factors. Importantly, though, all of the statistics converge on the same results.

– members from different -> members of different.

Thank you. We have made this correction (see pages 6 and 8).

– “we tested models by feeding their outputs into an autoencoder and measuring the error signal across habituation and test phases (see Methods).” I didn’t quite get this. To evaluate similarity within the network models, can’t one use a Euclidean norm or a cosine? Please clarify.

In the current study, we chose to use an autoencoder approach because it allowed us to better match evaluation of the models to infants (for review, see Yermolayeva & Rakison, 2014 – Psychological Bulletin). Specifically, unlike conventional measures of classification, autoencoders allow for measuring model categorization following exposure to just one exemplar, rather than abelled contrasting examples (e.g., supervised classifiers). Moreover, like infant learning during habituation, the learned representation of an autoencoder reflects the entire habituation video, rather than the similarity between individual frames, as might be measured by Euclidean or cosine measures. Most importantly, unlike other techniques, autoencoders can be tested using the same criteria as infants. We have provided further justification for the use of autoencoders on pages 6-7.

– “but did not differ from one another” – Meaning what? Low BF? If so, please give BF.

The lack of a difference is based on overlapping confidence intervals (see page 7).

– “it has remained unknown whether one-shot categorization is possible within the first year of human life.” Has this really never been established, even for simple categories? Anecdotally, infants seem to do one-shot learning all the time.

To the best of our knowledge, this is the first study to test one-shot categorization in human infants. A predominant perspective in developmental psychology is that one-shot categorization on the basis of shape is possible at around 4 years of age following more extensive linguistic and object experience (see page 3 and 12).

– “Moreover, V2 and V3 are evolutionarily preserved in primate and non-primate animals” I think the entire discussion of neural analogs here is misleading. I am not a bird expert, but I didn't think birds have homologous visual cortical areas to primates. But that doesn't matter, because functional organization is analogous when computational problems are analogous. In other words, birds don't generalize the same way we do because they have V2 areas like us, but because they are solving problems like us.

We have removed the aforementioned section from the General Discussion.

– "set was comprised of two…" -> "set comprised two…" Use of "comprised of" to mean "composed of" is colloquial. The US comprises states, not the other way around.

We have made this correction (see page 13).

Reviewer #3:

This study relates habituation in infants to categorization in neural networks, showing that infants learn (as evidenced by looking times) shape skeletons across surface form, while neural networks that lack an explicit skeletal representation do not show this generalization. A key aspect of the study is that infants are only exposed to one exemplar, suggesting that infants learn shape skeletons using "one-shot categorization". The study is well-conducted, using relatively large sample sizes, comprehensive statistical testing, and careful modelling. The manuscript is well-written and easy to follow. However, results may be alternatively explained by motion similarity and/or low-level visual similarity between habituation and test stimuli.

– The shapes are shown as videos during both habituation and test phases. While I understand that this was preferred for drawing the infants' attention, it complicates the interpretation of the results. First, it becomes hard to disentangle the learning of the shape skeleton from the learning of the motion trajectory. Second, the comparison with neural networks becomes more difficult as these neural networks are not sensitive to motion.

The reviewer raises a valid concern. However, we would point out that the skeletal model, like the neural networks, is not sensitive to motion, yet it performed comparably to infants, suggesting that motion information is not needed for models to succeed on one-shot categorization. We would also note that the neural networks were successful at categorizing objects by their surface forms from the same videos (see pages 9-10), suggesting that their deficit was related to the type of information used for categorization, not to the use of videos per se. Finally, we now include a model of motion flow (FlowNet), which we compare to infants. Although FlowNet performed above chance in both experiments, our data nevertheless suggest that the skeletal model was a better match to infants’ performance (see pages 6-9). Nevertheless, we now explicitly discuss the potential contribution of motion trajectory in forming object representations (see pages 11-12).

– Because surface form differed for both test objects, the same-skeleton object will be visually more similar to the habituated object than the different-skeleton object. Therefore, it cannot be ruled out that results reflect habituation to lower-level stimulus properties rather than shape skeleton. This is also suggested by recent fMRI results using these stimuli (Ayzenberg et al., Neuropsychologia 2022), showing that the skeletal model correlates with activity patterns throughout the visual system, including V1 (though the cross-surface form results are only shown for V3 and LO, as far as I could tell).

Please see our previous response (#3). Although it is true that the same-skeleton object was more similar than the different-skeleton object, in raw pixels, to the habituated object, infants’ looking times suggest that both test objects (same-skeleton and different-skeleton) were perceptually discriminable from the habituated object (see pages 6 and 8). Moreover, we also demonstrate that infants (and the Skeletal model) outperformed a model based on image-level similarity (Pixel model; see pages 6-9). Nevertheless, we now elaborate on the potential contribution of image-level similarity to categorization (see page 11).

Finally, we would note that although Ayzenberg et al. (2022) found that the Skeletal model was correlated with the response profile of V1, it did not explain unique variance in this region after controlling for low-level visual similarity. By contrast, the Skeletal model explained unique variance in areas V3 and LO, even when controlling for several other models of visual similarity, and even when surface forms varied. These findings suggest that there can be some covariation between the Skeletal model and other visual properties but, importantly, different models explain unique variance in human behavioral judgments and neural responses. These findings further highlight the importance of the current approach in which we compared infants to several models of vision, not only the Skeletal model.

I would recommend the authors to include a discussion of the possible contributions of motion and non-skeletal visual properties to the habituation results.

We thank the reviewer for their recommendation. We now include discussion of the possible contributions of motion and non-skeletal visual properties (see pages 11-12).

[Editors’ note: what follows is the authors’ response to the second round of review.]

In addition to the comments of the previous reviews, during the consultation of the appeal, the following questions came up:

1. What new insight do we gain from the infant study above and beyond what we already know from adults?

Although adults are capable of one-shot categorization, this says nothing of its developmental origins. In particular, because of adults’ extensive experiences, it is difficult to disambiguate the contributions of early-developing perceptual mechanisms, supervised object experience, and language. With infants, we side-step these issues. By testing children who are largely non-verbal and who have had minimal experience with different objects, we provide a unique test of the perceptual mechanisms that support one-shot categorization. Moreover, few studies have compared the categorization abilities of infants to different computational models. This approach allows us to better assess the mechanism(s) underlying one-shot categorization and provides a novel avenue by which to evaluate the biological plausibility of existing models. Indeed, a common perspective in the machine learning and neuroscience literatures is that ANNs may simply need more extensive training to match human performance. Our study provides empirical evidence against this perspective by showing that one-shot categorization can be achieved early in human development, when language and object experiences are limited. Thus, by exploring object categorization at this developmental timepoint, we elucidate the initial processes that may support object categorization throughout the lifespan and provide a strong constraint on the development of object recognition models. In the revised manuscript, we have further elaborated on our motivation to test infants (see page 3).

2. Why do you think that (dis)habitation is evidence of learning per se, rather than comparisons of perceptual similarity between items?

We would argue that category learning and comparisons of visual similarity are not mutually exclusive. In particular, there is strong evidence that both children and adults make category decisions on the basis of visual similarity and that basic-level categories are grouped on the basis of visual similarity (e.g., Sloutsky, 2003 – Trends in Cognitive Sciences). The key question of our work is: what kind of visual properties are infants using to determine similarity, and, consequently, group membership? Habituation paradigms provide an excellent method to address this question because they reveal what properties, from the habituation phase, infants generalize to the test phase. Because there was no point at which habituation and test stimuli were presented simultaneously, infants’ looking behaviors were based on what they retained (i.e., learned) from the habituation phase, not direct comparisons of visual information.

We would also note that extensive research with infants has shown that their looking times in the test phase reflect the learned regularities of the habituation phase, such that, like adults, their category boundaries change depending on the variability of the stimuli during habituation (e.g., Bomba & Siqueland, 1983 – Journal of Experimental Child Psychology). This work suggests that infants learn the relevant properties of a category from the habituation phase. We have now provided additional review of the habituation and infant categorization literatures (see page 4).

[Editors’ note: what follows is the authors’ response to the second round of review.]

The manuscript has been improved but there are some remaining issues that need to be addressed, as outlined below:

The reviewers were not convinced that the experiments convincingly demonstrate “one-shot” categorization. This concern can be addressed by adjusting the headline claim throughout (title, abstract, intro, results and discussion would need some modifications). A suggested alternative title could be: "The shape skeleton supports shape similarity judgments in human infants".

The design involves making a judgment about which objects appear to be more similar. This requires comparing the distances between presented items (in some feature space describing the stimuli). Making such a comparison doesn't involve learning anything on the basis of the experience. It doesn't involve generalization and there is no sense in which it is a 'one-shot' task, except that a given trial presents only a few items. But this is true of practically ANY experiment involving comparing a small number of items.

Here is an analogy: suppose the infants were shown three different (i.e., easily discriminable) patches of gray shades: two patches are relatively light shades of gray and one is a significantly darker shade. A habituation experiment like the one the authors performed would reveal that the infants see the two light grays as more similar than the darker one. But in what sense is this 'one shot categorization'? The infants wouldn't have learned a new category. There is no meaningful generalization. The experiment simply reflects the fact that similar grays look more similar than more different ones.

The same is true in the authors' experiments. The experiments demonstrate that objects with more similar skeletons appear more similar to infants. This is not a trivial result, and is worthy of publication in its own right (although similar findings have already been shown in adults). However not under the title of 'one-shot categorization'. Instead, it should be pitched (correctly) as the shape skeleton contributing substantially to judgments of shape similarity.

To reiterate: I think the authors have performed an elegant study with some interesting findings. I think their interpretation can certainly be discussed in the MS. However, the headline claim about one-shot categorization should be toned down. Otherwise, the term would 'one-shot' would not mean anything anymore.

Thank you for the additional opportunity to revise our manuscript. As suggested, we have changed the title of our manuscript to reflect the focus on shape similarity as opposed to one-shot categorization. The new title is: " Perception of an object’s global shape is best described by a model of skeletal structure in human infants." And we have revised the manuscript accordingly to reflect the new focus. The potential implications for one-shot categorization are now exclusively discussed in the general discussion.

Author details

1. Vladislav Ayzenberg

   Neuroscience Institute, Carnegie Mellon University, Pittsburgh, United States

   Contribution

   Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Validation, Visualization, Writing - original draft, Writing - review and editing

   For correspondence

   vayzenb@cmu.edu

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0003-2739-3935

2. Stella Lourenco

   Department of Psychology, Emory University, Atlanta, United States

   Contribution

   Conceptualization, Methodology, Project administration, Resources, Supervision, Writing - original draft, Writing - review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0003-3070-7122

• 748

  Page views

• 144

  Downloads

• 3

  Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.