An affordance boundary in the affordable world.

a, An illustration of a picnic scene, featuring objects of various sizes relative to human body. Example objects within the normal body size range are painted red, and those beyond green. We hypothesized qualitative differences between perceived affordances of these two kinds of objects. b, A demonstration of the object-action relation judgement task for human participants (top) and AI models (bottom). The question in task for human participants was presented in Chinese. c, The representational similarity matrix (RSM) for objects based on human rating of affordance similarity. Object sizes are denoted with red to green. Two primary clusters emerged in clustering analysis of the similarity pattern are outlined with black boxes. d, Left panel: The overall affordance similarity and that of each gender (left y-axis) as well as real-world size similarity (right y-axis) between neighboring size ranks. The error bars represent the standard error (SE). Right panel: The point clouds of pairwise correlations between objects from the same rank or neighboring ranks. Each colored dot represents the affordance similarity (y-axis) and the average real-world size (x-axis) of a specific object pair. The grey dots indicate the averaged size (x-axis) and pairwise similarity (y-axis) of object pairs in different rank compositions. Left to right: both from size rank 3, from size rank 3 and 4, both from size rank 4, from size rank 4 and 5, both from size rank 5, from size rank 5 and 6, and both from size rank 6. The horizontal error bars represent 95% confidence interval (CI) of the averaged object size in each pair, and the vertical error bars denote the CI of pairwise affordance similarity.

A disembodied origin of the affordance boundary.

a, The schematic diagram of the imagined size in the cat condition (top) and the elephant condition (bottom), with the mean estimated height reported by participants for each condition. b, The affordance similarity between neighboring size ranks for manipulated body sizes (Red line: cat-size body; Green line: elephant-size body). The dashed line marks the boundary of the human-size body. The red and green arrows indicate the corresponding boundary shift in each condition. c, The affordance similarity between neighboring size ranks for different large language models, and human data from Fig. 1d was re-drawn as a reference. The stars indicate significant contrasts between affordance similarities between neighboring data points. d, The trough value of each model at between size rank 4-5. The stars here indicate the significant trough value compared to zero. The error bars represent the estimated standard error (SE). *p<.05, **p<.01, ***p<.001.

Affordance representation in the visual streams.

a, An example block with the probe action “graspable”. The participants indicated whether each of the subsequently presented objects is graspable by pressing the corresponding button. The action probing question was presented in Chinese during the experiment. b, The ROIs included in this experiment. c, The activation of each condition in the pFs and SPL. The bars represent the contrast estimates of each condition versus baseline. The stars indicate the significant difference between congruent and incongruent condition. *p<.05, **p<.01, ***p<.001, otherwise non-significance. Error bars represent the standard error (SE).