Motion along the mental number line reveals shared representations for numerosity and space

  1. Caspar M Schwiedrzik  Is a corresponding author
  2. Benjamin Bernstein
  3. Lucia Melloni  Is a corresponding author
  1. The Rockefeller University, United States
  2. Northwestern University, United States
  3. Max Planck Institute for Brain Research, Germany
  4. Columbia University College of Physicians and Surgeons, United States
  5. New York University Langone Medical Center, United States
7 figures

Figures

Figure 1 with 1 supplement
Cross-adaptation paradigm to test the effect of motion direction on numerosity perception.

Each block started with a 30 s adaptor, consisting of two clouds of 400 dots each (50% black, 50% white). Dots in the bottom cloud always moved randomly (0% coherence). Dots in the top cloud moved leftward or rightward at 100% coherence (as indicated here by arrows). To maximize directional adaptation at the top location, each condition was tested in a separate session. Each trial started with a 5 s top-up adaptor. Dots in the bottom cloud would again always move in random directions. Dots in the top cloud would move rightward or leftward at 100% coherence. After a 400 ms inter-stimulus interval (ISI), we presented a ‘test’ stimulus consisting of randomly moving dots at the top location. After another 400 ms ISI, we presented the ‘probe’ stimulus, also consisting of randomly moving dots, at the bottom location. In Experiment 1, the ‘test’ contained 23:107 and the probe contained 30 randomly moving dots. Subjects indicated which cloud, the ‘test’ or the ‘probe’, was more numerous by pressing a button on a keyboard. This paradigm allowed use to investigate motion direction-numerosity cross-adaptation and also replicated the well-known numerical distance effect (Figure 1—figure supplement 1). We used the same basic task structure also in Experiments 2 and 6, but changed the number of dots in the adaptors, test, and probe stimuli.

https://doi.org/10.7554/eLife.10806.003
Figure 1—figure supplement 1
Numerical distance effect.

Our paradigm replicated the well-known numerical distance effect (Moyer and Landauer, 1967).When comparing numerosities, subjects typically take longer when numerosities are more similar, i.e., closer to each other on the mental number line, than when they are further apart. A parametric rmANOVA on mean reaction times from Experiment 1 revealed a significant effect of test size (F(9,90)=2.834, p=0.006). Interestingly, the numerical distance effect depended on the perceived, not the actual (green), numerosity of the test cloud, with the longest reaction times around the point of subjective equality (PSE, red), and shorter reaction times the further the test cloud was from that point in either direction. A Gaussian fit to the data which explained about 72% of the variance illustrates this effect (black line). Reaction times are represented as mean ± SEM, corrected for between-subject variability (Cousineau, 2005; Morey, 2008).

https://doi.org/10.7554/eLife.10806.004
Figure 2 with 3 supplements
Effect of motion direction on numerosity perception.

(a) There was a significant repulsive aftereffect of motion direction (left, right) on perceived numerosity (mean difference in PSE 7.12 dots, t(10)=2.555, p=0.029, dHedges=0.953; see inset) when adapting with large numerosities (400 dots) in Experiment 1. The scatter plot pits the PSE after rightward motion against the PSE after leftward motion per subject. It shows that adaptation to rightward motion led subjects to perceive the test cloud as less numerous than adaptation to leftward motion, as evidenced by the large fraction of dots above the equality line, which is indicative of a repulsive aftereffect. This was not due to a classical motion aftereffect (Figure 2—figure supplement 1 and Figure 2—figure supplement 2). (b) Psychometric functions of subject aba17. After adaptation to 400 rightward moving dots in Experiment 1, this subject perceived 70 dots in the test dot cloud to be equivalent to 30 dots in the probe dot cloud, thus underestimating the number of dots in the test. After adaptation to leftwards motion, aba17’s PSE was 20 dots lower than after adaptation to rightwards motion, although the number of dots in the two clouds were identical in both conditions. Note that all psychometric functions are shifted away from 30 dots, the probe size, because of a static numerosity adaptation effect that causes subjects to underestimate consecutively presented dot clouds (Burr and Ross, 2008). See Figure 2—figure supplement 3 for additional example subjects. (c) There was also a repulsive motion direction-numerosity cross-adaptation effect after adaptation to small numerosities (30 dots) in Experiment 2. When adapting with numerosities smaller than the probe (166 dots), subjects overestimate the number of dots in the test cloud (Burr and Ross, 2008). Leftward motion-numerosity cross-adaptation should exaggerate this overestimation effect relative to rightward motion. Indeed, the scatter plot shows that PSEs after leftward motion were consistently smaller than for rightward motion (mean difference 15.85 dots; t(9)=4.523, p=0.001, dHedges=1.017, two-sided). This indicates that, as when adapting with large numerosities, leftward motion shifts numerosity perception down the number line, while rightwards motion shifts numerosity perception up the number line. (d) Motion direction affected numerosity perception also directly, in the absence of adaptation. In Experiment 3, subjects compared the numerosities of coherently and incoherently moving dot clouds without prior adaptation. The scatter plot pits the PSE for rightwards motion against the PSE for leftwards motion per subject, and shows that, indeed, all but one subject perceived clouds of rightward moving dots as more numerous than randomly moving clouds (0% coherence) and clouds of leftward moving dots as less numerous than randomly moving clouds (0% coherence). Accordingly, the PSEs for rightwards vs. leftwards conditions were significantly different (mean difference right vs. left −2.08, t(9)=−2.47, p=0.035, dHedges=−0.707, two-sided). Thus, motion direction affects numerosity estimates also without a preceding adaptation phase. Note that since there is no adaptation, there is also no repulsive aftereffect and hence, the direct effect is of the opposite sign than the repulsive aftereffects in a and c. Data in insets are represented as mean ± SEM. All data shown here are publicly available at Figshare (Schwiedrzik et al., 2015).

https://doi.org/10.7554/eLife.10806.005
Figure 2—figure supplement 1
Adaptation paradigm to test classical motion aftereffects.

The design and all stimulus parameters of this control experiment were the same as for Experiment 1, but subjects now had to report the direction of motion instead of the numerosity of the dot clouds. Each block started with a 30 s adaptor, consisting of two clouds of 400 dots each (50% black, 50% white). Dots in the top cloud moved to the left or to the right at 100% coherence (as indicated here by arrows; panels i-iii), or randomly (panel iv). Dots in the bottom adaptor cloud always moved randomly (0% coherence). To maximize directional adaptation at the top location, each condition was tested in a separate block. Each trial started with a 5 s top-up adaptor. After a 400 ms inter-stimulus interval (ISI), we presented a test stimulus at the top location, with 60 dots moving in random directions. After another 400 ms ISI, we presented the probe stimulus, consisting of 30 dots moving at 100% coherence either in the same (panel i) or in the opposite direction (panel ii) of the adaptor. Subjects then had to press the ‘up’ button on the keyboard when they perceived that the motion direction was the same in the test and the probe cloud, or the ‘down’ button when they perceived the test and the probe cloud to move in different directions. This allowed us to test whether subjects perceived attractive or repulsive motion aftereffects: For example, if subjects consistently perceived an attractive motion aftereffect, we expected them to respond ‘same’ after leftwards adaption when the lower cloud was moving to the left (panel i). In contrast, if they perceived a repulsive motion aftereffect, we expected them to respond ‘same’ after leftwards adaption when the lower cloud was moving to the right (panel ii). To assure that subjects were following task instructions, we also presented coherent motion to the left or right in the upper cloud on 1/3 of the trials within a block, followed by motion in the same or opposite direction in the lower cloud (panel iii). Here, we expected subjects to respond ‘same’ whenever the two clouds were actually moving in the same direction. To control for response biases, we also adapted subjects to incoherent dot clouds above and below the fixation cross in a separate block to control for an overall propensity to respond ‘same’ or ‘different’ (panel iv).

https://doi.org/10.7554/eLife.10806.006
Figure 2—figure supplement 2
Control for classical motion aftereffects.

Gray bars represent the mean (± SEM), red circles represent the median percent ‘same’ responses as a function of condition. Subjects did not perceive attractive or repulsive classical MAEs when incoherently moving dots were presented at the test location (perceived direction), while performance on trials which contained coherent motion in both clouds was at ceiling (actual direction). Neither the mean nor the median percent ‘same’ responses were significantly different from 0 in the aftereffect conditions (mean: one sample t-tests, two-sided, all p>0.05; median: percentile bootstrap, all p>0.37), and the percentage of ‘same’ responses did not differ between attractive and repulsive aftereffect conditions (mean: t(10)=−1.7417, p=0.11, two-sided; median: percentile bootstrap, p>0.38). In line with these results, none of the subjects reported perceiving directional motion in the randomly moving test clouds during debriefing after the experiments. This indicates that the effect of motion direction on numerosity perception arises in the absence of a classical MAE (like other high-level MAEs, Whitney and Cavanagh, 2003), and thus likely in a different brain area. Hence, the control experiment implies that motion direction acts directly on numerosity perception, in line with an account where the same neurons code for motion direction and numerosity. All data shown here are publicly available at Figshare (Schwiedrzik et al., 2015).

https://doi.org/10.7554/eLife.10806.007
Figure 2—figure supplement 3
Additional example subjects from Experiment 1.

(a) Psychometric functions of subject cpb26. After adaptation to 400 rightward moving dots in Experiment 1, this subject perceived 56 dots in the test dot cloud to be equivalent to 30 dots in the probe dot cloud, thus underestimating the number of dots in the test. After adaptation to leftwards motion, cpb26’s PSE was 21 dots lower than after adaptation to rightwards motion, although number of dots in the two clouds were identical in both conditions. (b) Psychometric functions of subject tlq25. After adaptation to 400 rightward moving dots in Experiment 1, this subject perceived 61 dots in the test dot cloud to be equivalent to 30 dots in the probe dot cloud, thus underestimating the number of dots in the test. After adaptation to leftwards motion, tlq25’s PSE was 7 dots lower than after adaptation to rightwards motion, exhibiting overestimation relative to rightward motion adaptation. All data shown here are publicly available at Figshare (Schwiedrzik et al., 2015).

https://doi.org/10.7554/eLife.10806.008
Hypothetical effect of adaptation on numerosity-encoding neurons in LIP and VIP.

It has been shown that area LIP (dotted lines) contains neurons that encode numerosity with monotonically increasing firing rates, i.e., the larger the number of dots in the display, the more vigorous the neural response (Roitman et al., 2007). In contrast, one stage downstream from area LIP, neurons are narrowly tuned to numerosity in area VIP (solid lines), i.e., each neuron will respond vigorously only to a narrow range of numerosities, and less so when either more or less dots are displayed (Nieder et al., 2006). Adaptation has different effects on these different classes of tuning curves (black lines): In area VIP, adaptation will only affect neurons that encode the number of dots in the adapter (400 in Experiment 1) while neurons coding for numerosities far removed from the adapter, such as the ones in the test (23:107 in Experiment 1) and probe dot clouds (30 in Experiment 1), will not be affected. In contrast, in area LIP, adaptation will shift the entire tuning curve, hence even adaptation to 400 dots affects responses to numerosities in the range of 23:107. Thus, the adaptation effect we observe is likely to arise in area LIP, not VIP.

https://doi.org/10.7554/eLife.10806.009
Paradigm to test cross-adaptation in different reference frames.

The design of Experiment 6 closely followed that of Experiment 1, but introduced two saccades (arrows) between the top-up adaptor and the test/probe stimuli. Initial stimulus and fixation locations are depicted in blue, final stimulus and fixation locations in red. (a) In the ‘same location’ condition, subjects made a 14 dva leftward or rightward saccade after the top-up adaptor had been presented, and then back to the original fixation dot. The location of the test/probe stimuli and adaptor was the same in space and on the retina, depicted here as blue (initial) and red (final) half-cycles. This condition served to replicate the cross-adaptation effect found in Experiment 1. (b) In the ‘spatiotopic’ condition, subjects made two consecutive leftward or rightward saccades (first 14 dva, second 28 dva), and test/probe stimuli were presented at the same screen position as the top-up adaptor but on a different retinal position. Here, a cross-adaptation effect would occur in an area that has spatiotopic but locally restricted receptive fields, such as LIP. (c) In the ‘retinotopic’ condition, subjects also made two consecutive leftward or rightward saccades (first 14 dva, second 28 dva) but test/probe stimuli were presented above and below the new fixation dot. Hence, stimuli occurred at the same retinal location (relative to the subjects’ gaze), but at different location in spatiotopic coordinates. Here, the cross-adaptation effect would occur in early, retinotopically organized visual areas. (d) In the ‘full field’ condition subjects first made rightward or leftward saccaded (14 dva), and then back to the original fixation dot (14 dva, as in the ‘same location’ condition). Test/probe stimuli were however presented at a new location opposite to the second saccade direction. Here, a cross-adaptation effect would only occur if the effect takes place in an area with very large receptive fields, such as VIP.

https://doi.org/10.7554/eLife.10806.010
The reference frame of motion direction-numerosity cross-adaptation.

A non-parametric rmANOVA showed that there was a significant interaction between reference frame and motion direction (F(3,15)=4.58, p=0.018, η2=0.478). Only the ‘same location’ condition which replicated Experiment 1 with intervening saccades and the ‘spatiotopic’ condition with restricted receptive field sizes as predicted for area LIP showed a significant (both p<0.05) cross-adaptation effect from motion-direction on numerosity (see text for statistics). Together with the tuning function of the effect, this indicates that the influence of motion direction on numerosity takes place at an intermediate, analog stage of number processing, and not in early visual areas with a retinotopic frame of reference or in area VIP with receptive fields covering the entire visual field. Data are represented as mean ± SEM. All data shown here are publicly available at Figshare (Schwiedrzik et al., 2015).

https://doi.org/10.7554/eLife.10806.011
Hypothetical model of motion direction-numerosity cross-adaptation.

We can explain the cross-adaptation effect using a two-pool opponent coding model similar to what is often invoked to explain the classical MAE and other aftereffects (Mather and Harris, 1998). Panel (a) exemplifies the effect of motion direction during adaptation with large numerosities (Experiment 1, Figure 2a), panel (b) exemplifies the effect of motion direction during adaptation with small numerosities (Experiment 2, Figure 2c). The two black lines reflect the activity level of two broadly tuned pools of neurons that respond maximally to small or large numbers, respectively. The tuning curves are overlapping such that any given stimulus produces activity in both pools. Adaptation to number causes each pool to reduce its firing rate proportionally to how well the stimulus activates the respective pool in the non-adapted state, e.g., static or randomly moving dot clouds with large numbers of dots (as in Experiment 1, e.g., in the lower cloud, or in Burr and Ross, 2008) lead to stronger adaptation in the pool for large numbers (blue). After adaptation, this pool’s response is reduced, shifting the point at which the two tuning curves intersect to the either to the right (a), i.e., towards larger numbers, or to the left (b), i.e., towards smaller numbers. This also shifts the point of subjective equality (PSE) in a comparison task (gray), leading to an underestimation (a) or overestimation (b) in how many dots are perceived. During adaptation with large numerosities, rightwards motion exaggerates underestimation (a) by activating the pool for large numbers more strongly, moving perception up the number line (green, inset), subsequently causing a stronger cross-adaptation effect (underestimation). In contrast, leftwards motion reduces the effect, moving perception down the number line (orange, inset), subsequently causing a cross-adaption effect in the opposite direction. During adaptation with small numerosities, leftwards motion exaggerates overestimation (b) by activating the pool for small numbers more strongly, moving perception down the number line (green, inset), subsequently causing a stronger overestimation effect. Rightwards motion reduces overestimation, moving perception up the number line (orange, inset). Monotonically increasing and decreasing tuning curves are consistent with numerosity tuning curves measured in monkey LIP (Roitman et al., 2007) and with the fact that we could exert an effect on the perception dot numerosities far removed from the adaptor, e.g., perception of 23–107 dots when adapting with 400 dots in Experiment 1, which would not be expected if each number was represented by narrow tuning curves as those found in monkey VIP (Nieder et al., 2006). This was also supported by Experiment 5 which showed that there was no motion direction-numerosity cross-adaption when the number of dots in the adapter and the probe were matched (mean difference right vs. left −3.69, p=0.92, Wilcoxon signed rank test, one-sided; Bayesian posterior probability <0.18%), as predicted for two-pool opponent coding models (Robbins et al., 2007; Webster, 2011).

https://doi.org/10.7554/eLife.10806.012
Author response image 1
Motion direction-numerosity adaptation when adapting with small numerosities.

Leftward motion cross-adaptation leads to a stronger overestimation effect than rightward motion adaptation. The scatter plot shows that PSEs for leftward motion were consistently smaller than for rightward motion (mean difference in PSE 15.85 dots; t(9)=4.523, p=0.001, dHedges=1.017, two-sided). This indicates that as when adapting with large numerosities, leftward motion shifts numerosity perception down the number line, while rightwards motion shifts numerosity perception up the number line. A general main effect of static numerosity and/or texture density leading to overestimation independent of motion direction is also evident in a majority of subjects (all data points falling left and/or below the dashed lines).

https://doi.org/10.7554/eLife.10806.013

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Caspar M Schwiedrzik
  2. Benjamin Bernstein
  3. Lucia Melloni
(2016)
Motion along the mental number line reveals shared representations for numerosity and space
eLife 5:e10806.
https://doi.org/10.7554/eLife.10806