1 Introduction

Two observers presented with the exact same visual stimulus may experience a qualitatively very different percept (e.g., “the dress” Gegenfurtner et al., 2015). While intuitively this may seem puzzling, theories on consciousness and vision explain such phenomena by noting that human perception, rather than a direct reflection of the external (shared) world, emerges in a constructive process of adjusting incoming sensory signals against our idiosyncratic expectations, knowledge and experiences (Dennett, 1991; Friston, 2005; Rao & Ballard, 1999; Seth, 2019; von Helmholtz, 1867). This constructed understanding becomes especially apparent in color perception; even though color space is clearly defined in its physical and perceptual dimensions (Hurlbert, 2007), it still fails to capture the “what-it-is-like” aspect of an individual’s color experience (Gegenfurtner et al., 2015; Hurlbert, 2007). Not surprisingly, being able to measure, understand, and eventually describe the emergence of these subjective qualia is therefore one of most hotly debated questions in the neurosciences, philosophy and beyond (Baars, 2005; Chalmers, 1995; Seth, 2019; Tononi, 2004). Here, we present an extraordinary natural experiment of subjective color phenomenology; a condition called grapheme-color synesthesia (Hubbard & Ramachandran, 2005; Ramachandran & Hubbard, 2001; Simner et al., 2005; Van Leeuwen et al., 2011). For grapheme-color synesthetes, certain linguistic inducers (e.g., grapheme ‘4’) automatically and consistently trigger additional and idiosyncratic conscious color percepts (e.g., a bright-blue color) alongside the veridical sensory input.

Over the past decades, research has established that synesthesia is a ‘real’ condition that can reliably be identified using behavior, neurophysiology, and neuroimaging (Brang et al., 2011; Dixon et al., 2004; Hubbard & Ramachandran, 2005; Laeng et al., 2011; Ramachandran & Hubbard, 2001; Rouw & Root, 2019; Rouw et al., 2011). The most remarkable aspect of synesthesia is the subjective perceptual phenomenology of the induced colors, setting these sensations apart from color memory, thought, or amodal association. Furthermore, much like ordinary perception, the synesthetic experience is described as ‘automatic’ in the sense that it comes effortlessly (Cytowic, 1989; Dixon et al., 2000; Ramachandran & Hubbard, 2001) (albeit not pre-attentive, see Rich & Karstoft, 2013; Ward et al., 2010); the concurrent synesthetic color is ‘just there’, even if incongruent with the task at hand (Cytowic, 1989, 1993; Dixon et al., 2000; Laeng et al., 2011). Because each synesthete has a stable set of grapheme–color pairings, the color phenomenology can be examined independently of the physical properties of the inducing stimuli. Therefore, synesthesia might provide an exceptional window into how the brain’s constructive processes can generate additional, conscious content across modalities that goes all the way down to the level of sensory phenomenology.

The measurement of such sensory phenomenology primarily relies on subjective reports and introspection, methods often criticized for potential unreliability and susceptibility to biases or expectancy effects (e.g., Nisbett & Wilson, 1977; Schwitzgebel, 2008; Spener, 2024). These concerns extend to synesthesia research, where objective measurements are called for to corroborate subjective reports (Amin et al., 2011; Piazza et al., 2006; Simner et al., 2006). Instead, current paradigms capturing synesthesia employ objective measure, but fail to capture its phenomenology (Dixon et al., 2000; Eagleman et al., 2007; Rouw & Root, 2019; Ward et al., 2010). Behavioral and neurophysiological findings suggesting synesthetic colors behave like printed colors in turn have been questioned regarding replicability and interpretability (Hupé & Dojat, 2015; Rich & Karstoft, 2013; Rothen & Meier, 2009; Sagiv et al., 2006). In short, the lack of agreedupon objective methodology is a critical roadblock obstructing scientific examination of the extraordinary synesthetic phenomenology. By extension, this locks the condition’s potential to inform our understanding of the constructive topdown cross-modal processes that can generate additional conscious percepts.

We propose that pupillometry is the tool to break this gridlock. Pupil size constricts in response to externally increased brightness and dilates when brightness decreases. Remarkably, akin to our color phenomenology not directly following from physical color input, the pupillary light response does not strictly follow the physical light entering the eye, but reflects the percept as interpreted by the viewer (Binda et al., 2013a, 2013b; Cai et al., 2025; Goldflam, 1922; Haab, 1886; Koevoet, Naber, et al., 2025; Naber et al., 2011, 2013; Strauch, 2024). Research has revealed such sensitivity to perceptual phenomenology in unimodal contexts (e.g., phenomenological vividness of an (imagined) visual image Goldflam, 1922; Kay et al., 2022; Laeng & Sulutvedt, 2014). Building on this evidence, we hypothesized that the cross modal color phenomenology in synesthesia can, if truly sensory in nature, could likewise be inferred from changes in pupil size. Hereby, the direction and magnitude of these changes should provide a scaled response reflecting the brightness of the experienced synesthetic colors; pupillometry may thus provide both qualitative and quantitative characterizations of the synesthetic color phenomenology (see Figure 1a and b for an illustration of the rationale and proposed mechanism). If synesthetic cross-activations indeed reach all the way down to low-level (sensory) processes, pupillometry can track their precise temporal onset, as well as provide scaled measurements of their phenomenological properties (i.e., the relative change in pupil size corresponding to the brightness of the synesthetic color).

Figure 1:

To investigate this, we tested 16 grapheme-color synesthetes and two control groups of 16 participants each. Participants viewed graphemes (digits) on a computer screen during eyetracking, and indicated, after each trial, which color most closely matched with the respective grapheme (see Figure 1c for paradigm).

2 Results

2.1 More consistent and strongly coupled colors in synesthetes

Selected colors are visualized per participant in Figure 2a. Synesthetes reported colors more consistently (t(30) = 9.910, p < 0.001, d = 3.504, 95% CI = [2.370, 4.614] (in line with previous work, see (see e.g. Baron-Cohen et al., 1987; Eagleman et al., 2007; Mattingley et al., 2001)) and more strongly coupled to graphemes (t(30) = 12.690, p < 0.001, d = 4.487, 95% CI = [3.150, 5.801]) than controls (see Figure 2b). There was considerable variation in reported color lightness for all participants (see Figure 2c,

Figure 2:

2.2 Pupil responses reveal the quality of synesthetic color perception

Having established similar reported-color lightness levels between active controls and synesthetes, we next investigated whether the pupil light response betrays synesthetic color. Perceived, (covertly) attended, or even imagined brightness modulates pupil size in the same direction as changes in physical luminance - i.e., in both directions, constriction and dilation (see Mathôt & Van der Stigchel, 2015; Strauch et al., 2022, for reviews). We therefore expected bright perceived synesthetic colors to be betrayed by relative pupil constriction and dark synesthetic colors to be betrayed by relative pupil dilation. We did not expect nonsynesthetes to show pupil size alterations in accordance with the brightness of their associated colors. Pupil responses to reported color lightness were analyzed separately for synesthetes and active controls. A visual inspection of pertimepoint demeaned pupil traces for participants having at least 25 trials in above and below median lightness bins respectively (Figure 3) demonstrates larger pupil sizes for dark graphemes and smaller pupil sizes for light graphemes in synesthetes (mid row, Block 1). In controls, this was very weak, if present at all (top row, Block 1). As expected, when splitting pupil size for colored discs similarly along lightness, synesthete pupil size demonstrated descriptively even larger and earlier changes than for synesthetic color. Dependent-samples t-tests for averaged pupil size in response to graphemes (stimulus interval)

Figure 3:

2.2.1 Pupil size betrays the lightness of synesthetic color

To optimally account for the data structure, we next ran a linear-mixed effects model (LME) predicting pupil size. The LME effectively considers all trials and was fitted for the average pupil size between 800 ms and 4000 ms (reported in brackets) as well as separately for every timepoint in the stimulus interval (visualized in Figure 4 and Supplementary Figure 2). Starting from a full model containing all interactions between the following factors, the final LME for synesthetes (for both discretized and per-timepoint analyses) was determined using AIC-based backward selection while retaining coupling strength (self-reported strength of color-grapheme coupling) and PA score (projector-associator score, Rouw and Scholte (2007)) as pre-dictors for synesthetes, predicting average pupil size between 800 ms and 4000 ms (Wilkinson notation: Pupil size ∼ grapheme + lightness + coupling strength + PA score + lightness * coupling strength + lightness * PA score + lightness * coupling strength * PA score + (1 | participant)). The effects of all predictors over time on pupil size are depicted in Figure 4a (Supplementary Figure 2 for controls). We found significant modulations of pupil size by the lightness of the grapheme’s synesthetic color - sustained and in the to-be-expected time window. Specifically, the pupil constricted more for brighter reported colors, and dilated more for darker reported colors, as predicted (Average pupil size 800-4000 ms, t = −3.904, p < 0.001). Critically, in an LME ran on synesthetes and controls and using only graphemes and the interaction of group and lightness as predictors, we found lightness to predict pupil size in synesthetes (t = −2.754, p = 0.006), but not controls (t = −1.134, p = 0.257). Together, this demonstrates that 1) the pupil reveals the hidden qualia of synesthetic color (along the brightness axis) and 2) that such perception recruits the very same networks as are active during the perception of ‘real’ differently luminant stimuli itself, down to such a level that even the sensory organ is affected.

Figure 4:

2.2.2 Lightness of stronger color-grapheme couplings affects pupil size stronger in synesthetes

The pupil response to synesthetic color lightness was amplified for stronger reported coupling strengths between graphemes and colors (interaction coupling strength * lightness: pertimepoint from ±1700 ms see Figure 4b); average pupil size 800-4000 ms: t = −3.288, p = 0.001). In other words, synesthetes have metacognitive insight into the 3) strength of their synesthetic color perception as revealed by the pupil response. Furthermore, stronger effects of lightness for more projecting synesthetes for a short duration (±1950-2000 ms, average pupil size 800-4000 ms: t = −1.623, p = 0.105, see Figure 4c), tentatively suggest that nature of color percepts might affect the pupil response to synesthetic color lightness. Yet, we want to caution before interpreting this (borderline) effect strongly, not least due to the absence of statistical significance over the averaged interval. Finally, effects of reported color lightness on pupil size were strongest with higher indicated grapheme-color couplings for individuals with higher PA scores, i.e., less associating, more projecting synesthetes (three-way interaction, ±1900 ms-3300 ms; average pupil size 800-4000 ms: t = −2.092, p = 0.037, see Figure 4d). Together, the pupil therefore revealed both quality and quantity of self-reported synesthetic colors.

For controls a separate model was run, now without the PA score as predictor (not assessed for controls). Neither lightness (t = −0.835, p = 0.404), coupling strength (t = 0.469, p = 0.639), nor their interaction gained significance (t = −1.056, p = 0.291; all for average pupil size between 800 ms and 4000 ms). Critically, we also ran a LME with the three-way interaction of coupling strength, group, and lightness (Wilkinson notation: pupil = grapheme + group + coupling strength : group : lightness + (1 | participant)) with group (control or synesthete). This analysis revealed a significant three-way interaction between lightness, coupling strength, and group, indicating that reported lightness and coupling strength thereof affected pupil size stronger in synesthetes than in controls (coupling strength : lightness interaction in synesthetes: t = –3.75, p < 0.001; coupling strength : lightness interaction in controls: t = –0.617, p = 0.537). Furthermore, pupil size was marginally larger in controls than in synesthetes over the interval (t = 2.020, p = 0.052; see later sections for more indepth analyses).

Lastly, we tested whether higher color consistency, the goldstandard assessment of synesthesia (Eagleman et al., 2007), predicted stronger pupil responses according to color lightness. It did neither in synesthetes (significant predictors: lightness and interaction of coupling strength and lightness; very limited variance of consistency) nor in controls (no other significant predictors; see Supplementary Material for full analyses). Together, this demonstrates that the consistency of colors was, in this study, not found related to the pupil responses to color brightness.

2.3 Pupil responses demonstrate the automaticity of synesthetic colors

Having established the qualia and quantity of synesthetic colors through pupillary responses, we next turned to the other core defining feature of synesthesia: its presumed effortless nature. Synesthetes report the colors to emerge automatically - rather than via active and effortful cognitive (e.g., memory) processes.

2.3.1 Synesthetic colors affect pupil size delayed relative to physically presented colors

In Block 1, synesthetes viewed individual digits, and as the pupil revealed, an evoked characteristic (synesthetic) color. In Block 2, we then physically presented, for each digit, its previously determined average synesthetic color as a colored disc on the screen. At the center of each disc sat a gray rectangle whose luminance and pixel area matched the original digit’s lightness and size from Block 1 (see Figure 1c). As expected, physically presented color discs let the pupil constrict strongly in response to bright and dilate in response to dark colors, respectively. Equally expected, this effect was numerically and statistically more pronounced than the response to synesthetic colors (akin to stronger effects for direct fixation compared with covert attention only, see Binda et al., 2013a). Interestingly, the time course of the pupil response to physical vs synesthetic color differed markedly. Specifically, pupil size first responded significantly to physical luminance after 330 ms (see Supplementary Figure 2 for pertimepoint LME), but only responded significantly at about 870 ms to synesthetic lightness (see also Figure 3c vs e and Figure 4 for pertimepoint LME). Assuming that internally generated lightness does elicit a pupil light response with similar latency as the physically triggered lightness reflex arc (330 ms here), this implies that synesthetic perception has to emerge within 540 ms, including the recognition of the digit itself. This fast emergence makes it highly unlikely that synesthetes imagined a color after processing a grapheme, as this must take up more time (Dijkstra et al., 2018).

2.3.2 Reporting colors to graphemes is more effortful for controls than for synesthetes

Lastly, we reasoned that synesthetic color perception should be relatively low in effort. We therefore exploited another, distinct feature of pupillary responses: In absence of any luminance changes, pupils dilate more from baseline the more effort is exerted (e.g. Alnæs et al., 2014; Bumke, 1911; Kahneman, 1973; Koevoet, Zantwijk, et al., 2025; Mathôt, 2018; Strauch et al., 2022). Figure 5a visualizes pupil size change to baseline. Figure 5b depicts average pupil size change to baseline between 800 ms and 4000 ms per participant. Mental effort presents in task-evoked dilations, yet other factors simultaneously affect the pupil, such as the luminance and contrast change that comes about with trial onset and trends in the signal. To clear the pupil time course of such factors that are uninformative regarding the effort in response to the trial, we therefore calculated the first derivative to assess the velocity of pupillary changes (Butterworth filter, 18 Hz, order 3, 2.5 Hz lowpass, following our previous works (Douze et al., submitted; Ten Brink et al., 2024)). Figure 5c depicts this derivative, Figure 5d the average derivative per participant between 700 ms and 2000 ms. We found a stronger increase in pupil dilation for active (reporting colors) compared with passive controls (not reporting colors; t = −4.254, p < 0.001) and for active controls compared with synesthetes (t = −2.828, p = 0.007), but no difference between synesthetes and passive controls (t = 1.424, p = 0.161, all on the interval 700 ms-2000 ms, LME formula in Wilkinson notation: pupil = group + (1 | participant)). Together, this demonstrates that having to report a color after seeing the grapheme is associated with pupil dilation in controls. This pupil dilation is absent in synesthetes and in controls not having to indicate any color. We interpret this effect to reflect differences in effort, not least because reported lightness was similar for synesthetes and active controls overall (see Figure 2c,d). This higher degree of effort in active controls as compared with passive controls and synesthetes may not be so surprising, given that the task to report a color when not seeing a color is not trivial. We argue that the obtained difference between active controls and synesthetes performing the exact same task is an additional consequence of the reported ‘automatic’ (effortless) nature with which synesthetes can indicate colors for graphemes. Together, the time course of effects and the reduced degree of effort needed for synesthetes during the task provides converging (and perhaps conclusive) evidence that synesthetic percepts are indeed fast and effortless, in line with synesthetes’ subjective reports (Mattingley et al., 2001).

Figure 5:

3 Discussion

We demonstrate that pupil size changes reveal the qualia of synesthetic (grapheme-color) percepts. Specifically, pupils constrict when viewing digits that evoke brighter synesthetic colors and dilate to digits that evoke darker synesthetic colors. In contrast, non-synesthetes presented with the exact same physical input do not show modulation of pupil size to the brightness of their associated colors. Synesthetes (but not controls) showed high color consistency, in line with the diagnostic ‘gold standard’ maintained in the field (Eagleman et al., 2007; Rothen et al., 2013). While such standardized objective diagnostics (Eagleman et al., 2007; Laeng et al., 2011; Paulsen & Laeng, 2006; Rothen et al., 2013, e.g.,) reliably separate synesthetes from non-synesthetes, our findings directly corroborate the most discerning and -arguably-most fascinating characteristic of grapheme-color synesthesia; the reported phenomenology of the synesthetic color.

This offers practical and theoretical progress in clarifying the boundary between synesthetes and non-synesthetes (Deroy & Spence, 2013; Martino & Marks, 2001; Watson et al., 2014). Conceptual cross-modal correspondences, which are consistent at the group level, can also be observed in the general population (Rich et al., 2005; Rouw et al., 2014; Simner et al., 2005). Moreover, synesthesia-like Stroop effects can be induced in non-synesthetes through training (Colizoli et al., 2012; Meier & Rothen, 2009). However, as illustrated by the traditional Stroop effect (Stroop, 1935), neither consistent color associations nor Stroop-like conflicts depend on sensory color phenomenology. Our technique, linking synesthetic brightness and pupil size for the first time, maps out phenomenological features of cross-modal (grapheme-to-color) associations and, as shown here, has the potential to provide a decisive separation between synesthetes and controls. In synesthetes -but not in controls-pupillometry tracked the qualia of associated colors along the brightness axis indicating that similar networks are engaged as during perception of ‘real’ (printed) differently luminant stimuli.

The effect of color lightness on pupil size scaled with the indicated strength of individual grapheme–color couplings, physiologically validating synesthetes’ meta-cognitive insight into their own associations (Hubbard et al., 2006). We further-more found that the effect of color lightness on pupil size scaled with the indicated strength of individual grapheme–color couplings, physiologically validating synesthetes ‘ meta-cognitive insight into their own associations (Hubbard et al., 2006). In future work, per-trial ratings could take this a step further by assessing moment-to-moment fluctuations and their neural correlates.

Along with the conscious perception of color, a main feature of synesthetic color experiences is that they happen automatically (e.g., Simner & Bain, 2013). Note that ‘automaticity’ in this study means the reported effortless nature of the additional sensations, as synesthetic sensations are unlikely evoked pre-attentively (see (Mattingley, 2009)). As increases in mental effort are tightly coupled with pupillary dilations (e.g. Bumke, 1911; Kahneman, 1973; Koevoet et al., 2024; Mathôt, 2018; Strauch et al., 2022), we could put this to a direct test. Indeed, we found faster pupil dilation in active controls than in passive controls at constant physical stimulation. This stronger effort-linked increase in pupil dilation in active controls (preparing to report a color after viewing the digit) as compared with passive controls (observing the same digits but without color task) most likely reflected the non-trivial nature of the color task for non-synesthetes. Furthermore, active controls exhibited greater pupil dilation and thus effort than synesthetes, even though both groups received the same color-reporting task. Pupil size change rates showed no significant difference between synesthetes (asked to pick a color after viewing digits) and passive controls (no subsequent task). Unlike non-synesthetes, synesthetes thus measurably experience their synesthetic colors effortlessly, as their conscious color phenomenology allowed them to see and pick the right color, much like non-synesthetes view an actual (typeface) color.

Effort-evoked pupil dilation also speaks to an important theoretical question, as synesthetes often report richer mental imagery (Barnett & Newell, 2008; Spiller et al., 2015); could our findings reflect an active color-imagery strategy (akin to Goldflam, 1922; Kay et al., 2022; Laeng & Sulutvedt, 2014) rather than automatic color emergence? We deem this highly unlikely. First, generating and maintaining mental images is effortful and produces according pupil dilations (Bumke, 1911; Henderson et al., 2018; Kay et al., 2022; Kosslyn et al., 1988; Laeng & Sulutvedt, 2014). Second, the response timeline supports automatic generation. Synesthetic color affected pupil size after 870 ms. Assuming a constant pupil light response latency to external colors (here 330 ms, see (Bergamin & Kardon, 2003)) and internally generated colors, plus at least 150 ms for digit recognition (Park et al., 2014; Walla & Klimovic, 2025), this leaves only 390 ms for imagery - which in turn has been shown to be slower (500 ms+, based on MEG, Dijkstra et al., 2018). Our fast pupil responses support previous physiological studies showing early emergence of the synesthetic response (Brang et al., 2011), and a proposed synesthetic physiological mechanism of a recurrent loop from grapheme recognition to color perception during the forward sweep of visually presented information (Laeng et al., 2004). Future work could further specify this mechanism using pupillometry.

Pupillometry thus allows to measure several key aspects of synesthesia, most prominently the unique phenomenology, the precise temporal onset, time course and strength of these effects, but also the degree of effort evoked by color reports. The measure is further unobtrusive, relatively inexpensive, and has a high signal-to-noise ratio for a physiological marker. Perhaps most importantly, pupillometry physiologically tracks synesthetic brightness in the sensory organ itself. Further-more, this method adds a metric allowing between-trial or between-participant quantitative comparisons that is not present in subjective reports: if two individuals indicate seeing a ‘bright-blue’ experience on a questionnaire, does this mean that their experiences are the same? Can their intensities ever be compared? Pupillometry thus extends the synesthesia research toolbox by providing an objective metric with shared measurement unit for the quality and strength of inducer-to-concurrent links, suited both across and within individuals.

Taking a broader perspective, this work more informs about cross-modal processes: associations stored in a synesthete’s brain can guide the constructive mechanisms that generate additional conscious sensations. Consistent with this view, recent work has identified cross-modal predictive-coding mechanisms, in which unimodal predictions are integrated across distributed networks to form cross-modal representations (Huang et al., 2024). We suggest that synesthesia may exemplify such a cross-modal constructive process, extending to the level of additional sensory color phenomenology. Indeed, synesthesia is an important phenomenon in context of interdisciplinary consciousness research. Fascinatingly, a synesthete may see a presented number 4 as bright-blue, while knowing and seeing it as dark gray at the same time (Dixon et al., 2004; Mills, 1999). Such idiosyncratic “extra” challenge the leading frameworks of consciousness. Integrated Information Theory (Tononi, 2004) posits that each unified conscious moment corresponds to a single maximally integrated causal complex, yet (stronger projecting) synesthetes experience an additional color dimension bound to the same grapheme. Similarly, Global Workspace Theory holds that only one content is broadcast at a time (e.g. Baars, 1988, 2005; Dehaene et al., 2017), yet shape and synesthetic color co-occupy the global workspace without interference. By varying the luminance of inducers while measuring the pupil light response in synesthetes, future work may disambiguate the (relative) strength of these two contributors to perception. Similarly, the here demonstrated perceptual nature of synesthetic perception may inform research into internally generated percepts and their properties. With the findings and technique proposed here, it remains inaccessible what it ‘feels like to be a bat’ (Nagel, 1974), yet we may come closer to objectively measure the seamless ‘controlled hallucinations’ (Seth, 2019) that we experience as reality.

4 Methods

4.1 Participants, inclusion, and ethics

Sixteen synesthetes (M_age = 23.94, SD_age = 3.40, 13 women, 3 men), n = 16 age-matched control participants watching graphemes passively (M_age = 23.50, SD_age = 2.11, 9 women, 7 men), as well as n = 16 age-matched ‘active’ control participants watching graphemes and indicating colors forced-choice (M_age = 24.75, SD_age = 4.20, 14 women, 2 men), all with otherwise normal or corrected-to-normal vision took part in the tasks. No participants were excluded. All participants had normal or corrected-to-normal vision without eye diseases. Synesthetes and controls were recruited through a snowball sample using a survey that was sent to thousands of people in the Netherlands and neighboring countries, using messenger apps and forums (Whatsapp, Signal, Reddit). Furthermore, the survey was distributed to a worldwide synesthesia network (Day, 2016, 2025) and participants with synesthesia of a concurrently running study at the University of Amsterdam were approached. Respondents indicated their form of synesthesia if present. Controls indicated not having synesthesia. The experimental procedure was approved by Utrecht University’s Faculty of Social Sciences ethical review board (24-0521). We herein predicted our main finding - the pupil light response to reflect the quality of synesthetic color. All participants gave written informed consent prior to participation.

4.2 Apparatus

Gaze position and pupil size were recorded at 1000 Hz using an Eyelink 1000 desktop mount (SR Research, Ontario, Canada) in a brightness- and sound-attenuated, mostly dark laboratory. A chin- and forehead-rest limited head movements. Stimuli were presented using PsychoPy (v.2024.2.3; Peirce et al., 2019) on an ASUS ROG PG278Q monitor (2560 x 1440, 100 Hz) positioned 67.5 cm away from eye position. The monitor was not linearized. The eye-tracker was calibrated and validated (7 points) at the beginning of the session and recalibrated whenever necessary throughout the experiment.

4.3 Procedure, task, and stimuli

Before the experiment, grapheme-color synesthetes indicated where they see their synesthetic colors, (in the mind versus in the outside world) on the ‘projector-associator’ (PA) questionnaire, answering twelve questions on a five-point Likert scale (Rouw & Scholte, 2007). Furthermore, participants (active controls and synesthetes) assessed for each grapheme separately the subjective strength of color-grapheme couplings on a 5-point scale from 0 (’None’, no color coupling) to 4 (’Very strong’, very strong color coupling). This rating is referred to as coupling strength in this manuscript. See Supplementary Materials for the questionnaire used.

The experiment started with calibration and validation of the eyetracker. Next, participants began with Block 1 of the task (see Figure 1c). Participants were first presented with a fixation cross for 1 s on a dark gray screen, if gaze was successfully kept central during this baseline screen (within 1.5 °visual angle from the center), this was followed by a random single digit number (0-9, letter height: 1.42° visual angle), presented for 4 s. Digits occupying more physical space on the screen (e.g., ‘8’) were presented less bright than digits occupying less space on the screen (e.g., ‘1’), scaled between 65% and 75% of the luminance range of the screen. Subsequent to this decisive measurement phase, participants saw the same digit as before as a reference on the screen. Participants were asked to use their mouse to indicate hue, saturation, and lightness (HSL) using sliders. These sliders changed the color of a circle surrounding the reference digit. The herewith obtained per-trial lightness values (L of HSL) form the main predictor in the manuscript. Only synesthetes were allowed to press ‘space’ to indicate the absence of any synesthetic color in which case the lightness of the screen background was used. Active controls were forced to always indicate a color, passive controls did not indicate a color. Participants were instructed not to blink or look away from the central character during baseline and stimulus phase to prevent pupil foreshortening errors and luminance confounds (Strauch et al., 2022). Consequently, trials containing blinks or gaze deviating more than 1.5°visual angle from the center were discarded and had to be repeated in random sequence until 120 valid trials (i.e., 12 trials per digit/grapheme) were collected. Trials were followed by a 2 s interstimulus interval, indicated by a centrally presented ‘x’ during which participants could blink or look away. In total, pupil responses of 5,760 trials were assessed in Block 1 in total, 1,920 from each of the three groups.

Only in synesthetes, we subsequently assessed pupil light responses to physically presented colors (Block 2, see Figure 1c). Block 2 was similar to Block 1, except that no digits were presented during the stimulus phase, but a centrally presented colored disc of 1.98 °visual angle in diameter, again on a dark gray screen. Ten different colors were presented in random sequence, one per trial. The color of the disc corresponded to the average per digit recreated color by the synesthete. Additionally, a smaller gray square was presented in the center of the disc, corresponding the same grayness value and number of pixels as the corresponding digit during Block 1. I.e., if a participant chose an onaverage bright-blue for a ‘5’ in Block 1, they were presented with a bright-blue in Block 2 with a central gray box corresponding to the grayness and number of pixels contained in the ‘5’. Just as for Block 1, trials were followed by an interstimulus interval. Again, participants had to keep gaze in the center and not blink (trials with blinks were repeated in random sequence). Every ten trials, a pause screen allowed participants to take a break. Per color, 5 trials were assessed, i.e., 50 trials in total. In total, pupil responses of 800 trials were assessed from synesthetes in Block 2. The experiment took about 60 minutes in total for synesthetes and about 15 minutes for passive controls (40 for active controls).

4.4 Data processing

All data were processed using custom Python (v3.10) and R (v4.4.3) scripts.

4.4.1 Pupil data

Pupillometric data were preprocessed following (Mathôt & Vilotijević, 2023; Strauch et al., 2022). Data were first filtered for valid trials only and downsampled to 100 Hz and subtractively baseline corrected using the mean pupil size during the last 50 ms of the baseline directly preceding the stimulus phase. Statistical tests over time were not corrected for type-1 error, which is why we recommend caution before interpreting short and briefly significant intervals strongly and to rely on analyses performed on averages per timebin in doubt.

4.4.2 Color reports

Per-trial reported colors in RGB space were converted to HLS (hue, lightness, space) color space. The lightness domain was used to infer the qualia of synesthetic color. Color consistency was calculated following (Rothen et al., 2013) under slight adjustments. For each participant and grapheme, the mean pairwise Euclidean distance between all twelve RGB color selections was computed and subsequently averaged. Smaller values indicate smaller color distance and thus higher internal color consistency.

4.4.3 Questionnaire data

Questionnaire data (excluding the screening questionnaire) were collected on paper before the start of the eye-tracking task; responses were later digitalized. We calculated the projector-associator (PA) score following Rouw and Scholte (2007). Furthermore, we assessed the subjective strength of the coupling between each grapheme and color (referred to as ‘coupling strength’ elsewhere; see Supplementary Materials for the questionnaire).

Data availability

Full materials, data, and analyses are available via the Open Science Framework https://osf.io/b6d8j/.

Acknowledgements

We thank Tessie Hamers for assistance with piloting and all participants for their contribution.

Additional files

Supplementary Material

Additional information

Funding

Nederlandse Organisatie voor Wetenschappelijk Onderzoek (NWO)

https://doi.org/10.61686/EEGGV23807

• Christoph Strauch