Experience transforms crossmodal object representations in the anterior temporal lobes
Abstract
Combining information from multiple senses is essential to object recognition, core to the ability to learn concepts, make new inferences, and generalize across distinct entities. Yet how the mind combines sensory input into coherent crossmodal representations - the crossmodal binding problem - remains poorly understood. Here, we applied multi-echo fMRI across a four-day paradigm, in which participants learned 3-dimensional crossmodal representations created from well-characterized unimodal visual shape and sound features. Our novel paradigm decoupled the learned crossmodal object representations from their baseline unimodal shapes and sounds, thus allowing us to track the emergence of crossmodal object representations as they were learned by healthy adults. Critically, we found that two anterior temporal lobe structures - temporal pole and perirhinal cortex - differentiated learned from non-learned crossmodal objects, even when controlling for the unimodal features that composed those objects. These results provide evidence for integrated crossmodal object representations in the anterior temporal lobes that were different from the representations for the unimodal features. Furthermore, we found that perirhinal cortex representations were by default biased towards visual shape, but this initial visual bias was attenuated by crossmodal learning. Thus, crossmodal learning transformed perirhinal representations such that they were no longer predominantly grounded in the visual modality, which may be a mechanism by which object concepts gain their abstraction.
Data availability
Anonymized data are available on the Open Science Framework: https://osf.io/vq4wj/.Univariate maps are available on NeuroVault: https://neurovault.org/collections/LFDCGMAY/
-
Crossmodal Object Representations Rely on Integrative CodingOpen Science Framework, https://doi.org/10.17605/OSF.IO/VQ4WJ.
-
https://neurovault.org/collections/LFDCGMAY/NeuroVault, https://identifiers.org/neurovault.collection:12807.
Article and author information
Author details
Funding
Natural Sciences and Engineering Research Council of Canada (Alexander Graham Bell Canada Graduate Scholarship-Doctoral)
- Aedan Yue Li
Natural Sciences and Engineering Research Council of Canada (Discovery Grant (RGPIN-2020-05747))
- Morgan Barense
James S. McDonnell Foundation (Scholar Award)
- Morgan Barense
Canada Research Chairs
- Morgan Barense
Ontario Ministry of Research and Innovation (Early Researcher Award)
- Morgan Barense
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Ethics
Human subjects: All experiments described in this study were approved by the University of Toronto Ethics Review Board: 37590. Informed consent was obtained for all participants in the study.
Copyright
© 2024, Li et al.
This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.
Metrics
-
- 877
- views
-
- 156
- downloads
-
- 1
- citations
Views, downloads and citations are aggregated across all versions of this paper published by eLife.
Download links
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Further reading
-
- Neuroscience
Active inference integrates perception, decision-making, and learning into a united theoretical framework, providing an efficient way to trade off exploration and exploitation by minimizing (expected) free energy. In this study, we asked how the brain represents values and uncertainties (novelty and variability), and resolves these uncertainties under the active inference framework in the exploration-exploitation trade-off. Twenty-five participants performed a contextual two-armed bandit task, with electroencephalogram (EEG) recordings. By comparing the model evidence for active inference and reinforcement learning models of choice behavior, we show that active inference better explains human decision-making under novelty and variability, which entails exploration or information seeking. The EEG sensor-level results show that the activity in the frontal, central, and parietal regions is associated with novelty, while the activity in the frontal and central brain regions is associated with variability. The EEG source-level results indicate that the expected free energy is encoded in the frontal pole and middle frontal gyrus and uncertainties are encoded in different brain regions but with overlap. Our study dissociates the expected free energy and uncertainties in active inference theory and their neural correlates, speaking to the construct validity of active inference in characterizing cognitive processes of human decisions. It provides behavioral and neural evidence of active inference in decision processes and insights into the neural mechanism of human decisions under uncertainties.
-
- Genetics and Genomics
- Neuroscience
The mammalian suprachiasmatic nucleus (SCN), situated in the ventral hypothalamus, directs daily cellular and physiological rhythms across the body. The SCN clockwork is a self-sustaining transcriptional-translational feedback loop (TTFL) that in turn coordinates the expression of clock-controlled genes (CCGs) directing circadian programmes of SCN cellular activity. In the mouse, the transcription factor, ZFHX3 (zinc finger homeobox-3), is necessary for the development of the SCN and influences circadian behaviour in the adult. The molecular mechanisms by which ZFHX3 affects the SCN at transcriptomic and genomic levels are, however, poorly defined. Here, we used chromatin immunoprecipitation sequencing to map the genomic localization of ZFHX3-binding sites in SCN chromatin. To test for function, we then conducted comprehensive RNA sequencing at six distinct times-of-day to compare the SCN transcriptional profiles of control and ZFHX3-conditional null mutants. We show that the genome-wide occupancy of ZFHX3 occurs predominantly around gene transcription start sites, co-localizing with known histone modifications, and preferentially partnering with clock transcription factors (CLOCK, BMAL1) to regulate clock gene(s) transcription. Correspondingly, we show that the conditional loss of ZFHX3 in the adult has a dramatic effect on the SCN transcriptome, including changes in the levels of transcripts encoding elements of numerous neuropeptide neurotransmitter systems while attenuating the daily oscillation of the clock TF Bmal1. Furthermore, various TTFL genes and CCGs exhibited altered circadian expression profiles, consistent with an advanced in daily behavioural rhythms under 12 h light–12 h dark conditions. Together, these findings reveal the extensive genome-wide regulation mediated by ZFHX3 in the central clock that orchestrates daily timekeeping in mammals.