Concurrent category-selective neural activity across the ventral occipito-temporal cortex supports a non-hierarchical view of human visual recognition
- Université de Lorraine, CNRS, IMoPA, France
- Université de Lorraine, CHRU-Nancy, Service de Neurologie, France
- Université de Lorraine, CHRU-Nancy, Service de Neurochirurgie, France
Figures
Figure 1 with 1 supplement
Recording and quantifying time-domain face-selective activity in the ventral occipito-temporal cortex (VOTC).
(A) Stereotactic electroencephalography (SEEG) (depth) electrode arrays (white circles) shown on the reconstructed white matter surface of one of the participants (ventral view of the left hemisphere). Electrodes penetrate both gyral and sulcal cortical tissues. (B) The frequency-tagging paradigm to quantify face-selective neural activity: images of nonface objects appear at a rate of six stimuli per second (6 Hz) with variable face images presented every five stimuli (i.e. every 0.835 s). Each stimulation sequence lasts for 65 s (2 s showed here). (C) Representative examples of natural face images used in the study (actual images not shown for copyright reasons). (D) Top: example raw intracranial electroencephalography (EEG) signal measured at the bipolar recording contacts shown in panel A (red). The signal is shown from –1.5–20 s relative to the onset of a stimulation sequence. The time-series displayed is an average of two sequences. Above the time-series, red vertical ticks indicate the appearance of face image every 0.835 s and small black vertical ticks indicate the appearance of non-face objects every 0.167 s. Middle: time-by-frequency representation of SEEG data in the HFB range (30–160 Hz). The plot shows the percent signal change at each frequency relative to a pre-stimulus baseline period (–1.6to –0.3s). Periodic bursts of HFB activity at the frequency of face stimulation (i.e. 1.2 Hz) are visible. Bottom: modulation of HFB amplitude over time obtained by averaging time-frequency signals across 30–160 Hz. (E) HFB signal is transformed in the frequency domain to quantify face-selective amplitude as the sum of 12 face-selective frequency harmonics. (F) Time-domain averaged HFB response to face images shows both the periodic response to non-face objects at 6 HZ (cycle duration = 0.167 s) and the larger face-selective response starting ~0.1 s after face onset. Shaded area around the curve is standard error of the mean across face trials. (G) Mean face responses from two separate example recording contacts (in posterior temporal lobe :PTL and anterior temporal lobe: ATL) in which the 6 Hz response to non-face objects has (black traces) or has not (blue traces) been filtered out.
Figure 1—figure supplement 1
Response timing in face-selective response decrease.
(A) Time-domain face-selective high frequency broadband (HFB) activity averaged by main ventral occipito-temporal cortex (VOTC) region (OCC: occipital , PTL: posterior temporal lobe, ATL: anterior temporal lobe). HFB time-series were notch-filtered to remove the general visual response at 6 Hz and harmonics, leaving only face-selective signals. The maximum amplitude of each averaged waveform was normalized to –1 for visualization purposes only. Shaded area represents the standard error of the mean between participants. (B) Onset latency for each VOTC main region and for four latency estimation methods, together with 95% confidence intervals (percentile bootstrap).
Figure 2
Time-domain face-selective periodic increases.
Left column: Time by frequency response (percent signal change, psc; see scale value at the top of each plot, as well as number of recording contacts/participants, and color scale at the bottom) in the HFB range (30–160 Hz) averaged over face-selective contacts in each main VOTC region (OCC: occipital, bottom; PTL: posterior temporal lobe, middle; ATL: anterior temporal lobe, top). For each recording contact, the time-frequency data was segmented in epochs of about three face cycles (i.e. 3×0.833 s), averaged by contacts and then averaged over the three groups of contacts. Frequency axis is on the left. Green traces are the HFB amplitude envelope obtained by averaging over the 30–160 Hz range. Amplitude (psc) axis is on the right, shown in green. Right column: Frequency spectra averaged over the corresponding groups of recording contacts and showing the face-selective response (red circles) at multiples of 1.2 Hz (i.e. face stimulation frequency) and visual response at 6 Hz (black square, other harmonics not shown). This highlights the sharp decrease in general visual response (i.e. 6 Hz) relative to face-selective response from posterior to anterior VOTC.
Figure 3 with 1 supplement
Spatial organization and increase in abstraction of face-selective HFB activity inVOTC.
(A) Map of all VOTC recording contacts across the 140 individual brains displayed in the Talairach space in a transparent reconstructed cortical surface of the Colin27 brain (ventral view). Each circle represents a single recording contact. Face-selective contacts are color-coded according to their anatomical location in the original individual anatomy. White-filled circles correspond to contacts without significant face-selective activity. Values along the y-axis of the Talairach coordinate system (postero-anterior) are shown near the interhemispheric fissure. (B) VOTC maps of the local proportion of face-selective contacts relative to the number of recorded contacts. Black contour lines delineate local proportions significantly above zero (p<0.01, percentile bootstrap). (C) The number of face-selective contacts is shown for each anatomical region (region defined in each individual participant) and hemisphere. (D) Face-selectivity index (FSI) along the postero-anterior axis collapsed along the X dimension (medio-lateral). The shaded area shows the 99% confidence interval computed using a percentile bootstrap. (E) Map of the proportion of face-exclusive (i.e. face-selective without significant response to nonface-objects at 6 Hz and harmonics) relative to face-selective contacts across VOTC.
Figure 3—figure supplement 1
Face-selective recording contacts anatomical labels.
(A) Anatomical regions were defined in each individual hemisphere according to major anatomical landmarks. The ventral temporal sulci (COS, OTS, and midfusiform sulcus, i.e. MFS) serve as medial/lateral borders of regions, whereas three coronal reference planes containing anatomical landmarks (OCC-PTL: anterior tip of the parieto-occipital sulcus, i.e. POS, PTL-ATL: posterior tip of the hippocampus, i.e. HIP, ATL-TP: limen insulae) serve as an anterior/posterior boundary for each region. Electrode contacts were considered to be in the ATL if they were located anteriorly to the posterior tip of the hippocampus and posteriorly to the limen insulae. The schematic locations of these anatomical structures are shown on a reconstructed cortical surface of the Colin27 brain. Acronyms: TP: temporal pole; ATL: anterior temporal lobe; PTL: posterior temporal lobe; OCC: occipital lobe; PHG: parahippocampal gyrus; CoS: collateral sulcus; FG: fusiform gyrus; ITG: inferior temporal gyrus; MTG: middle temporal gyrus; OTS: occipito-temporal sulcus; CS: calcarine sulcus; IOG: inferior occipital gyrus; LG: lingual gyrus; ant: anterior; lat: lateral; med: medial. (B) Map of all face-selective recording contacts and displayed in the Talairach space. Each circle represents a single face-selective contact color-coded according to its anatomical location in the original individual anatomy (see legend on the right).
Figure 4 with 9 supplements
Concurrent face-selective activity across the VOTC.
(A) Mean time-domain face-selective HFB activity in each VOTC region (OCC: occipital, PTL, ATL) collapsed across hemisphere. HFB time-series were filtered to remove the general visual response at 6 Hz and harmonics, leaving only face-selective signals. The maximum amplitude of each averaged waveform was normalized to 1 for visualization purposes only (see Figure 4—figure supplement 3 for non-normalized waveforms). Shaded area represents the standard error of the mean between participants. Colored vertical lines indicate onset latencies across four different methods (see markers’ legend in panel B) and offset latencies (estimated at 5% of peak amplitude) for each VOTC region. (B) Onset latency for each VOTC main region with four latency estimation methods, together with 95% confidence intervals as error bars (percentile bootstrap). (C) Offset latency (with 95% confidence intervals) in each region. Asterisks indicate significant differences (p < 0.05, two-tailed permutation test, fdr-corrected) (D) Correlation of time-series between regions and 95% confidence intervals. (E) Between-region area under the curve (AUC) overlap. The arrow shows the directionality of the computed overlap. For instance, the arrow from ATL to OCC indicates the percentage of the total AUC of ATL (measured between onset and offset latencies) occupied by the AUC of OCC (determined between max(onset(OCC, ATL)) and min(offset(OCC, ATL))).
Figure 4—figure supplement 1
Estimating onset and offset latencies.
The onset latencies of face-selective activity were characterized using four different methods. The offset latency was characterized with one method (i.e. percent of peak). (A) In the z-score method, the HFB time-series was converted to z-score values by subtracting the mean amplitude in the baseline window of the time-series (i.e. before face onset: [-0.166–0 s]) and dividing by the standard deviation of the amplitude in the same baseline window. Z-scores were converted to p-values which were FDR-corrected (Benjamini and Hochberg, 1995). Onset latency was the first time-point after 40 ms (which was taken as a lower bound for physiologically plausible response latencies after face onset) at which p<0.05 (two-tailed), for at least 30 ms. (B) In the delta slope method, the slopes of the variation in HFB amplitude are computed at each time point within a 25 ms sliding window (see bottom plot). Onset latency is defined as the time (with a lower bound of 40 ms) at which the slope exceeds 2.32 times the mean and standard deviation of the slopes in the baseline window (i.e. an ‘acceleration’ of the electrophysiological signal manifesting the onset of the neural response), for a duration of at least 30 ms. (C) In the broken stick method, a regression-based method (Mordkoff and Gianaros, 2000), onset latency is defined as the intersection point (lower bound set to 40 ms) of two line segments (with variable slopes) best fitted (least square error fit) to the HFB signal between the start of the baseline window (–166 ms) and the end of the ramping-up portion of the signal before the first peak. (D) In the percent peak method, onset latency is defined as the first point, after 40 ms, that rises above 20% of the amplitude difference between the baseline window and the peak (maximum amplitude between 0 and 0.8 s), for at least 30 ms. Offset latency was defined using the ‘percent peak’ approach, as the point in time, after the onset latency, where the signal reaches below 5% of the amplitude difference between baseline and peak.
Figure 4—figure supplement 2
Additional parameters for timing analyses.
The timing of face-selective activity was also characterized by computing response overlap and response correlation across VOTC regions. (A) A parameter estimated the overlap between time-series for pairs of regions (e.g. regions A and B). The overlap is asymmetrical and is calculated separately for region A and region B as the ratio between the area under the curve (AUC) of the overlap between regions A and region B (i.e. summing the amplitude between the maximum of onset A and B and the minimum of offset A and B) and the total AUC for region A (for overlap of region A to B) or for region B (for overlap of region B to A). (B) For the last timing parameter, Pearson correlations were computed between the time-series of pairs of regions using data between 0 and 0.6 s relative to face onset (gray shaded area). This provided an estimate of the similarity between the response function of the two regions compared. The shape of the response is more similar between region A and B than between region A and C. Between region correlations were compared against within-region correlations.
Figure 4—figure supplement 3
Non-normalized time-courses.
Time-domain face-selective HFB activity in contacts showing response increase, averaged by main VOTC region (OCC, PTL, ATL) across hemisphere. HFB time-series were notch-filtered to remove the general visual response at 6 Hz and harmonics, leaving only face-selective signals. Time-series are original non-normalized versions of Figure 4 of the main text. Shaded area represents the standard error of the mean between participants.
Figure 4—figure supplement 4
Time-course face-selective contacts split by hemispheres.
Response timing in face-selective contacts grouped by main region separately for left (top) and right (bottom) hemispheres. (A) Mean time-domain face-selective HFB activity. HFB time-series were notch-filtered to remove the general visual activity at 6 Hz and harmonics, leaving face-selective signals only. The maximum amplitude of each averaged waveform was normalized to 1 for visualization purposes only. Shaded area represents the standard error of the mean between participants. Number of participants and contacts are indicated in the legend. (B) Onset latency for each VOTC main region and for 4 latency estimation methods, together with 95% confidence intervals (percentile bootstrap).
Figure 4—figure supplement 5
Response timing in core ventral occipito-temporal cortex (VOTC) face-selective regions.
Response timing in face-selective contacts showing response increase in core VOTC face-selective regions: Inferior occipital gyrus (IOG, N=65), lateral fusiform gyrus and OTS (latFG, N=123) and antFG+ (anterior fusiform gyrus, anterior occipitotemporal sulcus, anterior collateral sulcus, N=184). (A) Mean time-domain face-selective HFB activity in each core face-selective VOTC region collapsed across hemisphere. HFB time-series were filtered to remove the general visual response at 6 Hz and harmonics so only face-selective signal remains. The maximum amplitude of each averaged waveform was normalized to 1 for visualization purposes only. Shaded area represents the standard error of the mean between participants. (B) Onset latency for each VOTC main region and for four latency estimation methods, together with 95% confidence interval (percentile bootstrap). (C) Offset latency in each region. (D) Correlation of time-series between regions and 95% confidence intervals. (E) Between-region area under the curve (AUC) overlap. The arrow shows the directionality of the computed overlap. For instance, the arrow from ATL to OCC indicates the percentage of the total AUC of ATL (measured between onset and offset latencies) occupied by the AUC of its temporal overlap with OCC (determined between max(onset(OCC, ATL)) and min(offset(OCC, ATL))). (F) Variation of face-selective response latency along the postero-anterior axis for four estimation methods. For each method, each data point represents the onset latency measured from the time-series averaged over contacts collapsed across the medio-lateral X dimension within 20 mm segments (in the Y dimension). Thick lines are estimated onset latencies and shaded areas show the 99% confidence intervals expected under the null hypothesis that the postero-anterior location has no influence on the onset latency, accounting for the expected conduction delays between posterior and anterior VOTC. Green lines show the expected increase in response onset latency based on simple axonal conduction delays (Lemaréchal et al., 2022; van Blooijs et al., 2023) due to increasing distance from the occipital region, with reference to the latency averaged over the OCC region for the ‘z-score’ method. Lines show mean expected conduction velocity for direct cortico-cortical connections (~3.5 m/s) and ±1 std (1.7 and 5.3 m/s).
Figure 4—figure supplement 6
Face-selective timing in low frequency event-related potentials (ERPs).
(A) Map of all VOTC contacts showing a significant (Z>3.1) face-selective response in low-frequency ERPs, displayed in the Talairach space in a transparent reconstructed cortical surface of the Colin27 brain (ventral view). Each circle represents a single recording contact. Contacts are color-coded according to their anatomical location in the original individual anatomy. Low frequency signal is the electrophysiological response measured at each recording contact after bipolar re-referencing (Jacques et al., 2022), dominated by low frequency event-related voltage variations. Only contacts with split-half correlation in the time-domain (i.e. reliability)>0.66 (i.e. 75% of contacts) were kept to ensure reliable latency estimations. (B) Grand average face-selective ERP in each face-selective VOTC region collapsed across hemispheres. ERP time-series were filtered to remove the general visual response at 6 Hz and harmonics, leaving face-selective signals only. Averaged ERP were baseline-corrected relative to a [–0.166 0]s window before face onset and the absolute value was computed to allow averaging across contacts/participants despite variability in ERP polarity and morphology. ERP at individual recording contacts were averaged by participants and then across participants in each region. The maximum amplitude of each averaged waveform was normalized to 1 for visualization purposes only. Shaded area represents the standard error of the mean between participants. (C) Onset latency for each VOTC main region and for three latency estimation methods (same as in main text except for ‘z-score’ method, which is unusable with low frequencies), together with 95% confidence intervals (percentile bootstrap). (D) Offset latency in each region with 95% confidence intervals. (E) Correlation of time-series between regions and 95% confidence intervals. (F). Between-region area under the curve (AUC) overlap. The arrow shows the directionality of the computed overlap. For instance, the arrow from ATL to OCC indicates the percentage of the total AUC of ATL (measured between onset and offset latencies) occupied by the AUC of its temporal overlap with OCC (determined between max(onset(OCC, ATL)) and min(offset(OCC, ATL))). (G) Face-selective onset latency map across VOTC, collapsed across hemispheres and displayed for three different methods to estimate response onset latency. (H) Variation of face-selective response latency along the postero-anterior axis for three estimation methods. For each method, each data point represents the onset latency measured from the time-series averaged over contacts collapsed across the medio-lateral X dimension within 20 mm segments (in the Y dimension). Thick lines are estimated onset latencies and shaded areas show the 99% confidence intervals expected under the null hypothesis that the postero-anterior location has no influence on the onset latency, accounting for the expected conduction delays between posterior and anterior VOTC. Green lines show the expected increase in response onset latency based on axonal conduction delays (Lemaréchal et al., 2022; van Blooijs et al., 2023) due to increasing distance from the occipital region, with reference to the latency averaged over the OCC region for the ‘z-score’ method. Lines show mean expected conduction velocity for direct cortico-cortical connections (~3.5 m/s) and ±1 std (1.7 and 5.3 m/s). Note that response latency from few recording contacts located posterior to y = –85 (around and posterior/medial to the posterior transverse collateral sulcus), likely in early visual cortex (V1, V2v, V3v, hV4, Brewer et al., 2005; Winawer and Witthoft, 2015) and potentially driven by low-level cues (Natu et al., 2019), is up to 40 ms earlier than responses measured elsewhere in the VOTC, where little difference is observed.
Figure 4—figure supplement 7
HFB time-domain face-selective response in individual participants in OCC region.
Time courses are averaged across contacts within participants. The 6 Hz response to non-face objects has been filtered-out. Part 1: responses in the occipital cortex (OCC).
Figure 4—figure supplement 8
HFB time-domain face-selective response in individual participants in PTL region.
Time courses are averaged across contacts within participants. The 6 Hz response to non-face objects has been filtered-out. Part 2: responses in the posterior temporal lobe (PTL).
Figure 4—figure supplement 9
HFB time-domain face-selective response in individual participants in ATL region.
Time courses are averaged across contacts within participants. The 6 Hz response to non-face objects has been filtered-out. Part 3: responses in the anterior temporal lobe (ATL).
Figure 5 with 1 supplement
Concurrent functional connectivity between face-selective regions.
Each plot shows the group-averaged Pearson’s correlations between single-trial face-selective amplitude measured at two distinct face-selective regions (left: Inferior occipital gyrus (IOG) and latFG; middle: latFG and antFG+; right: IOG and antFG+) computed at each time point, representing functional connectivity between pairs of regions. Black contour lines indicate significant positive correlations (randomization test, p<0.01, FDR-corrected). Correlations were computed across –150–150 ms lags between regions to infer direction of connectivity. The black dashed diagonal line represents a 0 ms time-lag between regions. Correlations centered above the diagonal would indicate that face-selective activity in the more anterior region (e.g. latFG in left panel) correlates with but precedes activity in the posterior region (e.g. IOG in left panel), suggesting an information flow from anterior to posterior, and the reverse for correlations centered below the diagonal.
Figure 5—figure supplement 1
Concurrent functional connectivity between face-selective regions computed using mutual information.
Each plot shows the group-averaged mutual information (MI) between single trials face-selective amplitude measured at two distinct face-selective regions (left: inferior occipital gyrus IOG and latFG; middle: latFG and antFG+; right: IOG and antFG+) computed at each time point, representing functional connectivity between pairs of regions. MI was computed using the method and MATLAB code described in Ince et al., 2017. Statistics were performed using permutation tests as for Pearson correlations presented in the main manuscript. Black contour lines indicate significant MI (randomization test, p<0.01, FDR-corrected). MI was computed across –150–150 ms lags between regions to infer direction of connectivity. The black dashed diagonal line represents a 0 ms time-lag between regions. MI centered above the diagonal would indicate that face-selective activity in the more anterior region (e.g. latFG in left panel) is coupled with but precedes activity in the posterior region (e.g. IOG in left panel), suggesting an information flow from anterior to posterior, and the reverse for MI centered below the diagonal.
Figure 6 with 1 supplement
Mapping concurrent face-selective response onset latency in ventral occipito-temporal cortex (VOTC).
(A) Face-selective onset latency map across VOTC, collapsed across hemispheres and displayed for four different methods to estimate response onset latency (see methods). (B) Variation of face-selective response latency along the postero-anterior axis for four estimation methods. For each method, each data point represents the onset latency measured from the time-series averaged over contacts/participants collapsed across the medio-lateral X dimension within 20 mm segments (in the Y dimension). Thick lines are estimated onset latencies and shaded areas show the 99% confidence intervals expected under the null hypothesis that the postero-anterior location has no influence on the onset latency, accounting for the expected conduction delays between posterior and anterior VOTC (see methods: ‘HFB onset latency map’). Green lines show the expected increase in response onset latency based on simple axonal conduction delays (Lemaréchal et al., 2022; van Blooijs et al., 2023) due to increasing distance from the occipital region, with reference to the latency averaged over the occipital (OCC) region for the ‘z-score’ method. Lines show mean expected conduction velocity for direct cortico-cortical connections (~3.5 m/s) and ±1 std (i.e. 1.7 and 5.3 m/s).
Figure 6—figure supplement 1
Mapping response onset latency of face-selective contacts split by hemispheres.
(A) Face-selective onset latency map across VOTC, displayed for four different methods to estimate response onset latency. We collected HFB time-series from contiguous face-selective contacts located within voxels of 20×20 ×100 mm (swept across VOTC in steps of 3×3 ×100 mm), averaged collected time-series across contacts within participants and then across participants, and computed onset latency of face-selective response within the current voxels. (B) Variation of face-selective response latency along the postero-anterior axis for four estimation methods and split by hemisphere. For each method, each data point represents the onset latency measured from the time-series averaged over contacts collapsed across the medio-lateral X dimension within 20 mm segments (in the Y dimension). Thick lines are estimated onset latencies and shaded areas show the 99% confidence intervals expected under the null hypothesis that the postero-anterior location has no influence on the onset latency, accounting for the expected conduction delays between posterior and anterior VOTC. Green lines show the expected increase in response onset latency based on simple axonal conduction delays (Lemaréchal et al., 2022; van Blooijs et al., 2023) due to increasing distance from the occipital region, with reference to the latency averaged over the OCC region for the ‘z-score’ method. Lines show mean expected conduction velocity for direct cortico-cortical connections (~3.5 m/s) and ±1 std (1.7 and 5.3 m/s).
Author response image 1
Author response image 2
Author response image 3
Author response image 4
Tables
Table 1
Statistical comparisons of face-selective onset latencies across regions (OCC vs PTL, PTL vs ATL, OCC vs ATL) using permutation tests and equivalence testing (proportion of difference in ROPE and Bayes factor) for four different onset estimation methods.
Table contains the median latency difference between regions (Median diff.), effect size (Cohen’s d) of the difference, p-value of the permutation test (pval. FDR), number of participants included in each region.
| Latency method | Region comparison | Median difference (ms) | Cohen’s d | p-value FDR | Nb Participants region 1 | Nb Participants region 2 | Equivalence Prop in ROPE (%) | Bayes factor Cauchy prior (>3) |
|---|---|---|---|---|---|---|---|---|
| z-score | OCC vs PTL | 15.6 | 0.27 | 0.89568 | 25 | 48 | 32 | 0.9 |
| delta slope | OCC vs PTL | 5.9 | 0.02 | 0.89568 | 25 | 48 | 97 | 97 |
| Broken stick | OCC vs PTL | –3.9 | 0.09 | 0.89568 | 25 | 48 | 85 | 8.8 |
| % of peak | OCC vs PTL | –2 | 0.04 | 0.89568 | 25 | 48 | 86 | 10 |
| z-score | PTL vs ATL | –7.8 | 0.11 | 0.89568 | 48 | 75 | 78 | 7 |
| delta slope | PTL vs ATL | –7.8 | 0.04 | 0.89568 | 48 | 75 | 99 | 232.4 |
| Broken stick | PTL vs ATL | 3.7 | 0.05 | 0.9832 | 48 | 75 | 80 | 8.1 |
| % of peak | PTL vs ATL | –1 | 0.02 | 0.89568 | 48 | 75 | 90 | 16.4 |
| z-score | OCC vs ATL | 7.8 | 0.12 | 0.9832 | 25 | 75 | 60 | 2.3 |
| delta slope | OCC vs ATL | –3.9 | 0.09 | 0.89568 | 25 | 75 | 82 | 5.9 |
| Broken stick | OCC vs ATL | –1.1 | 0.01 | 0.89568 | 25 | 75 | 88 | 12.5 |
| % of peak | OCC vs ATL | –3.9 | 0.06 | 0.89568 | 25 | 75 | 91 | 15.2 |
Additional files
-
Supplementary file 1
Supplementary Tables A-D.
- https://cdn.elifesciences.org/articles/109640/elife-109640-supp1-v1.docx
-
MDAR checklist
- https://cdn.elifesciences.org/articles/109640/elife-109640-mdarchecklist1-v1.pdf
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Concurrent category-selective neural activity across the ventral occipito-temporal cortex supports a non-hierarchical view of human visual recognition
eLife 15:RP109640.
https://doi.org/10.7554/eLife.109640.3
