Graphic illustration of the horizontal smearing occurring when averaging multiple views of a face.

The tolerance of human face identity recognition to drastic appearance variations caused by varying lighting, viewpoint, facial expression, etc. has been proposed to emerge through averaging (Burton et al., 2005). With increased exposure to a face, an averaging mechanism would progressively whiten accidental variations in appearance while preserving stable cues to identity . Past illustrations (Burton, 2013; Burton et al., 2005) used varying lighting and expressions but moderate pose variations. Here we show that when pose varies largely from left to right profile, averaging horizontally smears the face suggesting that across encounters with a face, cues at orientations other than horizontal are whitened. As the observer learns the natural statistics of a person’s face, it seems plausible that they increasingly rely on horizontal cues for identity recognition. Images of two celebrities (George Clooney and Daniel Radcliffe) were sampled from the internet and sorted into three view categories: frontal, left-, and right-averted. In order to illustrate the horizontal smearing due to view variations, the averages were made of 40% left-averted, 40% right-averted, and 20% frontal views, in line with exposure to face views in natural viewing (Oruc et al., 2019). The luminance and RMS contrast of the averaged faces were set to a luminance of .5 and contrast of .4. Using this procedure, one can appreciate the emergence of the so-called bar code, namely the vertical arrangement of horizontally-oriented cues which carries the natural statistics of the face category and of face individual identity (Dakin & Watt, 2009).

Stimulus conditions.

Columns. Each identity was viewed from seven different viewpoints ranging from +75° to −75° in steps of 25°. Rows. All images were filtered in the Fourier domain to preserve only a selective range of orientation, from 0° (vertical) to 157.5° in steps of 22.5°.

A. Sensitivity of human observers to facial identity (d’) as a function of the orientation filter (0° to 180° in 22.5° steps), and face viewpoint (yaw: +75° to −75° in 25° steps). Dots and error bars represent mean d’ values and 95% confidence intervals across participants. Solid lines and shaded areas indicate the mean posterior predictions and 95% credible intervals from the Gaussian Bayesian multilevel model. B. Population-level mean parameters of the Gaussian Bayesian Multilevel model, plotted with 95% credible intervals as a function of face viewpoint. The 95% credible intervals reflect the uncertainty of the model. They indicate a 95% probability that the true population parameter lies within the specified range, given the observed data.

Posterior mean and 95% credible interval for each parameter of the Gaussian model, at each viewpoint.

Sensitivity (i.e. performance in the recognition task) of human and model observers and image energy across viewpoints and orientations.

Left column. 3D surf plots of the normalized energy/sensitivity across orientation and viewpoints. Middle column. Matrix representations of the normalized energy/sensitivity across orientation and viewpoints. Right column. Matrix representations of the Pearson correlation (non Fisher Z-transformed) of the normalized orientation distributions of energy/sensitivity across viewpoints.

Mean and 95% confidence interval of the (Fisher Z-transformed) partial correlation coefficients between human and each model orientation d’ profiles while controlling for the variance in image energy and alternate model.

Fisher Z-transformed Pearson partial correlation of the orientation sensitivity profiles between humans and each model, while controlling for the alternate model and image energy profiles.

Error bars show the 95% confidence intervals. The faded grey line depicts the maximally achievable correlation for the separate viewpoint conditions in the human dataset (see Methods for details).

Difference of the human-model partial correlation coefficients between viewpoint-selective and view-tolerant model observers.

Positive t values indicate a stronger correlation with the viewpoint-selective model and negative t values stronger correlation with the view-tolerant model.

View-average model observer.

A. Image averages of the different views of each face identity of the stimulus set. B. Sensitivity of the view-average model observer across views and orientations. From left to right: 3D surf plots of the normalized sensitivity across orientation and viewpoints, matrix representations of the normalized sensitivity across orientation and viewpoints, and matrix representations of the Pearson correlation of the normalized orientation distributions of sensitivity across viewpoints.