The structure of species discrimination signals across a primate radiation

Acceptance summary:

This study by Winters et al. makes significant contributions to our understanding of species discrimination processes by combining machine learning-based classification of facial feature patterns with a clever behavioral experiment approach in guenons, a closely-related group of primate species who often inhabit the same forests. In such situations, there may be strong selection on signals or cues (which could be phenotypic, olfactory, vocal, behavioral, or combinations thereof) to help individuals reliably identify other members of their same versus different species, for example to help them avoid mating with individuals from different species. Guenons have an incredible diversity of colorful facial feature variation. In their study, Winters et al. first used a computational approach to predict which facial regions might be most critical for distinguishing each species from others. They then performed behavioral "looking-time" experiments with individuals from two guenon species. One of these species has a primary defining facial feature characteristic that was predicted to be highly discriminatory on its own; in contrast, discrimination for the other species is more dependent on the combined presence of multiple facial coloration traits. The investigators presented individual monkeys with pairs of images, both representing actual coloration patterns of the same and different species (with individuals gazing significantly longer at images representing their own species) and with manipulations to represent artificial combinations of the presence or absence of the respective most discriminating feature. Individuals from the species with the dominant discriminatory feature gazed longer at images with that feature present, yet the same result was not observed among individuals from the species for which the top-ranked feature was predicted to be much less discriminatory on its own. While the looking-time results cannot be directly linked to mate choice behavior, the elegant design of this integrative study and its clear results provide insight into the diverse and multidimensional mechanisms that could plausibly be involved in such processes, even within a closely-related group of species.

Decision letter after peer review:

Thank you for submitting your article "The structure of species discrimination signals across a primate radiation" for consideration by eLife. Your article has been reviewed by four peer reviewers, including George H Perry as the Reviewing Editor and Reviewer #1, and the evaluation has been overseen by Andrew King as the Senior Editor. The following individual involved in review of your submission has agreed to reveal their identity: Tim Caro (Reviewer #4).

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

Summary:

This paper uses an integrative machine learning and behavioral experiment approach to examine facial coloration pattern variation among a radiation of geunon monkeys with respect to potential species recognition mechanisms. For two species, facial features classified as most important for identification of that species relative to other guenons were altered in photos, which were then displayed to test subjects of each species in a 3-part paired design (involving unaltered conspecific and heterospecific (both for species sharing and not sharing the dominant classification feature with the test species) and manipulated conspecific images) in a looking-time experiment. The experimental results differ between the two test subjects in a manner consistent with that which would have been predicted by the machine learning classifier in terms of the discriminatory power of the dominant feature versus necessity of multiple features in combination.

We had an extensive and productive reviewer consultation session about your manuscript. As the reviewing editor I ultimately chose to recommend a 'needs revisions' decision for your article because first, the reviewers believe that our essential revisions (listed below) can be conducted within a two month timeframe, and second, we would be enthusiastic about the article if (following re-evaluation) the revised analyses prove robust and if the biological significance of the results could be convincingly clarified. Yet I would like to emphasize that the reviewers have high expectations on these points; given their fundamental nature the revised manuscript would be thoroughly re-assessed. Given this status, I am also choosing to break the typical eLife mould by including the full reviews below, because they provide helpful context for the summarized essential revisions, while also making multiple other excellent points that I believe you would want to consider and address accordingly so that your work can reach its top potential.

Essential revisions:

1) Demonstrate that similar classifier results are obtained within a more realistic taxonomic context. That is, for each of the two test species, the model should ideally consider the species recognition problem at hand, e.g. with sympatric taxa only. If impossible given image data availability, at the least the authors should show that results with different subsets of available taxa are similar.

2) Analyze data from the three different types of trials separately; the grouping of data from across the different trial types could conflate results. Additionally, correct the specific statistical analysis approach errors as detailed by Reviewer #3.

3) Clarify the biological significance of the looking-time experimental results. The relationship of these results with species recognition processes was considered indirect at best and inferential. Specify what can concluded once these results are clarified; is this still an impactful result? If so, emphasize that aspect, while also still discussing potential implications for species recognition. As the reviewing editor, I noticed that when you had more space to describe your results you were appropriately careful (yet this was not the case in several summary sentences). However, even then I think it would be better to be more direct about this limitation, while simultaneously celebrating the explicit knowledge that is obtained if warranted.

Reviewer #1:

Overall I am highly impressed by this study and its contributions to our understanding of visual species recognition systems. The novel combination of computational prediction with experimentation in two non-human primate species is outstanding. The contrasting results between putty-nosed and mona monkeys are compelling and informative.

Specifically, individuals from both species gazed significantly longer at unaltered images of conspecifics versus heterospecifics when shown side-by-side (both for when the heterospecific species shown shared the predominant classification feature with the test species and when it did not). All monkeys were also exposed to an experimentally manipulated image of a conspecific species but for which only the predominant classification feature was switched to that from a dissimilar species side-by-side with an unaltered image of a heterospecific species but that shares the predominant classification feature with the test species.

In putty-nosed monkeys, the predicted highly discriminant nose-spot feature significantly explained gazing time, even in the manipulated conspecific vs. unmanipulated heterospecific experiment. The opposite was true for mona monkeys (white eyebrow patch feature), for whom the classifier had predicted a lower level of reliance on the predominant feature compared to putty-nosed monkeys. The study design used (including various condition randomizations) and statistical analyses were appropriate.

This study represents an outstanding experimental confirmation of an automated visual species recognition prediction for a species with a highly discriminant feature (putty-nosed monkeys), along with the mona monkey result in contrast demonstrating the combinatorial importance of multiple facial features (including when the otherwise top-ranked discriminant feature for that species is manipulated artificially!), highlighting species recognition system variability.

Essential revisions:

1) The authors should run the manipulated images that were used for their experiments back through the classifier (i.e. the approach used to generate the images for the experiments is different than the blurring approach used in the machine learning process) for confirmation that the level of uncertainty introduced matched the experimental results.

2) I was surprised that some key information on the monkeys included in the experiment was only available in the supplementary information. I had noted a number of larger concerns while reading the paper (e.g. potential lack of history of exposure to heterospecifics) that were then mostly alleviated when I explored the supplementary information for answers, and I expect the experience might be the same for future readers. Therefore I would encourage this information to be brought into the main manuscript Introduction/Results and Materials and methods.

2B) Were any of the sanctuary monkeys born in the wild (and thus potentially experienced heterospecific interactions outside of the caged sanctuary environment?). If so and if this information is known, is there a significant difference with captive-born individuals? Should this variable be included in the analytical model?

Reviewer #2:

This study addresses the role of facial features in species recognition in guenons. It includes two parts. In the first part, the authors used a computer vision technique (machine classification with partial occlusion applied onto face images encoded by eigenface scores) to identify which facial features influence species classification most. In the second part, they designed a behavioural experiment with two guenon species to test whether key features identified in the first part influence looking time in conspecifics.

This is the first study that uses classification+occlusion to identify potentially relevant features. This is a great methodological step forward in visual ecology. However, I feel that neither the classification part nor the behavioural experiments are appropriately designed to test the proposed hypotheses, and thus I am not sure that results do support conclusions. More specifically, the author's central statement that their "results objectively identify the signal components most likely to be useful to guenon receivers" may be incorrect, for two main reasons.

First, their machine classification, as any artificial or biological classification process, is not objective. Any classification or recognition task is inherently contextual. If a classifier, be it artificial or biological, was instructed to recognize red Ferrari among yellow Lamborghini and blue Hummer, it would not rely on the same cues as if it was instructed to recognize the same Ferrari among red Lamborghini and red Hummer. Thus, which features are keys for recognition critically depends on which stimuli need to be classified. In this manuscript, I am puzzled by the selection of species included in the classification analysis. For example, it does not seem relevant to include, in the same classification task, Miopithecus, Erythrocebus patas, Cercopithecus ascanius and C. neglectus. Males of Miopithecus weight 1.3 kgs, while patas males weight around 12 kgs. Moreover, these two species live in large monospecific groups, not in polyspecific groups that include Cercopithecus spp.. Cercopithecus ascanius is arboreal, while neglectus is semi-terestrial. Actually, mixed-species groups concern mostly arboreal species; with groups often made of one species of each Cercopithecus 'super-species' complex cephus, ascanius and nictitans (sometimes intermittently further including colobus mangabeys and mangabeys). If the authors want to investigate which features matter for species recognition, they first need to clearly define the perceptual task/challenge that each species faces in the wild, thus considering known species associations, the ecology of species and a minimum of morphological data (to avoid pooling together species which weigh differs by factor 10!). The 2011 article by Jaffe and Isbell, cited in the manuscript, and some older contributions by field primatologists Annie Gautier-Hion, Marc Colyn and others would be very useful. If the authors do not want to do such an analysis, at least they should demonstrate that results (their identification of key features) are robust to the selection of species included in the classifier. But putting all guenon species into a single classifier is for sure biologically irrelevant, and so appear the results.

Second, and more importantly, it is unclear to me how the behavioural experiment can support any conclusion about species recognition. In humans, object or face recognition is achieved during the first few hundred milliseconds (e.g., Cichy, Pantazis and Oliva, 2014. Resolving human object recognition in space and time. Nature neuroscience / Hsiao and Cottrell (2008). Two fixations suffice in face recognition. Psychological science). Most likely other old-world primates also achieve conspecific recognition within a few milliseconds. Eventually, how a difference of visual attention measured during a viewing period lasting several seconds is related to recognition? Results of this study show that the white nose of C. nictitans caries information that is relevant for this species specifically or, as the authors wrote, "that longer looking time at a particular face reflects level of interest". But this study does not bring any evidence that this information is about species identity. The authors cite Mendelson and Shaw's review, which argues that species recognition traits are not different from other sexual signals. This is may true in some model species, yet, 1) it may be beneficial to advertise conspecific identities for interactions that are not sexual; 2) at least in one old world primate, Homo sapiens, it has been shown that face detection, recognition and evaluations correspond to different perceptual domains, i.e. they have different neural bases (e.g. Tsao and Livingstone, 2009, Mechanisms of face perception. Annual Review of Neuroscience / Zhen, Fang and Liu, 2013. The hierarchical brain network for face recognition. PloS one, 8(3), e59886.); 3) these different neural bases likely translate into different selective forces on signals and, accordingly, recognition traits undergo displacement, while all sexual signals do not. Recognition traits are special, distinct from classical sexual signals, and evidence that a trait is sexually or socially important is no evidence that this trait is relevant for recognition.

Moreover, a match between results from the machine classification and the behavioural experiment does not prove the relevance of the behavioural experiment for identifying recognition traits. I am pretty certain that a saliency analysis (e.g. using saliency maps) would have revealed that the nictitans' white nose and mona's eyebrow patches are salient. Saliency may be a property of discriminative traits (thought not necessarily, especially in species that use long-distance species-specific signals to communicate, like calls – see the various studies by Jean-Pierre Gautier on the calls of guenons and their use in mixed-specie groups), but it can also be a property of sexual signals that are not recognition traits (ie not under selection for displacement), of traits signalling kinship, individual identity etc… (though, again, even this result would barely support any conclusion, since saliency influences bottom-up selective attention, which cannot be investigated by measuring looking times lasting 3 or 4 seconds).

Reviewer #3:

With great interest I have reviewed the manuscript titled "The structure of species discrimination signals across a primate radiation". Here the authors present a clever approach, not only identifying unique species-specific features via machine-learning, but also determining whether these features are perceived by two of the species within the radiation. The question of how signals may guide character displacement and species recognition is fundamentally important to our understanding of behavior and evolution. Tackling this question with sophisticated analytical tools and behavioral tests (without operant conditioning) in monkeys while acquiring appropriate sample sizes is impressive. And importantly, the manuscript is very well written and provides a thorough background of the literature/theory and discussion of the results.

The authors interpret that longer looking times for a particular face as increased interest, which seems justifiable. And the three experimental trials in each species also provide an adequate test of the effects of the selected features on attention. I do wish there were additional experimental for a negative control, such that other areas of the face were manipulated to see if attention was affected. However, given the limitations of the study system, I completely understand.

I do have some concerns about how the data were analyzed and presented:

It is unclear to me why a binomial family distribution was used when the "number of video frames" is the dependent variable. Binomial models assume a binary response variable (e.g. 0 or 1). Although a binary choice is provided here for each trial, the data analyzed is count data (duration). Thus, a negative binomial distribution (or Poisson) would be appropriate. It appears to me that this could dramatically impact the statistical interpretation. Furthermore, when testing out the code and data reported here, the models report a "Singular fit", which indicates that the random effects may be too complex for the data provided.

Furthermore, I do not understand the choice to analyze all three trials data for each species together given that stimuli were presented in pairs. It is my understanding that if a subject is looking at 1 image during the five seconds window it cannot be looking at the other image – thus making each data point non-independent (and only relevant to the trial). Wouldn't it be easier for interpretation to just analyze each trial separately? For example, the Results section describes differences in con- vs hetero-specific looking times; yet only in trial 1 was this comparison between natural images of each species (e.g. I wouldn't predict a difference in con vs het in Trial 2).

Reviewer #4:

This is a very exciting paper that provides convincing evidence that guenon faces are a way to avoid hybridization. It combined careful analyses of faces with very interesting behavioral titration of attention by 2 of the guenon species.