Research: Gender bias in scholarly peer review
- Max Planck Institute for Dynamics and Self-Organization, Germany
- Bernstein Center for Computational Neuroscience, Germany
- Yale University, United States
- Aix-Marseille University, France
Figures
Figure 1 with 2 supplements
Women review and author even less articles than expected from their numeric underrepresentation.
(a1) We represent peer-reviewing interactions as directed graphs, in which vertices denote scientists. In the editor-to-reviewer network every edge represents the act of an editor (source vertex) appointing a reviewer (target vertex) to review a manuscript (and the reviewer has accepted the invitation). Analogously, in the reviewer-to-author network edges represent a reviewer reviewing a manuscript of an author. (b) The development of the fraction of contributions for each gender are shown for editors, reviewers and authors. Since the start of the Frontiers journals in 2007 until 2015, women (circles) edit, review and author much less than 50% of manuscripts, as expected from their numeric underrepresentation. However, the actual numbers of reviewing and authoring contributions by women are even smaller than expected by chance, taking into account their numeric underrepresentation. This is revealed by comparison with a null hypothesis in which gender and number of contributions are assumed to be independent. To this end, we generated surrogate ensembles by shuffling the genders of scientists appearing in a given role in the network (a2). From the surrogate ensembles, we obtained 95% confidence intervals (CIs; shaded areas in b). *, **, *** over (under) the data symbols denote the data lying over (under) the 95%, 99%, 99.9% CIs. Note that for all three subnetworks, there is a noticeable, but extremely slow trend towards equity (dashed line) for the fraction of contributions. (c) The fraction of female contributors, ranked in increasing order of authoring contributions, for the 47 frontier journals, whose published articles were handled by at least 25 distinct editors. Women were underrepresented consistently across all fields and particularly severely in math-intensive disciplines.
Figure 1—figure supplement 1
Analysis of network topology.
(a) Number of articles (blue), as well as number of people (green) in the reviewer-to-author (left) and editor-to-reviewer (right) network grow approximately exponentially over the years. (b) At the same time, network connectivity also steadily gets denser, giving rise to a growing weak graph giant component (that is, the biggest connected component in the network, irrespective of edge directions) of socially interacting editors, reviewers and authors. The cluster sizes are normalized with the network sizes at each time point. The latest reviewer-to-author (editor-to-reviewer) network sampled at the cutoff date has a giant component reaching 95% (93%) of its global size. (c) The small-world-ness index (see methods) suggests that a small-world topology is robustly established. (d) Global undirected transitivity (see methods), (e) as well as average shortest path lengths (see methods) have stabilized in the 3-5 last considered years, despite the fact that the network keeps growing at an exponential pace.
Figure 1—figure supplement 2
Gender disparities vary between countries.
Based on scientists' affiliations, we plot the fractions of (a) editors, (b) reviewers and (c) authors in each country. In case of multiple affiliations we counted one contribution for each distinct country. Dark gray color indicates a fraction of female contributions aligned to the overall median value. Countries providing less than seven contributions are excluded from this analysis (white color).
Figure 2 with 1 supplement
Women are underrepresented in the fat tail of contributions.
A break-down of the number of individuals contributing a given number of times as editors, reviewers and authors (binned, x-axis is marking the bin edges) shows that the majority of scientists (a) edited, (b) reviewed or (c) authored (corresponding zooms for small contribution numbers are shown in e-f) only a small number of manuscripts. Chance levels (shaded) were derived from an ensemble of reference networks constructed as shown in Figure 1a. The underrepresentation of women in relation to these chance levels tends to increase towards the fat tail of the distribution, associated to the relatively few individuals that made many contributions. In the group of one-time authors or reviewers however, women are overrepresented. Time resolved distributions are shown in Figure 2—figure supplement 1.
Figure 2—figure supplement 1
Time- and gender-resolved histograms of the number of contributions.
For each year we calculate the histograms of contributions for men (a–c) and (e–f) women (see Figure 2 for the year 2015) and color code the normalized deviations from the expected median (i.e. the median of the shaded area in Figure 2). Bins for which less than 7 contributions were available in a given year are shown in white. In the course of time the network grows (Figure 1—figure supplement 1) and higher contribution numbers become available, but the overall pattern of deviations from the expectation remains significant for reviewers (b,e) and authors (c,f). The tendency towards equity is weaker at the distribution’s right tail. Data for editors (a,d) is more variable.
Figure 3
Editors have a same-gender preference for appointing reviewers.
(a) Female editors (orange) appoint significantly more female reviewers than expected under the gender-blind assumption (shaded area). At the same time, male editors (green) appoint less women than expected. The development of this trend over time is shown, including articles cumulatively until the indicated year. (b) Likewise, female/male reviewers review significantly more female/male-authored articles than expected. (c) Homophily is widespread across scientific fields, including those with relatively mild underrepresentation of women. We here report four example disciplinary groupings, with large numbers of contributions (from left to right, respectively, 13416, 4721, 4020, 5680) and the propensity of appointing a female reviewer depending on the editor's gender for each of these groupings. Only assignments by female neuroscience editors were not homophilic, otherwise the occurrence of same-gender preferences was general, arguing against heterogeneity between subfields as a cause for homophily in assignments. (d) Plotted here are distributions of a measure of inbreeding homophily. To control for baseline homophily at the level of a narrow local neighborhood, we measure, for each editor node, the actual number of reviewer assignments given to women and subtract the expected number, which would be observed if the considered editor appointed women with the same frequency as in his/her local vicinity. For male editors (green) the distribution is skewed towards an underrepresentation of female assignments (left-leaning), while for female editors the distribution is skewed towards an overrepresentation of female assignments. This highlights that homophily bias is detectable even at the level of the reachable narrow surrounding of each editor. (e) Histogram of the probability that an editor assigns as least as many reviews to people of the same gender as he/she actually does reveals that there's an excess of strongly inbreeding-homophilic editors (small Φhom -values) among both men and women compared to expectation (shaded area). Note that below Φhom < 0.1 there are only few strongly homophilic female editors. For male editors, significant homophily extends through many more editors until Φhom < 0.6. (f) Using all data until 2015, the probability that a women is appointed is above expectation (shaded areas) only for female editors and only when all or all but the most extremely inbreeding-homophilic editors are included in the analysis.
Additional files
-
Supplementary file 1
Comparison of female author contributions by country between Larivière et al. (2013) and the Frontiers series of journals.
- https://doi.org/10.7554/eLife.21718.008
-
Supplementary file 2
Reported fractions of female authors, reviewers and editors in previous studies.
- https://doi.org/10.7554/eLife.21718.009
-
Supplementary file 3
Network data.
Contains two files, one (graph_nodes.csv) giving a random ID per person in the network together with that person’s gender, the other (graph_edges.csv) indicating which IDs interact together with the roles of each ID (a: author, r: reviewer, e: editor).
- https://doi.org/10.7554/eLife.21718.010
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Research: Gender bias in scholarly peer review
eLife 6:e21718.
https://doi.org/10.7554/eLife.21718
