Making peer review constructive and collaborative is an important goal for eLife. As a result, we study issues facing peer review and provide data that can inform our own (and other journals’) editorial policies. Recently, two eLife Reviewing Editors, Jennifer Raymond and Cassidy Sugimoto, collaborated with colleagues and journal staff to explore the subject of implicit bias in peer review. The findings by Dakota Murray et al. were posted last month as a preprint on bioRxiv. Code and anonymized data have also been made available on GitHub.
Based on data on the outcomes of over 23,000 submissions, the preprint reports higher success rates for male versus female senior authors, and for authors based in the United States and Europe versus those based elsewhere. For initial submissions, the encouragement rate was slightly higher for male corresponding authors compared to female corresponding authors (30.6% versus 28.6%). After in-depth peer review, the proportion that went on to be accepted was also higher for male corresponding authors (53.4% versus 50.4%). Overall, the accept rates were 15.4% for male corresponding authors and 13.6% for female corresponding authors. Initial submissions with a corresponding author from the United States were most likely to be encouraged for in-depth review, with 39.2% encouraged, while submissions with a corresponding author from China were the least likely to be encouraged, with 12.6% encouraged.
There are likely to be a variety of contributing factors to these differences in decision outcomes. We also believe that the issues raised here do not only affect the review process at eLife, but that they also affect many other journals. However, it is worth noting that these effects varied according to the demographics of the reviewer team: male senior authors had a higher success rate than their female counterparts when reviewed by an all-male reviewer panel (56.8% versus 50.4%); when the reviewer panel was mixed gender, the gender difference was not statistically significant, and male and female senior authors had similar success rates. A similar "homophilic" effect was observed for nationality: acceptance rates were higher when at least one reviewer was from the same country as the senior author. This indicates that the outcome of the peer-review process is influenced not only by the quality of the research, but also in part by socio-demographic characteristics of the editors, reviewers, and authors. These findings raise important questions around the peer-review process, not least because such effects have the potential to perpetuate socio-demographic disparities in science and medicine more broadly.
The preprint has been submitted for formal peer review and will likely undergo revisions. However, given the importance of the findings, we are taking immediate steps to improve our peer-review process.
First, we have shared the results of the preprint with our editorial board, and encouraged the editors to use a diverse group of reviewers whenever possible.
Second, we will continue to take steps to enhance the diversity of the editorial board. When recruiting editors over the last few years, we have placed a greater emphasis on diversity, for example by increasing the gender balance and international representation of the editorial board, and by recruiting early-career researchers. We will continue these efforts and monitor our progress in improving the diversity of the editorial board.
Third, we will continue to conduct research on peer review, and will experiment with modifications to our peer-review process that focus on the ideal of fair and constructive review. The preprint adds to past research into our review process, and we are now initiating collaborations to analyze the effects of the reviewer consultation session. Furthermore, we recently announced a trial process in which the authors decide how to respond to the issues raised during peer review, and we will release some early results of this trial as soon as we can. We also have another trial underway in which a small number of Reviewing Editors have committed to use an early-career researcher within the review process for every paper they handle. We will use all such data, as well as input from editors and authors, to refine and improve the peer-review process we use at eLife.
The Senior Editors will continue to discuss changes that can be taken to respond to the findings of this preprint, and we welcome suggestions from the scientific community.
In addition, it will be important for other journals to release data on their review processes, including author and reviewer demographics. This will mean that different forms of review can be evaluated more effectively and compared with one another, and it will allow the scientific community to work on ways of eliminating biases during peer review.
---
Questions and comments are welcome. Please annotate publicly on the article or contact us at hello [at] elifesciences [dot] org.
For the latest updates on editorial policy and other news from eLife, sign up to receive our bi-monthly newsletter. You can also follow @eLife on Twitter.
