One of eLife’s founding principles is that “decisions about the fate of submitted papers should be fair, constructive, and provided in a timely manner” (Launching eLife, Part 2), and our review process has been designed to support this.
By engaging the reviewers in an open discussion, we encourage them to be constructive in their initial reviews, as they will need to be able to stand by their comments. We also publish the most substantive points from the decision letter after review, along with the reviewers’ names, where they allow.
Of course, peer review in the life sciences and biomedicine takes many forms, with various degrees of transparency. At one end of the spectrum, there’s open peer review, where the reviewers’ and authors’ identities are known to one another. Some journals go further and publish the referee reports. The BMJ and BMJ Open are well-known proponents of open review.
At the other end of the spectrum, there’s double-blind peer review, where the author names (and other potentially identifying information) are hidden from the reviewers and the reviewers’ identities are not revealed to the authors. This form of review is quite common in the social sciences and humanities, but it’s not commonly used in the life sciences. That said, the Nature journals offer double-blind peer review as an option, and eNeuro uses double-blind review as its default option.
The EMBO Journal has pioneered referee ‘cross-commenting’, such that referees can comment on the other reviews before an editorial decision is reached and the (unattributed) referee reports are also published along with the article.
Other forms of review, such as post-publication peer review, are also emerging, in forums such as PubPeer, while preprints allow authors to collect community feedback often before or during a formal review process at a journal. There are also journals that combine formal peer review with simultaneous public discussion (where any interested members of the scientific community can comment, so long as they’re named), as pioneered by Atmospheric Chemistry and Physics.
However, the most common form of review in the life sciences is single-blind review, where the reviewers know who the authors are but the reviewer identities are not typically revealed to the authors (although many journals optionally allow reviewers to share their names). Although it’s based upon single blind review, eLife has adjusted this process in a way that aims to increase “fairness, speed and transparency” (The eLife approach to peer review).
An important and unique feature of eLife’s consultative peer-review model is that the reviewers discuss their comments openly with each other and the Reviewing Editor, with the aim of reaching a decision they can all agree upon. To ensure that the reviews are submitted independently, the reviewers cannot see the other reviews (and the names of the other reviewers) until they have submitted their own report. Once the reports have been submitted, and if the reviewers support a resubmission, it’s the role of the Reviewing Editor to resolve any differences in the major comments through consultation with the reviewers, so that the decision letter sent to the author only includes the essential revision requirements.
By revealing the reviewers to one another, eLife’s approach to peer review is more transparent than conventional single-blind peer review and in general helps to facilitate a constructive experience for authors and reviewers. However, the majority of reviewers at eLife decide to remain anonymous to the authors. In the interests of being constructive and fair, we wanted to explore whether we could increase the transparency of the review process.
In eLife’s earliest years, 23% of reviewers agreed to share their name with the authors, but we wondered whether the way that we presented this option was sufficiently encouraging. We therefore tested some new wording in a controlled experiment. Group A (n=288) was shown the original wording: “Would you like to have your identity revealed to the authors in the decision letter after peer review? If 'Yes' and if the article is accepted, please note that your name will also be included in the published decision letter.” Group B (n=288) was shown more encouraging wording: “We would like to encourage you to share your identity with the authors within the decision letter sent to the authors in the interests of transparency. If the article is published by eLife, you would then be credited as one of the reviewers within the accompanying decision letter.”
In group A, 58 reviewers agreed to share their name (20%), while in group B 95 reviewers agreed to share their name (33%). Those shown the new wording were more likely to agree to sharing their name by 13 percentage points (95% CI 5%–20%). These results provide evidence of a strong positive effect of the new wording (p=0.0007), which we have since adopted for all reviewers since November 2015.
In the graph below, the proportion of reviewers willing to share their name with authors who have submitted to eLife is shown over time. The red arrow indicates when the new wording was introduced for all reviewers.
From December 2015 to July 2016, 30–36% of reviewers have agreed to share their name with the authors. For the three months from November 2015 to January 2016, 27% of reviewers agreed to share their name for papers rejected after peer review, while 42% of reviewers agreed to share their name with the authors for papers that were revised. As a result, readers will see more published decision letters that reveal the names of the reviewers (for example, as shown in this published paper, 'Single-fluorophore membrane transport activity sensors with dual-emission read-out').
Because we think that openness and transparency helps to encourage more constructive behaviours, we will be looking further into the effects of being more open in the peer-review process. For example, we plan to look at the proportion of reviewers who are now using tools such as Publons as a way to gain credit for their peer-review contributions, and we will continue to report our results.
