A reviewers' charter

Inside eLife
  • Views 252
  • Annotations

We were recently approached by two colleagues about an issue that is close to the heart of eLife – namely how to make the editorial process a fair, swift and constructive experience. At eLife, we have devised an editorial process that is founded on these aims, encompassing the roles of author, reviewer and editor. Mariann Bienz and Kathy Weston are proposing a ‘Reviewers’ Charter’ to focus the minds of researchers specifically on the role of reviewers. eLife supports the principles that Kathy and Mariann are putting forward, and we invite comment and further discussion of the idea -- through comments on this page, through Twitter (#reviewerscharter), or via email to staff [at] elifesciences [dot] org .

A guest post by Mariann Bienz and Kathy Weston

The decision by the Wellcome Trust, the Howard Hughes Medical Institute and the Max Planck Society to throw their weight behind reform of the current journals system, where researchers have to pay for the content that they’ve contributed, reviewed and edited for free, is great news for the academic community; perhaps it does herald an “academic spring”, as many hope. Perhaps also, this would be a good time to start a new debate about that other bane of the modern scientist’s life: the length of time it takes to get a paper published, and the emotional, intellectual and career damage that authors sometimes sustain in the process. It would be marvelous if we lived in an open access world in which papers were properly and fairly reviewed and published in the shortest possible time.

If the fight for open access, a battle pitched against considerable odds, now seems winnable, surely we can reform the reviewing system as well? It shouldn’t even be a fight – the protagonists are all on the same side, as all reviewers are authors, too.

Of course, the imperfection of peer review is an issue that has been exercising scientific minds for many years now (for some recent opinions, see below), but it is worth setting out again the two main problems with the currently fairly lousy process. Firstly, despite electronic publishing, the gap between submission and publication of a paper is longer than ever, because papers are kept too long in review. Delaying publication delays scientific progress, hinders careers and is poor value for money. Secondly, the delays are frequently due to reviewers asking for a long list of additional experiments. Reviewers often come to a paper with the attitude ‘what else could be done?’ rather than ‘what is essential to support the claims in the paper?’.

We propose a new solution to the problem; instead of taking revenge for past authorial suffering by writing reviews like those described above, what if reviewers were motivated by a combination of altruism, common sense and self interest (so that their future experience as an author might be similarly positive)?

If everybody agreed to be constructive and unbiased, and to reach a basic yes-no decision on the data and conclusions presented, papers would move faster, and, in turn, so would research.

In cases where more experiments really were needed to support the conclusions, reviewers could limit their requests to one or two essential ones, rather than the plethora of supplements frequently demanded now.

As a focus for making the change to universal fair and fast review, we suggest the creation of a Reviewers’ Charter, a sort of scientific Hippocratic Oath, where scientists could publicly declare their intention to review papers as fairly and rapidly as possible; in other words, to treat all papers they are asked to review as they would wish their own papers to be treated. By signing up to this charter, scientists would agree to adhere to a set of easy-to-follow and easy-to-absorb guidelines, and to act as ambassadors to spread these principles amongst their colleagues, especially their junior ones who might still be new to the game.

Below is a set of preliminary charter points, to serve as a basis for further discussion. Signatories to the charter would agree to review according to the following guidelines:

  1. To determine whether the data support the conclusions of the paper, and if they do, to facilitate publication of the paper as rapidly as possible, i.e. decide whether a yes-no decision is appropriate, or whether a yes-if decision is necessary.
  2. To request further experiments only as a last resort, and only if they are essential to validate the conclusions of the paper. No experiments extending the study beyond its conclusions, or with unreasonable cost or time implications, should be proposed. An estimate of time required for the additional work should be provided.
  3. To review papers within two weeks, and to decline to review at once if this is not possible.
  4. If asked to re-review a paper, to assess the revision solely on the criteria specified in the initial reviews.
  5. To request the editor’s permission before asking a lab member to review a manuscript, to state the involvement of the lab member in the review, and to ensure adherence to the charter guidelines (the enthusiastic dismembering of a paper is fine for journal club but not for the peer review process).
  6. To undertake to actively campaign for journals to introduce a policy of cross-reviewing – asking reviewers to comment on each other’s assessments of the paper in order to counteract any personal bias, misinterpretation of the data, or non-adherence to the reviewers’ charter.

If the Charter is successful, it could become a stipulation of funding bodies that grant holders should sign up for it. Journals could use the Charter list to select reviewers, and there could also be monitoring of recusants. Being a Charter signatory could become an essential part of being a practising scientist. It would be a big change, but maybe the time is ripe. The fresh green shoots of the academic spring, or another crop of stinging nettles? The choice is ours.

Mariann Bienz (MRC Laboratory of Molecular Biology, Cambridge, UK) and Kathy Weston (Freelance Science Writer and Consulting Editor,Disease Models and Mechanisms)

Acknowledgments

The authors would like to acknowledge the numerous colleagues who have helped to crystallise the idea of a peer review 'charter'. MB particularly thanks James Nelson (Stanford University, USA), for a very fruitful email conversation on the subject.

Further reading

Siegel V (2008). The promise of peer review.Dis Model Mech1,73-77

Raff M, Johnson A, Walter P (2008). Painful publishing.Science321, 36

Ploegh H (2011). End the wasteful tyranny of reviewer experiments.Nature472, 391

Petsko GA (2011). The one new journal we might actually need.Genome Biology12, 129