Many researchers have strong views on peer review. To find out what early-career researchers think we conducted a survey in which we asked 10 questions about different aspects of peer review. A total of 264 researchers took part in the survey, including 146 postdoctoral researchers (55% of the total), 61 group leaders (23%) and 51 PhD students (19%). The results are summarised below. Read more about peer review in this collection of articles.
Most of the people (92%) who responded to the survey had some reviewing experience. Researchers at later career stages were more likely to have reviewed, to the extent that all of the group leaders who took part in the survey had reviewed before. At the earliest career stages, researchers commonly worked with their advisor to review papers, although 37% of PhD students still performed their review without the assistance of their advisor.
To assess the knowledge of the participants, the survey asked several questions about what reviewers should look for when assessing a paper. In general, researchers felt that the role of peer reviewers is to check the underlying science of the paper and to ensure that this is clearly described. Reviewers generally felt that journal staff should catch minor issues like typos, but some were concerned that this copy-editing stage is not always thoroughly performed.
Apart from the options presented in the question, respondents also mentioned a number of other issues that they look out for when reviewing a paper. These included: checking for incomplete or manipulated data; checking that the experiments described have been reproduced sufficiently with enough replicates; availability of data and code; checking that appropriate statistics and experimental designs have been used; citations; and ethical issues.
Almost all (96%) of the survey participants believed that it is justified to ask researchers to perform extra experiments to support unjustified claims in the paper. Even then, several people commented that asking for extra experiments is a last resort, preferring instead to ask authors to tone down their claims.
Several commenters were also displeased with reviewers who ask for more experiments to cover their own interests. Despite this, 32% of survey respondents thought that it was fine to ask for more experiments to investigate a consequence of the findings that the authors hadn’t thought of. It is worth noting that the ethical guidelines for peer reviewers published by the Committee Of Publication Ethics (COPE) state that “it is not the job of the reviewer to extend the work beyond its current scope”.
The aspect of peer review that the highest proportion of survey participants – regardless of their career stage – found difficult was evaluating statistics.
Most of the respondents had learned to peer review by following advice from their advisor or by learning from the example of reviews that they had received. However, several commenters stated that that the main thing they learnt from other reviews was “what not to do”, and they tried to avoid making those same mistakes themselves.
When asked about different forms of support and training, the early-career researchers taking part in the survey showed a strong preference for working with their advisor on to review real papers.
The survey was conducted in September, 2017 on behalf of the eLife Early-Career Advisory Group. The group is particularly interested in opportunities for early-career researchers to gain experience in – and credit for – peer review and have helped to drive efforts at eLife including the early-career reviewer pool and adoption of CRediT.
The results will inform the ECAG’s future efforts.
Interested in finding out more about opportunities, events and issues that are important for early-career researchers? Sign up to the eLife Early-Career Community newsletter or follow @eLifeCommunity on Twitter.
