In the first of two Q&A articles to mark Peer Review Week 2016, we at eLife and OASPA asked two experienced reviewers and two early-career researchers for their views on recognition and innovation in peer review. The reviewers included Suzanne Pfeffer (SP), from eLife, and a contributor from Ubiquity Press (UP). Alecia Carter (AC) from eLife’s Early Career Advisory Group - a postdoctoral research fellow for over four years and semi-independent researcher for three years - and Hannes Müller Schmied (HMS) - an early-career researcher and reviewer for Copernicus Publications - also provided their thoughts. This is what they had to say.
On recognition in review
Do you think it is important to be recognised for your work in peer review? If so, why?
AC: Yes. Reviewing is an essential part of the publishing process and I feel that we are obliged to be a part of the academic community, like with mentoring and supervising. As a rule, I try to give as much as I take in these areas. Recognition is important to keep reviews constructive and fair.
UP: Yes, it is important for the time spent, the expertise, and the effort put into a constructive peer review to be recognised, because it is a very important contribution to knowledge and society.
SP: When I serve as an editor of papers in certain fields, I am often surprised how many potential reviewers decline invitations to review. As authors who ask others to take time to review our papers, it is important that we all do our part to support the review process. I think it is nice to be recognised. Right now, my recognition comes from colleagues who have appreciated my willingness to help the overall review process. They count on me to do that quickly and I try my best to help out whenever possible.
Do you think enough is being done to ensure recognition in peer review? What more, if anything, do you think could be done to improve this?
UP: Nowhere close to enough. Time spent [on peer review] should be considered as working hours by our employing institutions, and listing the work we have reviewed (not only in terms of quantity but also the types and significance of papers/books) should be considered in our professional development.
SP: Publons is a new venture that is trying to do this; time will tell how successful it is. Good reviewers are often asked to serve as editors, and when editors help authors, they are certainly appreciated by those colleagues.
AC: No, not enough is being done. Publons is attempting to credit reviewers for reviewing, but as this is an opt-in system, the sample is biased through self-selection - it's not representative of the quality and numbers of reviews done by the average academic. Transparent reviews next to published articles is another great initiative of some journals, such as eLife and PeerJ. Community service should be recognised more generally. Editors could rate the quality, thoroughness, and constructiveness of reviews, and this could be public.
Organisations such as Publons and ORCID iD allow reviewers to build a profile of their contributions to peer review. What, if anything, do you think might be next in terms of increasing recognition in peer review?
UP: Opening up peer review after publication and allowing reviewers to be credited will motivate them to take up more peer-review work and conduct it in a more constructive, responsible and accountable manner for the benefit of all parties involved.
HMS: The editor of a journal could perhaps give scores or grades for reviews in terms of objective criteria. The best-scored reviewers could get a voucher for publishing an article or can benefit in some other way, for example through a ranking. (This is just an idea, however; I don’t know if it would work out at all.)
AC: Rating reviews (by editors, authors and the community) could increase the recognition of (good) peer review.
On innovation in review
There are a number of innovations in peer review, such as the transparency provided by F1000Research, the ability to provide public comment through preprint servers, and more. Have you personally been involved in any such innovative peer-review processes? If so, what did you enjoy most about them?
AC: Yes, I put my reviews on Publons, have some reviews of my own papers on Publons, and have published several times in journals with transparent peer review (i.e. where the reviews are published alongside the paper). I enjoy the transparent review process the most. It allows more information about the study (particularly the justification for its design) to be shared, and gives early-career researchers insight into the review process and also into limitations or confounds of particular approaches.
HMS: Yes, with a couple of Copernicus journals, namely Hydrology and Earth System Sciences and Earth System Dynamics. I really enjoyed the open-review system and the fact that the reviews, as well as the author responses, are openly available. It increases the carefulness of reviews.
What, if anything, do you think might be next in terms of new innovations in peer review?
UP: That the humanities and social sciences follow the innovations of the hard sciences.
HMS: An increase in motivation for reviewers to provide high-quality reviews in time, for example through vouchers or other ways to say “thank you”.
AC: Feedback on reviews would be helpful, as would more (post-publication) reviews of papers. Peer review is necessary to ensure the quality of published science and the interpretation of findings; having a greater proportion of the community involved in a dialogue about published science would be constructive and perhaps move fields forward more quickly.
I think it’s also important to note professional reviewers as a reviewing innovation. I feel that professional reviewers would recognise the importance and value of constructive, critical feedback on scientific work. They would have a much better overview of a current field from performing many reviews of (recent) research in that field, and would be better able to give unbiased feedback as, presumably, they wouldn't be motivated by pushing their own rhetoric or by being scooped by another lab.
