PREreview — a new resource for the collaborative review of preprints

Journal clubs are a staple in academia. They are an opportunity for researchers to sit together and discuss what is new in science. Ironically, because journal clubs typically use published papers, the research is usually already old.

If you want to be one year behind, don’t read bioRxiv

– Jeff Leek

Journal clubs provide a valuable opportunity for researchers to discuss other scientists’ work and learn about discoveries in their field. But it can be frustrating when those detailed critiques remain within the walls of the journal club and are unlikely to reach the authors or the wider community.

Preprints provide a wealth of data that is current, accessible by all, and at a stage where community peer review can push science forward.

Reviewing preprints benefits the authors, as they receive early feedback on their manuscript. Early-career researchers benefit from the opportunity to develop their critical thinking and peer-review skills. And the whole scientific community benefits from access to scientists’ discussions about the latest discoveries.

We envision a process in which researchers collaboratively write their manuscripts online, submit them to a preprint server, request open feedback from the community, improve their manuscript, and re-post a new preprint version or submit to a journal for editorial peer review. We hope it will become standard practice for researchers to post their work on preprint servers, encouraged by the useful feedback they receive from other researchers reviewing their manuscript.

Our goal is to promote preprints and discussions about preprints by facilitating the initiation of preprint journal clubs at research institutions around the world. Let’s break down the walls of the traditional journal club by stimulating a discussion between the author and the wider scientific community. To this end, we have developed resources and recently launched PREreview.

PREreview is a web platform for hosting preprint reviews and sharing them openly with everyone. It features a public or private collaborative space to share the fruits of your journal club’s discussions. For each of our reviews, you may issue a digital object identifier (DOI), which allows your reviews to be easily cited, indexed by search engines, and/or promoted on social media.

PREreview has been built using Authorea’s technical infrastructure. We share Authorea’s vision of streamlining the whole process of publishing and we are excited by the recent collaboration between Authorea and bioRxiv that brings us one step closer to realising that vision.

With PREreview we hope to encourage other scientists to start their own preprint journal club – either as a class for students or by integrating preprints at traditional journal clubs. The idea of using preprint journal clubs to formalize peer-review training for early-career scientists came from the work of Dr. Prachee Avasthi, Assistant Professor at the University of Kansas Medical Center, and member of the eLife Early-Career Advisory Group. In the Fall of 2016, Dr. Avasthi led a course called “Analysis of Scientific Papers”, in which students critically evaluated preprints and made their reviews available to the authors and wider community.

Source: video attributed to ASAPbio and reused under the Creative Commons Attribution License.

It was a very positive experience for the authors and also for the students who had the opportunity to interact with scientists on very cutting edge work!

– Prachee Avasthi

Soon, we expect PREreview to support personalized group pages for your journal clubs, include a function to recommend reviews written by other groups on the same preprint, and provide an option for authors to solicit reviews via PREreview. We are also working with bioRxiv to enable automatic trackbacks so that your reviews are discoverable from the preprint server itself.

As a first step, try presenting a preprint at a journal club. To help you find a preprint, we have listed the main hosting servers for preprints in biological sciences. For other disciplines, check out the Wikipedia list of preprint servers or this list of preprint servers by Martyn Rittman.

If your audience is new to preprints, you could start by introducing the concept using our slide deck (long form or short form).

Once you’ve discussed a preprint at a journal club, we encourage you to write a review. Sign up for free at PREreview.org and start a new document:

Video credit: Authorea, used with permission.

Our step-by-step guide and peer-review guidelines will help you through the process.

When you present and review a preprint at a journal club, tell us! Tweet @PREreview_ using the #preprintjc hashtag.

To encourage more preprint reviewing at your institute, you could set up your own preprint journal club. We have written two email templates that you can adapt to invite your colleagues to join.

PREreview is only one of the spaces in which you can interact with preprints and contribute to making science more transparent and collaborative. Other platforms and tools include:

Hypothes.is: an online annotation tool for layered preprint commenting

Hypothes.is provides a free, online plugin to annotate any HTML, PDF or EPUB document on the web. The organization recently formed a collaboration with bioRxiv to develop layered discussions over preprints, and is forming partnerships with many others. Hypothes.is offers the possibility of making journal clubs more interactive, as each journal club can form its own private hypothes.is group and invited members can collaboratively add tagged annotations to the preprint, allowing efficient organizing, searching of comments, and replying to comments. At present, hypothes.is offers three levels of openness for annotations: a personal mode, where your annotations are visible only to yourself; a private mode, where your annotations are seen only by group members; and a public mode, where your annotations are visible to all hypothes.is users. The organisation is currently developing these options to increase flexibility of openness for hypothes.is groups.

Academic Karma: a preprint review platform

Academic Karma allows you to browse preprints and preprint reviews recently posted to its platform, and to post your own preprint review. Reviews can be endorsed by members and comments can be left in response to the reviews. Academic Karma indexes preprints based on their submission status, where authors intend to submit, and if they intend to submit to a journal. In exchange for your review, Academic Karma rewards you with Karma points, and removes Karma points from the preprint author to encourage preprint authors to review other preprints. On Academic Karma, reviews can be signed or anonymous.

PubPeer: an online journal club for post-publication peer review and preprints

PubPeer aims to improve the progression of science by providing a platform for community commenting on scientific research. The platform, launched late 2012, primarily focuses on post-publication reviews; however, PubPeer recently opened it up to preprint reviews. On PubPeer, reviews can be either signed or anonymous.

Publons: a peer-review record system and resource for peer-review training

Publons provides a record of your contributions to peer review. This includes official pre-publication review for academic journals and post-publication review. Publons recently allowed the addition of preprint reviews to users’ Publons Profile. It also launched Publons Academy, a free online resource that aims to train up the next generation of expert peer reviewers and help alleviate fears and concerns of early-career researchers in performing their first peer review.

Before building PREreview, we surveyed 102 scientists – mainly academics – and asked what they thought about discussing preprints at journal clubs.

Our results showed that the main obstacles to hosting a preprint journal club related to time and motivation. As researchers ourselves, we know only too well how precious time is to a busy scientist, so we wanted to smooth everyone’s first steps by developing a Preprint Journal Club Starter Kit, which includes information on preprints and how to start a journal club (all this information is available on PREreview).

More surprisingly, the majority of people polled expressed excitement and saw the value of preprint journal clubs for training early-career researchers in how to write a review. Even though 72% of our survey respondents had contributed to the peer-review process, only 18% had received any training. Therefore, included in our Starter Kit are preprint peer-review guidelines. We hope, as PREreview grows, the stock of preprint reviews will become an additional resource to help train early-career researchers in peer review.

We need your feedback to make PREreview as beneficial as possible!

We are recruiting volunteer beta testers to start a preprint journal club at their research institutions using our guidelines and PREreview platform.

Thanks to our recently awarded Mozilla Science Lab mini grant, we are able to financially support our beta testers to provide refreshments for two journal clubs. Read our full application on our GitHub repository and email us at preprintjc@gmail.com if you would like to be considered for beta testing.

Samantha Hindle (Twitter: @HindleSamantha) is an Assistant Professional Researcher at the University of California, San Francisco, and an ASAPbio Ambassador, dedicated to promoting preprint awareness and education both nationally and internationally.

Daniela Saderi (Twitter: @Neurosarda) is a Ph.D. candidate in the Neuroscience Graduate Program at Oregon Health & Science University and ASAPbio Ambassador, passionate about promoting transparency and reproducibility in science.

To become an advocate for preprints, you can join the ASAPbio Ambassador task force and share your activities with the community, ask questions and engage in conversations about preprints and preprint journal clubs on our Slack channel.

We would like to thank Jessica Polka and Prachee Avasthi for supporting and advising us since the very inception of this project. We also credit and thank Prachee for suggesting the name for our platform, Jeff Spohn for designing the PREreview logo, and the Twitter community for helping us decide on the name and logo.

For the latest in innovation, eLife Labs and new open-source tools, sign up for our technology and innovation newsletter. You can also follow @eLifeInnovation on Twitter.

Do you have an idea or innovation to share on eLife Labs? Please send a short outline to innovation [at] elifesciences.org.