By Peter Rodgers, Features Editor, eLife
A scientific paper is rarely the last word on any topic. Rather, it should move the relevant field of research forward in some way and, the authors would hope, influence the work of other scientists. But what happens if another scientist reads the paper and thinks "that's not right"? Or, worse still, tries to repeat the experiments in the paper and gets different results?
The answer is that there are quite a few options, ranging from posting a comment on the original paper or on a third-party website (like PubPeer) through blogging or tweeting about the paper to uploading a preprint or submitting a full manuscript to the journal that published it. However, posting a comment or tweeting or blogging will not change the 'scientific record', and journals are often reluctant to publish papers that challenge or contradict papers they have published [1]. In many cases uploading a preprint will be the most efficient way for a scientist to make public their concerns about a paper.
A number of journals have specific article types for such challenges but, again, they often seem reluctant to go down this route. eLife introduced such an article type in 2016 and this month we published our first two items of "Scientific Correspondence": an article that challenges an eLife article, along with a response from the authors of the original article. Further items of Scientific Correspondence are in the pipeline.
This blog post expands on the description of Scientific Correspondence given in our Author Guide [2], and also discusses how such challenges have been handled in the past.
Submitting an item of Scientific Correspondence
If you read an eLife paper and decide that you would like to submit a Scientific Correspondence that challenges its central findings, your first step should be to contact the corresponding author of the original paper in an effort to resolve matters. You will need to include evidence of these efforts in your initial submission: ideally, this evidence will be the email thread between you and the original author, with attachments as appropriate.
Scientific Correspondence must also be submitted within a year of the original eLife paper being published. Such time limits are common at journals: at Science and PNAS, for example, the time limits are three and six months respectively. Although Cell and Nature do not have time limits, they publish relatively few challenges [3].
Manuscripts submitted as Scientific Correspondence should be written in a measured tone; manuscripts not written in a measured tone will be sent back immediately to the authors for revision before being considered in detail. Articles should also read like scientific papers rather than referee reports. In particular, please take note of the following:
- please avoid asking lots of questions: for example, instead of asking "Why are no data provided to support the claim that...", please reword to read "No data are provided to support the claim that..."
- please avoid speculation and hyperbole: that is, please avoid words and phrases like "probably", "I wonder why", "conclusions might differ if" and "it defies belief"
- please do not ask for the original paper to be corrected or retracted; such requests belong in the cover letter, not in the manuscript.
Where possible, the initial submission is considered by the Senior Editor and the Reviewing Editor who handled the original paper (and by others if necessary). In the first instance the Editors will decide if any of the issues raised in the initial submission require a formal Correction to the original article: if a correction is required, the corresponding author of the original paper will be contacted and the Scientific Correspondence will be put on hold until the Correction has been finalized. If a Correction is published, the authors of the Scientific Correspondence will be asked to revise their submission accordingly: however, if publication of the Correction addresses all of the issues raised in the initial submission, the file on the Scientific Correspondence will be closed.
If the Editors agree that the initial submission represents a credible challenge to the central findings of the original paper (possibly after the publication of a correction), a full submission is invited; the initial submission is also sent to the authors of the original paper and they are given 14 days to submit a formal response. If the initial submission does not represent a credible challenge to the original paper, it is declined.
Once any response has been received, the Editors discuss the challenge and the response, and then decide between the following options:
- accept both for publication
- accept the challenge but reject the response
- reject both
- proceed with peer review of one or both manuscripts
- take another course of action.
If the Editors opt for peer review, the following outcomes are possible:
- the challenge and the response are both accepted
- the challenge is accepted but the response is rejected
- the challenge is rejected (which means there is no need to publish the response). Please note the manuscripts will typically be accepted or rejected at this stage of the process; revised manuscripts will not usually be requested.
While priority is given to manuscripts that challenge eLife papers, we will also consider very important contributions that challenge papers published elsewhere. As described above, the author must contact the corresponding author of the original paper in an effort to resolve matters (and include evidence of these efforts in their initial submission), and the Scientific Correspondence must be submitted within a year of the original paper being published.
Previous challenges to papers in eLife and other journals
In the past challenges to eLife papers have resulted in the publication of Corrections: see, for example, this correction by Jeay et al. Some challenges have also been published as regular research papers: see, for example, this article by Egelman (and this correction by Xu et al.) and this article by Craven (and this correction by Mukherji and O'Shea). And an eLife paper by Wang et al. (which was one of 30 or so papers in a variety of journals to suggest that a protein kinase called MELK is an attractive therapeutic target in human cancer) prompted two challenges (Lin et al.; Giuliano et al.), a follow-up paper from the original group (Huang et al.), and a correction to the original paper: see this commentary by Settleman et al. for more details.
eLife has also published a number of articles that challenge papers published in other journals. For example, an eLife paper by Hyafil and Moreno-Bote challenged a paper in Neuron by Lorteije et al., who responded with a paper in eLife. There are also a number of similar examples that highlight the ability of preprints to encourage scientific debate. For instance, an eLife paper by Weiß et al. challenged a paper on ancient DNA by Smith et al., who responded by posting a preprint on bioRxiv. And when a pair of eLife papers by Meister and Gerkin and Castro challenged a paper on olfactory stimuli by Bushdid et al., the latter responded via a preprint; moreover, the two eLife papers had also started life as preprints. Finally, another eLife paper by Meister (which also started life as a preprint) is notable for challenging aspects of three papers published in other journals: in this case the authors of two of the papers challenged responded via a comment.
Final thoughts
The aim of the process described above is to be fair to everyone – to the authors of the paper that is being challenged, to the authors of the Scientific Correspondence, to other researchers in the field, and to science more generally – and to allow scientific debate to take place in as timely a manner as possible. Let the debates begin!
Notes
[1] Rick Trebino of Georgia Tech has published an entertaining account of his efforts to publish a comment in an optics journal that published a paper criticizing some of Trebino's work.
[2] The observant reader will have noticed that Scientific Correspondence used to be called Research Exchanges, but this name failed to take off.
[3] The observant reader will also have noted that different journals use different names for articles that challenge other articles: for example, they are called Matters Arising at Cell, Brief Communications Arising at Nature, Letters at PNAS, and Technical Comments at Science.
