Why did you decide to have an ERA accompaniment to your own eLife paper?
Simon Tragust, author of ‘Formicine ants swallow their highly acidic poison for gut microbial selection and control’: I already had a complete annotated R script file prepared for the article, as I think that availability of raw data but also analysis of the data should go hand in hand together. From there it was only a small step to the ERA accompaniment, although this small step turned out to be a bit trickier than expected.
Matt Nolan, author of ‘Inter- and intra-animal variation in the integrative properties of stellate cells in the medial entorhinal cortex’: Several reasons. First, our article (Pastoll et al. 2020) had quite extensive statistical analyses within it. By making an ERA we hoped to make these analyses more accessible to any readers interested to dig a little deeper or to try to carry out similar analyses themselves. Second, we work hard within the lab to make sure our analyses are well documented and fully reproducible. We thought it would be great to see how well an ERA works in linking analysis code more closely to the published paper. We're also very keen in general on transparency and re-usability and we hope that, by showing an ERA can work, that other labs will give it a try too.
James Watson, author of ‘Collider bias and the apparent protective effect of glucose-6-phosphate dehydrogenase deficiency on cerebral malaria’: I was working on a paper at the time that seemed like a great fit for an ERA. I contacted eLife staff about it. That paper actually didn't get accepted by eLife, but eLife recontacted me to ask if I wanted to retroactively make a previously published paper into an ERA.
Karl Huang, author of ‘Evaluating the impact of open access policies on research institutions’: Our eLife article forms part of the research carried out within the Curtin Open Knowledge Initiative (COKI) project. COKI advocates for universities to become open knowledge institutions and for universities to embrace “openness” in knowledge production and dissemination. We aim to develop insights and tools to support universities and researchers on achieving those goals. As such, we found the development of an ERA to be strongly aligned to our own goals in supporting transparency and reproducibility in research. As an article that analyses global trends in open access to research outputs, we feel that it is even more important for readers to have as much access to, and the ability to engage with, the data as possible. We were certainly very happy and excited to be among the first authors to be involved with the ERA publication process and be able to support a project that reduces barriers to research.”
Matteo Mancini, author of ‘An interactive meta-analysis of MRI biomarkers of myelin’: We wanted to make our meta-analysis interactive right from the beginning, when we were still surveying the literature. We put together a Jupyter Notebook to make our results and figures easily reproducible, but for us it wasn’t enough: a Notebook in the supplementary materials will be noticed only from those who look for it. However, our goal was to make the reader able to explore our results through interactive visualization, so anyone could actually make sense of the results themselves. This is why we wanted to have everything directly accessible through the paper itself. When we got in touch with an eLife editor to see if there was interest in our paper, we mentioned immediately that we wanted to make it an ERA. We went even a step further: we made a Jupyter Book out of the original Notebook and we hosted it through the non-profit NeuroLibre project – in this way, during the review process the reviewers were able to just click on a link in the caption to see the interactive figure coming alive. The figures on NeuroLibre are still available to this day through the ERA itself.
Sophie de Buyl, author of ‘Stochastic logistic models reproduce experimental time series of microbial communities’: I think this should be done for all publications (for which it is relevant).
How has your published ERA enhanced the communication or representation of specific results? Are there any features that you’re particularly happy with?
Sophie de Buyl: Our code was already available on GitHub but now it is straightforward to reproduce our results.
Matteo Mancini: Meta-analyses and systematic reviews can be precious allies for researchers: they collect several studies in a rigorous way and are able to create an overall snapshot of what we know on a specific research question in statistical terms. Unfortunately, the statistical language can be a barrier, and in most cases the main outcome of a meta-analysis is a giant table and some graphs. "ERAs give space for creativity to break that barrier – you can give the power to a figure to ‘explain itself', and this is why, once again, being able to embed interactive figures directly in the paper was really a game-changer for us." We are proud that our ERA is the first example that leverages interactive figures. Also, the way ERAs are implemented on Stencila was the icing on the cake: the native support for Jupyter Notebooks on the platform meant that our figures, made with the Plotly framework, could be rendered directly – we did not have to do any extra work!
Simon Traugust: I think it is great that the code is not hidden away in the supplement or some repository. Other researchers might find it especially helpful to produce certain types of figures in R or would like to try out one type of analysis with data and code at hand.”
Matt Nolan: I'm not sure that it's necessarily enhanced the communication of the main message of our study to a general reader, although I can imagine that for some other studies this could really be the case. Rather, I think the benefit for our study is in increased transparency and ease of re-usability of the analyses.
James Watson: My ERA is quite a simple paper. I would see it as a good template for how ERA can be constructed and then used.
Karl Huang: Having the code right next to the text in the article allows readers to immediately see what choices we have made on data filtering and presentation. Inevitably, a static version of the article only represents a small part of the whole data. "Having an ERA means readers are able to dive deeper into the data" (e.g., change specific figures to focus on a different year).
And, more generally, how do you feel about your ERA publication overall?
Matt Nolan: Overall, we're very happy.
Matteo Mancini: The way ERA publication is implemented at eLife and Stencila is straightforward, and the Stencila team has been really supportive. I loved the overall experience and honestly I look forward to more chances to create ERAs!
Simon Traugust: I am overall happy and hope that in the future the process of producing an ERA publication will be more streamlined so that authors that wish to do so will not depend as much on the ERA publication team for help as I did.
James Watson: Awesome! It's a fantastic idea – the future, I hope, of computational research papers.
Karl Huang: In general, I think everything went pretty smoothly. Understandably the tools were under development at its initial phases. Our article was the first to be published in the form of a Jupyter Notebook (rather than R Markdown) so some work was required from the ERA team to support it. However, the ERA team was able to update the tools and implemented changes very efficiently. We also love the clear way in which our ERA publication is presented on the eLife website and how easy it is for readers to engage with the codes and data.
Would you consider having another article published as an ERA in future?
Matteo Mancini: I would consider leveraging ERAs for any publication platform that provides this chance. I think that they are the ideal tool for going beyond the conservative idea of what research outcomes should look like. ERAs provide two specific advantages over conventional papers, and for two different audiences: for those who are familiar with the topic, an ERA gives the chance to dive deeper in the methodology used and where the results come from; for readers from different fields, an "ERA provides the tools to go beyond field-specific language and potentially make cross-field interactions more viable.”
Matt Nolan: Yes, I would. I think this would be up to the team members on the project and whether they want to take the extra time to embed their code within the article, but I would strongly encourage it.
Sophie de Buyl, James Watson and Karl Huang: Yes!
Simon Traugust: Certainly!
Would you recommend ERA to other authors? Is there anything you’d like to say to authors who may be considering an ERA publication for their own work?
James Watson: Yes! I would say that if people are familiar with tools like R Markdown, then it’s quite easy to integrate the analyses into an ERA.
Matteo Mancini: I would definitely recommend ERA to other authors, especially for the ones who rely already on Jupyter Notebooks to summarize their analyses – in that case the additional work is virtually non-existent and the advantages are countless.
Simon Traugust: Definitely, although some more work will be required. But then, maybe in a not-so-far future something like ERA will become the standard.
Matt Nolan: Absolutely. I think this is a relatively easy way that as a community we can improve transparency in publishing our results and make it easier for people to build on previous work. It would be great to see people like funders, and hiring and promotion committees, give people credit if they make the extra effort to do this.
Karl Huang: Yes. Given the current global trends towards open research, it is imperative that we consider ‘openness’ as a vital part of research. The ERA process provides the environment and support that makes this process simple and efficient. So, definitely strongly recommended.
--
We are grateful to these authors for taking their time to share their feedback with us, and for helping us showcase how Executable Research Articles can help improve the transparency, reproducibility and discoverability of research content across a variety of research subjects. Executable Research Articles are an open-source technology available to all, and we encourage any authors or publishers interested in the format to [get in touch] for more information.
#
We welcome comments, questions and feedback. Please annotate publicly on the article or contact us at innovation [at] elifesciences [dot] org.
Do you have an idea or innovation to share? Send a short outline for a Labs blogpost to innovation [at] elifesciences [dot] org.
For the latest in innovation, eLife Labs and new open-source tools, sign up for our technology and innovation newsletter. You can also follow @eLifeInnovation on Twitter.
