Research Culture: Framework for advancing rigorous research
- National Institute of Neurological Disorders and Stroke, United States
- iBiology, United States
- Center for Teaching and Department of Biological Sciences, Vanderbilt University, United States
- Department of Kinesiology and Wisconsin Institute for Science Education and Community Engagement, University of Wisconsin - Madison, United States
- Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, United States
- Department of Brain and Cognitive Science, Institute of Medical Engineering and Sciences, the Picower Institute for Learning and Memory, and the Institute for Data Systems and Society, Massachusetts Institute of Technology, United States
- Department of Biology and Program in Neuroscience, Brandeis University, United States
- Harvard Graduate School of Education, Harvard University, United States
- Department of Public Health Sciences, Medical University of South Carolina, United States
- Planetary Science Institute, United States
- Cellular and Molecular Biology Graduate Program, University of Michigan, United States
- The Concord Consortium, United States
- Department of Neurological Surgery, Weill Cornell Medical College, United States
- Department of Biological Sciences, Duquesne University, United States
- Solomon H. Snyder Department of Neuroscience, Johns Hopkins School of Medicine, United States
- Carney Institute for Brain Science, Department of Neuroscience, Brown University, United States
- Centre for Clinical Brain Sciences, University of Edinburgh, United Kingdom
- Biomedical Graduate Education, Georgetown University Medical Center, United States
- MRC Integrative Epidemiology Unit, School of Psychological Science, University of Bristol, United Kingdom
- Oakland University, United States
- Department of Biological Sciences, The University of Texas at El Paso, United States
- National Centre for the Replacement, Refinement and Reduction of Animals in Research (NC3Rs), United Kingdom
- Department of Neurobiology, Northwestern University, United States
- Complexly, United States
- Center for Talented Youth and School of Education, Johns Hopkins University, United States
- New York Hall of Science, United States
Figures
Figure 1
Outline of an educational resource on the principles of rigorous research suitable for a variety of audiences.
We envision a comprehensive resource that can be used by scientists at all stages of their career to explore the principles of rigorous research at various levels of detail. We envision modules on a range of topics (such as reducing cognitive biases), each of which contains a number of topics (such as blinding), each of which contains a number of lessons (such as practical examples).
Tables
Table 1
Activities for communities of rigor champions to promote the principles of rigorous research.
| Community | Intra-organizational activities | Inter-organizational activities |
|---|---|---|
| Trainees | • Promote transparency and other rigorous practices among colleagues and mentors • Advocate for resources to facilitate rigorous research practices | • Share institutional resources and practices in education and training • Call for changes in institutional culture and policies |
| Researchers | • Transparently report all experiments, including neutral outcomes • Promote rigorous practices among colleagues and trainees • Call for changes to institutional culture, policies, and infrastructure | • Share effective training practices and useful laboratory resources • Coordinate with the broader scientific community to promote better incentive structures |
| Educators | • Suggest improvements to available resources that address rigor • Integrate rigorous research principles into all coursework | • Share resources and educational best practices • Share effective learning evaluation methods |
| Institutional Leaders | • Enact policies and support infrastructure to incentivize transparency and other rigorous research practices • Explicitly incorporate mentoring, collaboration, and rigorous research practices into promotion procedures • Initiate and share outcomes from piloted educational resources | • Support and promote communities of rigor champions • Disseminate policy changes, new initiatives, educational successes, and implementation strategies • Develop tangible outcome measures to evaluate impact |
| Journal Editors and Reviewers | • Promote thorough review of research practices in publications • Explicitly support research transparency and neutral outcomes • Educate reviewers on which scientific practices are valued by the journal | • Collaborate to implement best practices consistently across different publishers |
| Scientific Societies and Organizations | • Support the founding of communities of rigor champions • Compile and encourage best practices used by the scientific community • Host workshops and educational materials for members | • Promote and maintain communities of rigor champions • Encourage institutional policies that promote research quality and effective education |
| Funding Organizations | • Emphasize attention to rigor in peer review • Reward rigorous research practices and outstanding mentorship • Support infrastructure for transparent and rigorous science • Support educational resources and initiatives | • Support and promote communities of rigor champions • Share best practices for incentivizing rigorous research and educating scientists • Develop partnerships to support better training and facilitate cultural changes |
Table 2
Key elements of teaching and learning to include in an educational resource on the principles of rigorous research.
| Key element | Teaching and learning principle |
|---|---|
| Clear learning objectives | Define the learning objectives upfront, identify ways to measure achievement of these objectives, and then design activities to support learning (Bradforth et al., 2015). |
| Inquiry-based learning | Encourage students to pose their own questions, apply commonly used tools and methods to actively explore their questions, and provide evidence when explaining phenomena (Bradforth et al., 2015; Corwin et al., 2015; Minner et al., 2010; Handelsman et al., 2004). |
| Relevance | Provide feedback on real-world experiments, whether in the classroom or the laboratory, as a way to demonstrate relevance and stimulate interest. Opportunities for personalized application and discussion in the local setting with the help of a facilitator’s guide are particularly critical, as adults typically learn most effectively when given the opportunity for immediate personal utility and value (Walkington and Bernacki, 2018). Emphasize the ability to contribute to a larger purpose or gain social standing (Yeager et al., 2014). |
| Individuality | Include a range of approaches to teaching and learning to accommodate different levels of knowledge and skills, motivations, and senses of self-efficacy (Walkington and Bernacki, 2018; Raman, 2014). |
| Self-efficacy | Allow individuals to gain self-efficacy by experiencing a feeling of progress, being challenged in low-stakes environments, and working through confusing concepts successfully (D’Mello et al., 2014). This is more effective when the person feels psychologically safe to take risks and fail in front of their local scientific community. |
| Belonging | Facilitate learning, foster collaboration, and recognize diverse perspectives in order to encourage learners to gain agency and forge a connection with the intellectual community (Bjork et al., 2013; Brown and Adler, 2008). |
| Recognition of complexity | Include complexity and inconsistencies in training examples rather than simplification for the sake of a persuasive story (Howitt and Wilson, 2014; Coleman, 1987). This counteracts the drive to smooth over inconvenient but potentially important details and highlights the importance of confounding variables, potential artefactual influences, reproducibility, and robustness of the findings. |
| Cultivation of growth | Nurture positive behaviors, like acknowledging and learning from mistakes, rather than penalize imperfect practices (Alberts et al., 2015). Mentors at all career stages are encouraged to model these positive behaviors and to share their own failures, the drudgery and frustrations of science, and their approaches to coping emotionally and growing intellectually while maintaining rigorous research practices. |
| Assessment of behavioral change | Measure success via gains in learner competency and changes to their real-world approaches to research. Changes in laboratory practice could be assessed by user self-reports, by analysis of research presented at meetings (Silberberg et al., 2017) and in publications (MacLeod et al., 2015), or by querying scientists on whether discussions with their mentors and colleagues led to changes in laboratory and institutional culture. Collaborate from the beginning with individuals who specialize in assessment design in higher education settings (Bradforth et al., 2015). |
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Research Culture: Framework for advancing rigorous research
eLife 9:e55915.
https://doi.org/10.7554/eLife.55915
