A study has characterised the factors that influence the adoption of clinical research into practice by comparing the citation rates of medical research papers between 18 different specialities, article type, geography and time. The results are published today in eLife.
The work reveals that most clinical trials are never cited by the time they reach the point-of-care – where clinical trials are embedded into regular medical practice. Furthermore, less than one in two Phase III clinical trials are ever cited across 9 out of 18 medical specialities. Phase III clinical trials typically involve a large number of participants and are designed to evaluate the safety and effectiveness of a new medical intervention across the general population.
The results represent a preliminary step to understanding why increased research funding in the US has not yielded equivalent results in health outcomes. The findings could be used to monitor the efficiency of research investments and evaluate health policies influencing the translation of biomedical studies to clinical practice.
The lack of evidence to assess the effectiveness and efficiency of treatments is a contributing factor that underpins the unsustainable expenditure of the US healthcare system. This is made worse by a limited understanding of the factors that influence the translation of clinical research into practice.
“This may partially explain why, in the US healthcare systems, many patients do not receive the recommended services and many receive treatment that is not necessary or appropriate for them,” says co-author Moustafa Abdalla, resident in the Department of Surgery, at Massachusetts General Hospital, and fellow at Harvard Medical School, both in Boston, Massachusetts, US.
The team used the resource tool UpToDate, which provides current evidence-based clinical information at the point of medical practice, to assess what clinical research makes it into practice. By leveraging a dataset of more than 10,000 UpToDate articles, sampled every three months between 2011–2020, they were able to trace the path of research from initial publication to medical practice, compared to the 2.4 million uncited studies published during the same time window across 18 medical specialties.
The analysis revealed substantial variation in how different medical specialities adopt research with regards to frequency of literature citations and the quality-of-evidence incorporated. Across all specialities, only 0.4% to 2.4% of published clinical research is ever cited in UpToDate. Amongst the 18 specialities included in the analysis, neurology had the highest citation rate at 2.4% (2,057 of 85,843 research articles), and pathology had the lowest rate at 0.4% (317 of 69,343 research articles).
The proportion of citations varied substantially by the type of article the research was presented in. Practice guidelines represented the most likely article type to be cited, with 9 of the 18 specialties receiving citations at a rate of over 13%. Clinical trials were the second most likely to be cited, but were also the most variable. In 9 of the 18 specialties, the team observed that less than one in ten Phase III clinical trials were ever cited at the point-of-care.
The team next sought to characterise the effects of National Institutes of Health (NIH) funding on the number of citations a paper receives, primarily focusing on the US. They found that, across all specialties in the past decade, average annual NIH funding correlated strongly with the total number of citations. Given this strong correlation, the team then calculated the effective cost per new citation at the point-of-care. This was used to represent the approximate indirect cost of bringing clinical research to medical practice in NIH funding dollars. In urology – a branch of medicine that focuses on diseases of the urinary track – and nephrology – a specialty of medicine concerning the kidneys – the cost of bringing a new citation to the point-of-care was estimated at $48,086.18 per article (not including the intercept of the model which reflects set up and startup costs). On the other side of the spectrum, in pathology, it cost $874.85 for every additional citation.
Whilst UpToDate represents the largest point-of-care resource, its relevance varies by specialty and training status. Therefore, whilst the study uses UpToDate as a metric of translation of research, the authors say a citation does not necessarily indicate actual changes in practice. Instead, citations represent adoption of knowledge to support current approaches, inform new changes in practice, or highlight points of controversy.
“While the findings of our study are largely descriptive and explanatory, we have provided a baseline for monitoring the efficiency of research investments. This may eventually lead to the development of strategies to quantify the effectiveness of proposed reforms to the biomedical scientific process,” concludes co-author Mohamed Abdalla, Scientist, Institute for Better Health, Trillium Health Partners, Mississauga, Ontario, Canada.
eLife transforms research communication to create a future where a diverse, global community of scientists and researchers produces open and trusted results for the benefit of all. Independent, not-for-profit and supported by funders, we improve the way science is practised and shared. From the research we publish, to the tools we build, to the people we work with, we’ve earned a reputation for quality, integrity and the flexibility to bring about real change. eLife receives financial support and strategic guidance from the Howard Hughes Medical Institute, Knut and Alice Wallenberg Foundation, the Max Planck Society and Wellcome. Learn more at https://elifesciences.org/about.
To read the latest Medicine research published in eLife, visit https://elifesciences.org/subjects/medicine.
About the Institute for Better Health
As Trillium Health Partner’s (THP) research and innovation engine, the Institute for Better Health (IBH) is a core enabler of THP’s mission of a new kind of health care for a healthier community through the application of scientific expertise, innovative thinking and partnerships. Focused on generating cutting-edge science and innovation in health service delivery and population health, IBH leads practical research and innovation that shapes how we engage, design, deliver, and sustain health care to solve problems stretching from the bedside to the system.