Development and evaluation of a machine learning-based in-hospital COvid-19 disease outcome predictor (CODOP): a multicontinental retrospective study

  1. Riku Klén
  2. Disha Purohit
  3. Ricardo Gómez-Huelgas
  4. José Manuel Casas-Rojo
  5. Juan Miguel Antón-Santos
  6. Jesus Millan Nunez-Cortes
  7. Carlos Lumbreras
  8. Jose Manuel Ramos-Rincon
  9. Noelia Garcia Barrio
  10. Miguel Pedrera Jimenez
  11. Antonio Lalueza Blanco
  12. María Dolores Martin-Escalante
  13. Francisco Rivas-Ruiz
  14. Mari Ángeles Onieva-Garcia
  15. Pablo Young
  16. Juan Ignacio Ramirez
  17. Estela Edith Titto Omonte
  18. Rosmery Gross Artega
  19. Magdy Teresa Canales Beltrán
  20. Pascual Ruben Valdez
  21. Florencia Pugliese
  22. Rosa Castagna
  23. Ivan Huesped
  24. Bruno Boietti
  25. Javier A Pollan
  26. Nico Funke
  27. Benjamin Leiding
  28. David Gómez-Varela  Is a corresponding author
  1. University of Turku, Finland
  2. Max Planck Institute of Experimental Medicine, Germany
  3. University of Malaga, Spain
  4. General University Hospital of Alicante, Spain
  5. Gregorio Marañón University Hospital, Spain
  6. Hospital Universitario 12 De Octubre, Spain
  7. Hospital Costa del Sol, Spain
  8. Hospital Británico of Buenos Aires, Argentina
  9. Hospital Santa Cruz - Caja Petrolera de Salud, Bolivia
  10. Hospital of San Juan de Dios, Bolivia
  11. Hospital Honduras Medical Centre, Honduras
  12. Hospital Velez Sarsfield, Argentina
  13. Hospital Italiano de Buenos Aires, Argentina
  14. Clausthal University of Technology, Germany
  15. Max Planck Institute for Experimental Medicine, Germany

Abstract

New SARS-CoV-2 variants, breakthrough infections, waning immunity, and sub-optimal vaccination rates account for surges of hospitalizations and deaths. There is an urgent need for clinically valuable and generalizable triage tools assisting the allocation of hospital resources, particularly in resource-limited countries. We developed and validate CODOP, a machine learning-based tool for predicting the clinical outcome of hospitalized COVID-19 patients. CODOP was trained, tested and validated with six cohorts encompassing 29223 COVID-19 patients from more than 150 hospitals in Spain, the USA and Latin America during 2020-22. CODOP uses 12 clinical parameters commonly measured at hospital admission for reaching high discriminative ability up to nine days before clinical resolution (AUROC: 0·90-0·96), it is well calibrated, and it enables an effective dynamic risk stratification during hospitalization. Furthermore, CODOP maintains its predictive ability independently of the virus variant and the vaccination status. To reckon with the fluctuating pressure levels in hospitals during the pandemic, we offer two online CODOP calculators, suited for undertriage or overtriage scenarios, validated with a cohort of patients from 42 hospitals in three Latin American countries (78-100% sensitivity and 89-97% specificity). The performance of CODOP in heterogeneous and geographically disperse patient cohorts and the easiness of use strongly suggest its clinical utility, particularly in resource-limited countries.

Data availability

The raw patient data used in this study are not freely available due to legal restrictions of the ethical committees of the different hospitals. However, they can be accessed upon request to the Scientific Committees of these organisms. An exception to this is the patient data from the USA cohort, which has been published elsewhere. All the model's numerical data necessary to generate all figures can be found in the submitted source data tables. Furthermore, all supplementary tables can be found in Supplementary File 1.

Article and author information

Author details

  1. Riku Klén

    Turku PET Centre, University of Turku, Turku, Finland
    Competing interests
    The authors declare that no competing interests exist.
  2. Disha Purohit

    Max Planck Institute of Experimental Medicine, Goettingen, Germany
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-1442-335X
  3. Ricardo Gómez-Huelgas

    Internal Medicine Department, University of Malaga, Màlaga, Spain
    Competing interests
    The authors declare that no competing interests exist.
  4. José Manuel Casas-Rojo

    Internal Medicine Department, General University Hospital of Alicante, Madrid, Spain
    Competing interests
    The authors declare that no competing interests exist.
  5. Juan Miguel Antón-Santos

    Internal Medicine Department, General University Hospital of Alicante, Madrid, Spain
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-3443-1100
  6. Jesus Millan Nunez-Cortes

    Internal Medicine Department, Gregorio Marañón University Hospital, Madrid, Spain
    Competing interests
    The authors declare that no competing interests exist.
  7. Carlos Lumbreras

    Internal Medicine Department, Hospital Universitario 12 De Octubre, Madrid, Spain
    Competing interests
    The authors declare that no competing interests exist.
  8. Jose Manuel Ramos-Rincon

    Internal Medicine Department, General University Hospital of Alicante, Elche, Spain
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-6501-9867
  9. Noelia Garcia Barrio

    Data Science Unit, Hospital Universitario 12 De Octubre, Madrid, Spain
    Competing interests
    The authors declare that no competing interests exist.
  10. Miguel Pedrera Jimenez

    Data Science Unit, Hospital Universitario 12 De Octubre, Madrid, Spain
    Competing interests
    The authors declare that no competing interests exist.
  11. Antonio Lalueza Blanco

    Internal Medicine Departmen, Hospital Universitario 12 De Octubre, Madrid, Spain
    Competing interests
    The authors declare that no competing interests exist.
  12. María Dolores Martin-Escalante

    Internal Medicine Department, Hospital Costa del Sol, Marbella, Spain
    Competing interests
    The authors declare that no competing interests exist.
  13. Francisco Rivas-Ruiz

    Internal Medicine Department, Hospital Costa del Sol, Marbella, Spain
    Competing interests
    The authors declare that no competing interests exist.
  14. Mari Ángeles Onieva-Garcia

    Preventive Medicine Department, Hospital Costa del Sol, Marbella, Spain
    Competing interests
    The authors declare that no competing interests exist.
  15. Pablo Young

    Clinical Medicine service, Hospital Británico of Buenos Aires, Buenos Aires, Argentina
    Competing interests
    The authors declare that no competing interests exist.
  16. Juan Ignacio Ramirez

    Clinical Medicine service, Hospital Británico of Buenos Aires, Buenos Aires, Argentina
    Competing interests
    The authors declare that no competing interests exist.
  17. Estela Edith Titto Omonte

    Internal Medicine Service, Hospital Santa Cruz - Caja Petrolera de Salud, Santa Cruz, Bolivia
    Competing interests
    The authors declare that no competing interests exist.
  18. Rosmery Gross Artega

    Epidemiology Unit, Hospital of San Juan de Dios, Santa Cruz, Bolivia
    Competing interests
    The authors declare that no competing interests exist.
  19. Magdy Teresa Canales Beltrán

    Instituto Hondureno of social security, Hospital Honduras Medical Centre, Tegucigalpa, Honduras
    Competing interests
    The authors declare that no competing interests exist.
  20. Pascual Ruben Valdez

    Hospital Velez Sarsfield, Buenos Aires, Argentina
    Competing interests
    The authors declare that no competing interests exist.
  21. Florencia Pugliese

    Hospital Velez Sarsfield, Buenos AIres, Argentina
    Competing interests
    The authors declare that no competing interests exist.
  22. Rosa Castagna

    Hospital Velez Sarsfield, Buenos Aires, Argentina
    Competing interests
    The authors declare that no competing interests exist.
  23. Ivan Huesped

    Hospital Italiano de Buenos Aires, Buenos Aires, Argentina
    Competing interests
    The authors declare that no competing interests exist.
  24. Bruno Boietti

    Hospital Italiano de Buenos Aires, Buenos Aires, Argentina
    Competing interests
    The authors declare that no competing interests exist.
  25. Javier A Pollan

    Hospital Italiano de Buenos Aires, Buenos Aires, Argentina
    Competing interests
    The authors declare that no competing interests exist.
  26. Nico Funke

    Max Planck Institute of Experimental Medicine, Goettingen, Germany
    Competing interests
    The authors declare that no competing interests exist.
  27. Benjamin Leiding

    Institute for Software and Systems Engineering, Clausthal University of Technology, Clausthal, Germany
    Competing interests
    The authors declare that no competing interests exist.
  28. David Gómez-Varela

    Max Planck Institute for Experimental Medicine, Göttingen, Germany
    For correspondence
    gomez@em.mpg.de
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-2502-9419

Funding

Max Planck Society (Publication cost)

  • David Gómez-Varela

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Reviewing Editor

  1. Evangelos J Giamarellos-Bourboulis, National and Kapodistrian University of Athens, Medical School, Greece

Version history

  1. Preprint posted: September 22, 2021 (view preprint)
  2. Received: November 30, 2021
  3. Accepted: April 24, 2022
  4. Accepted Manuscript published: May 17, 2022 (version 1)
  5. Version of Record published: May 24, 2022 (version 2)

Copyright

© 2022, Klén et al.

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 1,364
    views
  • 263
    downloads
  • 5
    citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Riku Klén
  2. Disha Purohit
  3. Ricardo Gómez-Huelgas
  4. José Manuel Casas-Rojo
  5. Juan Miguel Antón-Santos
  6. Jesus Millan Nunez-Cortes
  7. Carlos Lumbreras
  8. Jose Manuel Ramos-Rincon
  9. Noelia Garcia Barrio
  10. Miguel Pedrera Jimenez
  11. Antonio Lalueza Blanco
  12. María Dolores Martin-Escalante
  13. Francisco Rivas-Ruiz
  14. Mari Ángeles Onieva-Garcia
  15. Pablo Young
  16. Juan Ignacio Ramirez
  17. Estela Edith Titto Omonte
  18. Rosmery Gross Artega
  19. Magdy Teresa Canales Beltrán
  20. Pascual Ruben Valdez
  21. Florencia Pugliese
  22. Rosa Castagna
  23. Ivan Huesped
  24. Bruno Boietti
  25. Javier A Pollan
  26. Nico Funke
  27. Benjamin Leiding
  28. David Gómez-Varela
(2022)
Development and evaluation of a machine learning-based in-hospital COvid-19 disease outcome predictor (CODOP): a multicontinental retrospective study
eLife 11:e75985.
https://doi.org/10.7554/eLife.75985

Share this article

https://doi.org/10.7554/eLife.75985

Further reading

    1. Epidemiology and Global Health
    2. Medicine
    3. Microbiology and Infectious Disease
    Edited by Diane M Harper et al.
    Collection

    eLife has published the following articles on SARS-CoV-2 and COVID-19.

    1. Computational and Systems Biology
    2. Genetics and Genomics
    Weichen Song, Yongyong Shi, Guan ning Lin
    Tools and Resources

    We propose a new framework for human genetic association studies: at each locus, a deep learning model (in this study, Sei) is used to calculate the functional genomic activity score for two haplotypes per individual. This score, defined as the Haplotype Function Score (HFS), replaces the original genotype in association studies. Applying the HFS framework to 14 complex traits in the UK Biobank, we identified 3619 independent HFS–trait associations with a significance of p < 5 × 10−8. Fine-mapping revealed 2699 causal associations, corresponding to a median increase of 63 causal findings per trait compared with single-nucleotide polymorphism (SNP)-based analysis. HFS-based enrichment analysis uncovered 727 pathway–trait associations and 153 tissue–trait associations with strong biological interpretability, including ‘circadian pathway-chronotype’ and ‘arachidonic acid-intelligence’. Lastly, we applied least absolute shrinkage and selection operator (LASSO) regression to integrate HFS prediction score with SNP-based polygenic risk scores, which showed an improvement of 16.1–39.8% in cross-ancestry polygenic prediction. We concluded that HFS is a promising strategy for understanding the genetic basis of human complex traits.