Starting from late February 2020, the COVID-19 outbreak struck the north of Italy causing more than 30,000 deaths in Lombardy alone, up to the end of March 2021. At the beginning of the outbreak, the Spedali Civili di Brescia (SCBH), the university hospital of one of the hardest hit cities in Europe, was faced with a ‘flash flood’ of severely ill patients seeking admission to the emergency department (ED). For several weeks, their number exceeded the available resources, obliging a continuous organizational restructuring of the hospital wards (Garrafa et al., 2020b).

In those weeks, given the limited evidence of clinically proven predictors (Marengoni et al., 2021; Wynants et al., 2020; Sperrin et al., 2020), prioritizing hospital admission of non-critical patients was an arduous task. Essentially, the criteria were based on the presence of fever, respiratory symptoms, and the level of blood oxygenation. A significant drawback of this approach was that patients referring to the ED with very similar clinical findings underwent inconsistent assessments. In this scenario, the availability of predictors would have been extremely beneficial, not only to triage patients, but also to monitor hospitalized patients and warn of exacerbation of the outbreaks.

Starting from March 2020, all patients referred to EDs underwent a chest X-ray at admission or within a few hours. With the purpose of grading pulmonary involvement and tracking changes objectively over time, a chest X-ray severity score was developed (Brescia X-ray score) (Borghesi and Maroldi, 2020; Maroldi et al., 2021; Borghesi et al., 2020a; Borghesi et al., 2020b). The score was able to predict in-hospital mortality in 302 patients. In addition to the chest X-ray severity score, a dedicated blood sampling profile was included in the COVID-19 ED work-up (Garrafa et al., 2020a). Among its 17 blood analytes, the sampling profile encompassed hemachrome, inflammation biomarkers such as C-reactive protein (CRP), lactate dehydrogenase (LDH), and ferritin, and coagulation markers (fibrinogen and D-dimer). Since that time, the medical literature began to encompass an increasing number of studies advocating the prognostic value of single or grouped blood parameters (Bonetti et al., 2020; Borghi et al., 2020; Avouac et al., 2021; Knight et al., 2020). All these parameters were present in our COVID-19 sampling profile.

This study aims to develop and validate an early-warning model (BS-EWM), predictive of in-hospital death, based on data that could easily be acquired on admission to the ED: age, simple blood biomarkers, and chest X-ray. The model was constructed based on the analysis of a cohort of 2872 COVID-19 patients treated in a single reference center over a 10-month period.

This paper adheres to the TRIPOD checklist for predictive model development and validation (Collins et al., 2015).

The study was approved by the local ethics committee (COVID-SURG-BS; NP 4059).

The dataset for the development, validation, and testing of the BS-EWM originated entirely from an Italian region, potentially limiting the generalizability of the risk score in other areas of the world. Additional validation studies from different geographic areas are welcomed. Furthermore, though the BS-EWM has been validated using blood sample values obtained by instruments that satisfy internal and external quality control, different equipment could lead to divergent results (Martens et al., 2021; Lippi et al., 2020). Therefore, it would be appropriate to harmonize the results. Another limit could have been the presence of missing values, though the BS-EWM has also performed adequately in this condition since it used a multiple imputation technique to overcome the problem. Finally, it is important to point out that the BS-EWM risk score should not be used for asymptomatic COVID-19 patients or for the pediatric population. It will be interesting, in the future, to verify if the BS-EWM could be applied by general practitioners to the unhospitalized population. This could allow the generalizability of the model outside the hospital context to be tested.

Though the BS-EWM has been developed on a cohort of 2106 patients belonging to the COVID-19 first wave, the model also demonstrated a sensitivity greater than 70% in the early prediction of high risk in patients in the second wave, when in-hospital mortality was 40% lower.

Several predictive models have recently been applied to COVID-19 cohorts with variable results, some of them previously developed to predict mortality for community-acquired pneumonia, such as the Pneumonia Severity Index, CURB-65, qSOFA, and MuLBSTA (Genc Yavuz et al., 2021; Lazar Neto et al., 2021; Artero et al., 2021), NEWS2 criteria (Myrstad et al., 2020; Gidari et al., 2020), and SCAP score (Anurag and Preetam, n.d.). Novel early-warning scores have been specifically built on COVID-19 patient series using different techniques such as parametric and non-parametric tests (Linssen et al., 2020) or artificial intelligence techniques such as the COVID-GRAM score (Liang et al., 2020).

While these models are mostly based on age and a set of vital (clinical) parameters, in addition to age, the BS-EWM depends on blood parameters. It is conceivable that blood analytes capture a snapshot at hospital admission signaling a specific bodily reaction to viral infection in terms of hyperinflammation, immune response, and thrombophilia. On the other hand, the other models are more influenced by the general status of the patient, which may be determined by concomitant and pre-existing diseases.

According to the International Federation of Clinical Chemistry (Bohn et al., 2020), no single biochemical or hematological marker is sufficiently sensitive or specific to predict the outcome of SARS-CoV-2 infection. Notably, the IFCC recommends that the interpretation of laboratory abnormalities should be based on groups of analytes (Bohn et al., 2020). In the BS-EWM, three analytes reached a significant value in predicting death: LDH, D-dimer, and NLR. LDH is a non-specific indicator of tissue damage (Bohn et al., 2020; Liang et al., 2020) that emerges as one of the most consistently elevated markers in patients at higher risk of developing adverse outcomes, probably because COVID-19 infection is characterized by systemic tissue damage. Another key feature of SARS-CoV-2 is the coagulopathy: high levels of D-dimers have been reported to correlate with unfavorable disease progression in several cohorts of patients. The coagulopathy linked to COVID-19 infection is likely to involve a complex interplay between pro-thrombotic and inflammatory factors, thus the combined analysis of both inflammatory and thrombophilic markers could play an important role in the early identification of patients at higher risk of unfavorable progression (Bohn et al., 2020; Lazzaroni et al., 2021). Finally, lymphopenia has become a hallmark of SARS-CoV-2. It has been demonstrated in almost all symptomatic patients, though in varying degrees. Disease severity has been correlated with the level of lymphocyte count reduction. A direct infection of lymphocytes, which express the coronavirus receptor ACE-2, is among the mechanisms proposed. A poor prognosis is also associated with an elevated neutrophil count combined with lymphopenia, resulting in a high NLR. The increase in granulocytes is the result of the cytokine storm induced by the virus and is responsible for tissue damage (Bonetti et al., 2020; Bohn et al., 2020).

A further remark concerning the blood analytes is that, in the BS-EWM, the thresholds of the single analytes (namely the point where the functions in Figure 4 become constant and the probability of death no longer increases/decreases) closely overlap with the values recently proposed by other authors (Webb et al., 2020; Caricchio et al., 2021). For completeness, the optimal threshold (computed through the Youden index) for each biomarker to predict the outcome (dead or alive) are reported in Supplementary file 1e.

The present study is not unique in encompassing radiological findings combined with blood analysis. The study by Schalekamp et al., 2021, integrated blood analysis parameters and radiological information derived by grading chest X-rays (0–8 scale points). Unlike the cited study, with the BS-EWM in this study, the radiological score did not reach a high relevance (rel VIM) in predicting high risk. This difference can be explained by the different approaches used to build the model (logistic regression vs. random forests) and by the high degree of correlation of the X-ray score with multiple blood analytes: ‘collinearity’ thus could have ‘stolen’ importance from the information provided by imaging. Nevertheless, at admission, the chest X-ray score of patients who subsequently died was significantly higher than for patients who survived. Furthermore, the chest X-ray score may provide additional stability to the model, playing an important role in the case of missing data in the blood sample counterpart.

Further, the BS-EWM delivers high prediction performance and only requires a limited number of readily available variables with easy operability, no time consuming, no extra money since these analytes are required for COVID-19 diagnosis and monitoring. An important and pragmatic aspect offered by the BS-EWM is that the biomarkers employed may be obtained by the emergency laboratory in less than an hour (Garrafa et al., 2020a) and, differently from other biomarkers (Kyriazopoulou et al., 2021), they are non-expensive and frequently used also in developing countries. It is important to note that the same methodology could be applied to other infections and be practical to triage people.

Most laboratories, including the small or peripheral ones, may provide results in a short time. At the Spedali Civili of Brescia, the BS-EWM is integrated within the Laboratory Information System (LIS). It works as a web-based calculator and is easy to interpret. The online calculator allows an easy assessment of the EWM, requiring only the entering of analyte values and X-ray score. The score is calculated even if some of the values are missing. Furthermore, in our center the system may be integrated with the electronic health record and the radiology information system, allowing a completely automatic data retrieval and entering, without any operator interaction. It provides a risk threshold of 0.5, above which patients are graded as having a potentially high death-risk, thus supporting closer clinical observation or admission to a high-intensive care ward. In patients yielding a low risk (score 0–0.49), the decision by clinicians to allocate them to a low-intensive care ward or to monitoring is further sustained. The online calculator allows an easy assessment of the EWM, requiring only the entering of analyte values and X-ray score. The score is calculated even if some of the values are missing. Furthermore, in our center the system may be integrated with the electronic health record and the radiology information system, allowing a completely automatic data retrieval and entering, without any operator interaction.

Finally, the need to regularly update models and closely monitor their performances over time and geographically should be underlined, given the rapidly changing nature of the disease and its management.

The dataset contained 2782 COVID-19 symptomatic patients, hospitalized between March and December 2020 at SCBH after referring to the ED. In all patients, the following variables were retrieved from the SCBH database: age, sex, length of hospitalization, Brescia X-ray score (Borghesi and Maroldi, 2020), alive/dead, and 17 blood analytes acquired at admission (D-dimer, fibrinogen, LDH, neutrophils, lymphocytes, NLR, lymphocytes %, neutrophils %, CRP, white blood cell (WBC) count, basophils, basophils %, eosinophils, eosinophils %, monocytes, monocytes %, standardized ferritin). Blood tests were acquired within 24 hr after admission to the hospital.

According to the two temporal peaks of incidence of the COVID-19 outbreak in Lombardy, the 2782 patients were divided into two groups: (i) MA including 2106 patients admitted during the first wave; (ii) MD including 676 patients in the second wave. Quantitative variables were described using mean (SD), median (IQR), and range (min–max), while categorical variables were reported as counts and percentages. The comparisons between groups were performed using the Wilcoxon rank-sum test for quantitative variables and Fisher’s exact test for qualitative variables.

The relationships between the 17 analytes and the Brescia X-ray score were inspected using the Spearman correlation coefficient, ρs, and visualizing results using a correlation plot (Dancelli et al., 2013; Marziano et al., 2019; Figure 2).

To estimate the BS-EWM, the outcome (alive/dead) was modeled using as covariates: (i) Brescia X-ray score, (ii) 17 analytes, (iii) age. Since most of the covariates analyzed were strongly correlated (multi-collinearity) (Figure 2) and their relationships with the outcome were non-linear, the BS-EWM was estimated using random forests (Breiman, 2001; Carpita and Vezzoli, 2012), a non-parametric machine-learning method (Vezzoli, 2011; Vezzoli et al., 2017). Moreover, the algorithm is able to manage missing values which are common in clinical studies. The ‘on-the-fly-imputation’ algorithm (Hong and Lynn, 2020) imputes data when it grows the forest handling interactions and non-linearity in the dataset.

Since the prevalence rate of death in the two waves was different (20% in MA vs. 12% in MD), a strategy to generalize results in unbalanced datasets was applied, adopting a rebalancing method able to improve the detection of patients with a high death-risk.

The EWM was developed using the 2106 patients in the first COVID-19 wave (MA 2020) when in-hospital death prevalence was 20%. Seventy percent of them (1474 patients) were used for training the model and the remainder (632 patients) for testing it. Patients were randomly assigned to the two subgroups, and further stratified according to the outcome (alive/dead). Consequently, both the training and testing subgroups included the same rate of deaths (20.09%) as the full sample (2106 patients). With such a ‘moderate’ incidence of death, the dataset was statistically unbalanced. This limitation could have implied the development of a model yielding unsatisfactory results in predicting new observations for the minority class, that is, patients with death as outcome. An approach to address this limitation is to oversample the minority class (deceased patients) and, subsequently, create the predictive model (BS-EWM). The SMOTE (Chawla et al., 2002) was chosen. The SMOTE function oversamples the minority class by using bootstrapping and k-nearest neighbor to synthetically create additional observations belonging to that class (dead). The procedure is combined with under-sampling of the majority class (alive). To determine the optimum number of k-groups into which to assign the dataset, a matrix containing the 17 analytes and the Brescia X-ray score was used to compute the hierarchical cluster analysis (Salvi et al., 2019; Codenotti et al., 2016). By means of silhouette analysis, k = 2 was determined as the optimal number of clusters into which to assign the dataset. Hence, a synthetic rebalanced dataset was obtained with an equal number of living and deceased patients (888 + 888). The rebalancing procedure enabled a risk score to be devised ranging from 0 to 1 with a threshold of 0.5 to separate non-severely affected from severely affected patients. Subsequently, the model was tested on the subgroup of 632 patients in the first wave excluding the training set. A further validation of the EWM was conducted on the 676 COVID-19 patients in the second wave (Wynants et al., 2020).

The rel VIM (Carpita and Vezzoli, 2012, Doglietto et al., 2020b) and the PDP (Friedman, 2001; Doglietto et al., 2020a) were extracted from the model for a better understanding of the relationship between outcome and covariates.

The predictions extracted from the random forests classification were interpreted as in-hospital death probability conditional on the combination of the values of analytes, Brescia X-ray score, and age in COVID-19 patients at admission to the ED.

The BS-EWM performance was evaluated by AUC of an ROC curve. The robustness of the model was compared to other models by running GBM (a machine-learning approach and competitor to random forests), and logistic regression, and computing the same metrics.

The BS-EWM score is available for use online (https://bdbiomed.shinyapps.io/covid19score). In the SCBH it is integrated within LIS returning the death-risk score directly from the medical report.

All the analyses were performed by R, version 4.0.0 (R Development Core Team, 2020). The code is available at 288 https://github.com/biostatUniBS/BS_EWS copy archived at swh:1:rev:7416ba71075402e6a0ed997e7aa6a527e93247b2 (Garrafa et al., 2021).

We are unable to share the dataset as it contains sensitive personal data collected during the pandemic at Spedali Civili di Brescia. We cannot share the full data since are data from patients. Interested researchers should contact the authors for any query related to data sharing and submit a project proposal Once defined the goal of the study, and the data needed authors will submit the potential project of collaboration to the IRB of Spedali Civili di Brescia to receive approval to access a deidentified dataset. Please note that other informations related to patients can be acquired, always following approval of IRB of Spedali Civili di Brescia, not only the ones studied in the paper. Anyway, following request to the authors, it will be possible to share processed version of the dataset ( e.g. an Excel sheet with numbers used to plot the graphs and charts of the manuscript). All code used to analyse the data can be found on GitHub at https://github.com/biostatUniBS/BS_EWS, (copy archived at swh:1:rev:7416ba71075402e6a0ed997e7aa6a527e93247b2).

1. 1. Artero A
   2. Madrazo M
   3. Fernández-Garcés M
   4. Muiño Miguez A
   5. González García A
   6. Crestelo Vieitez A
   7. García Guijarro E
   8. Fonseca Aizpuru EM
   9. García Gómez M
   10. Areses Manrique M
   11. Martinez Cilleros C
   12. Fidalgo Moreno MDP
   13. Loureiro Amigo J
   14. Gil Sánchez R
   15. Rabadán Pejenaute E
   16. Abella Vázquez L
   17. Cañizares Navarro R
   18. Solís Marquínez MN
   19. Carrasco Sánchez FJ
   20. González Moraleja J
   21. Montero Rivas L
   22. Escobar Sevilla J
   23. Martín Escalante MD
   24. Gómez-Huelgas R
   25. Ramos-Rincón JM
   26. SEMI-COVID-19 Network

   (2021) Severity Scores in COVID-19 Pneumonia: a Multicenter, Retrospective, Cohort Study

   J Gen Intern Med 36:1338–1345.

   https://doi.org/10.1007/s11606-021-06626-7

   • PubMed
   • Google Scholar
2. 1. Avouac J
   2. Drumez E
   3. Hachulla E
   4. Seror R
   5. Georgin-Lavialle S
   6. El Mahou S
   7. Pertuiset E
   8. Pham T
   9. Marotte H
   10. Servettaz A
   11. Domont F
   12. Chazerain P
   13. Devaux M
   14. Claudepierre P
   15. Langlois V
   16. Mekinian A
   17. Maria ATJ
   18. Banneville B
   19. Fautrel B
   20. Pouchot J
   21. Thomas T
   22. Flipo R-M
   23. Richez C
   24. FAI2R/SFR/SNFMI/SOFREMIP/CRI/IMIDIATE consortium and contributors
   25. FAIR/SFR/SNFMI/SOFREMIP/CRI/IMIDIATE consortium and contributors

   (2021) COVID-19 outcomes in patients with inflammatory rheumatic and musculoskeletal diseases treated with rituximab: a cohort study

   Lancet Rheumatol 3:e419–e426.

   https://doi.org/10.1016/S2665-9913(21)00059-X

   • PubMed
   • Google Scholar
3. 1. Bohn MK
   2. Lippi G
   3. Horvath A
   4. Sethi S
   5. Koch D
   6. Ferrari M
   7. Wang CB
   8. Mancini N
   9. Steele S
   10. Adeli K

   (2020) Molecular, serological, and biochemical diagnosis and monitoring of COVID-19: IFCC taskforce evaluation of the latest evidence

   Clinical Chemistry and Laboratory Medicine 58:1037–1052.

   https://doi.org/10.1515/cclm-2020-0722

   • PubMed
   • Google Scholar
4. 1. Bonetti G
   2. Manelli F
   3. Patroni A
   4. Bettinardi A
   5. Borrelli G
   6. Fiordalisi G
   7. Marino A
   8. Menolfi A
   9. Saggini S
   10. Volpi R
   11. Anesi A
   12. Lippi G

   (2020) Laboratory predictors of death from coronavirus disease 2019 (COVID-19) in the area of Valcamonica, Italy

   Clinical Chemistry and Laboratory Medicine 58:1100–1105.

   https://doi.org/10.1515/cclm-2020-0459

   • Google Scholar
5. 1. Borghesi A
   2. Maroldi R

   (2020) COVID-19 outbreak in Italy: experimental chest X-ray scoring system for quantifying and monitoring disease progression

   La Radiologia Medica 125:509–513.

   https://doi.org/10.1007/s11547-020-01200-3

   • PubMed
   • Google Scholar
6. 1. Borghesi A
   2. Zigliani A
   3. Golemi S
   4. Carapella N
   5. Maculotti P
   6. Farina D
   7. Maroldi R

   (2020a) Chest X-ray severity index as a predictor of in-hospital mortality in coronavirus disease 2019: A study of 302 patients from Italy

   International Journal of Infectious Diseases 96:291–293.

   https://doi.org/10.1016/j.ijid.2020.05.021

   • PubMed
   • Google Scholar
7. 1. Borghesi A
   2. Zigliani A
   3. Masciullo R
   4. Golemi S
   5. Maculotti P
   6. Farina D
   7. Maroldi R

   (2020b) Radiographic severity index in COVID-19 pneumonia: relationship to age and sex in 783 Italian patients

   La Radiologia Medica 125:461–464.

   https://doi.org/10.1007/s11547-020-01202-1

   • PubMed
   • Google Scholar
8. 1. Borghi MO
   2. Beltagy A
   3. Garrafa E
   4. Curreli D
   5. Cecchini G
   6. Bodio C
   7. Grossi C
   8. Blengino S
   9. Tincani A
   10. Franceschini F
   11. Andreoli L
   12. Lazzaroni MG
   13. Piantoni S
   14. Masneri S
   15. Crisafulli F
   16. Brugnoni D
   17. Muiesan ML
   18. Salvetti M
   19. Parati G
   20. Torresani E
   21. Mahler M
   22. Heilbron F
   23. Pregnolato F
   24. Pengo M
   25. Tedesco F
   26. Pozzi N
   27. Meroni PL

   (2020) Anti-Phospholipid Antibodies in COVID-19 Are Different From Those Detectable in the Anti-Phospholipid Syndrome

   Frontiers in Immunology 11:584241.

   https://doi.org/10.3389/fimmu.2020.584241

   • PubMed
   • Google Scholar
9. 1. Breiman L

   (2001) Random Forests

   Machine Learning 45:5–32.

   https://doi.org/10.1023/A:1010933404324

   • Google Scholar
10. 1. Caricchio R
    2. Gallucci M
    3. Dass C
    4. Zhang X
    5. Gallucci S
    6. Fleece D
    7. Bromberg M
    8. Criner GJ
    9. Temple University COVID-19 Research Group

    (2021) Preliminary predictive criteria for COVID-19 cytokine storm

    Annals of the Rheumatic Diseases 80:88–95.

    https://doi.org/10.1136/annrheumdis-2020-218323

    • PubMed
    • Google Scholar
11. 1. Carpita M
    2. Vezzoli M

    (2012) Statistical evidence of the subjective work quality: the fairness drivers of the job satisfaction

    Electron J Appl Stat Anal 5:89–107.

    https://doi.org/10.1285/i20705948v5n1p89

    • Google Scholar
12. 1. Chawla NV
    2. Bowyer KW
    3. Hall LO
    4. Kegelmeyer WP

    (2002) SMOTE: Synthetic Minority Over-sampling Technique

    Journal of Artificial Intelligence Research 16:321–357.

    https://doi.org/10.1613/jair.953

    • Google Scholar
13. 1. Codenotti S
    2. Vezzoli M
    3. Poliani PL
    4. Cominelli M
    5. Bono F
    6. Kabbout H
    7. Faggi F
    8. Chiarelli N
    9. Colombi M
    10. Zanella I
    11. Biasiotto G
    12. Montanelli A
    13. Caimi L
    14. Monti E
    15. Fanzani A

    (2016) Caveolin-1, Caveolin-2 and Cavin-1 are strong predictors of adipogenic differentiation in human tumors and cell lines of liposarcoma

    European Journal of Cell Biology 95:252–264.

    https://doi.org/10.1016/j.ejcb.2016.04.005

    • PubMed
    • Google Scholar
14. 1. Collins GS
    2. Reitsma JB
    3. Altman DG
    4. Moons KGM

    (2015) Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement

    BMJ 350:g7594.

    https://doi.org/10.1136/bmj.g7594

    • PubMed
    • Google Scholar
15. Book

    1. Dancelli L
    2. Manisera M
    3. Vezzoli M

    (2013) On Two Classes of Weighted Rank Correlation Measures Deriving from the Spearman’s ρ In

    In: Giudici P, Ingrassia S, Vichi M, editors. Statistical Models for Data Analysis, Studies in Classification, Data Analysis, and Knowledge Organization. Heidelberg: Springer International Publishing. pp. 107–114.

    https://doi.org/10.1007/978-3-319-00032-9_13

    • Google Scholar
16. 1. Doglietto F
    2. Vezzoli M
    3. Biroli A
    4. Saraceno G
    5. Zanin L
    6. Pertichetti M
    7. Calza S
    8. Agosti E
    9. Aliaga Arias JM
    10. Assietti R
    11. Bellocchi S
    12. Bernucci C
    13. Bistazzoni S
    14. Bongetta D
    15. Fanti A
    16. Fioravanti A
    17. Fiorindi A
    18. Franzin A
    19. Locatelli D
    20. Pugliese R
    21. Roca E
    22. Sicuri GM
    23. Stefini R
    24. Venturini M
    25. Vivaldi O
    26. Zattra C
    27. Zoia C
    28. Fontanella MM

    (2020a) Anxiety in neurosurgical patients undergoing nonurgent surgery during the COVID-19 pandemic

    Neurosurgical Focus 49:E19.

    https://doi.org/10.3171/2020.9.FOCUS20681

    • Google Scholar
17. 1. Doglietto F
    2. Vezzoli M
    3. Gheza F
    4. Lussardi GL
    5. Domenicucci M
    6. Vecchiarelli L
    7. Zanin L
    8. Saraceno G
    9. Signorini L
    10. Panciani PP
    11. Castelli F
    12. Maroldi R
    13. Rasulo FA
    14. Benvenuti MR
    15. Portolani N
    16. Bonardelli S
    17. Milano G
    18. Casiraghi A
    19. Calza S
    20. Fontanella MM

    (2020b) Factors Associated With Surgical Mortality and Complications Among Patients With and Without Coronavirus Disease 2019 (COVID-19) in Italy

    JAMA Surgery 155:691–702.

    https://doi.org/10.1001/jamasurg.2020.2713

    • PubMed
    • Google Scholar
18. 1. Friedman JH

    (2001) Greedy function approximation: A gradient boosting machine

    The Annals of Statistics 29:1189–1232.

    https://doi.org/10.1214/aos/1013203451

    • Google Scholar
19. 1. Garrafa E
    2. Brugnoni D
    3. Barbaro M
    4. Andreoli L
    5. Focà E
    6. Salvetti M
    7. Castelli F
    8. Franceschini F
    9. Piva S
    10. Muiesan ML
    11. Latronico N
    12. Levaggi R

    (2020a) Laboratory considerations amidst the coronavirus disease 2019 outbreak: the Spedali Civili in Brescia experience

    Bioanalysis 12:1223–1230.

    https://doi.org/10.4155/bio-2020-0109

    • PubMed
    • Google Scholar
20. 1. Garrafa E
    2. Levaggi R
    3. Miniaci R
    4. Paolillo C

    (2020b) When fear backfires: Emergency department accesses during the Covid-19 pandemic

    Health Policy 124:1333–1339.

    https://doi.org/10.1016/j.healthpol.2020.10.006

    • PubMed
    • Google Scholar
21. Software

    1. Garrafa E
    2. Vezzoli M
    3. Ravanelli M
    4. Farina D
    5. Borghesi A
    6. Calza S
    7. Maroldi R

    (2021) BS_EWS, version swh:1:rev:7416ba71075402e6a0ed997e7aa6a527e93247b2

    Software Heritage.

    https://archive.softwareheritage.org/swh:1:dir:a6165bf6958e6ebdf264bdf16186bd9e0c59438d;origin=https://github.com/biostatUniBS/BS_EWS;visit=swh:1:snp:6f1c9a504371377ab81e69db8b1f3f18e695485d;anchor=swh:1:rev:7416ba71075402e6a0ed997e7aa6a527e93247b2
22. 1. Genc Yavuz B
    2. Colak S
    3. Guven R
    4. Altundag İ
    5. Seyhan AU
    6. Gunay Inanc R

    (2021) Clinical Features of the 60 Years and Older Patients Infected with 2019 Novel Coronavirus: Can We Predict Mortality Earlier?

    Gerontology 67:433–440.

    https://doi.org/10.1159/000514481

    • PubMed
    • Google Scholar
23. 1. Gidari A
    2. De Socio GV
    3. Sabbatini S
    4. Francisci D

    (2020) Predictive value of National Early Warning Score 2 (NEWS2) for intensive care unit admission in patients with SARS-CoV-2 infection

    Infectious Diseases 52:698–704.

    https://doi.org/10.1080/23744235.2020.1784457

    • Google Scholar
24. 1. Hong S
    2. Lynn HS

    (2020) Accuracy of random-forest-based imputation of missing data in the presence of non-normality, non-linearity, and interaction

    BMC Medical Research Methodology 20:199.

    https://doi.org/10.1186/s12874-020-01080-1

    • PubMed
    • Google Scholar
25. 1. Knight SR
    2. Ho A
    3. Pius R
    4. Buchan I
    5. Carson G
    6. Drake TM
    7. Dunning J
    8. Fairfield CJ
    9. Gamble C
    10. Green CA
    11. Gupta R
    12. Halpin S
    13. Hardwick HE
    14. Holden KA
    15. Horby PW
    16. Jackson C
    17. Mclean KA
    18. Merson L
    19. Nguyen-Van-Tam JS
    20. Norman L
    21. Noursadeghi M
    22. Olliaro PL
    23. Pritchard MG
    24. Russell CD
    25. Shaw CA
    26. Sheikh A
    27. Solomon T
    28. Sudlow C
    29. Swann OV
    30. Turtle LC
    31. Openshaw PJ
    32. Baillie JK
    33. Semple MG
    34. Docherty AB
    35. Harrison EM
    36. ISARIC4C investigators

    (2020) Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: development and validation of the 4C Mortality Score

    BMJ 370:m3339.

    https://doi.org/10.1136/bmj.m3339

    • PubMed
    • Google Scholar
26. 1. Kyriazopoulou E
    2. Panagopoulos P
    3. Metallidis S
    4. Dalekos GN
    5. Poulakou G
    6. Gatselis N
    7. Karakike E
    8. Saridaki M
    9. Loli G
    10. Stefos A
    11. Prasianaki D
    12. Georgiadou S
    13. Tsachouridou O
    14. Petrakis V
    15. Tsiakos K
    16. Kosmidou M
    17. Lygoura V
    18. Dareioti M
    19. Milionis H
    20. Papanikolaou IC
    21. Akinosoglou K
    22. Myrodia DM
    23. Gravvani A
    24. Stamou A
    25. Gkavogianni T
    26. Katrini K
    27. Marantos T
    28. Trontzas IP
    29. Syrigos K
    30. Chatzis L
    31. Chatzis S
    32. Vechlidis N
    33. Avgoustou C
    34. Chalvatzis S
    35. Kyprianou M
    36. van der Meer JW
    37. Eugen-Olsen J
    38. Netea MG
    39. Giamarellos-Bourboulis EJ

    (2021) An open label trial of anakinra to prevent respiratory failure in COVID-19

    eLife 10:e66125.

    https://doi.org/10.7554/eLife.66125

    • PubMed
    • Google Scholar
27. 1. Lazar Neto F
    2. Marino LO
    3. Torres A
    4. Cilloniz C
    5. Meirelles Marchini JF
    6. Garcia de Alencar JC
    7. Palomeque A
    8. Albacar N
    9. Brandão Neto RA
    10. Souza HP
    11. Ranzani OT
    12. COVID Registry Team; Medical Students; Residents; Attending physicians

    (2021) Community-acquired pneumonia severity assessment tools in patients hospitalized with COVID-19: a validation and clinical applicability study

    Clinical Microbiology and Infection 27:1037.

    https://doi.org/10.1016/j.cmi.2021.03.002

    • PubMed
    • Google Scholar
28. 1. Lazzaroni MG
    2. Piantoni S
    3. Masneri S
    4. Garrafa E
    5. Martini G
    6. Tincani A
    7. Andreoli L
    8. Franceschini F

    (2021) Coagulation dysfunction in COVID-19: The interplay between inflammation, viral infection and the coagulation system

    Blood Reviews 46:100745.

    https://doi.org/10.1016/j.blre.2020.100745

    • PubMed
    • Google Scholar
29. 1. Liang W
    2. Liang H
    3. Ou L
    4. Chen B
    5. Chen A
    6. Li C
    7. Li Y
    8. Guan W
    9. Sang L
    10. Lu J
    11. Xu Y
    12. Chen G
    13. Guo H
    14. Guo J
    15. Chen Z
    16. Zhao Y
    17. Li S
    18. Zhang N
    19. Zhong N
    20. He J

    (2020) Development and Validation of a Clinical Risk Score to Predict the Occurrence of Critical Illness in Hospitalized Patients With COVID-19

    JAMA Internal Medicine 180:1081–1089.

    https://doi.org/10.1001/jamainternmed.2020.2033

    • Google Scholar
30. 1. Linssen J
    2. Ermens A
    3. Berrevoets M
    4. Seghezzi M
    5. Previtali G
    6. van der Sar-van der Brugge S
    7. Russcher H
    8. Verbon A
    9. Gillis J
    10. Riedl J
    11. de Jongh E
    12. Saker J
    13. Münster M
    14. Munnix IC
    15. Dofferhof A
    16. Scharnhorst V
    17. Ammerlaan H
    18. Deiteren K
    19. Bakker SJ
    20. Van Pelt LJ
    21. Kluiters-de Hingh Y
    22. Leers MP
    23. van der Ven AJ

    (2020) A novel haemocytometric COVID-19 prognostic score developed and validated in an observational multicentre European hospital-based study

    eLife 9:e63195.

    https://doi.org/10.7554/eLife.63195

    • PubMed
    • Google Scholar
31. 1. Lippi G
    2. Henry BM
    3. Hoehn J
    4. Benoit S
    5. Benoit J

    (2020) Validation of the Corona-Score for rapid identification of SARS-CoV-2 infections in patients seeking emergency department care in the United States

    Clinical Chemistry and Laboratory Medicine 58:e311–e313.

    https://doi.org/10.1515/cclm-2020-1121

    • Google Scholar
32. 1. Marengoni A
    2. Zucchelli A
    3. Vetrano DL
    4. Armellini A
    5. Botteri E
    6. Nicosia F
    7. Romanelli G
    8. Beindorf EA
    9. Giansiracusa P
    10. Garrafa E
    11. Ferrucci L
    12. Fratiglioni L
    13. Bernabei R
    14. Onder G

    (2021) Beyond Chronological Age: Frailty and Multimorbidity Predict In-Hospital Mortality in Patients With Coronavirus Disease 2019

    The Journals of Gerontology. Series A, Biological Sciences and Medical Sciences 76:e38–e45.

    https://doi.org/10.1093/gerona/glaa291

    • PubMed
    • Google Scholar
33. 1. Maroldi R
    2. Rondi P
    3. Agazzi GM
    4. Ravanelli M
    5. Borghesi A
    6. Farina D

    (2021) Which role for chest x-ray score in predicting the outcome in COVID-19 pneumonia?

    European Radiology 31:4016–4022.

    https://doi.org/10.1007/s00330-020-07504-2

    • PubMed
    • Google Scholar
34. 1. Martens RJH
    2. van Adrichem AJ
    3. Mattheij NJA
    4. Brouwer CG
    5. van Twist DJL
    6. Broerse JJCR
    7. Magro-Checa C
    8. van Dongen CMP
    9. Mostard RLM
    10. Ramiro S
    11. Landewé RBM
    12. Leers MPG

    (2021) Hemocytometric characteristics of COVID-19 patients with and without cytokine storm syndrome on the sysmex XN-10 hematology analyzer

    Clinical Chemistry and Laboratory Medicine 59:783–793.

    https://doi.org/10.1515/cclm-2020-1529

    • Google Scholar
35. 1. Marziano M
    2. Tonello S
    3. Cantù E
    4. Abate G
    5. Vezzoli M
    6. Rungratanawanich W
    7. Serpelloni M
    8. Lopomo NF
    9. Memo M
    10. Sardini E
    11. Uberti D

    (2019) Monitoring Caco-2 to enterocyte-like cells differentiation by means of electric impedance analysis on printed sensors

    Biochimica et Biophysica Acta. General Subjects 1863:893–902.

    https://doi.org/10.1016/j.bbagen.2019.02.008

    • PubMed
    • Google Scholar
36. 1. Myrstad M
    2. Ihle-Hansen H
    3. Tveita AA
    4. Andersen EL
    5. Nygård S
    6. Tveit A
    7. Berge T

    (2020) National Early Warning Score 2 (NEWS2) on admission predicts severe disease and in-hospital mortality from Covid-19 - a prospective cohort study

    Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine 28:66.

    https://doi.org/10.1186/s13049-020-00764-3

    • PubMed
    • Google Scholar
37. Software

    1. R Development Core Team

    (2020) R: A language and environment for statistical computing

    R Foundation for Statistical Computing, Vienna, Austria.

    https://www.R-project.org/
38. 1. Salvi A
    2. Vezzoli M
    3. Busatto S
    4. Paolini L
    5. Faranda T
    6. Abeni E
    7. Caracausi M
    8. Antonaros F
    9. Piovesan A
    10. Locatelli C
    11. Cocchi G
    12. Alvisi G
    13. De Petro G
    14. Ricotta D
    15. Bergese P
    16. Radeghieri A

    (2019) Corrigendum] Analysis of a nanoparticle‑enriched fraction of plasma reveals miRNA candidates for Down syndrome pathogenesis

    Ternational Journal of Molecular Medicine 44:768.

    https://doi.org/10.3892/ijmm.2019.4222

    • PubMed
    • Google Scholar
39. 1. Schalekamp S
    2. Huisman M
    3. van Dijk RA
    4. Boomsma MF
    5. Freire Jorge PJ
    6. de Boer WS
    7. Herder GJM
    8. Bonarius M
    9. Groot OA
    10. Jong E
    11. Schreuder A
    12. Schaefer-Prokop CM

    (2021) Model-based Prediction of Critical Illness in Hospitalized Patients with COVID-19

    Radiology 298:E46–E54.

    https://doi.org/10.1148/radiol.2020202723

    • PubMed
    • Google Scholar
40. 1. Sperrin M
    2. Grant SW
    3. Peek N

    (2020) Prediction models for diagnosis and prognosis in Covid-19

    BMJ 369:m1464.

    https://doi.org/10.1136/bmj.m1464

    • PubMed
    • Google Scholar
41. 1. Vezzoli M

    (2011) Exploring the facets of overall job satisfaction through a novel ensemble learning

    Electron J Appl Stat Anal 4:23–38.

    https://doi.org/10.1285/i20705948v4n1p23

    • Google Scholar
42. 1. Vezzoli M
    2. Ravaggi A
    3. Zanotti L
    4. Miscioscia RA
    5. Bignotti E
    6. Ragnoli M
    7. Gambino A
    8. Ruggeri G
    9. Calza S
    10. Sartori E
    11. Odicino F

    (2017) RERT: A Novel Regression Tree Approach to Predict Extrauterine Disease in Endometrial Carcinoma Patients

    Scientific Reports 7:10528.

    https://doi.org/10.1038/s41598-017-11104-4

    • PubMed
    • Google Scholar
43. 1. Webb BJ
    2. Peltan ID
    3. Jensen P
    4. Hoda D
    5. Hunter B
    6. Silver A
    7. Starr N
    8. Buckel W
    9. Grisel N
    10. Hummel E
    11. Snow G
    12. Morris D
    13. Stenehjem E
    14. Srivastava R
    15. Brown SM

    (2020) Clinical criteria for COVID-19-associated hyperinflammatory syndrome: a cohort study

    The Lancet. Rheumatology 2:e754–e763.

    https://doi.org/10.1016/S2665-9913(20)30343-X

    • PubMed
    • Google Scholar
44. 1. Wynants L
    2. Van Calster B
    3. Collins GS
    4. Riley RD
    5. Heinze G
    6. Schuit E
    7. Bonten MMJ
    8. Dahly DL
    9. Damen JAA
    10. Debray TPA
    11. de Jong VMT
    12. De Vos M
    13. Dhiman P
    14. Haller MC
    15. Harhay MO
    16. Henckaerts L
    17. Heus P
    18. Kammer M
    19. Kreuzberger N
    20. Lohmann A
    21. Luijken K
    22. Ma J
    23. Martin GP
    24. McLernon DJ
    25. Andaur Navarro CL
    26. Reitsma JB
    27. Sergeant JC
    28. Shi C
    29. Skoetz N
    30. Smits LJM
    31. Snell KIE
    32. Sperrin M
    33. Spijker R
    34. Steyerberg EW
    35. Takada T
    36. Tzoulaki I
    37. van Kuijk SMJ
    38. van Bussel B
    39. van der Horst ICC
    40. van Royen FS
    41. Verbakel JY
    42. Wallisch C
    43. Wilkinson J
    44. Wolff R
    45. Hooft L
    46. Moons KGM
    47. van Smeden M

    (2020) Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal

    BMJ 369:m1328.

    https://doi.org/10.1136/bmj.m1328

    • PubMed
    • Google Scholar

Author details

1. Emirena Garrafa

   1. Department of Molecular and Translational Medicine, University of Brescia, Brescia, Italy
   2. ASST Spedali Civili di Brescia, Department of Laboratory, Brescia, Italy

   Contribution

   Conceptualization, Data curation, Investigation, Methodology, Resources, Supervision, Validation, Writing – original draft, Writing – review and editing

   Contributed equally with

   Marika Vezzoli

   For correspondence

   emirena.garrafa@unibs.it

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0003-4761-6892

2. Marika Vezzoli

   Department of Molecular and Translational Medicine, University of Brescia, Brescia, Italy

   Contribution

   Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Supervision, Writing – original draft, Writing – review and editing

   Contributed equally with

   Emirena Garrafa

   Competing interests

   No competing interests declared

3. Marco Ravanelli

   1. Department of Medical and Surgical Specialties, Radiological Sciences and Public Health, University of Brescia, Brescia, Italy
   2. ASST Spedali Civili di Brescia, Department of Radiology, Brescia, Italy

   Contribution

   Conceptualization, Data curation, Supervision, Writing – review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-4841-2771

4. Davide Farina

   1. Department of Medical and Surgical Specialties, Radiological Sciences and Public Health, University of Brescia, Brescia, Italy
   2. ASST Spedali Civili di Brescia, Department of Radiology, Brescia, Italy

   Contribution

   Supervision, Writing – review and editing

   Competing interests

   No competing interests declared

5. Andrea Borghesi

   1. Department of Medical and Surgical Specialties, Radiological Sciences and Public Health, University of Brescia, Brescia, Italy
   2. ASST Spedali Civili di Brescia, Department of Radiology, Brescia, Italy

   Contribution

   Validation, Writing – review and editing

   Competing interests

   No competing interests declared

6. Stefano Calza

   Department of Molecular and Translational Medicine, University of Brescia, Brescia, Italy

   Contribution

   Conceptualization, Data curation, Funding acquisition, Software, Supervision, Writing – review and editing

   Competing interests

   No competing interests declared

7. Roberto Maroldi

   1. Department of Medical and Surgical Specialties, Radiological Sciences and Public Health, University of Brescia, Brescia, Italy
   2. ASST Spedali Civili di Brescia, Department of Radiology, Brescia, Italy

   Contribution

   Conceptualization, Data curation, Supervision, Writing – original draft, Writing – review and editing

   Competing interests

   No competing interests declared

• 985

  views

• 132

  downloads

• 22

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.