External validation of postnatal gestational age estimation using newborn metabolic profiles in Matlab, Bangladesh

Complications from preterm birth are the leading cause of death among children under five. Ultrasounds are routinely used in wealthy countries to track babies' development. In countries with limited resources, however, ultrasounds are rare, making it harder to estimate how many children are born prematurely. Blood tests may offer a way to determine whether a newborn was born too early when ultrasounds are not available. Many countries already require clinicians to collect a drop of blood from newborns via a heel-prick or from their umbilical cord. Testing these blood spots identifies babies at risk of rare conditions so they can receive prompt treatment.

Chemicals in the blood vary depending on how long the newborn spent growing in its mother’s womb. Scientists have developed a mathematical formula that can estimate a baby’s gestational age based on these chemicals. Using blood spots to estimate gestational age worked well when this strategy was tested in Canada, a high-income country. More tests are needed to determine if it works in low-income countries.

Now, Murphy et al. show their blood spot-testing strategy also reliably predicts the gestational age of babies in Matlab, Bangladesh. In the experiments, blood spots were collected from 1,069 newborns. This included 1,036 cord blood samples and 487 heel prick samples. Nearly all the samples came from full-term infants. A mathematical model estimated the infants' gestational age to within an average of one week of their true age when applied to heel-prick blood samples and to within two weeks of the baby’s true gestational age 94% of the time.

The model also provided reliable estimates of babies’ gestational ages when cord blood samples were tested, which is useful as the Bangladeshi parents were more comfortable with this method of blood collection. Using this strategy to estimate how many babies are born too early in low-income countries may help the countries develop strategies to reduce preterm births. The estimates might also help identify preterm babies who need special care.

Complications related to preterm birth are the leading cause of death among children under 5 years of age (March of Dimes, 2012). Estimating the burden of preterm birth in low-resource settings is challenging due to the absence of ultrasound technology and the unreliability of recall of last menstrual period. Commonly used estimates obtained late in gestation or postnatally (e.g. fundal height, and Ballard or Dubowitz scores) are subject to high inter-user variability and poor reliability in small for gestational age and preterm infants (Taylor et al., 2010; Spinnato et al., 1984; Robillard et al., 1992). In addition, data on preterm birth are not routinely documented in some countries and may not be classified according to international standards (Quinn et al., 2016), thus impeding the development of strategies for resource allocation to support global and local health initiatives. Strengthened data surveillance systems to more accurately assess and track changes in preterm birth across jurisdictions are urgently required (March of Dimes, 2012).

Algorithms based on newborn metabolic profiles in combination with clinical covariates such as sex and birthweight have demonstrated the potential to accurately categorize infants across preterm birth categories in high-resource settings (Jelliffe-Pawlowski et al., 2016; Ryckman et al., 2016; Wilson et al., 2016). Data from newborn screening programs in North America have been used to create models capable of estimating gestational age to within 1–2 weeks, but their performance among other infant populations is uncertain. Recent work has focused on refining these models and tailoring them for use across a range of environments and sub-populations, and has suggested that while the models perform well among infants from a variety of backgrounds, ethnicity-specific models may improve the models’ performance (Wilson et al., 2017; Hawken et al., 2017). More recent model iterations have been strenthened by the addition of variables such as newborn hemoglobin peak percentages (calculated from the ratio of fetal to adult hemoglobin levels), which have demonstrated strong associations with gestational age (Wilson et al., 2017). While these algorithms have the potential to provide reliable population estimates of preterm birth burden where prenatal ultrasound data are not available, the models’ generalizability to all infant populations, as well as the feasibility of collecting samples for analysis in low-resource settings, is uncertain. In this paper, we explore the performance of gestational age estimation models in an infant population born in Matlab, Bangladesh. We also comment on the effect of timing of sample collection on newborn metabolic profiles and the feasibility of newborn blood sample collection and analysis in this setting.

Performance of gestational age estimation models using heel prick data

We determined the performance of previously published metabolic gestational dating algorithms in heel prick-derived data from the Bangladeshi infant cohort. Results of linear regression analyses for heel prick metabolic profiles demonstrated optimal performance among term infants between 38 and 39 completed gestational weeks (Figure 1). Residual plots for each of the three models in both heel and cord samples are provided in Figure 2. In general, all models predicted gestational ages close to full term with the highest accuracy, while tending to overestimate gestational age in preterm infants and underestimate gestational age in post-term infants, in the Bangladesh cohort.

Figure 1

Download asset Open asset

Figure 2

Download asset Open asset

A baseline model including only clinical covariates (infant sex, birthweight and multiple birth status, Model 1) provided the least accurate estimation of gestational age relative to ultrasound-validated gestational age estimates, RMSE 1.46 weeks. By comparison, a model including analyte covariates (Model 2) had an RMSE of 1.35 weeks. A full model containing all clinical and analyte data (Model 3) demonstrated the lowest RMSE (best performance) of 1.07 weeks and correctly estimated gestational age to within 1 week for 63.9%, and within 2 weeks for 94.3% of all heel prick samples. Among small for gestational age infants, the full heel prick model had an RMSE of 1.12 weeks when growth restriction was defined as birthweight below the 10th percentile for gestational age and an RMSE of 1.30 weeks when defined as birthweight below the 3rd percentile for gestational age. By these definitions, our model accurately estimated gestational age to within 1 week for 62.8% and 53.4% of growth-restricted infants, respectively.

Dichotomous discrimination of gestational age

We evaluated the discrimination of gestational age across a dichotomous preterm birth threshold (≥37 weeks vs <37 weeks gestational age) (Figure 3). Gestational age estimation models performed best when applied to metabolic profiles derived from heel prick samples. For both types of samples, the best performance was achieved by the full model containing all clinical and analyte data (Model 3) (area under the curve [AUC] 0.945 (95% CI 0.890, 0.999) for heel prick profiles and AUC 0.894 (95% CI 0.853, 0.935) for cord blood profiles).

Figure 3

Download asset Open asset

Table 3

	AUC (lower, upper 95% confidence limits),
	A) Model 1: Sex, Multiple Birth Status, Birthweight Model	B) Model 2: Analytes, Sex, Multiple Birth Status Model	C) Model 3: Full Model
	0.840 (0.754, 0.925)	0.895 (0.823, 0.968)	0.945 (0.890, 0.999)
Bangladesh Cord	0.806 (0.755, 0.858)	0.823 (0.773, 0.873)	0.894 (0.853, 0.935)
Ontario Reference (Wilson et al., 2017)	0.915 (0.909, 0.921)	0.946 (0.941, 0.952)	0.967 (0.963, 0.971)

Data cannot be made publicly available according to our institutional guidelines on human subject research, but anonymized data and materials can be made available upon request to Dr Wilson and Dr Hawken. These requests will not need to be reviewed by our ethics board, but should be provided in writing. Numerical data files for Figures 1 and 2 are included as Source data 1.

1. 1. Alexander GR
   2. de Caunes F
   3. Hulsey TC
   4. Tompkins ME
   5. Allen M

   (1992) Validity of postnatal assessments of gestational age: a comparison of the method of Ballard et al. and early ultrasonography

   American Journal of Obstetrics and Gynecology 166:891–895.

   https://doi.org/10.1016/0002-9378(92)91357-G

   • PubMed
   • Google Scholar
2. 1. Büyükgebiz A

   (2013) Newborn screening for congenital hypothyroidism

   Journal of Clinical Research in Pediatric Endocrinology 5:8–12.

   https://doi.org/10.4274/jcrpe.845

   • PubMed
   • Google Scholar
3. 1. Campbell OM
   2. Cegolon L
   3. Macleod D
   4. Benova L

   (2016) Length of stay after childbirth in 92 countries and associated factors in 30 low- and Middle-Income countries: compilation of reported data and a Cross-sectional analysis from nationally representative surveys

   PLOS Medicine 13:e1001972.

   https://doi.org/10.1371/journal.pmed.1001972

   • PubMed
   • Google Scholar
4. 1. Committee on Obstetric Practice, the American Institute of Ultrasound in Medicine, and the Society for Maternal-Fetal Medicine

   (2017) Committee opinion no 700: methods for estimating the due date

   Obstetrics and Gynecology 129:e150–e154.

   https://doi.org/10.1097/AOG.0000000000002046

   • PubMed
   • Google Scholar
5. 1. Goto E

   (2013) Prediction of low birthweight and small for gestational age from symphysis-fundal height mainly in developing countries: a meta-analysis

   Journal of Epidemiology and Community Health 67:999–1005.

   https://doi.org/10.1136/jech-2012-202141

   • PubMed
   • Google Scholar
6. 1. Harrell FE

   (2001)

   Regression Modeling Strategies

   with Applications to Linear Models, Logistic and Ordinal Regression and Survival Analysis, Regression Modeling Strategies, Springer.

   • Google Scholar
7. 1. Hawken S
   2. Ducharme R
   3. Murphy MSQ
   4. Atkinson KM
   5. Potter BK
   6. Chakraborty P
   7. Wilson K

   (2017) Performance of a postnatal metabolic gestational age algorithm: a retrospective validation study among ethnic subgroups in Canada

   BMJ Open 7:e015615.

   https://doi.org/10.1136/bmjopen-2016-015615

   • PubMed
   • Google Scholar
8. 1. Jelliffe-Pawlowski LL
   2. Norton ME
   3. Baer RJ
   4. Santos N
   5. Rutherford GW

   (2016) Gestational dating by metabolic profile at birth: a California cohort study

   American Journal of Obstetrics and Gynecology 214:511.e1–511.e13.

   https://doi.org/10.1016/j.ajog.2015.11.029

   • Google Scholar
9. 1. Lee AC
   2. Mullany LC
   3. Ladhani K
   4. Uddin J
   5. Mitra D
   6. Ahmed P
   7. Christian P
   8. Labrique A
   9. DasGupta SK
   10. Lokken RP
   11. Quaiyum M
   12. Baqui AH
   13. Projahnmo Study Group

   (2016) Validity of newborn clinical assessment to determine gestational age in Bangladesh

   Pediatrics 138:e20153303.

   https://doi.org/10.1542/peds.2015-3303

   • PubMed
   • Google Scholar
10. 1. March of Dimes

    (2012)

    Partnership for maternal newborn and child health, save the children & WHO

    Born Too Soon: The Global Action Report on Preterm Birth.

    • Google Scholar
11. 1. Moons KG
    2. Donders RA
    3. Stijnen T
    4. Harrell FE

    (2006) Using the outcome for imputation of missing predictor values was preferred

    Journal of Clinical Epidemiology 59:1092–1101.

    https://doi.org/10.1016/j.jclinepi.2006.01.009

    • PubMed
    • Google Scholar
12. 1. Mundel T

    (2017)

    Innovation: how a 50-Year-Old drop of blood helps solve an urgent global health challenge

    Impatient Optimists.

    • Google Scholar
13. 1. Murphy MSQ
    2. Hawken S
    3. Atkinson KM
    4. Milburn J
    5. Pervin J
    6. Gravett C
    7. Stringer JSA
    8. Rahman A
    9. Lackritz E
    10. Chakraborty P
    11. Wilson K

    (2017) Postnatal gestational age estimation using newborn screening blood spots: a proposed validation protocol

    BMJ Global Health 2:e000365.

    https://doi.org/10.1136/bmjgh-2017-000365

    • PubMed
    • Google Scholar
14. Book

    1. Newborn Screening Ontario

    (2017)

    Newborn Screening Manual: A Guide for Newborn Care Providers

    NSO.

    • Google Scholar
15. 1. Oladipo OO
    2. Weindel AL
    3. Saunders AN
    4. Dietzen DJ

    (2011) Impact of premature birth and critical illness on neonatal range of plasma amino acid concentrations determined by LC-MS/MS

    Molecular Genetics and Metabolism 104:476–479.

    https://doi.org/10.1016/j.ymgme.2011.08.020

    • PubMed
    • Google Scholar
16. 1. Quinn JA
    2. Munoz FM
    3. Gonik B
    4. Frau L
    5. Cutland C
    6. Mallett-Moore T
    7. Kissou A
    8. Wittke F
    9. Das M
    10. Nunes T
    11. Pye S
    12. Watson W
    13. Ramos AA
    14. Cordero JF
    15. Huang WT
    16. Kochhar S
    17. Buttery J
    18. Brighton Collaboration Preterm Birth Working Group

    (2016) Preterm birth: case definition & guidelines for data collection, analysis, and presentation of immunisation safety data

    Vaccine 34:6047–6056.

    https://doi.org/10.1016/j.vaccine.2016.03.045

    • PubMed
    • Google Scholar
17. 1. R core team

    (2017) R: a language and environment for statistical computing

    R: a language and environment for statistical computing, 3.3.2, Vienna Austria, http://www.r-project.org/.

    http://www.r-project.org/

    • Google Scholar
18. 1. Robillard PY
    2. De Caunes F
    3. Alexander GR
    4. Sergent MP

    (1992)

    Validity of postnatal assessments of gestational age in low birthweight infants from a caribbean community 8071

    Journal of Perinatology 12:115–119.

    • PubMed
    • Google Scholar
19. 1. Ryckman KK
    2. Cook DE
    3. Berberich SL
    4. Shchelochkov OA
    5. Berends SK
    6. Busch T
    7. Dagle JM
    8. Murray JC

    (2012) Replication of clinical associations with 17-hydroxyprogesterone in preterm newborns

    Journal of Pediatric Endocrinology and Metabolism 25:301–305.

    https://doi.org/10.1515/jpem-2011-0456

    • PubMed
    • Google Scholar
20. 1. Ryckman KK
    2. Berberich SL
    3. Dagle JM

    (2016) Predicting gestational age using neonatal metabolic markers

    American Journal of Obstetrics and Gynecology 214:515.e1–51515.

    https://doi.org/10.1016/j.ajog.2015.11.028

    • Google Scholar
21. Software

    1. SAS Institute

    (2017) SAS/STAT, version 9.4

    SAS/STAT.

    https://www.sas.com/en_in/software/stat.html
22. 1. Slaughter JL
    2. Meinzen-Derr J
    3. Rose SR
    4. Leslie ND
    5. Chandrasekar R
    6. Linard SM
    7. Akinbi HT

    (2010) The effects of gestational age and birth weight on false-positive newborn-screening rates

    Pediatrics 126:910–916.

    https://doi.org/10.1542/peds.2010-0943

    • PubMed
    • Google Scholar
23. 1. Spinnato JA
    2. Sibai BM
    3. Shaver DC
    4. Anderson GD

    (1984)

    Inaccuracy of Dubowitz gestational age in low birth weight infants

    Obstetrics and Gynecology 63:491–495.

    • PubMed
    • Google Scholar
24. 1. Steyerberg E

    (2010)

    Clinical Prediction Models

    A Practical Approach to Development, Validation and Updating, Clinical Prediction Models, Springer-Verlag.

    • Google Scholar
25. 1. Taylor R
    2. Beyai S
    3. Owens S
    4. Denison F

    (2010) The external ballard examination does not assess gestational age accurately in a rural field setting in the Gambia

    Archives of Disease in Childhood - Fetal and Neonatal Edition 95:Fa103.

    https://doi.org/10.1136/adc.2010.189761.48

    • Google Scholar
26. 1. Therrell BL
    2. Padilla CD
    3. Loeber JG
    4. Kneisser I
    5. Saadallah A
    6. Borrajo GJC
    7. Adams J

    (2015) Current status of newborn screening worldwide: 2015

    Seminars in Perinatology 39:171–187.

    https://doi.org/10.1053/j.semperi.2015.03.002

    • Google Scholar
27. Book

    1. Tukey JW

    (1977)

    Analysis Edata

    Addison‐Wesley Publishing Company.

    • Google Scholar
28. Book

    1. van Buuren S

    (2013)

    Flexible Imputation of Missing Data

    CRC Press.

    • Google Scholar
29. Book

    1. WHO

    (2014)

    Every Newborn Action Plan

    WHO. ISBN 978-92-4-150744-8.

    • Google Scholar
30. 1. Wilson K
    2. Hawken S
    3. Potter BK
    4. Chakraborty P
    5. Walker M
    6. Ducharme R
    7. Little J

    (2016) Accurate prediction of gestational age using newborn screening analyte data

    American Journal of Obstetrics and Gynecology 214:513.e1–515.e13.

    https://doi.org/10.1016/j.ajog.2015.10.017

    • Google Scholar
31. 1. Wilson K
    2. Hawken S
    3. Murphy MSQ
    4. Atkinson KM
    5. Potter BK
    6. Sprague A
    7. Walker M
    8. Chakraborty P
    9. Little J

    (2017) Postnatal prediction of gestational age using newborn fetal hemoglobin levels

    EBioMedicine 15:203–209.

    https://doi.org/10.1016/j.ebiom.2016.11.032

    • PubMed
    • Google Scholar

Thank you for submitting your article "Postnatal gestational age estimation using newborn metabolic profiles in Matlab, Bangladesh" for consideration by eLife. Your article has been reviewed by three peer reviewers, and the evaluation has been overseen by a Reviewing Editor and Prabhat Jha as the Senior Editor. The following individuals involved in review of your submission have agreed to reveal their identity: Jessica Duby (Reviewer #1); Eric Ohuma (Reviewer #2).

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

Summary:

In this manuscript by Murphy et al., the authors study the accuracy of a metabolic based model to determine gestational age for neonates in Bangladesh when compared to a first trimester ultrasound. The authors make a strong case for the potential impact of a successful algorithm in determining population rates of preterm birth and small-for-gestational age in low-resource settings. This is a potentially important study but the description of the study design and execution is far from complete.

In addition, whilst there is no doubt an accurate estimate of GA is important, there are fundamental problems with this approach that is based on postnatal estimation of GA. First, it helps to perpetuate a standard of care that even WHO has abandoned, namely managing pregnancies without knowing the due date. This is the reason the new WHO guidelines recommend at least one ultrasound scan, in part to estimate gestational age prenatally.

In addition, the authors do not properly address the model's ability to distinguish small-for-gestational age (SGA) neonates, which should be one of the model's primary advantages over using birth weight alone as a surrogate marker for gestational age.

Essential revisions:

1) There are deviations from the cited, published protocol that remain unexplained by the authors. (Murphy et al., 2017)

Major differences between published protocol and current manuscript:

a) Sample size: The protocol's sample size requirements are calculated to be 3,500 participants. The current manuscript details 1523 samples from 1069 newborns with no discussion regarding the achieved sample size. As part of addressing sample size differences, it may benefit the authors to present a figure detailing number of eligible participants, number approached for consent, number consented, etc.

b) Models: The proposed models to test in the protocol are quite different than the models tested in the current manuscript and no justification is provided. For reference, the models listed in the protocol are: "1) birth weight alone; 2) combination of birth weight and fetal/adult haemoglobin levels; 3) combination of birth weight, haemoglobin levels, thyroid-stimulating hormone and 17-OHP (all non-mass-spectrometry-derived analytes); and 4) birth weight and the full panel of newborn screening analytes. Sex and multiple birth (yes, no) will be included in all models." (Murphy et al., 2017) For models used in the reported study, refer to the subsection “Validation of algorithms”.

2) SGA/large-for-gestational age (LGA) babies: The authors need to provide further analysis and discussion regarding the algorithm's performance in SGA/LGA identification. Specifically, the authors need to identify how many SGA/LGA neonates were in their sample and the models' accuracy for these important sub-groups. The authors had listed this analysis in their protocol and have previously performed similar sub-group analysis on the original Ontario cohort which can be used as a guide. (Wilson et al., 2016).

3) While the authors do sub-group analyses by birth weight (<2500g and ≥2500g) in Table 2, weight alone is unimportant when we consider the overarching goal of finding a model that can distinguish preterm birth from SGA and post-term birth from LGA. I would recommend replacing the columns under each of the sample types in Table 2 to be:

1) Overall;

2) SGA;

3) Appropriate-for-gestational age (AGA); and

4) LGA.

4) Imputation of missing values: As only 46 (11%) of observations were imputed, did the authors do a sensitivity analysis of the performance of the model with and without the imputed missing values to evaluate their influence on the results?

5) Prediction score (nomogram) – it would be useful for the authors to also show the relative contribution of each factor in their model. For example, in the baseline model with birthweight, infant sex and multiple births, what is the greatest predictor of GA? A quantification of the relative contribution of each of the factors is useful in understanding their contribution and justification as important variables the prediction of GA. Similarly, this should be done for models 2 and 3 as it allows one to make judgements on how much more improvements in prediction is provided for by including the clinical and analyte data. Also, a comparison of scores helps to discern whether some factors included in the model are correlated and thus are similar in their prediction of GA.

6) Would also be helpful for the author to show a plot of true GA vs. predicted GA (calibration) as this will evidently show the variability of the prediction as a function of GA as opposed to the aggregated estimates they have presented by GA in Figure 1.

7) The cohort was apparently nested within a PreSSMat study in Matlab but the authors do not provide key information on timing of ultrasound examination, birth examinations and weight and criteria used to categorize infants as either preterm, small for gestational age (SGA) or combinations thereof. Please add the necessary level of detail to the methodology and Results sections.

8) Given the issues with cord blood specificity and the wide range of times in collection of heel prick samples (from a few hours to up to 40 hours), one is left guessing as to the fidelity of design of the gold standard used in this study and early measures. Please explain when were these measurements obtained?

9) Please explain what standards were used for ultrasound based gestational age and add an appropriate justification for adoption of the chosen standards.

10) The overall rates of prematurity in the cohort determined from the study (both cord and heel prick samples) are implausibly low (given what is known from ANISA and AMANHI studies done in Bangladesh). Please explain the reasons for these differences.

11) The authors should comment on what is known about prematurity incidence from prior studies of maternal antenatal care and nutrition supplementation done in Bangladesh on large rural cohorts including Matlab, without that information, it is difficult to place this study in the context of what else is known.

Summary:

In this manuscript by Murphy et al., the authors study the accuracy of a metabolic based model to determine gestational age for neonates in Bangladesh when compared to a first trimester ultrasound. The authors make a strong case for the potential impact of a successful algorithm in determining population rates of preterm birth and small-for-gestational age in low-resource settings. This is a potentially important study but the description of the study design and execution is far from complete.

In addition, whilst there is no doubt an accurate estimate of GA is important, there are fundamental problems with this approach that is based on postnatal estimation of GA. First, it helps to perpetuate a standard of care that even WHO has abandoned, namely managing pregnancies without knowing the due date. This is the reason the new WHO guidelines recommend at least one ultrasound scan, in part to estimate gestational age prenatally.

Thank you for addressing this point. We agree that managing pregnancies without knowledge of gestational age is not recommended. We therefore do not seek to implement our postnatal estimation models as a clinical tool replace prenatal ultrasound. Rather, we propose that our models could have utility in deriving population-level estimates of preterm birth in geographies or locales where prenatal ultrasound data are frequently available. We have noted in the Introduction that these models are intended for use when ultrasound is not available.

“…these algorithms have the potential to provide reliable population estimates of preterm birth burden when prenatal ultrasound is not available”.

In addition, the authors do not properly address the model's ability to distinguish small-for-gestational age (SGA) neonates which should be one of the model's primary advantages over using birth weight alone as a surrogate marker for gestational age.

Thank you. One of the limitations of our best performing model is its reliance on birthweight as a covariate in estimating gestational age. Including birthweight in the models limits our ability to identify SGA or IUGR infants as the impact of birthweight on gestational age is confounded in these infants.

When birthweight is removed from the model, its performance deteriorates substantially among infants overall, but improves slightly in SGA infants. We have included model performance results for infants with birthweight below the 10^th and 3^rd percentiles for gestational age and sex as defined by a North American population, however a large proportion of these infants are not truly growth restricted in the context of the local Bangladesh population distribution.

Future work will seek to refine our model that excludes birthweight as a covariate in order to improve its utility in these populations. We discuss the utility of our current models with and without birthweight in SGA cases in greater detail below in response to the other SGA-related reviewer questions.

Essential revisions:

1) There are deviations from the cited, published protocol that remain unexplained by the authors. (Murphy et al., 2017)

Major differences between published protocol and current manuscript:

a) Sample size: The protocol's sample size requirements are calculated to be 3,500 participants. The current manuscript details 1523 samples from 1069 newborns with no discussion regarding the achieved sample size. As part of addressing sample size differences, it may benefit the authors to present a figure detailing number of eligible participants, number approached for consent, number consented, etc.

Thank you for giving us the opportunity to address the differences between our published protocol and our completed study. As the reviewer has noted, we ultimately collected and analysed fewer samples than we initially anticipated. This was due to administrative delays at the outset of the project, particularly while the study was reviewed by three research ethics boards in two countries. By the time the study was initiated, there was less time in which to recruit eligible participants before the end of the study. The sample size calculations presented in our protocol were presented across a range of scenarios, and sample sizes of 1500 and 3500 were projections for Zambia and Bangladesh respectively. The projected sample size of 3500 in Bangladesh was based on Power to detect differences in AUCs for models classifying <34 weeks vs. ≥34 weeks GA. We ended up recruiting fewer total subjects and unfortunately almost no infants <34 weeks, so we have only presented results classifying <37 weeks vs. ≥37 weeks. We still have good statistical power to report c-statistics with good precision. For continuous models, sample size estimates were also driven by the desire to have a minimum number of samples to validate using RMSE estimation (e.g., accurate within ± 1 week) in the smallest subgroups of interest. Thus, we cannot report on validation in <34 weeks. For example, where we only had 3 subjects, but we do have sufficient sample in 34-36 week preterm infants as well as term and post-term infants to report our findings among this group. The following statement has been added to the manuscript:

“Due to logistical challenges in initiating the study, fewer samples were collected than initially anticipated in our protocol, leading to low numbers of infants with gestational age below 34 weeks. Our methods of sample collection and analysis remain the same, recognizing these sampling limitations.”

b) Models: The proposed models to test in the protocol are quite different than the models tested in the current manuscript and no justification is provided. For reference, the models listed in the protocol are: "1) birth weight alone; 2) combination of birth weight and fetal/adult haemoglobin levels; 3) combination of birth weight, haemoglobin levels, thyroid-stimulating hormone and 17-OHP (all non-mass-spectrometry-derived analytes); and 4) birth weight and the full panel of newborn screening analytes. Sex and multiple birth (yes, no) will be included in all models." (Murphy et al., 2017) For models used in the reported study, refer to the subsection “Validation of algorithms”.

Models 1, 2 and 3 were conducted as described in the published protocol. This was an editorial omission. All three models included infant sex and multiple birth (yes/no). Model 1 additionally included birthweight, model 2 excluded birthweight and included analytes and pairwise interactions. Model 3 included birthweight, analytes and pairwise interactions in addition to sex and multiple birth (yes/no). This has been corrected in the Statistical Analysis section of the Materials and methods as follows:

“The performance of the following models was assessed:

Model 1: Baseline model containing only the clinical factors of infant sex, birthweight, and multiple birth (yes/no);

Model 2: Analytes model including infant sex, multiple birth (yes/no) and newborn screening analytes and pairwise interactions including acylcarnitines, amino acids, endocrine and enzyme markers;

Model 3: Full model containing both clinical and analyte data (infant sex, multiple birth (yes/no), birthweight, newborn screening analytes and pairwise interactions)”.

The intermediate models that limited analytes to fetal/adult hemoglobin ratio, and fetal/adult hemoglobin ratio+17OHP+TSH were not included in the results because, although these subgroups of analytes were more simply and cheaply measurable, the models including only these analytes did not reach the level of predictive accuracy that would make them useful. For the sake of clarity and parsimony we opted not to report the results for those subsets of analytes, which were initially chosen for pragmatic rather than scientific reasons.

2) SGA/large-for-gestational age (LGA) babies: The authors need to provide further analysis and discussion regarding the algorithm's performance in SGA/LGA identification. Specifically, the authors need to identify how many SGA/LGA neonates were in their sample and the models' accuracy for these important sub-groups. The authors had listed this analysis in their protocol and have previously performed similar sub-group analysis on the original Ontario cohort which can be used as a guide. (Wilson et al., 2016).

Thank you. We address this point and point 3 in one response, below.

3) While the authors do sub-group analyses by birth weight (<2500g and ≥2500g) in Table 2, weight alone is unimportant when we consider the overarching goal of finding a model that can distinguish preterm birth from SGA and post-term birth from LGA. I would recommend replacing the columns under each of the sample types in Table 2 to be:

1) Overall;

2) SGA;

3) Appropriate-for-gestational age (AGA); and

4) LGA.

Response to points 2 and 3:

Thank you, we agree that the performance of our models among SGA infants is important. We did not have access to local reference distributions of appropriate weight for gestational age, so we relied on our North American reference distributions, from which we applied cutpoints to define small and large for gestational age (below 3rd percentile, below 10th percentile, and above 90^th percentile for given infant sex and GA in weeks. We have added details to the Study Design section of the Materials and methods as follows:

“Small for gestational age was defined as SGA10, for cases where birthweight was below the 10^th percentile within categories of week of gestational age at delivery and infant sex. […] We also calculated SGA3, which identifies infants below the 3rd percentile within gestational age and sex categories and is much more likely to reflect infants who suffered intrauterine growth restriction, especially in low and middle income countries such as Bangladesh where birthweights are lower.”

Further, we have added results for SGA10 and SGA3 to Table 2. We have not reported LGA90 since only a very small proportion of infants (3 infants overall) fell into this group.

4) Imputation of missing values: As only 46 (11%) of observations were imputed, did the authors do a sensitivity analysis of the performance of the model with and without the imputed missing values to evaluate their influence on the results?

We did a complete case analysis and investigated the range of results in each of the 10 imputed datasets and determined that the results did not vary substantially in either the complete case analysis, or among the 10 independent imputations. We reported the average performance across the 10 imputed datasets for the validation results.

5) Prediction score (nomogram) – it would be useful for the authors to also show the relative contribution of each factor in their model. For example, in the baseline model with birthweight, infant sex and multiple births, what is the greatest predictor of GA? A quantification of the relative contribution of each of the factors is useful in understanding their contribution and justification as important variables the prediction of GA. Similarly, this should be done for models 2 and 3 as it allows one to make judgements on how much more improvements in prediction is provided for by including the clinical and analyte data. Also, a comparison of scores helps to discern whether some factors included in the model are correlated and thus are similar in their prediction of GA.

Thank you. In the current study, we are using a previously-developed model to score an external cohort for the purposes of external validation and assessment of generalizability. We did not report on the model building and the relative importance of factors in the final model in this study. We did however report on these details in our previously published studies where these models were developed. However, we appreciate the reviewer’s point that this information is important enough to reiterate, so we have added the following to the Statistical Analysis section of the Materials and methods:

“The fitted models used for scoring the Bangladesh data included numerous main effects and interaction terms including both analytes and clinical measures (sex, multiple birth, birthweight). However, there was a subset of predictors that were clearly the strongest contributors to the model in terms of independent contribution to explained variance: birthweight (in base and full models) and fetal/adult hemoglobin ratio, TSH, 17OHP, ALA, c5, C4DC and TYR (in the sex+multiple birtH⁺analytes and full models).”

6) Would also be helpful for the author to show a plot of true GA vs. predicted GA (calibration) as this will evidently show the variability of the prediction as a function of GA as opposed to the aggregated estimates they have presented by GA in Figure 1.

Thank you. In evaluating our models, we generated residual plots across the range of true gestational ages, but did not include these figures in the manuscript. We agree that these plots are very helpful in understanding the performance of these models above and beyond the aggregate RMSE plots we previously included. We have now included a new figure (Figure 2), which is a panel plot of the three models in both heel prick and cord blood models (6 individual residual plots in all).

7) The cohort was apparently nested within a PreSSMat study in Matlab but the authors do not provide key information on timing of ultrasound examination, birth examinations and weight and criteria used to categorize infants as either preterm, small for gestational age (SGA) or combinations thereof. Please add the necessary level of detail to the methodology and Results sections.

Thank you for this opportunity to provide additional details. Within the PreSSMat study, pregnant women visited the local clinic three times during pregnancy. The first visit to obtain an ultrasound occurred between 11-14 weeks’ gestation. Subsequent visits occurred at 22-24 weeks, 32 weeks, delivery, and six-weeks post-partum. Preterm births were defined as all births that occurred at <37 weeks’ gestation. “Very preterm births” were those that occurred at <32 weeks, and “extremely preterm births” were those that occurred at <28 weeks. As noted, above, we did not have access to local reference distributions of appropriate weight for gestational age, so we relied on our North American reference distributions, from which we applied cutpoints to define small and large for gestational age (below 3rd percentile, below 10th percentile, and above 90^th percentile for given infant sex and GA in weeks. This has been added to the Materials and methods section:

“In the PreSSMat cohort, pregnant women were followed prospectively along the pregnancy continuum, with scheduled visits at 11-14 (enrollment and ultrasound), 22-24, and 32 weeks’ gestation, at delivery, and at 6-weeks post-partum to collect socio-demographic and clinical data as well as biological specimens. […] The percentiles were calculated and applied based on a North American distribution of birthweight within sex and gestational age categories.”

8) Given the issues with cord blood specificity and the wide range of times in collection of heel prick samples (from a few hours to up to 40 hours), one is left guessing as to the fidelity of design of the gold standard used in this study and early measures. Please explain when were these measurements obtained?

Thank you. As noted above, the gold-standard against which our GA estimates were compared was first trimester ultrasound (see below our response to the following comment for more information). We have noted as a limitation that heel-prick samples should be collected at least 24 hours after birth, but given the rates of early mother-child discharge from hospital, heel-prick samples were often collected before this time out of necessity. The post-partum analyte fluctuations that occur within the first few days of birth are a limitation of cord and early heel-prick sampling, and we have noted that this may have affected the performance of our model. However, as ultrasound-validated GA estimates were obtained between 11-14 weeks’ gestation, we are confident that the gestational ages against which our model was compared were accurate to within one week.

9) Please explain what standards were used for ultrasound based gestational age and add an appropriate justification for adoption of the chosen standards.

Thank you. Ultrasound-based gestational age was determined via first trimester ultrasound (between 11-14 weeks). The American College of Obstetricians and Gynecologists recommends first-trimester ultrasound (up to 14 weeks) as the most accurate method of establishing and measuring gestational age (ACOG, 2017). In this study, a portable ultrasound machine was used for all participants, and included measurement of crown-rump length, biparietal diameter, occipito-frontal diameter, head and abdominal circumferences, and femur length. We have referenced the ACOG recommendations within our manuscript.

10) The overall rates of prematurity in the cohort determined from the study (both cord and heel prick samples) are implausibly low (given what is known from ANISA and AMANHI studies done in Bangladesh). Please explain the reasons for these differences.

Thank you for this comment. We believe there are two primary reasons for the lower rates of preterm birth seen in this cohort. First, the women in this cohort benefited from being in the service area for icddr,b, the organization implementing the study in Matlab. Compared to individuals outside of this service area, women in the icddr,b service area benefit from a high level of prenatal and other health care. Additionally, participation in this study allowed pregnancies to be monitored and antenatal care to be provided in ways that may not have occurred had these women not participated in the study. Second, many parents of very preterm and extremely preterm infants were reluctant to have their infants subjected to the sample collection procedures given the challenges already experienced by these infants, and thus did not consent to participate in the study. This was highlighted previously in our Discussion section, but we have made this limitation more explicit. As our objective was not to determine rates of preterm birth in this area however, we have not expanded further as to why our study contains relatively few preterm infants.

“The primary limitation of this study is the participation bias against very preterm and extremely preterm infants, whose parents expressed reluctance to subjecting their newborn to these collection procedures. As a result, we had a relatively small number of samples collected from very preterm and extremely preterm infants, limiting our ability to validate model performance in these sub-groups.”

11) The authors should comment on what is known about prematurity incidence from prior studies of maternal antenatal care and nutrition supplementation done in Bangladesh on large rural cohorts including Matlab, without that information, it is difficult to place this study in the context of what else is known.

Thank you. We believe that this falls outside the scope of our study, as our aim was not to estimate the incidence of preterm birth in Matlab, but rather to comment on our model’s ability to estimate GA in this context. The next step in the implementation of our model will be to use the model to estimate the incidence of preterm birth, but that was not our goal at this time.

Author details

1. Malia SQ Murphy

   Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada

   Contribution

   Writing—original draft, Project administration, Writing—review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-4566-4957

2. Steven Hawken

   1. Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
   2. Department of Epidemiology and Community Health, University of Ottawa, Ottawa, Canada

   Contribution

   Conceptualization, Data curation, Formal analysis, Validation, Methodology, Writing—review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-3341-9022

3. Wei Cheng

   Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada

   Contribution

   Data curation, Formal analysis, Visualization, Methodology, Writing—review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-1475-4079

4. Lindsay A Wilson

   Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada

   Contribution

   Project administration, Writing—review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-9910-3338

5. Monica Lamoureux

   Newborn Screening Ontario, Children’s Hospital of Eastern Ontario, Ottawa, Canada

   Contribution

   Project administration, Writing—review and editing

   Competing interests

   No competing interests declared

6. Matthew Henderson

   Newborn Screening Ontario, Children’s Hospital of Eastern Ontario, Ottawa, Canada

   Contribution

   Formal analysis, Writing—review and editing

   Competing interests

   No competing interests declared

7. Jesmin Pervin

   International Centre for Diarrhoeal Disease Research, Dhaka, Bangladesh

   Contribution

   Investigation, Writing—review and editing

   Competing interests

   No competing interests declared

8. Azad Chowdhury

   Dhaka Shishu (Children) Hospital, Dhaka, Bangladesh

   Contribution

   Writing—review and editing

   Competing interests

   No competing interests declared

9. Courtney Gravett

   Global Alliance to Prevent Prematurity and Stillbirth, Lynnwood, United Stares

   Contribution

   Project administration, Writing—review and editing

   Competing interests

   No competing interests declared

10. Eve Lackritz

    Global Alliance to Prevent Prematurity and Stillbirth, Lynnwood, United Stares

    Contribution

    Writing—review and editing

    Competing interests

    No competing interests declared

11. Beth K Potter

    Department of Epidemiology and Community Health, University of Ottawa, Ottawa, Canada

    Contribution

    Writing—review and editing

    Competing interests

    No competing interests declared

12. Mark Walker

    Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada

    Contribution

    Writing—review and editing

    Competing interests

    No competing interests declared

13. Julian Little

    Department of Epidemiology and Community Health, University of Ottawa, Ottawa, Canada

    Contribution

    Writing—review and editing

    Competing interests

    No competing interests declared

14. Anisur Rahman

    International Centre for Diarrhoeal Disease Research, Dhaka, Bangladesh

    Contribution

    Investigation, Writing—review and editing

    Competing interests

    No competing interests declared

15. Pranesh Chakraborty

    Newborn Screening Ontario, Children’s Hospital of Eastern Ontario, Ottawa, Canada

    Contribution

    Conceptualization, Investigation, Writing—review and editing

    Competing interests

    No competing interests declared

16. Kumanan Wilson

    1. Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
    2. Department of Epidemiology and Community Health, University of Ottawa, Ottawa, Canada

    Contribution

    Conceptualization, Supervision, Funding acquisition, Writing—review and editing

    For correspondence

    kwilson@ohri.ca

    Competing interests

    No competing interests declared

    "This ORCID iD identifies the author of this article:" 0000-0002-1741-7705

• 862

  Page views

• 83

  Downloads

• 14

  Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.