More than 50 million couples worldwide experience infertility. The most common treatment is in vitro fertilization (IVF). Fertility specialists collect eggs and sperm from the prospective parents. They combine the egg and sperm in a laboratory and allow the fertilized eggs to develop for five days into a multi-celled blastocyst. Then, the specialists select the healthiest blastocysts and return them to the patient's uterus.

Since 1978, more than 8 million children have been conceived through IVF. Yet, only about 30% of IVF attempts result in a successful birth. As a result, fertility patients often undergo multiple rounds of IVF, which can be expensive and emotionally draining. Several factors determine IVF success, one of which is the health of the blastocysts selected for transfer to the uterus. Specialists select the blastocysts using several criteria. But these human assessments are subjective and inconsistent in predicting which ones are most likely to result in a successful birth. Recent studies suggest artificial intelligence technology may help select blastocysts.

Liu et al. show that using artificial intelligence to assess blastocysts and fertility patient characteristics leads to more accurate predictions about which blastocysts are likely to result in a successful birth. In the experiments, the researchers trained an artificial intelligence computer program using pictures of 17,580 blastocysts with known birth outcomes and the parents' clinical characteristics. The model identified 16 parental factors associated with birth outcomes. The top 5 most predictive parental factors were maternal age, the day of blastocyst transfer to the uterus, how many eggs were present in the ovaries, the number of eggs retrieved and the thickness of the uterus lining. The program achieved the highest prediction of healthy births so far, compared to success rates listed in other studies.

Artificial intelligence-aided blastocyte selection using patient and blastocyst characteristics may improve IVF success rates and reduce the number of treatment cycles patient couples undergo. Before specialists can use artificial intelligence in their clinics, they must conduct confirmatory clinical studies that enroll patient couples to compare conventional methods and artificial intelligence.

Infertility is a global health issue, affecting more than 50 million couples worldwide (Mascarenhas et al., 2012). Since the birth of the first in vitro fertilization (IVF) child in 1978, over 8 million children were born with IVF treatment (Adamson et al., 2018). Among the various factors contributing to IVF outcomes, the quality of the blastocyst (day 5 embryo) selected for transfer is critical for the success of IVF treatment. Manual grading of blastocyst development stage, inner cell mass (ICM), and trophectoderm (TE) remains the most common method for blastocyst evaluation. While the blastocyst morphological grading is widely used in clinical practice, morphological grades of the development stage, ICM, and TE have shown limited predictive power on clinical outcomes (Seli et al., 2011; Reignier et al., 2019; Bartolacci et al., 2021; Ueno et al., 2021; Xiong et al., 2022). It is desired to identify features for accurate prediction of clinical outcomes of blastocysts.

To achieve this goal, the convolutional neural network (CNN) is expected to play a critical role. CNN is able to automatically detect discriminative features from images and has been the state-of-the-art method in various fields in medical imaging, such as lung cancer prediction (Ardila et al., 2019), breast cancer prediction (McKinney et al., 2020), and diabetic retinopathy screening (Bora et al., 2021). To apply CNN to predict the clinical outcome of a blastocyst, images of blastocysts with a known clinical outcome (e.g., pregnancy and live birth) are collected for the CNN model development. The area under the receiver operating characteristic (ROC) curve (AUC) is the most commonly used metric to evaluate and compare machine learning models on predicting clinical outcomes of blastocysts (Kragh and Karstoft, 2021a). The AUCs reported in the literature using CNN to predict clinical outcomes from blastocyst images range from 0.64 to 0.71 for pregnancy prediction (VerMilyea et al., 2020; Kragh et al., 2021b; Berntsen et al., 2022; Enatsu et al., 2022; Loewke et al., 2022), and are around 0.65 for live birth prediction (Miyagi et al., 2019; Nagaya and Ukita, 2021).

Besides using blastocyst images, attempts have also been made to use time-lapse videos for live birth prediction. These videos contain the entire development process from days 0 to 5–7. However, results in the literature show that CNN using static images achieved similar or slightly better accuracies in predicting clinical outcomes than using time-lapse videos, for instance, AUC=0.68–0.71 (VerMilyea et al., 2020; Enatsu et al., 2022; Loewke et al., 2022) versus 0.64–0.67 (Kragh et al., 2021b; Berntsen et al., 2022) for pregnancy prediction, and 0.66 (Miyagi et al., 2019) versus 0.65 (Nagaya and Ukita, 2021) for live birth prediction. A potential reason is that the redundant frames of images in time-lapse videos may work as noise causing the model to overfit and thus leading to a lower prediction accuracy (Zhu et al., 2018; Wu et al., 2021; Tao et al., 2022). Therefore, we opted to use static blastocyst images in this study.

Different from using blastocyst images alone to predict clinical outcomes, Miyagi et al., 2020 proposed to use blastocyst images together with maternal clinical features including maternal age, AMH, and BMI and reported an AUC of 0.74, the highest accuracy in literature (Miyagi et al., 2020). However, two questions remain elusive. First, the contribution of blastocyst images and the additional contribution of maternal clinical features to live birth prediction are unknown. Second, endometrium status-related features, such as endometrium thickness and pattern, are also critical factors impacting live birth outcomes (Ng et al., 2007; Bu et al., 2016; Mahutte et al., 2022), but were not considered.

In this study, we quantified the effect of blastocyst images and the combined effect of both blastocyst images and patient couple’s clinical features on live birth prediction. The live birth prediction model using only blastocyst images achieved an AUC of 0.67, which was significantly outperformed by the AUC of 0.77 achieved by the model using both blastocyst images and patient couple’s clinical features (p value<0.0001). Additionally, when endometrium status-related features (e.g., endometrium thickness and pattern) were excluded, the AUC of the model using both blastocyst images and patient couple’s clinical features significantly decreased to 0.74 (p value<0.0001), indicating that the inclusion of endometrium status-related clinical features helps improve live birth prediction accuracy. Sixteen patient couple’s clinical features were identified to be most related to live birth outcomes of blastocysts, among which maternal age, the day of blastocyst transfer, antral follicle count (AFC), retrieved oocyte number, and endometrium thickness measured before transfer are the top five features contributing to live birth prediction. Additionally, the CNN heatmaps showed that the CNN mainly focused on ICM and TE for live birth prediction, and the contribution of TE-related features was greater in the CNN trained with the inclusion of patient couple’s clinical features compared with the CNN trained with blastocyst images alone.

In this study, the individual effect of blastocyst images and the combined effect of patient couple’s clinical features for live birth prediction were quantified by comparing the AUC values of the model using only blastocyst images and the model using both blastocyst images and patient couple’s clinical features. An AUC of 0.67 was achieved with blastocyst images only while using both blastocyst images and patient couple’s clinical features led to a significantly higher AUC of 0.77 in live birth prediction. When endometrium status-related features were excluded from patient couple’s clinical features, the AUC of the live birth prediction model significantly decreased (p value=0.007) from 0.77 to 0.74, indicating the strong relevance of endometrium status-related features in live birth prediction. Sixteen patient couple’s clinical features were identified to be most related to live birth outcomes of blastocysts, among which maternal age, the day of blastocyst transfer, AFC, retrieved oocyte number, and endometrium thickness before transfer are the top five features contributing to live birth prediction.

This study was based on a comprehensive multimodal data set collected for blastocyst evaluation. The data set includes 17,580 blastocysts with known live birth outcomes, blastocyst images, and 16 patient couple’s clinical features. As shown in Figure 3, 16 patient couple’s clinical features comprehensively include maternal basal characteristics (age and BMI); hormone profiles measured after period (LH and FT4), on HCG day (PE2, P, and LH), and before transfer (E2); endometrium status-related features (endometrium thickness on HCG day and before transfer, endometrium pattern on HCG day); features related to oocytes (AFC, retrieved oocyte number); the day of blastocyst transfer; number of ovarian stimulation cycles; and paternal features (the ratio of grade A sperm after semen processing). For comparison, the data set studied by Miyagi et al., 2020 did not contain endometrium status-related features and key hormone profiles (e.g., P, E2, and LH). There are numerous IVF data sets containing over 100,000 records of clinical features and live birth outcomes (Nelson and Lawlor, 2011; McLernon et al., 2016; La Marca et al., 2021); however, there are no blastocyst images in these data sets, and thus, these data sets cannot be used for building models to evaluate blastocysts from their images.

To handle the multimodal data set, our proposed model was designed to integrate two modules including a CNN and an MLP to enable the model to simultaneously consider images and numerical clinical features for blastocyst evaluation. The large and comprehensive multimodal data set and the proposed CNN+MLP model resulted in the highest AUC value of 0.77 ever reported thus far for predicting live birth outcomes of blastocysts. They also enabled us to quantify the predictive power of each feature in predicting the live birth outcomes of blastocysts.

The blastocyst grading system introduced in 1999 (Gardner and Schoolcraft, 1999; Gardner, 1999) remains the most common method used by embryologists to evaluate blastocyst quality although the morphological grades of blastocyst development stage, ICM and TE have limited predictive power on live birth outcomes (e.g., AUC=0.58–0.61 for live birth prediction reported by Reignier et al., 2019; Bartolacci et al., 2021; Xiong et al., 2022). Since CNN became a state-of-the-art method for image-based classification, many attempts have been made to apply the CNN to blastocyst evaluation for predicting clinical outcomes (e.g., VerMilyea et al., 2020; Kragh et al., 2021b; Berntsen et al., 2022; Enatsu et al., 2022; Loewke et al., 2022; Miyagi et al., 2019; Nagaya and Ukita, 2021). Among these, the AUC values reported in the literature using blastocyst images only were around 0.65 for live birth prediction (Miyagi et al., 2019; Nagaya and Ukita, 2021). Similarly, we achieved an AUC of 0.67 (see Figure 2). Compared with the AUC of 0.58–0.61 reported in the literature using Gardner grades for live birth prediction, these results confirmed that CNN can achieve a higher prediction accuracy. As shown in Figure 4, the CNN mainly focuses on ICM and TE. Different from the Gardner-defined TE grade on the number of TE cells and the cohesiveness of TE cells as a whole, the CNN tends to focus on specific TE clusters. Understanding the heatmaps further requires more investigations.

Miyagi et al., 2020 used both blastocyst images and maternal clinical features (age, AMH, and BMI) to predict live birth outcomes and achieved an AUC of 0.74 (Miyagi et al., 2020). The additional contribution from the three maternal clinical features was not clear since no AUC was reported by using blastocyst images alone. Furthermore, despite their importance in pregnancy and live birth, endometrium status-related features were not considered in their work. Therefore, our study used a comprehensive data set and quantitatively compared the AUC values of live birth prediction using only blastocyst images versus using both blastocyst images and patient couple’s clinical features. We also quantified the usefulness of endometrium status-related features in working with blastocyst images to improve live birth prediction. Furthermore, we revealed that hormone profiles such as E2, LH, P, and FT4, features related to oocyte retrieval such as AFC and number of oocytes retrieved, and the ratio of grade A sperm after processing representing semen quality are able to work with blastocyst images to further improve the live birth prediction accuracy. Note that in this study, only the total testosterone (T) was analyzed, and free T or bioavailable T was not available for clinical feature analysis (see Supplementary file 1). This may cause potential bias in determining the significance of testosterone as a predictor of live birth.

Another finding of this study, by comparing the heatmaps of the CNN trained without and with the inclusion of patient couple’s clinical features, is that the weights of TE-related features increased (see Figure 4). A potential reason may be that TE and the endometrium status-related features (e.g., endometrium thickness and pattern) play critical roles when a blastocyst initiates implantation, and a positive live birth outcome is not possible without the success of this implantation process (Ahlström et al., 2011; Hill et al., 2013; Chen et al., 2014; Bakkensen et al., 2019).

In conclusion, in this retrospective study involving 17,580 blastocysts with known live birth outcomes, blastocyst images and 16 patient couple’s clinical features, we built a live birth prediction model based on multimodal blastocyst evaluation using both blastocyst images and patient couple’s clinical features. We quantified the individual effect of blastocyst images and the combined effect of patient couple’s clinical features on live birth prediction. Results demonstrated that using both blastocyst images and patient couple’s clinical features can significantly improve live birth prediction than using blastocyst images alone.

The proposed live birth prediction model improves the evaluation of a blastocyst in terms of its live birth potential for best blastocyst selection from multiple blastocysts of a patient. The next step is to validate the model’s prediction accuracy using prospectively collected data and verify its effectiveness in blastocyst selection via a randomized controlled trial (RCT). Patients enrolled in the RCT will be split into the study group and the control group (1:1 ratio). In the study group, the model selects a top blastocyst having the highest probability of live birth for transfer, and in the control group, embryologists select a top blastocyst based on their routine morphological grading for transfer. Live birth outcomes of both groups will be tracked and compared.

All processed data and code needed to reproduce the findings of the study are made openly available in deidentified form. This can be found in https://github.com/robotVisionHang/LiveBirthPrediction_Data_Code (copy archived at Liu et al., 2023), and attached to this manuscript. All codes and software used to analyze the data can also be accessed through the link. Due to data privacy regulations of patient data, raw data cannot be publicly shared. Interested researchers are welcome to contact the corresponding author with a concise project proposal indicating aims of using the data and how they will use the data. The project proposal will be firstly assessed by Prof. Yu Sun, Prof. Ge Lin, and then by the Ethics Committee of the Reproductive and Genetic Hospital of CITIC-Xiangya. There are no restrictions on who can access the data.

1. 1. Adamson GD
   2. de Mouzon J
   3. Chambers GM
   4. Zegers-Hochschild F
   5. Mansour R
   6. Ishihara O
   7. Banker M
   8. Dyer S

   (2018) International committee for monitoring assisted reproductive technology: world report on assisted reproductive technology, 2011

   Fertility and Sterility 110:1067–1080.

   https://doi.org/10.1016/j.fertnstert.2018.06.039

   • Google Scholar
2. 1. Ahlström A
   2. Westin C
   3. Reismer E
   4. Wikland M
   5. Hardarson T

   (2011) Trophectoderm morphology: an important parameter for predicting live birth after single blastocyst transfer

   Human Reproduction 26:3289–3296.

   https://doi.org/10.1093/humrep/der325

   • PubMed
   • Google Scholar
3. 1. Ardila D
   2. Kiraly AP
   3. Bharadwaj S
   4. Choi B
   5. Reicher JJ
   6. Peng L
   7. Tse D
   8. Etemadi M
   9. Ye W
   10. Corrado G
   11. Naidich DP
   12. Shetty S

   (2019) End-To-End lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography

   Nature Medicine 25:954–961.

   https://doi.org/10.1038/s41591-019-0447-x

   • PubMed
   • Google Scholar
4. 1. Bakkensen JB
   2. Brady P
   3. Carusi D
   4. Romanski P
   5. Thomas AM
   6. Racowsky C

   (2019) Association between blastocyst morphology and pregnancy and perinatal outcomes following fresh and cryopreserved embryo transfer

   Journal of Assisted Reproduction and Genetics 36:2315–2324.

   https://doi.org/10.1007/s10815-019-01580-0

   • PubMed
   • Google Scholar
5. 1. Bartolacci A
   2. Dal Canto M
   3. Guglielmo MC
   4. Mura L
   5. Brigante C
   6. Mignini Renzini M
   7. Buratini J

   (2021) Early embryo morphokinetics is a better predictor of post-ICSI live birth than embryo morphology: speed is more important than beauty at the cleavage stage

   Zygote 29:495–502.

   https://doi.org/10.1017/S0967199421000253

   • Google Scholar
6. 1. Berntsen J
   2. Rimestad J
   3. Lassen JT
   4. Tran D
   5. Kragh MF

   (2022) Robust and generalizable embryo selection based on artificial intelligence and time-lapse image sequences

   PLOS ONE 17:e0262661.

   https://doi.org/10.1371/journal.pone.0262661

   • Google Scholar
7. 1. Bora A
   2. Balasubramanian S
   3. Babenko B
   4. Virmani S
   5. Venugopalan S
   6. Mitani A
   7. de Oliveira Marinho G
   8. Cuadros J
   9. Ruamviboonsuk P
   10. Corrado GS
   11. Peng L
   12. Webster DR
   13. Varadarajan AV
   14. Hammel N
   15. Liu Y
   16. Bavishi P

   (2021) Predicting the risk of developing diabetic retinopathy using deep learning

   The Lancet. Digital Health 3:e10–e19.

   https://doi.org/10.1016/S2589-7500(20)30250-8

   • PubMed
   • Google Scholar
8. 1. Bu Z
   2. Wang K
   3. Dai W
   4. Sun Y

   (2016) Endometrial thickness significantly affects clinical pregnancy and live birth rates in frozen-thawed embryo transfer cycles

   Gynecological Endocrinology 32:524–528.

   https://doi.org/10.3109/09513590.2015.1136616

   • PubMed
   • Google Scholar
9. 1. Chen X
   2. Zhang J
   3. Wu X
   4. Cao S
   5. Zhou L
   6. Wang Y
   7. Chen X
   8. Lu J
   9. Zhao C
   10. Chen M
   11. Ling X

   (2014) Trophectoderm morphology predicts outcomes of pregnancy in vitrified-warmed single-blastocyst transfer cycle in a Chinese population

   Journal of Assisted Reproduction and Genetics 31:1475–1481.

   https://doi.org/10.1007/s10815-014-0317-x

   • PubMed
   • Google Scholar
10. 1. Enatsu N
    2. Miyatsuka I
    3. An LM
    4. Inubushi M
    5. Enatsu K
    6. Otsuki J
    7. Iwasaki T
    8. Kokeguchi S
    9. Shiotani M

    (2022) A novel system based on artificial intelligence for predicting blastocyst viability and visualizing the explanation

    Reproductive Medicine and Biology 21:12443.

    https://doi.org/10.1002/rmb2.12443

    • PubMed
    • Google Scholar
11. Conference

    1. Feng C
    2. Zhong Y
    3. Huang W

    (2021) Exploring Classification Equilibrium in Long-Tailed Object Detection

    IEEE/CVF International Conference on Computer Vision. pp. 3417–3426.

    https://doi.org/10.1109/ICCV48922.2021.00340

    • Google Scholar
12. Preprint

    1. Fu R
    2. Hu Q
    3. Dong X
    4. Guo Y
    5. Gao Y
    6. Li B

    (2020) Axiom-Based Grad-Cam: Towards Accurate Visualization and Explanation of Cnns

    arXiv.

    https://doi.org/10.48550/arXiv.2008.02312

    • Google Scholar
13. Book

    1. Gardner DK

    (1999)

    In-vitro culture of human blastocyst

    In: Gardner DK, editors. Towards Reproductive Certainty: Infertility and Genetics Beyond 1999. CRC Press. pp. 378–388.

    • Google Scholar
14. 1. Gardner DK
    2. Schoolcraft WB

    (1999) Culture and transfer of human blastocysts

    Current Opinion in Obstetrics & Gynecology 11:307–311.

    https://doi.org/10.1097/00001703-199906000-00013

    • PubMed
    • Google Scholar
15. 1. Hill MJ
    2. Richter KS
    3. Heitmann RJ
    4. Graham JR
    5. Tucker MJ
    6. DeCherney AH
    7. Browne PE
    8. Levens ED

    (2013) Trophectoderm grade predicts outcomes of single-blastocyst transfers

    Fertility and Sterility 99:1283–1289.

    https://doi.org/10.1016/j.fertnstert.2012.12.003

    • PubMed
    • Google Scholar
16. 1. Kragh MF
    2. Karstoft H

    (2021a) Embryo selection with artificial intelligence: how to evaluate and compare methods?

    Journal of Assisted Reproduction and Genetics 38:1675–1689.

    https://doi.org/10.1007/s10815-021-02254-6

    • PubMed
    • Google Scholar
17. 1. Kragh MF
    2. Rimestad J
    3. Lassen JT
    4. Berntsen J
    5. Karstoft H

    (2021b) Predicting embryo viability based on self-supervised alignment of time-lapse videos

    IEEE Transactions on Medical Imaging 41:465–475.

    https://doi.org/10.1109/TMI.2021.3116986

    • PubMed
    • Google Scholar
18. 1. La Marca A
    2. Capuzzo M
    3. Donno V
    4. Mignini Renzini M
    5. Giovane CD
    6. D’Amico R
    7. Sunkara SK

    (2021) The predicted probability of live birth in in vitro fertilization varies during important stages throughout the treatment: analysis of 114,882 first cycles

    Journal of Gynecology Obstetrics and Human Reproduction 50:101878.

    https://doi.org/10.1016/j.jogoh.2020.101878

    • PubMed
    • Google Scholar
19. Software

    1. Liu H
    2. Zhang Z
    3. Gu Y
    4. Dai C
    5. Shan G
    6. Song H
    7. Li D
    8. Chen W
    9. Lin G
    10. Sun Y

    (2023) LiveBirthPrediction_Data_Code, version swh:1:rev:55eda9aece25451f1b68e3941124d92e9d97cf82

    Software Heritage.

    https://archive.softwareheritage.org/swh:1:dir:04bbbb284ca95b4e8c2385bbac8f4460ceb5ae68;origin=https://github.com/robotVisionHang/LiveBirthPrediction_Data_Code;visit=swh:1:snp:9f5da846f1bf5d1b38540d13cb03e221bd9df153;anchor=swh:1:rev:55eda9aece25451f1b68e3941124d92e9d97cf82
20. 1. Loewke K
    2. Cho JH
    3. Brumar CD
    4. Maeder-York P
    5. Barash O
    6. Malmsten JE
    7. Zaninovic N
    8. Sakkas D
    9. Miller KA
    10. Levy M
    11. VerMilyea MD

    (2022) Characterization of an artificial intelligence model for ranking static images of blastocyst stage embryos

    Fertility and Sterility 117:528–535.

    https://doi.org/10.1016/j.fertnstert.2021.11.022

    • PubMed
    • Google Scholar
21. 1. Mahutte N
    2. Hartman M
    3. Meng L
    4. Lanes A
    5. Luo ZC
    6. Liu KE

    (2022) Optimal endometrial thickness in fresh and frozen-thaw in vitro fertilization cycles: an analysis of live birth rates from 96,000 autologous embryo transfers

    Fertility and Sterility 117:792–800.

    https://doi.org/10.1016/j.fertnstert.2021.12.025

    • PubMed
    • Google Scholar
22. 1. Mascarenhas MN
    2. Flaxman SR
    3. Boerma T
    4. Vanderpoel S
    5. Stevens GA
    6. Low N

    (2012) National, regional, and global trends in infertility prevalence since 1990: a systematic analysis of 277 health surveys

    PLOS Medicine 9:e1001356.

    https://doi.org/10.1371/journal.pmed.1001356

    • PubMed
    • Google Scholar
23. 1. McKinney SM
    2. Sieniek M
    3. Godbole V
    4. Godwin J
    5. Antropova N
    6. Ashrafian H
    7. Back T
    8. Chesus M
    9. Corrado GS
    10. Darzi A
    11. Etemadi M
    12. Garcia-Vicente F
    13. Gilbert FJ
    14. Halling-Brown M
    15. Hassabis D
    16. Jansen S
    17. Karthikesalingam A
    18. Kelly CJ
    19. King D
    20. Ledsam JR
    21. Melnick D
    22. Mostofi H
    23. Peng L
    24. Reicher JJ
    25. Romera-Paredes B
    26. Sidebottom R
    27. Suleyman M
    28. Tse D
    29. Young KC
    30. De Fauw J
    31. Shetty S

    (2020) International evaluation of an AI system for breast cancer screening

    Nature 577:89–94.

    https://doi.org/10.1038/s41586-019-1799-6

    • PubMed
    • Google Scholar
24. 1. McLernon DJ
    2. Steyerberg EW
    3. Te Velde ER
    4. Lee AJ
    5. Bhattacharya S

    (2016) Predicting the chances of a live birth after one or more complete cycles of in vitro fertilisation: population based study of linked cycle data from 113 873 women

    BMJ 355:i5735.

    https://doi.org/10.1136/bmj.i5735

    • PubMed
    • Google Scholar
25. 1. Miyagi Y
    2. Habara T
    3. Hirata R
    4. Hayashi N

    (2019) Feasibility of artificial intelligence for predicting live birth without aneuploidy from a blastocyst image

    Reproductive Medicine and Biology 18:204–211.

    https://doi.org/10.1002/rmb2.12267

    • PubMed
    • Google Scholar
26. 1. Miyagi Y
    2. Habara T
    3. Hirata R
    4. Hayashi N

    (2020) Predicting a live birth by artificial intelligence incorporating both the blastocyst image and conventional embryo evaluation parameters

    Artificial Intelligence in Medical Imaging 1:94–107.

    https://doi.org/10.35711/aimi.v1.i3.94

    • Google Scholar
27. 1. Nagaya M
    2. Ukita N

    (2021) Embryo grading with unreliable labels due to chromosome abnormalities by regularized Pu learning with ranking

    IEEE Transactions on Medical Imaging 41:320–331.

    https://doi.org/10.1109/TMI.2021.3126169

    • PubMed
    • Google Scholar
28. 1. Nelson SM
    2. Lawlor DA

    (2011) Predicting live birth, preterm delivery, and low birth weight in infants born from in vitro fertilisation: a prospective study of 144,018 treatment cycles

    PLOS Medicine 8:e1000386.

    https://doi.org/10.1371/journal.pmed.1000386

    • PubMed
    • Google Scholar
29. 1. Ng EHY
    2. Chan CCW
    3. Tang OS
    4. Yeung WSB
    5. Ho PC

    (2007) Endometrial and subendometrial vascularity is higher in pregnant patients with livebirth following art than in those who suffer a miscarriage

    Human Reproduction 22:1134–1141.

    https://doi.org/10.1093/humrep/del458

    • PubMed
    • Google Scholar
30. Preprint

    1. Paszke A
    2. Gross S
    3. Massa F
    4. Lerer A
    5. Bradbury J
    6. Chanan G
    7. Killeen T
    8. Lin Z
    9. Gimelshein N
    10. Antiga L
    11. Desmaison A

    (2019) Pytorch: An Imperative Style, High-Performance Deep Learning Library

    arXiv.

    https://doi.org/10.48550/arXiv.1912.01703

    • Google Scholar
31. 1. Raschka S

    (2018) MLxtend: providing machine learning and data science utilities and extensions to python’s scientific computing stack

    Journal of Open Source Software 3:638.

    https://doi.org/10.21105/joss.00638

    • Google Scholar
32. 1. Reignier A
    2. Girard JM
    3. Lammers J
    4. Chtourou S
    5. Lefebvre T
    6. Barriere P
    7. Freour T

    (2019) Performance of day 5 kidscore

    Journal of Assisted Reproduction and Genetics 36:2279–2285.

    https://doi.org/10.1007/s10815-019-01567-x

    • PubMed
    • Google Scholar
33. 1. Seli E
    2. Bruce C
    3. Botros L
    4. Henson M
    5. Roos P
    6. Judge K
    7. Hardarson T
    8. Ahlström A
    9. Harrison P
    10. Henman M
    11. Go K
    12. Acevedo N
    13. Siques J
    14. Tucker M
    15. Sakkas D

    (2011) Receiver operating characteristic (ROC) analysis of day 5 morphology grading and metabolomic viability score on predicting implantation outcome

    Journal of Assisted Reproduction and Genetics 28:137–144.

    https://doi.org/10.1007/s10815-010-9501-9

    • PubMed
    • Google Scholar
34. 1. Solorio-Fernández S
    2. Carrasco-Ochoa JA
    3. Martínez-Trinidad JF

    (2020) A review of unsupervised feature selection methods

    Artificial Intelligence Review 53:907–948.

    https://doi.org/10.1007/s10462-019-09682-y

    • Google Scholar
35. Conference

    1. Tan M
    2. Le Q

    (2021)

    Efficientnetv2: Smaller models and faster training

    In International Conference on Machine Learning 2021. pp. 10096–10106.

    • Google Scholar
36. 1. Tao H
    2. Lu M
    3. Hu Z
    4. Xin Z
    5. Wang J

    (2022) Attention-aggregated attribute-aware network with redundancy reduction convolution for video-based industrial smoke emission recognition

    IEEE Transactions on Industrial Informatics 18:7653–7664.

    https://doi.org/10.1109/TII.2022.3146142

    • Google Scholar
37. 1. Ueno S
    2. Berntsen J
    3. Ito M
    4. Uchiyama K
    5. Okimura T
    6. Yabuuchi A
    7. Kato K

    (2021) Pregnancy prediction performance of an annotation-free embryo scoring system on the basis of deep learning after single vitrified-warmed blastocyst transfer: a single-center large cohort retrospective study

    Fertility and Sterility 116:1172–1180.

    https://doi.org/10.1016/j.fertnstert.2021.06.001

    • PubMed
    • Google Scholar
38. 1. VerMilyea M
    2. Hall JMM
    3. Diakiw SM
    4. Johnston A
    5. Nguyen T
    6. Perugini D
    7. Miller A
    8. Picou A
    9. Murphy AP
    10. Perugini M

    (2020) Development of an artificial intelligence-based assessment model for prediction of embryo viability using static images captured by optical light microscopy during IVF

    Human Reproduction 35:770–784.

    https://doi.org/10.1093/humrep/deaa013

    • PubMed
    • Google Scholar
39. Conference

    1. Wu X
    2. Yang F
    3. Zhou T
    4. Lin X

    (2021) Rethinking the Impacts of Overfitting and Feature Quality on Small-scale Video Classification

    MM ’21. pp. 4760–4764.

    https://doi.org/10.1145/3474085.3479226

    • Google Scholar
40. 1. Xiong F
    2. Sun Q
    3. Wang S
    4. Yao Z
    5. Chen P
    6. Wan C
    7. Zhong H
    8. Zeng Y

    (2022) A nomogram to assist blastocyst selection in vitrified‐warmed embryo transfer cycles

    Journal of Obstetrics and Gynaecology Research 48:1816–1828.

    https://doi.org/10.1111/jog.15138

    • Google Scholar
41. Conference

    1. Zhu C
    2. Tan X
    3. Zhou F
    4. Liu X
    5. Yue K
    6. Ding E
    7. Ma Y

    (2018) Fine-grained video categorization with redundancy reduction attention

    In Proceedings of the European Conference on Computer Vision. pp. 136–152.

    https://doi.org/10.1007/978-3-030-01228-1

    • Google Scholar

Decision letter after peer review:

Thank you for submitting your article "Development and evaluation of a live birth prediction model for evaluating human blastocysts: a retrospective study" for consideration by eLife. Your article has been reviewed by 2 peer reviewers, and the evaluation has been overseen by a Reviewing Editor and Ricardo Azziz as the Senior Editor. The reviewers have opted to remain anonymous.

The reviewers have discussed their reviews with one another, and the Reviewing Editor has drafted this to help you prepare a revised submission.

Essential revisions:

1) Please, expand the section "Model architecture" and clarify the details regarding "decision-level features".

2) Provide the details of how parameter optimization was accomplished as well as the architectural details, i.e., the number of layers and nodes. What was the computational overhead for training these models?

3) Consider presenting the data for all significant predictors to justify the choice for inclusion in the model and verify if the important or top features MLP (using explanation methods) uses for prediction are the same as those inferred by the logistic regression.

4) Please, consider presenting in detail a weighted sampling approach used to tackle the imbalance issue.

5) The code is available only for generating figures 2, 4 reported in the paper. For figure 3, only data is available. Consider presenting this code for reproducibility purposes.

6) Please, improve the discussion of the potential applications of the proposed model in clinical settings and mention the method of testosterone assessment as a study limitation.

Reviewer #1 (Recommendations for the authors):

The authors could mention the method of testosterone assessment as a study limitation that may cause a potential bias regarding the estimation of significance of testosterone as a predictor of live birth.

The authors could consider presenting the data for all significant predictors (of example, in the supplemental Figure 3 data) and justify the choice for inclusion in the model. It will better demonstrate the correctness of the selection of predictors

Reviewer #2 (Recommendations for the authors):

I quite enjoyed reading the article. Overall, the motivation and concepts are well defined; however, the manuscript lacks the necessary methodological details, which hindered my ability to fully understand and appreciate the work.

The link between CNN and MLP architecture in the final integrated model is unclear. The section "Model architecture" needs to be expanded. It is not clear what "decision-level features" are from CNN or MLP. Are these features from the CNN's fully connected layer? In MLP, are they before the final layer? And how do authors concatenate these features? These details are important to understand final architecture.

How was parameter optimization accomplished? Architectural details, i.e., the number of layers and nodes, are missing. What was the computational overhead for training these models?

It is difficult to understand the discrepancy of features between CNN with clinical features and CNN without clinical features. Maybe it is because model architecture is not well defined. For the moment, it seems like CNN was trained independently, even in the concatenated version of the model. How can you explain the discrepancy between the activation maps of these two models?

Important features were identified using logistic regression. I do not observe the link presenting these features as important features when the model was built using MLP instead. Could you verify that the important or top features MLP (using explanation methods) uses for prediction are the same as those inferred by the logistic regression?

The imbalance issue was tackled using a weighted sampling approach. This approach needs to be detailed in the main text. And how were train, val, and test partitions built in view of the distribution of minority and majority classes? Did the author verify other approaches that can help resolve this issue?

The code to reproduce the model is missing.

Discussion regarding implementation in a clinical setting would be informative. How feasible is the model's deployment in a clinic? Maybe you can further elaborate on prospective clinical trial which was mentioned in line # 363.

Reviewer #3 (Recommendations for the authors):

The study is well designed as the Materials and methods were convincing. Results are supporting the Aims. Further studies will be important for establishing the Criteria.

The challenging issue is to keep reporting the effectiveness of this predictive procedure and publish it with LBR.

Essential revisions:

1) Please, expand the section "Model architecture" and clarify the details regarding "decision-level features".

The decision-level features are generated from the last fully connected layer in the CNN and the last fully connected layer in the MLP. Each decision-level feature has two variables used to classify the blastocyst into the positive or negative live birth outcome category. The two decision-level features are fused by the adding operation.

To clarify this issue, we have revised the “Model architecture” section, added text in Figure 1 to indicate the last fully connected layer, and provided the code to reproduce the model. The source code has been submitted with the revised manuscript and can also be accessed at https://github.com/robotVisionHang/LiveBirthPrediction_Data_Code.

The revised “Model architecture” section now reads:

“Figure 1 shows the architecture of the live birth prediction model based on multi-modal blastocyst evaluation. It consists of a CNN to process blastocyst images and an MLP to process patient couple's clinical features. Features from the CNN and the MLP are fused; thus, the model can be trained to simultaneously take into account both blastocyst images and patient couple's clinical features for live birth prediction. The last fully connected layer in the CNN and the last fully connected layer in the MLP each output a decision-level feature, which has two variables used to classify the blastocyst into the positive or negative live birth outcome category. The adding operation fuses decision-level features from the CNN and the MLP, and the result of addition is taken as the final output of the overall live birth prediction model.”

2) Provide the details of how parameter optimization was accomplished as well as the architectural details, i.e., the number of layers and nodes. What was the computational overhead for training these models?

To clarify this issue, we have added paragraph 3 in the “Model implementation and training” section:

“Model performance is subject to training hyperparameters (e.g., optimizer, learning rate, batch size, number of layers). Hence, an automatic hyperparameter-tuning tool is used, Facebook Ax (https://github.com/facebook/Ax), to search for the optimal hyperparameters for model training. The selected hyperparameters for training the model include a batch size of 16, an SGD optimizer with a learning rate of 0.008 and a momentum of 0.39, and three hidden layers in the MLP. A dropout layer follows each hidden layer in the MLP to prevent overfitting. The number of nodes in each hidden layer is 6836, 5657, and 468, respectively. The dropout rate in each dropout layer is 0.01, 0.07, and 0.67, respectively. The model was trained with four RTX A6000 GPUs. It took about 30 hours to search for the optimal hyperparameters and about an hour to train the model using the optimal hyperparameters.”

3) Consider presenting the data for all significant predictors to justify the choice for inclusion in the model and verify if the important or top features MLP (using explanation methods) uses for prediction are the same as those inferred by the logistic regression.

The AUC of the model using all significant predictors and blastocyst images is 0.75, lower than the AUC (0.77) achieved by the model using logistic regression-selected clinical features and blastocyst images, and the AUC (0.76) achieved by the model using MLP-selected clinical features and blastocyst images. Feature selection reduces the input feature dimensions by removing redundant features and features with limited predictive power, thus improving the model generalization capability.

To follow the reviewer’s suggestion, we used the sequential forward feature selection method to perform MLP-based feature selection. It is explainable in that features are sequentially added to an empty candidate set until the addition of further features does not increase the prediction accuracy. Note that we searched for the optimal MLP parameters (e.g., number of layers, nodes, dropout rate) and training parameters (e.g., optimizer, learning rate) to accurately evaluate the predictive power of each feature candidate.

The MLP-based feature selection method selected 14 clinical features. Compared with the 16 clinical features selected by logistic regression (LR), 12 of the 16 clinical features were selected by both MLP and logistic regression, and the top 9 features are the same. The two clinical features selected by MLP but not by LR include fresh semen (yes/no) and follicle stimulating hormone (FSH) on day 3 after period. The four clinical features selected by LR but not MLP include number of ovarian stimulation cycles, progesterone (P) on HCG day, maternal body mass index (BMI), and free thyroxine (FT4) on day 3 after period. The AUC (0.77) achieved by the model using LR-selected clinical features and blastocyst images and the AUC (0.76) achieved by the model using MLP-selected clinical features and blastocyst images show no significant difference (p-value = 0.95 > 0.05).

Furthermore, the MLP-based feature selection took a few days due to the processes of iterative feature searching/testing and tuning hyperparameters. The LR-based feature selection is much more efficient and only takes a few minutes.

To clarify this issue, we have now added the data regarding the AUC of the model using all significant predictors in and blastocyst images and the MLP-based feature selection results in Figure 3-supplement 1, 2.

4) Please, consider presenting in detail a weighted sampling approach used to tackle the imbalance issue.

To clarify this issue, we have added details regarding the weighted sampling approach in paragraph 2 in the “Model implementation and training” section:

“In the weighted sampling approach, the probability of each item to be selected is determined by its weight, and the weight of each item is assigned by inverse class frequencies. In this way, the weighted sampling approach rebalances the class distributions by oversampling the minority class and under-sampling the majority class.”

5) The code is available only for generating figures 2, 4 reported in the paper. For figure 3, only data is available. Consider presenting this code for reproducibility purposes.

We have provided the source code for generating figure 3. The source code has been submitted with the revised manuscript and can also be accessed at https://github.com/robotVisionHang/LiveBirthPrediction_Data_Code

6) Please, improve the discussion of the potential applications of the proposed model in clinical settings and mention the method of testosterone assessment as a study limitation.

1) The potential application of the proposed model in clinical settings is to improve blastocyst selection. Among the various factors contributing to IVF outcomes, the quality (i.e., developmental potential) of the selected blastocyst for transfer is a major factor determining IVF success. Existing approaches for evaluating and selecting blastocysts are based on manually observing blastocyst morphology grade, which have shown limited predictive power on live birth outcomes of blastocysts (e.g., AUC = 0.58-0.61).

The proposed model achieved a significantly higher accuracy in evaluating the live birth potential of blastocysts (AUC = 0.77) for best blastocyst selection to improve the live birth rate. When the proposed model is applied in clinical practice for blastocyst selection, it takes images of multiple blastocysts of a same patient and patient couple’s clinical features as input, outputs the live birth probability of each blastocyst, and identifies the best blastocyst having the highest live birth probability.

To clarify this issue, we have added following contents on page 13:

“The proposed live birth prediction model improves the evaluation of a blastocyst in terms of its live birth potential for best blastocyst selection from multiple blastocysts of a patient. The next step is to validate the model’s prediction accuracy using prospectively collected data and verify its effectiveness in blastocyst selection via a randomized controlled trial (RCT). Patients enrolled in the RCT will be split into the study group and the control group (1:1 ratio). In the study group, the model selects a top blastocyst having the highest probability of live birth for transfer, and in the control group, embryologists select a top blastocyst based on their routine morphological grading for transfer. Live birth outcomes of both groups will be tracked and compared.”

2) The testosterone (T) included in the clinical feature analysis is total T measuring both free T and bioavailable T.

To clarify this issue, we revised the name of testosterone in Supplementary file 1, which now reads: “Total testosterone on day 3 after period”.

We also mentioned this as a study limitation on page 12:

“Note that in this study, only the total testosterone (T) was analyzed, and free T or bioavailable T was not available for clinical feature analysis (see Supplementary file 1). This may cause potential bias in determining the significance of testosterone as a predictor of live birth.”

Reviewer #1 (Recommendations for the authors):

The authors could mention the method of testosterone assessment as a study limitation that may cause a potential bias regarding the estimation of significance of testosterone as a predictor of live birth.

To clarify this issue, we revised the name of testosterone in Supplementary Table Ⅰ, which now reads: “Total testosterone on day 3 after period”.

We also mentioned this as a study limitation on page 12:

“Note that in this study, only the total testosterone (T) was analyzed, and free T or bioavailable T was not available for clinical feature analysis (see Supplementary file 1). This may cause potential bias in determining the significance of testosterone as a predictor of live birth.”

The authors could consider presenting the data for all significant predictors (of example, in the supplemental Figure 3 data) and justify the choice for inclusion in the model. It will better demonstrate the correctness of the selection of predictors

The AUC of the model using all significant predictors and blastocyst images is 0.75, lower than the AUC (0.77) achieved by the model using logistic regression-selected clinical features and blastocyst images, and the AUC (0.76) achieved by the model using MLP-selected clinical features and blastocyst images. Feature selection reduces the input feature dimensions by removing redundant features and features with limited predictive power, thus improving the model generalization capability.

To clarify this issue, we have added the ROC curve comparisons Figure 3-supplement 1:

Reviewer #2 (Recommendations for the authors):

I quite enjoyed reading the article. Overall, the motivation and concepts are well defined; however, the manuscript lacks the necessary methodological details, which hindered my ability to fully understand and appreciate the work.

The link between CNN and MLP architecture in the final integrated model is unclear. The section "Model architecture" needs to be expanded. It is not clear what "decision-level features" are from CNN or MLP. Are these features from the CNN's fully connected layer? In MLP, are they before the final layer? And how do authors concatenate these features? These details are important to understand final architecture.

The decision-level features are generated from the last fully connected layer in the CNN and the last fully connected layer in the MLP. Each decision-level feature has two variables used to classify the blastocyst into the positive or negative live birth outcome category. The two decision-level features are fused by the adding operation.

To clarify this issue, we have revised the “Model architecture” section, added text in Figure 1 to indicate the last fully connected layer, and provided the code to reproduce the model. The source code has been submitted with the revised manuscript and can also be accessed at https://github.com/robotVisionHang/LiveBirthPrediction_Data_Code.

The revised “Model architecture” section now reads:

“Figure 1 shows the architecture of the live birth prediction model based on multi-modal blastocyst evaluation. It consists of a CNN to process blastocyst images and an MLP to process patient couple's clinical features. Features from the CNN and the MLP are fused; thus, the model can be trained to simultaneously take into account both blastocyst images and patient couple's clinical features for live birth prediction. The last fully connected layer in the CNN and the last fully connected layer in the MLP each output a decision-level feature, which has two variables used to classify the blastocyst into the positive or negative live birth outcome category. The adding operation fuses decision-level features from the CNN and the MLP, and the result of addition is taken as the final output of the overall live birth prediction model.”

How was parameter optimization accomplished? Architectural details, i.e., the number of layers and nodes, are missing. What was the computational overhead for training these models?

To clarify this issue, we have added paragraph 3 in the “Model implementation and training” section:

“Model performance is subject to training hyperparameters (e.g., optimizer, learning rate, batch size, number of layers). Hence, an automatic hyperparameter-tuning tool is used, Facebook Ax (https://github.com/facebook/Ax), to search for the optimal hyperparameters for model training. The selected hyperparameters for training the model include a batch size of 16, an SGD optimizer with a learning rate of 0.008 and a momentum of 0.39, and three hidden layers in the MLP. A dropout layer follows each hidden layer in the MLP to prevent overfitting. The number of nodes in each hidden layer is 6836, 5657, and 468, respectively. The dropout rate in each dropout layer is 0.01, 0.07, and 0.67, respectively. The model was trained with four RTX A6000 GPUs. It took about 30 hours to search for the optimal hyperparameters and about an hour to train the model using the optimal hyperparameters.”

It is difficult to understand the discrepancy of features between CNN with clinical features and CNN without clinical features. Maybe it is because model architecture is not well defined. For the moment, it seems like CNN was trained independently, even in the concatenated version of the model. How can you explain the discrepancy between the activation maps of these two models?

The CNN and the MLP are connected by the two decision-level features. The adding operation fuses decision-level features from the CNN and the MLP, and the addition result is taken as the final output of the overall live birth prediction model. Thus, the model weights of the CNN and the MLP were trained simultaneously.

Another finding of this study, by comparing the heatmaps of the CNN trained without and with the inclusion of patient couple's clinical features, is that the weights of TE-related features increased (see Figure 4). A potential reason may be that TE and the endometrium status-related features (e.g., endometrium thickness and pattern) play critical roles when a blastocyst initiates implantation, and a positive live birth outcome is not possible without the success of this implantation process (Ahlström et al., 2011; Hill et al., 2013; Chen et al., 2014; Bakkensen et al., 2019).

To clarify this issue, we have revised the “Model architecture” section. The explanation of the discrepancy between the heatmaps of the CNN trained without and with clinical features is added on page 12:

“Another finding of this study, by comparing the heatmaps of the CNN trained without and with the inclusion of patient couple's clinical features, is that the weights of TE-related features increased (see Figure 4). A potential reason may be that TE and the endometrium status-related features (e.g., endometrium thickness and pattern) play critical roles when a blastocyst initiates implantation, and a positive live birth outcome is not possible without the success of this implantation process (Ahlström et al., 2011; Hill et al., 2013; Chen et al., 2014; Bakkensen et al., 2019).”

Important features were identified using logistic regression. I do not observe the link presenting these features as important features when the model was built using MLP instead. Could you verify that the important or top features MLP (using explanation methods) uses for prediction are the same as those inferred by the logistic regression?

To follow the reviewer’s suggestion, we used the sequential forward feature selection method to perform MLP-based feature selection. It is explainable in that features are sequentially added to an empty candidate set until the addition of further features does not increase the prediction accuracy. Note that we searched for the optimal MLP parameters (e.g., number of layers, nodes, dropout rate) and training parameters (e.g., optimizer, learning rate) to accurately evaluate the predictive power of each feature candidate.

The MLP-based feature selection method selected 14 clinical features. Compared with the 16 clinical features selected by logistic regression (LR), 12 of the 16 clinical features were selected by both MLP and logistic regression, and the top 9 features are the same. The two clinical features selected by MLP but not by LR include fresh semen (yes/no) and follicle stimulating hormone (FSH) on day 3 after period. The four clinical features selected by LR but not MLP include number of ovarian stimulation cycles, progesterone (P) on HCG day, maternal body mass index (BMI), and free thyroxine (FT4) on day 3 after period. The AUC (0.77) achieved by the model using LR-selected clinical features and blastocyst images and the AUC (0.76) achieved by the model using MLP-selected clinical features and blastocyst images show no significant difference (p-value = 0.95 > 0.05).

Furthermore, the MLP-based feature selection took a few days due to the processes of iterative feature searching/testing and tuning hyperparameters. The LR-based feature selection is much more efficient and only takes a few minutes.

To clarify this issue, we have now added the data regarding the AUC of the model using all significant predictors and blastocyst images and the MLP-based feature selection results in Figure 3-supplement 1, 2:

The imbalance issue was tackled using a weighted sampling approach. This approach needs to be detailed in the main text. And how were train, val, and test partitions built in view of the distribution of minority and majority classes? Did the author verify other approaches that can help resolve this issue?

We have added details regarding the weighted sampling approach in paragraph 2, the “Model implementation and training” section. We also verified the approach of using weighted cross-entropy loss. We found that both approaches can be used to mitigate the prediction bias towards the majority class. The weighted sampling approach performed better and was selected for the final model training.

The stratified random sampling approach was used to ensure that all split datasets (training, validation, and testing datasets) have the same distribution of minority and majority classes.

To clarify this issue, we have revised the following contents in paragraph 2, the “Model implementation and training” section:

“The blastocysts were randomly split into 80%:10%:10% to construct the training, validation, and testing datasets. The stratified random sampling approach was used to ensure that all split datasets have the same distribution of minority and majority classes. Since the ratio of blastocysts with a positive live birth outcome in the dataset is 0.368, to mitigate the model's prediction bias towards the majority category (i.e., the negative live birth outcome), the weighted sampling approach, which can help rebalance the class distributions when sampling from an imbalanced dataset (Feng et al., 2021), was employed for training the model. In the weighted sampling approach, the probability of each item to be selected is determined by its weight, and the weight of each item is assigned by inverse class frequencies. In this way, the weighted sampling approach rebalances the class distributions by oversampling the minority class and under-sampling the majority class. We also verified the approach of using weighted cross-entropy loss, which assigns greater weights to the loss caused by the prediction error of minority classes. Both approaches helped mitigate the prediction bias towards the majority class, and the results showed that the weighted sampling approach outperformed the weighted cross-entropy loss method.”

The code to reproduce the model is missing.

We have provided the source code to reproduce the model. The source code has been submitted with the revised manuscript and can also be accessed at https://github.com/robotVisionHang/LiveBirthPrediction_Data_Code

Discussion regarding implementation in a clinical setting would be informative. How feasible is the model's deployment in a clinic? Maybe you can further elaborate on prospective clinical trial which was mentioned in line # 363.

The potential application of the proposed model in clinical settings is to improve blastocyst selection. Among the various factors contributing to IVF outcomes, the quality (i.e., developmental potential) of the selected blastocyst for transfer is a major factor determining IVF success. Existing approaches for evaluating and selecting blastocysts are based on manually observing blastocyst morphology grade, which have shown limited predictive power on live birth outcomes of blastocysts (e.g., AUC = 0.58-0.61).

The proposed model achieved a significantly higher accuracy in evaluating the live birth potential of blastocysts (AUC = 0.77) for best blastocyst selection to improve the live birth rate. When the proposed model is applied in clinical practice for blastocyst selection, it takes images of multiple blastocysts of a same patient and patient couple’s clinical features as input, outputs the live birth probability of each blastocyst, and identifies the best blastocyst having the highest live birth probability.

To clarify this issue, we have added following contents on page 13:

“The proposed live birth prediction model improves the evaluation of a blastocyst in terms of its live birth potential for best blastocyst selection from multiple blastocysts of a patient. The next step is to validate the model’s prediction accuracy using prospectively collected data and verify its effectiveness in blastocyst selection via a randomized controlled trial (RCT). Patients enrolled in the RCT will be split into the study group and the control group (1:1 ratio). In the study group, the model selects a top blastocyst having the highest probability of live birth for transfer, and in the control group, embryologists select a top blastocyst based on their routine morphological grading for transfer. Live birth outcomes of both groups will be tracked and compared.”

Author details

1. Hang Liu

   Department of Mechanical Engineering, University of Toronto, Toronto, Canada

   Contribution

   Conceptualization, Data curation, Software, Formal analysis, Validation, Investigation, Visualization, Methodology, Writing – original draft, Writing – review and editing

   Contributed equally with

   Zhuoran Zhang and Yifan Gu

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0001-7948-4236

2. Zhuoran Zhang

   School of Science and Engineering, The Chinese University of Hong Kong-Shenzhen, Shenzhen, China

   Contribution

   Conceptualization, Resources, Data curation, Software, Formal analysis, Validation, Investigation, Visualization, Methodology, Writing – original draft, Writing – review and editing

   Contributed equally with

   Hang Liu and Yifan Gu

   Competing interests

   No competing interests declared

3. Yifan Gu

   1. Institute of Reproductive and Stem Cell Engineering, School of Basic Medical Science, Central South University, Changsha, China
   2. Reproductive and Genetic Hospital of CITIC-Xiangya, Changsha, China

   Contribution

   Conceptualization, Resources, Data curation, Software, Formal analysis, Validation, Investigation, Visualization, Methodology, Writing – original draft, Writing – review and editing

   Contributed equally with

   Hang Liu and Zhuoran Zhang

   Competing interests

   No competing interests declared

4. Changsheng Dai

   Department of Mechanical Engineering, University of Toronto, Toronto, Canada

   Contribution

   Conceptualization, Data curation, Software, Formal analysis, Investigation, Writing – review and editing

   Competing interests

   No competing interests declared

5. Guanqiao Shan

   Department of Mechanical Engineering, University of Toronto, Toronto, Canada

   Contribution

   Conceptualization, Data curation, Software, Formal analysis, Investigation, Writing – review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-2570-769X

6. Haocong Song

   Department of Mechanical Engineering, University of Toronto, Toronto, Canada

   Contribution

   Data curation, Software, Investigation, Writing – review and editing

   Competing interests

   No competing interests declared

7. Daniel Li

   Department of Electrical and Computer Engineering, Toronto, Canada

   Contribution

   Data curation, Software, Validation, Investigation, Writing – review and editing

   Competing interests

   No competing interests declared

8. Wenyuan Chen

   Department of Mechanical Engineering, University of Toronto, Toronto, Canada

   Contribution

   Conceptualization, Resources, Data curation, Supervision, Validation, Investigation, Project administration, Writing – review and editing

   Competing interests

   No competing interests declared

9. Ge Lin

   1. Institute of Reproductive and Stem Cell Engineering, School of Basic Medical Science, Central South University, Changsha, China
   2. Reproductive and Genetic Hospital of CITIC-Xiangya, Changsha, China
   3. Key Laboratory of Reproductive and Stem Cell Engineering, National Health and Family Planning Commission, Changsha, China
   4. National Engineering Research Center of Human Stem Cells, Changsha, China

   Contribution

   Conceptualization, Resources, Supervision, Project administration, Writing – review and editing

   For correspondence

   linggf@hotmail.com

   Competing interests

   No competing interests declared

10. Yu Sun

    1. Department of Mechanical Engineering, University of Toronto, Toronto, Canada
    2. Department of Electrical and Computer Engineering, Toronto, Canada
    3. Institute of Biomedical Engineering, University of Toronto, Toronto, Canada
    4. Department of Computer Science, University of Toronto, Toronto, Canada

    Contribution

    Conceptualization, Resources, Data curation, Formal analysis, Supervision, Funding acquisition, Validation, Investigation, Visualization, Methodology, Writing – original draft, Project administration, Writing – review and editing

    For correspondence

    sun@mie.utoronto.ca

    Competing interests

    No competing interests declared

    "This ORCID iD identifies the author of this article:" 0000-0001-7895-0741

• 969

  Page views

• 151

  Downloads

• 0

  Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.