Prospective study of automated versus manual annotation of early time-lapse markers in the human preimplantation embryo.

How does automated time-lapse annotation (Eeva™) compare to manual annotation of the same video images performed by embryologists certified in measuring durations of the 2-cell (P2; time to the 3-cell minus time to the 2-cell, or t3-t2) and 3-cell (P3; time to 4-cell minus time to the 3-cell, or t4-t3) stages? Manual annotation was superior to the automated annotation provided by Eeva™ version 2.2, because manual annotation assigned a rating to a higher proportion of embryos and yielded a greater sensitivity for blastocyst prediction than automated annotation. While use of the Eeva™ test has been shown to improve an embryologist's ability to predict blastocyst formation compared to Day 3 morphology alone, the accuracy of the automated image analysis employed by the Eeva™ system has never been compared to manual annotation of the same time-lapse markers by a trained embryologist. We conducted a prospective cohort study of embryos (n = 1477) cultured in the Eeva™ system (n = 8 microscopes) at our institution from August 2014 to February 2016. Embryos were assigned a blastocyst prediction rating of High (H), Medium (M), Low (L), or Not Rated (NR) by Eeva™ version 2.2 according to P2 and P3. An embryologist from a team of 10, then manually annotated each embryo and if the automated and manual ratings differed, a second embryologist independently annotated the embryo. If both embryologists disagreed with the automated Eeva™ rating, then the rating was classified as discordant. If the second embryologist agreed with the automated Eeva™ score, the rating was not considered discordant. Spearman's correlation (ρ), weighted kappa statistics and the intra-class correlation (ICC) coefficients with 95% confidence intervals (CI) between Eeva™ and manual annotation were calculated, as were the proportions of discordant embryos, and the sensitivity, specificity, positive predictive value (PPV) and NPV of each method for blastocyst prediction. The distribution of H, M and L ratings differed by annotation method (P < 0.0001). The correlation between Eeva™ and manual annotation was higher for P2 (ρ = 0.75; ICC = 0.82; 95% CI 0.82-0.83) than for P3 (ρ = 0.39; ICC = 0.20; 95% CI 0.16-0.26). Eeva™ was more likely than an embryologist to rate an embryo as NR (11.1% vs. 3.0%, P < 0.0001). Discordance occurred in 30.0% (443/1477) of all embryos and was not associated with factors such as Day 3 cell number, fragmentation, symmetry or presence of abnormal cleavage. Rather, discordance was associated with direct cleavage (P2 ≤ 5 h) and short P3 (≤0.25 h), and also factors intrinsic to the Eeva™ system, such as the automated rating (proportion of discordant embryos by rating: H: 9.3%; M: 18.1%; L: 41.3%; NR: 31.4%; P < 0.0001), microwell location (peripheral: 31.2%; central: 23.8%; P = 0.02) and Eeva™ microscope (n = 8; range 22.9-42.6%; P < 0.0001). Manual annotation upgraded 82.6% of all discordant embryos from a lower to a higher rating, and improved the sensitivity for predicting blastocyst formation. One team of embryologists performed the manual annotations; however, the study staff was trained and certified by the company sponsor. Only two time-lapse markers were evaluated, so the results are not generalizable to other parameters; likewise, the results are not generalizable to future versions of Eeva™ or other automated image analysis systems. Based on the proportion of discordance and the improved performance of manual annotation, clinics using the Eeva™ system should consider manual annotation of P2 and P3 to confirm the automated ratings generated by Eeva™. These data were acquired in a study funded by Progyny, Inc. There are no competing interests. N/A.

Kaser DJ ，Farland LV ，Missmer SA ，Racowsky C ... -  《-》

A pilot randomized controlled trial of Day 3 single embryo transfer with adjunctive time-lapse selection versus Day 5 single embryo transfer with or without adjunctive time-lapse selection.

Compared to D5 selection with conventional morphology (CM), does adjunctive use of the Eeva™ test on D3 or D5 improve the clinical pregnancy rate (CPR) per transfer? The evidence is insufficient to conclude that adjunctive use of the Eeva™ test on D3 or D5 improves CPR per transfer as compared to D5 selection with CM. Time-lapse imaging is increasingly used for embryo selection, despite there being no class I data to support its clinical application. Pilot randomized controlled trial included 163 patients from August 2014 to February 2016. Patients up to age 41 years with a planned fresh autologous single embryo transfer (SET), less than four prior oocyte retrievals, and four or more zygotes were blocked according to age (<35, 35-37, 38-40 years) and randomized to one of three study arms: (1) D3 SET + EevaTM, (2) D5 SET + Eeva™ or (3) D5 SET with CM alone. All embryos were cultured in the same time-lapse system under identical conditions. Intention-to-treat (ITT) and as-treated analyses of the primary endpoint (CPR at 7 weeks) and secondary endpoint (ongoing pregnancy rate at 12 weeks) were performed. Multivariate regression analyses adjusted for patient age and ICSI. Of 478 eligible patients, 217 consented and 163 were randomized. Demographic characteristics were similar among the three study arms. There were no statistically significant differences in the clinical pregnancy rate or the ongoing pregnancy rate between the study arms for either the ITT or as-treated analyses (CPR ITT: D3 + Eeva™: 41.1% vs. D5 + Eeva™: 38.9% vs. D5 CM: 49.1%). This study was designed as a pilot randomized controlled trial and was not powered to detect a statistically significant difference at α < 0.05. Importantly, the study was terminated prematurely by the sponsor due to a change in funding priorities, so the sample size is limited and the results should be interpreted with caution due to the role of chance. Furthermore, these findings may not be generalizable to other time-lapse systems. Our findings do not support the clinical application of these time-lapse markers. This study was funded by Progyny, Inc. There are no competing interests. clinicaltrials.gov: NCT02218255. 14 August 2014. 3 September 2014.

Kaser DJ ，Bormann CL ，Missmer SA ，Farland LV ，Ginsburg ES ，Racowsky C ... -  《-》

Clinical validation of an automatic classification algorithm applied on cleavage stage embryos: analysis for blastulation, euploidy, implantation, and live-birth potential.

Is a commercially available embryo assessment algorithm for early embryo evaluation based on the automatic annotation of morphokinetic timings a useful tool for embryo selection in IVF cycles? The classification provided by the algorithm was shown to be significantly predictive, especially when combined with conventional morphological evaluation, for development to blastocyst, implantation, and live birth, but not for euploidy. The gold standard for embryo selection is still morphological evaluation conducted by embryologists. Since the introduction of time-lapse technology to embryo culture, many algorithms for embryo selection have been developed based on embryo morphokinetics, providing complementary information to morphological evaluation. However, manual annotations of developmental events and application of algorithms can be time-consuming and subjective processes. The introduction of automation to morphokinetic annotations is a promising approach that can potentially reduce subjectivity in the embryo selection process and improve the workflow in IVF laboratories. This observational, retrospective cohort study was performed in a single IVF clinic between 2018 and 2021 and included 3736 embryos from oocyte donation cycles (423 cycles) and 1291 embryos from autologous cycles with preimplantation genetic testing for aneuploidies (PGT-A, 185 cycles). Embryos were classified on Day 3 with a score from 1 (best) to 5 (worst) by the automatic embryo assessment algorithm. The performance of the embryo classification model for blastocyst development, implantation, live birth, and euploidy prediction was assessed. All embryos were monitored by a time-lapse system with an automatic cell-tracking and embryo assessment software during culture. The embryo assessment algorithm was applied on Day 3, resulting in embryo classification from 1 to 5 (from highest to lowest developmental potential) depending on four parameters: P2 (t3-t2), P3 (t4-t3), oocyte age, and number of cells. There were 959 embryos selected for transfer on Day 5 or 6 based on conventional morphological evaluation. The blastocyst development, implantation, live birth, and euploidy rates (for embryos subjected to PGT-A) were compared between the different scores. The correlation of the algorithm scoring with the occurrence of those outcomes was quantified by generalized estimating equations (GEEs). Finally, the performance of the GEE model using the embryo assessment algorithm as the predictor was compared to that using conventional morphological evaluation, as well as to a model using a combination of both classification systems. The blastocyst rate was higher with lower the scores generated by the embryo assessment algorithm. A GEE model confirmed the positive association between lower embryo score and higher odds of blastulation (odds ratio (OR) (1 vs 5 score) = 15.849; P < 0.001). This association was consistent in both oocyte donation and autologous embryos subjected to PGT-A. The automatic embryo classification results were also statistically associated with implantation and live birth. The OR of Score 1 vs 5 was 2.920 (95% CI 1.440-5.925; P = 0.003; E = 2.81) for implantation and 3.317 (95% CI 1.615-6.814; P = 0.001; E = 3.04) for live birth. However, this association was not found in embryos subjected to PGT-A. The highest performance was achieved when combining the automatic embryo scoring and traditional morphological classification (AUC for implantation potential = 0.629; AUC for live-birth potential = 0.636). Again, no association was found between the embryo classification and euploidy status in embryos subjected to PGT-A (OR (1 vs 5) = 0.755 (95% CI 0.255-0.981); P = 0.489; E = 1.57). The retrospective nature of this study may be a reason for caution, although the large sample size reinforced the ability of the model for embryo selection. Time-lapse technology with automated embryo assessment can be used together with conventional morphological evaluation to increase the accuracy of embryo selection process and improve the success rates of assisted reproduction cycles. To our knowledge, this is the largest embryo dataset analysed with this embryo assessment algorithm. This research was supported by Agencia Valenciana de Innovació and European Social Fund (ACIF/2019/264 and CIBEFP/2021/13). In the last 5 years, M.M. received speaker fees from Vitrolife, Merck, Ferring, Gideon Richter, Angelini, and Theramex, and B.A.-R. received speaker fees from Merck. The remaining authors have no competing interests to declare. N/A.

Valera MA ，Aparicio-Ruiz B ，Pérez-Albalá S ，Romany L ，Remohí J ，Meseguer M ... -  《-》

Development of a generally applicable morphokinetic algorithm capable of predicting the implantation potential of embryos transferred on Day 3.

Can a generally applicable morphokinetic algorithm suitable for Day 3 transfers of time-lapse monitored embryos originating from different culture conditions and fertilization methods be developed for the purpose of supporting the embryologist's decision on which embryo to transfer back to the patient in assisted reproduction? The algorithm presented here can be used independently of culture conditions and fertilization method and provides predictive power not surpassed by other published algorithms for ranking embryos according to their blastocyst formation potential. Generally applicable algorithms have so far been developed only for predicting blastocyst formation. A number of clinics have reported validated implantation prediction algorithms, which have been developed based on clinic-specific culture conditions and clinical environment. However, a generally applicable embryo evaluation algorithm based on actual implantation outcome has not yet been reported. Retrospective evaluation of data extracted from a database of known implantation data (KID) originating from 3275 embryos transferred on Day 3 conducted in 24 clinics between 2009 and 2014. The data represented different culture conditions (reduced and ambient oxygen with various culture medium strategies) and fertilization methods (IVF, ICSI). The capability to predict blastocyst formation was evaluated on an independent set of morphokinetic data from 11 218 embryos which had been cultured to Day 5. PARTICIPANTS/MATERIALS, SETTING, The algorithm was developed by applying automated recursive partitioning to a large number of annotation types and derived equations, progressing to a five-fold cross-validation test of the complete data set and a validation test of different incubation conditions and fertilization methods. The results were expressed as receiver operating characteristics curves using the area under the curve (AUC) to establish the predictive strength of the algorithm. By applying the here developed algorithm (KIDScore), which was based on six annotations (the number of pronuclei equals 2 at the 1-cell stage, time from insemination to pronuclei fading at the 1-cell stage, time from insemination to the 2-cell stage, time from insemination to the 3-cell stage, time from insemination to the 5-cell stage and time from insemination to the 8-cell stage) and ranking the embryos in five groups, the implantation potential of the embryos was predicted with an AUC of 0.650. On Day 3 the KIDScore algorithm was capable of predicting blastocyst development with an AUC of 0.745 and blastocyst quality with an AUC of 0.679. In a comparison of blastocyst prediction including six other published algorithms and KIDScore, only KIDScore and one more algorithm surpassed an algorithm constructed on conventional Alpha/ESHRE consensus timings in terms of predictive power. Some morphological assessments were not available and consequently three of the algorithms in the comparison were not used in full and may therefore have been put at a disadvantage. Algorithms based on implantation data from Day 3 embryo transfers require adjustments to be capable of predicting the implantation potential of Day 5 embryo transfers. The current study is restricted by its retrospective nature and absence of live birth information. Prospective Randomized Controlled Trials should be used in future studies to establish the value of time-lapse technology and morphokinetic evaluation. Algorithms applicable to different culture conditions can be developed if based on large data sets of heterogeneous origin. This study was funded by Vitrolife A/S, Denmark and Vitrolife AB, Sweden. B.M.P.'s company BMP Analytics is performing consultancy for Vitrolife A/S. M.B. is employed at Vitrolife A/S. M.M.'s company ilabcomm GmbH received honorarium for consultancy from Vitrolife AB. D.K.G. received research support from Vitrolife AB.

Petersen BM ，Boel M ，Montag M ，Gardner DK ... -  《-》

Inter- and intra-observer variability of time-lapse annotations.

Sundvall L ，Ingerslev HJ ，Breth Knudsen U ，Kirkegaard K ... -  《-》