Predicting Blastocyst Formation in IVF: Integrating DINOv2 and Attention-Based LSTM on Time-Lapse Embryo Images

1. Problem Statement

In Vitro Fertilization (IVF) relies heavily on selecting the optimal embryo for transfer, a process traditionally dependent on manual inspection of time-lapse imaging (TLM) data. This manual evaluation is time-consuming, subjective, and expensive.

• The Core Challenge: While full TLM videos provide continuous data, many IVF clinics lack complete time-lapse systems or suffer from interrupted imaging, resulting in sparse data (limited daily images rather than continuous video).
• The Gap: Existing deep learning models often require full video sequences or focus solely on the final blastocyst stage, failing to effectively utilize early and middle-stage growth patterns from incomplete datasets. There is a need for a model that can predict blastocyst formation accurately using only a few daily snapshots (e.g., one image per day) without relying on continuous high-frequency monitoring.

2. Methodology

The authors propose a novel hybrid deep learning architecture designed to handle sparse, sequential embryo images. The pipeline consists of three main stages:

A. Data Preprocessing

• Input: Time-lapse videos are segmented into a maximum of $m=7$ frames per embryo, spaced at 24-hour intervals ($\Delta t = 24h$) to mimic clinical observation routines.
• Image Processing: Raw images are converted to grayscale and resized to 518×518 pixels to match the input requirements of the feature extractor.
• Handling Variable Lengths: Since embryo development varies (some arrest early), sequences have different lengths. The authors use zero-padding to standardize all sequences to a maximum length ($L_{max}$), ensuring uniform input dimensions for the temporal model.

B. Spatial Feature Extraction (DINOv2)

• Model: The authors utilize DINOv2, a self-supervised Vision Transformer (ViT) foundation model pre-trained on 142 million unlabeled images.
• Mechanism: DINOv2 extracts high-dimensional visual embeddings from each frame without requiring manual labeling.
• Feature Engineering: The model extracts both the global [CLS] token embedding and the average-pooled per-patch embeddings. These are concatenated to form a 1×1536 dimensional embedding vector for each frame, capturing rich morphological and contextual features.

C. Temporal Modeling (LSTM with Multi-Head Attention)

• Architecture: The sequence of 1536-dimensional embeddings is fed into a Long Short-Term Memory (LSTM) network enhanced with a Multi-Head Attention (MHA) mechanism.
• LSTM Role: Captures dynamic temporal patterns and long-term dependencies in embryonic development (e.g., cell division rates, morphological changes).
• Attention Role: The MHA layer allows the model to re-evaluate the importance of specific frames relative to the entire sequence. This helps the model focus on critical developmental stages (e.g., blastocoel appearance) even if they are not in consecutive frames, addressing the limitation of standard LSTMs in capturing non-local dependencies.
• Classification: The aggregated temporal features are flattened and passed through a classification head to predict a binary outcome: Blastocyst (positive) vs. Non-Blastocyst (negative).
• Loss Function: A weighted binary cross-entropy loss is used to address class imbalance between the two categories.

3. Key Contributions

1. Hybrid Architecture: Development of a novel DINOv2-LSTM-MHA framework that effectively combines state-of-the-art spatial feature extraction with advanced temporal modeling.
2. Sparse Data Capability: The model is specifically designed to work with limited daily images (sparse input), reducing dependency on expensive, continuous time-lapse incubators and making it accessible to a wider range of clinics.
3. Attention Mechanism Integration: Demonstrates that integrating Multi-Head Attention significantly improves the model's ability to identify critical developmental milestones compared to standard sequential models.
4. Robustness: The approach reduces computational costs by processing fewer frames while maintaining high accuracy, offering a practical solution for real-world clinical constraints.

4. Experimental Results

The model was evaluated on a real-world dataset of 704 embryo videos (499 suitable for transfer, 182 discarded).

• Performance Metrics:
  • Accuracy: 96.4% (surpassing existing methods).
  • Precision/Recall/F1:
    • Blastocyst Class: ~0.971
    • Non-Blastocyst Class: ~0.918
  • AUC (Area Under Curve): 0.99, indicating exceptional discriminative power.
• Comparative Analysis:
  • The proposed model outperformed previous benchmarks, including a ResNet50-GRU model (93% accuracy) and various CNN/DenseNet combinations.
  • Crucially, it showed a better balance between precision and recall for the "Non-Blastocyst" class compared to the ResNet50-GRU model, which had a low recall (0.77) for negative cases.
• Ablation Study:
  • Removing the Multi-Head Attention layer (using standard LSTM only) dropped the accuracy from 96.43% to 93.86%, confirming the contribution of the attention mechanism.
• Generalizability: The model was tested on four external time-series datasets (Car, Financial Distress, Occupancy Detection, EEG Eye State), achieving high accuracy (e.g., 99% on Occupancy Detection), proving its adaptability to various sequential data types.

5. Significance and Limitations

Significance:

• Clinical Impact: The model assists embryologists in selecting viable embryos with greater confidence and efficiency, potentially reducing the number of IVF cycles required and lowering patient stress.
• Accessibility: By functioning effectively with sparse data, it democratizes advanced AI-assisted embryo selection for clinics lacking full time-lapse infrastructure.
• Scientific Contribution: Validates the use of foundation models (DINOv2) combined with attention mechanisms for medical time-series analysis.

Limitations:

• Data Scarcity: The study relies on a single dataset from one IVF center. The lack of publicly available, multi-center TLI datasets limits the generalizability of the findings across different laboratories and equipment.
• Future Work: The authors suggest future research should incorporate patient clinical profiles and validate the model on diverse, multi-center datasets to ensure robustness.

In conclusion, this paper presents a highly effective, accessible AI solution for IVF embryo selection that bridges the gap between advanced deep learning capabilities and the practical limitations of clinical imaging data.