Technical Summary: FUGC Benchmarking Semi-Supervised Learning for Cervical Segmentation

Problem Statement

Accurate segmentation of cervical structures in transvaginal ultrasound (TVS) is critical for assessing the risk of spontaneous preterm birth (PTB). However, the development of supervised deep learning models is hindered by the scarcity of labeled medical data, a challenge exacerbated by privacy constraints, the rarity of specific conditions, and the high cost of expert annotation. While semi-supervised learning (SSL) offers a solution by leveraging abundant unlabeled data, there is a significant gap in publicly available datasets and standardized evaluation frameworks specifically for SSL in cervical ultrasound segmentation. Existing benchmarks rely exclusively on fully supervised learning, and cervical segmentation itself presents unique difficulties, including signal dropouts, artifacts, missing boundaries, and anatomical variability.

Methodology

The paper introduces the Fetal Ultrasound Grand Challenge (FUGC), the first benchmark designed to evaluate SSL methods for cervical segmentation.

Dataset

The FUGC dataset comprises 890 TVS images acquired using GE Voluson E10 and E8 systems. The data is split into:

• Training: 500 images (50 labeled, 450 unlabeled).
• Validation: 90 images.
• Test: 300 images.
  Annotations for the anterior and posterior cervical lips were generated using a fine-tuned, ultrasound-adapted SAM model and subsequently refined by an experienced obstetric ultrasonographer (>10 years of experience). The dataset is publicly available under a CC BY 4.0 license.

Evaluation Framework

The challenge followed the Biomedical Image Analysis Challenges (BIAS) guidelines. Performance was evaluated using three metrics:

1. Dice Similarity Coefficient (DSC): Measured for the anterior lip (DSC A), posterior lip (DSC P), and overall cervix (DSC All).
2. Hausdorff Distance (HD): Measured for the same regions (HD A, HD P, HD All).
3. Runtime (RT): Inference time measured on specific hardware (Xeon Silver 4210R CPU, Quadro RTX 8000 GPU).

The final ranking utilized a weighted combination of these metrics: 0.4 for mDSC (average of DSC A and DSC P), 0.4 for mHD (average of HD A and HD P), and 0.2 for RT. A 402 ms threshold was applied to RT scoring to ensure highly accurate but slower methods were not unduly penalized.

Participating Solutions

The challenge attracted 10 winning teams (82 participants). The top-performing methods employed diverse strategies, including:

• Human-in-the-loop refinement: Iterative correction of pseudo-labels using annotation tools (Team T1).
• Mean Teacher Frameworks: Utilizing exponential moving average (EMA) teachers and consistency regularization (Teams T2, T3, T5, T6, T7, T8, T9).
• Foundation Models: Integration of Vision Transformers (ViT) such as DINOv2 and DINOv2-S as encoders (Teams T3, T5, T9).
• Multi-stage Pipelines: Two-stage or three-stage approaches involving initial pseudo-label generation (often via nnUNet) followed by refinement and retraining (Teams T1, T4, T7, T10).
• Data Augmentation: Extensive use of geometric transformations, photometric adjustments, and advanced techniques like Copy-Paste, CutMix, and Complementary Channel-Wise Dropout.

Key Results

• Overall Performance: The best-performing method (Team T1) achieved a final weighted score of 98.51%, balancing high accuracy with reasonable efficiency.
• Accuracy Metrics:
  • mDSC: The highest mDSC was 90.26% (Team T4).
  • mHD: The lowest (best) mHD was 38.88 (Team T4).
  • DSC All: Team T1 achieved 0.9336, while Team T4 achieved 0.9289.
• Efficiency Metrics:
  • Runtime: The fastest inference time was 32.85 ms (Team T6), while the slowest was 652.38 ms (Team T4).
• Trade-offs: A Pareto frontier analysis identified Teams T1, T2, and T4 as non-dominated solutions, offering optimal trade-offs between segmentation accuracy, boundary precision, and runtime. Team T1 secured the top rank by effectively balancing the accuracy of T4 with the efficiency of faster models.

Key Contributions

1. First SSL Benchmark for Cervical Ultrasound: FUGC establishes the first standardized, reproducible benchmark specifically for semi-supervised learning in transvaginal cervical segmentation, addressing a critical gap in the literature.
2. Comprehensive Dataset Release: The release of 890 TVS images with high-quality, expert-refined annotations provides a valuable resource for the medical imaging community.
3. Methodological Insights: The challenge demonstrates that competitive performance can be achieved with limited labeled data (50 images) through diverse SSL strategies, including offline pseudo-label refinement, online consistency regularization, and the integration of foundation models.
4. Open Science: All datasets, annotations, and source codes from the top 10 teams have been publicly released to foster transparency and further research.

Significance and Claims

The paper claims that FUGC serves as a catalyst for scalable, data-efficient cervical segmentation models, contributing to the quantitative study of uterine physiology and the development of AI tools for maternal-fetal medicine.

• Clinical Relevance: The results suggest that semi-supervised methods can approach the performance of state-of-the-art fully supervised algorithms (achieving DSC scores >93%), potentially enabling more accurate cervical length measurements and earlier, precise screening for PTB risk.
• Feasibility: The study demonstrates that well-designed SSL pipelines can mitigate the limitations of label scarcity. Furthermore, inference times as low as 32 ms indicate that these methods are moving toward real-time clinical applicability.
• Limitations: The authors modestly note that the reference annotations were provided by a single expert, limiting the assessment of inter-observer variability. Additionally, while some methods achieved low latency, further optimization is required for strict real-time deployment on edge devices, and future work should include more diverse clinical cohorts, particularly preterm cases.