Adapting Foundation Models for Annotation-Efficient Adnexal Mass Segmentation in Cine Images

1. Problem Statement

The evaluation of adnexal masses (ovarian lesions) via ultrasound is a critical but challenging clinical task. Accurate differentiation between benign and malignant lesions is vital for patient outcomes, yet current workflows rely heavily on subjective operator interpretation, leading to significant inter-observer variability.

• Challenges: Ultrasound images suffer from speckle noise, acoustic shadowing, and low contrast, making boundary delineation difficult.
• Data Scarcity: Traditional automated segmentation methods (e.g., U-Net variants) rely on fully supervised training, which requires massive amounts of high-quality, pixel-level annotations. These are time-consuming to produce and often insufficient to handle domain shifts or data scarcity in clinical settings.
• Goal: Develop a segmentation framework that achieves state-of-the-art accuracy while being highly efficient in terms of annotation requirements (label-efficient).

2. Methodology

The authors propose a label-efficient segmentation framework that leverages the robust semantic priors of a self-supervised foundation model combined with a dense prediction decoder.

Architecture

• Backbone (Encoder): The model utilizes a frozen DINOv3 (Self-Distilled Vision Transformer) backbone.
  • DINOv3 is pre-trained on massive natural image datasets, providing robust semantic representations.
  • Since DINOv3 expects 3-channel RGB input, grayscale ultrasound cine frames are converted to pseudo-RGB via channel replication.
  • The encoder extracts hierarchical feature maps ($F_0, F_1, F_2, F_3$) at different depths, discarding classification tokens to focus on spatial localization.
• Decoder: A Dense Prediction Transformer (DPT) style decoder is employed.
  • It hierarchically reassembles multi-scale features from the encoder.
  • The process involves resampling features to a common embedding space, reassembly via residual fusion blocks (merging coarse semantic context with fine-grained spatial details), and a final prediction head that outputs a binary segmentation mask.
• Training Strategy:
  • The DINOv3 backbone is frozen during training to preserve pre-trained knowledge and reduce computational cost.
  • Only the decoder and prediction head are trained in a fully supervised manner using a combination of Binary Cross-Entropy (BCE) and Dice Loss to address class imbalance and boundary ambiguity.

3. Key Contributions

1. Novel Architecture Adaptation: The first application of a frozen DINOv3 backbone with a DPT-style decoder specifically for adnexal mass segmentation in ultrasound cine images.
2. Comprehensive Benchmarking: A rigorous evaluation against established Convolutional Neural Network (CNN) baselines (U-Net, U-Net++, DeepLabV3, MAnet) using patient-level splits to prevent data leakage.
3. Efficiency Analysis: A systematic investigation into:
   • Backbone Scale: How model size (Small vs. Base vs. Large) affects performance.
   • Data Starvation: Performance retention when training data is drastically reduced.
   • Resolution Effects: The impact of input image resolution on learning efficiency.

4. Experimental Results

The model was evaluated on a clinical dataset of 7,777 annotated frames from 112 patients.

Quantitative Performance (224×224 Resolution)

The proposed DINOv3-based method outperformed all CNN baselines:

• Dice Score: 0.945 (vs. 0.939 for the best baseline, U-Net).
• Boundary Accuracy: Reduced the 95th-percentile Hausdorff Distance (HD95) by 11.4% compared to U-Net++.
• Mean Surface Distance (MSD): Reduced by 14.1% compared to U-Net.
• Sensitivity: Improved by nearly 1.4%.

Efficiency Analysis Findings

• Data Starvation (The "Retention Index"):
  • When trained on only 25% of the data, the DINOv3-based models retained ~96% of their full-data performance (Retention Index $\approx$ 0.96).
  • In contrast, CNN baselines (U-Net, etc.) dropped to ~80-90% retention, struggling significantly with limited data.
• Backbone Scale Saturation:
  • Scaling from DINOv3-Small to Base yielded marginal gains.
  • Scaling to DINOv3-Large resulted in performance degradation (overfitting), suggesting that for this specific medical task, excessive capacity amplifies noise rather than improving generalization. The "Base" or "Small" variants represent the Pareto-optimal frontier.
• Resolution Effects:
  • Counter-intuitively, increasing input resolution from 224×224 to 512×512 degraded performance.
  • The authors attribute this to a "flatter" optimization landscape where the frozen backbone's gradient signals become less informative at higher resolutions, hindering convergence.

5. Significance and Conclusion

• Annotation Efficiency: The study demonstrates that leveraging large-scale self-supervised foundations allows for high-performance segmentation with significantly fewer labeled examples. This is crucial for medical domains where expert annotations are scarce and expensive.
• Robustness: The framework acts as a strong structural regularizer, stabilizing learning against the inherent noise (speckle, shadowing) and boundary ambiguity of ultrasound imaging.
• Practical Implications: The results suggest that for boundary-sensitive medical tasks, moderately sized, distilled transformer backbones offer a superior trade-off between accuracy, robustness, and computational efficiency compared to both larger foundation models and traditional CNNs.
• Future Outlook: While the results are promising, the authors note the need for multi-center validation to confirm cross-site generalization and further evaluation of the downstream impact on malignancy risk stratification.

Repository: The code is available at github.com/FrancescaFati/MESA.