Data-Efficient Surgical Phase Segmentation in Small-Incision Cataract Surgery: A Controlled Study of Vision Foundation Models

1. Problem Statement

The paper addresses the challenge of surgical phase segmentation in Small-Incision Cataract Surgery (SICS), a procedure widely performed in low-resource settings.

• Core Challenge: Developing robust models is difficult due to the scarcity of labeled surgical videos. Phase segmentation requires dense, frame-level annotations across long procedures, which is expensive and time-consuming to produce.
• Specific Constraints: The target dataset, SICS-155, contains only 155 videos with 19 distinct phases. The data exhibits severe long-tail class imbalance (e.g., "cautery" accounts for only 1.7% of frames) and short contiguous phase durations (median ~6.5 seconds for some phases).
• Goal: To determine if Vision Foundation Models (VFMs) pre-trained on massive unlabeled datasets can provide superior, data-efficient representations for surgical workflow understanding compared to traditional supervised encoders, without requiring extensive task-specific retraining.

2. Methodology

The authors propose a controlled, factorized framework to isolate the quality of visual representations from temporal modeling capabilities.

A. Controlled Pipeline Design

Instead of training end-to-end (which is computationally prohibitive for large models on small datasets), the study decouples the process:

1. Frozen Visual Encoder: A pre-trained encoder extracts features from frames. These encoders are frozen during the downstream task.
2. Feature Caching: Features are extracted offline and cached to decouple expensive encoding from lightweight temporal learning.
3. Shared Temporal Head: All encoders feed into the same temporal model, MS-TCN++, under identical training settings (optimization, splits, loss functions).
   • Rationale: This ensures that performance differences are strictly attributable to the feature representation quality, not variations in the temporal architecture or training hyperparameters.

B. Encoders Compared

The study compares two categories of encoders:

• Supervised Baselines:
  • ResNet-50: Pre-trained on ImageNet (2D).
  • I3D: Pre-trained on Kinetics (3D video).
• Self-Supervised Foundation Models:
  • DINOv3: Image-based Vision Transformers (ViT-B, ViT-L, ViT-7B) pre-trained on LVD-1689M.
  • V-JEPA2: Video-based Vision Transformers (ViT-L, ViT-g) pre-trained by Meta.
  • Note on V-JEPA2: Since it operates on clips (64 frames), a sliding window with stride 4 and linear interpolation is used to generate per-frame features, which may smooth rapid transitions.

C. Domain Adaptation Strategy: CataractFT

To explore data efficiency further, the authors introduce CataractFT, a transfer learning workflow:

1. SSL Continuation: Self-supervised pre-training on 1,000 unlabeled cataract videos (Cataract-1K) using the encoder's native objective (DINO self-distillation or JEPA masked prediction).
2. LoRA Adaptation: Optional Low-Rank Adaptation (LoRA) fine-tuning on 56 annotated Cataract-1K and 101 Cataract-101 surgeries.
3. Freezing: The adapted encoder is frozen, and features are extracted for the SICS-155 temporal head.

3. Key Contributions

1. Controlled Benchmark: A systematic evaluation of supervised vs. self-supervised foundation models for surgical phase segmentation, using a shared MS-TCN++ head and stratified 5-fold cross-validation to eliminate confounding variables.
2. Empirical Scaling Analysis: An investigation into how model size and pretraining scale affect performance in a low-label medical setting, interpreting trends within the DINOv3 family (ViT-B < ViT-L < ViT-7B).
3. CataractFT Workflow: A practical pipeline for leveraging unlabeled related-domain data (phacoemulsification) via SSL continuation and LoRA, analyzing when transfer helps or hurts.
4. Practical Guidance: Evidence that foundation models can serve as "drop-in" replacements for supervised encoders in low-label medical video tasks, provided the temporal model is fixed.

4. Results

A. Cross-Model Comparison (SICS-155)

• Foundation Models Outperform Supervised Baselines: Self-supervised models generally outperformed ResNet-50 and I3D.
• Top Performer: DINOv3 ViT-7B achieved the best overall results:
  • Accuracy: 83.4%
  • Edit Score: 87.0
  • Macro-F1: 76.5%
• Scaling Trend: Within the DINOv3 family, performance scaled with model size (ViT-B < ViT-L < ViT-7B).
• V-JEPA2 Underperformance: Despite being video-native, V-JEPA2 underperformed. The authors attribute this to the stride-4 extraction/interpolation smoothing rapid phase transitions and potential domain shift from generic video pretraining.

B. Domain Adaptation (CataractFT)

The impact of transfer learning was mixed and backbone-dependent:

• ViT-B (Smaller Model): Benefited significantly from CataractFT + LoRA.
  • Accuracy increased by +2.86% and Macro-F1 by +2.87% compared to the base.
  • LoRA helped the smaller model specialize for the cataract domain.
• ViT-L (Larger Model): Showed negative transfer for aggregate metrics.
  • Accuracy and F1 scores dropped compared to the base model.
  • However, Edit Score (temporal consistency) improved slightly with SSL-only continuation (no LoRA), suggesting the larger model's representation was already robust, and LoRA introduced over-specialization that fragmented temporal sequences.
• Phase-Specific Effects: Gains were concentrated in phases with distinct visual signatures (e.g., cautery, sideport), while amorphous phases (e.g., cortical wash) saw little improvement or regression.

5. Significance and Implications

• Data Efficiency: The study demonstrates that large, self-supervised foundation models can achieve state-of-the-art performance in surgical phase segmentation with minimal task-specific labels, making them ideal for low-resource medical settings.
• Deployment Trade-offs: While ViT-7B offers peak accuracy, ViT-L may offer a better accuracy-to-compute trade-off for real-world deployment (ViT-7B requires ~13.4 GB VRAM).
• Transfer Learning Nuance: Simply adding more data or fine-tuning on related domains does not guarantee improvement. The effectiveness of transfer depends heavily on the backbone capacity and the nature of the phase (localized instruments vs. amorphous processes).
• Metric Importance: The study highlights the necessity of reporting both frame-level metrics (accuracy, F1) and segment-level metrics (Edit Score). Adaptation can improve frame accuracy while degrading temporal smoothness (introducing "choppy" predictions), a critical failure mode in surgical workflows.

Conclusion: The paper establishes that modern vision foundation models are highly transferable to surgical workflow understanding. It provides a reproducible, controlled benchmark and practical guidelines for teams building surgical AI systems with limited annotated data, advocating for a "feature caching + fixed temporal head" approach.