AURORA: Adaptive Unified Representation for Robust Ultrasound Analysis

1. Problem Statement

Ultrasound imaging faces significant challenges in automated analysis due to high variability across different scanners, operators, and anatomical targets. These variations (e.g., probe pose, speckle statistics, vendor differences) cause models trained in one setting to generalize poorly to new hospitals or clinical conditions.

Furthermore, existing foundation models in medical imaging are often single-task, meaning a separate model is required for segmentation, detection, classification, or landmark regression. This limits efficiency and hinders the development of a unified system capable of handling the diverse requirements of the Foundation Model Challenge for Ultrasound Image Analysis (FMC-UIA), which demands a single model to perform all these tasks across unseen domains.

2. Methodology: AURORA Framework

The authors propose AURORA, a unified multi-task framework built upon a Qwen3-VL transformer visual encoder. The architecture is designed to bridge the gap between global transformer token representations and dense spatial predictions.

Core Architecture Components:

• Backbone (Visual Encoder): The model utilizes the Qwen3-VL-8B vision encoder (discarding the language model to save memory). Inputs are resized to a fixed square resolution ($448 \times 448$) and processed as 3-channel grayscale images.
• Token-to-Pyramid Bridging:
  • Intermediate token activations are extracted from four transformer blocks at different depths.
  • These tokens are reshaped into spatial feature maps ($h \times w$) and projected to a shared channel width ($D=256$) using a learnable $1\times1$ convolution.
  • A Feature Pyramid Network (FPN) with top-down and lateral connections fuses these maps into a four-level hierarchy ($P_2$ to $P_5$) at scales $\{1, 1/2, 1/4, 1/8\}$.
• Dual Feature Pathways:
  1. Dense Pathway (for Segmentation & Detection): Operates on the multi-scale FPN features.
  2. Global Pathway (for Classification & Regression): Operates on a pooled representation created by concatenating Global Average Pooled (GAP) vectors from all pyramid levels ($e = [GAP(P_2); \dots; GAP(P_5)]$).

Task-Specific Heads:

• Segmentation Head: Uses the full multi-scale pyramid. Features are refined via residual blocks and channel recalibration. An Atrous Spatial Pyramid Pooling (ASPP) module is applied at the coarsest level to expand the receptive field. A decoder with gated skip connections upsamples features to the input resolution.
• Detection Head: An anchor-free grid predictor operating on the finest pyramid level ($P_2$). It concatenates $P_2$ with upsampled $P_3$ and $P_4$ (plus coordinate channels) to predict bounding boxes and objectness scores.
• Classification Head: Operates on the pooled global embedding $e$. It uses a stack of gated MLP residual blocks to predict class logits.
• Regression Head: Also operates on the global embedding $e$. It predicts normalized coordinates ($[0, 1]$) for landmarks using a linear regressor with a sigmoid activation.

Optimization & Training Strategy:

• Task-Aware Sampling: To address task imbalance, a temperature-based sampler mixes task frequencies with a loss-Exponential Moving Average (EMA).
• Dynamic Loss Balancing: Uses selective loss reweighting with a dynamic factor ($\gamma=0.5$) clamped between $[0.25, 4.0]$ to prevent dominant tasks from overshadowing others.
• Training Setup: Trained on $448 \times 448$ inputs using BF16 mixed precision, gradient checkpointing, and an effective batch size of 16.

3. Key Contributions

1. Unified Multi-Task Framework: A single model based on a pretrained Qwen3-VL encoder that jointly handles segmentation, detection, classification, and landmark regression, eliminating the need for task-specific models.
2. Token-to-Map Bridging Mechanism: A novel method to convert intermediate transformer token activations into multi-scale spatial feature maps via an FPN, enabling the model to support both dense pixel-level predictions and global coordinate reasoning.
3. Efficient Optimization Strategy: Implementation of task-aware sampling and selective loss balancing to manage heterogeneous supervision and improve training stability across diverse ultrasound tasks.

4. Experimental Results

The model was evaluated on the FMC-UIA 2026 benchmark, which includes 27 subtasks across unseen domains.

• Performance Gains: The score improved from 67% to 85% on the validation set during development. On the official test set, the model achieved an average score of 81.84% across all tasks.
• Ablation Studies:
  • Backbone: Qwen3-VL-8B consistently outperformed Qwen2.5-VL-7B, confirming that backbone capacity and pretraining quality are critical for generalization.
  • Architecture: Introducing the Token Pyramid with FPN fusion yielded substantial improvements for dense prediction tasks compared to using the backbone alone.
• Task-Specific Performance:
  • Classification: Achieved high discrimination (e.g., AUC > 0.97 for fetal head position).
  • Detection: Solid localization performance (e.g., Thyroid nodule detection IoU of 0.717).
  • Segmentation: Strong performance on lung and cervix (Dice > 0.85), though cardiac multi-class segmentation showed degradation due to anatomical complexity.
  • Regression: Showed variability in Mean Radial Error (MRE), suggesting challenges in geometric fidelity when predicting landmarks from pooled features.

5. Significance and Conclusion

AURORA demonstrates that a single, unified foundation model can effectively adapt to the diverse and challenging nature of ultrasound analysis. By leveraging a large-scale vision-language encoder (Qwen3-VL) and adapting its intermediate representations for medical tasks, the framework achieves competitive performance across heterogeneous objectives.

Limitations & Future Work:

• Negative Transfer: Potential interference between tasks when sharing a single backbone.
• Small Targets: Reduced sensitivity in detecting very small or ambiguous targets.
• Resolution Loss: Resizing inputs to a fixed resolution may cause a loss of fine geometric details.
• Future Directions: The authors plan to explore stronger task decoupling (e.g., adapters), scale-aware detection mechanisms, and resolution-aware preprocessing to further enhance robustness.

The code is publicly available, providing a strong baseline for future research in unified medical imaging foundation models.