Evidential learning driven Breast Tumor Segmentation with Stage-divided Vision-Language Interaction

1. Problem Statement

Breast cancer is a leading cause of death among women, and Magnetic Resonance Imaging (MRI), specifically Dynamic Contrast-Enhanced MRI (DCE-MRI), is a critical tool for diagnosis due to its high sensitivity. However, automated tumor segmentation using deep learning faces significant challenges:

• Low Contrast & Blurred Boundaries: There is often a subtle intensity difference between tumor regions and adjacent normal tissues, making precise contour localization difficult.
• Limitations of Image-Only Models: Traditional deep learning methods (e.g., U-Net variants) rely solely on visual features, which often struggle to distinguish tumors in low-contrast scenarios or handle ambiguous boundaries, leading to false positives or negatives.
• Uncertainty Quantification: Standard probabilistic outputs (Softmax) tend to be overconfident, failing to adequately represent the uncertainty inherent in blurred medical image boundaries.

2. Methodology: TextBCS Framework

The authors propose TextBCS, a novel framework that integrates text prompts describing tumor attributes with image data. The architecture consists of three core components:

A. Stage-Divided Vision-Language Interaction (SVLI)

Unlike previous methods that fuse text and image features only at the final stage or via shallow connections, SVLI facilitates deep, bidirectional interaction at every down-sampling stage of the encoder.

• Bidirectional Cross-Attention: At each stage $s$, the module alternately processes visual and textual features.
  • Vision Query: Visual features act as queries, while text features serve as keys and values to extract text-aware visual features.
  • Language Query: Text features act as queries, while the updated visual features serve as keys and values to extract vision-aware text features.
• Stage-Divided Cross-Modal Alignment Loss: A contrastive loss is applied at every feature level (not just the final output) to align low-level and high-level image-text representations in a shared embedding space. This ensures the model utilizes text guidance throughout the feature extraction process.

B. Evidential Learning (EL) for Uncertainty Estimation

To address the issue of overconfident predictions on blurred boundaries, the authors replace the standard Softmax layer with an Evidential Learning strategy based on Subjective Logic and the Dirichlet Distribution.

• Evidence Accumulation: Instead of predicting probabilities directly, the network outputs non-negative "evidence" values ($e$) for each class.
• Dirichlet Distribution: These evidences parameterize a Dirichlet distribution ($Dir(p|\alpha)$), where $\alpha = e + 1$. This models the distribution of the segmentation probabilities.
• Uncertainty Quantification: The model explicitly calculates an "uncertainty mass" ($u$). In cases of blurred boundaries or ambiguous regions, the model assigns higher uncertainty, preventing it from making unreliable, overconfident predictions.
• Loss Function: The training objective includes a cross-entropy loss reformulated for evidential learning ($L_{ice}$) and a Kullback-Leibler (KL) divergence term ($L_{KL}$) to regularize the evidence and minimize uncertainty for incorrect classes.

C. Total Loss Function

The model is optimized using a weighted sum of four losses:
$$L_{total} = L_{Dice} + \lambda_1 L_{ice} + \lambda_2 L_{KL} + \lambda_3 L_{con}$$
Where $L_{Dice}$ ensures segmentation accuracy, $L_{ice}$ and $L_{KL}$ handle evidential uncertainty, and $L_{con}$ enforces image-text alignment.

3. Key Contributions

1. First Text-Guided DCE-MRI Segmentation: Introduces TextBCS, the first method to utilize text prompts (e.g., location, shape, size, number) to guide breast tumor segmentation in DCE-MRI.
2. Stage-Divided Interaction: Proposes the SVLI module, which ensures continuous and deep fusion of visual and linguistic information at every encoder stage, significantly improving the model's ability to locate lesions in low-contrast scenarios.
3. Uncertainty-Aware Segmentation: Incorporates Evidential Learning to quantify segmentation uncertainty. By modeling the distribution of probabilities via a variational Dirichlet function, the model avoids "false signals" in ambiguous boundary regions.
4. Comprehensive Validation: Demonstrates state-of-the-art performance on a public dataset, outperforming both traditional image-only networks and existing text-guided vision-language models.

4. Experimental Results

The method was evaluated on the Duke-Breast-Cancer-MRI dataset (922 patients, 3,876 slices). Text prompts were generated by radiologists based on lesion attributes.

• Performance Metrics:
  • Dice Score: 85.33% (outperforming the best baseline TransUNet by 2.19% and the best text-guided method MGCA by 1.05%).
  • mIoU: 76.08%.
• Ablation Studies:
  • Adding SVLI to the baseline UNet improved Dice by 2.87%.
  • Adding EL improved Dice by 1.65%.
  • Combining both yielded the full 3.79% improvement over the baseline.
• Statistical Significance: Paired t-tests confirmed that the improvements over all compared methods (including UNet, CLIP, and LViT) are statistically significant ($p < 0.05$).
• Qualitative Analysis: Visualizations showed TextBCS produces results more consistent with ground truth, particularly in multi-lesion cases and regions with low contrast. Saliency maps confirmed that text prompts effectively guide the network's attention to the correct tumor regions.
• Robustness: The model showed stability against variations in text prompt phrasing (e.g., "irregular shape" vs. "irregularly shaped"), though performance dipped slightly with highly complex, unseen vocabulary.

5. Significance and Future Implications

• Clinical Relevance: By leveraging radiological descriptions (text) alongside images, the model mimics the clinical reasoning process, potentially aiding radiologists in difficult cases where visual contrast is poor.
• Reliability: The integration of evidential learning provides a mechanism for the model to express "doubt" in uncertain regions, which is crucial for clinical safety and decision support.
• Deployment Potential: The authors discuss practical deployment strategies, such as using Large Language Models (LLMs) to automatically generate text prompts from raw scans or extracting prompts from existing radiology reports, reducing the need for manual annotation.
• Limitations: The study notes that ambiguous text prompts (e.g., "large" without a scale reference) can lead to segmentation errors, suggesting future work should focus on fine-grained prompt engineering and multi-modal prompt types (e.g., points/boxes).

In conclusion, TextBCS represents a significant advancement in medical image segmentation by effectively bridging the gap between visual data and semantic knowledge, while simultaneously addressing the critical issue of prediction uncertainty in medical diagnostics.