💻 computer science

Cross-Modal MRI Ovary Segmentation in Endometriosis Using Unpaired TVUS Prototype Priors

This paper proposes a dual-branch framework that leverages TVUS-derived prototype priors to align features across modalities, significantly improving the challenging task of cross-modal ovary segmentation in endometriosis MRI compared to state-of-the-art methods.

Original authors: Xingjian Kang, Lina Felsner, Dominik Perrin, Daiqi Liu, Jasmin Arjomandi, Franziska Mathis-Ullrich, Alexandra Stoll, Katharina Breininger

Published 2026-08-20

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Xingjian Kang, Lina Felsner, Dominik Perrin, Daiqi Liu, Jasmin Arjomandi, Franziska Mathis-Ullrich, Alexandra Stoll, Katharina Breininger

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Endometriosis is a painful, chronic condition where tissue similar to the lining of the uterus grows outside of it, often forming cysts on the ovaries. To diagnose and treat this disease, doctors rely heavily on two different types of medical imaging: transvaginal ultrasound, which uses sound waves to create pictures from inside the body, and magnetic resonance imaging, or MRI, which uses powerful magnets to see deep into the pelvis. While ultrasound is excellent at showing the specific texture of ovarian cysts, MRI provides a broader, clearer view of the surrounding anatomy. However, a significant challenge remains: the ovaries are small, and their boundaries can look very similar to the surrounding organs and tissues on an MRI scan. This makes it difficult for computer programs, known as artificial intelligence, to automatically and accurately draw a line around the ovary to help doctors measure it. Without this precise outline, it is harder to track the disease or plan surgery.

Researchers have long hoped that combining the strengths of both ultrasound and MRI could solve this problem, but a major hurdle has stood in the way. In the real world, a patient rarely gets an ultrasound and an MRI of the exact same ovary at the exact same moment, meaning the two images do not line up perfectly. This lack of matched pairs has prevented computers from learning how to translate the clear features seen in ultrasound directly to the MRI images. A new study from a team of scientists in Germany proposes a clever workaround. Instead of trying to match individual patients across both machines, they taught the computer to learn a general "idea" of what an ovary looks like from hundreds of ultrasound images, and then used that general knowledge to guide the computer when it looked at MRI scans.

The team built a system that operates like a two-lane highway. One lane processes ultrasound images, and the other processes MRI images. They started with a powerful, pre-trained computer model designed to understand medical images, which they adapted to focus specifically on the ovaries. First, they fed the system a large collection of ultrasound images where the ovaries were already marked by human experts. The computer analyzed these images to build a mental library, or a "prototype bank," of what a normal ovary and a cyst-filled ovary typically look like in terms of shape and texture. This library acts as a reference guide, a population-level standard of what an ovary should be.

Once this guide was established, the researchers turned their attention to the MRI scans. They did not need to pair these MRI scans with specific ultrasound images. Instead, they instructed the computer to look at an MRI scan and try to find the ovary. As the computer made its guess, it compared its findings against the reference guide built from the ultrasound data. If the computer's understanding of the MRI image drifted away from the established shape and features of an ovary, the system gently pulled it back toward the correct pattern. This process allowed the computer to learn the difficult task of spotting ovaries on MRI by borrowing the clarity it had already learned from ultrasound, all without needing the two types of scans to be taken from the same person at the same time.

The results of this approach were significant. When the team tested their new system on a dataset of MRI scans from eighty-one patients with endometriosis, it outperformed several existing methods. The most common standard for measuring how well a computer outlines an organ is a score called the Dice coefficient, which ranges from zero to one hundred percent, with higher numbers indicating better accuracy. The new method achieved a score of 61.0 percent, which was a substantial improvement over other advanced techniques that scored around 28.9 percent or slightly higher with full training. The researchers noted that this improvement was not just a small statistical gain; the computer's outlines became much more compact and accurate, reducing the number of times it mistakenly identified healthy tissue as part of the ovary.

However, the study also revealed the limits of this progress. While the new method was the best among those tested, the researchers acknowledged that a score of 61 percent means there is still room for improvement, and the system does not yet perfectly outline every single ovary in every image. They found that the way they trained the system mattered greatly. Specifically, they discovered that the computer needed a short period of "warm-up" training on the ultrasound data alone before it could effectively learn from the MRI data. If they tried to train both lanes of the highway at the same time from the very beginning, or if they skipped the warm-up entirely, the system performed much worse. This suggests that establishing a solid foundation of what an ovary looks like from the clearer ultrasound images is a crucial first step before attempting to apply that knowledge to the more ambiguous MRI scans.

The team also explored how different settings affected the system's performance. They found that the balance between the standard image-matching task and the new "reference guide" task was delicate. If the system relied too heavily on the ultrasound guide, it did not learn the MRI features well enough; if it relied too little, it failed to correct its mistakes. Through careful testing, they identified a specific setting that worked best, proving that the method is robust but requires precise tuning. The researchers concluded that while their approach successfully bridges the gap between two different imaging worlds without needing matched patient data, the current limitations of the available data mean that the technology is still evolving. They plan to continue refining the system, particularly by teaching it to better recognize the specific tissues that sit right next to the ovary, which currently confuse the computer. This work represents a meaningful step forward in using artificial intelligence to help doctors see the invisible details of endometriosis, offering a path toward more precise diagnosis and care.

Technical Summary: Cross-Modal MRI Ovary Segmentation in Endometriosis Using Unpaired TVUS Prototype Priors

Problem Statement

Accurate segmentation of ovaries in Magnetic Resonance Imaging (MRI) is critical for localizing lesions in patients with endometriosis, particularly ovarian endometriomas (chocolate cysts). However, this task remains highly challenging due to the small size of ovaries and their ambiguous boundaries with surrounding pelvic structures. While Transvaginal Ultrasound (TVUS) often captures more distinctive ovary-specific features, obtaining paired TVUS–MRI examinations for supervised cross-modal learning is difficult in clinical practice. Existing studies largely focus on single-modality analysis or disease classification, leaving cross-modal ovarian segmentation largely unexplored. There is a scarcity of methods capable of effectively transferring anatomical knowledge from TVUS to MRI without requiring paired data.

Methodology

The authors propose a dual-branch framework designed to leverage unpaired TVUS ovary masks to learn a population-level ovarian prior, which is then transferred to improve MRI ovary segmentation. The core components of the methodology are:

1. Dual-Branch Architecture with MedSAM3 Backbone

The framework utilizes MedSAM3, a promptable concept segmentation foundation model, as the backbone for both the TVUS and MRI branches.

Parameter-Efficient Fine-Tuning (PEFT): Both branches employ Low-rank Adaptation (LoRA) to adapt the pretrained MedSAM3 model.
Branch Configuration:
- TVUS Branch: The backbone and LoRA modules are frozen after a warm-up phase. A projection head is updated via an Exponential Moving Average (EMA) from the MRI branch.
- MRI Branch: The LoRA modules and projection head are trainable.
Feature Extraction: The Vision Transformer (ViT) backbone generates multi-scale feature representations via a Feature Pyramid Network (FPN). These features are mapped to normalized embeddings using a lightweight projection head (1×1 convolution, GroupNorm, ReLU, L2 normalization).

2. Prototype Bank Construction

To inject anatomical knowledge from TVUS into the MRI branch, the authors construct a population-level ovarian foreground prior from unpaired TVUS ovary masks.

Prototype Memory Bank: This bank stores foreground ( $p_{fg}^{TVUS}$ ) and background ( $p_{bg}^{TVUS}$ ) prototypes.
Extraction: Foreground and background features ( $q_{fg}, q_{bg}$ ) are extracted from TVUS inputs via masked average pooling.
Update Mechanism: Prototypes are updated using an EMA strategy:
$p_{fg}^{TVUS}(t) = \alpha p_{fg}^{TVUS}(t-1) + (1-\alpha) q_{fg}^{TVUS}(t)$
where $\alpha$ is the momentum coefficient. This ensures a stable prototype prior that represents population-level anatomy rather than batch-specific noise.

3. Prototype Contrastive Learning

Since the datasets are unpaired, spatial correspondence is not enforced. Instead, the framework enforces cross-modal anatomical consistency in the feature space.

Objective: MRI ovary foreground features ( $q_{fg}^{MRI}$ ) are encouraged to move closer to the TVUS ovary foreground prototypes ( $p_{fg}^{TVUS}$ ) while being pushed away from TVUS background prototypes ( $p_{bg}^{TVUS}$ ) and MRI background features ( $q_{bg}^{MRI}$ ).
Loss Function: An InfoNCE contrastive loss ( $\mathcal{L}_{proto}$ ) is employed:
$\mathcal{L}_{proto} = -\log \frac{\exp(\text{sim}(q_{fg}^{MRI}, p_{fg}^{TVUS})/\tau)}{\exp(\text{sim}(q_{fg}^{MRI}, p_{fg}^{TVUS})/\tau) + \sum_{z \in \{p_{bg}^{TVUS}, q_{bg}^{MRI}\}} \exp(\text{sim}(q_{fg}^{MRI}, z)/\tau)}$
Total Loss: The contrastive loss is combined with the standard segmentation loss ( $\mathcal{L}_{seg}$ ) weighted by $\lambda$ :
$\mathcal{L}_{total} = \mathcal{L}_{seg} + \lambda \mathcal{L}_{proto}$

4. Training Strategy

Warm-up Pretraining: The TVUS branch is first pre-trained independently for three epochs to stabilize the feature space and extract reliable prototypes.
Cross-Modal Training: The TVUS model is then frozen, and the MRI branch is trained for 10 epochs using the TVUS-derived prototypes.

Key Contributions

The paper outlines three primary contributions:

Dual-Branch Framework: A novel architecture for MRI ovary segmentation built upon MedSAM3, capable of learning modality-specific features and cross-modal anatomical representations.
Population-Level Ovarian Prior: The introduction of a prototype bank constructed from unpaired TVUS ovary masks, providing anatomical guidance for MRI representation learning without requiring paired data.
Prototype Contrastive Learning: A strategy to enforce cross-modal anatomical consistency, regularizing MRI feature representations toward anatomically meaningful ovarian features derived from TVUS.

Experimental Results

The framework was evaluated on public endometriosis-related datasets: OTU-2D (TVUS) and UT-EndoMRI (MRI).

Quantitative Performance: The proposed dual-branch approach achieved a mean Dice coefficient (mDICE) of 61.0% and a mean Average Precision (mAP) of 27.3%.
- This represents an improvement of over 5 percentage points in mDICE compared to fully fine-tuned MedSAM3 (54.6%) and significantly outperforms zero-shot MedSAM3 (14.9%) and 2D nnU-Net (28.9%).
Qualitative Observations: The method produced more compact predictions and reduced false-positive responses in surrounding pelvic regions compared to baselines. While zero-shot models struggled with localization and nnU-Net failed to identify the ovarian region, the proposed method refined boundary predictions.
Ablation Studies:
- Prototype Bank: Removing the prototype bank reduced mDICE from 61.02% to 57.74%, confirming its role in improving segmentation quality.
- Warm-up: A 3-epoch TVUS-only warm-up yielded the best results. Joint warm-up or no warm-up led to performance drops.
- Parameters: The framework showed robustness to variations in the contrastive loss weight ( $\lambda$ ) and EMA momentum ( $\alpha$ ), with optimal performance at $\lambda=0.015$ and $\alpha=0.95$ .

Significance and Claims

The authors claim that this work demonstrates the feasibility of using unpaired TVUS data to construct a useful cross-modal prototype prior for text-promptable MRI ovary segmentation.

Modest Performance Claims: While the method outperforms state-of-the-art baselines, the authors acknowledge that the overall performance on MRI ovary segmentation remains "modest." They attribute this to the limited availability of ovary annotations in the original dataset and the challenge of background prototypes being dominated by easy pixels rather than anatomically similar structures.
Methodological Insight: The study highlights that direct transfer from foundation models (zero-shot) is insufficient for this specific task and that target-domain adaptation via unpaired cross-modal priors is a viable strategy.
Future Directions: The authors suggest future work should focus on constructing "near-background" prototypes using dilated masks, exploring self-supervised pretraining for better feature alignment, and extending the framework to 3D volumetric segmentation and other modalities (e.g., laparoscopic video).