💻 computer science

Detecting ovarian endometriomas from ultrasound using vision transformers and cross-modality transfer learning

This paper presents a pioneering machine learning model that detects ovarian endometriomas from ultrasound images using vision transformers and cross-modality transfer learning, achieving high performance despite data scarcity and imbalance while demonstrating the transferability of low-level features across medical imaging modalities.

Original authors: Matthew Watson, Miliani Fraser-Fletcher, Tom Willshare, Molly Jowsey, Noura Al Moubayed

Published 2026-07-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Matthew Watson, Miliani Fraser-Fletcher, Tom Willshare, Molly Jowsey, Noura Al Moubayed

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine your body is a bustling city, and sometimes, a few stubborn neighborhoods (the ovaries) get covered in a sticky, dark goo called an endometrioma. This isn't just a messy room; it's a chronic condition affecting about 1 in 10 women of reproductive age, causing pain, fatigue, and trouble having babies. For years, the only way to confirm this "sticky goo" was to send in a construction crew with drills and cameras (laparoscopic surgery) to look inside. It's invasive, expensive, and often delayed because the symptoms look like other city problems.

Recently, doctors started using ultrasound machines—basically, high-tech flashlights that can see inside the body without cutting—to spot these cysts. But here's the catch: reading these flashlights requires a super-specialized detective. There aren't enough of them, and they are busy. So, the city gets stuck in traffic, and patients suffer.

Enter the computer scientists. They asked a bold question: Can we teach a robot to spot this sticky goo in the ultrasound flashlights?

The Big Experiment: Teaching a Robot with the Wrong Textbook
The team tried to train a super-smart AI brain (called a Vision Transformer) to find these cysts. But they hit a wall: there are very few public photos of these cysts available to teach the robot. It's like trying to teach a student to identify rare tropical birds when you only have a handful of blurry photos.

To fix this, they tried a clever trick called "transfer learning." Think of it like this: instead of teaching the robot to recognize birds from scratch, they first taught it to recognize everything in the medical world—X-rays of broken bones, CT scans of lungs, and MRI pictures of brains. They used a model pre-trained on 1.2 million of these different medical images (called OmniRad).

The researchers found that this "medical school" background helped the robot learn much faster than if they had just taught it using a general textbook of everyday photos (like cats and cars). Even though the robot was trained mostly on X-rays and MRIs, it learned to recognize the basic "shapes and patterns" of medical tissue. When they then showed it the ultrasound flashlights, it was surprisingly good at spotting the sticky goo, even though ultrasound looks very different from an X-ray.

The Results: A New High Score
The team tested their best robot on a set of 469 ultrasound images it had never seen before.

The robot trained on general photos (cats/cars) got a score of 0.677 (on a scale where higher is better).
The robot trained on other medical images (OmniRad) scored 0.808 to 0.813 (depending on how they tweaked the math).

That's a big jump! The authors suggest that this proves the robot learned to recognize the "low-level" building blocks of medical images, which work across different types of scans, even when the body parts are totally different.

What the Robot Actually "Sees"
To make sure the robot wasn't just cheating (like memorizing the red ink a doctor wrote on the photo), the team used a special tool called SHAP to see what the robot was looking at.

The Good News: When the robot found a cyst, it focused right on the cyst. It ignored the green boxes and red text that doctors sometimes draw on the images. This means it's looking at the actual medical features, not the "clues" the humans left behind.
The Bad News: The team couldn't prove the robot works on everyone. The data they used came from just one place, and they didn't have information about the patients' ages or backgrounds. So, while the robot is great at this specific test, we don't know yet if it would work just as well on patients from a different hospital or with different skin tones. The authors are careful to say this is a "preliminary analysis," not a final proof for the whole world.

Why This Matters (But Isn't a Magic Wand Yet)
The authors ran a simulation to see if this robot could help doctors decide who needs surgery. They found that using the robot's advice would be better than the current "treat everyone" or "treat no one" approaches. It suggests that AI could help triage patients, sending the ones who really need surgery to the top of the list and saving time for everyone else.

However, the paper is very clear: this is not a finished product ready for your local clinic. It's the first time anyone has used publicly available data to build a model like this. The authors hope this opens the door for more collaboration. They suggest that future work needs to test these models on data from many different hospitals to make sure they aren't biased and that they actually work in the real world.

In short, the team built a robot that learned to spot a specific type of ovarian cyst by first studying other medical scans. It performed better than robots trained on general photos, and it seems to be looking at the right things. But before it can replace a doctor's flashlight, it needs to pass more tests to prove it works for everyone, everywhere.

Technical Summary: Detecting Ovarian Endometriomas from Ultrasound Using Vision Transformers and Cross-Modality Transfer Learning

Problem Statement
Endometriosis is a chronic inflammatory condition affecting approximately 10% of reproductive-age women globally, often leading to severe delays in diagnosis due to symptom overlap with other gynecological conditions and the historical reliance on invasive laparoscopic surgery for definitive diagnosis. While standardized ultrasound approaches (e.g., the IDEA framework) offer non-invasive diagnostic potential, they currently depend on highly trained specialists, creating clinical bottlenecks. Although machine learning (ML) has shown promise in other diagnostic imaging tasks, its application to endometriosis detection via ultrasound is hindered by a lack of large-scale, public datasets. Existing studies often rely on small, private datasets with limited external validation, bias analysis, or generalization evaluation. Furthermore, the scarcity of public ultrasound data specific to gynecological regions necessitates investigating whether models can leverage transfer learning from other, non-ultrasound medical imaging modalities to overcome data limitations.

Methodology
The study utilizes the MMOTU-2D dataset, a publicly available collection of 1,469 ultrasound scans from 294 patients. The authors reformulate the dataset's original tumor segmentation task into a binary classification problem (endometriosis vs. non-endometriosis). Images containing "chocolate cysts" (endometriomas) are labeled as positive, while all other tumor types and normal ovaries are labeled as negative. The dataset is split into 1,000 training images and 469 test images, ensuring patient-level separation.

Model Architecture: The base model is a Vision Transformer (ViT) with input dimensions of 224x224.
Pre-training Strategies: Two pre-training regimes are compared:
1. General Corpus: ViT pre-trained on ImageNet.
2. Medical Corpus: OmniRad-Small, a ViT pre-trained on 1.2 million radiology images (primarily X-ray, CT, and MRI), explicitly confirmed to exclude the MMOTU-2D dataset to prevent data leakage.
Training Configuration: To address the heavily imbalanced class distribution (22.9% positive), the authors compare standard cross-entropy loss against weighted cross-entropy loss. Training involves a learning rate of $5 \times 10^{-5}$ for 10 epochs with a batch size of 32 (effective batch size of 128 via gradient accumulation).
Evaluation Metrics: Primary evaluation focuses on Average Precision (AP) due to class imbalance, supplemented by AUROC, F1 score, precision, recall, and accuracy. Decision Curve Analysis (DCA) is employed to assess clinical net benefit against "treat-all" and "treat-none" strategies.
Explainability: GradientSHAP is used to visualize feature importance and verify that models are not relying on spurious correlations (e.g., hard-coded annotations in the images).

Key Results

Performance: The OmniRad-Small model with weighted cross-entropy loss achieved the highest performance, with an AUROC of 0.923 and an Average Precision (AP) of 0.808. This outperformed the ImageNet-pre-trained ViT (AP: 0.677) and the OmniRad model without weighted loss (AP: 0.807).
Impact of Weighted Loss: Applying weighted loss significantly improved recall and F1 scores across both pre-training regimes, demonstrating the necessity of handling class imbalance in this domain.
Cross-Modality Transfer: Despite the significant domain shift between the pre-training modalities (X-ray, CT, MRI) and the target modality (Transvaginal Ultrasound), the OmniRad model consistently outperformed the general ImageNet baseline. This suggests that medical pre-training facilitates the learning of low-level feature/pattern recognition generalizable across different imaging modalities and anatomical regions.
Clinical Utility: Decision curve analysis showed that all models provided a greater net benefit than "treat-all" or "treat-none" strategies across clinically relevant thresholds (0.05 to 0.83).
Explainability: SHAP analysis indicated that the best-performing model focused on the endometrioma cysts themselves rather than hard-coded annotations (e.g., bounding boxes or text overlays) present in the ultrasound images, suggesting the model learned clinically meaningful features rather than shortcut learning.

Significance and Claims
The authors claim this is the first study to train machine learning models for endometriosis detection using publicly available ultrasound data. The study demonstrates that:

High-performance models (AP: 0.81) can be trained on small, imbalanced public datasets for endometrioma detection, matching or exceeding results from previous studies that used private datasets.
Cross-modality transfer learning is highly effective in medical imaging; pre-training on diverse radiological modalities (non-ultrasound) substantially improves performance on ultrasound tasks compared to general image pre-training. This challenges the prevailing preference for intra-modality transfer learning and suggests a path forward for low-resource domains where modality-specific data is scarce.
The proposed approach has the potential to democratize endometriosis diagnosis, reducing reliance on specialist acquisition and analysis, thereby shortening diagnostic delays and mitigating socioeconomic and geographic biases in access to care.

The paper concludes by emphasizing the need for future work to validate these techniques on external datasets to assess generalizability and bias, followed by prospective studies to evaluate cost-effectiveness and clinical impact.

Technical Summary: Detecting Ovarian Endometriomas from Ultrasound Using Vision Transformers and Cross-Modality Transfer Learning

More like this