💻 computer science

Large-Scale Multi-Cancer Detection by Learning Segmentation from Reports

The paper introduces R-Super, a novel framework that leverages over 100,000 radiology reports to train tumor segmentation models without requiring scarce manual masks, thereby enabling large-scale multi-cancer detection that significantly outperforms both existing AI models and human radiologists.

Original authors: Pedro Bassi, Xinze Zhou, Wenxuan Li, Szymon Płotka, Jakob Wasserthal, Jieneng Chen, Ibrahim Ethem Hamamci, Sezgin Er, Jakub Prządo, Zheren Zhu, Gorkem Durak, Wiktoria Romańczyk, Yavuz Taktak, Melih Ak

Published 2026-07-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Pedro Bassi, Xinze Zhou, Wenxuan Li, Szymon Płotka, Jakob Wasserthal, Jieneng Chen, Ibrahim Ethem Hamamci, Sezgin Er, Jakub Prządo, Zheren Zhu, Gorkem Durak, Wiktoria Romańczyk, Yavuz Taktak, Melih Akan, Yuhan Wang, Scott Ye, Qi Chen, Ashwin Kumar, Lizhou Wu, Guang Zhang, Bjoern Menze, Jarosław Ćwikła, Yuyin Zhou, Frank Miller, Yunhe Gao, Akshay Chaudhari, Curtis Langlotz, Ulas Bagci, Sergio Decherchi, Andrea Cavalli, Arkadiusz Sitek, Kang Wang, Yang Yang, Alan Yuille, Zongwei Zhou

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a massive library of medical "X-ray movies" (CT scans), but for most of them, the only clue about what's wrong is a written note from a doctor. For decades, teaching computers to find tumors was like trying to learn to draw a map without ever seeing the actual territory. You needed a radiologist to painstakingly trace every single tumor on a 3D scan, pixel by pixel, creating a "mask." This was slow, expensive, and for many types of cancer (like those in the spleen, gallbladder, or adrenal glands), these maps simply didn't exist.

Enter R-Super, a new AI framework that changes the game. Instead of waiting for those rare, hand-drawn maps, R-Super learns to find tumors by reading the doctors' written reports.

The "Ghost Hunter" Analogy
Think of a CT scan as a giant, dark cave. A radiologist's report is like a treasure hunter's journal left behind. The journal doesn't give you a GPS coordinate for the treasure (the tumor); it just says, "There's a 3-centimeter dark rock in the spleen," or "Two small bright spots in the bladder."

Old AI models were like blindfolded searchers who needed someone to point exactly where the rock was before they could learn. R-Super is different. It's like a detective who reads the journal, then scans the cave. It uses four special "rules" (loss functions) to guess where the treasure must be:

The Volume Rule: If the report says the rock is 3 cm wide, the AI knows the total amount of "rock" it finds must add up to that size.
The Ball Rule: If the report says there are two rocks, the AI looks for two distinct "bubbles" of high probability, each fitting the size described, and tries to make the brightest spots inside those bubbles match the description.
The Brightness Rule: If the report says the tumor is "dark" (hypoattenuating), the AI learns to ignore bright spots and focus on the dark ones.
The Danger Rule: If a pathology report says the tumor is "malignant" (bad), the AI learns to flag it as dangerous.

The Big Discovery
The researchers tested this on a massive dataset of 127,496 CT scans paired with reports from the USA, Turkey, and Switzerland. They focused on 7 tumor types that are notoriously hard to spot and for which public "maps" (masks) are scarce or non-existent: spleen, gallbladder, prostate, bladder, uterus, esophagus, and adrenal tumors.

Here is the surprising part: R-Super learned to find these tumors using only the written reports, without ever seeing a single hand-drawn mask during its main training.

When they tested this report-trained AI against 9 leading public AI models (including fancy ones from Google, Microsoft, and Stanford), R-Super won by a landslide. It was 9% to 22% more accurate at finding malignant tumors than the best existing public AI. Even more impressively, it found 56% more malignant tumors than a group of 6 human radiologists in a head-to-head test, while making the same number of false alarms.

What It Is NOT
It's important to know what this paper doesn't say.

It does not say that AI can now replace doctors. The paper explicitly states that R-Super is best used as a "flagging system" to help doctors spot things they might miss, not as an autonomous diagnostician.
It does not claim that reports are perfect. The system uses a large language model (LLM) to read the reports, and while that model is very good (98% accurate), it isn't perfect. However, the paper shows that R-Super is robust enough to handle these occasional reading errors.
It does not claim that this works for every cancer type immediately. The study focused specifically on those 7 difficult tumor types where masks were missing.
It does not say this is a "solved" problem for clinical use. The authors note that the system was tested on past data (retrospective), and future studies are needed to see how it works in real-time hospitals.

The "Double-Boost" Effect
The paper also suggests something even cooler: if you do have a few hand-drawn masks (they used 870 of them), you can mix them with the reports. This "report-plus-mask" training made the AI even better, boosting its ability to find malignant tumors by 11% and its ability to draw the tumor outline (DSC) by 14% compared to using masks alone.

The Bottom Line
The paper suggests that we have been sitting on a goldmine of data. For years, we treated medical reports as just "notes," but R-Super proves they are actually a massive, underused source of training data. By turning words into "ghostly" training signals, this framework allows AI to learn from 127,000+ cases instead of being stuck with a few hundred.

While the authors caution that more testing is needed before this becomes a standard tool in every hospital, the results suggest that AI can now detect these tricky tumors with a level of accuracy that matches or even exceeds human experts, all by learning to read the room (or rather, the report) instead of waiting for a hand-drawn map.

Technical Summary: Large-Scale Multi-Cancer Detection by Learning Segmentation from Reports

Problem Statement
Despite the performance of artificial intelligence (AI) in tumor detection, the field of semantic segmentation for medical imaging faces a critical bottleneck: the scarcity of tumor masks. While over 300 million CT scans are performed annually, generating voxel-wise tumor outlines (masks) is slow, expensive, and labor-intensive, often requiring radiologists to spend over 30 minutes per 3D scan. Consequently, public datasets contain few tumor masks, which are concentrated in a limited number of organs (e.g., liver, lungs, kidneys). For seven specific tumor types—spleen, gallbladder, prostate, bladder, uterus, esophagus, and adrenal tumors—public masks are either non-existent or extremely rare (fewer than 60 in some cases). This lack of supervision prevents the training of accurate segmentation models for these cancers, limiting AI's ability to detect incidental or early-stage tumors that radiologists often miss. Conversely, nearly every CT scan is accompanied by a radiology or pathology report containing detailed, albeit non-localized, descriptions of tumor characteristics (number, size, location, attenuation, and malignancy). Existing AI approaches, such as Vision-Language Models (VLMs) or classifiers, utilize these reports for image-level labels but fail to provide the per-voxel localization required for clinical verification.

Methodology: The R-Super Framework
The authors propose R-Super (Report Supervision), a training framework that converts routine radiology and pathology reports into localized training signals for tumor segmentation, eliminating the need for tumor masks during the initial training phase. The framework operates on the premise that reports can serve as a scalable source of localized supervision.

Data Extraction: A frozen Large Language Model (LLM), specifically Qwen 3 30B, is used to extract structured tumor characteristics from unstructured reports. This includes tumor count, rough anatomical location (organ/slice), diameter, attenuation (hypo-, hyper-, or iso-), and malignancy status (derived from pathology reports or high-confidence radiology descriptions).
Loss Functions: R-Super introduces four novel loss functions that enforce the segmentation model to produce outputs consistent with the extracted report descriptions:
- Volume Loss ( $L_{vol}$ ): Minimizes the dissimilarity between the tumor volume estimated from the segmentation output and the volume estimated from reported diameters (approximated as spheres or ellipsoids). It also penalizes segmentations occurring outside the reported organ.
- Ball Loss ( $L_{ball}$ ): Based on the Expectation-Maximization (EM) framework, this loss locates the most probable tumor positions. It employs a "Ball Convolution" (a non-learnable spherical kernel matching the reported diameter) to slide over the segmentation output within the reported organ. It identifies the sphere with the highest probability sum and maximizes the top- $N$ voxels (where $N$ is the reported volume) within that sphere, while minimizing probabilities outside an uncertainty margin.
- Attenuation Loss ( $L_{att}$ ): Enforces consistency between the segmented tumor's brightness (mean and standard deviation of HU values) and the attenuation description (hypo/hyper/iso) in the report.
- Malignancy Loss ( $L_{mal}$ ): A variation of the Ball Loss that separately optimizes segmentation channels for benign and malignant tumors based on pathology or radiology reports.
Training Strategy: The framework can be trained on CT-Report pairs alone, CT-Mask pairs alone, or a combination of both. When masks are available, standard Dice and Cross-Entropy losses are applied to those pairs, while the R-Super losses are applied to report pairs. The framework is architecture-agnostic; the authors utilized MedFormer (a U-Net-based CNN-transformer hybrid) for their experiments.
Active Learning Loop: The trained model can assist radiologists in creating new masks faster, which are then used to fine-tune the model alongside the vast report dataset, creating a scalable active learning cycle.

Key Contributions

R-Super Framework: The first method to directly convert unstructured medical reports into voxel-wise supervision for tumor segmentation, addressing the mask scarcity bottleneck.
Large-Scale Dataset: Creation of a new dataset comprising 127,496 CT-Report pairs (42 million 2D slices) from UCSF, Stanford, and affiliated institutions, covering seven tumor types with scarce or no public masks.
Public Release: The authors are releasing code, over 22,000 CT scans and reports, and the first public tumor segmentation masks for bladder, gallbladder, spleen, and prostate tumors, as well as expanded masks for uterus, esophagus, and adrenal tumors.
Novel Loss Functions: The development of Volume, Ball, Attenuation, and Malignancy losses that effectively translate textual descriptions into spatial constraints.

Results
The study evaluated R-Super across four hospital cohorts in three countries (USA, Turkey, Switzerland) and compared it against 9 leading public AI models (including VLMs like MedGemma, Merlin, and segmentation models like ULS and FLARE) and 6 alternative training strategies.

Performance vs. Public Models: R-Super, trained on reports alone (no masks), significantly outperformed all 9 public AI models in detecting the 7 target tumor types. In malignant tumor detection, it surpassed the best public model by +9% to +22% in average balanced accuracy across external test sets.
Report vs. Mask Training: Training R-Super on 116,514 reports (without masks) outperformed a standard segmentation model trained on 870 masks for tumor detection. When combining reports and masks, the model (R-Super†) achieved a +11% increase in sensitivity and +14% increase in DSC compared to mask-only training.
Comparison to Radiologists: In a reader study involving 6 radiologists on a dataset focused on small tumors (50% external data), R-Super† detected 56% more malignant tumors than the average radiologist at a matched false-positive rate (96% specificity). It surpassed radiologists in sensitivity for 6 of the 7 tumor types and matched them for uterus tumors.
Scaling: Performance scaled positively with the number of training reports, showing no saturation even at 100,000+ reports.

Significance and Claims
The paper claims that R-Super demonstrates a paradigm shift in medical AI: moving from reliance on scarce, manually annotated tumor masks to leveraging abundant, routinely generated medical reports as a source of localized supervision. The authors assert that R-Super is the first public AI model shown to match or surpass radiologist performance in detecting these seven specific tumor types on CT scans. By effectively learning from reports, the framework enables tumor segmentation models to scale beyond the limitations of mask scarcity, advancing the feasibility of automated, incidental cancer detection. The authors position R-Super not as an autonomous diagnostic tool, but as an assistive system capable of flagging probable cancer cases that would otherwise remain undetected on routine CT scans, thereby facilitating timely follow-up examinations. The study emphasizes that while the approach shows technical feasibility and strong external generalization, prospective multi-center studies are required before clinical implementation.

Technical Summary: Large-Scale Multi-Cancer Detection by Learning Segmentation from Reports

More like this