💻 computer science

Foundation Model-Enabled Efficient Data Sampling (FEEDS): A label-efficient training strategy for pan-cancer, multi-tracer PET/CT datasets

The paper introduces FEEDS, a label- and compute-efficient training strategy that leverages foundation model embeddings to select the most informative unlabeled PET/CT cases for annotation, achieving fully-labeled-level performance across diverse cancer types and tracers with 70% less annotation effort.

Original authors: Biratal Raj Wagle, Bashirul Azam Biswas, Grant Chau, Matthew E. Maeder, Muhammad Azeem Arshad, Michael S. Leapman, James B. Yu, Indrani Bhattacharya

Published 2026-08-12

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Biratal Raj Wagle, Bashirul Azam Biswas, Grant Chau, Matthew E. Maeder, Muhammad Azeem Arshad, Michael S. Leapman, James B. Yu, Indrani Bhattacharya

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to spot hidden treasures in a massive, chaotic attic. This isn't just any attic; it's a medical one, filled with thousands of 3D scans of human bodies. The "treasures" are cancer lesions, which can look like tiny specks or huge blobs, hiding in different organs and showing up differently depending on the type of scan used. To teach the robot, you need to show it examples of these treasures. But here's the catch: a human expert has to draw a perfect outline around every single treasure in every single scan. This is like asking a master artist to paint a detailed map of every grain of sand on a beach. It takes forever, costs a fortune, and there just aren't enough expert artists to go around.

For a long time, scientists tried to solve this by either showing the robot every picture (which is impossible because of the time it takes to draw the maps) or by letting the robot guess on the pictures it hasn't seen yet and hoping it learns from its mistakes (which often leads to the robot getting confused and making up fake treasures). The big question in this corner of science is: How do we teach a robot to be a master treasure hunter without making the human experts draw millions of maps? We need a way to pick the best few pictures to show the human, so that the robot learns the most from the least amount of work.

This is where a new strategy called FEEDS (Foundation Model-Enabled Efficient Data Sampling) comes in. Think of FEEDS as a super-smart librarian who has read every book in the world but hasn't seen the specific "treasure maps" yet. Instead of asking the librarian to read every single book to find the best ones to show the robot, FEEDS uses a special "vibe check" tool (called a foundation model) to look at the unmarked scans. This tool can instantly tell how different one scan is from another, like recognizing that one picture is a "sunny beach" and another is a "stormy forest."

The paper explains that FEEDS works in three simple steps. First, it takes all the unmarked scans and gives them a "vibe score" based on their visual features. Second, it looks at the small pile of scans the human experts have already marked (the "fixed labeled data") and asks, "Which unmarked scans are the most different from what we already know?" It specifically hunts for the weird, rare, or unusual cases that the robot hasn't seen before. Third, it asks the human experts to draw the maps only for those specific, unique cases. These new maps are then added to the robot's training pile, and the robot is trained just once.

The researchers tested this idea using a massive collection of cancer scans from different hospitals. They found that by using FEEDS to pick the most diverse and interesting cases, they could train a robot that was almost as good as one trained on every single scan in the database, but they only had to ask the human experts to draw maps for 30% of the data. In fact, the robot trained with FEEDS was better at finding real cancer spots and making fewer mistakes (like thinking healthy tissue was cancer) than robots trained by just picking random pictures. Even when they tested the robot on brand-new data from different hospitals it had never seen before, it still performed incredibly well, proving that it learned the "rules" of the game rather than just memorizing the specific pictures it was shown.

The authors suggest that this method is a game-changer because it saves a huge amount of time and effort. Instead of a slow, expensive process of labeling everything, or a confusing process of letting the robot guess and correct itself over and over, FEEDS offers a one-step, efficient path. It suggests that by being smart about which data we label, we can build powerful medical tools without overworking the doctors. The paper shows that this approach works across different types of cancer and different scanning machines, making it a practical tool for the future of medical imaging.

Technical Summary: FEEDS (Foundation Model-Enabled Efficient Data Sampling)

Problem Statement
Automated lesion segmentation in whole-body Positron Emission Tomography/Computed Tomography (PET/CT) is critical for cancer detection, staging, and treatment planning. However, developing robust, pan-cancer, multi-tracer models requires large, annotated datasets. Creating these datasets is time- and expertise-intensive due to the heterogeneity of lesion size, distribution, and appearance across different cancer types and radiotracers (e.g., FDG vs. PSMA). Consequently, models trained on limited labeled data often lack the accuracy and generalizability required for clinical use. Existing approaches to mitigate label scarcity—such as unsupervised pretraining, semi-supervised learning (SSL), and active learning—typically require iterative training cycles, significant computational resources, or are restricted to single-tracer or single-disease settings. Furthermore, many studies rely solely on voxel-level metrics (e.g., Dice coefficient), failing to evaluate clinical utility at the lesion or anatomical region level, which is essential for treatment planning.

Methodology
The authors propose FEEDS (Foundation Model-Enabled Efficient Data Sampling), a label- and compute-efficient training strategy designed to select the most informative and diverse unlabeled cases for expert annotation prior to model training. The method operates in three distinct steps:

Foundation Model Feature Extraction: For each 3D PET image, a Maximum Intensity Projection (MIP) is generated and normalized. These MIPs are passed through a pre-trained DinoV2 vision foundation model (trained on 142 million natural images) to extract 768-dimensional image-level token representations.
Diversity-Based Selection: The method calculates the cosine distance between the embeddings of a small, fixed labeled pool ( $N_L$ ) and the unlabeled pool ( $N_U$ ) for each specific tracer type (FDG or PSMA). For each unlabeled case, the minimum distance to the labeled set is computed. The cases with the largest distances (i.e., those least represented in the current labeled set) are selected for annotation. This process is performed independently for each tracer to preserve the FDG:PSMA ratio while prioritizing underrepresented scan patterns, such as rare cancer types or unusual uptake distributions.
Segmentation Model Training: The newly annotated diverse cases are combined with the initial fixed labeled pool to train a single nnU-Net model for whole-body lesion segmentation. This constitutes a one-step training paradigm, avoiding the iterative loops required by active learning or SSL.

Key Contributions

One-Step Training Paradigm: Unlike active learning or SSL methods that require iterative retraining or pretraining, FEEDS performs a single upfront case selection followed by a single supervised training phase, significantly reducing computational costs.
Tracer and Disease Agnosticism: The strategy is designed for pan-cancer, multi-tracer datasets without requiring task-specific adaptation of the feature extractor.
Comprehensive Evaluation: The study moves beyond standard voxel-level metrics to include lesion-level (counting and detection) and anatomical region-level (liver, lung, high-risk bones) evaluations, which are directly relevant to clinical treatment planning and eligibility.
Public Availability: The code is made publicly available on GitHub.

Experimental Results
The authors trained and validated FEEDS using the AutoPET-III dataset (1,043 training scans) and evaluated performance on three held-out sets: AutoPET-III, Deep-PSMA, and an internal Dartmouth-Hitchcock Medical Center (DHMC) dataset.

Performance vs. Random Sampling: FEEDS consistently outperformed random sampling strategies across all metrics. At a 30% labeling budget (10% fixed + 20% FEEDS-selected), FEEDS achieved a Dice score comparable to models trained on 100% of the labeled data (e.g., 0.645 vs. 0.655 on AutoPET-III validation) while significantly reducing the False Negative Volume (FNVol) compared to random sampling.
Generalizability: FEEDS generalized effectively across all three test sets and both FDG and PSMA tracers. Notably, on the DHMC dataset (which used a different, newer scanner), the model trained with 100% of the AutoPET-III data performed worse than the 10% baseline, whereas FEEDS (30% budget) achieved a higher Dice score and lower false-positive volume, suggesting better robustness to domain shift.
Clinical Utility:
- Lesion Level: FEEDS achieved higher lesion sensitivity (0.743) and Positive Predictive Value (PPV) (0.821) compared to random sampling and even surpassed the 100% labeled model in PPV.
- Region Level: FEEDS demonstrated superior performance in high-risk anatomical regions (liver, lung, high-risk bones), achieving lower False Negative Volumes, which is critical for avoiding missed high-risk lesions.
- False Positives: FEEDS significantly reduced False Positive Volume (FPVol) compared to random sampling, particularly in disease-negative cases, indicating better specificity.
Comparison with Other Methods: FEEDS outperformed Determinantal Point Process (DPP) sampling and pseudo-label-based semi-supervised learning (which performed worse than the 10% baseline, likely due to noisy pseudo-labels).

Significance and Claims
The paper claims that FEEDS addresses the challenge of label scarcity by providing a practical approach to constructing representative and diverse annotation queues from large, unannotated clinical repositories. By leveraging foundation model embeddings to identify "gaps" in the training distribution, FEEDS enables the construction of high-performance segmentation models with 70% less annotation burden than fully supervised training.

The authors emphasize that FEEDS is designed for practical clinical deployment. It allows radiologists to focus annotation efforts on the most informative cases rather than random samples, thereby optimizing limited expert time. The method is presented as a scalable solution for large, unannotated medical image repositories, requiring no iterative training and offering strong generalizability across different scanners, tracers, and cancer types. The study concludes that FEEDS matches the performance of fully-labeled (100%) training with a fraction of the data, making it a viable strategy for accelerating the development of clinical-grade AI tools in oncology.

Technical Summary: FEEDS (Foundation Model-Enabled Efficient Data Sampling)

More like this