🤖 machine learning

ADMIL: Attention-Distilled Multiple Instance Learning for Selective Foundation Model Inference in Pathology

The paper introduces ADMIL, a selective-compute framework that distills an attention-based multiple instance learning teacher's attention into a lightweight PriorNet model to select only the most informative tiles for expensive foundation model inference, thereby achieving full-teacher performance while reducing computational costs by over 98% in pathology slide analysis.

Original authors: Duncan Stothers, Ren-Chin Wu, William Lotter

Published 2026-08-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Duncan Stothers, Ren-Chin Wu, William Lotter

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of modern medicine, digital pathology has transformed how doctors examine tissue. Instead of looking through a microscope at a single glass slide, pathologists now work with massive digital images called whole-slide images. These images are so large and detailed that they are broken down into thousands of tiny square pieces, known as tiles, each containing a small view of the tissue. To understand what is happening in a patient's body, computer systems must analyze these tiles to find signs of disease, such as cancer. In recent years, powerful artificial intelligence models, often called foundation models, have been trained to recognize complex patterns in these tiles with incredible accuracy. These models act as expert eyes, capable of spotting subtle differences between healthy and diseased tissue. However, there is a significant problem with using these models in a real hospital: they are extremely slow and expensive to run. Because a single tissue slide can contain tens of thousands of tiles, the computer must process every single one of them to make a diagnosis. This exhaustive approach creates a bottleneck, making it difficult to use these advanced tools quickly enough for everyday patient care.

Researchers at Dana-Farber Cancer Institute and Harvard Medical School have developed a new method to solve this problem without sacrificing accuracy. They created a system called ADMIL, which stands for Attention-Distilled Multiple Instance Learning. The core idea is simple but powerful: not every tile on a slide is equally important. When a trained artificial intelligence model looks at a slide, it naturally focuses its attention on a very small number of regions that hold the most critical information, while largely ignoring the rest. The new system learns to mimic this behavior. It uses a lightweight, fast computer program to scan the entire slide first and predict which few tiles are the most important. Only those selected tiles are then sent to the expensive, slow expert model for detailed analysis. By skipping the vast majority of unimportant tiles, the system dramatically speeds up the process.

The researchers tested this approach on three different medical challenges involving breast cancer, prostate cancer, and lymph node metastasis. In each case, they started with a fully trained expert model that had already learned to diagnose the disease by looking at every single tile. They then trained their new, fast predictor to guess which tiles the expert model would have paid attention to. Once the predictor learned this skill, they built a second version of the diagnosis system that only looked at the top few tiles chosen by the predictor. The results were striking. For breast cancer slides, the system achieved the same high level of accuracy as the full expert model by analyzing just four tiles out of thousands. For prostate cancer, it needed only eight tiles. Even for the more complex lymph node task, it required just 128 tiles to match the performance of the exhaustive system. This means the new method avoids running the expensive computer model on more than 98 percent of the tiles, saving a massive amount of computing power and time.

To ensure that this speed did not come from simply guessing or getting lucky, the researchers compared their method against several alternatives. They tested what would happen if they just picked random tiles to analyze, and found that this approach performed much worse, especially when the number of tiles was small. They also compared their system to a theoretical "perfect" version that knew exactly which tiles the expert model would choose. Their new method came very close to this perfect scenario, proving that it was successfully learning the specific patterns the expert model used to make decisions. The study also showed that the fast predictor was not just picking tiles at random; it was correctly identifying the specific shapes and structures in the tissue that matter for the diagnosis. When the researchers looked at the images, they saw that the tiles selected by the new system contained the same cancerous features that the expert model had highlighted.

The study suggests that the heavy computational cost of current artificial intelligence in pathology is often unnecessary. The expert models are capable of finding the right answer, but they waste energy processing background information that does not change the final result. By teaching a smaller, faster model to act as a gatekeeper, the system can filter out the noise and focus only on the signal. This approach does not replace the powerful expert models; instead, it allows them to be used more efficiently. The researchers found that the method worked consistently across different types of cancer and different sizes of tissue samples. They also tested whether the system was sensitive to how much data it was trained on, finding that it remained robust even when trained with less information. While the system introduces a small amount of extra work during the training phase, the savings during actual use are enormous. This work offers a practical path forward for bringing advanced, high-accuracy artificial intelligence into clinical settings where speed and cost are critical factors, ensuring that patients can receive the benefits of these powerful tools without waiting for slow computer processing.

Technical Summary: ADMIL (Attention-Distilled Multiple Instance Learning)

Problem Statement
Pathology foundation models have become the standard for generating tile-level representations in Whole Slide Image (WSI) analysis. The prevailing pipeline involves partitioning a WSI into thousands of foreground tiles, embedding every tile using a large foundation model encoder, and aggregating these features via Attention-Based Multiple Instance Learning (ABMIL). While effective for tasks like metastasis detection and tumor subtyping, this exhaustive inference strategy is computationally prohibitive for clinical deployment. A single WSI may contain tens of thousands of tiles, making the latency and compute costs of running a foundation model on every tile a significant bottleneck.

This inefficiency arises from a mismatch between the computational cost and the actual information usage: trained ABMIL models rarely utilize all tiles equally. Instead, slide-level predictions are typically supported by a highly concentrated attention distribution over a small subset of morphologically informative regions. Consequently, expensive encoders are applied broadly even though the final classifier relies on a narrow evidence set.

Methodology: ADMIL Framework
The authors propose ADMIL, a selective-compute framework designed to reduce inference costs while preserving slide-level performance. The approach consists of two distinct stages:

Stage 1: Attention Distillation (PriorNet Training)
- A frozen, full-bag ABMIL teacher model (using a foundation model like Virchow2) is first trained on the complete set of foreground tiles.
- The teacher's attention distribution ( $\alpha$ ) serves as the ground truth for a lightweight image encoder called PriorNet.
- PriorNet, implemented as an EfficientNet-B0, is trained to predict the teacher's attention weights for each tile directly from raw pixel data.
- The training objective minimizes the Kullback-Leibler (KL) divergence between the teacher's attention distribution and PriorNet's predicted scores. To manage training costs, the authors employ attention-guided sampling, training on a subset of tiles (e.g., 10%) that includes the top-attended tiles and a random sample of the remainder.
Stage 2: Selective Inference and Student Training
- For a user-defined budget $K$ , PriorNet scores the entire foreground pool of a slide during inference.
- The top- $K$ tiles with the highest scores are selected.
- The expensive foundation model is invoked only on this selected subset to generate embeddings.
- A separate "Top-K ABMIL" student model is trained using these embeddings from the selected tiles to predict the slide label.

Key Contributions

Formulation of Selective Compute: The paper formulates attention distillation as a policy for selective computation in foundation model-based ABMIL. It demonstrates that a lightweight model trained via KL divergence can effectively recover task-relevant evidence from a teacher.
Substantial Efficiency Gains: Across three major benchmarks (BRACS, PANDA, and CAMELYON16), ADMIL achieves performance matching the full-teacher baseline while avoiding 98.5% to 99.8% of foundation model tile embeddings. This translates to a 98.3%–99.6% reduction in model inference FLOPs.
Fidelity Analysis: The authors provide quantitative and qualitative evidence that PriorNet recovers the teacher's tile ordering with high fidelity. The method relies on task-relevant selection rather than simple tile-count reduction, as evidenced by controls showing that random subsampling fails to match performance at low budgets.

Results

Performance vs. Budget: ADMIL matches the full-teacher headline performance at very small tile budgets:
- BRACS: $K=4$ tiles (99.8% embedding reduction).
- PANDA: $K=8$ tiles (98.5% embedding reduction).
- CAMELYON16: $K=128$ tiles (99.0% embedding reduction).
Attention Fidelity: PriorNet exhibits strong positive agreement with the teacher. Spearman correlation (ranking agreement) is consistently higher than Pearson correlation, indicating that PriorNet successfully learns the ordering of tile importance rather than the exact magnitude of attention weights.
Qualitative Validation: Visual inspection of attention maps confirms that the student model, guided by PriorNet, focuses on histologically plausible regions (e.g., tumor cells or high-grade carcinoma morphology) consistent with the teacher and ground truth.
Robustness: Sensitivity analysis shows the method is robust to the fraction of training tiles used for distillation (tested at 5%, 10%, and 20%), suggesting the approach is not narrowly tuned to a specific supervision fraction.

Significance and Claims
The paper claims that ADMIL offers a practical path for deploying pathology foundation models in clinical settings where latency and compute costs are critical. By demonstrating that nearly all expensive tile encodings can be removed without sacrificing slide-level performance, the authors argue that the "mismatch" between broad encoder application and narrow classifier focus can be resolved through distillation.

The authors emphasize that ADMIL does not replace the foundation model or the ABMIL architecture; rather, it acts as a gatekeeper that restricts access to the expensive encoder to only the most likely informative regions. This allows the system to retain the high-capacity representations of the foundation model where they matter most while avoiding the vast majority of evaluations. The work suggests that the teacher's implicit localization behavior can be distilled early enough to enable highly streamlined inference, providing a simple and extensible mechanism for more efficient clinical deployment.

Limitations
The authors note several limitations:

ADMIL depends on the stability and relevance of the teacher's attention; if the teacher attends to confounders or distributes evidence diffusely, the selector's reliability may decrease.
The critical budget $K_{crit}$ is a post-hoc summary; a production system would need to select $K$ based on validation data.
Validation is currently limited to three benchmarks and a specific base foundation model (Virchow2).
While inference is accelerated, the method introduces additional overhead during the training phase (training PriorNet and the student).

Technical Summary: ADMIL (Attention-Distilled Multiple Instance Learning)

More like this