💻 computer science

Explanation Stability of Test-Time Adaptation in Computational Pathology: A Large-Scale Benchmark

This paper presents a large-scale benchmark demonstrating that test-time adaptation methods in computational pathology exhibit significant variability in explanation stability, revealing that high accuracy does not guarantee reliable model interpretations and highlighting the need for new evaluation metrics to ensure clinical auditability.

Original authors: R. G. Bahumanya, Harshith V. M., Shreyank N. Gowda, Anala M. R

Published 2026-08-10

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: R. G. Bahumanya, Harshith V. M., Shreyank N. Gowda, Anala M. R

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to recognize different types of mushrooms in a forest. You show it thousands of pictures from your local woods, and it gets really good at spotting them. But then, you take the robot to a different forest where the light is dimmer, the leaves are a different shade of green, and the mushrooms look slightly squashed. The robot gets confused. To fix this without showing it new pictures with labels, you let the robot "learn on the fly" while it's walking through the new forest. This is called Test-Time Adaptation (TTA). It's like the robot adjusting its glasses or its focus while it's still moving, trying to make sense of the new world without stopping to ask for help.

However, there's a catch. In the real world, especially in medicine, we don't just want the robot to be right; we want to know why it thinks it's right. If the robot says, "That's a poisonous mushroom," a doctor needs to see where the robot is looking to trust it. Does it see the poisonous gills, or is it just guessing because the mushroom is red? This "looking" is called an explanation. The big question this paper asks is: When we let the robot adapt to the new forest on the fly, does it keep looking at the same parts of the mushroom, or does it suddenly start staring at the wrong things?

This paper is a massive detective story about exactly that. The researchers set up a giant experiment in the world of computational pathology, which is basically using computers to look at microscope slides of tissue to find diseases like cancer. They wanted to see if the "on-the-fly" learning tricks (TTA) that make models smarter also make them more confusing or unreliable in how they explain their decisions.

They didn't just guess; they ran a huge benchmark. They tested 17 different adaptation methods (17 different ways for the robot to learn on the fly) across 5 different types of AI brains (ranging from older, blocky "convolutional" networks to newer, fancy "transformer" ones). They ran these tests on two major datasets of tissue images and performed a staggering 2,958 adaptation runs. That's a lot of running!

Here is what they found, and it's a bit surprising.

First, not all learning methods are created equal. Some methods are like a careful librarian who adjusts the books but never moves the shelves; these methods leave the robot's "gaze" almost exactly where it started. Others are like a chaotic toddler who rearranges the whole room while playing. The researchers found that methods that constantly update the robot's internal "teacher" (called continual methods like CoTTA and RoTTA) caused the biggest mess. They made the robot look at completely different parts of the tissue image after adapting. In contrast, methods that kept the main brain frozen and only tweaked the edges left the explanations almost perfectly stable.

Second, the type of AI brain matters a lot. The older, blocky "convolutional" networks (like ResNet-50 and EfficientNet) were very sensitive. When they tried to adapt, their explanations went haywire. But the newer, more advanced "foundation models" and "transformers" were much more chill. They could adapt to the new data without changing where they were looking. It's like how a seasoned detective might adjust their theory without changing their focus, while a rookie might get distracted by everything.

The most important finding, however, is a warning. The paper shows that being right doesn't mean being stable, and being stable doesn't mean being right.

They discovered a "silent failure" mode. Some methods kept the robot's gaze perfectly steady (high explanation stability), but the robot's confidence became totally wrong, or it started making more mistakes. It was like a student who keeps pointing at the same spot on a map but gets the destination wrong every time. If you only checked if the robot was pointing at the right spot, you'd think everything was fine. But if you checked if the robot was actually finding the right mushroom, you'd see it was failing.

The researchers also found that the "strength" of the adaptation mattered. The more the robot tried to learn in one go, the more its gaze drifted. But the size of the group of images it looked at at once (batch size) didn't really change anything.

So, what's the takeaway? The paper argues that we can't just measure if a medical AI is accurate anymore. We have to measure three things at once:

Accuracy: Is it finding the disease?
Calibration: Is it confident when it should be?
Explanation Stability: Is it still looking at the right tissue features after it adapts?

The authors suggest that for doctors to trust these AI tools, we need a new "scorecard" that includes this explanation stability. If an AI adapts to a new hospital's microscope but starts looking at the wrong parts of the slide, even if it gets the diagnosis right by accident, it's a risk. The paper provides a new tool called the Explanation Stability Index (ESI) to measure this, and they've released all their code and data so others can check it.

In short, this paper tells us that in the race to make AI smarter on the fly, we shouldn't forget to ask: "Are you still looking at the right thing?" Because sometimes, the smartest-looking robot is the one that has lost its way.

Technical Summary: Explanation Stability of Test-Time Adaptation in Computational Pathology

1. Problem Statement

Deep learning models in computational pathology are highly sensitive to domain shifts arising from variations in staining protocols, scanner hardware, and patient cohorts. While Test-Time Adaptation (TTA) offers a practical mechanism to update models on unlabeled target data without retraining, current evaluation paradigms focus almost exclusively on target-domain accuracy.

The paper identifies a critical gap: clinical deployment requires not only accurate predictions but also reliable explanations (e.g., saliency maps indicating where the model is looking). It remains unknown whether TTA methods, which modify model parameters online, inadvertently alter these explanations. A method might maintain high accuracy while shifting attention to non-diagnostic tissue regions, or conversely, preserve explanations while degrading calibration. The authors posit that explanation stability is a distinct reliability axis that is currently unmeasured in TTA benchmarks.

2. Methodology

Experimental Setup

The authors conducted a large-scale benchmark involving 2,958 adaptation runs across:

Datasets: Two histopathology benchmarks with natural domain shifts:
- Camelyon17: Multi-hospital lymph node tumor detection (binary).
- NCT-CRC-HE: Nine-class colorectal tissue classification.
Architectures: Five models spanning convolutional networks, transformers, and foundation models:
- Convolutional: ResNet-50, EfficientNet-B3.
- Transformers: Swin-T, ViT-B/16.
- Foundation Model: Phikon-v2.
TTA Methods: 17 distinct methods grouped by mechanism:
- Normalization: BN-Adapt.
- Entropy: Tent, EATA, DeYO, ROID, COME, REM.
- Sharpness: SAR, SoTTA.
- Continual: CoTTA, RoTTA, RMemSafe.
- Prototype/Gradient-free/Adapter: T3A, LAME, SICL, ViDA.
Attribution Operators: Four families of explainability methods:
- Integrated Gradients (primary, architecture-agnostic).
- Grad-CAM, Grad-CAM++, and Attention Rollout.

The Explanation Stability Index (ESI)

To quantify the movement of model explanations, the authors introduced the Explanation Stability Index (ESI).

Process: Saliency maps ( $S_{pre}$ and $S_{post}$ ) are min-max normalized, resized to a $224 \times 224$ grid, and converted to probability distributions.
Metrics: ESI aggregates six measures:
- Similarity: Structural Similarity (SSIM), Cosine Similarity, Rescaled Spearman Rank Correlation, Top-20% Region-of-Interest IoU.
- Divergence: Symmetric KL Divergence, Wasserstein-1 Distance (EMD).
Calculation: The index is an equal-weighted average of these six terms, bounded between 0 and 1. An ESI of 1.0 indicates an unchanged explanation.
Evaluation: The study measures three coupled effects: change in explanation (ESI), change in discrimination ( $\Delta$ AUC), and change in calibration ( $\Delta$ ECE).

3. Key Results

A. Systematic Hierarchy of Stability

TTA methods exhibit a sharp, systematic difference in how much they perturb explanations:

Frozen-Backbone Methods: Methods that do not update the backbone (e.g., No-TTA, LAME, T3A, SICL) preserve explanations almost perfectly, with ESI $\approx$ 1.000.
Continual Methods: Methods that accumulate updates over time (e.g., CoTTA, RoTTA) cause the largest drift, with the lowest mean ranks in stability.
Normalization/Entropy Methods: BN-Adapt and Tent show low pooled ESI values (0.786 and 0.849), but their disruption is heavily concentrated on convolutional models.

B. Architecture and Shift Dependence

Convolutional Sensitivity: Convolutional networks (ResNet-50, EfficientNet-B3) are substantially more sensitive to adaptation-induced drift than transformer or foundation-model backbones. EfficientNet-B3 showed the lowest mean ESI (0.806).
Transformer Robustness: Swin-T, ViT-B/16, and Phikon-v2 maintained ESI values near 1.00. This is attributed to the adaptation protocol for these models, which freezes the backbone and only updates normalization blocks and the linear head.
Task Difficulty: The harder multi-class NCT-CRC-HE task induced larger drift than the binary Camelyon17 task for continual methods.

C. Decoupling of Stability and Quality

The most significant finding is that explanation stability is largely decoupled from adaptation quality:

Silent Failures: The study identified 54 "silent calibration failures" where accuracy remained stable ( $|\Delta$ AUC| < 0.02) but calibration degraded ( $\Delta$ ECE > 0.05). 94% of these failures occurred in methods (LAME, T3A) that preserved explanations perfectly (ESI = 1.0).
Weak Correlation: Partial Spearman correlation between ESI and $\Delta$ ECE (controlling for $\Delta$ AUC) was weak ( $\rho = +0.089$ ).
Conclusion: A method can preserve explanations while degrading accuracy/calibration, or move explanations while maintaining accuracy. Therefore, ESI cannot be inferred from accuracy or calibration alone.

D. Drivers of Drift

Adaptation Strength: Increasing the number of adaptation steps (e.g., in Tent) monotonically decreased ESI.
Batch Size: Changing batch size had negligible effect on ESI.
Mechanism: Drift is driven by how much of the feature hierarchy is modified. Continual methods and BN-statistic updates in convolutional networks affect features broadly, whereas frozen-backbone approaches do not.

4. Contributions

Formalization of Explanation Stability: The paper defines explanation stability as a distinct reliability axis for TTA and introduces the ESI, a bounded index combining similarity and divergence metrics.
Large-Scale Benchmark: The authors provide a comprehensive evaluation of 17 TTA methods across 5 architectures and 2 datasets, yielding 2,958 paired measurements of accuracy, calibration, and explanation stability.
Empirical Findings: The study establishes that explanation drift is method-dependent, architecture-dependent, and shift-dependent. Crucially, it demonstrates that explanation stability does not substitute for calibration or accuracy monitoring, as "silent failures" can occur where explanations appear stable while model reliability degrades.

5. Significance and Claims

The paper argues that explanation stability is a distinct reliability axis for TTA in computational pathology.

Clinical Relevance: In clinical workflows, pathologists rely on explanations to verify that models attend to plausible tissue regions. If TTA shifts these attributions without changing accuracy, the model may become "clinically auditable" even if numerically robust.
Deployment Implications: The authors recommend that TTA evaluation in medical imaging must report discrimination, calibration, and explanation stability together. Relying on accuracy alone or explanation-only metrics can miss critical failure modes.
Future Direction: The findings suggest that future TTA methods should consider regularizing explanation movement or constraining updates to smaller parameter subsets to prevent unnecessary relocation of attention, particularly in convolutional backbones.

The authors explicitly state that ESI is an integrity measure, not a safety certificate. A high ESI indicates the visual rationale is stable, but does not guarantee the prediction is correct or well-calibrated. Conversely, a low ESI indicates a shift in rationale, which warrants closer inspection but does not necessarily imply a worse prediction.