⚡ electrical engineering

Useful nonrobust features are ubiquitous in biomedical images

This paper demonstrates that deep learning models in medical imaging utilize nonrobust, non-interpretable features to boost standard accuracy, but relying on these features creates a trade-off where in-distribution performance improves at the expense of robustness to distribution shifts.

Original authors: Coenraad Mouton, Randle Rabe, Niklas C. Koser, Nicolai Krekiehn, Christopher Hansen, Jan-Bernd Hövener, Claus-C. Glüer

Published 2026-04-27

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Coenraad Mouton, Randle Rabe, Niklas C. Koser, Nicolai Krekiehn, Christopher Hansen, Jan-Bernd Hövener, Claus-C. Glüer

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a judge at a high-stakes baking competition. You have two different types of contestants:

Contestant A (The "Robust" Baker): They bake based on the fundamentals—flour, sugar, and heat. You can tell exactly why their cake is good: it’s fluffy, it’s golden-brown, and it smells like vanilla. Even if the kitchen lights flicker or the oven temperature fluctuates slightly, the cake remains a cake.

Contestant B (The "Nonrobust" Baker): They have discovered a "cheat code." They’ve realized that if they sprinkle a specific, microscopic pattern of salt in a very particular way, the judges' taste buds are tricked into thinking the cake is delicious. To a human, the cake looks normal, but that tiny, invisible salt pattern is doing all the heavy lifting.

This paper is about how Artificial Intelligence (AI) in medical imaging acts like Contestant B.

The Core Discovery: The "Invisible Cheat Codes"

When doctors train AI to look at X-rays, CT scans, or pathology slides, the AI doesn't just look at the "fluffy cake" parts (like the shape of an organ or the texture of a tumor). It also finds "nonrobust features"—tiny, invisible patterns in the pixels that are completely meaningless to a human doctor but are incredibly predictive for the computer.

The researchers proved this by doing a clever experiment: they took medical images and "flipped" those tiny invisible patterns to point to the wrong diagnosis. Even though the images looked perfectly normal to a human doctor, the AI was completely fooled because it was relying on those "cheat codes" rather than the actual anatomy.

The "Accuracy vs. Reliability" Tug-of-War

The paper reveals a frustrating dilemma for scientists, which we can call the Medical Trade-off:

The High-Score Trap (In-Distribution Accuracy): If you let the AI use every trick in the book—including those invisible, nonrobust cheat codes—it will get incredibly high scores on standard tests. It looks like a genius!
The Reality Crash (Out-of-Distribution Performance): The moment you take that AI out of the lab and put it in a real hospital—where the lighting is different, the scanner is a different brand, or there is a bit of digital "noise"—the AI fails miserably. Because it was relying on those fragile "cheat codes" instead of the actual anatomy, a tiny change in the image makes the AI lose its way.

The Summary: A Tale of Two Doctors

The researchers found that:

Standard AI is like a student who memorizes the exact patterns of a practice exam. They get an A+ on the practice test, but if the real exam changes even one word, they fail.
Robust AI (AI trained to ignore the "cheat codes") is like a student who actually learns the subject. They might get a slightly lower score on the practice test, but they are much more likely to pass the real exam in the real world.

Why does this matter to you?

If you are ever diagnosed by an AI, you want to know: Is this machine looking at my actual anatomy (the "fluffy cake"), or is it just reacting to digital noise (the "salt pattern")?

The paper concludes that we face a choice: Do we want an AI that is "perfect" in a controlled lab setting, or an AI that is slightly less "accurate" on paper but much more trustworthy and stable when it's actually treating a human being? Currently, the most accurate models are often the most fragile.

Technical Summary: Useful Nonrobust Features are Ubiquitous in Biomedical Images

1. Problem Statement

Deep Neural Networks (DNNs) are increasingly used in high-stakes medical imaging tasks (radiology, pathology, etc.). However, research in natural images has shown that DNNs often rely on nonrobust features: predictive patterns that are highly susceptible to small adversarial perturbations and are completely uninterpretable to humans.

The authors address two critical questions in the biomedical domain:

Do DNNs learn useful, well-generalizing nonrobust features from various medical imaging modalities?
How essential are these features for achieving high performance, and what is the cost of removing them?

2. Methodology

The study utilizes the MedMNIST collection, covering diverse modalities including CT, radiography, ultrasound, and histopathology. The researchers employed a three-pronged experimental approach:

Isolating Nonrobust Features: To prove nonrobust features exist and are predictive, the authors used a "feature-switching" technique. They trained a base model, then constructed a new training set ( $\hat{D}_{train}$ ) where labels were reassigned to adversarial examples that forced the model to predict a target class via nonrobust cues while keeping the robust (anatomical) features intact. If a model trained on this "mislabeled" but anatomically correct dataset performs better than chance, it proves the existence of useful nonrobust features.
Isolating Robust Features: The authors used adversarial training via the TRADES loss function. This method optimizes a trade-off between standard cross-entropy loss (accuracy on clean data) and a Kullback-Leibler (KL) divergence term (consistency between clean and adversarial examples), effectively forcing the model to ignore nonrobust cues.
Evaluation Framework:
- In-distribution (ID): Standard test accuracy on original datasets.
- Out-of-distribution (OOD): Performance under controlled corruptions (contrast, noise, blur, etc.) using MedMNIST-C.
- Adversarial Robustness: Resistance to strong attacks using AutoAttack.

3. Key Contributions

First Systematic Investigation: This is the first study to provide a systematic analysis of the generalization properties of robust vs. nonrobust features specifically within medical imaging.
Quantification of the Trade-off: The paper formally characterizes the robustness-accuracy trade-off in medical AI, demonstrating that features which boost standard accuracy often degrade reliability under distribution shifts.
Saliency Map Analysis: The study provides visual evidence (via saliency maps) showing that robust models focus on stable, anatomically relevant structures, while nonrobust models rely on "noise-like" patterns that shift drastically under perturbation.

4. Results

Nonrobust Features are Predictive: In all five MedMNIST datasets, models trained only on nonrobust features achieved accuracy significantly above chance (e.g., ~81% on PneumoniaMNIST), confirming these features are highly useful for in-distribution classification.
The Accuracy-Robustness Trade-off:
- In-distribution: Models using all features (base models) performed best. Relying only on robust features resulted in a performance drop of 2% to 12%.
- Out-of-distribution: Robust models (adversarially trained) significantly outperformed base models. For example, in PathMNIST, OOD accuracy jumped from 24.89% to 45.78% when using robust features.
Adversarial Vulnerability: Base models and nonrobust-only models were almost entirely defeated by adversarial attacks (accuracy $\leq 0.04\%$ ), whereas robust models maintained significant accuracy (57%–74%).

5. Significance and Implications

The findings present a "practical dilemma" for the deployment of medical AI:

Deployment Strategy: If the goal is maximum accuracy on a specific, controlled dataset, nonrobust features are beneficial. However, if the model will encounter different scanners, noise levels, or potential adversarial threats, prioritizing robust features is essential for safety.
Trustworthiness: Because nonrobust features are uninterpretable and fragile, relying on them undermines the ability of clinicians to trust and justify AI-driven decisions.
Clinical Decision-Making: The paper suggests that future research must determine whether clinicians prefer a "more accurate but fragile" model or a "slightly less accurate but more stable and interpretable" one.