Probabilistic Robustness in Medical Image Classification

Technical Summary: Probabilistic Robustness in Medical Image Classification

Problem Statement
While Deep Learning (DL) has demonstrated strong performance in medical image classification tasks—ranging from MRI and CT analysis to skin cancer and lung disease diagnosis—its deployment in safety-critical clinical settings remains hindered by concerns regarding trustworthiness. Existing robustness research predominantly focuses on Adversarial Robustness (AR), which adopts a worst-case perspective by seeking the single deterministic perturbation within a norm ball that maximizes prediction loss. The authors argue that this worst-case formulation is less representative of real-world medical applications, where perturbations are often stochastic (e.g., equipment degradation, camera defocus) rather than deterministic attacks. Consequently, there is a need for a more practical measure of model trustworthiness that accounts for the likelihood of errors under realistic, random perturbations.

Methodology
To address this gap, the study investigates Probabilistic Robustness (PR), a statistical approach that evaluates the overall likelihood of a model maintaining correct predictions under random perturbations. The methodology involves three key components:

1. Definition of Probabilistic Robustness: Unlike AR, which solves for $\delta^* = \arg \max_{\|\delta\| \leq \gamma} L(x+\delta, y; \theta)$, PR defines robustness as the expected probability that the model's prediction remains unchanged under a random perturbation $\delta$ drawn from a local input distribution $Pr(\cdot|x)$. Formally, $R(x, \gamma) = \mathbb{E}_{\delta \sim Pr(\cdot|x), \|\delta\| \leq \gamma} [I\{f_\theta(x+\delta) = y\}]$.
2. Natural Corruption Settings: The authors construct a set of standard natural corruption settings for medical image classification by adapting perturbation types widely studied in general computer vision to the medical imaging domain.
3. Experimental Setup: The study utilizes the MedMNIST v2 dataset, specifically the PathMNIST subset (9-class colorectal pathology images with ~107k samples). Two commonly used CNN backbones, ResNet-18 and ResNet-50, are evaluated. Performance is measured using standard metrics (Accuracy and AUC) on clean images, alongside PR assessment under six specific natural corruptions: Defocus, Motion blur, Stain, Saturation, Pixelation, and Brightness variations. The PR evaluation follows the benchmarking protocol defined in PRBench.

Key Contributions
The paper outlines three primary contributions:

1. Investigation of PR in Medical Context: The work highlights the practical relevance of probabilistic robustness for safety-critical medical diagnosis systems, arguing it is more suitable than worst-case AR for realistic perturbation scenarios.
2. Standardized Corruption Framework: The authors construct and standardize a set of natural corruption settings specifically for medical image tasks by adapting general computer vision perturbations, enabling a more realistic robustness evaluation protocol.
3. Systematic Evaluation: A comprehensive evaluation of ResNet-18 and ResNet-50 on MedMNIST v2 is conducted, analyzing PR under medical-image corruption settings in addition to conventional predictive accuracy.

Results
The experimental results on PathMNIST reveal several critical insights:

• Substantial Performance Drop: While the models achieve high clean-image accuracy (e.g., ResNet-18: 90.68% ACC, 98.46% AUC), their probabilistic robustness drops significantly under natural corruptions. For instance, under brightness variations, PR falls to 62.84% for ResNet-18 and 67.91% for ResNet-50.
• Corruption Dependency: Robustness is highly dependent on the type of corruption. Both models remain relatively robust against Pixelation, Stain, and Saturation but are significantly more vulnerable to Motion blur and Defocus. Brightness variations also cause substantial degradation.
• Clean Performance vs. Robustness: Higher clean-image performance does not guarantee better probabilistic robustness. Although ResNet-50 achieves higher ACC and AUC than ResNet-18, it does not consistently demonstrate superior PR across all corruption types.

Significance and Claims
The paper claims that Probabilistic Robustness offers a more practical perspective than worst-case robustness for assessing the trustworthiness of DL models under realistic perturbations. By providing a statistically grounded reference, this work aims to support the safer deployment of deep learning systems in medical imaging. The study emphasizes that image quality issues common in practice (e.g., equipment degradation) can lead to less trustworthy predictions, even for models with high standard accuracy, necessitating evaluation frameworks that go beyond deterministic adversarial examples.