Unsupervised Causal Prototypical Networks for De-biased Interpretable Dermoscopy Diagnosis

This paper proposes CausalProto, an unsupervised causal prototypical network that leverages structural causal modeling and information bottleneck constraints to disentangle pathological features from environmental confounders, thereby achieving de-biased, interpretable, and high-accuracy dermoscopy diagnosis without compromising performance.

The Gist

Imagine you are training a new medical student to diagnose skin conditions using photos of moles (dermoscopy images).

The Problem: The "Shortcut" Student

In the past, we used powerful AI models (like deep learning) to do this. But these models are like black boxes: they give an answer, but you have no idea why. To fix this, researchers created "Prototypical Networks." Think of these as students who learn by looking at a "cheat sheet" of perfect examples (prototypes). When they see a new mole, they say, "This looks 90% like the 'Melanoma' example on my cheat sheet, so I'll diagnose it as Melanoma."

Here's the catch: Real-world photos are messy. They have shadows, different lighting, or even the patient's hair or a ruler next to the mole.

• The Shortcut: A standard AI student gets lazy. Instead of learning what a mole looks like, it learns that "if there's a ruler in the photo, it's probably cancer." Or, "if the photo is taken with a specific phone camera, it's likely a specific disease."
• The Result: The AI becomes a "shortcut learner." It gets high scores on tests but fails in real life because it's guessing based on the background (the ruler, the lighting) rather than the actual disease. This is dangerous for doctors.

The Solution: CausalProto (The "Detective" Student)

The authors of this paper created a new system called CausalProto. Think of this as a super-smart detective student who refuses to take shortcuts.

Here is how it works, using a simple analogy:

1. The Two-Brain System (Disentanglement)

Imagine the AI has two separate brains working at the same time:

• Brain A (The Pathologist): This brain is strictly forbidden from looking at the background. It only looks at the mole itself. Its job is to find the real disease features.
• Brain B (The Photographer): This brain looks only at the background, the lighting, the ruler, and the camera type. It ignores the mole entirely.

The system forces these two brains to be completely independent. If Brain A starts thinking about the ruler, Brain B gets a penalty. This ensures the "Pathologist" brain only learns about the disease, not the environment.

2. The "Cheat Sheet" Cleanup (Prototypes)

Instead of one messy cheat sheet, the system creates two:

• The Pure Evidence Book: This contains only pictures of the actual disease patterns (like a perfect map of a melanoma).
• The Noise Dictionary: This contains pictures of all the "bad stuff" (shadows, rulers, hair).

3. The "What If?" Test (Causal Intervention)

When the detective student looks at a new patient photo, it doesn't just say, "It looks like the cheat sheet." It performs a mental experiment called "Do-Calculus."

• The Question: "If I took this photo and removed all the background noise (the ruler, the bad lighting) using my Noise Dictionary, would it still look like cancer?"
• The Action: It mathematically "averages out" all the possible backgrounds. It asks, "Does this mole look like cancer in a bright room? In a dark room? With a ruler? Without a ruler?"
• The Result: If the answer is "Yes, it looks like cancer in every scenario," then the diagnosis is solid. If it only looks like cancer when there's a ruler, the system says, "No, that's a trick."

Why This Matters

• No More Guessing: The AI stops guessing based on the camera or the ruler. It focuses 100% on the skin.
• Transparency: When the AI makes a diagnosis, it can show you the "Pure Evidence Book" page it matched. It says, "I think this is Melanoma because it looks exactly like this specific patch of skin, and I ignored the ruler in the corner."
• Better Accuracy: Surprisingly, by ignoring the "easy shortcuts," the AI actually gets better at diagnosing than the old "black box" models. It proves you don't have to choose between a smart AI and a transparent AI.

The Bottom Line

CausalProto is like teaching an AI to be a true doctor rather than a trickster. It learns to separate the disease from the distractions, ensuring that when it gives a diagnosis, it's based on real medical evidence, not on the type of camera used to take the picture. This makes AI safe and trustworthy enough to be used in real hospitals.

Technical Summary

1. Problem Statement

Deep learning models for dermoscopy image analysis face two critical challenges:

• The Black-Box Nature: Standard models lack transparency, hindering clinical trust and deployment in safety-critical environments.
• Shortcut Learning & Bias: Even interpretable models, such as Prototypical Networks, suffer from selection bias in real-world clinical data. These models often learn "shortcuts" by encoding environmental confounders (e.g., image artifacts, lighting, specific camera types) as predictive features rather than genuine pathological signs.
  • In a Structural Causal Model (SCM) framework, this creates an active backdoor path ($Y \leftarrow S \rightarrow X$), where spurious correlations ($S$) dictate the diagnosis ($Y$) instead of the true causal factors ($C$).
  • This leads to spurious visual evidence (e.g., heatmaps highlighting artifacts instead of lesions), rendering the diagnosis unreliable despite high apparent accuracy.

2. Methodology: CausalProto

The authors propose CausalProto, an unsupervised causal prototypical network designed to purify the visual evidence chain by decoupling pathological features from environmental confounders. The framework is built upon a Structural Causal Model (SCM) and consists of three core components:

A. Unsupervised Disentanglement via Information Bottleneck

To separate causal features ($Z_C$) from spurious features ($Z_S$) without requiring environmental annotations:

• Dual-Branch Encoders: The network uses two parallel encoders, $f_c(\cdot)$ and $f_s(\cdot)$, to map input images $X$ into causal latent variables $Z_C$ and spurious latent variables $Z_S$.
• Mutual Information (MI) Minimization: An Information Bottleneck constraint is enforced to strictly minimize the mutual information between $Z_C$ and $Z_S$, ensuring orthogonal disentanglement.
• vCLUB Approximation: Since exact MI calculation is intractable, the authors use the Variational Contrastive Log-Ratio Upper Bound (vCLUB) to approximate and minimize the dependency between the two latent spaces during training.

B. Causal Prototypical Metric for Reasoning

To ensure interpretability:

• Dual Prototype Spaces: The model learns two distinct prototype libraries:
  1. Causal Prototype Library ($P_C$): Captures genuine pathological patterns.
  2. Spurious Prototype Library ($P_S$): Models environmental artifacts.
• Prototype Projection: Causal prototypes are constrained to be projections of real training image instances, ensuring they represent verifiable clinical examples rather than abstract vectors.
• Interpretability: Diagnosis is based on the similarity between the input's causal features and the causal prototypes, mimicking human case-based reasoning.

C. Backdoor Adjustment via do-Calculus

To eliminate the influence of confounders during inference:

• Interventional Prediction: Instead of predicting $P(Y|X)$, the model estimates the interventional probability $P(Y|do(X))$.
• Expectation Pooling (NWGM): The model performs a backdoor adjustment by marginalizing over the learned spurious dictionary $P_S$. Using the Normalized Weighted Geometric Mean (NWGM), it averages the predictions across the diverse contexts captured in $P_S$.
• Result: This mathematically blocks the backdoor path, ensuring the final diagnosis is driven solely by the purified causal evidence in $P_C$.

D. Overall Objective Function

The model is trained end-to-end using a joint loss function:
$$L = L_{CE} + \lambda_1 L_{cluster} + \lambda_2 L_{proto} + \beta L_{MI}$$
Where $L_{CE}$ is cross-entropy for causal prediction, $L_{cluster}$ ensures diversity in the spurious dictionary, $L_{proto}$ aligns features with prototypes, and $L_{MI}$ enforces disentanglement.

3. Key Contributions

1. Mechanism Definition: Rigorously defines the spurious evidence generation mechanism in medical vision, identifying the vulnerability of prototypical networks to confounding factors.
2. Unsupervised Disentanglement: Achieves strict orthogonal separation of pathological and environmental features using a variational mutual information upper bound, eliminating the need for expensive environmental annotations.
3. Causal Intervention Dictionary: Introduces an unsupervised confounding prototype library and utilizes do-calculus to perform efficient expectation pooling, effectively marginalizing spurious noise.
4. Breaking the Trade-off: Demonstrates that it is possible to achieve both superior diagnostic accuracy and high-purity visual interpretability, overcoming the traditional accuracy-interpretability trade-off.

4. Experimental Results

The model was evaluated on three public dermoscopy datasets: HAM10000, ISIC 2019, and PAD-UFES-20.

• Quantitative Performance:
  • CausalProto achieved State-of-the-Art (SOTA) performance across all datasets.
  • On HAM10000, it improved Balanced Accuracy (BAcc) by 4.1% over the strongest baseline (CausalVAE) and 9.5% over standard black-box ResNet-50.
  • It consistently outperformed other prototype-based models (ProtoPNet, PIP-Net) and robust causal models (Group DRO, FactorVAE).
• Ablation Studies:
  • Removing the MI penalty ($L_{MI}$) caused a sharp drop in prototype purity and accuracy, confirming that feature orthogonality is critical for de-biasing.
  • Removing the causal intervention ($do$-calc) maintained disentanglement but failed to block residual shortcuts, proving that marginalization is essential for final prediction.
• Qualitative Analysis:
  • Prototype Purity: CausalProto achieved significantly higher prototype purity (0.82 vs. ~0.58 for baselines), meaning the visual evidence matched the diagnosis class.
  • Visualizations: Heatmaps generated by CausalProto focused strictly on intrinsic pathological regions (e.g., lesion borders, pigment networks) and successfully ignored ubiquitous artifacts (e.g., hair, rulers, ink marks) that misled baseline models.

5. Significance

• Clinical Trust: By providing transparent, high-purity visual evidence that aligns with expert logic, CausalProto addresses the "black-box" barrier in medical AI.
• Robustness: The framework fundamentally shifts diagnosis from fitting observational biases to interventional reasoning, making models robust against dataset shifts caused by environmental confounders.
• Future Direction: While currently limited to image-level features, the authors suggest that future iterations could incorporate multi-modal clinical priors to capture complex non-visual confounders, further enhancing the reliability of AI in high-stakes clinical environments.