📊 statistics

Certified Causal Defense with Generalizable Robustness

This paper proposes GLEAN, a novel certified defense framework that leverages causal factor learning to disentangle causal relations from spurious correlations, thereby enabling models to maintain theoretical robustness guarantees against adversarial attacks even when facing distribution shifts across different data domains.

Original authors: Yiran Qiao, Yu Yin, Chen Chen, Jing Ma

Published 2026-05-29

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yiran Qiao, Yu Yin, Chen Chen, Jing Ma

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Smart" Student Who Cheats

Imagine you are training a student (an AI model) to identify animals in photos.

The Goal: The student should learn that a lion has a mane and a tiger has stripes.
The Cheat: In your training class, you accidentally only show lions in the savanna and tigers in the jungle. The student gets a perfect score, but they aren't actually learning about the animals. They are "cheating" by memorizing the background (the spurious correlation). They think, "If I see grass, it's a lion. If I see trees, it's a tiger."

The Disaster: When you take this student to a new classroom (a different data domain) where lions are in the jungle and tigers are in the savanna, the student fails miserably. They get confused because their "cheat code" no longer works.

In the world of AI security, this is a huge problem. Even if the student is "robust" (hard to trick) in the first classroom, they fall apart in the second one. Furthermore, standard security checks (Certified Defense) can't guarantee the student will be safe in that new classroom because the rules of the game have changed.

The Solution: GLEAN (The "Truth-Seeking" Detective)

The authors propose a new framework called GLEAN. Think of GLEAN as a detective who refuses to look at the background scenery. Instead, the detective focuses entirely on the essential, unchangeable features of the object (the causal factors).

Here is how GLEAN works, step-by-step:

1. Separating the "Real" from the "Fake"

GLEAN uses a special lens to split every image into two parts:

The Causal Factors (The Real Deal): These are the things that actually cause the label. For a lion, it's the mane and the face shape. These stay the same whether the lion is in a zoo or the wild.
The Spurious Factors (The Distractions): These are the accidental clues, like the background color or the lighting. These change from domain to domain.

GLEAN teaches the AI to ignore the distractions and focus only on the "Real Deal."

2. The "Unbreakable Shield" (Lipschitz Constraint)

Usually, when you try to prove an AI is safe, you have to check every possible way an attacker could tweak the image. This is like trying to count every grain of sand on a beach.

GLEAN uses a mathematical trick called Lipschitz continuity. Imagine the AI's brain is a rubber sheet. If you poke the sheet (add noise to the image), the rubber sheet stretches, but it can't stretch too far.

By forcing the AI to be a "tight" rubber sheet (1-Lipschitz), the authors can mathematically guarantee that if you poke the image slightly, the answer won't change.
Because the AI is only looking at the "Real Deal" (causal factors) and the rubber sheet is tight, this guarantee holds true even when the student moves to a new classroom.

3. The "Blindfolded" Test (Randomized Smoothing)

To prove the AI is robust, GLEAN uses a technique called Randomized Smoothing.

Imagine you are trying to identify a friend in a foggy room. You can't see them clearly.
GLEAN adds a little bit of "fog" (random noise) to the image many, many times.
If the AI says "It's a lion" 99% of the time despite the fog, we can mathematically prove that a small amount of fog (an attack) won't trick it.
Because GLEAN focuses on the unchangeable "Real Deal," this foggy test works even in the new classroom where the backgrounds are different.

Why This Matters (The Results)

The paper tested this on three different scenarios:

CMNIST: A fake dataset where numbers are colored red or green to trick the AI.
CelebA: Real photos of celebrities where the AI might get confused by hair color vs. smiling.
DomainNet: A massive dataset with photos from different real-world sources.

The Outcome:
In every test, GLEAN was significantly better than previous methods.

Old Methods: When the background changed, their safety guarantee (Certified Radius) dropped to almost zero. They were like the student who failed the new test.
GLEAN: Maintained a high safety guarantee even when the data distribution shifted. It proved that by focusing on the cause (the animal) rather than the correlation (the background), you can build an AI that is both smart and provably safe in new environments.

Summary Analogy

Think of existing AI defenses as bodyguards who memorize the route. If the route changes (domain shift), the bodyguard gets lost and the VIP (the prediction) is vulnerable.

GLEAN is a bodyguard who memorizes the VIP's face. No matter where the VIP goes, what they are wearing, or what the background looks like, the bodyguard knows exactly who they are and can guarantee their safety against any small attempt to disguise them.

Technical Summary: GLEAN - Certified Causal Defense with Generalizable Robustness

Problem Statement

Machine learning models, particularly deep neural networks, are vulnerable to adversarial perturbations. While certified defense methods (e.g., Randomized Smoothing) provide theoretical guarantees that a model's prediction remains constant within a specific perturbation radius (robust radius) for a given input, they largely fail to generalize this robustness across domains with distribution shifts.

Existing certified defenses struggle when applied to unseen domains because models often rely on spurious correlations (e.g., background color correlating with object class) rather than invariant causal factors. These spurious correlations vary across domains; when the correlation shifts in a test domain, the model's prediction confidence drops or becomes incorrect. This leads to a significant degradation in the Average Certified Radius (ACR), effectively nullifying the theoretical guarantees of robustness in new environments.

Current approaches lack a unified framework that simultaneously:

Eliminates the negative impact of spurious correlations on robustness.
Defends against perturbations on the latent causal factors that determine labels.
Provides theoretical guarantees for certified robustness in the context of domain shifts.

Methodology: The GLEAN Framework

The authors propose GLEAN (GeneraLizable cErtified cAusal defeNse), a framework that integrates causal inference into certified defense to achieve robustness generalization. The approach is built on three core components:

1. Causal View and Invariant Learning

GLEAN models data generation using a causal graph involving:

$Y$ : The label.
$X$ : The input (e.g., image).
$C$ : Causal factors (invariant features determining $Y$ ).
$S$ : Spurious factors (non-causal features correlated with $Y$ in specific domains).
$D$ : The domain variable.

The framework relies on the Causal Invariance Assumption: The conditional probability $P(Y|C)$ remains invariant across domains, whereas $P(Y|S)$ varies. GLEAN aims to learn a representation of $C$ from $X$ that is invariant to domain shifts, thereby filtering out the influence of $S$ .

2. Certifiable Causal Factor Learning

To extract causal factors robustly, GLEAN employs a Causal Encoder ( $\Psi$ ) trained with two specific mechanisms:

Invariant Risk Minimization (IRM): The encoder is trained to minimize prediction loss while penalizing the gradient of the loss with respect to a dummy classifier weight. This encourages the learned latent representations to be invariant across training domains, effectively disentangling causal factors from spurious correlations.
Lipschitz Constraints: To enable certification in the latent space, the encoder is constrained to be 1-Lipschitz (using Cayley transforms for linear layers and GroupSort for activations). This ensures that the mapping from input space to latent causal space does not amplify perturbations excessively, which is a prerequisite for deriving theoretical bounds.

3. Causally Certified Defense Strategy

The defense process operates in two stages:

Latent Space Certification: Randomized Smoothing (RS) is applied to the latent causal representation $z = \Psi(x)$ . By adding Gaussian noise $\eta \sim \mathcal{N}(0, \sigma^2 I)$ to $z$ , a smoothed classifier $g_\beta$ is formed. Theoretical guarantees (Theorem 1) ensure that predictions remain constant within a certified radius $CR_z$ in the latent space.
Input Space Mapping: Using the Lipschitz property of the encoder ( $\|\Psi(x_1) - \Psi(x_2)\| \le L\|x_1 - x_2\|$ ), the certified radius in the latent space is mapped back to the input space. Theorem 2 establishes that the certified radius in the input space is $CR \ge CR_z / L$ . This provides a theoretical guarantee that the entire classifier $f$ is robust against perturbations in the input space, even across different domains, provided the causal factors are correctly identified.

Key Contributions

Problem Identification: The paper identifies and analyzes the underexplored problem of generalizing certified robustness across domains, highlighting the role of spurious correlations in degrading certified radii.
Novel Framework (GLEAN): It proposes a causality-inspired framework that combines invariant risk minimization with Lipschitz-constrained networks. This allows the model to exclude spurious correlations and perform certified defense on latent causal factors.
Theoretical Guarantees: The authors derive new theorems (Theorem 1 and Theorem 2) that provide theoretical bounds for certified robustness in unseen domains. This bridges the gap between empirical observations of robustness generalization and rigorous theoretical certification.
Empirical Validation: Extensive experiments on synthetic (CMNIST) and real-world datasets (CelebA, DomainNet) demonstrate that GLEAN significantly outperforms existing certified defense baselines (Gaussian, MACER, SmoothAdv, Consistency) in terms of both certified accuracy and Average Certified Radius (ACR) under domain shifts.

Experimental Results

The authors evaluated GLEAN against standard Randomized Smoothing-based baselines on three datasets:

CMNIST: A synthetic dataset with a strong spurious correlation between digit color and class, which is reversed in the test domain.
CelebA: A real-world dataset with spurious correlations between facial attributes (e.g., smiling and hair color).
DomainNet: A dataset with complex domain shifts involving different backgrounds and styles.

Key Findings:

Superior Generalization: GLEAN achieved the highest certified accuracy and ACR across all datasets and radii. For instance, on CMNIST, GLEAN achieved an ACR of 0.2466, significantly outperforming the next best baseline (SmoothAdv at 0.0518).
Robustness to Shift: While baselines suffered severe performance drops when moving from training to test domains (e.g., accuracy dropping from ~72% to ~21% on CMNIST for standard methods), GLEAN maintained high robustness.
Ablation Studies: Removing the invariance penalty caused a drastic drop in performance, confirming that capturing causal factors is essential. Removing Lipschitz constraints resulted in slightly lower performance, validating the necessity of the constraint for theoretical certification.
Parameter Sensitivity: The framework showed robustness to hyperparameter choices, with performance plateauing after a certain threshold for the invariance penalty weight ( $\lambda$ ).

Significance and Claims

The paper claims that GLEAN addresses a critical limitation in current adversarial defense: the inability to transfer theoretical robustness guarantees to new data domains. By shifting the focus from raw input features to invariant causal factors, the framework mitigates the vulnerability caused by spurious correlations.

The authors position GLEAN not just as an improvement in metrics, but as a methodological shift that provides theoretical support for cross-domain robustness. They argue that this approach offers a path toward deploying machine learning models in critical applications (like autonomous driving and healthcare) where security and reliability must be maintained despite environmental changes. The work suggests that future research should further explore causality-inspired defenses and unified approaches for generalizing adversarial robustness.