🤖 machine learning

Sample-Specific Noise Injection For Diffusion-Based Adversarial Purification

This paper proposes Sample-specific Score-aware Noise Injection (SSNI), a framework that adaptively adjusts the noise injection level for each input sample based on its deviation from the clean data distribution, thereby significantly improving the accuracy and robustness of diffusion-based adversarial purification methods.

Original authors: Yuhao Sun, Jiacheng Zhang, Zesheng Ye, Chaowei Xiao, Feng Liu

Published 2026-02-16

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yuhao Sun, Jiacheng Zhang, Zesheng Ye, Chaowei Xiao, Feng Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Digital Laundry" Problem

Imagine you have a very smart, but easily confused, security guard (the AI Classifier). This guard is great at recognizing faces, but hackers have found a way to put invisible "glitter" (adversarial noise) on people's faces. To the naked eye, the person looks normal, but the glitter tricks the guard into thinking a dog is a cat, or a friend is an enemy.

To fix this, researchers use a "Digital Laundry" machine (a Diffusion Model). The idea is simple:

Wash: Throw the dirty image into the machine and add a bunch of soap bubbles (Gaussian noise) to scrub away the invisible glitter.
Spin: Spin the machine to remove the bubbles and the glitter, leaving a clean image behind.
Check: Show the clean image to the security guard.

The Problem: The "One-Size-Fits-All" Mistake

In previous versions of this "Digital Laundry," the machine had a single dial for how much soap to use. Let's call this the Noise Level ( $t^*$ ).

The researchers set the dial to a fixed number (e.g., 100 bubbles) for every single image, no matter what.

Why this fails:

Scenario A (The Lightly Stained Shirt): Imagine an image that is already 99% clean, with just a tiny speck of glitter. If you blast it with 100 bubbles of soap, you might scrub the shirt so hard that you tear the fabric or wash away the actual design. The image becomes unrecognizable.
Scenario B (The Muddy Boot): Imagine an image covered in thick mud (heavy adversarial noise). If you only use 10 bubbles, the mud stays. The machine fails to clean it, and the guard still gets confused.

The old method was like using a high-pressure hose on a delicate flower and a gentle mist on a muddy boot. It just didn't work for everyone.

The Solution: SSNI (The "Smart Soap Dispenser")

The authors of this paper propose a new system called SSNI (Sample-specific Score-aware Noise Injection).

Think of SSNI as a Smart Soap Dispenser that looks at every piece of clothing before deciding how much soap to use.

The "Sniffer" (Score Network): Before washing, the machine uses a special sensor (a pre-trained AI called a score network) to sniff the image. It measures how "far away" the image is from being a normal, clean photo.
- Analogy: If the image is a clean white t-shirt, the sensor says, "Smells fresh! Only a little soap needed."
- Analogy: If the image is a muddy boot, the sensor says, "Smells terrible! We need a massive amount of soap!"
The "Adjustable Dial" (Reweighting): Based on that sniff, the machine automatically adjusts the soap level ( $t^*$ ) for that specific image.
- Clean images get a gentle mist (low noise injection) so their details aren't destroyed.
- Dirty images get a heavy blast (high noise injection) to ensure all the adversarial "glitter" is scrubbed away.
The Result: The laundry comes out perfectly clean. The security guard can now easily tell the difference between a dog and a cat, and the system is much harder for hackers to trick.

Why This Matters

The paper proves that one size does not fit all. By customizing the "washing" process for every single image, they achieved two major wins:

Better Accuracy: They didn't accidentally ruin the clean images (which happened with the old "one-size-fits-all" method).
Better Security: They successfully cleaned the heavily attacked images that the old method missed.

Summary in a Nutshell

Old Way: Use the same amount of "noise" to clean every image. Result: Some images get ruined, others stay dirty.
New Way (SSNI): Use a smart sensor to measure how dirty each image is, then apply the exact right amount of noise to clean it without damaging it.
Outcome: A smarter, safer, and more accurate AI defense system that treats every image like a unique individual.

The authors call this Sample-specific Score-aware Noise Injection, but you can just think of it as "The Tailored Wash Cycle for AI."

1. Problem Statement

Context: Deep Neural Networks (DNNs) are vulnerable to adversarial examples (AEs), where imperceptible perturbations cause misclassification. Adversarial Purification (AP) is a defense strategy that uses generative models to remove adversarial noise before classification. Diffusion-Based Purification (DBP) has emerged as a leading AP method, utilizing the denoising capabilities of diffusion models.

The Core Issue: Existing DBP methods (e.g., DiffPure, GDMP, GNS) rely on a sample-shared constant noise level ( $t^*$ ) for the forward diffusion process. They inject a fixed amount of Gaussian noise into all input samples (both clean and adversarial) before reversing the process to recover the clean image.

Flaw 1 (Insufficient Denoising): For heavily perturbed adversarial samples, a fixed $t^*$ (e.g., 100) may be too small to sufficiently corrupt the adversarial pattern, leaving residual noise that the reverse process cannot remove.
Flaw 2 (Over-Denoising): For clean or lightly perturbed samples, the same fixed $t^*$ may be too large, causing excessive disruption of semantic information. This leads to a degradation in clean accuracy because the reverse process struggles to recover the original semantics from an over-corrupted state.

Hypothesis: The optimal noise level $t^*$ is not universal; it should vary on a sample-specific basis. Cleaner samples require less injected noise, while more perturbed samples require more aggressive noise injection to effectively "reset" them to the data manifold.

2. Methodology: Sample-specific Score-aware Noise Injection (SSNI)

The authors propose SSNI, a framework that adaptively adjusts the noise injection level ( $t^*$ ) for each sample based on its deviation from the clean data distribution.

Key Components:

Score Norm as a Metric:
- The method leverages the score function ( $\nabla_x \log p_t(x)$ ), which represents the gradient of the log-probability density.
- Intuition: Samples closer to the high-density region of clean data (clean samples) have low score norms. Samples far from the distribution (adversarial examples) have high score norms.
- Observation: Empirical analysis (Figure 2) confirms that score norms scale linearly with the perturbation budget ( $\epsilon$ ).
Expected Perturbation Score (EPS):
- To avoid sensitivity to a single reference noise level, SSNI uses EPS, which aggregates score norms across a spectrum of noise levels ( $t \sim U(0, t_S)$ ).
- $EPS(x) = \mathbb{E}_{t \sim U(0, t_S)} [\nabla_x \log p_t(x)]$ .
- The magnitude $\|EPS(x)\|$ serves as the metric for the sample's deviation.
Adaptive Reweighting Functions:
SSNI replaces the constant $t^*$ with a sample-specific $t(x)$ calculated via a reweighting function $f$ :
$t(x) = f(\|EPS(x)\|, t^*)$
Two specific functions are proposed:
- Linear Reweighting ( $SSNI\text{-}L$ ): Normalizes the score norm to scale $t^*$ linearly.
- Non-linear Reweighting ( $SSNI\text{-}N$ ): Uses a sigmoid function to smoothly adjust $t^*$ , allowing it to approach a maximum bound for highly perturbed samples.
- Bias Term: An optional bias $b$ is added to allow the reweighted $t^*$ to exceed the original baseline $t^*$ , ensuring sufficient noise injection for strong attacks.
Workflow:
- Input sample $x$ $\rightarrow$ Compute $\|EPS(x)\|$ using a pre-trained score network.
- Compute adaptive $t(x)$ using the reweighting function.
- Inject noise up to step $t(x)$ (Forward Process).
- Reverse diffusion from $t(x)$ to 0 to recover the purified image.
- Classify the result.

3. Key Contributions

Discovery of Sample-Specific Optimal Noise: The paper establishes that a single global $t^*$ is suboptimal. It demonstrates that optimal noise levels vary significantly across samples depending on their perturbation strength.
SSNI Framework: A general, plug-and-play framework that integrates with existing DBP methods (DiffPure, GDMP, GNS) to enable sample-specific noise injection without retraining the diffusion model or classifier.
Theoretical Justification: The authors provide theoretical proofs (Appendix K) linking score norms to noise levels, showing that different noise levels result in distinct score norms, validating the use of score norms as a proxy for determining the required noise injection depth.
Efficiency: The reweighting process is lightweight, adding minimal computational overhead (approx. 0.5s on CIFAR-10 and 5s on ImageNet-1K per image) compared to the full diffusion process.

4. Experimental Results

The method was evaluated on CIFAR-10 and ImageNet-1K against strong adaptive white-box attacks (PGD+EOT, BPDA+EOT, AutoAttack, DiffAttack).

Accuracy-Robustness Trade-off:
- Clean Accuracy: SSNI significantly improves standard accuracy (e.g., +3.58% on DiffPure for CIFAR-10) by injecting less noise into clean samples, preserving their semantics.
- Robust Accuracy: SSNI maintains or improves robustness against adversarial examples by injecting sufficient noise to remove strong perturbations.
- Comparison: Across all baselines (DiffPure, GDMP, GNS) and architectures (WideResNet, ResNet-50, Swin-Transformer), SSNI-N consistently outperforms the original methods in both metrics simultaneously.
Specific Gains (CIFAR-10, PGD+EOT $\ell_\infty$ ):
- DiffPure + SSNI-N: Standard Accuracy increased from 89.71% to 93.29%; Robust Accuracy increased from 47.98% to 48.63%.
- GDMP + SSNI-N: Standard Accuracy increased from 92.45% to 94.08%; Robust Accuracy increased from 36.72% to 40.95%.
Generalization: The method generalizes well to different attack types (including Diff-PGD and DiffAttack) and different model architectures (Transformers).

5. Significance

Paradigm Shift: SSNI challenges the prevailing assumption in diffusion-based purification that a fixed noise schedule is sufficient. It introduces the concept of adaptive noise injection tailored to the specific "health" of each input sample.
Practical Deployment: By improving clean accuracy without sacrificing robustness, SSNI addresses a major bottleneck in adversarial defense: the trade-off where robust defenses often degrade performance on clean data.
Modularity: As a general framework, SSNI can be seamlessly integrated into any existing DBP pipeline, making it a versatile tool for enhancing the robustness of DNN-based systems in critical applications (e.g., autonomous driving, security).

In summary, the paper demonstrates that tailoring the purification process to the specific characteristics of each input sample leads to superior defense performance, effectively leveraging the full potential of diffusion models for adversarial purification.