🤖 machine learning

A combination of noise and bilateral filters achieve supralinear and scalable adversarial robustness in CNNs

This paper demonstrates that combining Gaussian noise and bilateral filtering as a preprocessor yields supralinear and scalable adversarial robustness in CNNs, achieving state-of-the-art performance with significantly reduced computational cost, parameters, and training data compared to existing defenses.

Original authors: Nicolas Stalder, Benjamin F. Grewe, Matteo Saponati, Pau Vilimelis Aceituno

Published 2026-06-02

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Nicolas Stalder, Benjamin F. Grewe, Matteo Saponati, Pau Vilimelis Aceituno

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart robot that can recognize pictures of cats, dogs, and cars. This robot is incredibly fast and accurate, but it has a weird weakness: if you add a tiny, invisible speck of "static" to a picture (something a human eye can't even see), the robot might suddenly think a cat is a toaster. These invisible tricks are called adversarial attacks.

For a long time, the only way to make these robots tougher was to "train them harder" by showing them millions of these tricky pictures. But this is like trying to learn a language by reading every book in the world; it takes forever, costs a fortune in electricity, and the robot still might get confused by a new type of trick it hasn't seen before.

This paper introduces a much simpler, cheaper, and smarter solution. The authors found that combining two very simple tools creates a defense that is stronger than the sum of its parts.

Here is how it works, using some everyday analogies:

The Two Tools

The "Static" (Gaussian Noise):
Imagine you are trying to sneak up on a guard (the robot) to trick him. If you stand perfectly still, he sees you clearly. But what if you start shaking slightly, like a leaf in the wind? The guard gets confused because he can't lock onto your exact position.
- In the paper: They add random "static" (noise) to the image. This makes the robot's view fuzzy. If an attacker tries to place a trick in a very specific, precise spot, the random shaking moves that spot around, causing the trick to miss its target.
The "Smart Sponge" (Bilateral Filter):
Imagine you have a muddy window. You could wipe it with a wet rag, but that might smear the mud and blur the whole picture. A "bilateral filter" is like a magical sponge that only wipes away the mud (the noise) but leaves the edges of the picture (the cat's ear, the car's wheel) perfectly sharp.
- In the paper: This filter cleans up the image, smoothing out the weird, jagged patterns attackers use, while keeping the important details clear.

The Magic Combination: Why 1 + 1 = 3

The authors discovered that these two tools fight different kinds of bad guys.

The Static is great at stopping tricks that are very precise and thin (like a needle).
The Sponge is great at stopping tricks that are clustered together near the edge of what's allowed.

If you use just the Static, a clever attacker can find a way to hide their trick in a "thick" cluster that the shaking doesn't move enough. If you use just the Sponge, a clever attacker can hide their trick in a "thin" line that the sponge smooths out too much.

But when you use both?
It's like having a guard who is both shaking (Static) and holding a magical sponge (Filter).

If the attacker tries a "thin needle" trick, the shaking messes it up.
If the attacker tries a "thick cluster" trick, the sponge wipes it away.
The Result: The paper calls this "supralinear" growth. It means the protection you get from using both is much bigger than just adding the protection of one to the protection of the other. It's a multiplicative effect.

The Real-World Win

The authors tested this on a standard computer vision system (a CNN) and found amazing results:

Cheaper: They achieved top-tier security using only 35% of the computing power usually required.
Faster: They trained the robot in one-third the time and with one-third the data.
Smaller: They used a robot model that was 50% smaller (fewer "neurons") than the usual champions.
Better: Despite being smaller and cheaper, their system actually ranked second on the world's toughest security tests (beating many much larger, more expensive systems).

The Bottom Line

The paper argues that instead of just brute-forcing our way to security by training bigger and bigger robots, we can use a simple, smart pre-processing step. By adding a little bit of "static" and then "cleaning" the image with a smart filter before the robot even looks at it, we make the robot naturally much harder to trick.

It's a low-cost, high-reward upgrade that makes AI safer without needing to rebuild the whole system from scratch.

Technical Summary: A Combination of Noise and Bilateral Filters for Adversarial Robustness

Problem Statement

Deep neural networks, particularly Convolutional Neural Networks (CNNs), exhibit significant vulnerability to adversarial examples—imperceptible, targeted pixel-level perturbations that cause incorrect predictions. This poses a critical barrier to real-world deployment in safety-critical domains such as autonomous driving and facial recognition.

Current state-of-the-art (SotA) defenses, primarily adversarial training, are computationally expensive (requiring roughly 9× more FLOPs than standard training) and often scale poorly with model size. Furthermore, these defenses are frequently tailored to specific attack types encountered during training, potentially failing against unseen perturbations. While simpler techniques like adding Gaussian noise or applying image filters (e.g., bilateral filtering) have shown modest improvements in robustness with low computational cost, they have historically been developed in isolation. No single filtering or noise method has proven broadly effective across all attack types, and a systematic evaluation of their combined effect is lacking.

Methodology

The authors propose a principled approach combining two distinct defense mechanisms: Gaussian noise addition and bilateral filtering. This combination is applied as a preprocessor during both training and inference.

Theoretical Framework

The paper provides a mathematical analysis demonstrating that noise and filters enhance robustness through complementary mechanisms:

Gaussian Noise: Effective against adversarial attacks that are distributed in "thin filaments" around the image manifold. Noise introduces random deviations that disrupt these specific attack geometries but may fail if attacks are clustered into large spheres.
Bilateral Filtering: Effective against attacks clustered near the edge of the maximum allowed perturbation norm. The filter attenuates deviations while preserving sharp edges, pushing perturbed images back toward the image manifold. However, it struggles if the attack is already very close to the manifold.

The authors argue that because these methods target fundamentally different shapes of adversarial volumes ( $V_a(x)$ ), their combination should theoretically cover a broader range of potential attacks than either method alone, potentially yielding supralinear robustness gains.

Experimental Setup

Datasets: Primarily CIFAR-10, with ablation studies on ImageNet10.
Models: Standard CNNs (EfficientNet-B0) and Wide Residual Networks (WRN) of varying sizes (WRN-28-4, WRN-28-10, WRN-82-12).
Training Protocol: Adversarial training following the TRADES protocol, augmented with synthetic data generated by an elucidating diffusion model (EDM).
Preprocessor Configuration: The pipeline applies zero-mean Gaussian noise followed by multiple iterations of bilateral filtering. The authors note that while the order of operations is theoretically non-commutative, they recommend adding noise first to preserve clean accuracy.

Key Contributions

Theoretical Insight: Mathematical proof that Gaussian noise and filtering operate via distinct mechanisms, protecting against different geometric distributions of adversarial attacks.
Empirical Validation of Supralinearity: Demonstration that combining these methods yields a supralinear gain in adversarial robustness. In ablation studies, the combined model's performance gain exceeded the sum of the individual gains for four out of six tested perturbations.
Efficient SotA Integration: A framework that, when combined with adversarial training, achieves higher robust accuracy than previous SotA defenses while drastically reducing computational requirements.
Scalability: Evidence that the preprocessor scales efficiently across model sizes, significantly reducing the compute needed to reach competitive robustness levels.

Experimental Results

Ablation Studies (CIFAR-10)

Supralinear Effect: On the CIFAR-10 dataset, the combination of Noise + Bilateral Filter achieved a robust accuracy of 36.5% against $L_\infty$ attacks (AutoAttack), compared to 25.5% for noise alone and 1.0% for the filter alone. The combined gain (36.2%) exceeded the linear sum of individual gains (26.1%).
Clean Accuracy Trade-off: While all preprocessing variants reduced clean accuracy, the drop for the combined model was less than the sum of the individual drops, suggesting a synergistic effect on the decision boundary.

Integration with State-of-the-Art Models

The authors integrated their preprocessor with WRN models trained via adversarial training and evaluated them on RobustBench using AutoAttack:

Performance: The best model (WRN-82-12 with preprocessor) achieved 74.32% robust accuracy on AutoAttack, ranking second on the benchmark and surpassing the previous SotA (WRN-94-16, 73.71%) by 0.6%.
Efficiency: This performance was achieved using:
- A model with 50% fewer parameters than the SotA.
- ~33% of the training epochs.
- ~15% of the training data (synthetic samples).
- Only ~35% of the training FLOPs.
Scalability: The method matched the accuracy of competing models while using 2–8× less total compute across three orders of magnitude.
Generalization: The method showed improved robustness against various attacks (APGD- $L_\infty$ , EoTPGD, APGD- $L_2$ ) and perturbation strengths, including those not seen during training (e.g., larger perturbation distances).

Significance and Claims

The paper claims to offer a principled, easily integrable, and computationally efficient framework for enhancing adversarial robustness. Its primary significance lies in:

Theoretical Grounding: Moving beyond heuristic combinations by demonstrating that noise and filters target complementary attack geometries.
Cost-Effectiveness: Achieving SotA-level robustness with a fraction of the computational resources (FLOPs, data, and epochs) required by current methods.
Simplicity: Utilizing a simple preprocessor that requires negligible overhead during inference, unlike complex denoising neural networks or ensemble methods.

The authors remain modest regarding the mechanism, noting that while the combination works, the exact underlying geometry of adversarial attacks in high-dimensional spaces remains an open question for future theoretical exploration. They also acknowledge diminishing returns as model sizes and training resources increase, suggesting that large models may implicitly learn similar transformations, though the preprocessor still accelerates convergence.