🤖 machine learning

CEAR: Certified Ensemble Adversarial Robustness in DNNs

This paper introduces CEAR, an ensemble-based defense method that combines empirical training with randomized smoothing and novel voting mechanisms to achieve superior certified robustness against adaptive white-box attacks across multiple datasets.

Original authors: Daniel Sadig, Mohammadreza Maleki, Hamed Karimi, Reza Samavi

Published 2026-06-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Daniel Sadig, Mohammadreza Maleki, Hamed Karimi, Reza Samavi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a team of expert guards (a Deep Neural Network) protecting a valuable building. Their job is to recognize visitors and let the right ones in. However, there are clever thieves (adversarial attacks) who can make tiny, almost invisible changes to a visitor's appearance—like a slightly different hat or a subtle shift in posture—to trick the guards into letting a criminal in.

The paper introduces a new security system called CEAR (Certified Ensemble Adversarial Robustness). Instead of relying on just one guard or one type of trick to stop the thieves, CEAR uses a combination of three smart strategies to make the security team nearly impossible to fool.

Here is how CEAR works, broken down into simple concepts:

1. The Team of Diverse Guards (The Ensemble)

Most security systems use a single guard or a team of identical guards. If a thief figures out how to trick one, they can trick them all.

The CEAR Solution: CEAR hires a whole team of different guards (an "ensemble"). But here's the twist: each guard sees the world slightly differently.
The Analogy: Imagine giving each guard a different pair of slightly foggy glasses. One pair is a little more foggy than the other. Because they see the visitor through different "lenses," a trick that confuses Guard A might not confuse Guard B. This makes it much harder for a thief to find a single trick that works on the whole team.

2. The "Soft" Training (Distillation with Temperature)

Usually, guards are trained to be very strict: "This is a cat, that is a dog." But strict guards are easily confused by small changes.

The CEAR Solution: The system uses a "Head Teacher" to train the guards. Instead of just saying "This is a cat," the Head Teacher says, "This looks 80% like a cat, but it has a little bit of a dog in it."
The Analogy: This is like teaching a child to recognize animals by showing them how similar a wolf is to a dog, rather than just memorizing rigid definitions. This makes the guards' decision-making "smoother." If a thief changes a cat's ear just a tiny bit, the guard doesn't suddenly panic and think it's a car; they still confidently say, "It's still a cat."

3. The "Static" Inference (Noisy Logits)

When a visitor actually arrives at the door, the thieves try to study the guards' reactions to figure out how to break in.

The CEAR Solution: Every time a visitor is checked, the system adds a tiny bit of random "static" or "noise" to the image before the guards look at it.
The Analogy: Imagine the guards are looking at the visitor through a window that is constantly vibrating slightly. Even if the thief tries to calculate exactly how to move their face to trick the guard, the constant vibration makes the calculation useless. The thief can't predict exactly what the guard will see next.

4. The Smart Voting System

Once all the guards have made their decision, how does the team decide who is right? CEAR uses two different voting methods depending on how confident the guards are.

Method A: The "Geometric Median" (For confident situations)
- How it works: If the guards are mostly agreeing, this method finds the "middle ground" that ignores the outliers.
- The Analogy: Imagine a group of friends guessing the weight of a watermelon. If one friend guesses 5 lbs and another guesses 500 lbs, but everyone else says 20 lbs, the "Geometric Median" ignores the crazy guesses and sticks to the realistic consensus. It's great when everyone is pretty sure.
Method B: The "Robust Weighted" Vote (For confusing situations)
- How it works: If the visitor is hard to identify (high noise), this method listens more to the guards who have historically been the most accurate at handling difficult cases.
- The Analogy: If the watermelon is wrapped in a blanket and hard to see, you don't listen to everyone equally. You listen more to the friend who is known to be the best at guessing wrapped objects. This ensures the team doesn't get swayed by the guards who are currently confused.

5. The "Certificate of Safety"

The most exciting part of CEAR is that it doesn't just hope it's safe; it proves it.

The Analogy: Most security systems say, "We think we are safe." CEAR is like a safety inspector who walks around the building and says, "I can mathematically prove that no matter how much the thief shakes the door (up to a certain limit), the lock will not break."
The paper shows that CEAR can prove its safety over a much larger range of "shaking" (perturbations) than previous methods.

What Did They Find?

The researchers tested this system on three different "training grounds" (datasets):

MNIST: Simple handwritten numbers (like a basic security test).
CIFAR10: Colorful pictures of everyday objects (like a busy street).
TinyImageNet: More complex, high-resolution images (like a crowded airport).

The Results:

Better Safety: CEAR proved it could withstand stronger attacks than previous methods.
Less Transferability: A trick that worked on one guard in the team didn't work on the others, so the thieves couldn't reuse their tricks.
Smart Adaptation: The system knew when to use the "Geometric Median" (when things were clear) and when to switch to the "Robust Weighted" vote (when things were messy), keeping the security tight in both scenarios.

In short, CEAR is a security team that uses diverse perspectives, smooth training, random noise, and smart voting to mathematically guarantee that they won't be tricked by clever thieves, even when the thieves know exactly how the guards work.

Technical Summary: CEAR: Certified Ensemble Adversarial Robustness in DNNs

Problem Statement
Deep Neural Networks (DNNs) achieve high accuracy on standard test sets but remain highly vulnerable to adversarial perturbations—small, imperceptible input modifications that drastically alter predictions. This vulnerability hinders deployment in safety-critical applications. Existing defense mechanisms fall into two categories:

Empirical Defenses: Methods like adversarial training and defensive distillation improve robustness during training but lack formal guarantees and often fail against strong, adaptive white-box attacks (e.g., C&W, AutoAttack).
Certified Defenses: Methods like randomized smoothing provide provable guarantees that predictions remain invariant within a specific perturbation bound. However, individual classifiers often struggle to maintain high certified accuracy at larger perturbation radii.
While ensemble methods have shown promise in improving robustness through aggregation, existing approaches often suffer from high adversarial transferability (where attacks on one network succeed on others) and steep declines in certified accuracy as perturbation budgets increase.

Methodology
The authors propose CEAR (Certified Ensemble Adversarial Robustness), a framework that unifies empirical and certified defense mechanisms within an ensemble training pipeline. The method operates in two phases:

Robust Training:
- Variable Gaussian Augmentation (VGA): Unlike standard ensembles that share uniform noise, CEAR trains each network in the ensemble with a distinct noise level. A base standard deviation ( $\sigma_{tr}$ ) is perturbed by a random variable $\beta$ sampled from a uniform distribution, resulting in unique noise distributions ( $\sigma_{tr} + \beta$ ) for each student network. This diversity reduces adversarial transferability.
- Distillation with Temperature Scaling (DTS): A teacher network is trained to generate soft labels (probability distributions smoothed by a temperature parameter $t$ ). Student networks are then trained on Gaussian-augmented inputs using these soft labels. This process smooths decision boundaries and captures inter-class similarities, improving generalization against unforeseen attacks.
Robust Inference:
- Noisy Logits: To prevent attackers from recovering original logits (a vulnerability in standard distillation), Gaussian noise is injected into the input data at inference time. This creates inherently stochastic logits without modifying model parameters, obfuscating gradients against iterative attacks like C&W.
- Aggregation Mechanisms: CEAR employs two voting strategies depending on the confidence regime:
  - Geometric Median (GM): Aggregates softmax probabilities using the geometric median. This is robust to outliers and effective in high-confidence, low-perturbation regions.
  - Robust Weighted Ensemble (RW): Assigns weights to each student network based on their individual certified accuracy on a held-out validation set. Networks with higher certified accuracy contribute more to the final prediction. This mechanism is designed to handle larger perturbation budgets and lower-confidence regimes.
Certified Verification:
The paper extends randomized smoothing to verify ensemble-based classifiers. It defines a smoothed ensemble classifier $G$ and uses Monte Carlo sampling to compute a lower confidence bound on the probability of the predicted class. If this bound exceeds 0.5, the model is certified as robust within a calculated radius $R$ .

Key Contributions

CEAR Framework: A unified defense framework combining DTS and VGA in an ensemble pipeline to promote decision-boundary smoothness and diversity. It leverages noisy logits at inference to hinder gradient-based attacks and utilizes GM and RW voting to recover clean accuracy while improving robustness.
Extended Verification: The authors extend the randomized smoothing verification method to formally verify the robustness of ensemble-based classifiers, deriving a corollary for the robustness radius of an ensemble.
Empirical Validation: Extensive experiments on MNIST, CIFAR10, and TinyImageNet demonstrate that CEAR achieves higher average certified accuracy at larger radii and exhibits greater resilience to gradient-based white-box attacks compared to state-of-the-art baselines (RandSmooth and SWEEN).

Experimental Results

Certified Accuracy: On MNIST and CIFAR10, CEAR variants outperform baselines. Specifically, CEAR(GM) achieves superior certified accuracy in high-confidence regions (small radii), while CEAR(RW) maintains higher accuracy as perturbation radii increase and model confidence degrades. For example, on CIFAR10 with $\sigma_v=1.0$ , CEAR(RW) achieves 20% certified accuracy at $R=1.0$ , surpassing baselines.
Robustness Radius: CEAR(RW) consistently achieves larger certified robustness radii than baselines, particularly in low-confidence regions under high perturbations.
Transferability: CEAR significantly reduces attacker success rates against transferable attacks. On CIFAR10 under the $\ell_\infty$ norm, CEAR(RW) reduced the attacker success rate to 46.00%, compared to 96.56% for RandSmooth and 60.10% for SWEEN.
Computational Overhead: As an ensemble method, CEAR incurs linear computational growth with the number of networks ( $M$ ). However, the authors note this overhead is acceptable for safety-critical applications where verifiable guarantees are required.

Significance and Claims
The paper claims that CEAR addresses the fundamental challenge of enhancing robustness at larger perturbation radii by effectively combining ensemble diversity with certified defense mechanisms. The authors emphasize that the choice of aggregation strategy is critical: optimized geometric median aggregation is preferable when ensemble predictions are confident, while fixed robust weighting is superior in high-uncertainty adversarial regimes. The work demonstrates that integrating variable noise distributions during training with adaptive or robust voting mechanisms can yield provable guarantees that outperform existing single-network and uniform-ensemble defenses. The authors conclude by noting future work will investigate the systematic influence of temperature scaling in DTS on ensemble diversity and calibration.