🤖 machine learning

NERO-Net: A Neuroevolutionary Approach for the Design of Adversarially Robust CNNs

This paper introduces NERO-Net, a neuroevolutionary framework that designs inherently robust CNN architectures by optimizing for high post-attack accuracy without relying on adversarial training during the search process, achieving superior performance on both clean and adversarial CIFAR-10 benchmarks.

Original authors: Inês Valentim, Nuno Antunes, Nuno Lourenço

Published 2026-03-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Inês Valentim, Nuno Antunes, Nuno Lourenço

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are building a team of security guards for a high-tech museum. Your goal is to find the perfect guard who can spot a fake painting (an "adversarial example") instantly, even if the fake looks almost identical to the real one to the naked eye.

Most security firms today use a method called "Adversarial Training." This is like taking your guards and forcing them to practice with a thousand different forgeries every day. They get really good at spotting those specific fakes, but they often become confused by real paintings or forget how to do their other jobs. They are "over-trained" for the specific tricks they've seen.

The authors of this paper, Inês Valentim and her team, asked a different question: "What if we just build a guard who is naturally tough and smart, without forcing them to memorize every possible trick?"

They created a system called NERO-Net (Neuroevolution of adveRsarily Robust artificial neural Networks). Here is how it works, using simple analogies:

1. The Evolutionary Zoo (Neuroevolution)

Instead of hiring a human architect to design the perfect security guard, the authors let nature do the designing. They created a digital "zoo" where thousands of different neural network "creatures" are born.

The Parents: These creatures have different body shapes (architectures). Some are tall and thin, some have many eyes, some have strong legs.
The Mutation: Every generation, the creatures have babies. These babies are slightly different from their parents—maybe they have an extra layer of skin, or their brain is connected differently.
The Survival of the Fittest: The system tests these creatures. The ones that are best at their job survive and become the parents of the next generation. The weak ones die out.

2. The "Fair Test" (The Secret Sauce)

Here is the clever part. In most similar experiments, the researchers would teach the creatures how to spot fakes while they were evolving. This is like teaching a baby how to swim while throwing it into a stormy ocean; it learns to survive the storm, but it might not learn how to swim properly.

NERO-Net does something different:

Standard Training: The creatures are only taught to recognize real paintings (clean data). They are never shown the fakes during their "childhood" (the evolutionary search).
The Fitness Test: When it's time to see who survives, the researchers throw a few fakes at them.
The Goal: They only keep the creatures that are naturally good at spotting fakes without having been specifically trained to do so.

If a creature is good at spotting fakes even though it never saw them before, it means its body structure (architecture) is inherently robust. It's like a guard who has such good eyesight and reflexes that they can spot a fake even without having studied forgery techniques.

3. The "Warm-Up" Phase

At the very beginning, the creatures are very weak. If the researchers started testing for fakes immediately, no one would pass, and the evolution would stop.

The Analogy: Imagine a sports coach. You don't start a rookie by throwing a fastball at them immediately. You let them hit a few soft tosses first to build confidence.
The Solution: NERO-Net has a "warm-up" period where it only cares about how well the creatures recognize real paintings. Once they are decent at that, the researchers start adding the "fake painting" tests to the mix.

4. The Results: The "Super Guard"

After running this evolutionary process for 100 generations, they found a "champion" creature.

The Champion: This specific neural network architecture was naturally very strong.
The Stats: Even though it was only trained on normal data (no fake training), it could still correctly identify about 47% of the fake images. (Most other models without special training would get almost 0%).
The Bonus: When they gave this champion a little bit of extra training (standard training) and then a little bit of "adversarial training" (showing it some fakes), it became even stronger, reaching 40% accuracy against the most sophisticated attacks.

Why This Matters

Think of it like building a house.

Traditional Method: You build a normal house, then try to reinforce the doors and windows with heavy steel after you've built it. It's sturdy, but it's a patchwork job.
NERO-Net Method: You design the house from the ground up with thick, reinforced walls and a smart layout. Even before you add the steel doors, the house is naturally harder to break into because of how it was built.

The Takeaway

The paper proves that how you build a neural network matters just as much as how you train it. By using evolution to find the best "body plans" for these AI models, we can create systems that are naturally tougher against hackers and tricks, without needing to spend years teaching them every possible trick in the book.

It's a shift from "teaching the AI to be tough" to "growing the AI to be tough by nature."

1. Problem Statement

Deep Neural Networks (DNNs), particularly Convolutional Neural Networks (CNNs), are highly susceptible to adversarial examples—inputs with imperceptible perturbations that cause misclassification. While Adversarial Training (AT) is the standard defense, it is computationally expensive and often treats robustness as a byproduct of the training process rather than an intrinsic property of the architecture.

Existing Neuroevolution (NE) and Neural Architecture Search (NAS) approaches for robustness face two main limitations:

Search Space Constraints: Many rely on cell-based search spaces with fixed backbones, limiting the discovery of novel topologies.
Confounding Factors: Most integrate adversarial training during the search. This makes it difficult to distinguish whether robustness stems from the architecture itself or the training procedure.

The authors aim to design CNNs with intrinsic adversarial robustness using neuroevolution, isolating architectural influence by avoiding adversarial training during the evolutionary loop.

2. Methodology: NERO-Net

NERO-Net is a neuroevolutionary framework built upon Fast-DENSER, a hierarchical evolutionary algorithm. The core innovation lies in decoupling architecture search from adversarial training.

A. Fitness Function ( $F_\beta$ )

Instead of relying solely on clean accuracy, NERO-Net employs a composite fitness function that co-optimizes clean performance and adversarial robustness:
$F_\beta = (1 + \beta^2) \cdot \frac{C \times A}{C + \beta^2 \cdot A}$
Where:

$C$ : Clean accuracy.
$A$ : Adversarial accuracy (measured via FGSM attacks during the search).
$\beta$ : A weight factor (set to 4) balancing the importance of clean vs. adversarial performance.

Key Strategy: Candidates are trained using standard (non-robust) methods. If an architecture achieves high accuracy against FGSM without AT, it proves the robustness is intrinsic to the topology.

B. Search Space and Representation

To expand the search space beyond standard cell-based approaches, the authors modified the genotype representation:

Layer Blocks: Instead of evolving single layers, the grammar encodes blocks (e.g., convblock containing Conv-BN-Activation sequences). This mimics modern CNN structures (like ResNet/DenseNet) while allowing internal parameter evolution.
Flexible Connectivity: The "levels back" parameter was updated to allow skip connections that skip intermediate layers (not just the immediate predecessor), enabling more complex topologies.
Parameter Types: New parameter types (int_power2, int_power10) were introduced to efficiently search large ranges for hyperparameters like learning rates and batch sizes.

C. Evolutionary Process

Warm-up Period: The search begins with a "warm-up" phase where only clean accuracy is optimized. Once a threshold ( $\tau = 0.80$ ) is met, the fitness function switches to the composite $F_\beta$ . This prevents the algorithm from getting stuck in local optima early in the search.
Ill-Fitted Detection: The system flags and penalizes "degenerate" solutions (e.g., models that predict a single class for all inputs or suffer numerical instability) to ensure valid architectural exploration.
Seed Initialization: The initial population includes a known high-performing architecture (NSGA-Net) as a seed to accelerate convergence.

3. Key Contributions

Intrinsic Robustness Discovery: Demonstrated that neuroevolution can discover architectures that are robust to adversarial attacks without adversarial training during the search phase.
Novel Fitness Mechanism: Introduced a weighted harmonic mean fitness function that balances clean and adversarial accuracy, using FGSM as an efficient estimator during the search.
Expanded Search Space: Developed a flexible genotypic representation supporting tunable computational blocks and diverse topological connectivity (skip connections), moving beyond rigid cell-based NAS.
Multi-Threat Robustness: Showed that architectures optimized for $L_\infty$ threats also exhibit unexpected robustness against $L_2$ perturbations.

4. Experimental Results

The framework was evaluated on CIFAR-10 with a focus on $L_\infty$ -robustness ( $\epsilon = 8/255$ ).

A. Evolutionary Search Results

Best Individual (Post-Search): Achieved 87.49% clean accuracy and 33.25% adversarial accuracy against FGSM using standard training.
Significance: This proves the architecture itself provides a baseline of robustness, as no adversarial examples were used during the evolutionary training loop.

B. Post-Evolution Training

The best individual was subjected to further training to maximize its potential:

Extended Standard Training:
- Clean Accuracy: 93.47%
- FGSM Adversarial Accuracy: 47.88%
- Observation: The model showed inherent robustness to $L_2$ attacks (58.35% accuracy) despite being optimized only for $L_\infty$ .
Adversarial Training (AT):
- When the best architecture underwent standard AT (7-step PGD), clean accuracy dropped to 84.08%, but robustness against the AutoAttack (AA) ensemble reached 40.40%.
- Comparison: While the final AT performance was slightly lower than a baseline NAS model (NSGA-Net) under AT, the NERO-Net model significantly outperformed the baseline when using only standard training.

C. Multi-Threat Generalization

The evolved architecture demonstrated multi-threat robustness. Although optimized solely for $L_\infty$ threats, it maintained superior performance against $L_2$ bounded perturbations compared to the baseline, suggesting the discovered topology generalizes well across different threat models.

5. Significance and Conclusion

Architectural vs. Training Robustness: NERO-Net provides strong evidence that robustness is not solely a function of training data augmentation but can be an intrinsic property of the network topology.
Efficiency: By avoiding adversarial training during the expensive evolutionary search, the method reduces computational overhead while still discovering robust candidates.
Future Directions: The authors note that while the discovered architectures are promising, they are larger (26M parameters) than typical NAS models. Future work will focus on optimizing adversarial training settings specifically for these evolved architectures and exploring zero-cost proxies to further reduce computational costs.

In summary, NERO-Net successfully bridges the gap between automated architecture design and adversarial robustness, proving that neuroevolution can yield CNNs that are naturally resilient to attacks, even before specialized defense mechanisms are applied.