⚡ electrical engineering

Learning Neural Network Controllers with Certified Robust Performance via Adversarial Training

This paper presents an adversarial training method that jointly synthesizes neural network controllers and dissipativity certificates to formally guarantee robust closed-loop performance on nonlinear dynamical systems, achieving certified robust regions up to 78 times larger than linear matrix inequality-based approaches.

Original authors: Neelay Junnarkar, Yasin Sonmez, Murat Arcak

Published 2026-04-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Neelay Junnarkar, Yasin Sonmez, Murat Arcak

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to balance a broom on its hand (an inverted pendulum). You want the robot to be so good at this that it never drops the broom, even if someone pushes it, if the wind blows, or if the robot's own sensors are slightly off.

In the world of engineering, this is called Robust Control. The problem is that while modern "Neural Network" robots are incredibly smart and learn fast, they are also a bit of a "black box." We don't always know why they make a decision, and we can't easily prove they will never fail in a dangerous situation.

This paper presents a new way to train these robots so we can mathematically guarantee they will stay safe, even when things go wrong.

Here is the breakdown of their method using simple analogies:

1. The Problem: The "Black Box" vs. The "Safety Net"

The Neural Network (The Brain): It's like a genius student who learns by trial and error. It gets really good at balancing the broom, but if you ask it, "Why did you move your hand left?" it might just say, "I felt like it." We can't trust it in a hospital or a self-driving car without proof.
The Old Way (LMI): Engineers used to try to prove safety by simplifying the robot's brain into a basic, linear math problem. It's like trying to describe a complex, swirling hurricane using only a straight line. It's safe, but it's very conservative. It says, "We can only trust the robot if the wind is barely blowing," which wastes the robot's potential.
The New Goal: We want to prove the robot is safe without simplifying its brain too much, so we can trust it in bigger, messier, real-world scenarios.

2. The Solution: Two Tools Working Together

The authors combine two powerful tools to solve this:

Tool A: The "Stress-Test Machine" (α,β-CROWN)

Think of this as a super-fast video game simulator.

What it does: It takes the robot's complex brain and runs millions of "what-if" scenarios in a split second. It tries to break the robot by finding the exact combination of wind, pushes, and sensor errors that would make it drop the broom.
The Analogy: Imagine a security guard trying to break into a bank vault. If the guard can't find a way to open the vault after trying every possible combination, the vault is certified as "secure." This tool is the ultimate security guard for the robot's logic.

Tool B: The "Safety Certificate" (Dissipativity & QCs)

Think of this as a mathematical "energy budget" or a safety net.

What it does: It defines a rule: "As long as the robot stays within this specific energy zone, it cannot fall."
The Analogy: Imagine the robot is walking on a tightrope. The "Certificate" is a giant, invisible safety net that expands and shrinks. The goal is to make this net as big as possible so the robot has plenty of room to move without falling.
Handling Uncertainty: The paper also deals with "unknowns" (like a gust of wind we didn't predict). They use a tool called Quadratic Constraints (QCs) to put a "fence" around these unknowns. It doesn't know exactly where the wind will hit, but it knows the wind won't be stronger than a certain limit.

3. The Secret Sauce: "Adversarial Training"

This is the most creative part. Usually, you train a robot to do a task, and then you check if it's safe. This paper does them at the same time.

The Game: Imagine a game of "Cat and Mouse."
- The Cat (The Robot): Tries to balance the broom and stay inside the safety net.
- The Mouse (The Adversary): Tries to push the robot out of the safety net by finding the weakest point (the "counterexample").
The Process:
1. The Mouse tries to break the robot.
2. The Robot learns from the breakage and gets stronger.
3. The Safety Net (the Certificate) also reshapes itself to cover the new, stronger robot.
4. They repeat this thousands of times.
The Result: By the end, the Robot is so strong, and the Safety Net is so perfectly shaped, that the Mouse literally cannot find a way to break it.

4. The Results: A Massive Leap Forward

The authors tested this on a simulated inverted pendulum (the broom-balancing robot).

The Old Method (LMI): Could only guarantee safety for a tiny, tiny area. It was like saying, "The robot is safe only if it stands perfectly still."
The New Method:
- Just using the "Stress-Test Machine" (without training) made the safe area 51 times bigger.
- Training the robot and the safety net together made the safe area 78 times bigger than the old method.

Summary

Think of it like this:
In the past, to prove a car was safe, we built a tiny, slow test track and said, "If it doesn't crash here, it's safe."
This paper says, "Let's build a massive, realistic obstacle course. Let's have a robot learn to drive it while a super-computer tries to crash it at every turn. If the robot survives the super-computer's worst attacks, we can mathematically prove it's safe for the real world."

The takeaway: They found a way to make AI controllers not just "smart," but provably safe in chaotic, real-world environments, expanding the zone of safety by nearly 80 times compared to old methods.

1. Problem Statement

Neural Network (NN) controllers demonstrate superior empirical performance on nonlinear dynamical systems but lack formal guarantees required for safety-critical deployment. While recent works have addressed nominal stability, guaranteeing robust performance (stability and disturbance attenuation) in the presence of external disturbances and model uncertainty remains a significant challenge.

Existing verification methods face a trade-off:

Linear Matrix Inequalities (LMIs) with Integral Quadratic Constraints (IQCs): Computationally efficient but introduce significant conservativeness by bounding nonlinearities with quadratic constraints.
Exact Verification (e.g., $\alpha,\beta$ -CROWN): Provides tight bounds for known nonlinearities but struggles to scale when combined with complex uncertainty sets or when used for controller synthesis.

The core problem is to jointly synthesize an NN controller and a robust dissipativity certificate that formally guarantees closed-loop performance under uncertainty, while minimizing the conservativeness inherent in traditional LMI-based approaches.

2. Methodology

The authors propose a framework that combines Adversarial Training with Formal Verification using $\alpha,\beta$ -CROWN. The approach leverages the complementary strengths of two techniques:

$\alpha,\beta$ -CROWN: Used for the direct, tight analysis of known nonlinear dynamics (e.g., NN activations, plant nonlinearities like $\sin(\theta)$ ).
Quadratic Constraints (QCs): Used exclusively to characterize non-parametric model uncertainties (e.g., gain bounds, sector bounds).

A. Theoretical Foundation: Robust Dissipativity

The system is modeled as a feedback interconnection of a nominal plant, an NN controller, and an uncertainty operator $\Delta$ . The goal is to certify Robust Dissipativity on a bounded set $(X, D, \Delta)$ , defined by:

Robust Forward Invariance (RFI): The system state remains within a bounded sublevel set $\Omega_{V,\rho}$ for all time.
Dissipation Inequality: A storage function $V(x)$ satisfies $V(x_K) - V(x_0) \leq \sum s(d_k, e_k)$ , ensuring performance bounds (e.g., $\ell_2$ -gain or passivity).

The authors derive sufficient conditions (Theorem 1) that can be verified via $\alpha,\beta$ -CROWN. These conditions require finding a storage function $V$ and QC multipliers such that specific logical implications hold for all states within the certified region.

B. Adversarial Training Algorithm

The synthesis process (Algorithm 1) involves two stages: Training and Verification.

1. Architecture:

Controller: A Recurrent Implicit Neural Network (RINN), chosen for compatibility with LMI initialization and robustness analysis.
Storage Function ( $V_\phi$ ): A hybrid function combining a quadratic term ( $x^T P x$ ) and a neural network modulation term ( $1 + \alpha \tanh(\psi(x))$ ). This allows the certificate to start as a standard quadratic Lyapunov function and evolve into a more expressive non-quadratic shape.

2. Training Loop:
The training minimizes a total loss function $L_{total}$ composed of three terms:

Sample Loss ( $L_{sample}$ ): Penalizes violations of the dissipativity condition on uniformly sampled states.
Adversarial Loss ( $L_{adv}$ ): Uses Projected Gradient Descent (PGD) to find "worst-case" counterexamples (states where the condition fails) and penalizes them. This guides the network to harden against specific vulnerabilities.
Anchor Loss ( $L_{anchor}$ ): Encourages the certified region to expand by penalizing the storage function value at "anchor" states located just outside the current verified boundary.

3. Domain Expansion:
Once the condition is satisfied for a current bounding box, the algorithm simulates trajectories to find a larger reachable set, updates the bounding box, and continues training.

4. Formal Verification:
After training, the final parameters are fixed. The authors use $\alpha,\beta$ -CROWN to perform a binary search (bisection) on the level set $\rho$ to find the maximum volume of the certified region $\Omega_{V,\rho}$ where the robust dissipativity conditions hold strictly.

3. Key Contributions

Joint Synthesis Framework: A novel method to co-train NN controllers and dissipativity certificates, moving beyond fixed-controller verification.
Hybrid Verification Strategy: Successfully integrating $\alpha,\beta$ -CROWN (for nonlinear dynamics) and QCs (for uncertainty) to avoid the high conservativeness of pure LMI/IQC approaches while maintaining tractability.
Adversarial Training for Control: Adapting adversarial training techniques (PGD) to the control synthesis domain, using counterexamples to the robust dissipativity condition to guide the learning of both the controller and the certificate.
Novel Storage Function: The design of a hybrid storage function that initializes as a quadratic form (warm-started via LMI) and learns nonlinear corrections, enabling larger certified regions.

4. Experimental Results

The method was tested on a torque-actuated inverted pendulum under two scenarios:

Scenario A: $\ell_2$ -Gain Bound (Disturbance Attenuation)

Goal: Maximize the volume of the state space where the system attenuates disturbances.
Results:
- LMI Baseline: Certified volume = 0.16.
- Before Training (using $\alpha,\beta$ -CROWN): 51.2 $\times$ larger than LMI.
- Fixed Controller (Trained Certificate): 67.5 $\times$ larger than LMI.
- Joint Training (Controller + Certificate): 78.1 $\times$ larger than the LMI baseline.
- Significance: Joint training provided a 1.52 $\times$ improvement over the "Before Training" verification and 1.16 $\times$ over training the certificate alone.

Scenario B: Robust Stability (Model Uncertainty)

Goal: Certify stability under non-parametric uncertainty (sector-bounded gain variation).
Results:
- LMI Baseline: 0.80.
- Joint Training: 10.2 $\times$ larger than LMI.
- Improvement: Joint training yielded a 1.35 $\times$ improvement over the "Before Training" baseline.

Computation Time:
While the proposed method is more computationally intensive (minutes to hours vs. seconds for LMI), the trade-off yields significantly larger certified safety regions, which is critical for practical deployment.

5. Significance and Conclusion

This paper addresses a critical gap in control theory: the lack of scalable, non-conservative methods for certifying NN controllers in safety-critical environments.

Reduction of Conservatism: By using $\alpha,\beta$ -CROWN for the known nonlinearities, the method avoids the "loose" bounds typical of IQC-based LMI methods, resulting in certified regions up to 78 times larger.
Scalability: The adversarial training approach allows the system to learn complex, non-convex certificates that standard convex optimization (LMI) cannot find.
Practical Impact: The framework provides a pathway to deploy high-performance NN controllers in safety-critical applications (e.g., robotics, autonomous vehicles) with mathematically rigorous guarantees of robustness against disturbances and model errors.

The authors conclude that while computation time is higher, the substantial increase in the certified safe operating region makes this approach superior for real-world applications where maximizing the safety envelope is paramount. Future work aims to extend this framework to full Integral Quadratic Constraints (IQCs).