⚡ electrical engineering

Synthesizing Neural Network Controllers with Closed-Loop Dissipativity Guarantees

This paper proposes a method to synthesize neural network controllers for uncertain nonlinear plants by formulating a linear matrix inequality based on integral quadratic constraints, which is then integrated into a projection-based training algorithm to maximize reward while guaranteeing closed-loop dissipativity, stability, and $L_2$ gain bounds.

Original authors: Neelay Junnarkar, Murat Arcak, Peter Seiler

Published 2026-06-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Neelay Junnarkar, Murat Arcak, Peter Seiler

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to balance a broom on its hand (an inverted pendulum) or to drive a cart with a wobbly, flexible pole on top. You want the robot to be incredibly smart and efficient, using the least amount of energy possible. To do this, you decide to use a "brain" made of a neural network (a type of AI that learns by trial and error).

However, there's a catch: Neural networks are notorious for being unpredictable. If you just let them learn freely, they might find a clever trick to minimize energy by simply turning off and letting the broom fall, or they might react strangely to a sudden gust of wind, causing a crash. In safety-critical situations, you can't afford that.

The Big Idea: The "Safety Cage"
This paper proposes a way to train these smart neural network controllers so they are guaranteed to stay safe, while still being free to learn how to be efficient.

Think of it like training a race car driver.

Standard Training: You tell the driver, "Go as fast as you can." They might win the race, but they might also crash into the wall because they didn't know the limits.
This Paper's Method: You put the driver in a car with a safety cage (a roll cage). You still tell them, "Go as fast as you can," but the cage guarantees that no matter how hard they push, the car won't flip over or fly apart. The driver can still learn to drive incredibly well, but they are physically prevented from doing something catastrophic.

How Does the "Safety Cage" Work?
The authors use a mathematical concept called Dissipativity. In simple terms, this is a way of measuring energy.

Imagine the system (the robot and the plant) as a bucket.
Dissipativity guarantees that the bucket never overflows. Even if you pour in a lot of "energy" (disturbances, wind, errors), the system has a way to absorb or release that energy so it doesn't explode or become unstable.
The paper creates a "safety cage" that ensures this energy balance is always maintained.

The Magic Trick: Turning a Puzzle into a Straight Line
Usually, making a neural network obey strict safety rules is like trying to solve a puzzle where the pieces keep changing shape. It's incredibly hard to calculate.

The authors modeled the neural network controller in a special way (using "Implicit Neural Networks") that looks mathematically similar to the plant it is controlling.
They then used a tool called Integral Quadratic Constraints (IQCs). Think of IQCs as a universal translator. They translate the complex, messy behavior of the neural network's "brain" (its activation functions) and the plant's unknown quirks into a simple, standardized language.
By speaking this same language, they could turn the safety problem into a Linear Matrix Inequality (LMI).
- Analogy: Imagine trying to find a path through a foggy, twisting forest (non-linear problem). The authors found a way to draw a straight, flat highway (convex LMI) that runs parallel to the forest. You can drive your car (train your AI) on this highway easily and quickly, knowing it stays within the safe boundaries of the forest.

The Training Process: The "Check and Correct" Loop
The paper describes a training method that works like a strict coach:

The Sprint (Learning): The neural network takes a step to learn and improve its score (reward).
The Safety Check: The coach immediately checks if this new step breaks the safety rules (the dissipativity guarantee).
The Correction (Projection): If the step is unsafe, the coach doesn't just say "no." They gently push the controller back onto the safe path. It's like a GPS that says, "You tried to drive off the cliff, but I've automatically rerouted you to the nearest safe road that is still very close to where you wanted to go."
Repeat: This happens over and over. The AI learns to be efficient, but it is constantly nudged back into the safety cage.

The Results: Smart and Safe
The authors tested this on two scenarios:

The Inverted Pendulum: Balancing a pole.
The Flexible Rod on a Cart: Driving a cart with a wobbly, bending pole.

They compared their "Safe Neural Network" (D-RINN) against:

A standard, old-school linear controller (safe but not very smart).
A standard neural network with no safety rules (smart but dangerous).

The Outcome:
The "Safe Neural Network" was the winner. It performed almost as well as the dangerous, unconstrained neural network (using very little energy), but it guaranteed that the system would never become unstable. It beat the old-school linear controller by a wide margin in efficiency, while keeping the same level of safety.

In Summary
This paper provides a recipe for building AI controllers that are both super-smart and provably safe. It uses a mathematical "safety cage" to ensure that even as the AI learns complex behaviors, it can never cross the line into chaos. This allows engineers to use the power of modern AI in critical systems without fearing that the AI will suddenly decide to do something reckless.

Technical Summary: Synthesizing Neural Network Controllers with Closed-Loop Dissipativity Guarantees

Problem Statement
Neural network controllers, particularly those trained via reinforcement learning (RL), offer significant expressiveness for complex nonlinear control tasks but often lack formal safety guarantees. Standard training methods can result in controllers that are sensitive to inputs or exhibit unexpected behaviors, posing risks in safety-critical applications. While verification methods exist to analyze closed-loop stability of pre-trained networks, they do not inherently facilitate the synthesis of controllers that satisfy robustness constraints during the training process. Furthermore, existing synthesis methods often face limitations regarding the class of plants they can handle (e.g., restricted to linear time-invariant systems) or the complexity of the neural network architectures they support (e.g., memoryless vs. dynamic).

This paper addresses the challenge of synthesizing neural network controllers for nonlinear and uncertain plants that simultaneously maximize a reward function and satisfy hard constraints on closed-loop dissipativity. Dissipativity is utilized as a unifying framework to certify properties such as stability, $L_2$ gain bounds (robustness to disturbances), and passivity.

Methodology
The proposed approach models both the plant and the controller using a unified framework based on the interconnection of a Linear Time-Invariant (LTI) system and an uncertainty block.

System Modeling:
- Plant: The plant is modeled as an uncertain LTI system where uncertainties and nonlinearities are described using Integral Quadratic Constraints (IQCs). This allows the representation of unmodeled dynamics and nonlinearities.
- Controller: The neural network controller is modeled as a Recurrent Implicit Neural Network (RINN), also known as an equilibrium network. This architecture encompasses common feedforward and recurrent structures. The controller is represented as an LTI system interconnected with an activation function block, where the activation functions are also characterized by IQCs (specifically, sector-bounded and slope-restricted constraints).
Dissipativity Certification:
- The authors derive sufficient conditions for an uncertain LTI system to be dissipative with respect to a specified quadratic supply rate.
- By leveraging the IQC descriptions of both the plant uncertainty and the controller activation functions, they formulate a condition for the closed-loop system's dissipativity.
- Initially, this results in a Bilinear Matrix Inequality (BMI) involving the controller parameters, storage function matrix ( $P$ ), and IQC multipliers.
Convexification and Synthesis:
- To make the synthesis computationally tractable, the authors employ a change of variables (based on Scherer et al., 1997) to convert the BMI into a Linear Matrix Inequality (LMI). This transformation allows the problem to be solved efficiently using semidefinite programming (SDP).
- The LMI is affine in the new decision variables, enabling the simultaneous optimization of the controller structure and the dissipativity certificate.
Training Algorithm:
- A projection-based training algorithm is proposed. The process alternates between a standard RL step (e.g., Proximal Policy Optimization) to maximize reward and a dissipativity-enforcing step.
- If the updated controller parameters violate the dissipativity condition, a projection step solves an SDP to find the nearest controller parameters (in the transformed space) that satisfy the LMI constraints.
- An alternative regularization-based method is also discussed, where a penalty term is added to the loss function, though the projection-based method is shown to be more robust in experiments.

Key Contributions

Unified Framework: The paper presents a method to synthesize neural network controllers for a broad class of nonlinear and uncertain plants by modeling both the plant and the controller as interconnections of LTI systems and IQC-bounded blocks.
Convex Synthesis Conditions: It derives convex (LMI) conditions for closed-loop dissipativity, enabling the use of efficient SDP solvers for controller synthesis.
Projection-Based Training: A novel training procedure is introduced that integrates RL with hard dissipativity constraints via projection, ensuring that the final controller satisfies robustness guarantees while optimizing performance.
Architectural Flexibility: The use of Implicit Neural Networks (INNs) allows the method to support dynamic controllers with memory (RINNs), extending beyond static or purely feedforward architectures used in prior work.

Experimental Results
The approach was validated on two simulation examples:

Inverted Pendulum: The goal was to stabilize the pendulum with minimal control effort. The proposed Dissipative RINN (D-RINN) achieved significantly lower control effort (higher reward) compared to a constrained LTI controller while maintaining the same stability guarantees. It outperformed unconstrained neural networks, which failed to stabilize the system under the reward function.
Flexible Rod on a Cart: This example involved a flexible rod modeled as a rigid body with uncertainty to capture flexibility. The D-RINN was trained to guarantee an $L_2$ gain bound from disturbances to the state. The D-RINN achieved higher rewards (better state regulation with less control effort) than the LTI controller with the same robustness guarantees. Unconstrained neural networks achieved high rewards but failed to provide stability guarantees.

In both cases, the D-RINN demonstrated that formal robustness guarantees do not necessarily preclude high performance; in fact, the neural network controller outperformed the linear controller with identical constraints.

Significance and Claims
The authors claim that their primary contribution is the simultaneous guarantee of robustness to exogenous disturbances and model uncertainty through the unifying framework of dissipativity. They assert that this method offers a computationally tractable way to synthesize neural network controllers that are not only expressive but also formally certified.

The paper positions this work as an advancement over existing methods by:

Handling a larger class of dynamical systems (nonlinear and uncertain plants) compared to methods restricted to LTI plants.
Utilizing a more flexible characterization of neural networks (dynamic, output-feedback) compared to memoryless or state-feedback-only approaches.
Not requiring the existence of a pre-stabilizing base controller, unlike some Youla-Kucera parameterization approaches.

The authors conclude that while the current LMI-based approach has computational scaling limitations, it provides a rigorous foundation for safe, learning-based control, with future work focusing on reducing conservatism and improving numerical stability.

More like this