Learning Neural Controllers with Optimality and Stability Guarantees Using Input-Output Dissipativity

Technical Summary: Learning Neural Controllers with Optimality and Stability Guarantees Using Input-Output Dissipativity

Problem Statement
Deep learning has shown promise in solving complex nonlinear control tasks across robotics, transportation, and power systems. However, these methods typically lack formal guarantees for critical properties such as stability. While learning-based approaches using counterexample-guided inductive synthesis (e.g., via SMT solvers) can verify stability for neural controllers, they often fail to provide guarantees for optimality over an infinite horizon. Existing methods frequently optimize performance over finite sample trajectories, leading to "myopic" designs that do not ensure long-term optimality or global stability. The core challenge addressed in this paper is designing neural controllers that simultaneously guarantee asymptotic stability and optimality with respect to user-defined cost functions for general nonlinear systems.

Methodology
The authors propose a framework based on input-output dissipativity theory, a generalization of Lyapunov theory that utilizes storage functions and supply rate functions to characterize energy dissipation in input-output pairs.

1. Theoretical Framework:

   • The system is modeled as $\dot{x} = f(x) + g(x)u$.
   • The authors define a storage function $V(x)$ (analogous to a Lyapunov function) and a supply rate function $w(x, u)$ in a Quadratic-Storage-Resistive (QSR) form: $w(x, u) = x^\top Q(x)x + 2x^\top S(x)u + u^\top R(x)u$.
   • Stability: If the system is dissipative with $w(x, u) < 0$ under a specific control law, the closed-loop system is stable. The paper derives a structured control law $\pi(x) = -R(x)^{-1}S(x)^\top x$ that guarantees stability if specific matrix inequalities (involving $\Delta(x) = S(x)R(x)^{-1}S(x)^\top - Q(x)$) are positive definite.
   • Optimality: The authors prove that this structured control law is the optimal solution to an infinite-horizon optimal control problem. The associated cost functional is derived from the learned storage and supply rate functions. Furthermore, they show that if additional regularity conditions are met, the cost functional becomes a nonlinear quadratic form, allowing the controller to be shaped toward specific user-defined objectives.

2. Learning Algorithm (Counterexample-Guided Inductive Synthesis):

   • Neural Parameterization: The storage function $V(x)$ and the matrix functions $Q(x), S(x), R(x)$ are parameterized by feed-forward neural networks.
   • Dimensionality Reduction: A key technical contribution is a reformulation of the dissipativity condition. Instead of sampling over the joint state-input space (which is difficult for high-dimensional inputs), the authors derive a condition (Proposition 8) that depends only on the state space $x$. This involves a matrix inequality $M(x) \succ 0$ derived via Schur complement, significantly reducing the sampling dimensionality.
   • Training Loop: The training employs a hybrid approach:
     • Learner: Optimizes neural network parameters using loss functions derived from the dissipativity conditions (positive definiteness of $V$, $R$, and $\Delta$, and the matrix inequality $M$).
     • Verifier: Uses Satisfiability Modulo Theories (SMT) solvers (specifically dReal) and Projected Gradient Descent (PGD) to formally verify the conditions over the region of interest.
     • Counterexamples: If the verifier finds a violation, the counterexample is added to the training set to refine the neural networks. This loop continues until formal guarantees are satisfied.

Key Contributions

• Novel Framework: A new method for designing neural controllers that provides provable stability and infinite-horizon optimality guarantees by learning input-output dissipativity.
• Theoretical Guarantees: Proof that the derived control law stabilizes any system sharing the same dissipativity property and is the optimal solution to a specific optimal control problem with a learned cost functional.
• Dimensionality Reduction: A reformulation of the dissipativity condition that reduces the sampling space from the state-input product space to the state space alone, making the learning of multi-input systems more efficient.
• Cost Shaping: A mechanism to shape the learned cost functional to match user-defined objectives, bridging the gap between theoretical optimality and practical performance metrics.

Experimental Results
The framework was validated on three benchmark environments: an Electric Circuit, an Inverted Pendulum (static and dynamic control), and a Rod on a Cart.

• Baselines: The method was compared against Neural Control Lyapunov Functions (NCLF), Neural ODEs (NODE), and a variant without cost shaping (OURS-NCFS).
• Stability: Unlike NODE, which failed to stabilize the system in some scenarios despite lower training costs, the proposed method provided formal stability guarantees verified by dReal.
• Optimality: The proposed method (OURS) consistently achieved lower cumulative costs than NCLF and, in several cases (e.g., Electric Circuit, Inverted Pendulum), matched or outperformed NODE while maintaining stability.
• Linear Case: In the Electric Circuit experiment (a linear system), the learned controller successfully replicated the theoretical Linear Quadratic Regulator (LQR) solution, demonstrating the framework's ability to recover known optimal solutions.

Significance and Claims
The paper claims that its primary significance lies in unifying stability and optimality guarantees within a learning-based control framework. By leveraging dissipativity theory, the authors argue that their approach overcomes the limitations of purely trajectory-based training (which lacks infinite-horizon guarantees) and traditional Lyapunov-based learning (which often lacks explicit optimality certificates). The ability to learn the supply rate and storage functions directly allows the controller to inherit optimality properties that can be shaped toward specific user-defined objectives. The authors note that while learning these functions under hard constraints remains challenging, their reformulation and verification-guided synthesis provide a robust path toward safe and optimal neural control.