⚡ electrical engineering

Stability-Preserving Online Adaptation of Neural Closed-loop Maps

This paper introduces a stability-preserving online adaptation mechanism for neural network-based controllers that guarantees closed-loop $\ell_p$ -stability during updates by modeling controllers as causal operators with bounded $\ell_p$ -gain and deriving specific time-scheduled or state-triggered update conditions.

Original authors: Danilo Saccani, Luca Furieri, Giancarlo Ferrari-Trecate

Published 2026-03-25

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Danilo Saccani, Luca Furieri, Giancarlo Ferrari-Trecate

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are driving a car on a winding mountain road. The road is slippery, the weather is changing, and sometimes a deer jumps out in front of you.

The Problem:
Most self-driving cars today are like drivers who memorized a single route perfectly. They are great at following that specific path, but if the road changes or a new obstacle appears, they get confused. They can't "learn" while driving without risking a crash.

If you try to swap the driver mid-drive for a "better" one who knows the new route, you might cause a panic. The car might jerk, swerve, or spin out because the two drivers have different styles. In engineering terms, switching controllers can destabilize the system.

The Solution:
This paper introduces a clever "safety net" that allows a self-driving car to swap its brain (its control software) with a smarter, more up-to-date version while it is still moving, without ever losing control.

Here is how they do it, using some simple analogies:

1. The "Gain" Budget (The Speed Limit)

Think of the car's controller as a driver's aggressiveness.

A low-gain driver is cautious. They turn the steering wheel gently and brake softly. They are slow to react but very stable.
A high-gain driver is aggressive. They yank the wheel and slam the brakes to correct a mistake instantly. They are fast but risky.

The authors realized that you can only switch to a new, more aggressive driver if the car is currently very calm (moving slowly and straight). If the car is already swerving wildly, you can't bring in an aggressive driver; you need to wait until the car settles down.

They created a mathematical "budget" that says: "You are only allowed to switch to a new, smarter controller if the car's current wobble is small enough to handle the new driver's intensity."

2. The Two Switching Strategies

The paper proposes two ways to decide when to swap the driver:

The Alarm Clock Method (Time-Scheduled):
Imagine you set an alarm to check the car every 2 seconds. When the alarm rings, you check: "Is the car calm enough to swap drivers?"
- Yes? Swap the brain for a smarter one.
- No? Keep the current driver and wait for the next alarm.
  This is simple and predictable.
The "Wait for Calm" Method (State-Triggered):
Instead of a clock, you just wait. You keep the current driver until the car naturally slows down and straightens out. The moment the car is calm enough, you instantly swap in the new, smarter driver.
This is more efficient because you don't waste time checking when the car is still swerving; you only act when it's safe.

3. The "IMC" Architecture (The Crystal Ball)

To make this work, the car needs a special setup called Internal Model Control (IMC).
Imagine the car has a crystal ball inside it. This crystal ball simulates the road ahead.

The car drives.
The crystal ball predicts where the car should be.
If the car drifts off course (due to wind or a bump), the difference between the prediction and reality is the "error."
The new controller only needs to fix this error, not the whole car. This makes it much easier to prove that the new driver won't crash the car.

4. Why This is a Big Deal

Usually, in AI and robotics, there is a trade-off:

Stability: The system is safe but dumb (can't adapt).
Adaptability: The system is smart but risky (might crash while learning).

This paper breaks that trade-off. It proves that you can have both. You can keep updating the controller to handle new obstacles, changing weather, or broken parts, and the math guarantees the car will never spin out of control, no matter how many times you swap the brain.

Real-World Results

The authors tested this on two scenarios:

The Mountain Pass: Two robots had to navigate a narrow valley. When they suddenly hit a "wind gust" (a disturbance), the old, static controllers crashed or got stuck. The new "online adapting" controllers swapped their brains mid-drive, adjusted to the wind, and got through safely.
The Moving Obstacles: A robot had to track a path while dodging obstacles that were moving around. The new method allowed the robot to constantly update its plan to avoid the moving targets, performing much better than robots that tried to plan ahead without updating.

The Bottom Line

This paper gives engineers a safety license to let their AI controllers learn and change while they are working. It's like teaching a pilot to swap flight plans mid-air without ever losing altitude, ensuring that even as the world gets more chaotic, our machines stay safe and steady.

1. Problem Statement

Modern control systems operate in dynamic environments where objectives and disturbances change over time. While Nonlinear Optimal Control (NOC) offers a framework to optimize performance, enforcing closed-loop stability while optimizing general nonlinear costs is computationally difficult and often requires restrictive assumptions.

Existing approaches using Neural Networks (NNs) to parametrize controllers (e.g., via System Level Synthesis or Internal Model Control) can guarantee stability for a fixed controller. However, they lack a principled mechanism for online adaptation. A critical challenge is that switching from one stabilizing policy to another—even if both are individually stable—can destabilize the closed-loop system during the transient switching phase (a known issue in switched-systems theory).

The Core Problem: How can we continuously update a neural-network-based controller online to adapt to changing conditions and disturbances while mathematically guaranteeing that the closed-loop system remains $\ell_p$ -stable after every update?

2. Methodology

The authors propose a framework based on Internal Model Control (IMC) architecture and gain-budgeted triggering conditions.

A. Theoretical Foundation: IMC and Operator Parametrization

System Model: The system is modeled as a discrete-time, time-varying nonlinear system with process noise: $x_t = f_{t-1}(x_{t-1}, u_{t-1}) + w_t$ .
IMC Architecture: The controller is parametrized by a free, causal operator $M$ (implemented as a Recurrent Equilibrium Network - REN). The control law is defined as $u = M(x - F(x, u))$ , where $F$ is the system dynamics operator.
Stability Guarantee: Based on Theorem 1, if the operator $M$ belongs to the space of finite $\ell_p$ -gain operators ( $M \in L_p$ ), the closed-loop system is guaranteed to be $\ell_p$ -stable. This decouples stability from optimality; the controller can be an approximate solution to the optimization problem, provided the gain constraint is met.

B. The Online Adaptation Mechanism

The authors introduce a mechanism to switch between different operators $M^{(i)}$ (parameterized by $\theta^{(i)}$ ) at discrete time instants $t_i$ . To prevent instability during switching, they derive a gain-budgeted update condition:

Triggering Condition: An update is only permitted if the system state at the switching time is sufficiently small relative to the "gain budget" of the new controller.
- Condition: $|x_{t_i}| \leq \epsilon^{(i)}$
- Where $\epsilon^{(i)}$ is derived from the inequality: $\gamma(F) \cdot (\gamma(M^{(i)}) + 1) \cdot \epsilon^{(i)} \leq r^{(i)}$ .
- Here, $\gamma(F)$ is the gain of the plant, $\gamma(M^{(i)})$ is the gain of the new controller, and $r^{(i)}$ is a non-negative scalar sequence in $\ell_p$ .
Two Implementation Schemes:
- Time-Scheduled (Algorithm 1): Updates are attempted at fixed intervals ( $t_{i+1} = t_i + t_{opt}$ ). At each interval, the algorithm checks if the current state $|x_{t_i}|$ is small enough to allow a new controller with a specific gain bound. If the condition is met, the new controller is optimized; otherwise, the old one is retained.
- State-Triggered (Algorithm 2): The update period is not fixed. The system waits until the state naturally decays to a threshold $\epsilon^{(i)}$ (calculated based on a pre-defined maximum gain $\bar{\gamma}$ ) before triggering an update. This reduces unnecessary computations when the state is large.
Optimization: At each valid trigger, the new controller parameters $\theta^{(i)}$ are found by solving a finite-horizon optimization problem (minimizing a truncated cost) subject to the gain constraints. Crucially, the optimization does not need to reach a global or even local minimum; it only needs to satisfy the gain constraint to ensure stability.

3. Key Contributions

Stability-Preserving Update Condition: A novel theoretical condition derived from finite-gain and small-gain arguments that guarantees $\ell_p$ -stability for a switched closed-loop system under repeated online controller updates.
Practical Triggering Mechanisms: Two implementable schemes (Time-Scheduled and State-Triggered) that trade off computational effort against adaptivity while strictly enforcing the stability condition.
Decoupling of Stability and Optimality: The framework allows for approximate, early-stopped, or budgeted optimization (e.g., using fewer samples or shorter horizons) without compromising stability, as long as the gain budget is respected.
Empirical Validation: Demonstration on nonlinear benchmarks showing consistent performance improvements over static offline baselines and naive receding-horizon open-loop controllers.

4. Results

The approach was tested on two nonlinear robotic tasks:

Mountain Problem (Collaborative Navigation):
- Scenario: Two agents navigating a narrow valley with obstacles, subject to abrupt impulse disturbances.
- Performance: The proposed online method reduced the average cost by 35.1% (nominal) and 40.3% (perturbed) compared to a state-of-the-art offline-trained neural controller.
- Insight: The ability to update the controller online allowed the system to adapt to previously unseen disturbance events that the static controller could not handle.
Dynamic Obstacle Avoidance:
- Scenario: Tracking a circular path while avoiding moving obstacles with perturbed sinusoidal trajectories.
- Performance: The online method outperformed both the offline controller and a Receding-Horizon Open-loop (RHO) planner. It achieved lower costs with smaller variability.
- Insight: The state-triggered approach effectively handled the time-varying nature of the obstacles and disturbances, maintaining stability while optimizing performance.

5. Significance

This work addresses a fundamental gap in learning-based control: the lack of safety guarantees during the adaptation phase.

Safety: It provides a rigorous mathematical guarantee that switching controllers will not destabilize the system, a critical requirement for real-world deployment.
Flexibility: By decoupling stability from optimality, it enables the use of computationally cheaper, approximate optimization methods online, making real-time adaptation feasible on embedded hardware.
Future Impact: The framework opens pathways for integrating reinforcement learning with stability guarantees, potentially allowing policy improvement during learning rather than just after convergence. It also suggests future directions for handling model-plant mismatch and updating system models online.