🤖 machine learning

Learned Lyapunov Shielding for Adaptive Control

This paper proposes a learned Lyapunov shielding framework that augments the Slotine–Li adaptive controller with a structured-quadratic Lyapunov function, a residual Soft Actor–Critic policy, and a physics-informed neural network to ensure closed-form safety filtering and exponential stability while significantly improving tracking performance on Euler–Lagrange systems with unmodeled dynamics.

Original authors: Giansalvo Cirrincione, Adriano Fagiolini

Published 2026-05-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Giansalvo Cirrincione, Adriano Fagiolini

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot arm to move a heavy box from point A to point B. You have two ways to do this:

The "Textbook" Way: You give the robot a perfect physics textbook (the Slotine–Li controller). It knows exactly how heavy the box is and how the arm moves. It works great, but if the box is slightly heavier than expected, or if the joints are sticky, the robot gets confused and moves clumsily.
The "Gambler" Way: You let the robot learn by trial and error (standard Reinforcement Learning). It might eventually learn to move the box perfectly, but it might also crash into the wall or drop the box because it hasn't learned the rules of physics yet.

This paper proposes a third way: A "Smart Coach" that combines the best of both. It takes the reliable textbook robot and adds a "learning assistant" that fixes mistakes, but it puts a safety net around the whole system so the robot can never do something dangerous.

Here is how the paper's "Smart Coach" works, broken down into simple parts:

1. The Safety Net (The "Lyapunov Certificate")

In the world of robotics, a Lyapunov function is like a "stability thermometer." It measures how "unstable" the robot is. If the temperature goes up, the robot is in trouble.

The Problem: Usually, proving a robot is safe requires complex math that is hard to do in real-time.
The Paper's Trick: They created a special, structured "thermometer" (a Learned Lyapunov Certificate) that is guaranteed to be positive (meaning it always measures something real) by its very design.
The Result: Because of this special design, the computer can instantly calculate a "safety filter." Think of this as a traffic cop standing in front of the robot. If the robot tries to make a move that would make the "thermometer" spike (dangerous), the traffic cop instantly blocks that move and nudges the robot back to a safe path. The paper proves this traffic cop always has a valid move to make; it never gets stuck saying, "I can't stop you!"

2. The Learning Assistant (The "Residual Policy")

The textbook robot (Slotine–Li) is good, but it doesn't know about sticky joints or weird friction.

The Solution: They added a Soft Actor–Critic (SAC) policy. Think of this as a student who watches the textbook robot and says, "Hey, the textbook says move left, but I feel the joint is sticky, so let's add a little extra push to the right."
The Catch: This student is allowed to make suggestions, but only if the Safety Net (the traffic cop) says it's okay. This allows the robot to learn from experience without ever breaking the safety rules.

3. The Physics Detective (The "PINN")

Sometimes the robot doesn't know why it's slipping.

The Solution: They added a Physics-Informed Neural Network (PINN). This is like a detective that looks at the robot's movement and tries to figure out the "hidden forces" (like unmodeled friction or a heavy payload) that the textbook didn't account for.
How it helps: The detective feeds this information to the Safety Net. The Safety Net then knows, "Ah, the robot is slipping because of friction, not because it's out of control," and adjusts the safety rules accordingly.

What Did They Find? (The Results)

The team tested this system on a robot arm with two joints (2-DOF) and then on a more complex seven-joint arm (like a human arm, the Franka Panda).

The "Sweet Spot" Gain: When the robot was carrying a weight it had seen during training (the "centroid"), the new system was 41% better at tracking the target path than the old textbook method. The "learning assistant" and the "safety net" worked together to smooth out the movement.
The "Specialization" Problem: The system was amazing at the weights it practiced with, but if you gave it a weight it had never seen (too light or too heavy), it sometimes got worse than the old textbook method. It was like a student who studied hard for a specific test but struggled when the questions changed slightly.
The "Warm-Start" Trap: They discovered a weird bug. If you take a trained robot and try to "fine-tune" it for a new task without starting over, it can get stuck in a bad habit. It's like a student who memorized the answers to last year's test so well that they can't learn the new material. The paper says you have to start from scratch (retrain from zero) when changing tasks to avoid this.
Scalability: They proved this system works even on a big, complex 7-joint robot, not just the small 2-joint one.

The Bottom Line

This paper introduces a way to make robots safer and smarter at the same time.

It uses a mathematically guaranteed safety filter (the traffic cop) so the robot never crashes.
It uses learning to fix the mistakes of traditional physics models.
It proves that this combination is stable and works in the real world, provided you don't try to "tweak" a trained robot for a totally new job without retraining it first.

In short: It's a robot controller that learns from experience but is strictly supervised by a mathematically perfect safety guard.

Technical Summary: Learned Lyapunov Certificates for Safe Residual Reinforcement Learning on Euler–Lagrange Systems

Problem Statement
Adaptive controllers for Euler–Lagrange systems, such as the classic Slotine–Li controller, rely on the linear-in-parameters property of rigid-body dynamics and a priori knowledge of mass, Coriolis, and gravity terms. However, practical robotic systems often exhibit unmodeled nonlinearities (e.g., friction hysteresis, joint elasticity, soft-body effects) that resist clean parameterization. While Reinforcement Learning (RL) offers a path to learn these residuals, standard RL lacks formal stability guarantees and often fails to converge when applied to safety-critical control tasks. Existing approaches to learned Lyapunov functions often rely on generic neural architectures (e.g., MLPs with tanh activations) that require post-hoc verification or complex optimization to ensure positive-definiteness, often leading to infeasible safety filters during deployment.

Methodology
The authors propose a framework that augments the Slotine–Li adaptive controller with three learned components, integrated into a constrained Soft Actor–Critic (SAC) policy:

Structured-Quadratic Lyapunov Function ( $V_\psi$ ): Instead of a generic neural network, the Lyapunov candidate is parameterized as $V_\psi(x) = x^\top P_\psi(x) x$ , where $P_\psi(x) = L_\psi(x)L_\psi(x)^\top + \epsilon I$ . Here, $L_\psi$ is a lower-triangular MLP. This Cholesky parameterization guarantees positive-definiteness by construction.
Residual SAC Policy: A Soft Actor–Critic agent learns a bounded residual torque $\Delta \tau$ to correct the analytic Slotine–Li baseline. The policy is trained to maximize reward (tracking performance) subject to a constraint on the Lyapunov decrease violation.
Physics-Informed Neural Network (PINN): A network $f_\theta$ estimates unmodeled dynamics (residuals) and is trained via a physics loss that enforces the Euler–Lagrange equation structure. This model is used to estimate the drift field for the safety filter.

Safety Filter and Stability Mechanism
The core innovation is a closed-form safety filter derived from the single affine constraint $\dot{V}_\psi + \alpha V_\psi \leq 0$ . Because the system dynamics are affine in control ( $\dot{x} = h(x) + g(x)\tau$ ) and $V_\psi$ is quadratic, the constraint becomes linear in $\tau$ ( $b(x)^\top \tau \leq c(x)$ ).

Closed-Form Projection: The filter projects any raw policy output $\tau_{raw}$ onto the safe half-space via a simple Euclidean projection, avoiding the need for online Quadratic Programming (QP) solvers.
Feasibility Condition: The paper proves that this filter is globally feasible if a "drift-decay condition" holds on the set $Z$ where the control authority vanishes ( $b(x)=0$ ). Specifically, the natural drift dynamics must already be Lyapunov-decreasing on this set.
Robustness: A robust version of the filter incorporates the PINN approximation error ( $\delta$ ) and the gradient bound of $V_\psi$ ( $L_V$ ) to ensure stability even when the learned dynamics model is imperfect.

Key Contributions
The paper establishes five theoretical and empirical contributions:

Closed-Form Feasibility: The structured-quadratic parameterization yields a single-constraint QP with a closed-form solution. The authors prove global feasibility under a verifiable drift-decay condition (Theorem 4).
Stability Guarantees: Exponential stability is proven under exact shielding. A robust extension is provided where the stability margin depends explicitly on the PINN approximation error (Theorem 13).
Convergence: The joint updates of the policy, Lyapunov certificate, and Lagrange multiplier are shown to converge almost surely to a KKT point using a three-timescale stochastic approximation scheme (Theorem 10).
Generalization: A PAC (Probably Approximately Correct) generalization bound is derived, certifying the learned Lyapunov function over compact sets based on finite samples (Proposition 14).
Empirical Validation: Extensive experiments on a 2-DOF manipulator and a 7-DOF Franka Emika Panda demonstrate the framework's efficacy and scalability.

Experimental Results

2-DOF Manipulator: On a system with nonlinear friction and variable payload, the learned Lyapunov certificate was identified as the dominant source of performance gain. Compared to the analytic Slotine–Li baseline, the proposed method reduced tracking error by 41% under nominal friction and 24% under aggressive friction at the training distribution centroid. Ablation studies confirmed that the learned certificate ( $V_\psi$ ) provided significantly more gain than the PINN or the SAC policy alone.
7-DOF Scalability: A study on a 7-DOF Franka Panda (35-dimensional state space) confirmed that the full pipeline (Transformer-based PINN, structured $V_\psi$ , closed-form shield) converges cleanly at industrial scale. When compared to an exact Computed-Torque baseline (which has access to true dynamics), the framework achieved tracking within 1.7% of the baseline, confirming it acts as a certified safety filter rather than a performance booster when the baseline is already optimal.
Failure Modes: The study identified a "warm-start pathology" where a certificate trained under a specific reward structure becomes adversarial to new reward objectives, necessitating fresh initialization. It also noted numerical instability near states where $\nabla V_\psi \approx 0$ , requiring a gating mechanism to prevent torque amplification.

Significance and Claims
The paper claims that the structured-quadratic parameterization is the central design choice that enables the entire framework. By ensuring positive-definiteness by construction, it allows for a computationally efficient, closed-form safety filter that guarantees feasibility without online optimization. The work demonstrates that learned components can be integrated into classical adaptive control to handle unmodeled dynamics while retaining rigorous stability guarantees. The authors emphasize that the framework's value lies in the joint learning of the policy, certificate, and dynamics residual under formal safety constraints, rather than in any single component in isolation. The results suggest that such frameworks are practically usable across manipulator scales, provided the analytic baseline is not already perfect and the training distribution covers the operational range.