⚡ electrical engineering

Kernel-Based Learning of Stable Nonlinear Systems

This paper proposes a kernel-based nonlinear identification procedure that leverages enhanced reproducing kernel theory and the Representer Theorem to systematically learn discrete-time dynamical models with guaranteed stability properties, such as input-to-state stability, while demonstrating through numerical results that these constraints improve long-term simulation accuracy with minimal impact on short-term prediction.

Original authors: Matteo Scandella, Michelangelo Bin, Thomas Parisini

Published 2026-08-06

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Matteo Scandella, Michelangelo Bin, Thomas Parisini

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to predict the future. You show it a video of a swinging pendulum or a bouncing ball, and you ask it to guess where the object will be next. In the world of science, this is called "system identification." It's like trying to figure out the secret recipe of a cake just by tasting a few bites. But here's the catch: a good recipe doesn't just taste right for the first bite; it has to keep working forever. If your robot learns a recipe that makes the ball bounce higher and higher until it flies off into space, that's a disaster. In math and engineering, we call this "stability." A stable system is like a well-behaved dog that stays in the yard, no matter how much you throw a ball. An unstable one is a dog that chases the ball into traffic. For decades, scientists have been great at teaching robots to be stable when the world is simple and linear, like a straight line. But the real world is messy, curved, and nonlinear—like a rollercoaster. Teaching a robot to predict a rollercoaster's path without it crashing is a huge, unsolved puzzle.

This paper, written by Matteo Scandella, Michelangelo Bin, and Thomas Parisini, tackles that exact rollercoaster problem. They propose a new way to teach computers to learn these messy, nonlinear systems while guaranteeing they stay stable. Think of their method as a "safety harness" for the learning process. Usually, when a computer learns, it just tries to fit the data as closely as possible, even if that means learning a crazy, unstable pattern. The authors built a mathematical framework using something called "kernels" (which are like flexible, shape-shifting lenses that help the computer see patterns) and added strict rules to the learning process. These rules act like a bouncer at a club, ensuring that only "stable" models get in. They showed that by tweaking how the computer chooses its learning settings (called hyperparameters), they can force the model to be stable in several specific ways, like ensuring it doesn't go wild when the input gets big, or ensuring that small mistakes in prediction don't grow into huge errors over time.

The researchers didn't just dream this up; they tested it. They ran simulations on two different types of systems: one that was already known to be stable and another that was a bit more complex. They compared their "safety-harnessed" learning method against standard learning methods that don't care about stability. The results were promising. In the simulations, their method learned models that were just as good at predicting the next step as the standard methods. But here's the magic: when they let the models run for a long time to simulate the future, the standard models started to drift and go crazy, while the authors' stable models stayed on track. They even tested this on a model of a potassium ion channel in a neuron (a tiny part of a brain cell), showing that their method could keep the simulation accurate over long periods where other methods failed.

The paper suggests that this approach is a systematic way to solve a problem that has been open for a long time. However, the authors are careful to note that their "safety harness" isn't free. Sometimes, being too strict about stability can make the model slightly less accurate at fitting the data, especially if the model is very complex. They found that for some types of mathematical "lenses" (kernels), it's easier to guarantee stability than for others. For instance, they showed that certain popular lenses simply cannot be used to guarantee the strictest type of stability unless you change them first. But overall, their work provides a solid, mathematically proven toolkit for engineers who need to build predictive models that won't crash and burn when used in the real world. It's a step forward in making artificial intelligence not just smart, but also safe and reliable.

Technical Summary: Kernel-Based Learning of Stable Nonlinear Systems

Problem Formulation
The paper addresses a critical gap in system identification: the lack of learning procedures that systematically guarantee specific stability properties for learned nonlinear discrete-time models. While stability is essential for robustness in prediction and simulation—particularly over long time horizons—existing methods often fail to ensure that the identified model inherits the stability of the underlying system. Classical approaches for linear time-invariant (LTI) systems exist, but the nonlinear case remains largely open due to the diversity of stability notions and modeling techniques. Current nonlinear methods, such as those using neural networks or Koopman operators, often struggle with numerical constraint violations or rely on two-step procedures that first identify an unstable model and then modify it. The authors propose a unified framework to directly learn stable nonlinear predictors that satisfy bounded-input-bounded-state (BIBS) stability, input-to-state stability (ISS), and their incremental counterparts ( $\delta$ BIBS, $\delta$ ISS), while also ensuring stability with respect to prediction errors.

Methodology
The proposed approach extends the standard kernel-based regularized regression framework by embedding stability constraints directly into the learning algorithm and the hyperparameter selection process.

Regularized Kernel Learning: The predictor function $f$ is selected from a Reproducing Kernel Hilbert Space (RKHS) by solving a regularized least-squares problem. The solution is guaranteed to exist in the form of a linear combination of kernel functions (Representer Theorem).
Stability via Viability Sets: To enforce stability, the authors introduce the concept of a "viability set" ( $\Omega_k$ ) for the kernel hyperparameters. This set restricts the selection of hyperparameters to those that ensure the resulting RKHS functions satisfy specific sufficient conditions for stability.
Constrained Optimization: The learning procedure involves a bilevel optimization problem:
- Outer Loop (Model Selection): Hyperparameters are selected to minimize a cost function (e.g., Empirical Bayes or cross-validation) subject to the constraint that they belong to the viability set $\Omega_k$ .
- Inner Loop (Predictor Selection): The predictor $f$ is obtained by solving the regularized regression problem with an additional constraint on the RKHS norm ( $|f|_{k_\eta}^2 \leq \chi$ ). This norm constraint is crucial for satisfying the stability conditions derived in the theoretical analysis.
Theoretical Conditions: The paper derives sufficient conditions on the kernel function (specifically regarding its growth and metric properties) that guarantee the learned system $\Sigma(f)$ possesses the desired stability properties. These conditions link the kernel's behavior to the stability of an auxiliary dynamical system that models both the input-output dynamics and the prediction error evolution.

Key Contributions

Systematic Stability Enforcement: The paper provides a systematic, non-parametric method to learn nonlinear models with guaranteed stability properties (BIBS, ISS, $\delta$ BIBS, $\delta$ ISS) without relying on specific model structures like ARMAX or neural networks.
Unified Framework for Stability and Error: The methodology guarantees stability not only with respect to exogenous inputs but also with respect to prediction errors. This is significant for simulation, as it prevents the indefinite accumulation of errors over infinite time horizons.
Kernel Viability Analysis: The authors analyze the "viability" of various common kernel structures (linear, polynomial, Gaussian, Matérn, and stationary kernels). They establish that while stationary kernels are generally suitable for BIBS stability, they are not suitable for ISS unless modified (e.g., by composition with a degenerate kernel).
Theoretical Guarantees: The paper proves that if the kernel hyperparameters are selected from a specific viability set and the RKHS norm is constrained, the resulting predictor guarantees the desired stability properties.

Results
The theoretical findings are validated through numerical experiments:

Synthetic Models: Monte Carlo simulations on two synthetic discrete-time systems (one ISS and one $\delta$ ISS) demonstrate that the proposed constrained method achieves prediction and simulation performance comparable to unconstrained methods while providing rigorous stability guarantees.
Hodgkin-Huxley Neuron Model: A realistic application involving the identification of potassium ion gate dynamics in a neuron model shows that:
- Unconstrained models (Ha) exhibit large simulation errors over time, rendering them unsuitable for long-term simulation.
- Models constrained for $\delta$ BIBS stability with weak asymptotic gain (Hb) offer a balance, maintaining good simulation performance without significant loss in prediction accuracy.
- Models constrained for $\delta$ ISS (Hc) guarantee bounded simulation errors relative to prediction errors but incur a slight reduction in prediction performance.
- Crucially, the constrained models prevent the divergence of simulation errors, confirming the theoretical benefit of stability with respect to prediction errors.

Significance and Claims
The paper claims to offer a flexible and systematic solution to the open problem of learning stable nonlinear systems. By leveraging reproducing kernel theory, the method allows for the use of various kernel functions and assessment criteria (like Empirical Bayes) while ensuring stability. The authors emphasize that the stability guarantees cover both the system's response to inputs and the accumulation of prediction errors, which is vital for simulation applications.

The authors modestly acknowledge limitations: the stability conditions may be conservative, potentially leading to a loss in prediction performance compared to unconstrained models, and the computational complexity may increase with the model dimension. However, they argue that these trade-offs are acceptable given the robustness guarantees provided, particularly for long-horizon simulations where error accumulation is a primary concern. The work represents a step toward bridging the gap between data-driven learning and control-theoretic stability requirements.

More like this