📊 statistics

Learning to control switching nonlinear systems with Koopman operator regression

This paper proposes a control framework for nonlinear systems with finite action spaces that uses Koopman operator regression in a reproducing kernel Hilbert space to learn linear switching predictive models, which are then employed in model predictive control with theoretical guarantees on learning rates and sub-optimality.

Original authors: Edoardo Caldarelli, Oleksii Kachaiev, Cesare Molinari, Lorenzo Rosasco

Published 2026-07-14

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Edoardo Caldarelli, Oleksii Kachaiev, Cesare Molinari, Lorenzo Rosasco

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to balance a wobbly, unpredictable stick on its finger. The stick doesn't just fall down; it twists, spins, and reacts in wild, non-straight ways depending on how the robot pushes it. This is what scientists call a "nonlinear system," and it's notoriously hard to control because the math gets messy fast.

This paper introduces a clever trick to tame that chaos. Instead of trying to solve the messy, twisting math directly, the authors suggest "lifting" the problem into a different world—a higher-dimensional space where the rules suddenly become simple and straight. Think of it like taking a tangled ball of yarn and magically stretching it out until it's a perfectly straight line. In this new world, the chaotic stick behaves like a predictable, straight-moving object.

The Magic Ladder: Koopman Operators
The tool they use to do this stretching is called the Koopman operator. In the real world, the stick's movement is a complicated curve. But in this "lifted" world, the movement is just a simple switch. If the robot pushes left, the stick moves one way; if it pushes right, it moves another. It's like a train that only has a few tracks to choose from. The authors show that even though the original system is a wild nonlinear beast, we can find a family of these "train tracks" (linear operators) that describe its behavior perfectly, as long as the robot has a limited set of moves to choose from.

Learning from a Few Snapshots
Here's the catch: the robot doesn't know the tracks yet. It has to learn them. The authors teach the robot by showing it a bunch of "snapshots" of the stick's movement. They use a method called Koopman operator regression (a fancy way of saying "learning the pattern from data") to figure out exactly how those tracks look.

They proved mathematically that if you give the robot enough snapshots, it can learn these tracks with high accuracy. The more data you feed it, the closer the learned tracks get to the real ones. They didn't just guess this; they derived specific rates showing how the error shrinks as the number of data points grows. For example, with enough data, the error in predicting the next step drops at a specific rate (scaling with $n^{-1/6}$ in the fastest scenario), meaning the model gets sharper and sharper.

The "Look-Ahead" Strategy: Model Predictive Control
Once the robot knows the tracks, it still has to decide which one to take at every moment. The paper uses a strategy called Model Predictive Control (MPC). Imagine the robot is a chess player who doesn't just look at the next move, but simulates the next 10 or 15 moves in its head to see which path leads to the best outcome.

The authors show that even if the robot is only looking a short distance ahead (a finite "predictive horizon"), it can still do a great job. They proved that if the robot looks far enough ahead (specifically, if the horizon $T$ is large enough relative to a constant $C$ derived from the system's cost), the strategy becomes almost as good as the perfect, infinite-horizon plan. The "sub-optimality" (how much worse it is than the perfect plan) drops exponentially as the robot looks further ahead.

What About Mistakes?
Since the robot learned the tracks from data, it might make small mistakes. The paper tackles this head-on. They showed that even with these learned, slightly imperfect tracks, the robot's performance doesn't crash. Instead, the final cost (how well it balanced the stick) stays within a predictable bound. The worse the learning error, the slightly worse the final result, but the relationship is smooth and controlled. They didn't just say this happens; they wrote down the exact formula showing how the learning error translates into control error.

The Test Drive: The Duffing Oscillator
To prove this wasn't just theory, the authors tested it on a famous wobbly system called the Duffing oscillator. They simulated the robot controlling this system with two different sets of moves: a symmetric set (pushing left or right with equal force) and an asymmetric set (adding a stronger "push" option).

In their simulations, they found that:

More data helps: When they increased the number of training snapshots from a few to $10^6$ , the robot's performance improved significantly.
Looking further helps: When they increased the "look-ahead" horizon $T$ from 1 to 15 steps, the robot stabilized the system much better. With a short look-ahead ( $T=1$ ), the system wandered around with multiple attractors (it couldn't decide where to settle). With a long look-ahead ( $T=15$ ), it smoothly stabilized right at the center.
The cost function matters: They used a specific cost function that included a discount factor $\lambda = 0.9999$ to ensure the robot cared about the long-term future without getting stuck in infinite loops.

What They Don't Claim
It's important to note what this paper doesn't say. They don't claim this works for any system with infinite control options; they specifically require a finite set of actions (like a switch with a few positions). They also don't claim the system becomes perfectly stable in the limit if the control set is finite; instead, they use a time-varying cost to handle the fact that the system might just stay bounded rather than settling perfectly to zero. They avoid assuming the system is "ergodic" (a specific statistical property about time averages), which makes their method more flexible than some previous approaches.

The Bottom Line
The authors have built a bridge between messy, real-world nonlinear chaos and clean, linear math. They showed that by "lifting" the problem, learning the rules from data, and using a smart "look-ahead" strategy, you can control complex systems effectively. They proved this works mathematically and backed it up with simulations on a classic wobbly system. While they haven't tested this on a real physical robot yet (that's a future step), the math and the computer simulations suggest it's a solid, reliable way to teach machines how to handle the unpredictable.

Technical Summary: Learning to Control Switching Nonlinear Systems with Koopman Operator Regression

Problem Statement

This work addresses the challenge of identifying and controlling nonlinear dynamical systems where the control space $U$ is finite. The system dynamics are governed by a nonlinear switching flow map $f: X \times U \to X$ , described by the difference equation $x_{t+1} = f(x_t, u_t)$ . The primary difficulties in controlling such systems are twofold:

Nonlinearity and Unknown Dynamics: The flow map $f$ is generally unknown and nonlinear, making the solution of the infinite-horizon optimal control problem computationally intractable.
Finite Control Sets: The finiteness of the control set can lead to closed-loop dynamics that are ultimately bounded but not asymptotically stable in the limit, potentially resulting in unbounded optimal control costs.

To address these issues, the authors propose a data-driven framework that transforms the nonlinear switching problem into a linear switching problem within a Reproducing Kernel Hilbert Space (RKHS), leveraging the Koopman operator formalism.

Methodology

1. Koopman Lifting in RKHS

The core methodology involves lifting the state space $X$ into a high-dimensional (potentially infinite-dimensional) observable space $\mathcal{H}$ via a feature map $\psi: X \to \mathcal{H}$ .

Linear Switching Dynamics: For a finite control set $U$ , the nonlinear dynamics are represented as a family of linear operators $\{K_u\}_{u \in U}$ acting on the lifted state $z_t = \psi(x_t)$ . The evolution is governed by $z_{t+1} = K_{u_t} z_t$ .
Operator Construction: The authors define the Koopman operators $K_u$ on the RKHS $\mathcal{H}$ by extending the pre-Koopman operators defined on the image of the state space. This extension relies on the assumption that the kernel is strictly positive definite and that the operators are uniformly bounded on a dense subspace of $\mathcal{H}$ .
Avoidance of Invariant Measures: Unlike some existing literature, this approach does not require the existence or knowledge of an invariant measure (e.g., for ergodic dynamics) to define the operators.

2. Data-Driven Identification via Regression

Since the operators $K_u$ are unknown, they are estimated from finite data samples using nonparametric regression in the RKHS.

Learning Problem: For each control $u$ , the operator $\hat{K}_u$ is learned by solving a regularized operator-valued regression problem (Tikhonov regularization) to minimize the error between the predicted lifted state and the observed next state.
Convergence Guarantees: The paper derives learning rates for the approximation error $\|\hat{K}_u - K_u\|$ . These rates depend on the sample size $n$ , the kernel properties, and a "source condition" parameter $r \in (1/2, 1]$ which quantifies the smoothness of the target operator relative to the kernel geometry. The analysis does not rely on ergodicity or time-reversibility assumptions.

3. Model Predictive Control (MPC)

To solve the infinite-horizon optimal control problem in the lifted space, the authors employ Model Predictive Control (MPC).

Time-Varying Stage Cost: To handle the potential lack of asymptotic stability due to finite actions, the stage cost $\ell(t, z, u)$ is explicitly time-varying (e.g., incorporating a discount factor). This ensures the optimal control cost remains bounded.
Exact vs. Inexact MPC:
- Exact MPC: Assumes the true Koopman operators are known. The paper establishes that the sub-optimality gap decays exponentially with the predictive horizon $T$ , provided a controllability assumption holds.
- Inexact MPC: Uses the learned surrogate models $\hat{K}_u$ . The paper quantifies the sub-optimality gap by combining the statistical learning error (from the Koopman regression) with the robustness properties of the MPC algorithm.

Key Contributions

Linear Switching Model for Finite Controls: The study formalizes a linear, Koopman-based predictive model specifically for nonlinear dynamics with finite control spaces, avoiding the bilinear or linear time-invariant assumptions common in other works.
Nonparametric Learning with Convergence Bounds: The authors demonstrate how to learn these Koopman models via nonparametric regression in RKHSs. They derive learning bounds that do not rely on ergodicity or time-reversibility, offering a more general theoretical framework.
Robust MPC Formulation: The paper formulates an infinite-horizon, unconstrained optimal control problem with a time-varying stage cost to accommodate finite action sets. It solves this using MPC and provides a theoretical performance analysis.
Sub-Optimality Analysis:
- For exact Koopman dynamics, the paper derives sub-optimality bounds showing exponential decay with the horizon length.
- For learned (surrogate) dynamics, the paper provides a high-probability bound linking the statistical learning error to the closed-loop control performance. This bridges the gap between statistical learning theory and robust control.
Empirical Validation: The proposed algorithms are implemented and tested on a Duffing oscillator benchmark, empirically validating the theoretical findings regarding the impact of the predictive horizon and the number of training samples on control performance.

Results and Findings

Learning Rates: The theoretical analysis shows that the one-step prediction error of the approximated Koopman model scales as $O(n^{-1/6})$ in the fastest regime (depending on the source condition parameter $r$ ). This matches rates found in supervised learning, suggesting the estimates are sharp.
MPC Performance:
- The sub-optimality gap for the exact MPC algorithm decays exponentially with the predictive horizon $T$ .
- When using learned models, the sub-optimality gap is bounded by a term dependent on the learning error $\epsilon(n, \delta)$ . As the number of samples $n$ increases, the learned model becomes more accurate, and the closed-loop cost approaches the optimal value.
Numerical Simulations: Experiments on the Duffing oscillator confirm that:
- Increasing the predictive horizon $T$ improves controller performance, leading to better stabilization near the origin.
- Increasing the number of training samples $n$ reduces the control cost, validating the theoretical learning bounds.
- The method works effectively with both symmetric and asymmetric control sets, with the latter showing faster approach to switching surfaces.

Significance and Claims

The paper claims to provide a rigorous theoretical framework for controlling switching nonlinear systems using data-driven Koopman operators. Its significance lies in:

Bridging Theory and Practice: It connects statistical learning theory (RKHS regression) with control theory (MPC) to provide provable performance guarantees for data-driven control.
Generalization: By avoiding assumptions like ergodicity or specific system structures (e.g., bilinearity), the approach is applicable to a broader class of nonlinear systems with finite control inputs.
Quantifiable Robustness: The work explicitly quantifies how estimation errors from finite data propagate to control performance, offering a "high-probability bridge" between learning and control.
Practical Implementation: The authors provide an open-source implementation and demonstrate that the theoretical sub-optimality bounds are relevant in practice, as evidenced by the Duffing oscillator simulations.

The authors remain modest in their claims, noting that the theoretical analysis relies on specific assumptions (e.g., boundedness, source conditions) and that future work could extend these results to different sampling strategies or verify assumptions for specific classes of nonlinear systems. They also acknowledge that real-time application may require approximations for the combinatorial search inherent in the optimal control problem.