🤖 machine learning

Learning dynamically inspired bases for Koopman and transfer operator approximation

This paper proposes a machine learning approach that dynamically adapts orthonormal basis functions to efficiently approximate Koopman and transfer operators, thereby enabling accurate recovery of spectral properties, eigenfunctions, and invariant measures for complex nonlinear dynamical systems.

Original authors: Gary Froyland, Kevin Kühl

Published 2026-03-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Gary Froyland, Kevin Kühl

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict the weather, the movement of a flock of birds, or the flow of traffic. These are nonlinear dynamical systems. They are messy, chaotic, and incredibly hard to predict because a tiny change in the beginning can lead to a massive, unexpected change later (the "butterfly effect").

For a long time, scientists have tried to solve this by pretending these messy systems are actually linear (simple and straight). If you can turn a chaotic system into a simple one, you can use standard math tools to predict it.

This paper introduces a new tool called SABON (Single Autoencoder Basis Operator Network) to do exactly that. Here is how it works, explained through simple analogies.

1. The Problem: The Wrong Lens

Imagine you are trying to describe the shape of a cloud.

The Old Way: Scientists used to say, "Let's describe this cloud using a grid of perfect squares." Or, "Let's use a grid of perfect circles."
The Problem: Clouds aren't made of squares or circles. If you force a square grid onto a fluffy cloud, you get a very bad, blocky approximation. You miss the details. In math terms, they were using "pre-defined bases" (like squares or standard waves) that didn't fit the specific shape of the system they were studying.

2. The Solution: A Custom-Tailored Suit

The authors propose a new method: Don't force the system to fit your math; let the math learn the shape of the system.

Think of it like a 3D body scanner at a tailor shop.

Instead of giving you a suit that comes in standard sizes (Small, Medium, Large), the scanner measures your exact body shape.
It then creates a custom suit (a "basis") that fits your unique curves perfectly.
SABON does this for math. It looks at the chaotic data and "learns" a set of custom-shaped building blocks (functions) that fit the system's specific behavior perfectly.

3. How SABON Works (The Four-Step Dance)

The paper describes a neural network architecture that acts like a four-step dance to learn these custom blocks:

The Encoder (The Scanner): It looks at the data and tries to figure out what the "custom building blocks" should look like. It's like the tailor measuring your shoulders, waist, and hips.
The Projector (The Translation): It takes a complex input (like a weather pattern) and translates it into a simple list of numbers based on those custom blocks.
The Linear Map (The Simple Math): Here is the magic trick. In the world of these custom blocks, the chaotic system behaves like a simple, straight line. The network learns a simple matrix (a table of numbers) to predict how the system moves from one state to the next.
The Reconstructor (The Suit Maker): It takes the simple prediction and turns it back into the complex shape of the real world.

4. Why is this better than the old way?

The paper tested this on three types of systems:

The Circle (Simple): Imagine a point spinning around a circle. The old methods used standard waves (like a guitar string). SABON learned the exact waves needed to describe that spin perfectly.
The Cat Map (Moderately Chaotic): Imagine stretching and folding a piece of dough (like making pasta). The dough gets stretched in one direction and squished in another.
- Old Method: Used a grid of squares. The squares got stretched and squished, becoming useless for prediction.
- SABON: Learned "stretched-out" building blocks that naturally follow the direction the dough is being pulled. It's like using long, thin noodles instead of square pasta to describe the stretch.
The Conjugated Cat (Very Chaotic): Imagine the dough is being stretched, but the table it's on is also curving and twisting.
- Old Method: The square grid was completely confused.
- SABON: It learned blocks that curved and twisted with the dough. It found the "hidden geometry" of the chaos.

5. The "Secret Sauce": Orthogonality and Sparsity

The paper mentions two technical tricks that make this work well:

Orthogonality: Imagine your custom building blocks are like a set of rulers. If they are all leaning on each other, it's hard to measure. SABON forces them to be at perfect 90-degree angles to each other. This makes the math stable and easy to solve.
Sparsity: Sometimes, the system only cares about a small part of the space at a time. SABON learns to focus on those specific areas (like a spotlight) rather than trying to describe the whole dark room at once.

The Big Picture

SABON is a machine learning tool that automatically invents the best possible "language" to describe a chaotic system.

Instead of forcing a system to speak English (standard math), SABON listens to the system, learns its native dialect, and then translates the chaos into a simple, linear story that we can easily predict.

Why does this matter?
If we can predict chaotic systems better, we can:

Predict weather patterns more accurately.
Control the flow of electricity in a smart grid.
Understand how diseases spread through a population.
Design better robots that can move through complex environments.

The authors proved that this method works better than the traditional "one-size-fits-all" math approaches, especially when the system is messy, twisted, and chaotic.

1. Problem Statement

Nonlinear dynamical systems are often analyzed using Koopman and Transfer (Perron–Frobenius) operators, which lift the nonlinear evolution of states to a linear (but infinite-dimensional) evolution of observables (functions). While this framework enables the use of linear techniques for nonlinear problems, practical application requires finite-dimensional approximations.

Key Challenges:

Basis Selection: Traditional methods (e.g., Extended Dynamic Mode Decomposition, EDMD) rely on predefined basis functions (monomials, Fourier modes). These are often ill-suited for complex, anisotropic, or chaotic dynamics, leading to poor spectral recovery and high approximation errors.
Invariance: Finding a finite-dimensional subspace that is invariant (or nearly invariant) under the operator is difficult. If the chosen subspace does not align with the system's intrinsic geometry (e.g., stable/unstable manifolds), the approximation fails to capture emergent behaviors like invariant measures (SRB measures).
Separation of Learning: Existing neural approaches often learn the operator and the embedding separately, or optimize for state reconstruction rather than spectral accuracy, failing to adapt the basis to the specific dynamics of the operator.

2. Methodology: Single Autoencoder Basis Operator Network (SABON)

The authors propose SABON, a neural architecture that jointly learns a data-driven orthonormal basis and the linear operator representation within that basis.

Architecture Components

SABON consists of four stages:

Encoder ( $E$ ): A stack of vector-valued neural networks that learns a set of basis functions $\{\phi_j\}_{j=1}^N$ . These functions are evaluated pointwise on a discretized state space grid.
Projection ( $P$ ): Projects an observable $g$ onto the learned basis using inner products: $P(g) = (\langle g, \phi_1\rangle, \dots, \langle g, \phi_N\rangle)^\top$ .
Latent Map ( $G$ ): A learnable linear map (matrix) that approximates the action of the Transfer/Koopman operator in the coefficient space.
Reconstructor ( $R$ ): Lifts the transformed coefficients back to the function space via linear combination: $R(a) = \sum a_j \phi_j$ .

Loss Function

The network is trained to minimize a composite loss function $J$ :

Operator Approximation Error ( $E_1$ ): Measures the relative $L^2$ error between the true operator action $L(g)$ and the reconstructed approximation $R \circ G \circ P(g)$ .
Sparsity Penalty ( $E_2$ ): An optional $L^1$ norm penalty on basis functions to encourage local support (useful for identifying localized structures).
Reconstruction Penalty ( $E_3$ ): Ensures the subspace can accurately represent $k$ -step iterates ( $L^k g$ ), forcing the basis to adapt to the anisotropic geometry of the dynamics over time.
Orthogonality: The inner-product structure of the projection block naturally drives the learned basis toward orthogonality during gradient descent, improving the conditioning of the loss landscape.

Theoretical Foundation

The authors prove a Universal Approximation Theorem for this setting. Unlike previous works (e.g., BasisONet) that use separate encoders for input and output spaces, SABON uses a single neural basis for both, which is sufficient because Transfer and Koopman operators map $L^2(X)$ to $L^2(X)$ . The theorem guarantees that for any continuous linear operator, there exists a neural basis and a linear map that can approximate the operator arbitrarily well on compact sets.

3. Key Contributions

Joint Learning Framework: SABON simultaneously optimizes the basis functions and the operator matrix, ensuring the basis is dynamically adapted to the system rather than being pre-defined.
Automatic Discovery of Anisotropy: The method automatically discovers basis functions that align with the stable and unstable manifolds of chaotic systems, overcoming the limitations of isotropic bases (like Fourier modes).
Spectral Recovery: The framework enables the accurate recovery of spectral properties, including eigenvalues, eigenfunctions, and invariant measures (SRB measures), even for systems where these are distributions rather than smooth functions.
Efficiency: By learning a compact, adapted basis, SABON achieves higher accuracy with fewer basis functions compared to traditional methods.

4. Numerical Results

The authors validated SABON on three examples of increasing complexity:

A. Circle Rotation (Linear/Integrable)

Setup: Rotation on a circle ( $T(\theta) = \theta + \alpha$ ).
Result: SABON successfully recovered the analytic eigenvalues ( $\lambda_k = e^{-ik\alpha}$ ) and eigenfunctions ( $e^{ik\theta}$ ).
Observation: The learned basis functions coincided perfectly with the theoretical Fourier modes, demonstrating that SABON can recover known invariant subspaces.

B. Weakly Nonlinear Perturbed Cat Map (Anosov)

Setup: A perturbation of Arnold's cat map on the 2-torus.
Result: The learned basis functions aligned with the contracting and expanding directions of the map.
Comparison: When approximating the SRB measure (the physical invariant measure), SABON significantly outperformed a Fourier basis of the same cardinality (324 functions).
- Error Reduction: SABON reduced the relative $L^2$ and $H^{-1}$ approximation errors by a factor of ~4 compared to Fourier.
- Reason: Fourier modes are isotropic (aligned with axes), whereas the SRB measure for hyperbolic systems is concentrated along unstable manifolds. SABON learned "ridge-like" functions aligned with these manifolds.

C. Strongly Nonlinear Conjugated Cat Map

Setup: A diffeomorphic conjugation of the perturbed cat map, introducing curved stable/unstable foliations and significant non-volume preservation.
Result: The learned basis functions adapted to the curved geometry of the dynamics, showing pronounced curvature not present in the linear case.
Performance: Despite the increased geometric complexity, SABON maintained superior accuracy over Fourier bases in reconstructing the SRB measure. The Fourier basis failed to capture the fine-scale curved structures, while SABON's learned basis provided a high-fidelity approximation.

5. Significance and Conclusion

Overcoming Isotropy: The paper demonstrates that for chaotic, anisotropic systems, fixed isotropic bases (like Fourier) are fundamentally inefficient. Data-driven bases that align with the system's intrinsic geometry (stable/unstable manifolds) are essential for accurate spectral analysis.
Spectral Gap and Invariant Measures: SABON successfully identifies the spectral gap and recovers the leading eigenfunction (SRB measure) even when the measure is a distribution (not a smooth function), a task where standard $L^2$ methods often struggle.
Future Directions: The framework is extendable to high-dimensional and partially observed systems. Future work aims to derive quantitative rates relating network depth/width to approximation error.

In summary, SABON represents a significant advancement in operator-theoretic machine learning, moving from "fixed basis" approximations to dynamically inspired, adaptive bases that capture the true geometric structure of nonlinear dynamical systems.