🤖 machine learning

Deep Spectral Learning of Embedded Latent Transfer Operators for Stochastic Dynamical Systems

This paper introduces the Deep Spectral Encoder (DSE), a novel operator-based latent state-space model that learns time-invariant nonlinear feature maps via functional canonical correlation analysis to enable stable and superior spectral learning, sequential Bayesian filtering, and dynamic mode decomposition for stochastic nonlinear dynamical systems under noise and partial observability.

Original authors: Ryogo Tanaka, Yoshinobu Kawahara

Published 2026-06-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ryogo Tanaka, Yoshinobu Kawahara

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict the future path of a chaotic system, like a double pendulum swinging wildly or a stock market fluctuating with noise. The problem is that the data you see (the "observations") is messy, incomplete, and often distorted. Traditional methods often try to build a giant, complex map of every single detail, which can get confusing and break down when the data gets noisy.

This paper introduces a new method called Deep Spectral Encoder (DSE). Think of it as a smart translator and a crystal ball combined. Here is how it works, using simple analogies:

1. The Translator: Turning Chaos into a Clean Language

Imagine you are watching a foreign movie with terrible subtitles. The words (the raw data) are jumbled and hard to understand.

The Encoder: The DSE uses a neural network (a type of AI) to act as a translator. It takes the messy, raw video frames or data points and translates them into a "clean language" of features.
The Goal: Instead of trying to understand the raw pixels or numbers directly, the system converts them into a structured, mathematical "state" that captures the essence of the system's behavior, stripping away the noise.

2. The Crystal Ball: Finding the Hidden Rules

Once the data is translated into this clean language, the system needs to figure out how the system evolves over time.

The "Spectral" Part: The authors use a technique called Stochastic Realization. Imagine you have a pile of past events and a pile of future events. The system looks for the hidden connections between them, kind of like finding the common thread in a tangled ball of yarn. It uses a mathematical tool (Canonical Correlation Analysis) to find the most important "directions" or patterns that link the past to the future.
The Result: It identifies a small set of "latent states" (hidden variables) that act as the true coordinates of the system. It's like realizing that even though a pendulum looks like it's moving randomly in 3D space, it's actually just swinging back and forth along a single, predictable line.

3. The Rulebook: Learning the Laws of Motion

Now that the system has the clean coordinates, it needs to learn the rules of the game.

The Transfer Operator: Instead of guessing the rules with a complex neural network that might overfit (memorize the noise), DSE learns a simple, linear "rulebook" (a matrix) that tells it how the state moves from one second to the next.
The "Closed-Form" Trick: Usually, learning these rules requires a slow, trial-and-error process. This paper uses a clever mathematical shortcut (Ridge Regularization) to calculate the perfect rulebook instantly, like solving a math equation on paper rather than playing a video game to figure out the physics. This makes the learning very stable and reliable.

4. The Two-Stage Training: Learning to Walk Before Running

A common problem in AI is that if you try to teach the translator and the rulebook at the same time, they might get confused and "cheat" (the translator might just memorize the answer instead of learning the language).

The Solution: The authors use a Staged Training approach.
- Stage 1: They freeze the translator and just teach the rulebook how to work with the current data.
- Stage 2: Once the rulebook is solid, they unlock the translator and fine-tune everything together.
Analogy: It's like a student first learning the grammar rules of a language (Stage 1) before trying to write a novel (Stage 2). This prevents the student from just memorizing the story without understanding the language.

What Did They Prove?

The authors tested this method on two main scenarios:

A Chaotic Pendulum: They used images of a four-link pendulum swinging. Even when the images were blurry, had missing parts (occlusion), or were noisy, DSE predicted the next frame better than other top methods (like LSTMs or standard Kalman filters).
Oscillators (Waves): They tested it on mathematical waves (Van der Pol and Stuart-Landau oscillators) with added noise. They showed that DSE could accurately recover the "spectrum" (the true frequencies and rhythms) of the system, even when the data was very noisy. Other methods got the frequencies wrong or unstable.

The Bottom Line

The paper claims that Deep Spectral Encoder is a robust way to predict complex, noisy, and partially hidden systems. By combining deep learning (to translate messy data) with classical spectral math (to find the stable rules), it creates a model that is:

Stable: It doesn't fall apart when data is noisy.
Accurate: It predicts future steps better than current state-of-the-art methods.
Interpretable: Because it uses linear operators, you can actually look at the "rulebook" and understand the system's underlying rhythms and frequencies, rather than treating the AI as a black box.

In short, it's a method that teaches a computer to ignore the noise, find the hidden rhythm, and predict the future with mathematical precision.

Technical Summary: Deep Spectral Learning of Embedded Latent Transfer Operators for Stochastic Dynamical Systems

Problem Statement
The paper addresses the challenge of forecasting and understanding stochastic nonlinear dynamical systems using time-series data. Real-world observations often exhibit non-stationarity due to nonlinear evolution, observational noise, and partial observability, rendering simple linear models ineffective. While classical approaches like sequential Bayesian filtering (e.g., Kalman filters) and spectral learning methods (e.g., subspace identification, Hidden Markov Models) exist, they face limitations. Traditional filtering often relies on iterative latent state inference which can be computationally expensive or unstable, while existing spectral methods typically depend on fixed feature representations (e.g., kernel-based embeddings) that may not capture complex nonlinear structures effectively. The authors propose a method that combines the statistical robustness of spectral learning with the representational power of deep learning, while retaining the stability of closed-form operator estimation.

Methodology: Deep Spectral Encoder (DSE)
The authors propose Deep Spectral Learning (DSL), instantiated as the Deep Spectral Encoder (DSE). This is an operator-based latent state-space model that learns embedded latent transfer operators within finite-dimensional deep feature spaces.

Operator-Theoretic Formulation:
The method models the system at the level of probability laws embedded in Hilbert spaces. It utilizes mean embeddings to represent marginal distributions and conditional covariance operators to represent linear maps between these distributions. Specifically, the latent dynamics are characterized by an embedded latent transfer operator ( $T_e$ ) and the observation mechanism by an embedded observation operator ( $O_e$ ). These operators propagate the mean embeddings of the state and observation distributions linearly, even if the underlying dynamics are nonlinear.
Stochastic Realization with Deep Features:
To construct latent state coordinates, the method employs functional Canonical Correlation Analysis (CCA) between past and future observation blocks within a learnable feature space.
- A time-invariant neural encoder maps observation windows to feature vectors.
- A shallow "head" network processes delay vectors of these features to create block-wise past and future features.
- Functional CCA is performed on these blocks to identify the predictive subspace (the minimal splitting subspace), yielding low-dimensional latent state coordinates ( $x(t)$ ).
Closed-Form Operator Estimation:
Unlike deep latent state models that learn dynamics via iterative backpropagation through time (which can lead to degenerate solutions), DSE estimates the transition and observation operators using ridge-regularized closed-form solutions.
- The operators are estimated as Galerkin projections of the conditional covariance operators onto subspaces spanned by learned neural dictionaries.
- The estimation follows a two-stage deep feature procedure: first, estimating the conditional expectation of features given the state; second, fitting the structural function (read-out maps) on these predicted features.
Staged Training Strategy:
To prevent the encoder from absorbing temporal structures (which would render the learned operators trivial), the authors employ a staged training schedule:
- Phase 1: The encoder and decoder are frozen. The dictionaries, read-out maps, and operators are optimized using alternating updates and block cross-fitting to ensure the operators learn the dynamics independently of the representation.
- Phase 2: The encoder and decoder are unfrozen for end-to-end refinement, optimizing the reconstruction loss while maintaining the causal path.
Downstream Applications:
- Sequential State Estimation: The learned operators enable a feature-space version of the Kalman filter, propagating and updating state moments linearly without explicit Gram matrix constructions.
- Spectral Mode Decomposition: The transfer operator is decomposed via eigen-decomposition to recover Koopman spectral modes, separating oscillatory frequencies from growth/decay rates.

Key Contributions

Deep Spectral Learning Framework: A novel method that learns embedded latent transfer operators in finite-dimensional deep feature spaces while preserving the operator-based state-space formulation.
Deep Spectral Encoder (DSE): A concrete architecture that strictly separates representation learning (via neural encoders) from dynamics modeling (via closed-form operator estimation).
Stable Operator Estimation: The use of ridge-regularized closed-form solutions for operator estimation, which coincides with Galerkin projections, ensures statistical stability and avoids the degeneracy often seen in joint end-to-end training of latent state models.
Generalization of Existing Methods: The framework generalizes sequential Bayesian filtering and Koopman spectral analysis to feature spaces defined by deep neural networks.

Experimental Results
The authors evaluate DSE on several scenarios involving noise and partial observability:

Quad-Link Pendulum (Image Sequences): DSE demonstrated superior performance in one-step and multi-step prediction compared to baselines including Recurrent Kalman Networks (RKN), LSTMs, and the kernel-based ELTO-Kalman filter. It showed robustness under both clean and corrupted (occlusion/distortion) regimes.
Nonlinear Oscillators (Van der Pol and Stuart-Landau): In recovering Koopman spectra under observation and process noise, DSE achieved lower mean eigenvalue estimation errors compared to Extended DMD (eDMD), Hankel DMD (hDMD), Subspace DMD (sDMD), and ELTO. DSE exhibited greater stability (lower standard deviation) than competing methods, particularly as noise levels increased.

Significance and Claims
The paper claims that DSE offers a robust alternative for modeling stochastic nonlinear dynamics by bridging the gap between the statistical rigor of spectral learning and the flexibility of deep learning.

Robustness: The method maintains stable forecasting and spectrum recovery even under significant noise and partial observability, outperforming both iterative deep learning baselines and fixed-kernel spectral methods.
Interpretability: By learning explicit finite-dimensional Galerkin coordinates and operators, the model allows for direct spectral analysis (Koopman modes) and sequential filtering in a manner that is computationally efficient and statistically well-behaved.
Modularity: The staged training strategy effectively mitigates the issue of degenerate representations where encoders absorb temporal dynamics, ensuring that the learned operators genuinely capture the system's evolution.

The authors position this work as a step toward making finite-dimensional representation spaces in spectral realization learnable, thereby enhancing the applicability of operator-theoretic methods to complex, real-world time-series data.