🤖 machine learning

Structured Noise Adaptation for Sequential Bayesian Filtering with Embedded Latent Transfer Operators

This paper proposes a novel ELTO-based Bayesian filtering approach that overcomes the limitations of simplified noise models by introducing a structured parameterization enabling both data-driven learning and dynamic adaptation to non-stationary processes, thereby improving state estimation performance in noisy, time-varying environments.

Original authors: Naichang Ke, Pongpisit Thanasutives, Yoshinobu Kawahara

Published 2026-06-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Naichang Ke, Pongpisit Thanasutives, Yoshinobu Kawahara

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to follow a friend walking through a crowded, chaotic city. Your goal is to guess exactly where they are at every second, even though you can only see them through a foggy window and the crowd keeps bumping into them, pushing them off course.

This is the problem of Sequential State Estimation. In the world of math and robotics, we use tools called Kalman Filters to do this. Think of a Kalman Filter as a very smart, predictive GPS. It has two jobs:

Predict: "My friend was walking north, so they should be a bit further north now."
Update: "Wait, I just saw them through the window, and they are actually near a coffee shop. I need to adjust my guess."

The Problem: The "Static" GPS

For a long time, these filters had a major flaw. They assumed the "noise" (the fog, the crowd, the bumps) was always the same. It was like driving a car with a GPS that assumes the traffic is always light, even when you hit a massive jam. If the traffic suddenly gets heavy (a non-stationary process), the GPS gets confused, makes bad guesses, and you lose track of your friend.

Previous methods tried to fix this by either:

Ignoring the bad data: Pretending the crowd doesn't exist (which doesn't work well).
Learning a single "average" noise model: This works if the traffic is always average, but fails if the traffic suddenly changes from light to a gridlock.

The Solution: The "Smart, Adaptable" GPS (ELTO-AKF)

The authors of this paper, Naichang Ke and colleagues, created a new version of this filter called ELTO-AKF.

Here is how it works, using a simple analogy:

1. The "Structured" Blueprint (The SB Structure)

Imagine the filter's "noise model" is a giant, complex spreadsheet of numbers that describes how messy the world is.

Old way: Trying to fill out every single cell in this massive spreadsheet is slow and prone to errors. If you get one number wrong, the whole thing breaks.
New way (ELTO-AKF): The authors invented a "Scalar-Block" (SB) structure. Instead of filling out every cell, they organize the spreadsheet into neat, smaller blocks where each block is just a simple number multiplied by a standard pattern.
- Analogy: Instead of trying to describe every single brick in a wall individually, you describe the wall as "10 rows of identical red bricks." It's much easier to manage, faster to calculate, and less likely to collapse.

2. The Two-Step Learning Process

The magic of ELTO-AKF is that it learns the noise model in two clever ways at the same time:

Step A: The "Long-Term Memory" (Data-Driven Learning)
Before the filter starts tracking, it looks at a lot of past data to learn the general rules of the game. It figures out, "On average, how much does the crowd usually push people?" It finds the best "static" settings for its spreadsheet.
- Metaphor: This is like studying a map of the city to learn the usual traffic patterns before you even leave the house.
Step B: The "Real-Time Reflex" (Dynamic Adaptation)
Once the filter is running, it doesn't just stick to the map. It watches for sudden changes. If your friend suddenly stops or runs, the filter notices the "error" (the difference between where it thought they were and where it saw them). It instantly tweaks its settings to handle this new, sudden chaos.
- Metaphor: This is like a driver who sees a sudden roadblock and immediately swerves, rather than sticking to the pre-planned route.

Why This Matters (The Results)

The paper tested this new filter in three main scenarios:

The Swinging Pendulum: They simulated a pendulum swinging in a noisy room. When the noise got very high, old filters failed or got confused. ELTO-AKF kept tracking the pendulum accurately because it could adapt to the sudden noise spikes.
The LiDAR Tracker: They simulated tracking a car moving through a city with changing speeds and turns. When the car suddenly changed its behavior, ELTO-AKF adjusted instantly, while the old filters lagged behind or lost the target.
The "Shock Wave" Test (Burgers' Equation): This is a complex math problem involving fluids that can suddenly form "shock waves" (sudden, violent changes).
- The Result: When they used ELTO-AKF to clean up the noisy data, they were able to correctly discover the mathematical equation governing the fluid. Without the filter, the data was too messy to figure out the equation. With the filter, they found the correct formula, even when the fluid behavior changed abruptly.

The Bottom Line

The paper claims that by organizing the "noise" into a smart, block-like structure and allowing the filter to learn both long-term patterns and short-term surprises, ELTO-AKF is much better at tracking things in messy, changing environments.

It doesn't just guess; it learns the rules of the chaos and then adapts its guesses the moment the chaos changes. This makes it a powerful tool for anything from robotics to understanding complex natural systems, provided the data is noisy and unpredictable.

Technical Summary: Structured Noise Adaptation for Sequential Bayesian Filtering with Embedded Latent Transfer Operators

Problem Statement
Sequential state estimation, particularly via Kalman filtering, is fundamental to applications in robotics, navigation, and financial modeling. However, a critical limitation persists in existing approaches: the reliance on simplified, fixed noise models that fail to adapt to non-stationary processes. While methods like the Kernel Kalman Rule (KKR) and Embedded Latent Transfer Operators (ELTO) have enabled data-driven state estimation in Reproducing Kernel Hilbert Spaces (RKHS) by bypassing explicit parametric assumptions, they typically inherit the limitation of fixed noise covariances. Optimizing these noise models is challenging; suboptimal covariances severely degrade performance. Current solutions are often bifurcated: model-based approaches (e.g., Adaptive Extended Kalman Filters) dynamically tune noise using residuals but rely on parametric assumptions, while data-driven approaches learn time-invariant parameters but lack mechanisms to handle abrupt dynamic shifts. There remains a gap in constructing a unified approach that leverages both parameter learning strategies to improve filtering in high-dimensional, non-stationary contexts.

Methodology
The authors propose ELTO-AKF (ELTO-based Adaptive Kalman Filtering), a novel method that integrates a new structured parameterization for noise models into the ELTO framework. The methodology consists of three core components:

Sparse Structure Parameterization (Scalar-Block): To facilitate tractable learning of noise covariance matrices ( $Q$ and $R$ ) while ensuring numerical stability, the authors introduce a Scalar-Block (SB) structure. In this structure, the covariance matrix is partitioned into $k \times k$ sub-blocks, where each sub-block is a scaled identity matrix. To guarantee the Symmetric Positive-Definite (SPD) property required for valid covariance matrices, the SB structure is parameterized via Cholesky decomposition ( $A = LL^\top$ ), where $L$ is a block lower-triangular matrix with scaled identity sub-blocks. This reduces computational complexity compared to full-rank matrices and ensures stability during updates.
Data-Driven Optimization (Time-Invariant Learning): The method employs a derivative-free optimization strategy (CMA-ES) to learn an optimal, time-invariant noise model. The objective is to minimize the Mean Squared Error (MSE) between the filter's pre-image estimates and a validation dataset. This process identifies a robust global noise model that accounts for the underlying system dynamics without requiring explicit probability density functions.
Dynamic Parameter Adaptation (Time-Variant Tracking): To address non-stationarity where noise levels or dynamics change abruptly, the method incorporates dynamic adaptation mechanisms inspired by the Adaptive Extended Kalman Filter (AEKF). The algorithm tracks the innovation (prediction error) and residual (update error) to estimate process and measurement noise covariances. Crucially, these updates are performed on the Cholesky factors ( $L$ and $M$ ) of the noise matrices rather than the matrices directly. A sparse projection operator enforces the SB structure and rectifies eigenvalues to maintain the SPD property. Memory factors ( $\alpha, \beta$ ) control the adaptation rate, allowing the filter to balance between tracking sudden changes and maintaining overall system trends.

Key Contributions

Structured Parameterization: The introduction of a Scalar-Block structure for noise covariance matrices, which provides a computationally tractable representation while guaranteeing the SPD property through Cholesky factorization.
ELTO-AKF Algorithm: A unified filtering method that couples the data-driven learning of a robust, time-invariant noise model with dynamic parameter adaptation. This addresses the research gap between static data-driven optimization and dynamic model-based adaptation.
Robustness in Non-Stationary Environments: The demonstration that structured noise adaptation significantly improves dynamic state estimation performance in noisy, time-varying environments, outperforming baselines that rely on fixed covariances or unstructured adaptive methods.

Experimental Results
The authors evaluated ELTO-AKF across four distinct scenarios:

Nonlinear Pendulum Denoising: In simulations with varying process and observation noise, ELTO-AKF outperformed model-based AEKFs, sampling-based filters (SPKF), and neural-aided filters (CKFNet), particularly under high noise conditions where other methods suffered from numerical divergence or fixed prior limitations.
Non-Stationary LiDAR Trajectories: On synthetic LiDAR data with piecewise non-stationary dynamics (alternating straight motion and turns), ELTO-AKF demonstrated superior performance in both "Full Optimization" (tuned initial parameters) and "Pure Adaptation" (relying solely on dynamic updates) settings. It also showed better generalization to unseen noise levels compared to the standard ELTO-KF.
High-Dimensional Lorenz-96 Systems: Experiments on chaotic systems with dimensions up to $D=1000$ revealed that while the standard ELTO-KF often diverged due to the loss of the SPD property in high-noise, high-dimensional settings, ELTO-AKF remained numerically stable and competitive.
Downstream Applications:
- PDE Discovery: When applied to denoise data for discovering governing Partial Differential Equations (specifically the viscous Burgers' equation with shock waves), ELTO-AKF significantly reduced identification errors compared to un-denoised data or standard ELTO-KF.
- Ecological Time Series: On real-world Canadian lynx and snowshoe hare data, the method successfully smoothed observations to facilitate the discovery of Lotka-Volterra dynamics, outperforming the baseline in recovering original observations from doubly noisy inputs.

Significance and Claims
The paper claims that ELTO-AKF represents a significant advancement in sequential Bayesian filtering by enabling structured noise adaptation. By integrating data-driven parameter optimization with dynamic adaptation, the method effectively handles the non-stationarity of real-world processes where traditional filters fail. The authors emphasize that their approach provides robust covariance representations for noisy, non-stationary processes without requiring ground truth latent states or explicit likelihood functions.

The authors modestly acknowledge limitations, noting that the current RKHS formulation bypasses explicit probability density functions, rendering gradient-based likelihood methods (like Maximum Marginal Likelihood Estimation) inapplicable. Consequently, the method relies on derivative-free optimization and predefined memory factors for adaptation. Future work is suggested to explore structured likelihood approximations within the RKHS to enable principled gradient-based optimization and fully adaptive strategies for memory factors.