📊 statistics

Amortized Filtering and Smoothing with Conditional Normalizing Flows

This paper introduces AFSF, a unified amortized framework that leverages conditional normalizing flows and a recurrent encoder to efficiently approximate both filtering distributions and smoothing trajectories for high-dimensional nonlinear dynamical systems, while also enabling extrapolation and providing diagnostic capabilities through a flow-based particle filtering variant.

Original authors: Tiangang Cui, Xiaodong Feng, Chenlong Pei, Xiaoliang Wan, Tao Zhou

Published 2026-04-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Tiangang Cui, Xiaodong Feng, Chenlong Pei, Xiaoliang Wan, Tao Zhou

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to track a lost hiker in a dense, foggy forest using only a shaky, intermittent radio signal. You know the hiker moves according to certain rules (they might walk uphill, avoid rivers), but the signal is noisy, and sometimes it cuts out completely.

Filtering is guessing where the hiker is right now based on everything you've heard so far.
Smoothing is looking back after the rescue is over and drawing the most accurate map of the entire path the hiker took, using all the clues you have from start to finish.

For decades, scientists have struggled to do this for complex systems (like weather patterns or stock markets) because the math gets incredibly messy when the "forest" is huge and the rules are non-linear. Old methods either oversimplify the problem (assuming the hiker walks in a straight line) or get so computationally heavy that they crash the computer.

This paper introduces a new, smart solution called AFSF (Amortized Filtering and Smoothing with Conditional Normalizing Flows). Here is how it works, using simple analogies:

1. The Problem with Old Methods

The "Gaussian" Trap: Old methods often assume the hiker's location follows a simple bell curve. But in reality, the hiker might be stuck in a cave or climbing a cliff. The bell curve breaks down, and the prediction fails.
The "Particle" Overload: Other methods use thousands of "guesses" (particles) to track the hiker. But as the forest gets bigger (high dimensions), almost all the guesses become useless, leaving you with no idea where the hiker is.
The "One-Off" Bottleneck: New AI methods exist, but they usually have to retrain or re-optimize for every single new hiker. If you want to track a million hikers, you have to do the math a million times. That's too slow for real-time use.

2. The AFSF Solution: The "Smart Summarizer"

The authors built a system that learns the rules of the forest once and then applies them instantly to any situation.

The Core Idea: The "Travel Log" (Recurrent Encoder)
Imagine instead of listening to every single crackle of the radio for 10 hours, you have a Smart Summarizer.

As the radio signal comes in, this AI reads the history and compresses it into a single, fixed-size "Travel Log" (a summary statistic).
Whether the hiker has been walking for 1 minute or 1,000 minutes, the Travel Log is always the same size. It captures the essence of the journey without getting bogged down in the length of the tape.

The Two-Way Street (Forward and Backward Flows)
Once the AI has the Travel Log, it uses two powerful tools (called Conditional Normalizing Flows) to do the heavy lifting:

The Forward Flow (The "Now" Predictor):
- Analogy: Think of this as a GPS Navigator. It takes the current Travel Log and instantly calculates the most likely spot where the hiker is right now.
- It doesn't just guess a single point; it draws a map of probabilities, showing exactly where the hiker could be and how uncertain you are.
The Backward Flow (The "Retro-Inspector"):
- Analogy: Think of this as a Time-Travel Detective. Once the hiker is found, this tool looks at the final location and the Travel Log to figure out, "Given where they ended up, where must they have been 5 minutes ago?"
- It works backward step-by-step, refining the path.

The Magic Trick: Shared Memory
The genius of AFSF is that the Forward Flow and the Backward Flow share the same Travel Log.

Because they are both looking at the same summary of the history, they "agree" on the story.
This shared memory acts like a glue, preventing the AI from hallucinating a path that makes sense for "now" but contradicts "then." It forces the system to be consistent, making the final path reconstruction (smoothing) much more accurate.

3. Why This is a Game Changer

Amortized (One-Time Learning): The AI learns the "laws of the forest" during training. Once trained, it can process any new hiker instantly. It doesn't need to re-learn for every new radio signal. It's like learning to drive a car once, then being able to drive any car of that model instantly.
Extrapolation: The paper shows that even if the AI is trained on 500 hours of data, it can accurately predict what happens at hour 1,000. It learns the structure of the movement, not just the specific data points.
Particle Filter Upgrade: They also created a hybrid version that uses these AI tools to guide "guesses" (particles), making them smarter and less likely to get lost in the fog.

4. Real-World Tests

The authors tested this on four very different "forests":

Heat Diffusion: Tracking how heat spreads through a metal rod (a physics problem).
Stock Volatility: Tracking the hidden "fear" in the stock market based on noisy price data.
Fluid Dynamics: Tracking complex swirling fluids (Burgers' equation).
Weather Chaos: Tracking the famous "Lorenz System," which models chaotic weather patterns.

In all cases, AFSF was more accurate and stable than the current state-of-the-art methods, especially when the systems got huge and complex.

Summary

Think of AFSF as a universal, instant-replay camera for complex systems.

Old methods were like trying to reconstruct a movie by guessing frame-by-frame with a blurry lens.
This new method is like having a smart editor who watches the whole movie, creates a perfect summary of the plot, and then instantly generates the most accurate version of every scene (both the current one and the past ones) without ever needing to re-watch the whole thing from scratch.

It turns a computationally impossible nightmare into a fast, reliable, and accurate tool for understanding our chaotic world.

1. Problem Statement

The paper addresses the challenge of Bayesian filtering and smoothing for high-dimensional, nonlinear dynamical systems.

The Goal: Estimate hidden, time-varying system states ( $u_t$ ) from noisy observations ( $y_t$ ). Specifically, the goal is to approximate the filtering distribution $p(u_t | y_{1:t})$ and the smoothing distribution (entire trajectory) $p(u_{1:t} | y_{1:t})$ .
Limitations of Existing Methods:
- Gaussian Approximations (e.g., Kalman Filters): Fail when posteriors are highly nonlinear or non-Gaussian due to reliance on moment-closure assumptions.
- Sequential Monte Carlo (Particle Filters): Suffer from "particle degeneracy" in high dimensions, requiring exponentially large sample sizes to maintain accuracy, especially over long time horizons.
- Deep Generative Models: While capable of representing complex posteriors, they often treat filtering and smoothing as separate tasks or rely on costly per-instance optimization, making them unsuitable for online, large-scale applications.

2. Methodology: AFSF Framework

The authors propose AFSF, a unified, amortized framework that combines Recurrent Neural Networks (RNNs) and Conditional Normalizing Flows (CNFs).

Core Architecture

Recurrent Summary Encoder:
- Instead of conditioning on the full, variable-length observation history $y_{1:t}$ , the model uses a multi-layer LSTM to encode the history into a fixed-dimensional summary statistic $s_t$ .
- $s_t = \text{Enc}(y_{1:t}; \psi)$ .
- This ensures the conditioning dimension remains constant regardless of the time horizon, enabling scalability and extrapolation.
Dual Conditional Normalizing Flows:
The framework learns two distinct flows conditioned on the same shared summary statistic $s_t$ :
- Forward Flow ( $p_{\theta_1}(u_t | s_t)$ ): Approximates the filtering distribution $p(u_t | y_{1:t})$ . This enables real-time, online state inference.
- Backward Flow ( $p_{\theta_2}(u_t | u_{t+1}, s_t)$ ): Approximates the backward transition kernel $p(u_t | u_{t+1}, y_{1:t})$ .
- Smoothing: The full trajectory smoothing distribution is recovered by combining the terminal filtering distribution with the learned backward flows via standard backward recursion (Rauch-Tung-Striebel style).
Training Objective:
- The model is trained on simulated trajectories using a joint maximum likelihood objective.
- The loss function is a weighted sum of the negative log-likelihoods for the forward flow and the backward flow.
- Key Insight: Sharing the summary network ( $\psi$ ) between the forward and backward flows acts as an implicit regularizer. It forces the two conditional distributions to remain consistent with a unified representation of the observation history, preventing the degradation of smoothing accuracy often seen in decoupled training.
Flow-Based Particle Filtering Variant:
- The authors also develop a particle filtering variant where the learned flows serve as:
  - A proposal distribution for propagating particles ( $p(u_t | y_t, u_{t-1})$ ).
  - A predictive density for reweighting particles ( $p(y_t | u_{t-1})$ ).
- This approach reduces weight degeneracy compared to standard bootstrap filters. When explicit model factors are available, an Effective Sample Size (ESS) diagnostic is used to quantify particle degeneracy.

3. Key Contributions

Unified Amortized Framework: AFSF is the first framework to jointly learn filtering and smoothing distributions using a shared summary statistic and conditional flows, enabling coherent online filtering and trajectory-level smoothing without per-instance optimization.
Implicit Regularization via Shared Summaries: By coupling the forward and backward flows through a single summary network, the method induces consistency across latent state trajectories, significantly improving smoothing performance compared to decoupled approaches.
Extrapolation Capability: Because the model learns the underlying temporal evolution structure rather than just fitting specific trajectories, it can generalize and extrapolate beyond the training time horizon.
Flow-Based Particle Filtering: Introduces a novel particle filter variant that leverages learned flows for proposals, offering a robust alternative for online inference with built-in diagnostics (ESS) when model densities are known.

4. Numerical Results

The authors evaluated AFSF on four diverse benchmark problems:

1D Linear Advection-Diffusion:
- Compared against exact analytical solutions.
- Result: AFSF achieved high accuracy (low KL divergence) and demonstrated robustness as state dimension increased ( $n=10$ to $50$). It successfully extrapolated beyond the training horizon.
Two-Factor Stochastic Volatility Model:
- A highly nonlinear financial model.
- Result: AFSF and the flow-based particle filter significantly outperformed the state-of-the-art FBF method. The method was also tested on real S&P 500 data, successfully tracking latent volatility during market turbulence (e.g., COVID-19 outbreak).
Burgers' Equation (Nonlinear PDE):
- A high-dimensional fluid dynamics problem.
- Result: AFSF maintained slight but consistent advantages over FBF across various noise levels, demonstrating effectiveness in complex PDE-constrained data assimilation.
Lorenz Systems (Single and Two-Scale):
- Single-Scale: Outperformed FBF across dimensions $K=10$ to $50$.
- Two-Scale: Demonstrated superior flexibility in handling systems where the effective transition density is intractable (due to marginalizing fast variables), a scenario where FBF fails.
- Smoothing: The shared summary statistic was proven critical; using independent summaries led to catastrophic error accumulation in smoothing.

5. Significance

Scalability: By amortizing inference (learning a single set of parameters for all time steps and trajectories), AFSF avoids the computational bottleneck of per-instance optimization, making it suitable for large-scale, high-dimensional systems.
Robustness: The method handles strong nonlinearity, non-Gaussianity, and high dimensions where traditional filters (EnKF) and particle filters fail.
Practicality: The framework requires only a simulator for training (simulation-based inference) and does not require explicit knowledge of the transition or observation density functions, making it applicable to complex real-world systems where analytical models are unavailable.
Extrapolation: The ability to generalize beyond the training time horizon is a crucial advancement for long-term forecasting and control applications.