📊 statistics

Nonlinear filtering based on density approximation and deep BSDE prediction

This paper introduces a novel offline-trained, online-applicable Bayesian filter that leverages deep backward stochastic differential equations and neural networks to approximate filtering densities via a nonlinear Feynman–Kac representation, with its theoretical convergence and error bounds validated under a parabolic Hörmander condition and numerical examples.

Original authors: Kasper Bågmark, Adam Andersson, Stig Larsson

Published 2026-04-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Kasper Bågmark, Adam Andersson, Stig Larsson

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to track a lost hiker in a dense, foggy forest. You can't see them directly, but every few minutes, a drone flies overhead and sends you a blurry, noisy photo of where they might be. Your goal is to figure out exactly where the hiker is at any given moment, despite the fog and the bad photos.

This is the Filtering Problem. It's a classic challenge in math and engineering used for everything from self-driving cars to tracking disease spread.

Here is how this paper solves that problem, explained simply:

1. The Old Ways: Too Simple or Too Slow

Traditionally, there are two main ways to solve this:

The "Gaussian" Way (Kalman Filter): This assumes the hiker moves in a straight line and the photos are perfect. It's fast and easy, but if the hiker starts running in circles or the photos are very blurry, this method fails miserably.
The "Guess-and-Check" Way (Particle Filter): This method throws thousands of imaginary hikers into the forest, sees which ones match the photos, and keeps the best ones. It works for complex situations, but if the forest is huge (high-dimensional), you need so many imaginary hikers that your computer crashes. This is called the "Curse of Dimensionality."

2. The New Idea: The "Deep Learning" Detective

The authors of this paper propose a new, hybrid method that combines the best of both worlds. They use Deep Learning (AI) to act as a super-smart detective.

Here is the step-by-step analogy of their method:

Step A: The Prediction (The "Foggy Walk")

Between photos, the hiker is moving through the fog. Mathematically, this is described by a complex equation (the Fokker-Planck equation) that predicts how the "cloud of possible locations" spreads out.

The Problem: Solving this equation for a high-dimensional forest is like trying to paint a 100-dimensional picture on a 2D canvas. It's impossible with old tools.
The Solution: The authors use a trick called Deep BSDE. Imagine instead of trying to paint the whole picture at once, you send a swarm of tiny, invisible drones (simulated paths) through the forest. You train a neural network (an AI brain) to learn the pattern of where the hiker is likely to be based on where these drones go. This AI learns to predict the "fog" without needing a grid or a map.

Step B: The Update (The "Photo Moment")

When the drone sends a new photo, you have to update your guess.

The Trick: This part is actually easy! You just look at the photo, see where the hiker appears, and adjust your AI's prediction to match. This is done instantly using a simple formula (Bayes' rule).

Step C: The Loop

The process repeats: Predict (using the AI) $\rightarrow$ Update (using the photo) $\rightarrow$ Predict $\rightarrow$ Update.

3. Why is this paper special?

The authors didn't just build the tool; they proved it works mathematically.

The "Training" Phase: Before the system can track the hiker in real-time, it needs to be trained. The authors show how to train the AI "offline" (in a computer lab) using simulated data. Once trained, the AI is ready to work "online" (in real-time) with new, real photos.
The Proof: They proved a mathematical "safety net." They showed that if you make your time steps smaller (check the drone more often), your answer gets closer and closer to the truth. Specifically, they proved the error shrinks at a predictable rate (like $\sqrt{\text{time step}}$ ).
The Test: They tested this on two scenarios:
1. A Linear Forest: Where the hiker moves predictably. The new method matched the perfect solution.
2. A Chaotic Forest: Where the hiker moves in a complex, non-linear way (like a "bistable" process, bouncing between two valleys). Even here, the method worked well, converging to the right answer as they increased the computer power.

The Big Picture Metaphor

Think of the old methods as trying to navigate a maze:

Kalman Filter is like assuming the maze is a straight hallway. It works great until you hit a turn.
Particle Filters are like sending 10,000 people into the maze to find the exit. It works, but it's exhausting and slow.
This New Method is like hiring a Genius AI Guide. You train the guide on thousands of practice mazes (offline). Once trained, the guide can instantly tell you the best path through any new maze, even a massive, twisting one, just by looking at a few clues.

Summary

This paper introduces a new way to track moving objects in complex, noisy environments. It uses Artificial Intelligence to solve the hardest part of the math (predicting movement in high dimensions) and proves that this AI approach is mathematically sound. It opens the door to tracking things in 100-dimensional spaces (like complex financial markets or weather systems) where previous methods used to fail.

1. Problem Statement

The paper addresses the Bayesian filtering problem for hybrid systems where the state evolves continuously in time via a Stochastic Differential Equation (SDE), while observations are made at discrete time points.

System Dynamics: The state $S_t$ follows an SDE:
$dS_t = \mu(S_t)dt + \sigma(S_t)dB_t$
Observations: Discrete measurements $O_k$ are taken at times $t_k$ :
$O_k = h(S_{t_k}) + V_k$
where $V_k$ is additive noise.
Goal: Compute the conditional probability density $p(S_{t_k} | O_{1:k})$ of the latent state given the history of observations.
Challenges:
- Nonlinearity & Non-Gaussianity: Classical methods like the Kalman Filter fail when dynamics or noise are non-Gaussian.
- Curse of Dimensionality: Particle filters (Sequential Monte Carlo) and grid-based PDE solvers (Fokker-Planck) become computationally intractable as the state dimension $d$ increases due to sample degeneracy or exponential grid growth.

2. Methodology

The authors propose a novel recursive filter that combines density approximation with the Deep Backward Stochastic Differential Equation (Deep BSDE) method. The approach transforms the filtering problem into a sequence of optimization problems solvable by neural networks.

A. Recursive Formulation

The filtering process is decomposed into alternating steps:

Prediction Step: Propagates the density forward in time using the Fokker-Planck PDE.
Update Step: Incorporates new observations using Bayes' rule (multiplying the predicted density by the likelihood function).

B. Nonlinear Feynman-Kac Representation

To avoid solving the high-dimensional Fokker-Planck PDE directly, the authors leverage the connection between parabolic PDEs and BSDEs.

They introduce an auxiliary drift $b$ and a process $X$ satisfying an SDE.
They derive a Nonlinear Feynman-Kac formula (Theorem 2.1) which represents the solution to the prediction PDE as the solution to a Backward Stochastic Differential Equation (BSDE).
Crucially, they select a specific auxiliary drift $b$ (Eq. 10) such that the driver function $f$ in the BSDE depends only on the state $x$ and the solution $u$ , simplifying the numerical approximation.

C. Deep BSDE Optimization

The prediction step is reformulated as an optimization problem:

The goal is to find a function $w$ (representing the density at the start of the interval) that minimizes the mean squared error between the terminal condition of the BSDE and the target function derived from the previous step's density and the current likelihood.
Neural Network Approximation: The unknown functions in the BSDE (the solution $Y$ and the gradient $Z$ ) are approximated by neural networks ( $w_\theta$ and $v_\theta$ ).
Training: The networks are trained offline using Monte Carlo sampling of the underlying SDE trajectories. The loss function is the squared difference between the BSDE terminal value and the target.
Hybrid Scheme: The method is recursive. The output of the Deep BSDE solver at time $t_k$ serves as the input (initial condition) for the update step, which is applied in closed form, and then feeds into the next Deep BSDE prediction step for $t_{k+1}$ .

3. Key Contributions

Novel Filter Architecture: Proposes a density-based recursive filter that utilizes the Deep BSDE method for the prediction step and closed-form updates for observations. This is specifically designed to handle high-dimensional, nonlinear, and non-Gaussian settings.
Theoretical Error Bounds: Proves a hybrid a priori-a posteriori error bound (Theorem 3.1).
- The bound depends on the time discretization step ( $\tau^{1/2}$ ) and the residual error of the optimization (the difference between the learned solution and the theoretical optimum).
- The proof relies on the parabolic Hörmander condition (a hypoellipticity condition ensuring the diffusion generates a smooth density) and smooth boundedness of coefficients.
- This extends previous convergence results for Deep BSDEs to a setting involving a sequence of uncoupled forward-backward systems.
Convergence Verification: Demonstrates theoretically and empirically that the method achieves a convergence rate of order $1/2$ with respect to the time discretization, consistent with the Euler-Maruyama approximation used in the BSDE solver.

4. Numerical Results

The authors validate the method on two 1D examples (to verify convergence rates) and reference a separate study [4] for high-dimensional scalability (up to 100 dimensions).

Example 1: Ornstein-Uhlenbeck Process (Linear):
- The system is linear with Gaussian noise. The exact solution is known (Kalman Filter).
- Result: The numerical error decreases as the time discretization $N$ increases. The convergence rate matches the theoretical $O(N^{-1/2})$ .
Example 2: Bistable Process (Nonlinear):
- The system has a cubic drift (double-well potential), leading to a bimodal distribution. No closed-form solution exists; a Particle Filter serves as the reference.
- Result: Despite the violation of strict smoothness assumptions (cubic drift), the method converges with order $1/2$ until limited by optimization saturation (network capacity or Monte Carlo noise).
Observation: For coarse discretizations, error accumulates over time. For fine discretizations, the error stabilizes, indicating the method successfully mitigates the curse of dimensionality in the tested regimes.

5. Significance and Outlook

High-Dimensional Capability: Unlike traditional particle filters which suffer from sample degeneracy in high dimensions, this method scales effectively (demonstrated in [4] up to $d=100$ ) because it relies on neural network approximation rather than particle ensembles.
Offline Training, Online Application: The heavy computational cost (training the neural networks) is performed offline. Once trained, the filter can be applied online to new observation streams with low latency.
Theoretical Rigor: The paper provides a rare rigorous convergence analysis for deep learning-based filtering, establishing a solid theoretical foundation for using Deep BSDEs in Bayesian inference.
Future Work: The authors suggest extending the method to more complex observation models and further investigating the stability of the method under weaker regularity conditions.

In summary, this paper presents a robust, theoretically grounded framework for solving high-dimensional nonlinear filtering problems by bridging stochastic analysis (BSDEs) and deep learning, offering a viable alternative to particle filters in regimes where dimensionality renders traditional methods ineffective.