⚡ electrical engineering

The Variational Approach in Filtering and Correlated Noise

This paper demonstrates that the standard variational formulation of nonlinear filtering fails when signal and observation noises are correlated due to mutual singularity of measures, and proposes a generalized conditional variational principle that preserves noise correlations to recover the original formulation for independent noises while providing a new free energy characterization for the linear correlated-noise case.

Original authors: Sharan Srinivasan, Vijay Gupta, Harsha Honnappa

Published 2026-04-06

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Sharan Srinivasan, Vijay Gupta, Harsha Honnappa

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to track a lost hiker (the Signal) in a dense forest using a shaky, noisy radio signal (the Observation). Your goal is to guess where the hiker is at any given moment based on the static and garbled words you hear.

In the world of mathematics and engineering, this is called Filtering. For decades, mathematicians had a brilliant, elegant recipe for solving this problem when the hiker's movements and the radio noise were completely unrelated. This recipe, created by Mitter and Newton, was like a perfect map: it said, "The most likely path the hiker took is the one that minimizes a specific 'cost' (a mix of how surprising the path is and how well it fits the radio data)."

The Problem: The "Shared Noise" Trap
However, real life is messy. Often, the hiker and the radio share a common problem. Maybe a storm is blowing through the forest, causing the hiker to stumble and the radio to crackle at the same time. In math terms, the Signal and the Observation share a "common noise source."

The authors of this paper, Sharan Srinivasan, Vijay Gupta, and Harsha Honnappa, discovered a shocking truth: The old recipe completely breaks down when there is shared noise.

Here is why, using a simple analogy:

The Old Recipe (Independent Noise): Imagine the hiker walks randomly, and the radio static is random. You can treat them as two separate stories. You can say, "Given the radio noise, what is the chance the hiker is here?" easily.
The New Reality (Shared Noise): Now, imagine the storm (the shared noise) pushes the hiker and jams the radio simultaneously. The hiker's path and the radio signal are now tightly locked together. If you try to separate them into two independent stories (as the old recipe requires), the math explodes. It's like trying to describe a dance by looking at the dancer and the music separately, when they are actually moving in perfect, inseparable sync. The "cost" calculation becomes infinite or undefined because the two paths are so entangled that they are mathematically "mutually singular"—they live in different universes that don't overlap in the way the old formula expects.

The Discovery: Why the Old Map Fails
The paper proves that whenever the signal and observation share a noise source, the mathematical condition required for the old "Free Energy" formula to work is impossible to satisfy. No matter how cleverly you try to adjust the reference point, the math says: "You can't do this." The old method doesn't just give a slightly wrong answer; it gives no answer at all.

The Solution: A New, Smarter Map
But don't worry! The authors didn't just point out a hole in the road; they built a bridge across it.

They introduced a Conditional Variational Principle.

The Old Way: "Here is the hiker's general history (the Prior). Now, adjust it based on the radio."
The New Way: "The hiker's history depends on the radio noise right now. Let's create a Reference Map that already includes the shared storm."

Instead of starting with a generic map of where the hiker might be, they start with a map that assumes the hiker and the radio are already dancing to the same stormy tune. They then calculate the "energy" or "cost" of the hiker's specific movements relative to this new, shared map.

The Result
This new approach:

Fixes the broken math: It works perfectly for the "shared noise" scenario where the old method failed.
Keeps the old magic: If you turn off the shared noise (make the storm stop), this new method automatically simplifies back into the old, famous Mitter-Newton formula. It's a "super-version" of the old map.
Gives a clear picture: In simple cases (like linear movements), they can write down an exact formula for the best guess of where the hiker is.

In a Nutshell
Think of the old method as trying to predict a couple's date by looking at their individual schedules separately. It works if they are strangers. But if they are a couple sharing a life (shared noise), you have to look at their joint schedule.

This paper says: "The old way of looking at them separately is mathematically impossible when they are a couple. But here is a new way to look at their shared schedule that gives you the perfect answer, and it still works if they happen to be strangers."

They have successfully updated the toolkit for tracking systems in the real world, where noise and signals are often inextricably linked.

1. Problem Statement

The paper addresses a fundamental limitation in the variational formulation of nonlinear filtering, specifically the framework proposed by Mitter and Newton [9].

The Mitter-Newton Framework: This approach characterizes the filtering distribution (the conditional distribution of a signal $X$ given observations $Y$ ) as the unique minimizer of a free energy functional. This functional is the sum of the relative entropy (with respect to a prior) and an expected energy term.
The Critical Assumption: The validity of this framework relies on Assumption 2.1: The joint path measure $P_{XY}$ must be absolutely continuous with respect to a product measure $P_X \otimes \lambda_Y$ (where $P_X$ is the prior and $\lambda_Y$ is a reference measure for the observation).
The Failure Case: This assumption holds when the signal and observation are driven by independent Brownian motions (allowing the use of the Cameron-Martin-Girsanov theorem). However, the paper investigates the scenario where the signal and observation share a common noise source (correlated noise).
Core Question: Does the Mitter-Newton variational formulation hold when signal and observation diffusions share a common noise source? If not, can it be salvaged, and how can a variational principle be recovered?

2. Methodology

The authors employ a combination of measure-theoretic probability, stochastic calculus, and variational analysis to address the problem.

Discrete-Time Intuition: They first analyze a discrete-time Gaussian system with correlated noise to demonstrate that the Radon-Nikodym derivative between the joint measure and the product measure degenerates (blows up or vanishes) in the continuum limit, suggesting mutual singularity.
Measure-Theoretic Proofs:
- They prove a general proposition (Proposition 4.1) stating that if $P_{XY} \ll P_X \otimes \lambda_Y$ for any $\sigma$ -finite $\lambda_Y$ , then $P_{XY} \ll P_X \otimes P_Y$ .
- They utilize the Feldman-Hájek theorem for the linear Gaussian case to show that Gaussian measures on infinite-dimensional path spaces are either equivalent or mutually singular. By comparing Cameron-Martin spaces, they prove singularity.
- For the general nonlinear case, they construct an explicit set with full measure under the joint distribution but null measure under the product distribution, using quadratic variation arguments and the Hörmander condition to ensure density existence.
Conditional Variational Principle: Recognizing that the standard prior-based approach fails, they propose a new formulation. Instead of comparing the posterior to a static prior $P_X$ , they compare it to a conditional reference measure $Q_{X|Y}$ that preserves the noise correlation structure.

3. Key Contributions

A. Proof of Failure for Correlated Noise

The paper rigorously proves that the Mitter-Newton assumption fails fundamentally when signal and observation share noise:

Mutual Singularity: The joint measure $P_{XY}$ and the product measure $P_X \otimes P_Y$ are mutually singular ( $P_{XY} \perp P_X \otimes P_Y$ ).
No Salvage Possible: There exists no choice of reference measure $\lambda_Y$ such that $P_{XY} \ll P_X \otimes \lambda_Y$ .
Mechanism: Conditioning on the observation path $Y=y$ in a correlated noise setting collapses the support of the signal distribution relative to the prior. Specifically, the posterior $P_{X|Y}(\cdot, y)$ is singular with respect to the prior $P_X$ , making the standard Gibbs representation $dP_{X|Y} \propto e^{-H} dP_X$ impossible.

B. Development of a Conditional Variational Principle

The authors introduce a generalized variational formulation that resolves the singularity issue:

New Assumption (6.1): Instead of requiring absolute continuity with respect to a product measure, they require the posterior $P_{X|Y}(\cdot, y)$ to be absolutely continuous with respect to a conditional reference measure $Q_{X|Y}(\cdot, y)$ for almost every $y$ .
The Reference Measure: The reference measure $Q$ is chosen to preserve the noise coupling structure. In the linear case, this is the law of the signal process driven by the same noise path as the observation but with drift terms removed.
Free Energy Characterization: The filtering distribution is shown to be the unique minimizer of a free energy functional:
$\min_{\tilde{P}} \left\{ h(\tilde{P} \mid Q_{X|Y}(\cdot, y)) + \langle H(\cdot, y), \tilde{P} \rangle \right\}$
where $h$ is the relative entropy with respect to the conditional reference, and $H$ is an energy term derived from the drift difference.

C. Explicit Solution for Linear Correlated Noise

For linear Gaussian systems with correlated noise, the authors derive an explicit form for the variational principle:

Reference Measure ( $\mu_y$ ): A Gaussian measure where the signal follows the noise path of the observation ( $W_t = y_t$ ) but has no drift.
Energy Function ( $H$ ): A functional involving the drift correction term $(A - \sigma_1 C)X_t$ and the Girsanov density relative to the "private" noise $B$ .
Consistency: They demonstrate that when the correlation vanishes ( $\sigma_1 = 0$ ), this new formulation strictly reduces to the original Mitter-Newton formulation.

4. Key Results

Theorem 5.4 (Linear Case): For linear Gaussian diffusions with correlated noise, $P_{XY}$ and $P_X \otimes P_Y$ are mutually singular because their Cameron-Martin spaces differ.
Theorem 5.6 (Nonlinear Case): Under the Hörmander condition, the joint and product measures are mutually singular for general nonlinear diffusions with correlated noise.
Theorem 6.4 (Generalized Variational Principle): Under the new conditional assumption, the filtering measure is the unique minimizer of the conditional free energy.
Proposition 6.6: Provides the explicit Radon-Nikodym derivative and energy functional for the linear correlated-noise case, yielding a concrete free energy characterization.

5. Significance and Implications

Theoretical Correction: The paper corrects a potential misconception that the Mitter-Newton variational approach is universally applicable. It establishes that the independence of noise sources is not merely a technical convenience but a necessary condition for the standard formulation.
New Framework for Correlated Systems: By shifting from a static prior to a conditional reference measure, the authors provide a rigorous variational tool for systems where signal and observation are coupled by common noise (a common scenario in finance, physics, and engineering).
Unification: The new framework strictly generalizes the existing theory. It contains the independent noise case as a special sub-case, ensuring continuity with established results while extending applicability.
Future Directions: The paper identifies the extension of this conditional variational principle to the general nonlinear setting as an open problem, suggesting that while the linear case is solved, the computational and theoretical challenges for nonlinear correlated noise remain.

In summary, this paper fundamentally redefines the variational approach to filtering for correlated noise, proving the failure of the classical product-measure assumption and successfully constructing a robust alternative based on conditional reference measures that respect the underlying noise structure.