From the Linear Quadratic Regulator (LQR) to the (Deterministic) Kalman Filter in Two Easy Steps

This paper presents a two-step tutorial demonstrating how to derive the deterministic Kalman filter from the Linear Quadratic Regulator (LQR) by first converting the state estimation problem into a purely quadratic LQR formulation using homogeneous coordinates, and then partitioning the resulting solution to recover the traditional Kalman filter dynamics and Riccati equation.

The Gist

Imagine you are trying to figure out exactly where a lost hiker is in a dense forest. You have two sources of information, but both are flawed:

1. Your Map (The Model): You know the hiker's general path and speed, but the terrain is tricky, and they might stumble or take a detour.
2. Your Binoculars (The Measurements): You can see the hiker occasionally, but the trees block your view, and the image is blurry.

The Kalman Filter is the mathematical tool that combines these two imperfect sources to guess the hiker's true location. Usually, this is taught as a complex statistical problem involving "noise" and "probability."

This paper by Bassam Bamieh offers a different, simpler way to look at it. It argues that you don't need to think about random chance at all. Instead, you can treat this as a deterministic puzzle: "What is the simplest possible story that explains what we saw?"

Here is the paper's "Two Easy Steps" to solve this puzzle, explained with everyday analogies.

The Core Idea: "Occam's Razor" for Math

The paper starts with a principle called the Minimal Uncertainty Principle. Imagine you are a detective trying to reconstruct a crime scene. There are infinite ways the crime could have happened.

• Story A: The suspect ran 5 miles, tripped 10 times, and the witness was hallucinating.
• Story B: The suspect walked 1 mile, stumbled once, and the witness was slightly blurry-eyed.

The paper says: Choose Story B. Why? Because it requires the least amount of "weirdness" (uncertainty) to make the facts fit. In math terms, we want the story where the "errors" (the stumbling and the blurry vision) are as small as possible.

Step 1: The "Homogeneous Coordinates" Trick

The first hurdle is that the math for this "simplest story" problem is messy. It has a mix of squared terms (like "distance squared") and straight-line terms (like "distance"). It's like trying to bake a cake where the recipe calls for "2 cups of flour" and "a pinch of salt," but the mixing bowl only accepts ingredients in a specific "squared" format.

The Solution: The paper suggests a magic trick called Homogeneous Coordinates.

• The Analogy: Imagine you have a 2D drawing on a piece of paper. To make the math work, you add a third dimension—a "1" attached to the side of your drawing. Suddenly, your 2D problem becomes a 3D problem where everything fits perfectly into a neat, symmetrical box.
• What it does: By adding this extra "1" to the system, the messy "mixed" math problem transforms into a perfectly clean, purely "squared" math problem.
• The Result: This clean problem is exactly the same as a Linear Quadratic Regulator (LQR). If you know how to solve an LQR problem (which is like finding the most fuel-efficient way to drive a car), you can now solve this messy estimation problem.

Why this matters: The paper points out a cool insight here. In control problems (like driving a car), the "extra" math usually represents a pre-planned feedforward signal. In estimation problems (like tracking the hiker), that same "extra" math represents the observer—the part of the system that learns and updates its guess over time.

Step 2: The "Time Reversal" and "Final Guess"

Now that we have a clean, squared problem, we need to solve it. But there's a catch: In a standard driving problem, you know where you started. In this estimation problem, we don't know where the hiker started. We only know where they are now (or rather, we are trying to figure out where they are now based on past data).

The Solution: The paper uses a clever two-part maneuver:

1. Assume the End: Pretend for a moment that you do know where the hiker ended up at the final moment. If you know the start and the end, the "simplest path" between them is easy to calculate.
2. Time Reversal: The math for "starting at A and ending at B" is the mirror image of "starting at B and ending at A." The paper flips the problem in time. Instead of asking "How do we get from start to finish?", it asks "If we are at the finish, how did we get here?"
3. Optimize the Guess: Since we don't actually know the final position, we take the answer from step 2 and ask: "Which final position makes the total 'weirdness' (uncertainty) the smallest?"

The Result: When you do this optimization, the messy equations magically simplify into the famous Kalman Filter equations.

• The "Observer Gain" (how much you trust the map vs. the binoculars) pops out naturally.
• The "Riccati Equation" (the complex math that updates the filter) appears as the solution to this "cost-to-arrive" problem.

The Big Picture: Certainty vs. Information

The paper concludes with a fascinating re-interpretation of the math.

• In the traditional (stochastic) view, the filter calculates a "Covariance Matrix," which tells you how uncertain you are. A big number means "I have no idea."
• In this paper's view, the math calculates an "Information Matrix" (or "Certainty Matrix").
  • The Analogy: Think of a bowl. If the bowl is very steep and deep, a marble placed inside will roll quickly to the bottom. This means you are very certain about the bottom's location. If the bowl is flat, the marble can roll anywhere; you are uncertain.
  • The paper argues that the matrix $S$ in their equations measures the steepness of the bowl. A large $S$ means the "bowl" is steep, meaning the filter is very confident in its estimate.

Summary

This paper doesn't invent a new filter; it rewrites the recipe.

1. It says: "Stop thinking about random noise. Think about finding the simplest, least-error explanation for your data."
2. It uses a mathematical trick (homogeneous coordinates) to turn a messy problem into a clean, standard control problem.
3. It uses time reversal to solve that problem, revealing that the Kalman Filter is just the optimal way to minimize uncertainty in a deterministic world.

It's a "tutorial" that strips away the scary probability theory to show that the Kalman Filter is fundamentally about efficiency and simplicity: choosing the path that requires the fewest assumptions.

Technical Summary

Technical Summary: From LQR to the Deterministic Kalman Filter

Problem Formulation
The paper addresses the deterministic state estimation problem for linear time-varying systems. The system is modeled by the equations $\dot{x}(t) = Ax(t) + w(t)$ and $y(t) = Cx(t) + v(t)$, where the output $y(t)$ is known, but the process disturbance $w(t)$, measurement noise $v(t)$, and initial state $x_i$ are unknown. The objective is to find the state trajectory $\hat{x}(t)$ consistent with the system dynamics that minimizes a quadratic cost function representing the "size" of the uncertainty triple $(w, v, x_i)$. This cost functional, $J$, is affine-quadratic in the state and inputs due to the presence of the known measurement signal $y(t)$ within the quadratic term $(y - C\hat{x})^*V(y - C\hat{x})$. The paper frames this as an "input design" problem rather than a stochastic estimation problem, adhering to a "Minimal Uncertainty Principle" analogous to Occam's razor: select the trajectory requiring the least assumptions (smallest uncertainty norm).

Methodology: The "Two Easy Steps"
The author derives the Kalman filter equations through a two-step transformation of the affine-quadratic optimization problem into a standard Linear Quadratic Regulator (LQR) framework:

1. Homogenization via Homogeneous Coordinates:
   The first step converts the affine-quadratic cost (containing quadratic, linear, and constant terms) into a purely quadratic cost. This is achieved by embedding the system into a higher-dimensional state space using "homogeneous coordinates." An auxiliary scalar state $\alpha$ is appended to the state vector $x$, constrained such that $\alpha(t) \equiv 1$. This transforms the original system and cost into a larger system with state $\xi = [x^T, 1]^T$ and a purely quadratic objective. This embedding reveals that controllers for affine-quadratic problems inherently contain dynamical components (unlike memoryless purely quadratic controllers), which correspond to the feedforward dynamics in tracking or the observer dynamics in estimation.

2. Time Reversal and Final-State Optimization:
   The second step utilizes the "LQR with final conditions" formulation. Unlike the standard LQR which specifies an initial state and minimizes a "cost-to-go," this dual problem specifies a final state and minimizes a "cost-to-arrive."

   • The estimation problem is first solved assuming the final state $\hat{x}(t)$ is known (fixed). This yields a solution characterized by a matrix Differential Riccati Equation (DRE) running forward in time, denoted as $S(t)$, and an auxiliary vector $s_1(t)$.
   • Since the final state is actually unknown, the optimal estimate is found by further minimizing the resulting "cost-to-arrive" function with respect to the final state variable. This optimization yields the optimal state estimate $\hat{x}(t) = -S^{-1}(t)s_1(t)$.
   • By differentiating this relationship and substituting the dynamics of $S(t)$ and $s_1(t)$, the paper derives a differential equation for $\hat{x}(t)$ directly. This equation takes the form of a causal observer: $\dot{\hat{x}} = A\hat{x} + L(y - C\hat{x})$, where the gain $L$ is derived from the solution $S(t)$.

Key Contributions and Results

• Derivation of the Deterministic Kalman Filter: The paper provides a streamlined derivation of the deterministic Kalman filter (state estimator) by explicitly disentangling the steps of time reversal, homogeneous coordinate embedding, and final-state optimization.
• Connection to LQ-Tracking: The methodology demonstrates a structural equivalence between the deterministic estimation problem and the Linear-Quadratic (LQ) tracking (servomechanism) problem. In LQ-tracking, the auxiliary dynamics provide the anti-causal feedforward term; in estimation, they provide the causal observer dynamics.
• Information Filter Formulation: The resulting estimator is presented in the "information filter" form. The matrix $S(t)$ is identified as the solution to a forward-time DRE, which is the inverse of the error covariance matrix found in the stochastic Kalman filter.
• Deterministic Interpretation of Information: The paper offers a deterministic interpretation of the "information matrix." Rather than relying on probabilistic covariance, $S(t)$ is interpreted as a "certainty matrix." The curvature of the cost-to-arrive function (a quadratic bowl) around the optimal estimate is determined by $S(t)$. The eigenvectors of $S(t)$ with large eigenvalues correspond to directions of high certainty (steep curvature), while small eigenvalues correspond to high uncertainty.

Significance and Claims
The paper claims to offer a "tutorial" perspective that demystifies the derivation of the Kalman filter by grounding it in deterministic optimal control theory. It argues that the preference for deterministic versus stochastic formulations is often a matter of taste rather than logical necessity, citing Willems and Gauss. The primary significance lies in the "two easy steps" approach, which:

1. Unifies the treatment of affine-quadratic problems (like tracking and estimation) with standard quadratic problems (LQR) via homogeneous coordinates.
2. Clarifies the role of time reversal and the "cost-to-arrive" function in deriving optimal observers.
3. Provides a rigorous deterministic justification for the Kalman filter equations without invoking stochastic calculus, relying instead on least-squares principles and the equivalence of input design problems.

The author explicitly avoids introducing new applications or experimental proposals, focusing instead on the theoretical unification of existing concepts (LQR, homogeneous coordinates, and duality) to explain the structure of the optimal estimator.