🤖 machine learning

From Gradients to Riccati Geometry: Kalman World Models for Single-Pass Learning

This paper introduces Kalman World Models, a gradient-free framework that replaces backpropagation with recursive Bayesian filtering and Kalman gain adaptation to enable online, robust training of state-space models and transformers grounded in control theory.

Original authors: Andrew Kiruluta

Published 2026-03-17

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Andrew Kiruluta

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to speak or recognize objects. For the last decade, the standard way to do this has been Backpropagation.

Think of Backpropagation like a strict teacher grading a final exam.

The robot tries to answer a question (Forward Pass).
The teacher checks the answer, sees the mistakes, and then walks backward through the robot's entire brain to figure out exactly which neuron made the error and how much to tweak it (Backward Pass).
The robot updates its weights and tries again.

This works great, but it's slow and memory-hungry. The teacher has to remember every single step the robot took to grade it correctly. If the robot is huge (like a modern AI), this "backward walk" becomes a logistical nightmare.

This paper proposes a completely different approach: "Kalman World Models."

Instead of a teacher grading a final exam, imagine the robot is a pilot flying a plane in foggy weather.

The Core Idea: The Pilot vs. The Grader

In this new system, the AI isn't trying to minimize a "loss" (a score of how bad it did). Instead, it's constantly estimating its own state in real-time.

The Old Way (Backprop): "I got the answer wrong. Let me rewind my memory, find the exact moment I went off course, and fix it."
The New Way (Kalman Filtering): "I'm flying through fog. I see a mountain (the data). My map says I should be over the ocean. I'm not sure if I'm wrong or if the map is wrong. So, I take a tiny, calculated step toward the mountain, but I adjust my confidence based on how foggy it is."

Key Concepts Explained with Analogies

1. The "Innovation" Signal (The Surprise)

In the old way, the AI calculates a "gradient" (a mathematical slope) to know which way to go.
In the new way, the AI looks at the Innovation.

Analogy: Imagine you are walking home. You expect to see a red mailbox. You turn the corner, and there is a blue mailbox.
The Innovation: The difference between what you expected (Red) and what you saw (Blue).
The Update: Instead of calculating a complex slope, the AI simply says, "Okay, I was wrong. I need to adjust my internal map slightly toward the blue mailbox." The size of the adjustment depends on how confident the AI was in its original guess. If it was very confident, it adjusts a little. If it was guessing, it adjusts a lot.

2. The "Kalman Gain" (The Smart Adjuster)

This is the secret sauce. In standard AI, we use a "learning rate" (a fixed knob that says "adjust by 0.01").
In this paper, the AI calculates a Kalman Gain on the fly.

Analogy: Think of the Kalman Gain as a smart volume knob.
- If the AI is very sure of its current location (low uncertainty), the knob turns down low. It makes tiny, careful adjustments.
- If the AI is lost and confused (high uncertainty), the knob turns up high. It makes big, bold changes to its internal map.
Why it matters: The AI doesn't need a human to set the learning rate. It figures out how much to learn based on how confused it currently is.

3. "World Models" (The Internal Map)

The paper suggests treating the AI's internal thoughts (activations) not as static data, but as a moving state.

Analogy: Imagine a detective solving a mystery.
- Old Way: The detective writes down clues, then goes back to the beginning of the notebook to rewrite the whole theory from scratch every time a new clue appears.
- New Way: The detective keeps a "working theory" in their head. When a new clue arrives, they just tweak the theory slightly to fit the new fact. They don't rewrite the whole book; they just update the current page.
This allows the AI to learn continuously. It doesn't need to stop and re-train on old data; it just updates its current state as new data arrives.

4. Fixing the "Hallucinations" (Activation Correction)

One of the coolest parts of the paper is applying this to Large Language Models (LLMs) like the one you are talking to now.

The Problem: Sometimes LLMs get confident but wrong (hallucinations).
The Solution: The paper suggests that when the AI generates a word, it can immediately check: "Did that word make sense?" If the word was unexpected (high innovation), the AI can correct its own internal thought process before generating the next word.
Analogy: It's like a musician playing a solo. If they hit a wrong note, they don't stop the concert to rewrite the sheet music. They immediately adjust their next few notes to smooth out the mistake and get back on track. This happens in real-time, without stopping the flow.

5. The "Koopman Lift" (The Magic Lens)

The paper mentions a complex math trick called "Koopman operators."

Analogy: Imagine trying to predict the path of a swirling leaf in a tornado. It's chaotic and hard to predict (non-linear).
The Trick: Instead of trying to predict the leaf's crazy path directly, you put on a pair of special glasses (the Koopman lift). Through these glasses, the leaf's path looks like a perfect, straight line.
The Result: The AI learns to see the world through these "glasses" where everything is simple and linear. Then, it uses simple, fast math (Kalman filtering) to predict the future. It's much easier to navigate a straight line than a tornado.

Why Should We Care?

It's Online: You don't need to store massive amounts of data to train. The AI learns as it goes, like a human.
It's Robust: Because it tracks its own uncertainty, it knows when it's confused and adjusts accordingly. It's less likely to be fooled by weird data.
It's Memory Efficient: It doesn't need to remember the entire "backward walk" through the brain. It just needs to remember its current "state" and "confidence."
It's Safer: By treating the AI as a dynamic system that corrects itself in real-time, it might be less prone to the "catastrophic forgetting" (where AI learns a new thing and forgets everything old) that plagues current models.

The Bottom Line

This paper is saying: "Stop treating AI like a student taking a test. Start treating it like a pilot flying a plane."

Instead of grading the past and rewinding, let the AI constantly estimate its position, measure its uncertainty, and make small, smart corrections as it flies forward. It's a shift from static optimization to dynamic estimation.

Based on the paper "From Gradients to Riccati Geometry: Kalman World Models for Single-Pass Learning" by Andrew J. Kiruluta, here is a detailed technical summary.

1. Problem Statement

Modern deep learning is dominated by backpropagation (reverse-mode automatic differentiation) and stochastic gradient descent (SGD). While effective, this paradigm faces significant limitations:

Memory Overhead: It requires storing all intermediate activations for the backward pass, leading to memory complexity proportional to model depth and sequence length ($O(LdT)$).
Static Optimization: It treats parameters as fixed unknowns optimized offline, lacking inherent mechanisms for online adaptation or uncertainty quantification.
Distribution Shift: Standard optimizers struggle with continual learning and catastrophic forgetting because they do not explicitly track parameter confidence or uncertainty.
Historical Limitations: Previous attempts to use Extended Kalman Filters (EKF) for neural network training in the 1990s failed to scale due to computational costs and reliance on local linearization, leading to their abandonment in favor of gradient methods.

The paper posits that learning should be reframed not as deterministic optimization, but as sequential Bayesian state estimation in a dynamical system.

2. Methodology: Kalman World Models (KWM)

The authors propose Kalman World Models (KWM), a framework that replaces gradient descent with recursive Bayesian filtering.

A. Core Conceptual Shift

Parameters as Latent States: Model parameters $\theta_t$ are treated as dynamic latent states evolving over time, rather than static constants.
Learning as Filtering: Training becomes an online filtering process where the "error signal" is an innovation term ( $y_t - \hat{y}_t$ ), and parameter updates are driven by a Kalman Gain ( $K_t$ ) derived from propagated uncertainty (covariance), not a heuristic learning rate.
State-Space Formulation: The network is modeled as:
$x_{t+1} = f_\theta(x_t, u_t) + w_t$
$y_t = h_\theta(x_t) + v_t$
where parameters are augmented into the state vector $z_t = [x_t, \theta_t]^T$ .

B. Key Theoretical Innovations

Kalman–Natural Gradient Equivalence:
The paper proves that under locally linear Gaussian observation models, the Kalman update is mathematically equivalent to a Natural Gradient Descent step.
- The Kalman covariance matrix $P_t$ acts as the inverse Fisher Information metric ( $F^{-1}$ ).
- The update rule $\theta_{t+1} = \theta_t + K_t(y_t - \hat{y}_t)$ is shown to be a Riemannian steepest descent step, unifying control theory and information geometry.
Activation-Level Innovation Correction:
Unlike classical EKF which only updates weights, KWM applies Kalman-style corrections to hidden activations in Transformers.
- Hidden states $h_t$ are treated as latent states subject to measurement updates based on token residuals.
- This allows for online correction of representations during decoding without backpropagation, acting as a nonlinear observer to mitigate hallucinations and distribution shifts.
Koopman-Lifted World Models:
To address the nonlinearity of deep networks (which usually requires repeated Jacobian linearization in EKF), the authors introduce Koopman operator theory.
- Nonlinear dynamics are "lifted" into a higher-dimensional feature space where evolution is exactly linear ( $z_{t+1} = K z_t$ ).
- This allows for exact Kalman filtering in the lifted space, eliminating the approximation errors associated with local linearization.
Structured Covariance for Scalability:
To handle billions of parameters, the full covariance matrix $P_t$ (which scales as $O(p^2)$ ) is approximated using:
- Block-diagonal structures (ignoring inter-layer correlations).
- Low-rank factorizations ( $P_t \approx U_t U_t^T + \delta I$ ).
- Kronecker-factored approximations (similar to K-FAC).
  These reduce complexity to near-linear scaling ($O(pr)$) while preserving stability.

3. Key Contributions

Formal Equivalence: Proved that Kalman filtering updates are equivalent to Natural Gradient Descent with Fisher preconditioning, grounding recursive filtering in information geometry.
Filtering-as-Training Principle: Reinterpreted learning as sequential Bayesian inference, enabling online adaptation and explicit uncertainty quantification without reverse-mode differentiation.
Activation Correction: Introduced a novel mechanism to correct internal transformer activations via innovation terms, improving robustness to distribution shifts.
Koopman Integration: Combined Koopman operator lifting with Kalman filtering to enable exact linear filtering for nonlinear neural dynamics, avoiding repeated Jacobian computations.
Stability Guarantees: Provided rigorous stability analysis (via Riccati equations and Lyapunov functions) showing exponential convergence under persistent excitation and bounded noise, even with low-rank approximations.

4. Experimental Results

The authors evaluated KWM on three benchmarks against SGD, Adam, K-FAC, and classical EKF:

Sequential MNIST: KWM achieved comparable accuracy to Adam (97.8% vs 97.5%) but with significantly reduced sensitivity to learning rate initialization and faster covariance stabilization.
WikiText-2 (Language Modeling): KWM matched the perplexity of Adam (88.4 vs 88.7) but demonstrated superior robustness under synthetic distribution shifts (token dropout/vocabulary perturbation), maintaining lower perplexity (98.2 vs 110.3 for Adam).
Continual Learning: On permuted MNIST and streaming tasks, KWM reduced catastrophic forgetting by ~30% compared to Adam and 18% compared to K-FAC, leveraging the covariance matrix to protect high-confidence parameters.

Table 1 Summary:

Method	Seq. MNIST Acc.	WikiText-2 PPL	Shifted PPL	Avg. Forgetting
Kalman World Model	97.8%	88.4	98.2	0.17
Adam	97.5%	88.7	110.3	0.24
K-FAC	97.7%	87.9	105.6	0.21
Classical EKF	97.2%	90.8	112.7	0.26

5. Significance and Impact

Paradigm Shift: Moves beyond the "gradient descent" dogma, offering a principled alternative grounded in control theory and Bayesian inference.
Online & Continual Learning: Provides a natural mechanism for online learning and continual adaptation without catastrophic forgetting, as uncertainty is explicitly tracked.
Robustness: The innovation-based correction mechanism offers a new approach to mitigating hallucinations and handling distribution shifts in Large Language Models (LLMs).
Theoretical Unification: Bridges the gap between second-order optimization (Natural Gradient), recursive filtering (Kalman), and operator theory (Koopman), creating a unified mathematical framework for learning.
Scalability: By utilizing structured covariance approximations, the method addresses the historical scalability issues of EKF, making it potentially viable for modern deep architectures.

In conclusion, the paper argues that neural networks should be viewed as stochastic dynamical estimators governed by Riccati geometry. This perspective enables single-pass, online learning with built-in uncertainty quantification and robustness, offering a compelling alternative to the memory-intensive and static nature of backpropagation.