🧬 biology

Online Generalised Predictive Coding

This paper introduces Online Generalised Predictive Coding (ODEM), an extension of Dynamic Expectation Maximisation that enables simultaneous online inference of latent states, learning of model parameters, and uncertainty estimation in dynamic environments through a separation of temporal scales, demonstrating its robustness even with mismatched or chaotic generative models.

Original authors: Mehran H. Z. Bazargani, Szymon Urbas, Adeel Razi, Thomas Brendan Murphy, Karl Friston

Published 2026-05-05

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mehran H. Z. Bazargani, Szymon Urbas, Adeel Razi, Thomas Brendan Murphy, Karl Friston

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine your brain is a detective trying to solve a mystery in real-time. Every second, new clues (sensory data) arrive, and the detective must instantly update their theory about what's happening, learn the rules of the game, and figure out how much they can trust their own senses.

This paper introduces a new mathematical tool called ODEM (Online Dynamic Expectation Maximisation) that helps a computer do exactly what this detective does. It's a way for a machine to "guess" the hidden causes of the world around it, learn from mistakes, and adjust its confidence, all while the world keeps moving.

Here is a breakdown of how it works, using simple analogies:

1. The Three Jobs of the Detective (Triple Estimation)

Most computer programs try to do one thing at a time. ODEM tries to do three things simultaneously, which the authors call "online triple estimation":

Tracking the Hidden State: Figuring out where things are right now (e.g., "Is that bird flying left or right?").
Learning the Rules: Figuring out how the world works (e.g., "Birds usually fly in a curve, not a straight line").
Estimating Uncertainty: Deciding how much to trust the data (e.g., "It's foggy, so my eyesight is unreliable; I should trust my memory of bird flight patterns more").

The paper argues that to be truly "online," the system can't wait to see all the data before making a decision. It must make a guess, update it instantly, and move on to the next second of data without looking back.

2. The "Smooth" Trick (Generalised Coordinates)

Imagine you are trying to predict where a car will be in one second.

Old Way: You only look at where the car is right now. If the car suddenly turns, you might be late to the prediction.
ODEM's Way: It looks at the car's position, its speed, its acceleration, and even how fast the acceleration is changing (jerk).

The paper calls this using "Generalised Coordinates of Motion." Think of it like a movie camera that doesn't just take a photo of the car's location, but also records its speed and how hard the driver is pressing the gas pedal. By knowing the "smoothness" of the movement, the system can predict the future much better, even if the car is moving in a chaotic, unpredictable way.

3. The Speed Trap (Separating Time Scales)

One of the biggest challenges in this "detective work" is that some things change fast, and some change slow.

Fast: The position of a moving object (changes every millisecond).
Slow: The rules of the game or how much noise is in the air (changes over minutes or hours).

ODEM uses a clever trick called separation of temporal scales. It treats the "fast" updates (tracking the object) and the "slow" updates (learning the rules) as two different jobs happening at different speeds.

The Fast Job: Updates constantly, like a reflex.
The Slow Job: Updates only occasionally, like learning a new habit.

This prevents the system from getting confused. It doesn't try to rewrite the laws of physics every time a bird flaps its wings; it only updates the laws if the pattern persists over a long time.

4. The Test: The "Wrong" Map

To prove their system works, the researchers set up a difficult test.

The Reality: They created a complex, chaotic world using a mathematical model called the Generalised Lotka-Volterra (think of it as a simulation of three species of animals interacting and fighting for food).
The Detective: They gave the ODEM system a "map" based on a completely different mathematical model called the Lorenz system (often used to simulate weather patterns).

Usually, if you try to navigate a jungle using a map of a city, you get lost. However, because ODEM used the "smooth motion" trick (Generalised Coordinates) and the "speed trap" trick (separating time scales), it managed to track the animals' movements surprisingly well, even though its internal map was fundamentally wrong about the type of world it was in.

5. The Result: A Better Detective

The paper shows that ODEM can:

Track hidden states accurately even when the world is chaotic.
Learn the correct "rules" (parameters) over time, even if it started with the wrong ones.
Adjust its confidence (uncertainty) based on how noisy the data is.

The authors conclude that this method offers a biologically inspired way for machines to learn and adapt in real-time, much like a human brain does, without needing to stop and re-analyze the entire history of events every time a new piece of data arrives.

In short: ODEM is a smart, fast-learning system that guesses the future by looking at how things move smoothly, learns the rules slowly, and knows when to trust its eyes versus its memory—all while the world keeps spinning.

Technical Summary: Online Generalised Predictive Coding

Problem Statement

The paper addresses the challenge of online triple estimation in dynamic environments. In the context of brain-inspired computation, this involves simultaneously performing three tasks in real-time as sensory data arrives:

Online hidden state inference: Estimating the latent dynamical states of the world.
Online parameter learning: Inferring the unknown parameters of the generative model (GM).
Online uncertainty estimation: Estimating the precision (inverse variance) of state and observation noise.

The primary difficulty lies in the online nature of the problem: unlike offline schemes that can revisit the entire time-series to converge on a solution, an online system must process each datum exactly once. It cannot iteratively refine internal states or parameters before the next observation arrives. This necessitates computational schemes capable of rapid, accurate updates under strict temporal constraints, particularly when the functional form of the true generative process (GP) differs from the model used for inference.

Methodology

Theoretical Framework

The proposed method, Online Dynamic Expectation Maximisation (ODEM), is grounded in the Free Energy Principle (FEP) and Variational Inference (VI). It frames perception as the minimisation of Variational Free Energy (VFE), which serves as an upper bound on the negative log-model evidence. The VFE is decomposed into an accuracy term (fitting the data) and a complexity term (penalising divergence from prior beliefs), adhering to Occam's razor.

To make the VFE tractable for continuous-time dynamics, the authors employ the Laplace approximation, approximating the posterior distribution with a Gaussian. This reduces the VFE to a sum of precision-weighted prediction errors.

Generalised Coordinates of Motion (GCM)

A core component of ODEM is the use of Generalised Coordinates of Motion (GCM). Instead of modeling only the position of hidden states ( $x_t$ ), the model infers a vector of higher-order temporal derivatives (velocity, acceleration, jerk, etc.): $\tilde{x}_t = \{x_t, x'_t, x''_t, \dots\}$ .

This allows the model to capture the smooth dynamics of the system without requiring closed-form transition densities or Monte Carlo simulations (unlike Kalman or Particle filters).
The model assumes that random fluctuations in the GP possess a degree of smoothness, relaxing the standard Wiener process assumption.
Generalised observations ( $\tilde{y}_t$ ) are constructed by approximating higher-order derivatives of the sensory input via finite differences.

Separation of Temporal Scales

ODEM implements a crucial separation of temporal scales, inspired by Haken's Synergetics, to manage the three estimation tasks:

Fast D-step (State Inference): Occurs at every observation. It continuously updates the hidden states using gradient-based minimisation of the VFE, regularised by the curvature of the VFE surface (Ozaki scheme). This tracks fast-moving external states.
Slow E-step (Parameter Learning): Occurs less frequently (every interEM observations). It updates model parameters ( $\theta$ ) using accumulated gradients over a window of data. This reflects the slower timescale of synaptic plasticity or changing contexts.
Slow M-step (Uncertainty/Precision Estimation): Occurs at the same slow timescale as the E-step. It updates the precision hyperparameters ( $\lambda$ ) for state and observation noise. This is interpreted as an attentional mechanism, modulating the weighting of prediction errors.

Algorithm

The ODEM algorithm iterates through observations $y_t$ :

Generalise the observation to $\tilde{y}_t$ .
Perform the D-step: Update the mean of the state posterior ( $\tilde{\mu}_x$ ) using a Jacobian-based update rule.
Accumulate gradients for parameters and precisions.
At intervals defined by interEM, perform the E-step and M-step: Update the means and covariances of the parameter ( $\theta$ ) and precision ( $\lambda$ ) posteriors using Robbins-Monro learning rates, then reset the gradient accumulators.

Key Contributions

ODEM Algorithm: The introduction of an online variant of the Dynamic Expectation Maximisation (DEM) algorithm, specifically designed for triple estimation in non-linear, potentially chaotic systems.
Generalised Predictive Coding Implementation: A Python/PyTorch implementation of ODEM that utilises GCM to model dynamics without closed-form transition densities.
Robustness to Model Mismatch: Demonstration that ODEM can successfully track latent states even when the functional form of the GM differs fundamentally from the GP (e.g., using a Lorenz model to infer a Generalised Lotka-Volterra process), provided higher orders of motion are used.

Experimental Results

The authors validated ODEM using a Generalised Lotka-Volterra (GLV) generative process (GP) with smooth, correlated noise. They tested two GMs:

Scenario-Same: A GLV-GM (matching the GP).
Scenario-Different: A Lorenz-GM (mismatched functional form).

Key Findings:

Impact of GCM Order: Increasing the order of motion ( $k_x$ ) from 2 to 3 significantly improved tracking performance, particularly in the Scenario-Different case. With $k_x=2$ , the Lorenz-GM struggled to track the GLV-GP; with $k_x=3$ , tracking improved substantially. In the Scenario-Same case, the GLV-GM performed well with both orders, suggesting GCM is most critical when structural assumptions mismatch.
Convergence: Posterior estimates for parameters ( $\mu_\theta$ ) and precisions ( $\mu_\lambda$ ) stabilised over time. The uncertainty (credible intervals) around parameter estimates reduced faster with higher orders of motion.
Free Action (FA): FA values (a proxy for model evidence) were lower for the Scenario-Same (GLV-GM) than Scenario-Different (Lorenz-GM). While higher orders of motion improved accuracy, they also increased complexity, resulting in similar overall FA values across orders.
Precision Ratio Sensitivity: Performance was highly sensitive to the prior precision ratio $C$ (the ratio of expected observation noise precision to state noise precision), highlighting the importance of tuning the relative trust in sensory data versus internal dynamics.

Significance and Claims

The paper claims that ODEM offers a biologically inspired solution to online inference, learning, and uncertainty estimation. By separating temporal scales, it mimics the brain's ability to perform rapid state estimation (fast neuronal dynamics) while simultaneously adapting slower parameters (synaptic plasticity) and attention (neuromodulation).

The authors assert that ODEM provides a scalable framework for studying adaptive inference in both biological and artificial agents. Crucially, they demonstrate that inference in Generalised Coordinates of Motion allows a model to track complex, non-linear dynamics even when the model's structural assumptions differ from the true generative process, provided the model can capture the smoothness of the dynamics through higher-order derivatives. The work bridges predictive coding, variational inference, and online dynamical systems modelling, offering a method that does not require batch optimisation or retrospective smoothing.