📊 statistics

Bayesian Inference and Learning in Gaussian Process State-Space Models with Particle MCMC

This paper presents a fully Bayesian framework for joint state estimation and system identification in nonlinear nonparametric state-space models by placing Gaussian process priors on transition dynamics and employing tailored Particle Markov Chain Monte Carlo samplers to efficiently infer the joint smoothing distribution while preserving model expressivity and enabling computational scalability through sparse approximations.

Original authors: Roger Frigola, Fredrik Lindsten, Thomas B. Schön, Carl E. Rasmussen

Published 2026-06-04

📖 5 min read🧠 Deep dive

CC BY 3.0

Original authors: Roger Frigola, Fredrik Lindsten, Thomas B. Schön, Carl E. Rasmussen

Original paper licensed under CC BY 3.0 (http://creativecommons.org/licenses/by/3.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to figure out how a mysterious, invisible machine works. You can't see the machine's gears or levers (the internal "state"), and you don't know the exact rules it follows to move from one moment to the next (the "dynamics"). All you have are blurry, noisy snapshots of what the machine looks like at different times (the "measurements").

This paper presents a new, highly flexible way to solve this puzzle using Bayesian Inference and Gaussian Processes. Here is the breakdown in everyday terms:

The Problem: The "Black Box" Machine

In many real-world systems (like weather, stock markets, or robot movements), things change over time based on hidden rules.

The Hidden State: Think of the machine's internal position. You can't see it directly.
The Dynamics: This is the rulebook that says, "If the machine is here, it will move there next." Usually, we try to guess this rulebook by assuming it looks like a simple line or a specific curve. But real life is messy; the rules might be wiggly, complex, and unpredictable.
The Noise: Your snapshots are blurry. You might see a shadow that looks like a circle, but it could actually be a square.

The Solution: A "Shape-Shifting" Rulebook

The authors propose a method that doesn't force the machine to follow a simple, pre-defined rulebook. Instead, they use a Gaussian Process (GP).

The Analogy: Imagine you are trying to draw a line connecting a series of dots on a piece of paper.

Old Way (Parametric): You decide beforehand, "I will only use a straight ruler." If the dots curve, your drawing will be wrong.
This Paper's Way (Non-parametric): You use a piece of elastic rubber. You pin the rubber down at the dots you do know, and let the rubber naturally stretch and curve to fill in the gaps. The rubber is flexible enough to learn any shape the data suggests, without you having to guess the shape in advance.

The Magic Trick: "Collapsing" the Mystery

The hardest part of this problem is that you don't know the rubber band's shape (the rulebook) and you don't know where the dots are (the hidden state). They depend on each other.

The authors' clever trick is to marginalize (or "collapse") the rulebook out of the equation.

The Metaphor: Imagine you are trying to find a lost hiker in a forest. Usually, you need a map of the forest to find them. But here, the map itself is missing.
The Trick: Instead of trying to draw the map and find the hiker at the same time, the authors say, "Let's pretend we don't care about the specific shape of the map for a moment. Let's just calculate the probability of where the hiker could be, considering every possible map at once."
By doing this mathematically, they remove the need to guess the complex rulebook first. This leaves them with a clearer picture of where the hidden state is likely to be.

The Engine: Particle MCMC (The "Swarm of Explorers")

To actually calculate these probabilities, they use a method called Particle Markov Chain Monte Carlo (PMCMC), specifically a version called PGAS.

The Analogy:
Imagine you have a swarm of 20 explorers (particles) trying to trace the path of the hidden hiker through time.

The Problem: If the explorers just guess randomly, they might all get stuck in the same dead end, or they might forget where they started (a problem called "path degeneracy").
The Solution (PGAS): The explorers work in a team. They keep a "leader" path (a specific trajectory they know is good). As they move forward, they constantly check: "If I were to swap my current path with the leader's path, would it make sense?"
The Result: This allows the swarm to explore many different possibilities for the hidden state and the complex rulebook simultaneously, ensuring they don't get stuck in one bad guess. They effectively "sample" the most likely history of the machine.

The Payoff: Learning the Rules

Once the explorers have mapped out the most likely path of the hidden state (the "smoothing distribution"), the authors can finally figure out the rulebook.

Because the rubber band (Gaussian Process) is flexible, once they know where the points actually were, they can draw the exact curve that connects them.
They can then predict what the machine will do next, even in situations they haven't seen before.

Why This Matters (According to the Paper)

Flexibility: Unlike older methods that force the system to be a simple line or curve, this method can learn complex, wiggly, non-linear behaviors.
Robustness: Even if the initial guess about the system is wrong (like using a "Model B" that is clearly different from reality), the method corrects itself and finds the true behavior.
Efficiency: They developed a way to do this without the computer crashing under the weight of the math, even for long sequences of data, by using "sparse" techniques (focusing on key points rather than every single point).

Summary

The paper introduces a way to learn how a complex, hidden system works by treating the rules of the system as a flexible, shape-shifting rubber band. By using a smart team of digital explorers (Particle MCMC) to trace the hidden path first, they can then perfectly reconstruct the rules that govern the system, even when the data is noisy and the rules are unknown.

Technical Summary: Bayesian Inference and Learning in Gaussian Process State-Space Models with Particle MCMC

Problem Statement

State-space models (SSMs) are widely used for modeling time series and dynamical systems, typically defined by a latent state trajectory $x_{0:T}$ and a state transition function $f$ . A significant challenge arises when the dynamics $f$ are unknown and highly nonlinear. Traditional approaches often resort to parametric representations of $f$ or use Expectation-Maximization (EM) algorithms that can lead to overfitting, particularly in high-dimensional state spaces or when the smoothing distribution is uncertain.

While Gaussian Processes (GPs) offer a flexible, nonparametric prior for $f$ , applying them to SSMs is difficult because the latent states $x_{0:T}$ are unobserved. Existing methods often approximate the GP using a finite set of target points, effectively rendering the model parametric, or rely on fixed models of transition dynamics for smoothing. The core problem addressed by this paper is how to perform fully Bayesian inference and learning in nonlinear, nonparametric SSMs without sacrificing the model's expressivity or incurring prohibitive computational costs.

Methodology

The authors propose a fully Bayesian framework where a Gaussian Process prior is placed directly over the unknown state transition function $f$ . The observation model $g$ is assumed to be known (possibly parameterized), while the dynamics $f$ are treated as a random function.

1. Marginalization of the Transition Function
Instead of inferring the joint posterior over the states, hyper-parameters, and the function $f$ simultaneously, the authors marginalize out the transition function $f$ . This "collapsing" approach allows the method to respect the nonparametric nature of the model. By conditioning on the state trajectory $x_{0:T}$ , the marginal prior $p(x_{0:T} | \theta)$ can be expressed as a product of Gaussians. Although this distribution is not Gaussian itself (due to the non-linear dependence of the mean and covariance on the states), it retains a structure amenable to sequential inference.

2. Particle Markov Chain Monte Carlo (PMCMC)
To sample from the resulting joint smoothing distribution $p(x_{0:T}, \theta | y_{0:T})$ , the authors employ a Particle Markov Chain Monte Carlo (PMCMC) sampler, specifically Particle Gibbs with Ancestor Sampling (PGAS).

Sequential Monte Carlo (SMC): The algorithm uses SMC to approximate the smoothing distribution. However, standard SMC suffers from path degeneracy, where particle diversity is lost for time steps far from the final observation.
PGAS: To mitigate path degeneracy and enable inference over hyper-parameters $\theta$ , PGAS is used. It combines Gibbs sampling steps for the state trajectory and hyper-parameters. Crucially, it uses a Conditional Particle Filter with Ancestor Sampling (CPF-AS) to sample the state trajectory. In CPF-AS, one particle trajectory is fixed (conditioned upon) at each step, while the others are sampled. The "ancestor sampling" step allows the fixed trajectory to resample its ancestors, ensuring the Markov kernel leaves the exact smoothing distribution invariant and improving mixing rates.

3. Sparse GP Construction
To address the computational complexity, which naively scales as $O(T^4)$ or $O(T^3)$ , the authors introduce a sparse GP construction using the Fully Independent Conditional (FIC) prior. By introducing a set of inducing variables, the computational complexity is reduced to $O(M^2T)$ , where $M \ll T$ is the number of inducing points. The implementation details involve efficient updates of Cholesky factorizations to maintain this complexity during the forward recursion of the particle filter.

4. Learning the Dynamics
Once samples from the smoothing distribution $p(x_{0:T}, \theta | y_{0:T})$ are obtained, the posterior over the transition function $f$ is available in closed form. The predictive distribution for a new test point is formulated as a mixture of Gaussians, where each component corresponds to a sample from the smoothing distribution. This mixture correctly captures multimodality arising from ambiguity in the measurements.

Key Contributions

Fully Nonparametric Inference: The paper presents a method that retains the full nonparametric expressivity of Gaussian Processes in SSMs, avoiding the need to discretize the function space or assume a specific parametric form for the dynamics.
Marginalization Strategy: By marginalizing out the transition function $f$ , the authors decouple the learning of the dynamics from the state estimation, allowing for exact inference of the smoothing distribution given the state trajectory.
PGAS for SSMs: The application of Particle Gibbs with Ancestor Sampling to GP-SSMs enables efficient sampling from the joint smoothing distribution and hyper-parameter posterior, overcoming the path degeneracy issues of standard particle filters.
Computational Efficiency: The integration of sparse GP methods (FIC prior) reduces the computational burden, making the approach scalable to longer time series.

Experimental Results

The authors evaluate their approach on two benchmarks:

Nonlinear System Benchmark: A system with bimodal smoothing densities (due to sign ambiguity in observations). The GP-SSM successfully captured the bimodal nature of the state trajectory and learned the dynamics accurately, even when initialized with a poor parametric mean function. The Root Mean Square Error (RMSE) for the learned dynamics was significantly lower than parametric baselines and comparable to the ground truth model with known parameters.
Cart and Pole System: A reinforcement learning benchmark with a 4-dimensional state space. Using only 100 noisy observations, the model learned a dynamics representation capable of one-step-ahead predictions. The resulting predictive distribution (a mixture of Gaussians) correctly reflected higher uncertainty in regions of the state space far from the training data and lower uncertainty near training data.

Significance and Claims

The paper claims that its primary significance lies in providing a vital component for inference in GP-based SSMs: the ability to learn the GP model itself in a fully Bayesian manner. Unlike previous works that assumed the GP dynamics were already learned or used parametric approximations, this approach allows for adaptive model complexity.

The authors emphasize that their method preserves the nonparametric expressivity of the model, enabling it to capture complex dynamical phenomena that parametric models might miss. They note that the approach is particularly robust in scenarios with high uncertainty in the smoothing distribution, where traditional MAP-based or EM-based approaches might overfit. The use of PGAS ensures that the inference is not limited by the fixed model assumptions of conventional smoothing techniques, offering a flexible framework for system identification in nonlinear, nonparametric settings.