📊 statistics

Compositionality in algorithms for smoothing

This paper establishes a categorical connection between the Backward Filtering Forward Guiding (BFFG) smoothing algorithm and optics by demonstrating that their forward and backward maps collectively define a lax monoidal functor from Markov kernels to optics.

Original authors: Moritz Schauer, Frank van der Meulen, Andi Q. Wang

Published 2026-03-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Moritz Schauer, Frank van der Meulen, Andi Q. Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Solving a Mystery with a Time Machine

Imagine you are a detective trying to solve a mystery. You know where the suspect started (the Root) and you have a few blurry photos of where they ended up (the Leaves). However, you don't know exactly what the suspect did in between. You want to reconstruct their entire path, step-by-step, to understand their story.

In the world of statistics and machine learning, this is called Smoothing. It's about figuring out the hidden history of a system based on partial observations.

The paper introduces a specific method called BFFG (Backward Filtering Forward Guiding) to solve this. The authors, Moritz Schauer, Frank van der Meulen, and Andi Q. Wang, have done something brilliant: they took this complex mathematical algorithm and explained it using the rules of Category Theory (a branch of math that studies how things fit together).

Here is the breakdown of their discovery:

1. The Two-Step Dance: Looking Back and Looking Forward

The BFFG algorithm works like a two-step dance to reconstruct the suspect's path:

Step 1: The Backward Filter (Looking at the Clues)
Imagine you are at the end of the timeline, looking at the final photo. You work your way backward in time, asking: "If the suspect ended up here, what were the likely clues at the previous step?" You are gathering "guidance" from the future to inform the past.
Step 2: The Forward Guide (Walking the Path)
Now, you start from the beginning again. But this time, you don't just walk blindly. You use the "guidance" you gathered in Step 1 to steer your steps. You walk forward, but you are being gently pulled toward the correct path by the clues you found earlier.

The Analogy: Think of it like hiking a mountain in the fog.

Backward: You look at the summit (where you want to go) and trace a path down to the base, marking the best trail markers.
Forward: You start at the base and hike up, following those specific trail markers you just drew, ensuring you don't get lost in the fog.

2. The Secret Sauce: "Optics" and "Lenses"

The paper's main claim is that this two-step dance isn't just a random trick; it has a deep, structural logic. The authors use a concept from category theory called Optics (or Lenses).

The Analogy: The Two-Way Mirror
Imagine a special window (an Optic) that connects two rooms.

The Forward Pass: You look through the window from Room A to Room B. You see a message (the "internal state") that tells you what's happening in B.
The Backward Pass: You look back from Room B to Room A, using that message to adjust what you see in A.

The authors show that the BFFG algorithm is essentially a machine that builds these two-way windows.

3. The "Lego" Discovery (Compositionality)

This is the most exciting part of the paper. In math, Compositionality means: If I build a big machine out of small blocks, does the big machine behave exactly like the small blocks glued together?

The authors proved that Yes, it does.

The Analogy: Building with Lego Bricks
Imagine you have a long chain of events (a path from start to finish).

The Old Way: You might think you have to solve the whole chain as one giant, messy puzzle.
The New Way (The Paper's Insight): You can break the chain into tiny Lego bricks (individual steps). You can figure out the "two-way window" (the Optic) for each tiny brick. Then, you just snap the bricks together.

The magic is that snapping the bricks together creates the exact same result as solving the whole chain at once.

If you have a missing sensor: You can pretend the path is one long jump, or you can split it into two smaller jumps. The math says it doesn't matter; the result is the same.
If you have a complex path: You can break it down into simple, easy-to-solve steps, solve those, and snap them back together.

4. Why Does This Matter?

Why should a non-mathematician care?

Simplifying the Complex: It allows computer scientists to break down massive, scary algorithms into tiny, manageable pieces. Instead of writing one giant, error-prone code, they can write small, reusable "blocks" and snap them together.
Flexibility: If you need to change the middle of a process (like adding a new sensor or removing a step), you don't have to rewrite the whole program. You just swap out the specific "brick" (the optic) for that section.
Reliability: Because the pieces fit together perfectly (mathematically proven), you know the whole system will work correctly even if you change the parts.

Summary

The paper takes a sophisticated statistical tool used for predicting hidden paths (BFFG) and reveals its "skeleton." They show that this tool is built from modular, interchangeable parts (Optics) that snap together perfectly.

In a nutshell: They proved that you can build a complex time-traveling detective story by snapping together simple, two-way mirrors, and the story will always make sense, no matter how you arrange the mirrors. This makes building better AI and statistical models easier, faster, and more reliable.

1. Problem Statement

The paper addresses the challenge of Bayesian smoothing in partially observed stochastic systems, specifically those modeled as Hidden Markov Models (HMMs) or more generally, stochastic processes on directed trees.

The Smoothing Problem: Given observations at the leaf nodes of a tree, the goal is to infer the distribution of the latent states at all interior (unobserved) nodes.
Computational Difficulty: Exact smoothing often requires closed-form solutions (e.g., Kalman filters for linear Gaussian systems) or is restricted to finite state spaces. For general non-linear/non-Gaussian systems, exact computation is intractable.
Existing Solutions: The Backward Filtering Forward Guiding (BFFG) algorithm (Mider et al., 2021) was proposed as a general framework. It uses a backward pass to compute "guiding functions" (approximations of likelihoods) and a forward pass to sample from a guided process, reweighting samples to recover the true posterior.
The Gap: While BFFG is effective, its structural properties regarding compositionality were not fully understood. Specifically, it was unclear how the algorithm behaves when complex transitions are decomposed into simpler steps or when multiple independent processes are combined. The paper seeks to provide a rigorous mathematical framework to understand these compositional properties using Category Theory.

2. Methodology

The authors employ Category Theory, specifically the concepts of Markov Kernels, Optics, and Functors, to abstract and analyze the BFFG algorithm.

A. Mathematical Framework

Markov Kernels ($Stoch$): The authors treat stochastic transitions as morphisms in the category $Stoch$, where objects are measurable spaces and arrows are Markov kernels. Composition follows the Chapman–Kolmogorov equations.
Optics: They utilize Optics (introduced by Riley, 2018) as a categorical structure for bidirectional data flow. An optic consists of:
- A First Map (Forward Pass in Optics / Backward Filter in BFFG): Generates an internal state (message) from the input.
- A Second Map (Backward Pass in Optics / Forward Guiding in BFFG): Consumes the internal state and the output to produce a result.
- Note: The authors explicitly map the BFFG "backward filtering" step to the optic's "first map" and the "forward guiding" step to the "second map," reversing the intuitive temporal naming to fit the categorical definition.
Functoriality: The core methodology involves defining a functor $O$ that maps the category of Markov kernels (specifically pairs of kernels representing the true dynamics and the guiding approximation) to the category of Optics.

B. The Construction

The paper defines a specific optic $O(\kappa, \tilde{\kappa})$ for a pair of kernels:

$\kappa$ : The true transition kernel.
$\tilde{\kappa}$ : An approximate "guiding" kernel (chosen to make the backward pass tractable).
First Map ( $B_{\tilde{\kappa}}$ ): Computes the pullback of a function (the guiding function) via $\tilde{\kappa}$ .
Second Map ( $F_{\kappa, \tilde{\kappa}}$ ): Performs the $h$ -transform (importance sampling step) using the true kernel $\kappa$ and the guiding function derived from the first map.

3. Key Contributions

1. Categorical Formulation of BFFG

The primary contribution is the demonstration that the BFFG algorithm is not just a heuristic procedure but a functor.

Theorem 8: The BFFG algorithm defines a contravariant functor from the opposite category of pairs of Markov kernels ( $Stoch^2$ $S t oc h^{2}$ ) to the category of Optics ($Optic(Set)$).
- This proves that sequential composition is preserved: Composing two kernels and then applying BFFG yields the same result (up to equivalence) as applying BFFG to each kernel individually and composing the resulting optics.

2. Lax Monoidality for BFFS

The paper investigates parallel composition (tensor products).

Theorem 11: In the specific case where the guiding kernel equals the true kernel ( $\tilde{\kappa} = \kappa$ $\tilde{κ} = κ$ ), known as Backward Filtering Forward Sampling (BFFS), the functor is lax monoidal.
- This means the algorithm preserves parallel composition of independent processes, up to a natural transformation (specifically, handling weights and marginals).
- Crucial Distinction: The general BFFG (where $\tilde{\kappa} \neq \kappa$ ) is not lax monoidal. The failure arises because approximating kernels introduces global weights that do not factorize cleanly across independent branches (a phenomenon related to confounding). However, the authors argue (Remark 17) that this failure is practically irrelevant for Monte Carlo inference because the resulting constants cancel out in self-normalized estimators.

3. Structural Insights for Implementation

By establishing these categorical properties, the paper provides a "higher-order structure-preserving core" for smoothing algorithms. This allows practitioners to:

Decompose complex transitions into simpler edges without altering the algorithm's outcome.
Introduce latent variables (splitting edges) to make intractable kernels tractable, knowing the compositionality guarantees the result remains consistent.

4. Key Results

Equivalence of Composition: The paper proves that the order of operations (compose kernels first vs. compose optics first) does not matter for the resulting smoothing distribution.
- Example: If a transition $X_{t-1} \to X_{t+1}$ is intractable, one can introduce a latent node $X_t$ and split the transition into $X_{t-1} \to X_t$ and $X_t \to X_{t+1}$ . The BFFG algorithm applied to the split graph yields the same result as applying it to the original single edge.
Handling Missing Data: The compositionality result implies that missing observations (where a node has no leaf observation) can be handled by simply composing the adjacent kernels, effectively "collapsing" the graph.
Approximation Robustness: The framework validates the use of approximate guiding kernels ( $\tilde{\kappa}$ ) in the backward pass. Even though the guiding step is approximate, the functorial structure ensures that the composition of these approximations over a tree is mathematically consistent.

5. Significance

Theoretical Unification: The paper bridges the gap between computational statistics (specifically smoothing algorithms) and category theory. It moves beyond viewing these algorithms as ad-hoc numerical procedures to viewing them as structural morphisms in a category.
Algorithm Design: It provides a rigorous justification for modular algorithm design. Developers can build complex smoothing algorithms by composing smaller, verified components (edges) rather than re-deriving the entire algorithm for a new graph structure.
Handling Complexity: The results explain why techniques like splitting time steps in Stochastic Differential Equations (SDEs) or introducing auxiliary variables work. The categorical framework guarantees that these structural changes preserve the underlying probabilistic semantics.
Future Directions: By framing BFFG as a functor into Optics, the paper opens the door to applying other categorical tools (e.g., string diagrams, monoidal categories) to optimize and analyze Bayesian inference algorithms, potentially leading to new, more efficient inference schemes.

In summary, the paper establishes that the BFFG algorithm is a compositional structure, formally proving that its behavior on complex graphs is entirely determined by its behavior on individual edges, thereby providing a robust theoretical foundation for scalable Bayesian inference.