📊 statistics

Empirical Measures and Strong Laws of Large Numbers in Categorical Probability

This paper establishes a unified categorical framework for the Glivenko–Cantelli theorem, the strong law of large numbers, and de Finetti's theorem by introducing "empirical sampling morphisms" within quasi-Markov categories to formalize the convergence of empirical measures from first principles.

Original authors: Tobias Fritz, Tomáš Gonda, Antonio Lorenzin, Paolo Perrone, Areeb Shah Mohammed

Published 2026-05-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Tobias Fritz, Tomáš Gonda, Antonio Lorenzin, Paolo Perrone, Areeb Shah Mohammed

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to figure out the "true nature" of a mysterious crowd based only on a long list of people walking by. This is essentially what probability theory does: it tries to understand the hidden rules (the distribution) that generate a sequence of random events (the samples).

This paper, titled "Empirical Measures and Strong Laws of Large Numbers in Categorical Probability," by Tobias Fritz and colleagues, is a high-level mathematical adventure. It doesn't just look at numbers; it tries to rebuild the entire logic of probability theory using a new language called Category Theory. Think of Category Theory as a "universal translator" that describes how things connect and flow, rather than just calculating specific numbers.

Here is the story of their discovery, broken down into simple concepts and analogies.

1. The Problem: The "Infinite" Sequence

In the real world, if you flip a coin 1,000 times, you can count how many heads you got. If you flip it a million times, you get a better idea. But what if you flip it forever?

Mathematicians have known for a long time (thanks to the Law of Large Numbers) that if you keep flipping a coin, the percentage of heads will eventually settle down to the true probability (50%). This is the "Strong Law."

However, there's a catch. Not every infinite sequence of coin flips settles down. Some sequences might oscillate forever (like 1, 0, 1, 0, 1, 0... but with increasingly long pauses). For these "bad" sequences, you can't define a true probability.

The authors ask: Can we build a mathematical machine that takes an infinite sequence as input and outputs the "true" probability distribution, but only if the sequence is "good" enough to have one?

2. The Solution: The "Empirical Sampling Machine"

The authors propose a new kind of mathematical object called an Empirical Sampling Morphism.

Think of this as a specialized vending machine:

The Input: You feed it an infinite stream of data (like a long list of numbers or coin flips).
The Output: If the stream is "well-behaved," the machine spits out a single sample drawn from the "average" of that stream (the empirical measure).
The Catch: If the stream is chaotic and never settles down, the machine refuses to work. It doesn't give you a wrong answer; it simply says, "I can't process this."

In the language of the paper, this is a partial morphism. It's a function that only works on a specific subset of inputs (the "good" sequences).

3. The Rules of the Machine

To ensure this machine makes sense, the authors give it two strict rules (axioms):

Rule 1: The Shuffle Rule (Permutation Invariance)
Imagine you have a list of 1,000 numbers. If you shuffle the first 10 numbers around, the "average" nature of the list shouldn't change. The machine must give the same result regardless of the order of the input, as long as the overall collection of data is the same. It ignores the order and looks only at the "bulk" of the data.
Rule 2: The Self-Consistency Rule (Empirical Adequacy)
This is a bit like a mirror test. If you take a sequence generated by a fair coin, feed it into the machine to get an "average coin," and then use that average to generate a new sequence, the new sequence should look statistically identical to the original. The machine must be consistent with itself.

4. The Big Discovery: Rebuilding Probability from Scratch

The authors didn't just build this machine for one specific case (like coin flips). They built a theoretical framework (using "Quasi-Markov Categories") that allows them to prove three massive, famous theorems simultaneously, using only the rules of their machine:

The de Finetti Theorem: This says that if a sequence of events looks random and exchangeable (order doesn't matter), it must have been generated by a hidden "average" distribution. The authors prove this is a natural consequence of their machine's rules.
The Glivenko–Cantelli Theorem: This is the "uniform" version of the Law of Large Numbers. It says that the entire shape of the data distribution (not just the average) converges to the truth.
The Strong Law of Large Numbers: The classic result that the average of your samples converges to the true expected value.

The Magic: Usually, proving these three theorems requires heavy, complex math (measure theory). The authors show that if you accept the existence of their "Empirical Sampling Machine" and its two rules, all three theorems fall out automatically, like dominoes.

5. Making it Real: The "Partial" Machine

A major hurdle was that in the real world (specifically with real numbers), you can't always define this machine for every possible infinite sequence.

The Analogy: Imagine trying to calculate the average height of an infinite line of people. If the line includes some people who are infinitely tall, the average breaks.
The Fix: The authors constructed a specific version of this machine for real numbers (like the height of people or stock prices). They defined exactly which sequences are "good" (those where the average settles down and doesn't explode to infinity) and which are "bad."

They proved that for these "good" sequences, the machine works perfectly and recovers the standard results we use in statistics today.

Summary

In simple terms, this paper is a unified theory of randomness.

The authors built a conceptual "black box" (the Empirical Sampling Morphism) that takes infinite data and outputs the underlying probability. By defining exactly how this box should behave (ignoring order and being self-consistent), they were able to derive the most important laws of probability (de Finetti, Glivenko–Cantelli, and Strong Law) as logical consequences.

They showed that these laws aren't just lucky coincidences of math; they are the inevitable result of how we define "averaging" over infinite data. It's a new, cleaner, and more structured way to understand why the "Law of Large Numbers" works.

Technical Summary: Empirical Measures and Strong Laws of Large Numbers in Categorical Probability

Problem Statement
The paper addresses the challenge of formulating fundamental limit theorems of probability theory—specifically the Glivenko–Cantelli theorem and the Strong Law of Large Numbers (SLLN)—within the framework of categorical probability. While categorical probability has successfully abstracted concepts like independence, conditioning, and the de Finetti theorem using Markov categories, these frameworks typically assume total morphisms (morphisms defined for all inputs).

The specific difficulty addressed here is the nature of empirical measures for infinite sequences. In standard measure theory, the empirical measure of an infinite sequence $(x_i)_{i \in \mathbb{N}}$ is defined as the limit of relative frequencies. However, this limit does not exist for every sequence (e.g., oscillating sequences), and even when it exists, it may fail to define a valid probability measure (e.g., failing $\sigma$ -additivity, as in the sequence $1, 2, 3, \dots$ over $\mathbb{N}$ ). Consequently, the operation of "sampling from an empirical measure" is inherently a partial operation. Standard Markov categories, which require all morphisms to be total, are insufficient to model this phenomenon directly. The authors seek to develop a synthetic (axiomatic) framework that accommodates these partialities while recovering the standard measure-theoretic results.

Methodology
The authors employ a two-pronged approach combining abstract category theory with concrete measure-theoretic constructions:

Quasi-Markov Categories: To handle partiality, the authors work within quasi-Markov categories. Unlike standard Markov categories where every morphism is total, quasi-Markov categories allow for morphisms that are "quasi-total." A morphism $f: A \to X$ is quasi-total if it is either undefined or produces a normalized output for every input. This framework supports a canonical partial order (extension relation) where $f \sqsupseteq g$ if $f$ extends $g$ .
$\sigma$ -Continuity: To manage infinite tensor products (Kolmogorov products) and infinite sequences, the authors introduce the assumption of $\sigma$ -continuity. This ensures that hom-sets possess countable directed meets (intersections of domains) and that these meets are preserved by composition and tensor products. This property is shown to correspond to the $\sigma$ -continuity of probability measures in measure theory.
Axiomatization of Empirical Sampling: The core of the methodology is the definition of an empirical sampling morphism $es_X: X^\mathbb{N} \to X$ $e s_{X} : X^{N} \to X$ . This morphism takes an infinite sequence as input and outputs a sample from its empirical measure. The authors propose two axioms for this morphism:
- Permutation Invariance: The morphism is invariant under finite permutations of the input sequence.
- Empirical Adequacy: If a distribution on $X^\mathbb{N}$ is exchangeable, then sampling from it is equivalent to iterated empirical sampling applied to a sampled sequence.
Concrete Constructions: The authors construct explicit empirical sampling morphisms for standard Borel spaces (finite sets, $\mathbb{N}$ , and $\mathbb{R}$ ) within the category of partial Markov kernels, Partial(BorelStoch). They carefully define the domains of these morphisms to ensure the resulting limits satisfy $\sigma$ -additivity and, for $\mathbb{R}$ , uniform convergence of cumulative distribution functions (CDFs).

Key Contributions

Framework for Partiality: The paper develops the theory of quasi-Markov categories, specifically establishing the existence and properties of Kolmogorov products in this setting. This includes proving that Partial(BorelStoch) is a $\sigma$ -continuous quasi-Markov category.
Definition of Empirical Sampling Morphisms: The authors provide a rigorous categorical definition of empirical sampling morphisms via the axioms of permutation invariance and empirical adequacy. They demonstrate that these morphisms exist for all standard Borel spaces.
Synthetic Proofs of Limit Theorems: Using the existence of empirical sampling morphisms and the splitting of balanced idempotents, the authors derive abstract versions of:
- The de Finetti Theorem: Showing that exchangeable morphisms factor through a distribution object.
- The Glivenko–Cantelli Theorem: Establishing the uniform convergence of empirical measures to the underlying distribution.
- The Strong Law of Large Numbers: Deriving the convergence of empirical averages to the expected value.
Unification: The work provides a joint proof of the de Finetti theorem, Glivenko–Cantelli theorem, and SLLN from first principles within a single categorical framework, rather than treating them as separate measure-theoretic results.

Results

Representability: The existence of empirical sampling morphisms implies that the category is observationally representable. This means that for every object $X$ , there exists a distribution object $PX$ (representing probability measures on $X$ ) and a sampling morphism $samp: PX \to X$ .
Synthetic Glivenko–Cantelli Theorem: In any quasi-Markov category satisfying the authors' assumptions (including the existence of empirical sampling morphisms), the resampling morphism $es^{(N)}$ splits through the distribution object. This leads to the result that for any exchangeable morphism $f$ , the empirical measure is almost surely equal to the underlying distribution.
Recovery of Standard Results: By instantiating the abstract theorems in Partial(BorelStoch):
- The synthetic Glivenko–Cantelli theorem recovers the standard theorem for real-valued random variables, stating that the empirical CDF converges uniformly to the true CDF almost surely.
- The synthetic SLLN recovers Kolmogorov's SLLN for real-valued random variables with finite first moments. The authors note that a specific restriction on the domain of the empirical sampling morphism for $\mathbb{R}$ is necessary to ensure the convergence of the empirical average (handling the case where the mean might not exist).

Significance
The paper claims that this work provides a structured proof of three fundamental probabilistic results from first principles. The authors argue that their joint proof is shorter and possesses enhanced conceptual clarity compared to traditional purely measure-theoretic treatments. By identifying empirical sampling morphisms as a key concept, the paper suggests that these morphisms are an invaluable tool for translating laws of large numbers into categorical language. The authors posit that this realization facilitates further progress in categorical probability, potentially aiding in the development of other limit theorems, such as ergodic theorems, within the same framework. The work emphasizes that while translating probabilistic concepts to category theory is often non-trivial, the resulting abstraction reveals the essential structural properties (permutation invariance and empirical adequacy) that underpin these classical theorems.