A complete characterization of testable hypotheses

This paper completes Le Cam's program by establishing a necessary and sufficient condition for the existence of nontrivial hypothesis tests between two sets of probability measures without requiring a common dominating measure, specifically by utilizing the closures of their convex hulls within the space of bounded finitely additive measures.

The Gist

Imagine you are a detective trying to solve a mystery. You have two suspects, Suspect P and Suspect Q. You don't know exactly who they are, but you have a list of possible "profiles" (probability distributions) for each of them.

Your job is to design a test (a set of rules or a lie detector) that can tell them apart.

• If the test says "It's Q!" when it's actually P, that's a false alarm (Type I error).
• If the test says "It's P!" when it's actually Q, that's a missed clue (Type II error).

A good test is one where you can be confident that if it says "Q," it's actually Q, and not just a fluke. In math-speak, this is called a "nontrivial" or "strictly unbiased" test.

The Old Rule (The "Common Language" Problem)

For decades, statisticians had a golden rule (proven by Lucien Le Cam) for knowing when a good test exists. The rule said:

"You can tell P and Q apart if and only if their profiles are far enough apart in a specific way called 'Total Variation Distance'."

The Catch: This rule only worked if P and Q spoke a "common language." In math terms, they had to share a "dominating measure."

• Analogy: Imagine P and Q are two groups of people. The old rule only worked if everyone in both groups spoke English. If P spoke English and Q spoke French, the rule broke down.
• The Problem: In the real world of modern statistics (non-parametric statistics), many problems involve groups that don't share a common language. The old rule went silent, leaving detectives stuck.

The New Discovery (The "Infinite Library" Solution)

This paper, written by Larsson, Ramdas, and Ruf, fixes the broken rule. They say: "Don't worry about the common language. We can still tell them apart, but we need to look at them through a different lens."

Here is the simple breakdown of their solution:

1. The "Convex Hull" (Mixing the Profiles)

First, imagine you can mix and match the profiles. If Suspect P could be 50% "Profile A" and 50% "Profile B," you create a new "mixed profile." The set of all possible mixes is called the Convex Hull.

• Analogy: If P is a bag of red marbles and Q is a bag of blue marbles, the Convex Hull is a bag containing every possible shade of purple you could make by mixing them.

2. The "Closure" (Filling in the Gaps)

Sometimes, you can get infinitely close to a specific profile by mixing, but you can never actually reach it with a finite mix.

• Analogy: Imagine trying to reach the number 1 by adding 0.9, then 0.99, then 0.999... You get closer and closer, but you never quite touch 1 with a finite number of steps.
• The Old Mistake: The old rule assumed that if you got close enough, you were there. But in complex statistics, getting "close" isn't enough; you need to include the "ghost" profiles that you can only reach in the limit.

3. The Secret Ingredient: "Finitely Additive Measures" (The Ghosts)

This is the paper's big innovation. To make the rule work for every possible scenario (even the ones without a common language), the authors say we must expand our definition of a "profile."

They introduce Finitely Additive Measures.

• Analogy: Think of a standard probability measure as a bucket of water. It has a definite amount.
• The New Concept: A "finitely additive" measure is like a bucket that can hold water, but also allows for "phantom water" that exists at the very edge of infinity. It's a mathematical object that behaves like a probability distribution but can capture things that standard distributions can't (like a point mass at "infinity").

Why do we need ghosts?
In some tricky cases, the "closest" profile between P and Q isn't a real, standard distribution. It's a "ghost" distribution that lives in the limit. If you ignore these ghosts, you might think P and Q are far apart (and a test exists), when in reality, they are touching at the ghostly edge (and no test exists). Or vice versa.

The Final Verdict (The New Rule)

The authors prove that a good test exists if and only if the "Ghost-Enhanced" versions of P and Q are far enough apart.

• Old Rule: Distance between P and Q > Threshold? (Only works if they share a common language).
• New Rule: Distance between the Closures of their Mixtures (including the Ghosts) > Threshold? (Works always, no matter how weird the situation is).

Why This Matters

1. It Completes the Puzzle: Lucien Le Cam (a giant in the field) hinted at this solution decades ago but never wrote it down formally. This paper finishes his work.
2. It Handles the Impossible: It solves problems in modern data science where data is messy, infinite, or doesn't fit standard models (like testing if a distribution is symmetric or if a mean is bounded).
3. It's Practical: Even though the math involves "ghosts" (finitely additive measures), the result tells us exactly when we can trust our statistical tests and when we are fooling ourselves.

Summary Analogy

Imagine you are trying to separate two crowds of people in a dark room.

• The Old Way: You could only separate them if they were wearing different colored shirts (a common reference).
• The Problem: In the dark, no one is wearing shirts.
• The New Way: The authors say, "Don't look at the shirts. Look at the shadows they cast on the wall, including the shadows of the people who almost stood in the light but didn't quite get there."
• The Result: If the shadows of the two crowds are distinct, you can separate them. If the shadows merge (even at the very edge of the wall), you can't.

This paper gives us the map to read those shadows correctly, ensuring we never mistake one crowd for another, no matter how dark the room gets.

Technical Summary

Here is a detailed technical summary of the paper "A complete characterization of testable hypotheses" by Martin Larsson, Aaditya Ramdas, and Johannes Ruf.

1. Problem Statement

The paper addresses a fundamental question in statistical hypothesis testing: Given two sets of probability measures, $\mathcal{P}$ (null hypothesis) and $\mathcal{Q}$ (alternative hypothesis), under what conditions does a nontrivial (strictly unbiased) test exist?

A test $\phi$ is defined as a measurable function $\phi: \Omega \to [0, 1]$. It is nontrivial if its worst-case power strictly exceeds its worst-case level:
$$\sup_{\mu \in \mathcal{P}} \mathbb{E}_\mu[\phi] < \inf_{\nu \in \mathcal{Q}} \mathbb{E}_\nu[\phi]$$
Equivalently, a nontrivial test exists if and only if the minimax risk $R(\mathcal{P}, \mathcal{Q}) < 1$, where the risk is the sum of worst-case Type I and Type II errors.

The Gap in Existing Literature:

• Le Cam's Theorem (1955/2012): If $\mathcal{P}$ and $\mathcal{Q}$ share a common dominating measure $\gamma$, a nontrivial test exists if and only if the convex hulls of $\mathcal{P}$ and $\mathcal{Q}$ are separated by a Total Variation (TV) distance greater than zero.
• The Limitation: Many important nonparametric problems (e.g., distributions with a specified mean, symmetric distributions, or TV/Wasserstein balls) do not admit a common dominating measure. In these "nondominated" settings, Le Cam's condition is silent or fails.
• The Challenge: Previous attempts to generalize this using standard weak closures or weak topologies on probability measures ($M_1$) have proven insufficient, as they either fail to capture necessary separation or are too large (allowing intersections where perfect tests should exist).

2. Methodology

The authors resolve this by shifting the mathematical framework from countably additive probability measures to bounded finitely additive measures.

• Space of Measures ($ba$): They utilize the space $ba$, the dual of the Banach space of bounded measurable functions $L$. This space includes both countably additive measures ($M_1$) and purely finitely additive measures (charges).
• Topology: The key innovation is the use of the weak-$\ast$ topology ($\sigma(ba, L)$) on $ba$.
  • This is the weakest topology ensuring that the expectation map $\mu \mapsto \mathbb{E}_\mu[\phi]$ is continuous for any bounded measurable test $\phi$.
  • By the Banach-Alaoglu theorem, the set of probability charges $ba_1$ is weak-$\ast$ compact.
• Convex Hulls and Closures: The authors define $\text{co}^*(\mathcal{P})$ and $\text{co}^*(\mathcal{Q})$ as the weak-$\ast$ closures of the convex hulls of the hypothesis sets within $ba$.
• Minimax Theorem: The proofs rely on Fan's Minimax Theorem (1953). The compactness of $\text{co}^*(\mathcal{P})$ and $\text{co}^*(\mathcal{Q})$ in the weak-$\ast$ topology is crucial for establishing the equality between the minimax risk and the geometric separation of the sets.

3. Key Contributions and Results

A. Main Theorem (Theorem 1.5)

The paper provides a necessary and sufficient condition for the existence of a nontrivial test in arbitrary settings (dominated or nondominated):

$$\exists \text{ nontrivial test } \iff d_{TV}(\text{co}^*(\mathcal{P}), \text{co}^*(\mathcal{Q})) > 0$$

More quantitatively, the minimax risk is exactly characterized by the TV distance between these closures:
$$R(\mathcal{P}, \mathcal{Q}) = 1 - d_{TV}(\text{co}^*(\mathcal{P}), \text{co}^*(\mathcal{Q}))$$
Crucially, the infimum in the TV distance is achieved by some $\mu^* \in \text{co}^*(\mathcal{P})$ and $\nu^* \in \text{co}^*(\mathcal{Q})$.

B. Resolution of Le Cam's Program

The authors clarify that Le Cam was aware of the limitations of his earlier results but did not formalize the general solution.

• They show that Le Cam's Theorem 1.1 is a corollary of their Theorem 1.5 when a dominating measure exists (Proposition 1.6).
• They distinguish their approach from Le Cam's "generalized tests" (Theorem 1.11), which allow for functionals that are not representable by bounded measurable functions and are thus statistically uninterpretable. The authors' result restricts tests to standard measurable functions but requires the hypothesis sets to be closed in $ba$.

C. Comparison with Other Topologies

The paper demonstrates why other topologies fail:

• Standard Weak Closure: Too large. Example 1.4 shows that weak closures of disjoint sets can intersect, falsely implying no test exists when a perfect test does.
• TV Closure: Too small. Example 1.3 shows that the TV closure of convex hulls may not be large enough to capture the true minimax risk in nondominated settings.
• Weak-$\ast$ Closure in $ba$: The "Goldilocks" solution. It is the largest extension that preserves the risk of any test while maintaining the compactness required for minimax theorems.

D. Relationship to $e$-variables and Effective Null Hypothesis

The paper connects its findings to recent work on $e$-variables (Larsson et al., 2025).

• They define the Effective Null Hypothesis ($P_{eff}$) via the bipolar of $\mathcal{P}$.
• Theorem 3.3: They prove that $P_{eff} \cap M_1 = \text{co}^*(\mathcal{P}) \cap M_1$.
• Significance: While $P_{eff}$ and $\text{co}^*(\mathcal{P})$ differ (the latter contains finitely additive elements), they coincide regarding countably additive measures. This explains why previous work on singleton alternatives ($Q=\{\nu\}$) could avoid $ba$, but general composite hypotheses cannot.

E. Corollary on $e$-variables

Corollary 3.7: A bounded $e$-variable uniformly powered against $\mathcal{Q}$ exists if and only if $d_{TV}(\text{co}^*(\mathcal{P}), \text{co}^*(\mathcal{Q})) > 0$. This bridges the gap between hypothesis testing and the theory of $e$-values in nonparametric settings.

4. Significance and Implications

1. Completeness: This work provides the first complete, necessary, and sufficient characterization of testability for any pair of hypothesis sets, removing the restrictive assumption of a common dominating measure.
2. Necessity of Finite Additivity: The paper argues that finite additivity is not merely a mathematical convenience or a philosophical stance (as in de Finetti's subjective probability) but an unavoidable mathematical consequence of characterizing testability in general nonparametric settings. The "optimal" separating measures often lie outside the space of countably additive measures.
3. Practical Verification: The results allow statisticians to verify if a candidate test is minimax optimal by finding the "closest" pair of measures in the weak-$\ast$ closure (Corollary 1.9).
4. Theoretical Unification: It unifies the theory of hypothesis testing, minimax risk, and $e$-variables under a single geometric framework involving the geometry of $ba$.

5. Conclusion

The paper settles a long-standing open problem in statistical theory. By introducing the weak-$\ast$ topology on the space of bounded finitely additive measures, the authors demonstrate that the existence of a nontrivial test is determined by the separation of the weak-$\ast$ closed convex hulls of the hypotheses. This result generalizes classical theorems by Le Cam and Kraft, corrects previous misconceptions about weak closures, and establishes finite additivity as a fundamental component of the geometry of statistical testing.