📊 statistics

Weak Information Geometry: Riemannian Structures from Distributional Inference Functions and Stein Discrepancies

This paper extends the framework of information geometry by demonstrating that a broad class of parametric models, including those lacking classical likelihoods or scores, can be endowed with Riemannian structures via weak inference functions and Stein discrepancies, which induce a family of Godambe metrics that generalize and dominate the classical Fisher-Rao geometry.

Original authors: Rodrigo Labouriau

Published 2026-07-14

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Rodrigo Labouriau

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to map the shape of a mysterious island. In the old school of statistics, you could only draw this map if the island had a smooth, continuous surface—a "density"—that you could walk on. If the island was made of jagged rocks, or if it was a ghostly fog that didn't stick to the ground, the old mapmakers would throw up their hands and say, "We can't measure this. No map possible." They relied on a tool called the "score function," which is like a compass that only works on smooth terrain.

This paper, written by R. Labouriau in July 2026, says: "Hold on. We can map everything, even the jagged rocks and the ghostly fog, if we change our tools."

The New Compass: The "Instrument"

Instead of trying to walk on the island's surface, the author suggests we use a measurement instrument. Think of this instrument not as part of the island, but as a special probe or a radar dish we hold in our hands.

The Old Way: You need a smooth surface (a probability density) to use your compass. If the surface is broken (like the Cantor distribution, which is a fractal dust with no smooth parts), the compass breaks.
The New Way: You shine a light (the instrument) onto the island. The light bounces off, and you measure the echo. The paper proves that as long as you have a good probe, you can calculate a "Godambe information" metric. This metric is a new kind of map that tells you how much information you have about the island's shape, even if the island is made of pure chaos or doesn't exist in the usual sense.

The "Godambe" Map

The paper introduces a new map called the Godambe–Riemannian manifold.

What it is: A way to measure distance and curvature on a statistical model.
How it works: It uses two numbers from your probe: how sensitive it is to changes (Sensitivity) and how much it wiggles (Variability). Combine them, and you get a perfect map.
The Catch: Unlike the old compass, which was the only valid map, this new method gives you a family of maps. Different probes give you slightly different maps.
- Analogy: Imagine measuring a mountain. A laser rangefinder gives you one map; a satellite photo gives you another. Both are true, but they highlight different features. The paper argues that in complex, broken worlds, there is no single "perfect" map. The "best" map depends entirely on which tool you chose to use.

The Four Impossible Islands

To prove this works, the author builds maps for four types of islands that the old school said were unmappable:

The Shifting Shore (Uniform Scale Model): Imagine a beach where the water line moves depending on how high the tide is. The old compass fails because the "ground" changes under your feet. The new instrument works fine.
The Shifted Exponential: A model where the starting point of the data depends on the parameter. Again, the old compass breaks, but the new probe measures the echo perfectly.
The Fractal Dust (Cantor Location Model): This is the big one. Imagine an island made of a Cantor set—a shape that is so full of holes it has no area, no density, and no "ground" to stand on. The old school says, "No likelihood exists here; we can't even write down a formula." The new paper says, "Watch this." They use a bounded instrument (like a gentle tap) and successfully draw a flat, smooth map of this fractal dust.
The Confused Mixture (Stratified Finite Mixtures): Imagine a group of people where you know some belong to Group A and some to Group B, but for others, you only have a probability (e.g., "60% chance they are A"). The old compass (the score function) gets confused and gives a biased reading. The new instrument ignores the confusion and finds the true shape.

The Heat Equation and the "Spectral Gap"

The paper also looks at a dynamic system: a lattice (a grid) of heat flowing through a wire, but the heat is driven by "alpha-stable noise" (a wild, unpredictable noise that doesn't have a standard variance).

The Finding: They calculated the map for this system and found something cool. The speed at which the map stabilizes (becomes steady) is directly tied to the spectral gap of the grid.
The Analogy: Think of a drum. When you hit it, the sound fades away. The speed at which it goes silent depends on the drum's shape. The paper shows that the "statistical map" of this noisy heat flow stabilizes at the exact same rate the drum's sound fades. If the heat flow is unstable (the drum is broken), the map collapses to zero. It's a direct link between the physics of the system and the geometry of the data.

What the Paper Rules Out

The paper is very clear about what it rejects:

It rejects the idea that a "density" (a smooth surface) is required to do geometry.
It rejects the idea that there is always one single, "canonical" (perfect) map for a model. In these new, messy models, there is no single "Fisher metric" to rule them all. The map is relative to the tool you use.
It argues against the idea that if a likelihood function exists, the score function is always a good tool. In the "stratified mixture" example, the score function exists but is biased (it lies), so the old method fails even though a formula exists.

How Sure Are They?

The authors are mathematically certain about their main claims.

They prove (with theorems and propositions) that if you have a valid instrument, you get a smooth Riemannian metric.
They demonstrate (with closed-form calculations) that this works for the Cantor set, the uniform distribution, and the shifted exponential.
They verify (via numerical simulation) the stability rates for the lattice heat equation, showing the math matches the computer code.
They suggest (based on transversality theory in companion papers) that "generic" instruments (randomly chosen good tools) will almost always work, but they don't claim it's a magic bullet for every single bad choice of tool.

The Bottom Line

This paper is a toolkit for the messy world. It tells us that we don't need to wait for data to be "nice" and smooth to understand it. By using "instruments" (probes) instead of "densities" (surfaces), we can draw beautiful, precise maps of fractals, shifting boundaries, and chaotic noise. The trade-off is that we lose the single "perfect" map, but we gain the ability to map anything. As the author puts it, we trade the "Chentsov–Markov invariance" (the old rule of a single perfect map) for existence (the ability to map the unmappable).

Technical Summary: Weak Information Geometry

Problem Statement
The classical differential-geometric theory of statistical models, pioneered by Fisher, Rao, Amari, and others, relies on the Fisher–Rao metric. This metric is defined via the Fisher information matrix, which requires the existence of a probability density function with respect to a dominating $\sigma$ -finite measure, the differentiability of the log-likelihood, and the finiteness of the information matrix. These conditions fail in several important classes of statistical models:

Parameter-dependent support: Models where the support of the distribution changes with the parameter (e.g., uniform scale, shifted exponential), rendering the score function biased or undefined.
Absence of dominating measures: Singular continuous families (e.g., Cantor location models) where no likelihood or score function exists at all.
Absence of moments: Models with heavy tails (e.g., Student $t$ with $\nu \le 1$ , $\alpha$ -stable processes) where classical moments diverge.
Biased scores in dominated models: Stratified finite mixtures where the score function is a biased inference function, leading to inconsistent maximum likelihood estimators despite the existence of a formal likelihood.

In these scenarios, the Fisher–Rao construction is either unavailable or inferentially meaningless, preventing the application of information geometry tools such as geodesics, curvature analysis, and natural gradient methods.

Methodology
The paper proposes a generalization of statistical models by shifting the representation of a probability law from a density to a tempered distribution $T_\theta \in \mathcal{S}'(\mathbb{R}^k)$ . In this framework, a law is identified with the continuous linear functional mapping a test function $g \in \mathcal{S}(\mathbb{R}^k)$ (Schwartz space) to its expectation $\langle T_\theta, g \rangle = \mathbb{E}_\theta[g(X)]$ .

Information is extracted not from the local behavior of a likelihood, but through an instrument (or measurement device). The paper utilizes two classes of instruments:

Weak Regular Inference Functions: Generalizations of Godambe's regular inference functions where unbiasedness and differentiability are expressed as pairings between the distributional representation $T_\theta$ and test functions, rather than pointwise identities involving densities.
Weak Stein Representations: Operators $A_\theta$ acting on test functions such that $\langle T_\theta, A_\theta g \rangle = 0$ .

For a chosen instrument $\psi(\cdot, \theta)$ , the paper defines the weak sensitivity $S(\theta) = -\mathbb{E}_\theta[\partial_\theta \psi]$ and weak variability $V(\theta) = \mathbb{E}_\theta[\psi \psi^\top]$ . The Godambe information matrix is then constructed as:
$G(\theta) = S(\theta)^\top V(\theta)^{-1} S(\theta)$
This matrix serves as a Riemannian metric on the parameter space $\Theta$ . The framework relies on the assumption that the model map $\theta \mapsto \langle T_\theta, g \rangle$ is weakly differentiable (Assumption 2.3), allowing the use of a weak Bartlett identity to relate sensitivity to the derivative of the distribution.

Key Contributions and Results

Extension of Riemannian Statistical Models (Theorem 1.1):
The paper proves that if a parametric family is equipped with an instrument possessing full-rank sensitivity and positive-definite variability, the Godambe information $G(\theta)$ defines a smooth Riemannian metric. This extends the class of models admitting a Riemannian structure far beyond the Fisher–Rao class.
- Recovery of Fisher–Rao: When a density exists and the score function is an admissible instrument, $G(\theta)$ coincides with the Fisher information $I(\theta)$ .
- Loewner Ordering: Whenever the Fisher information exists, $G(\theta) \preceq I(\theta)$ in the Loewner order. Equality holds if and only if the instrument is a non-singular linear transformation of the score.
Existence of Instruments in Non-Regular Models (Proposition 1.2):
The paper demonstrates that suitable instruments exist for models that fail the Fisher–Rao conditions for distinct reasons:
- Uniform Scale & Shifted Exponential: Parameter-dependent support is handled via moment-based or transform-based (characteristic function) instruments, yielding smooth metrics.
- Cantor Location Family: A singular continuous model with no dominating measure admits a Godambe metric via bounded transform instruments (sinusoids) or moment instruments. The metric is flat ( $G(\theta) \equiv 8$ ) for the moment instrument.
- Stratified Finite Mixtures: In dominated models where the score is biased, "stratification-agnostic" instruments (unbiased under every component law) recover a valid Riemannian metric, whereas the likelihood-based route fails.
- $\alpha$ -Stable Lattice SPDE: A dynamical system driven by $\alpha$ -stable noise (where variance is undefined) admits a closed-form Godambe metric derived from eigenmode probes.
Stein Discrepancy Equivalence (Section 6):
The paper establishes that quadratic Stein discrepancies induce the same local geometry as Godambe metrics. Specifically, if a Stein discrepancy is defined via a quadratic form of expectation residuals with optimal weighting $V(\theta)^{-1}$ , the resulting metric is exactly the Godambe information. This links Stein methods directly to information geometry. Furthermore, Reproducing Kernel Hilbert Space (RKHS) Stein discrepancies are shown to generate a hierarchy of geometries, aggregating a continuum of identities.
Geometric Stability and Spectral Gaps (Section 5):
In the lattice stochastic heat equation example, the paper shows that the rate at which the Godambe metric stabilizes to its stationary value is governed by the spectral gap of the underlying discrete Laplacian. This provides a quantitative bridge between the dynamical stability of the system and the geometric stability of the statistical model.
Inferential Separation and Block-Diagonality (Section 8):
The paper provides a geometric characterization of inferential separation (nonformation). The vanishing of the off-diagonal block of the Godambe metric ( $G_{\alpha\beta} = 0$ ) corresponds to the orthogonality of interest and nuisance parameters.
- Automatic Orthogonality: In symmetric location-scale models, using odd (sine) and even (cosine) instrument pairs automatically yields a block-diagonal metric, achieving exact inferential separation even in models without scores (e.g., Cauchy, Cantor).
- Bhapkar–Godambe Projection: A constructive method is presented to block-diagonalize the metric by projecting the interest instrument onto the orthocomplement of the nuisance instruments.

Significance and Claims
The paper claims that the differential-geometric treatment of statistical models survives the removal of the density-based representation, provided the metric is derived from an instrument rather than a likelihood.

Existence vs. Canonicity: The extension trades the Chentsov–Markov invariance (uniqueness of the Fisher metric) for existence. In non-regular models, there is no canonical metric; instead, there is a family of Godambe metrics indexed by the choice of instrument.
Instrument-Relative Geometry: The metric is interpreted as the geometry of the model as seen through a specific instrument. Different instruments may yield different geometries, and in non-regular models (like the uniform scale), the family may lack a maximal element (no Fisher-type bound), reflecting phenomena like superefficiency.
Utility: The framework allows for the application of geometric tools (geodesics, curvature, natural gradients) to models previously considered "pathological" or inaccessible, including those with parameter-dependent support, singular measures, and infinite variance.

The paper concludes that the Godambe–Riemannian structure provides a unified, inference-function-based characterization of statistical models, decoupling the existence of a geometric structure from the existence of a likelihood function.