📊 statistics

No-prior Bayes reIMagined: probabilistic approximations of inferential models

This paper proposes a reimagined "No-prior Bayes" framework that constructs posterior distributions by deriving inner probabilistic approximations from data-driven inferential models, thereby achieving exact uncertainty quantification and asymptotic efficiency without relying on default priors.

Original authors: Ryan Martin

Published 2026-06-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ryan Martin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but you have no prior knowledge of the suspect. You walk into the room with a blank slate. In the world of statistics, this is called the "no-prior" problem.

For decades, the standard way to solve this has been to pretend you do have a hunch (a "default prior") and then update that hunch with the evidence you find. This is like a detective saying, "I'm going to guess the butler did it, just to get started," and then adjusting that guess based on the fingerprints found. The problem, as this paper points out, is that if your initial guess was just a made-up placeholder, your final conclusion might feel confident but actually be unreliable. It's like building a house on a foundation you invented just to make the math work.

The New Approach: "ReIMagined" Inference

The author, Ryan Martin, proposes a different path. Instead of starting with a made-up guess, he suggests starting with a tool that is mathematically guaranteed to be honest about what it doesn't know.

Here is the step-by-step breakdown of his idea, using simple metaphors:

1. The "Possibility Map" (The IM)

First, the author creates a "Possibility Map." Imagine you are looking at a landscape of hills and valleys representing different possible suspects (parameters).

Traditional Bayes: Tries to draw a smooth, perfect probability curve over the landscape, assuming every point has a specific chance of being true.
The IM (Inferential Model): Instead of a smooth curve, it draws a "fuzzy" map. It highlights the areas where the evidence is strong and leaves the rest vague. It doesn't pretend to know the exact odds; it just says, "This area is definitely possible, and this area is definitely not."

This map is built on Possibility Theory. Think of probability as a bucket of water that must be poured out completely (the total must equal 100%). Possibility is more like a spotlight. The brightest spot (the most likely suspect) is fully illuminated (100%), but the edges of the light can fade out without needing to fill the whole room. This "fuzziness" is actually a feature, not a bug, because it prevents the system from making up facts.

2. The "Inner Probabilistic Approximation"

Now, here is the clever twist. The author knows that most people (and software) are used to working with smooth probability curves, not fuzzy maps. They want a single, clean answer.

So, he takes that honest, fuzzy "Possibility Map" and asks: "What is the simplest, most honest probability curve that fits entirely inside this map?"

He calls this the Inner Probabilistic Approximation.

The Metaphor: Imagine the Possibility Map is a large, irregularly shaped cookie cutter. The "Inner Approximation" is the largest, smooth, round cookie you can cut out that fits perfectly inside that cutter without touching the edges.
Why do this? Because the original map is mathematically proven to be reliable (it won't lie to you). By cutting a probability curve out of the inside of that map, the new curve inherits that reliability. It's a "safe" probability distribution.

3. Why is this better?

The paper argues that traditional "No-Prior Bayes" methods often suffer from False Confidence.

The Risk: A traditional method might say, "I am 99% sure the suspect is the Butler," when in reality, the evidence only supports a 50% chance. It's overconfident.
The Fix: The new method is calibrated. If it says there is a 90% chance of something, it actually happens 90% of the time in the long run. It avoids the trap of being confidently wrong.

4. How it works in practice

The paper shows that this new method:

Agrees with the old methods when they are right: In simple, symmetrical situations (like measuring the height of a group of people), this new method gives the exact same answer as the famous "Jeffreys prior" or "Fisher's fiducial" methods.
Fixes the hard problems: In tricky situations where the math gets messy (like the Behrens-Fisher problem, which involves comparing two groups with different and unknown variances), the old methods often fail or give misleading results. The new method handles these "messy" cases with the same reliability as the simple ones.

Summary

Think of the paper as a new way to bake a cake when you don't have a recipe.

Old Way: You guess the ingredients (the prior), mix them, and hope the cake tastes right. Sometimes it does, but often it's a disaster because your guess was wrong.
New Way: You first check the pantry to see exactly what ingredients you definitely have (the Possibility Map). Then, you bake a cake using only those ingredients, ensuring the cake is guaranteed to be edible (the Inner Approximation).

The result is a statistical method that doesn't need to make up a backstory to work. It starts with the data, builds a safe "fuzzy" boundary around the truth, and then extracts a clean, reliable probability answer from the center of that safety zone.

Technical Summary: No-prior Bayes reIMagined

1. Problem Statement

The paper addresses the fundamental challenge of statistical inference when genuine prior information is lacking. The standard "go-to" strategy in such scenarios is Objective Bayes (or "no-prior Bayes"), which combines a "default prior" (e.g., Jeffreys prior, right Haar prior) with the likelihood via Bayes's theorem. Other approaches include generalized fiducial inference and confidence distributions.

The author argues that while these constructions are natural, they suffer from a critical flaw: reliability. When a default prior is used in the absence of genuine prior beliefs, the resulting posterior probabilities lack a coherent "updating" interpretation. More importantly, the False Confidence Theorem (Balch et al. 2019; Martin 2019) demonstrates that no such no-prior Bayes solution possesses strong reliability properties. Specifically, it is not a rare event for these methods to assign high probability to false hypotheses or low probability to true ones. Consequently, the "belief-forming inferential weight" of these posteriors is questionable.

The paper posits that the root of this issue is the insistence on probabilistic uncertainty quantification (additive measures) when the data alone cannot support a single probability distribution.

2. Methodology: The "reIMagined" Approach

The proposed solution, termed reIMagined, shifts the foundational framework from probability to possibility theory before returning to a probabilistic approximation. The methodology proceeds in three stages:

A. Possibilistic Inferential Models (IMs)

Instead of starting with a prior, the method begins with a possibilistic Inferential Model (IM).

Construction: The IM utilizes the relative likelihood function $R(x, \theta)$ . To ensure validity, it applies a "validification" step (a version of the probability-to-possibility transform) to generate a possibility contour $\pi_x(\theta)$ :
$\pi_x(\theta) = P_\theta \{ R(X, \theta) \leq R(x, \theta) \}$
Properties: This contour defines a possibility measure $\Pi_x$ . Crucially, this construction guarantees validity:
$\sup_{\theta \in T} P_\theta \{ \pi_X(\theta) \leq \alpha \} \leq \alpha$
This implies that the IM assigns small possibility values to the true parameter only rarely, ensuring that the associated confidence sets $C_\alpha(x) = \{ \theta : \pi_x(\theta) \geq \alpha \}$ have exact frequentist coverage. This output is a coherent imprecise probability (possibility measure) that avoids the false confidence phenomenon.

B. Inner Probabilistic Approximation

Recognizing that many practitioners prefer probabilistic outputs, the paper proposes extracting an inner probabilistic approximation from the possibilistic IM.

Definition: The goal is to find a probability distribution $Q_x$ that is "dominated" by the possibility measure ( $Q_x \preceq \Pi_x$ ) and is "maximally diffuse."
Characterization (Theorem 1): A probability $Q_x$ belongs to the credal set of $\Pi_x$ if and only if it can be represented as a mixture of kernels supported on the level sets (confidence sets) $C_\alpha(x)$ :
$Q_x(\cdot) = \int_0^1 K_\alpha^x(\cdot) M_x(d\alpha)$
where $M_x$ is a probability measure on $[0, 1]$ and $K_\alpha^x$ is supported on $C_\alpha(x)$ .
The "Inner" Solution: An inner probabilistic approximation $Q_x^*$ is defined as a member of this credal set that achieves equality in the coverage condition:
$Q_x^* \{ C_\alpha(x) \} = 1 - \alpha \quad \text{for all } \alpha \in [0, 1]$
This corresponds to taking the marginal distribution $M_x$ as Uniform $(0, 1)$ and the kernel $K_\alpha^x$ as a uniform distribution on the boundary $\partial C_\alpha(x)$ .
Sampling Strategy: A sample from $Q_x^*$ $Q_{x}^{*}$ can be drawn via a two-step procedure:
1. Sample a level $\alpha \sim \text{Unif}(0, 1)$ .
2. Sample a point $\theta$ uniformly from the boundary of the confidence set $C_\alpha(x)$ .

C. Computation

Since exact confidence set boundaries are often intractable, the paper adopts a computational strategy (from Martin 2026a) that approximates the IM contour using a "stitched" collection of Gaussian possibility measures. This involves finding a vector $\sigma$ such that a Gaussian ellipsoid $C_\alpha^\sigma(x)$ snugly bounds the true IM level set $C_\alpha(x)$ , allowing for efficient Monte Carlo sampling.

3. Key Contributions and Results

A. Theoretical Properties

Agreement with Invariant Models (Theorem 3): For statistical models invariant under a group of transformations (e.g., location-scale families), the proposed inner probabilistic approximation coincides with the standard no-prior Bayes solution based on the right Haar prior. This establishes that the new method recovers the "gold standard" solutions in cases where a consensus exists.
Asymptotic Efficiency (Theorems 4 & 5): The paper proves a version of the Bernstein–von Mises theorem for the possibilistic IM. As sample size $n \to \infty$ , the IM contour converges to a Gaussian possibility contour, and the inner probabilistic approximation converges to a Gaussian distribution with covariance matching the Cramér–Rao lower bound. Thus, the method is asymptotically efficient.
Probability Matching (Theorem 2): Credible sets constructed using ranking functions comonotone with the possibility contour (e.g., the likelihood ratio) are probability matching, meaning their Bayesian credibility equals their frequentist coverage.

B. Marginalization and Over-Confidence

The paper investigates the risks of marginalizing these distributions.

The Risk: Standard probabilistic marginalization of a joint confidence distribution often leads to over-confidence (anti-conservative intervals) for marginal parameters, particularly under non-linear transformations.
The Solution: The paper distinguishes between two approaches:
1. Extension-based Marginalization: Marginalizing the IM first (via the extension principle) and then approximating. This preserves validity.
2. Probabilistic Marginalization: Marginalizing the inner approximation directly.
Boundary-Surjectivity (Theorem 6): The paper shows that the marginal of an inner approximation is an inner approximation of the marginal IM if and only if the mapping is boundary-surjective. In cases where this fails (e.g., non-linear functions), the direct probabilistic marginal may be over-confident. The recommended strategy is to perform marginalization at the possibilistic level (or via profiling) first, then extract the inner approximation.

C. Empirical Illustrations

Behrens–Fisher Problem: The paper applies the method to the classic Behrens–Fisher problem (inference on the difference of means with unknown, unequal variances).
- Result: The inner probabilistic approximation of the profile-likelihood-based marginal IM yields a solution that is exact in coverage (unlike the Welch $t$ -test or Jeffreys prior Bayes) and more efficient (shorter expected interval length) than the conservative right Haar prior solution.
Gamma and Directional Data: Illustrations on Gamma models and directional data (von Mises distribution) demonstrate that the inner approximation closely matches the right Haar prior Bayes solution in invariant cases but corrects for over-confidence in non-invariant or marginal cases.

4. Significance and Claims

The paper claims to resolve the "most important unresolved problem in statistical inference" (Efron 2013): the use of Bayes's theorem without prior information.

Reliability First: The core claim is that by prioritizing reliability (calibration) through a possibilistic framework, one avoids the false confidence inherent in all existing no-prior probabilistic solutions.
Reconciliation: The "reIMagined" approach bridges the gap between frequentist validity and Bayesian-like reasoning. It provides a data-dependent probability distribution that:
- Has exact coverage properties (via the underlying IM).
- Is asymptotically efficient.
- Agrees with established solutions (right Haar priors) in invariant models.
- Avoids the need for arbitrary prior specification.
Practicality: The paper provides a concrete Monte Carlo algorithm for computing these approximations, making the theory applicable to complex problems like the Behrens–Fisher problem where traditional solutions are either conservative or lack consensus.

In summary, the paper argues that the path to reliable no-prior inference is not to find a "better" default prior, but to construct a possibilistic IM first and then extract a maximally diffuse inner probabilistic approximation from it. This yields a "posterior" that is both probabilistic in form and valid in frequentist performance.