Bayes with No Shame: Admissibility Geometries of Predictive Inference

This paper demonstrates that predictive inference is governed by four distinct, pairwise non-nested admissibility geometries—Blackwell risk dominance, anytime-valid supermartingales, marginal coverage, and Cesàro approachability—each offering a unique certificate of optimality and proving that admissibility is irreducibly relative to the chosen criterion rather than a universal property.

The Gist

Imagine you are a weather forecaster. Your job is to predict if it will rain tomorrow. You have a notebook, and every day you write down your prediction.

This paper asks a very deep question: What does it mean to be a "good" forecaster?

The authors, Nicholas Polson and Daniel Zantedeschi, argue that there isn't just one definition of "good." In fact, there are four completely different ways to be a champion, and being a champion in one way doesn't mean you are a champion in the others. They call this "Admissibility Geometries."

To make this easy to understand, let's use the metaphor of a Gym and Four Different Sports.

The Core Idea: "No Shame"

The paper uses a clever metaphor: Shame.

• If you use a strategy that is clearly worse than another available strategy, you should feel "shame." You are leaving points on the table.
• A "No-Shame" strategy is one where you cannot be beaten. No other method can do strictly better than you in every possible scenario.

The paper says: You can be "No-Shame" in four different ways, but you can't be "No-Shame" in all four at once.

Here are the four "Sports" (Geometries) and their champions:

---

1. The Bayesian Athlete (Blackwell Admissibility)

The Goal: Be the best on average, based on a specific "hunch" (a prior belief).
The Metaphor: Imagine you are playing a game where you have a secret map (a prior belief) about where the treasure is. The Bayesian is the player who uses that map perfectly.

• How they win: They minimize their "regret" (loss) based on their map.
• The Certificate: A Supporting Hyperplane. Think of this as a judge holding up a sign that says, "Given your specific map, no one could have done better than you."
• The Catch: If your map is wrong, or if you don't have a map at all, this "win" doesn't count.
• Real-world example: A weather forecaster who assumes "it rains 30% of the time" and updates their prediction perfectly based on that assumption.

2. The Safe Gambler (Anytime-Valid Admissibility)

The Goal: Never get caught cheating, no matter when you stop the game.
The Metaphor: Imagine you are betting on coin flips. You want to prove the coin is unfair. The Safe Gambler uses a special "E-process" (a running score).

• How they win: They ensure that their score never grows too fast by luck. If they stop betting at any random moment (even if they peek at the results early), they are still safe from false alarms.
• The Certificate: A Non-Negative Supermartingale. Think of this as a "safety net" that guarantees you won't go broke due to bad luck, even if you stop whenever you want.
• The Catch: They aren't trying to predict the weather perfectly; they are just trying to prove a point without getting caught lying.
• Real-world example: A clinical trial for a new drug. You want to stop the trial early if the drug works, but you can't let the data peeking trick you into thinking it works when it doesn't.

3. The Crowd Surfer (Marginal Coverage Validity)

The Goal: Be right often enough, on average, without caring about the specific details.
The Metaphor: Imagine you are throwing a net to catch fish. You don't care if you catch the exact right fish every time. You just want to make sure that 95% of the time, the fish you think you caught is actually in the net.

• How they win: They use Conformal Prediction. They look at the crowd (exchangeable data) and say, "Based on how the crowd behaves, I'm 95% sure the answer is in this box."
• The Certificate: An Exchangeability Rank. It's like a popularity contest. "95% of the time, the answer falls in this range."
• The Catch: They don't care about the probability of the event, just the coverage. They might give a huge, useless net (e.g., "It will rain or not rain") just to be safe.
• Real-world example: A self-driving car saying, "I am 95% sure the pedestrian is within these 5 feet," even if it doesn't know exactly where they are.

4. The Marathon Runner (CAA Admissibility)

The Goal: Get it right in the long run, even if you stumble in the short run.
The Metaphor: Imagine a runner who trips and falls a lot in the first mile, but by the end of the marathon, their average speed is perfect. They use Defensive Forecasting.

• How they win: They use a "fixed-point" trick. They don't need a map (prior) or a safety net. They just keep adjusting their strategy so that, over a long time, their mistakes average out to the best possible performance.
• The Certificate: A Cesàro Steering Argument. It's like saying, "I might be wrong today, but if you watch me for a year, I will be perfect on average."
• The Catch: They might be terrible at predicting the next coin flip, but they are great at the long-term average.
• Real-world example: A stock trading bot that makes wild guesses every day but eventually learns the market trend so well that its long-term profit matches the best possible strategy.

---

The Big Reveal: The "Criterion Separation"

The paper's most important finding is that these four champions cannot be the same person.

• The Bayesian (who needs a map) is not the Safe Gambler (who needs a safety net).
• The Crowd Surfer (who just wants a net) is not the Marathon Runner (who cares about long-term averages).
• The Safe Gambler is not the Crowd Surfer.

Why? Because they are playing on different fields with different rules.

• If you try to be the Bayesian, you might fail the "Safe Gambler" test because you peeked at the data.
• If you try to be the Crowd Surfer, you might fail the "Bayesian" test because your net is too wide and you didn't minimize your error.

The "Shame" Lesson

The authors say: Don't feel shame if your method isn't perfect in every way.

• If you are a Bayesian, you are "No-Shame" regarding your map.
• If you are a Safe Gambler, you are "No-Shame" regarding your safety.
• If you are a Crowd Surfer, you are "No-Shame" regarding your coverage.

But, if you try to use a Bayesian method to solve a "Safe Gambler" problem, you will feel shame because you broke the rules of the game.

Summary in One Sentence

There is no single "best" way to predict the future; there are four different ways to be "good," and you have to pick the one that fits the specific game you are playing, because you can't win all four games at the same time.

Technical Summary

Here is a detailed technical summary of the paper "Bayes with No Shame: Admissibility Geometries of Predictive Inference" by Nicholas G. Polson and Daniel Zantedeschi.

1. Problem Statement

The paper addresses a fundamental fragmentation in statistical inference and machine learning. Four distinct research programs—Bayesian decision theory, safe anytime-valid inference (e-processes), conformal prediction, and online learning/defensive forecasting—all claim to provide "optimal" predictive procedures. However, these claims rely on different definitions of optimality, different performance metrics, and different structural constraints.

The core problem is that there is no unified framework to compare these methods. A procedure that is optimal under one criterion (e.g., minimizing Bayesian risk) may be strictly dominated under another (e.g., failing to control Type-I error at arbitrary stopping times or failing to provide valid coverage). The authors ask: Are these notions of optimality compatible, or are they structurally irreconcilable?

2. Methodology and Framework

The authors develop a unified decision-theoretic framework extended to allow for infinite risk (essential for proper scoring rules like log loss). They define a statistical decision problem via a tuple $(\Theta, A, L, \mathcal{X}, P)$ and analyze the geometry of the risk set $\mathcal{R}$, which maps decision rules to their risk vectors.

The methodology involves:

1. Geometric Characterization: Defining admissibility as lying on the lower boundary ($\partial_- \mathcal{R}$) of the risk set.
2. Constraint-Based Optimization: Framing all four criteria as instances of a Constrained Bayes Problem:
   $$\min_{\delta \in \mathcal{D}} \int R(\theta, \delta) d\Pi(\theta) \quad \text{subject to} \quad \delta \in \mathcal{F}$$
   where $\mathcal{F}$ is a feasibility set defined by the specific validity requirement of the criterion.
3. Separation Theorems: Constructing specific counterexamples (primarily in the Bernoulli and Gaussian models) to prove that the classes of admissible procedures under the four criteria are pairwise non-nested.
4. Martingale Analysis: Investigating the role of martingale coherence (self-consistency) as a bridge between criteria, showing it is necessary for some but insufficient for others.

3. The Four Admissibility Geometries

The paper isolates four distinct "geometries" of optimality, each with a unique certificate of optimality:

| Geometry | Criterion | Feasible Set ($\mathcal{F}$) | Certificate of Optimality | Key Characteristic |
| :--- | :--- | :--- | :--- :--- |
| B: Blackwell | Risk Dominance | $\mathcal{D}$ (Unconstrained) | Supporting Hyperplane Prior | Pointwise optimality; no rule has uniformly lower risk. Equivalent to Bayes optimality. |
| A: Anytime-Valid | Type-I Error Control | $\mathcal{C}_{AV}$ (e-processes) | Non-negative Supermartingale | Validity at every stopping time $\tau$. Requires $E_\tau \le 1$. |
| C: Coverage | Marginal Validity | $\mathcal{C}_{Cov}$ (Prediction Sets) | Exchangeability Rank | Guarantees $P(Y \in \hat{C}) \ge 1-\alpha$ under exchangeability. |
| D: CAA | Cesàro Approachability | $\mathcal{C}_{CAA}$ (Approachability) | Fixed-Point / Minimax | Time-averaged risk converges to $\partial_- \mathcal{R}$; no per-round prior required. |

4. Key Contributions and Results

A. The Criterion Separation Theorems

The central theoretical result is that the four classes of admissible procedures ($B, A, C, D$) are pairwise non-nested.

• $B \not\subseteq A$ and $A \not\subseteq B$: A Bayes-optimal point predictor (Blackwell admissible) is not an e-process (fails anytime-validity constraints). Conversely, an e-process does not minimize a proper scoring rule.
• $B \not\subseteq C$ and $C \not\subseteq B$: A Bayes point predictor does not produce a prediction set with valid coverage. A conformal set (valid coverage) does not minimize any proper scoring rule.
• $A \not\subseteq C$ and $C \not\subseteq A$: E-processes do not provide coverage sets; conformal sets do not control Type-I error at stopping times.
• $D$ (CAA) Separation: Defensive forecasters (CAA-admissible) achieve long-run calibration via fixed-point arguments but are not Bayes-optimal at any finite sample size and do not satisfy the other criteria.

B. The Role of Martingales

The paper clarifies the status of martingale coherence:

• Necessary for Blackwell: Every Bayes rule is a martingale under its predictive measure.
• Necessary and Sufficient for Anytime-Valid: Admissibility in the e-process class is equivalent to being a non-negative martingale.
• Not Sufficient for Blackwell: The Plug-in MLE ($\hat{p}_n = S_n/n$) is a martingale under its own predictive law but is strictly dominated by the Bayes rule (Beta(1/2, 1/2)) under log loss because it assigns zero probability to realizable events, leading to infinite risk.
• Not Necessary for Coverage/CAA: Conformal prediction and defensive forecasting do not require martingale properties.

C. The "No-Shame" Principle

The authors introduce the metaphor of "shame" to describe inadmissibility. A procedure is "shameless" (admissible) if it lies on the lower boundary of the risk set relative to a specific criterion.

• Moral Pluralism: Just as moral frameworks can be internally coherent but mutually irreconcilable, statistical optimality is irreducibly criterion-relative. A procedure can be "shameless" under Blackwell risk dominance but "shameful" (dominated) under coverage validity, and vice versa.

D. Constructive vs. Cesàro Admissibility

The paper distinguishes between:

• Constructive Admissibility: Every action is Bayes-optimal with respect to a specific prior (requires a "witness" prior at every step).
• Cesàro Admissibility: Only the time-averaged risk converges to the boundary (e.g., Defensive Forecasting). This allows for strategies that reach the optimal frontier in the limit without being optimal at any finite step.

5. Significance and Implications

1. Unified Design Principle (Constrained Bayes): The paper proposes that all four geometries can be viewed through a single lens: Constrained Bayes Optimization. The analyst first specifies a validity constraint ($\mathcal{F}$) relevant to their domain (e.g., "must control Type-I error" or "must have valid coverage"), and then minimizes Bayesian risk within that constraint.
2. Evaluation of LLMs and Forecasters: The results explain why "calibration" (martingale coherence under the model) is insufficient for admissibility. A Large Language Model (LLM) can be perfectly calibrated under its own predictive distribution yet be strictly dominated by a regularized (Bayesian) alternative that shrinks toward a prior.
3. Clinical Trials and Sequential Testing: The separation clarifies that an e-process (safe anytime-valid) is not necessarily the most powerful Neyman-Pearson test, and a powerful test is not necessarily safe for continuous monitoring. Practitioners must choose the geometry based on the operational constraint (stopping time flexibility vs. power).
4. Conformal Prediction: The paper validates that conformal wrappers provide a distinct type of validity (marginal coverage) that is orthogonal to the loss-minimization properties of the underlying black-box predictor.

Conclusion

"Bayes with No Shame" argues that there is no single "best" predictor. Instead, there are four distinct, geometrically incompatible geometries of optimality. The choice of an optimal procedure is not a matter of approximation or "taste," but a structural consequence of the validity constraints imposed by the user. The paper provides the mathematical machinery to navigate these trade-offs, ensuring that practitioners do not deploy procedures that are inadmissible relative to their own specified goals.