📊 statistics

When Is Generalized Bayes Bayesian? A Decision-Theoretic Characterization of Loss-Based Updating

This paper provides a decision-theoretic characterization distinguishing belief posteriors from decision posteriors, demonstrating that loss-based updating coincides with standard Bayes only when the loss is the negative log-likelihood, while establishing that generalized Bayes emerges as the optimal rule for decision posteriors under sequential coherence and separability via an entropy-penalized variational representation.

Original authors: Kenichiro McAlinn, Kōsaku Takanashi

Published 2026-02-03

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Kenichiro McAlinn, Kōsaku Takanashi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Question: Is it a "Belief" or a "Strategy"?

Imagine you are trying to figure out the best route to work.

The "Belief" approach (Standard Bayes): You have a map (a model) and you know the rules of traffic (probability). You update your belief about the traffic based on new data. If you see a red light, you believe the road is blocked. This is a conditional belief: "Given what I know, this is what I think is true."
The "Strategy" approach (Generalized Bayes/Loss-based): You don't have a perfect map. Instead, you have a goal: "Get to work with the least stress." You try different routes and assign a "pain score" (loss) to each one based on how bad the traffic was. You then pick a route strategy that minimizes this pain. This is a decision rule: "Given my goal, this is the best thing to do."

The paper's main point: For years, people have been using the "Strategy" approach (using pain scores instead of traffic maps) but calling it a "Belief." They have been using the same words (like "posterior," "Bayes factor," and "evidence") for both. The authors say: Stop doing that. They are fundamentally different things, and mixing them up leads to confusion and wrong conclusions.

1. The "Magic Switch" (When does Strategy become Belief?)

The authors ask: When can we treat a "Strategy" (Loss-based) exactly like a "Belief" (Standard Bayes)?

They found a specific "magic switch."

The Rule: A strategy is only a true "Belief" if your "pain score" (loss) is mathematically identical to the negative log of a probability map (likelihood).
The Analogy: Imagine you are grading students.
- If you grade them based on a strict rubric that matches a specific, pre-existing theory of intelligence, your grades represent a belief about their intelligence.
- If you grade them based on a custom rubric you made up to minimize classroom chaos, your grades represent a strategy to keep the class calm.
- The Catch: You can only call your custom rubric a "belief" if it happens to match the strict theory perfectly. If it doesn't, it's just a strategy.

Conclusion: Unless your "pain score" is exactly the negative log-likelihood, you are not doing Bayesian belief updating. You are doing decision-making optimization.

2. The "Normalizing Constant" Trap (Why "Evidence" is Fake here)

In standard statistics, there is a number called the "Marginal Likelihood" (or the "Normalizing Constant"). People use this number to say, "Model A has more evidence than Model B." It's like a scorecard for truth.

The paper shows that in the "Strategy" world, this scorecard is broken.

The Analogy: Imagine you are comparing two restaurants.
- Belief World: You have a fixed menu. The price of the meal is the "evidence." If the price changes, the value changes.
- Strategy World: You are comparing restaurants based on a "hunger score."
  - Restaurant A gives you a score of 10.
  - Restaurant B gives you a score of 8.
  - But, you can add a "service fee" of $5 to both scores without changing which restaurant you prefer.
  - Restaurant A is now 15, Restaurant B is 13.
  - The difference (the "evidence") is now different, even though your choice of restaurant hasn't changed at all.

The Problem: In the "Strategy" world, you can add arbitrary numbers to your pain scores (as long as they are the same for all options) without changing your decision. However, this changes the "Normalizing Constant" (the scorecard). Therefore, you cannot use that scorecard to claim one model is "more true" than another. It's just an artifact of how you wrote the math.

The Solution: Don't use the scorecard. Instead, look at the outcome. Did the strategy actually get you to work faster? Use predictive performance (did it work?) rather than theoretical evidence (does the math look pretty?).

3. The "Randomness" Problem (Why you need a special kind of preference)

In standard decision theory (von Neumann-Morgenstern), if you are perfectly rational, you should never randomize. If Option A is better than Option B, you pick A 100% of the time. You don't flip a coin.

The Paradox: Generalized Bayes often produces a "posterior" that is a mix of many options (a probability distribution). It says, "There is a 30% chance it's A, 70% chance it's B."
The Paper's Finding: If you are a standard rational agent, you shouldn't be doing this. You should just pick the single best option.
The Fix: To justify using a mix (a randomized decision rule), you must have non-linear preferences. You must prefer uncertainty or diversity for its own sake.
- Analogy: A standard rational person eats only the tastiest apple. A person with "non-linear preferences" might eat a mix of apples because they like the variety or because they want to avoid the risk of the one "best" apple being rotten.
- The paper argues that Generalized Bayes works because it acts like this second type of person: it uses "entropy" (a penalty for being too certain) to keep options open.

4. The "Coherence Book" (What you can and cannot claim)

The authors created a "Coherence Book" (a cheat sheet) to tell researchers what they are allowed to say depending on which method they use.

Feature	Belief Posterior (Standard Bayes)	Decision Posterior (Generalized Bayes)
What is it?	A belief about what is true.	A rule for what to do.
The "Prior"	A belief about the world.	A baseline or starting point for the rule.
Evidence	You can calculate "Bayes Factors" to prove one model is better.	No. You cannot calculate Bayes Factors. You must compare models by how well they predict the future.
The "Learning Rate"	Fixed by the math of the model.	Must be chosen by you. It's a knob you turn to control how much you trust the data vs. your baseline.
Consistency	If you learn new info, your beliefs update logically.	If you update step-by-step, you get the same result as updating all at once.

Summary: The "Design Recipe"

The paper concludes that Generalized Bayes is a powerful tool, but it is not "Bayes without a model." It is "A model of how you choose under a specific loss."

If you want to use it, you must be honest about:

The Loss: What are you trying to minimize? (Stress? Cost? Error?)
The Baseline: What is your starting point?
The Calibration: How did you set the "temperature" (learning rate)?
The Goal: Are you trying to find the truth (Belief) or make the best decision (Strategy)?

The Bottom Line: If you are using Generalized Bayes, stop pretending you are calculating "probabilities of truth" unless your math is exactly the same as standard probability. Instead, embrace it as a robust, flexible tool for making decisions in a messy world where perfect models don't exist. Just don't call it "evidence" in the old sense.

Technical Summary: "When Is Generalized Bayes Bayesian? A Decision-Theoretic Characterization of Loss-Based Updating"

Problem Statement

The literature on "generalized Bayes" (including Gibbs posteriors, quasi-posteriors, and loss-based updating) has grown significantly, offering a framework to update distributions by exponentiating cumulative loss rather than likelihoods. While these methods are widely used in robust inference, econometrics, and inverse problems, the terminology "posterior" creates operational ambiguity. Standard Bayesian posteriors represent conditional beliefs derived from a joint probability model (Savage/Anscombe-Aumann framework), supporting specific interpretations like betting odds and evidential use of marginal likelihoods. In contrast, loss-based updates are often driven by user-chosen loss functions without an underlying sampling model. The paper addresses the critical question: Under what conditions does a loss-based update retain the coherence properties of a Bayesian belief posterior, and when must it be interpreted strictly as a decision-theoretic object?

Methodology and Framework

The authors employ a decision-theoretic approach to distinguish between two distinct objects often conflated in the literature:

Belief Posteriors: Conditional beliefs about uncertain states derived from preferences over acts, satisfying dynamic consistency axioms (Savage, 1954; Anscombe and Aumann, 1963).
Decision Posteriors: Randomized decision rules (distributions over actions/parameters) chosen to optimize a preference functional after observing data.

The paper formalizes loss-based updating as an operator $\Psi$ that maps a baseline distribution $\pi$ and a cumulative loss $L_n$ to a rule $q(\cdot|x)$ . The analysis relies on axioms regarding sequential coherence (batching), separability across independent subproblems, and the structure of preferences over randomized rules.

Key Contributions and Results

1. The Bayesian Equivalence Divider (Theorem 1)
The paper establishes a sharp necessary and sufficient condition for a generalized Bayes posterior to be interpretable as an ordinary Bayesian belief posterior.

Result: A loss-based posterior $q(\theta|x) \propto \pi(\theta)\exp\{-\eta \ell(\theta; x)\}$ coincides with a Bayesian posterior derived from a joint model if and only if the loss function $\ell(\theta; x)$ is, up to a data-only term $c(x)$ and a scale factor, the negative log-likelihood:
$\ell(\theta; x) = -\frac{1}{\eta} \log p_\theta(x) + c(x)$
Implication: If the loss is not the negative log-likelihood (e.g., criterion-function losses in econometrics), the resulting distribution cannot be interpreted as a conditional belief state. It must be treated as a decision posterior. The paper provides a diagnostic (Corollary 1) based on the "partition function" $A(\theta) = \int \exp\{-\eta \ell(\theta; x)\} d\mu(x)$ ; if $A(\theta)$ depends on $\theta$ , the update is not Bayesian.

2. Decision-Theoretic Foundation for Generalized Bayes (Corollary 2)
For cases where the loss is not log-likelihood, the paper provides a positive characterization of generalized Bayes as an optimal randomized decision rule.

Methodology: The authors assume a decision maker optimizes a variational functional trading off expected loss against a deviation cost from a baseline $\pi$ .
Key Axiom: Separability across independent subproblems (Axiom A4). If the action space and baseline factorize, and losses are additive, the optimal rule must factorize.
Result: Under sequential coherence and separability, the unique deviation cost that satisfies these axioms is the Kullback-Leibler (KL) divergence (up to scale). Consequently, the optimal rule is the generalized Bayes (Gibbs) form:
$q^*(\cdot|x) = \arg\min_{q} \left\{ \int \ell(\theta; x) q(d\theta) + \frac{1}{\eta} D_{KL}(q \parallel \pi) \right\}$
Significance: This justifies generalized Bayes not as "Bayes without a model," but as the optimal rule for a decision maker with specific preferences over randomized rules, where the learning rate $\eta$ is a structural parameter of the preference specification, not a fixed property of a likelihood.

3. Non-Canonicity of Marginal Likelihood and Bayes Factors (Section 4)
The paper demonstrates that the "generalized marginal likelihood" (the normalizing constant $Z(x)$ of the Gibbs posterior) is not a canonical evidential quantity outside the belief-posterior regime.

Result: The decision posterior $q(\cdot|x)$ is invariant to adding any data-only term $c(x)$ to the loss. However, the normalizing constant $Z(x)$ rescales by $e^{-\eta c(x)}$ .
Implication: Bayes factors constructed from generalized marginal likelihoods are ill-defined without additional structure, as they depend on arbitrary conventions (e.g., centering of empirical likelihoods). The paper proposes shift-invariant predictive scoring (e.g., prequential log predictive density) as the canonical alternative for model comparison in the decision-posterior regime.

4. Degeneracy of Linear Preferences (Section 5)
The paper addresses the interpretation of randomization in loss-based posteriors.

Result: If preferences over randomized decision rules satisfy the standard von Neumann-Morgenstern (vNM) independence axiom, the optimal rule collapses to a point mass (degenerate) on the best pure action.
Implication: Non-degenerate generalized posteriors (which are common in practice) require nonlinear preferences over decision rules (e.g., entropy penalties). Randomization is not merely a tie-breaking mechanism but a structural feature of the decision maker's risk/regularization preferences.

Significance and "Coherence Book"

The paper concludes by providing a "coherence book" (Table 1) that itemizes which Bayesian properties survive in the decision-posterior regime and which do not:

Belief Posteriors: Support exchangeability, the likelihood principle, and well-defined marginal likelihoods.
Decision Posteriors: Support sequential coherence (operator consistency) and optimality under specified loss, but do not support exchangeability as a belief constraint, the likelihood principle, or canonical evidential summaries like Bayes factors.

Practical Implications:

Learning Rate Calibration: In the decision-posterior regime, the learning rate $\eta$ is not fixed by a probabilistic model. It is a structural component that must be explicitly calibrated (e.g., via loss-likelihood bootstrap or safe learning-rate selection) to ensure robustness or coverage.
Reporting Standards: The authors recommend that practitioners using generalized Bayes explicitly report the loss function, the calibration method for $\eta$ , the interpretation of the output (belief vs. decision), and the model comparison criterion used (favoring predictive scoring over $Z(x)$ ).

Conclusion
The paper clarifies that generalized Bayes is a coherent mechanism for selecting randomized decision rules under explicit loss and baseline specifications. It recovers standard Bayesian belief updating only in the specific case where the loss is the negative log-likelihood. Outside this case, the method is a decision-theoretic regularization tool, and its properties (such as the lack of canonical evidence) must be understood within that framework rather than through the lens of classical belief coherence.