📊 statistics

From Bayes' Rule to Bayes Rules: Optimal Information Processing and Axiomatic Foundations Beyond Probability

This paper establishes a canonical update rule for possibilistic inference, derived from both information-conservation and axiomatic perspectives, which combines prior and likelihood via product and supremum normalization while introducing a learning-rate parameter to control epistemic strength.

Original authors: Jeremie Houssineau, Badr-Eddine Chérief-Abdellatif

Published 2026-07-10

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Jeremie Houssineau, Badr-Eddine Chérief-Abdellatif

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery. You have a hunch about who the culprit is (your prior belief), and you find a new clue at the scene of the crime (your data). In the world of standard statistics, you combine these two using a famous recipe called Bayes' Rule to get your final answer.

But what if your hunch isn't a precise probability (like "70% sure")? What if it's more like a fuzzy feeling, a "possibility"? Maybe you think the culprit could be the butler, or the gardener, or the cook, but you don't have exact numbers. This is where Possibility Theory comes in. It's a way of handling uncertainty that doesn't rely on adding up numbers, but on finding the "maximum" or "supremum" of what's possible.

This paper, titled "From Bayes' Rule to Bayes Rules," asks a big question: If we stop using standard probabilities and start using these fuzzy "possibility" maps, is there still a single, perfect way to update our beliefs?

The Big Discovery: One Rule to Rule Them All

The authors, Jeremie Houssineau and Badr-Eddine Chérief-Abdellatif, argue that yes, there is. Even in this fuzzy world, there is a "canonical" (the one true) way to update your beliefs.

They found this rule by looking at the problem from two completely different angles, like two detectives solving the same case with different tools:

The "Information Conservation" Detective: This detective believes that when you combine your old hunch with new clues, you shouldn't accidentally invent new facts or lose old ones. You should just rearrange what you already know. By insisting that no information is created or destroyed, they derived a specific formula.
The "Logical Coherence" Detective: This detective asks, "If I update my beliefs step-by-step, or if I look at a smaller part of the puzzle, does my logic still make sense?" By setting up a set of strict logical rules (axioms) that any sensible update must follow, this detective arrived at the exact same formula.

The Formula:
The rule they both found is surprisingly simple. To get your new belief (the posterior), you multiply your old hunch (the prior) by the new clue (the likelihood), and then you divide by the biggest number in the result to make sure everything fits in the "1 to 0" scale.

In math-speak, it looks like this:
$\text{New Belief} = \frac{\text{Old Hunch} \times \text{New Clue}}{\text{The Biggest Number in the Result}}$

This is the "sup-normalised prior-likelihood product." It's the fuzzy-world cousin of the classic Bayes' Rule.

The "Magic Knob" (The Learning Rate)

Here is the twist that the paper is very careful about. In this fuzzy world, there is a special "knob" or parameter (let's call it $w$ ) that controls how strong your belief is.

What the paper proves: The rule works perfectly for any setting of this knob.
What the paper rules out: You cannot figure out the perfect setting for this knob just by looking at the data. The paper explicitly states that this strength parameter is not identifiable from the evidence alone.

Think of it like this: If you are trying to tune a radio, the data tells you what song is playing, but it doesn't tell you how loud the volume should be. You have to decide the volume based on how much you trust the radio station in the first place. The paper argues that trying to "learn" this volume knob from the data alone is a dead end; it's a choice you make about your own confidence, not a fact hidden in the numbers.

What This Paper Says "No" To

The authors are very clear about what this rule is not:

It is not a way to invent new information. If your update rule makes your belief more certain than the data and your old hunch justify, you have "created" information, which the paper says is a mistake.
It is not a way to lose information. If your update makes you less certain than you should be, you have "lost" information.
It is not a method that requires you to know the exact "true" probability of how the data was generated. In fact, this whole system is designed for when you don't have those exact probabilities.

How Sure Are They?

The authors are extremely confident in their findings, but they are precise about the scope.

Proven: They have mathematically proved (using rigorous logic and axioms) that if you want to conserve information or stay logically consistent in a possibility-based world, you must use this specific update rule. It's not a guess; it's a derivation.
Not a Simulation: They didn't just run computer simulations to see if it works; they built a logical framework that forces this rule to be the only answer.
Not a "Solved" Problem for Everything: They admit that while they found the rule, they haven't solved the problem of how to pick the perfect "volume knob" ( $w$ ) for every situation. They suggest that this knob needs to be calibrated (adjusted) based on external goals, like making sure your predictions cover the right amount of ground, rather than being learned automatically from the data.

The Takeaway for a Curious Teenager

Imagine you are building a robot that learns from the world. If you tell the robot to use standard math, it adds up probabilities. But if the world is messy and you can't be sure of the numbers, you tell the robot to use "possibility" math.

This paper tells the robot: "Here is the only way to update your brain without lying to yourself or forgetting what you knew. Multiply your old guess by the new clue, and normalize it. And by the way, don't try to guess how 'strong' your belief should be just by looking at the clues; you have to decide that strength yourself."

It's a new set of rules for a fuzzy world, proving that even when things aren't perfectly clear, there is still a perfect way to think about them.

Technical Summary: From Bayes' Rule to Bayes Rules

Problem Statement
Standard Bayesian inference relies on probability theory, where uncertainty is additive and updated via Bayes' rule. However, both classical statistics and modern machine learning frequently depart from this framework, utilizing loss-based updates, tempered posteriors, or evidence-fusion schemes where uncertainty is not naturally additive. A fundamental question remains: what makes Bayes' rule the "right" information processing rule, and how does this depend on the mathematical representation of information? Specifically, when uncertainty about a fixed parameter is represented not by a probability distribution but by a possibility function (the maxitive analogue of a probability distribution), what is the principled update rule?

Methodology
The paper approaches this problem by treating inference as information processing. It derives the canonical update rule for possibilistic inference from two complementary foundational viewpoints, mirroring classical probabilistic derivations but adapting them to the structure of possibility theory.

View 1: Optimal Information Processing (Information Conservation)
- Framework: The authors reformulate the Information Conservation Principle (ICP), traditionally associated with Zellner [1988] and scalar cross-entropies, into a possibilistic setting.
- Key Mechanism: Instead of compressing information into a scalar (like entropy), the paper utilizes the pointwise partial order inherent to possibility functions ( $f \preceq g \iff \forall \theta, f(\theta) \leq g(\theta)$ ).
- Derivation: By introducing a pointwise information score $S$ satisfying order-consistency, normalisation, and additivity under independent conjunction (product t-norm), the authors derive a canonical logarithmic information map: $I[f](\theta) = -\log f(\theta)$ .
- Conservation: An Information Processing Rule (IPR) is defined as optimal if it induces no information loss and no information creation. This is formalized via an information imbalance functional $\Delta(\text{IPR})$ , which is a function over the parameter space rather than a scalar. The condition $\Delta(\text{IPR}) = 0$ (pointwise equality) uniquely identifies the update rule.
View 2: Axiomatic Approach (Coherence)
- Framework: The authors propose a possibilistic analogue of the coherence axioms for generalised Bayesian inference established by Bissiri et al. [2016].
- Axioms: The update rule $\psi[\ell, \pi]$ $ψ [ℓ, π]$ (mapping a prior $\pi$ $π$ and loss $\ell$ $ℓ$ to a posterior) must satisfy:
  - Sequential coherence (A2.1).
  - Restriction compatibility (A2.2).
  - Monotonicity in loss (A2.3).
  - Invariance to vacuous losses and additive shifts (A2.4, A2.5).
- Derivation: Unlike the probabilistic case where normalisation is an integral, the possibilistic derivation requires establishing that maxitive restriction preserves pairwise odds and that binary updates cohere on triples. These axioms force the posterior to take a specific functional form.

Key Contributions and Results

The Canonical Update Rule:
Both derivations converge on the same unique update rule, termed the possibilistic Bayes' rule:
$\pi(\theta | y) = \frac{\pi(\theta) L(\theta | y)}{\sup_{\vartheta \in \Theta} \pi(\vartheta) L(\vartheta | y)}$
where $\pi(\theta)$ is the prior possibility function and $L(\theta | y)$ is the likelihood (or evidence).
- If the input is a sampling distribution $p_\theta(y)$ , $L$ is the likelihood.
- If the input is a generic loss $\ell(\theta, y)$ , $L(\theta | y) \propto \exp(-w \ell(\theta, y))$ , where $w \geq 0$ is a learning-rate parameter.
Role of the Learning Rate ( $w$ ):
The paper clarifies that the parameter $w$ (controlling epistemic strength) is exogenous to both derivations.
- In View 1, $w$ is an encoding choice when transforming a loss into an evidence possibility function.
- In View 2, $w$ is a constant left undetermined by the coherence axioms.
- Result: The strength $w$ is not identifiable from the normalising evidence alone. Maximising the possibilistic evidence merely drives admissible strengths to their smallest values. Thus, $w$ represents a modelling choice regarding epistemic strength, not a data-identifiable quantity.
Information Loss and Creation:
By utilizing the pointwise partial order, the paper distinguishes between information loss and creation as genuine, observable phenomena, unlike in the probabilistic setting where information is a scalar summary.
- An IPR creates no information if the posterior is everywhere dominated by the Bayes update ( $g \preceq \pi(\cdot|y)$ ).
- An IPR loses no information if the posterior dominates the Bayes update ( $g \succeq \pi(\cdot|y)$ ).
- The Bayes rule is the unique IPR that simultaneously avoids both loss and creation.
Connection to Existing Work:
- The derived rule aligns with the sup-normalised prior-likelihood product proposed in recent imprecise probabilistic literature (e.g., Martin [2026]).
- The information imbalance function corresponds to the integrand in the maxitive Donsker–Varadhan duality (Singh et al. [2025]), showing that the Bayes rule is the unique common optimiser of dual consistency bounds.

Significance and Claims
The paper claims to establish that Bayes' rule is not a single formula but a member of a family of rules indexed by the underlying calculus of information (additive vs. maxitive) and the nature of the data-to-parameter link.

Foundational Justification: The paper argues that the sup-normalised product is not merely a heuristic or a convenient choice for combining a likelihood with partial prior information; it is forced by fundamental principles of information conservation and coherence in a possibilistic setting.
Clarification of Generalised Bayes: It resolves ambiguity regarding the learning rate in generalised Bayesian updates by demonstrating that in the possibilistic framework, this parameter is intrinsically non-identifiable from data, serving instead as a calibration of epistemic strength.
Scope: The work is foundational and axiomatic. It does not propose new empirical algorithms or extensive experimental comparisons but rather provides the theoretical underpinning for why the sup-normalised update is the canonical choice for possibilistic inference. It suggests that this framework naturally extends to continuous parameter spaces without the measure-theoretic complications often required in probabilistic extensions, as possibility functions are defined pointwise.

In summary, the paper demonstrates that when uncertainty is represented by possibility functions, the "right" way to process information leads inevitably to a sup-normalised update rule, with the learning rate acting as an external calibration parameter rather than a learnable statistic.