📊 statistics

Subjective Risk Decomposition: A New View for Uncertainty Quantification

This paper proposes a novel framework for uncertainty quantification where epistemic and aleatoric uncertainty measures are derived as consequences of decomposing a subjective risk based on a strictly proper loss, thereby unifying existing metrics and establishing a foundation for a learning-theoretic approach to UQ.

Original authors: Raghad Alamri, Michele Caprio, Gavin Brown

Published 2026-07-17

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Raghad Alamri, Michele Caprio, Gavin Brown

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Mystery of the Crystal Ball

Imagine you are trying to predict the future. Maybe you're guessing the score of a soccer match, the weather for tomorrow, or what a friend will say next. In the world of computer science, specifically in a field called Machine Learning, we build digital "crystal balls" (models) to make these guesses. But here's the tricky part: a model can be confident and still be wrong. That's why scientists care deeply about Uncertainty Quantification. It's not just about getting the answer right; it's about knowing how sure the model is.

To understand this paper, we need to know about two main types of "not knowing." First, there's Aleatoric uncertainty. Think of this as the noise in the system—the things you just can't control, like a sudden gust of wind changing a soccer ball's path. It's the "irreducible" messiness of the real world. Second, there's Epistemic uncertainty. This is the "I don't know what I don't know" kind. It's the uncertainty that comes from the model itself, perhaps because it hasn't seen enough data or is confused about the rules. For a long time, scientists have argued fiercely over how to measure these two types of uncertainty, trying to write down strict rules (axioms) to define them.

The Paper's Big Idea: Stop Arguing, Start Decomposing

This paper, titled "Subjective Risk Decomposition," suggests that all this arguing might be missing the point. The authors, Raghad Alamri, Michele Caprio, and Gavin Brown, propose a new way to look at the problem. Instead of trying to define uncertainty as a mysterious, standalone object, they suggest that uncertainty is actually just a consequence of how we choose to evaluate our models.

Imagine you are a teacher grading a student's essay. You have a specific rubric (a set of rules) for how to give points. If you change the rubric, the grade changes, even if the essay stays the same. The authors argue that "uncertainty" is like that grade. It's not a magical number floating in the air; it's the result of a specific calculation based on a specific "rubric" (which they call a strictly proper loss).

The paper introduces a concept called Subjective Risk. In simple terms, this is a way of asking: "If the model believes the world is a certain way, how bad would it be if the real world turned out differently?" The authors show that if you take this "Subjective Risk" and break it apart (decompose it), the pieces you get are exactly the different types of uncertainty scientists have been trying to define for years.

How the Magic Trick Works

The authors use a mathematical tool called a Bregman divergence (think of it as a special kind of ruler that measures the distance between two different ways of seeing the world). When they apply this ruler to the "Subjective Risk," it naturally splits into three distinct parts:

Bias (The Systematic Error): This is the part where the model's average belief is just plain wrong compared to reality. It's like a compass that always points slightly north instead of true north. The paper argues this isn't "uncertainty" in the way we usually think; it's just a mistake in the model's grounding.
Variance (The Epistemic Part): This is the "I'm confused" part. It measures how much the model's predictions jump around when it sees slightly different data. If the model is a jumpy, nervous student who gives a different answer every time you ask, this number is high. The authors show that famous measures like Mutual Information (a complex math term for how much the model's internal state tells us about its prediction) are just special versions of this variance.
Entropy (The Aleatoric Part): This is the "noise" part. It's the inherent randomness of the situation that no amount of data can fix. It's the "gust of wind" in our soccer analogy.

Why This Changes Everything

The paper suggests that all the different formulas scientists have invented over the years to measure uncertainty aren't actually competing enemies. They are just different flavors of the same underlying recipe.

The "Gal" Method: One famous way to measure uncertainty (by a researcher named Gal) uses a specific type of math called "log-loss." The authors show that if you use this specific loss, your decomposition naturally gives you the famous "Mutual Information" formula.
The "Sale" Method: Another group of researchers uses a different math (squared loss). The authors show that if you use that loss, you get a different formula (variance-based), which is also valid, just for a different type of problem.

The paper explicitly argues against the idea that we need to write new rules (axioms) to decide which uncertainty measure is "correct." Instead, they say the "correct" measure is simply the one that naturally pops out of the specific loss function you chose for your problem. If you pick a loss that cares about being "mode-seeking" (focusing on the most likely outcome), you get one type of uncertainty. If you pick a loss that cares about "mean-seeking" (averaging everything out), you get another.

The Connection to Learning Theory

The authors also take a step further and connect this idea to Statistical Learning Theory, which is the math behind how machines learn from data. They introduce new terms like "Subjective Approximation Gap" and "Subjective Estimation Error."

They find that the "Epistemic Uncertainty" (the reducible part) is actually a mix of two things:

How much the model disagrees with itself (the variance/Mutual Information part).
How much the model is limited by the specific type of model it is (the approximation gap).

This is a crucial insight. It suggests that when we say a model has "high epistemic uncertainty," we might be mixing up two different problems: the model is just confused (variance), or the model is the wrong kind of tool for the job (approximation). The paper suggests that the famous "Mutual Information" measure only captures the first part (the confusion) and might be underestimating the total "reducible" uncertainty because it ignores the second part.

The Bottom Line

In short, this paper doesn't invent a new way to calculate uncertainty. Instead, it provides a unified map. It tells us that the many different maps we've been using are all correct, but they are just looking at the same mountain from different angles.

The authors suggest that we should stop treating uncertainty as a mysterious primitive that needs to be defined from scratch. Instead, we should look at the Subjective Risk—the cost of being wrong according to our own beliefs—and break it down. The pieces we find are the uncertainty measures we've been looking for all along. It's a shift from "What is uncertainty?" to "How did we choose to measure it?"

The paper is careful to note that this is a theoretical framework. It explains why different measures exist and how they relate, but it doesn't claim to have solved every practical problem in the field. It's a new lens that makes the existing landscape much clearer, suggesting that the disagreements in the field are often just differences in the "rulers" (loss functions) people are using, rather than contradictions in the nature of uncertainty itself.

Technical Summary: Subjective Risk Decomposition: A New View for Uncertainty Quantification

Problem Statement
The field of Uncertainty Quantification (UQ) currently suffers from a proliferation of competing definitions for aleatoric (irreducible) and epistemic (reducible) uncertainty. While frameworks like Gal (2016) provide influential information-theoretic decompositions, the literature contains numerous alternative measures motivated by different modeling assumptions and application domains. This has led to debates regarding the axiomatic foundations of UQ terms and whether the aleatoric/epistemic dichotomy itself is meaningful. Existing approaches often treat uncertainty measures as primitive objects requiring independent justification, or they rely on "all combinations" strategies that generate candidate measures without explaining why specific measures behave differently or which is appropriate for a given scenario.

Methodology
The authors propose a unified framework where uncertainty measures are not primitives but consequences of higher-level modeling decisions, specifically the choice of a strictly proper loss function and the evaluation of risk.

Subjective Risk as a Foundation: The paper adopts the decision-theoretic notion of subjective risk (Savage, 1971), defined as the expected loss of a model $q$ evaluated against a true distribution $p$ , but where the expectation is taken over the model's own beliefs ( $q$ ) rather than the true data-generating process ( $p$ ). Formally, for a strictly proper loss $\ell$ , the subjective risk is $R_q(p) = \mathbb{E}_{\hat{Y} \sim q}[\ell(p, \hat{Y})]$ .
- Unlike the objective risk ( $\mathbb{E}_{Y \sim p}[\ell(q, Y)]$ ), which answers "how correct is the model?", the subjective risk answers "how realistic is the model?" by penalizing the model for assigning mass to outcomes the true distribution deems implausible.
- This formulation utilizes the reverse argument of the loss (evaluating $p$ under $q$ ), which corresponds to reverse cross-entropy or reverse KL-divergence in the case of log-loss.
Decomposition via Bregman Divergences: Leveraging the properties of strictly proper losses, the authors show that the expected subjective risk admits a reverse bias–variance decomposition.
- The expected subjective risk is decomposed into three terms:
  $\mathbb{E}_\theta [R_{q_\theta}(p)] = \underbrace{B_\phi(\bar{q}, p)}_{\text{Bias}} + \underbrace{\mathbb{E}_\theta [B_\phi(q_\theta, \bar{q})]}_{\text{Variance}} + \underbrace{\mathbb{E}_\theta [H_\ell(q_\theta)]}_{\text{Generalized Entropy}}$
- Here, $\bar{q}$ is the right Bregman centroid (the mixture of predictive distributions), $B_\phi$ is the Bregman divergence induced by the loss, and $H_\ell$ is the generalized entropy.
- The Variance term is identified as the measure of epistemic uncertainty, and the Generalized Entropy term is identified as aleatoric uncertainty.
Extension to Learning Theory: The authors extend this framework to statistical learning theory by defining subjective analogues of standard concepts:
- Excess Subjective Risk: The difference between the expected subjective risk of a model and the best-in-class subjective model.
- Subjective Approximation Gap: The error due to the hypothesis class not containing the optimal subjective model (analogous to approximation error).
- Subjective Estimation Error: The error due to finite data and parameter uncertainty (analogous to estimation error).

Key Contributions and Results

Unification of Existing Measures: The paper demonstrates that numerous existing uncertainty measures are special cases of this decomposition, derived by choosing specific loss functions and hypothesis classes:
- Information-Theoretic Measures (Gal, 2016): By using log-loss on categorical distributions, the decomposition recovers Gal's epistemic uncertainty as the mutual information $I(\hat{Y}; \Theta)$ (the Bregman variance) and aleatoric uncertainty as the conditional entropy $H(\hat{Y}|\Theta)$ .
- Gaussian Regression: For Gaussian models with unknown mean and variance, the framework recovers the differential entropy and mutual information terms used in regression UQ.
- Label-wise Uncertainty: Using squared error loss (Brier score) on Bernoulli distributions recovers the "label-wise" uncertainty measures proposed by Sale et al. (2024).
Explanation of Divergence: The framework explains why different measures (e.g., entropy-based vs. variance-based) behave differently. These differences are not contradictions but consequences of the geometry of the loss function. Losses with high curvature near the probability simplex boundary (like log-loss) are more sensitive to "hallucinations" (assigning mass to implausible events) than losses with flatter geometries (like squared loss).
Learning-Theoretic Insights:
- The authors show that mutual information (Gal's epistemic measure) is a systematic under-estimate of the true subjective estimation error. Mutual information captures only the variance component (posterior disagreement) but ignores the additional error induced by restricting predictions to a specific hypothesis class.
- They establish conditions under which the "best-in-class" model for objective risk coincides with that of subjective risk, linking model optimality under reality to optimality under internal beliefs.
Analysis of Epistemic Variants: The paper analyzes various proposals for epistemic uncertainty (e.g., Expected Pairwise KL, Reverse Mutual Information) and shows how they relate to symmetric and asymmetric components of the mutual information decomposition.

Significance and Claims
The paper claims to provide a "new view" for uncertainty quantification by shifting the perspective from defining axioms for uncertainty primitives to deriving them as consequences of modeling choices.

Theoretical Foundation: The authors argue that uncertainty measures need not be debated as independent primitives. Instead, they are naturally induced once a strictly proper loss and a modeling scenario are fixed. This provides a common theoretical foundation for disparate measures in the literature.
Bridging UQ and Learning Theory: By introducing subjective analogues of excess risk, approximation error, and estimation error, the paper lays the groundwork for a full learning-theoretic framework for UQ. It clarifies the relationship between epistemic uncertainty and standard learning-theoretic notions like estimation error.
Practical Implication: The approach suggests a new methodology for practitioners: given a specific modeling scenario and a chosen strictly proper loss, the corresponding epistemic and aleatoric terms are automatically induced by the risk decomposition, removing the need to select between competing ad-hoc definitions.

The authors position this work as a first step toward a comprehensive framework, emphasizing that it unifies existing proposals rather than proposing a single new "best" measure, and highlights the role of loss geometry in determining the behavior of uncertainty quantification.