📊 statistics

CLEAR: Calibrated Learning for Epistemic and Aleatoric Risk

The paper proposes CLEAR, a calibration method that combines distinct parameters for aleatoric and epistemic uncertainty to significantly improve the conditional coverage and efficiency of predictive intervals in regression tasks across diverse real-world datasets.

Original authors: Ilia Azizi, Juraj Bodik, Jakob Heiss, Bin Yu

Published 2026-07-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ilia Azizi, Juraj Bodik, Jakob Heiss, Bin Yu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict the future, like guessing how much a house will sell for or how fast a car will go. You want your guess to be right, but you also want to know how "shaky" that guess is. In the world of machine learning, there are two main reasons why your guess might be shaky, and for a long time, most tools only looked at one of them.

The Two Types of "Shakiness"

Think of Aleatoric Uncertainty as the "static" on a radio. It's the noise that's just there, like a measurement error or a random gust of wind. No matter how much you study the radio or how many times you listen, that static won't go away unless you fix the radio itself. It's the inherent chaos of the data.

Then there's Epistemic Uncertainty. This is the "I've never been here before" feeling. Imagine you are a mapmaker who has only drawn maps for the city center. If you try to guess what the terrain looks like in a distant, unexplored forest, your guess is shaky because you lack data, not because the forest is noisy. If you went out and collected more data, this kind of shakiness would disappear.

The Problem with Old Maps

Most old methods for making predictions were like a mapmaker who only cared about the static. They would draw a safety zone (a prediction interval) around their guess to cover the "static," but they completely ignored the fact that they were guessing about a place they had never visited. This meant that in familiar areas, their maps were okay, but as soon as you moved to the edge of the known world (extrapolation), their safety zones became dangerously narrow, giving a false sense of security.

Other methods tried to fix the "never been here" problem by making huge, wide safety zones everywhere, just in case. But this made the maps useless because the zones were so wide they didn't tell you anything specific.

Enter CLEAR: The Smart Balancer

The authors of this paper, published at the ICLR 2026 conference, introduced a new method called CLEAR (Calibrated Learning for Epistemic and Aleatoric Risk). Think of CLEAR as a super-smart mapmaker who carries two different rulers: one for the "static" and one for the "unknown territory."

Instead of just guessing how much to widen the safety zone, CLEAR uses two special knobs, called $\gamma_1$ and $\gamma_2$ , to balance these two types of uncertainty.

One knob adjusts for the noise (aleatoric).
The other adjusts for the lack of data (epistemic).

The magic happens because CLEAR doesn't just set these knobs to a fixed number. It looks at the data and asks, "Right now, which type of shakiness is the bigger problem?" If the data is noisy, it turns up the first knob. If the data is sparse (like that distant forest), it turns up the second knob. It finds the perfect mix to make the safety zone as tight as possible while still being safe.

What They Found (and What They Didn't)

The researchers tested CLEAR on 17 different real-world datasets, ranging from house prices to energy efficiency. They also ran simulations to see how it behaved in controlled environments.

Here is what the data showed:

The Results: In these tests, CLEAR managed to shrink the width of the prediction intervals by an average of 28.3% and 17.5% compared to older methods that were calibrated separately. Crucially, it did this while still keeping the "safety" level exactly where it was supposed to be (maintaining nominal coverage).
The "What Ifs": The paper explicitly argues against the idea that you should just add the two uncertainties together in a fixed 1-to-1 ratio (like setting $\lambda = 1$ ). They show that this fixed approach is often suboptimal because the balance changes depending on the data.
The Limits: The paper notes that while CLEAR works great on these datasets, it relies on the quality of the base models it uses. If the base models are terrible, CLEAR can't fix them. Also, the theoretical guarantees they discuss mostly hold up in "asymptotic" scenarios (meaning with infinite data) or in simulations; in the real world with finite data, they rely on empirical evidence that it works well, rather than a mathematical proof that it must work in every single impossible edge case.

The "Why" Behind the Magic

The authors suggest that the reason CLEAR works so well is that it fixes a specific flaw in how older methods behave.

Old "noise-only" methods (like CQR) tend to be over-confident in places where there is very little data. They draw tiny boxes in the unknown forest.
Old "data-lack" methods (like ensembles) tend to be over-cautious in places with lots of data, drawing huge boxes in the busy city center.

CLEAR acts like a mediator. It realizes that in the city, the "noise" ruler is the one to trust, and in the forest, the "unknown" ruler is the one to trust. By letting the data decide the ratio between the two, it creates a safety zone that is tight where it should be and wide where it needs to be.

The Bottom Line

This paper suggests that by treating uncertainty as two separate ingredients that need to be mixed in the right proportion, rather than a single lump, we can make machine learning predictions that are both safer and more precise. It's not a magic wand that solves every problem in the universe, but in the tests they ran, it consistently outperformed the competition, proving that sometimes, the best way to handle the unknown is to know exactly what kind of unknown you are dealing with.

Technical Summary: CLEAR: Calibrated Learning for Epistemic and Aleatoric Risk

1. Problem Statement

Accurate uncertainty quantification (UQ) is critical for reliable predictive modeling, yet existing methods often fail to address both sources of uncertainty in a balanced manner.

Aleatoric Uncertainty: Arises from inherent data variability (measurement noise, missing covariates) and cannot be reduced by gathering more data.
Epistemic Uncertainty: Arises from limited data, model misspecification, or subjective choices in data processing. It is reducible with more data or better models but is often large in extrapolation regions.

Current approaches typically address only one type or combine them with fixed, non-adaptive ratios (e.g., setting the weight of epistemic to aleatoric uncertainty as 1:1). This leads to poor conditional coverage, where prediction intervals are well-calibrated on average (marginal coverage) but fail to provide reliable coverage for specific subgroups or in extrapolation regions. Furthermore, existing methods often implicitly fix the scaling parameters between these uncertainty components, which may be suboptimal as the relative importance of each source varies across datasets and prediction tasks.

2. Methodology: CLEAR

The authors propose CLEAR (Calibrated Learning for Epistemic and Aleatoric Risk), a calibration framework that adaptively balances aleatoric and epistemic uncertainty using two distinct parameters, $\gamma_1$ and $\gamma_2$ .

2.1 Prediction Interval Formulation

CLEAR constructs prediction intervals $C(x)$ for a point estimate $\hat{f}(x)$ as:
$C(x) = \left[ \hat{f}(x) \pm \left( \gamma_1 \times \text{aleatoric}_{\pm}(x) + \gamma_2 \times \text{epistemic}_{\pm}(x) \right) \right]$
where $\gamma_2 = \lambda \gamma_1$ . The parameter $\lambda$ controls the trade-off between the two uncertainty types:

$\lambda = 0$ : The interval reflects only aleatoric uncertainty.
$\lambda \to \infty$ : The interval reflects only epistemic uncertainty.
$\lambda > 0$ : A data-driven balance is achieved.

2.2 Estimation Components

CLEAR is compatible with any pair of estimators but demonstrates performance using:

Aleatoric Estimator: Quantile regression models trained on residuals ( $Y_i - \hat{f}(X_i)$ ) rather than raw targets. The authors argue that fitting quantiles directly on targets can distort aleatoric estimates if the mean model underfits, whereas fitting on residuals is more stable. An ensemble of these models (bagging) is used to estimate conditional quantiles $\hat{q}^{ale}_{\alpha/2}$ and $\hat{q}^{ale}_{1-\alpha/2}$ .
Epistemic Estimator: Intervals derived from the Predictability–Computability–Stability (PCS) framework. This involves training an ensemble of models across different data pre-processing versions and model architectures, then bootstrapping the top-performing models. The point-wise quantiles of this ensemble define the uncalibrated epistemic uncertainty band $\hat{q}^{epi}_{\alpha/2}$ and $\hat{q}^{epi}_{1-\alpha/2}$ .

2.3 Calibration Procedure

The method employs a two-step calibration process on a validation/calibration dataset:

Grid Search for $\lambda$ : A grid of positive values $\Lambda$ is evaluated. For each candidate $\lambda$ , the scaling factor $\gamma_1$ is determined via standard split conformal prediction to ensure marginal coverage of $1-\alpha$ .
Optimization: The optimal $\lambda^*$ is selected by minimizing the Quantile Loss (equivalent to the pinball loss or interval score) on the validation set. This metric balances both coverage and interval width, incentivizing truthfulness.

2.4 Theoretical Properties

The paper establishes that under mild assumptions (consistency of base estimators and a compact grid for $\lambda$ ), CLEAR achieves asymptotic conditional validity. While the epistemic component vanishes asymptotically in the limit of infinite data, it plays a crucial role in finite samples by preventing under-coverage in data-sparse regions where standard conformal methods (like CQR) fail.

3. Key Contributions

Dual-Parameter Calibration: The introduction of two distinct calibration parameters ( $\gamma_1$ and $\gamma_2$ , linked by $\lambda$ ) to balance the scales of aleatoric and epistemic uncertainty, moving beyond fixed-ratio combinations.
Residual-Based Aleatoric Estimation: Demonstrating that fitting quantiles on residuals yields more sensible and stable aleatoric uncertainty estimators compared to fitting directly on targets.
PCS and CQR Integration: The first combination of PCS ensemble perturbation intervals (for epistemic uncertainty) with Conformalized Quantile Regression (CQR) on residuals (for aleatoric uncertainty).
Large-Scale Benchmarking: Extensive evaluation across 17 diverse real-world regression datasets and synthetic simulations.

4. Experimental Results

The paper evaluates CLEAR against baselines including CQR (aleatoric only), PCS ensembles (epistemic only), and their individually calibrated versions.

Performance Metrics: CLEAR consistently achieves lower Normalized Calibrated Interval Width (NCIW) and Quantile Loss while maintaining nominal coverage (95%).
Quantitative Gains: Across 17 real-world datasets, CLEAR achieved an average improvement of 28.3% in interval width compared to the individually calibrated aleatoric baseline (ALEATORIC-R) and 17.5% compared to the epistemic baseline (PCS), while maintaining coverage.
Robustness: CLEAR outperformed baselines on 15 of the 17 datasets. It was particularly effective in scenarios dominated by high aleatoric or epistemic uncertainty.
Generalizability: The method was also tested with Deep Ensembles (epistemic) and Simultaneous Quantile Regression (aleatoric), showing similar improvements (e.g., 28.6% width reduction with Deep Ensembles).
Case Study: In an Ames Housing case study, CLEAR adaptively shifted its weighting ( $\lambda$ ) based on data characteristics. In low-information regimes (few features), it prioritized aleatoric uncertainty; in high-feature, low-data regimes, it drastically shifted weight to epistemic uncertainty, correctly identifying the dominant source of uncertainty.

5. Significance and Claims

The paper claims that CLEAR offers a significant improvement over classical methods that treat uncertainty types in isolation or rely on fixed, non-adaptive combinations.

Conditional Coverage: By adaptively balancing uncertainty sources, CLEAR maintains approximate conditional coverage across the input space, particularly in extrapolation regions where standard conformal methods often under-cover.
Interpretability: The learned parameter $\lambda$ provides insight into which source of uncertainty (epistemic vs. aleatoric) dominates a specific dataset or region, aiding practitioners in understanding model limitations.
Robustness: The dual-parameter approach allows the method to compensate for the shortcomings of base estimators (e.g., if aleatoric uncertainty is over- or underestimated, $\gamma_1$ can correct the scale).

The authors acknowledge limitations, noting that the accuracy of CLEAR depends on the quality of the base estimators and that overfitting of parameters can occur with very small calibration datasets. They suggest future work could extend CLEAR to classification, time series, and active learning scenarios.