📊 statistics

Extreme Conformal Prediction: Reliable Intervals for High-Impact Events

This paper proposes a novel method that bridges extreme value statistics and conformal prediction to generate reliable, informative prediction intervals for high-impact events, effectively overcoming the limitation of classical conformal methods that produce uninformative, infinitely wide intervals when high confidence levels are required relative to limited calibration data.

Original authors: Olivier C. Pasche, Henry Lam, Sebastian Engelke

Published 2026-04-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Olivier C. Pasche, Henry Lam, Sebastian Engelke

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Too Big to Fail" Prediction

Imagine you are a weather forecaster. Usually, you are asked to predict if it will rain tomorrow. You can easily say, "There is a 90% chance of rain," and give a range of how much water might fall. This is easy because you have lots of data about normal rain.

But now, imagine a city asks you: "What is the chance of a 'once-in-a-millennium' flood? We need to build a dam that never fails, even if the water rises higher than anything we've ever seen."

This is the problem the paper tackles. In the world of statistics, this is called an "extreme confidence level."

If you try to use standard prediction tools to answer this, they break. Here's why:

The Data Gap: You only have 100 years of rain data. You are being asked to predict something that might happen once every 10,000 years.
The "Infinite" Answer: Standard math looks at your 100 years of data, sees that the highest flood was 10 meters, and says, "I can't be 99.999% sure the water won't go higher than 10 meters." So, it gives you an answer of "Infinity."
The Useless Result: A prediction of "The water could be anywhere from 0 to Infinity" is technically safe (it covers the truth), but it is useless. You can't build a dam based on "Infinity."

The Solution: "Extreme Conformal Prediction"

The authors propose a new method that combines two powerful ideas: Conformal Prediction (a safety net for AI) and Extreme Value Theory (the science of the impossible).

Think of it like this:

1. The Safety Net (Conformal Prediction)

Imagine you have a very smart AI that predicts river levels. But AI is sometimes overconfident or slightly wrong.

Standard Conformal Prediction is like putting a "safety margin" around the AI's guess. If the AI says "10 meters," the safety net says, "Okay, let's add 2 meters just in case."
It checks its work against past data (calibration) to make sure the net is big enough to catch the real answer 95% of the time.

2. The Telescope (Extreme Value Theory)

The problem is, when you need a safety net for a "once-in-a-millennium" flood, the standard safety net runs out of data. It doesn't know how to stretch that far.

Extreme Value Theory is like a telescope. It doesn't just look at the data you have; it looks at the shape of the tail of the data distribution.
It uses a mathematical tool called the Generalized Pareto Distribution (GPD). Think of this as a "shape-shifter" that studies the highest few floods in your history and figures out the mathematical curve of how extreme events behave. It allows you to extrapolate (guess) what happens beyond your data.

How the New Method Works (The Recipe)

The authors combine these two tools into a new recipe:

Train the AI: First, they use a smart machine learning model to predict river levels.
Check the Errors: They look at how wrong the AI was in the past (the "nonconformity scores").
The Telescope Step: Instead of just looking at the highest error in the past 100 years, they use the "GPD Telescope" to model the shape of the errors. This lets them mathematically estimate what the error would be if the flood were 10 times bigger than anything seen before.
The Double-Check (Conservative Safety): Because guessing the future is risky, they don't just pick the single best guess. They calculate a confidence interval for that guess. They pick the upper bound of that interval.
- Analogy: If the telescope says the flood might be 100 meters, but there's a 5% chance it could be 110, they build the dam for 110 meters. They play it safe.

Why This Matters: The Flood Example

The paper tested this on real data from the Aare River in Switzerland.

The Goal: Predict the water flow for the next day with 99.999% confidence (extreme safety).
The Old Way: The standard method said, "We don't know, the answer is infinite." (Useless).
The New Way: The new method gave a specific number, like "The water will not exceed 650 cubic meters per second."
The Result: When they tested it, the new method actually kept the water under that limit almost every time. It provided a finite, useful number that was still safe enough to protect cities.

The Trade-off: Being "Over-Safe"

The authors admit their method is a bit "over-conservative."

Analogy: Imagine you are packing for a trip. A normal person packs for the weather forecast. This new method packs for the weather forecast plus a hurricane, plus a snowstorm, plus a heatwave, just to be absolutely sure you don't get wet or cold.
Is that bad? For building dams, managing bank reserves, or preventing wildfires, no. It is better to build a slightly bigger dam than to have the river overflow. The paper argues that being "too safe" is better than being "wrong" when lives and economies are at stake.

Summary in One Sentence

This paper invents a new statistical "telescope" that allows AI to make safe, finite predictions for catastrophic events (like massive floods) that are too rare to be found in historical data, preventing the AI from giving up and saying "I don't know."

1. Problem Statement

Context: Conformal prediction (CP) is a standard framework for constructing prediction intervals (PIs) with rigorous marginal coverage guarantees ( $P(Y_{test} \in \hat{C}(X_{test})) \geq 1-\alpha$ ) using black-box machine learning models. It relies on a calibration set to estimate a nonconformity score quantile.

The Challenge: In high-impact scenarios (e.g., flood forecasting, financial crises), regulators often require extremely high confidence levels (e.g., $1-\alpha = 0.9999$ ).

The Limitation: Classical CP methods fail when the desired confidence level exceeds the capacity of the calibration set size ( $n_c$ ). Specifically, if $\alpha < 1/(n_c + 1)$ , the required empirical quantile of the calibration scores is undefined (or infinite).
Consequence: The resulting prediction intervals become trivially infinite ( $(-\infty, \infty)$ ), offering no practical utility. Even when $\alpha$ is slightly above this limit, the variance of the empirical quantile estimator is prohibitively large, leading to unreliable intervals.
Gap: Existing methods cannot provide finite, informative, and statistically valid intervals for extreme quantiles when data is limited.

2. Methodology

The authors propose Extreme Conformal Prediction, a framework that bridges Conformal Prediction with Extreme Value Theory (EVT) to extrapolate prediction intervals beyond the range of observed data.

Core Workflow

The method follows a two-step process similar to Conformalized Quantile Regression (CQR) but integrates EVT for the calibration step:

Base Model Training (Step 1):
- Train a flexible, black-box Extreme Quantile Regression model (e.g., Neural Networks, Gradient Boosting, Random Forests) to estimate conditional quantiles $\hat{Q}_{1-\alpha}(x)$ on a training set.
- This model handles the extrapolation of the conditional quantile beyond the training data range.
Extreme Calibration (Step 2):
- Compute nonconformity scores $S_i = s(X_i, Y_i)$ on an independent calibration set.
- Instead of using the empirical quantile of $S$ (which fails for extreme $\alpha$ ), the method models the tail of the score distribution using the Generalized Pareto Distribution (GPD) via the Peaks-Over-Threshold (POT) approach.
- Extrapolation: Fit a GPD to scores exceeding a high threshold $u$ . Use this fitted distribution to estimate the extreme quantile $\hat{q}_\alpha$ corresponding to the desired confidence level $1-\alpha$ .

Conservative Confidence Interval Construction

To ensure the marginal coverage guarantee holds despite estimation bias in the GPD parameters, the authors do not use the point estimate of the GPD quantile directly. Instead, they construct a confidence interval (CI) for the quantile:

They define a target level $1-\alpha$ and split it into two components ( $\alpha_1, \alpha_2$ ) such that $(1-\alpha_1)(1-\alpha_2) \geq 1-\alpha$ .
They compute the upper endpoint ( $U_Q$ ) of a $(1-\alpha_2)$ -confidence interval for the $(1-\alpha_1)$ -quantile of the score distribution.
Theorem 3.1 proves that using this upper endpoint as the conformal threshold guarantees $P(Y_{test} \in \hat{C}(X_{test})) \geq 1-\alpha$ .
Implementation: The authors recommend using the Profile Likelihood method to construct this CI, as it is the most conservative and handles asymmetry best. They also propose a "GPD safeprofile" hybrid: if the profile likelihood encounters numerical instability (common in extreme cases), it falls back to a nonparametric bootstrap CI.

Handling Non-Stationarity

For data with distribution shifts (e.g., seasonal river flow), the authors extend the method to Weighted Conformal Prediction:

Calibration scores are assigned weights $w_i$ based on their similarity to the test point (e.g., temporal proximity or seasonal block matching).
The GPD log-likelihood is modified to a weighted log-likelihood to infer the quantile CI, ensuring the calibration accounts for non-exchangeable data.

3. Key Contributions

Novel Framework: First method to successfully construct finite, informative prediction intervals for extreme confidence levels ( $1-\alpha \gg n_c/(n_c+1)$ ) by combining CP with EVT.
Theoretical Guarantee: Provides a rigorous proof (Proposition 3.1) that the proposed conservative CI-based approach maintains the marginal coverage guarantee even when extrapolating beyond the data range.
Robustness: Introduces a "safeprofile" strategy that mitigates numerical instability in extreme quantile estimation, ensuring the method is practically deployable.
Flexibility: The method is agnostic to the base model, allowing the use of any black-box extreme quantile regressor (including deep learning) without requiring specific asymptotic guarantees from the base model.
Non-Stationary Extension: Adapts the framework to handle time-varying distributions via weighted likelihoods.

4. Results

Simulation Study

Setup: Tested on synthetic data with heavy-tailed (Student's $t$ ) and light-tailed (Gaussian) noise, varying calibration sizes ( $n_c \in \{10^3, 10^{3.5}, 10^4\}$ ) and extreme confidence levels ( $\alpha \in \{10^{-3}, \dots, 10^{-5}\}$ ).
Findings:
- Classical CP: Produced infinite (uninformative) intervals for all extreme levels where $\alpha < 1/(n_c+1)$ .
- Simple GPD: Provided finite intervals but suffered from undercoverage due to estimation bias.
- Proposed Method (GPD Profile/Bootstrap): Consistently achieved coverage $\geq 1-\alpha$ . The Profile Likelihood variant was the most conservative and reliable, satisfying coverage guarantees in all scenarios.
- Base Model Impact: The method corrected undercoverage even when the base quantile regression model was poorly fitted (e.g., linear model on non-linear data).

Application: Flood Risk Forecasting

Dataset: Daily river discharge (Aare river, Bern, Switzerland) and precipitation data (1930–1999).
Task: One-day-ahead interval forecasts for extreme water flow.
Challenges: Strong seasonality (violating exchangeability) and the need for extreme confidence levels ($0.9999$).
Results:
- Classical CP: Failed at high confidence levels ( $>0.99$ ), yielding infinite intervals.
- Extreme Conformal Prediction: Produced finite, informative intervals that strictly satisfied the target coverage levels (e.g., 99.99%).
- Weighted Approach: Successfully adapted to seasonal drifts, providing tighter intervals during low-risk seasons and wider ones during high-risk periods.
- Comparison: Outperformed both raw quantile predictions (which undercovered) and classical conformalized predictions (which were either infinite or undercovered).

5. Significance and Implications

Regulatory Compliance: Enables the construction of valid risk intervals for high-stakes applications (finance, infrastructure, safety) where regulators demand near-certainty ( $>99.9\%$ ) that cannot be met with standard empirical methods.
Bridging Theory and Practice: Solves the "extrapolation problem" in conformal prediction. While classical CP is limited to the empirical support of the data, this method leverages EVT to safely extrapolate into the tail.
Trade-off Acknowledgement: The authors note that while the method provides finite-sample validity for the marginal coverage, it relies on asymptotic EVT assumptions. However, they argue this is a necessary and unavoidable trade-off to achieve extrapolation, as finite-sample bounds for extrapolated quantiles are theoretically impossible without distributional assumptions.
Open Source: The authors provide an R package (ExtremeConformal) and a dependency for profile-likelihood CI estimation, facilitating immediate adoption by practitioners.

In summary, this paper solves a critical bottleneck in risk-sensitive machine learning: how to provide statistically rigorous, finite prediction intervals for events that are rarer than the available data can empirically support.