📊 statistics

Multivariate Standardized Residuals for Conformal Prediction

This paper proposes a multivariate conformal prediction framework that utilizes Mahalanobis distance-based standardized residuals to effectively handle heteroskedasticity and output correlations, thereby achieving improved conditional coverage and enabling practical extensions like missing value handling without requiring expensive sampling.

Original authors: Sacha Braun, Eugène Berta, Michael I. Jordan, Francis Bach

Published 2026-05-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sacha Braun, Eugène Berta, Michael I. Jordan, Francis Bach

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a weather forecaster. Your job isn't just to predict "It will rain tomorrow." It's to say, "It will rain, and I'm 90% sure the amount will be between 1 and 3 inches."

In the world of machine learning, this "amount between 1 and 3 inches" is called a conformal prediction set. It's a safety net that guarantees your prediction is correct a certain percentage of the time (e.g., 90% of the time).

However, there's a catch. Most standard methods only guarantee that the safety net works on average across all days. They don't guarantee it works for specific days.

The Problem: On a calm, predictable day, your net might be huge (1 to 10 inches), wasting your time. On a chaotic, stormy day, your net might be too small (1 to 1.1 inches), and you'll get caught in the rain. This is called heteroskedasticity—the uncertainty changes depending on the situation.

The paper you provided, "Multivariate Standardized Residuals for Conformal Prediction," introduces a new way to fix this, especially when predicting multiple things at once (multivariate).

Here is the breakdown in simple terms:

1. The Core Idea: The "Smart Elastic Band"

Imagine you are trying to catch a ball thrown by a friend.

Old Method (Naive): You use a rigid, fixed-size net. Sometimes the ball is easy to catch, and the net is too big. Sometimes the ball is wild, and the net is too small.
The Paper's Method: You use a smart, stretchy elastic band.
- If the ball is being thrown gently (low uncertainty), the band shrinks tight around the ball.
- If the ball is being thrown wildly (high uncertainty), the band stretches out to catch it.
- The Twist: The authors figured out how to do this when you are catching multiple balls at once that might be bouncing off each other (correlated outputs).

2. How It Works: The "Whitening" Trick

In math terms, the paper takes the error (the difference between the prediction and reality) and "whitens" it.

The Analogy: Imagine you are trying to measure how far a car is from the center of a track.
- If the track is a perfect circle, you just measure the distance.
- But what if the track is an oval, or a squashed circle, or a weird shape that changes depending on where you are?
- The authors' method learns the shape of the track (the local covariance) for every single moment. It then stretches or squashes the space so the track looks like a perfect circle again.
- Once the space is "normalized" (whitened), they can draw a perfect circle (an ellipsoid in higher dimensions) around the prediction. This circle is their safety net.

3. Why This is a Big Deal

The paper claims three main victories:

It's Smarter: It adapts to the specific situation. If the data is noisy, the net gets bigger. If it's clean, the net gets smaller. This makes the prediction much more useful than methods that use a "one-size-fits-all" net.
It's Fast: Other methods that try to do this involve complex, slow sampling (like running a million simulations to guess the shape). The authors' method uses a closed-form solution.
- Analogy: Instead of guessing the shape of a cloud by watching it for an hour, they have a formula that tells them the shape instantly. It's computationally cheap.
It Handles "Missing Pieces" and "New Views":
- Missing Data: Sometimes, you don't have all the data (e.g., you know the temperature but not the humidity). The paper shows how to still draw a valid safety net using just the pieces you have, without breaking the math.
- Partial Revelation: Imagine you predict a patient's blood sugar and cholesterol. If you measure the blood sugar first, you can instantly shrink the safety net for the cholesterol because the two are related. The paper allows you to update the net instantly without re-training the whole model.
- Transformations: If you want to predict the difference between two values (e.g., Profit = Revenue - Cost), the method can draw a net directly for that difference, rather than drawing a net for Revenue and Cost separately and hoping they overlap correctly.

4. What They Proved

The authors didn't just guess; they did the math.

They proved that if the errors follow a certain "elliptical" shape (like a stretched circle or oval), their method guarantees that the safety net works perfectly for every single input, not just on average.
Even if the data isn't a perfect oval, they showed that their method still gets very close to perfect accuracy as you feed it more data.

5. The Results

They tested this on fake data and real-world data (like predicting house prices or taxi routes).

The Outcome: Their "smart elastic band" (Mahalanobis distance) created safety nets that were tighter (more precise) and more accurate (hit the target 90% of the time) than other methods.
Speed: It was significantly faster than the other complex methods, making it practical for real-time use.

Summary

The paper solves the problem of "how do I draw a perfect, custom-shaped safety net around multiple predictions at once, even when the data is messy, missing, or changing?"

They answer: "Learn the local shape of the uncertainty, stretch the space to make it a circle, and draw a perfect net." This results in predictions that are not just statistically valid, but actually useful and precise for the specific situation at hand.

Technical Summary: Multivariate Standardized Residuals for Conformal Prediction

Problem Statement
Conformal prediction offers a model-agnostic framework for constructing predictive sets with guaranteed finite-sample marginal coverage. However, a significant limitation is the difficulty of achieving conditional coverage, where validity holds for specific observed inputs rather than just on average. This is particularly critical in heteroskedastic settings where prediction uncertainty varies with the input features. Existing approaches face trade-offs:

Univariate methods: Techniques like "standardized residuals" (normalizing errors by local variance) improve conditional coverage in univariate regression but do not naturally extend to multivariate outputs.
Multivariate density estimation: Methods that estimate the full conditional density can yield valid sets but are computationally expensive, often requiring sampling, and struggle with the "curse of dimensionality."
Fixed-shape methods: Approaches producing fixed-size or non-adaptive sets fail to capture input-dependent uncertainty.
Sampling-based methods: Recent strategies to improve conditional coverage often introduce randomness and variability for identical inputs, undermining reproducibility.

Methodology
The authors propose a framework that extends the concept of standardized residuals to the multivariate setting. The core idea is to learn a feature-dependent local covariance matrix to "whiten" the residuals, effectively decoupling output correlations and standardizing local variance.

Modeling: Given a feature vector $X$ and a multivariate response $Y \in \mathbb{R}^k$ , the method learns two functions:
- A predictor $f_\theta(X)$ estimating the conditional mean $\mathbb{E}[Y|X]$ .
- A covariance estimator $\Sigma_\varphi(X)$ estimating the conditional covariance matrix.
  These can be learned jointly via maximum likelihood (assuming a Gaussian approximation) or applied on top of a pre-existing black-box point predictor.
Non-conformity Score: The method defines a non-conformity score based on the Mahalanobis distance:
$S_{\text{Mah}}(X, Y) = \|\Sigma_\varphi(X)^{-1/2}(Y - f_\theta(X))\|_2$
This score measures the distance of a residual from the mean, scaled by the local covariance structure.
Conformal Sets: The resulting prediction set $C_\alpha(X_{\text{test}})$ is an ellipsoid centered at $f_\theta(X_{\text{test}})$ , with a shape and size determined by $\Sigma_\varphi(X_{\text{test}})$ and a threshold derived from the calibration set. This allows the set to adapt to heteroskedasticity.
Extensions: The framework naturally supports several practical extensions:
- Missing Outputs: By marginalizing the Gaussian density to observed dimensions and using the Chi-squared CDF, the method constructs valid scores even when calibration data has missing values.
- Partially Revealed Outputs: If part of the output $Y_r$ is revealed at test time, the method uses conditional Gaussian properties to refine the prediction set for the hidden part $Y_h$ without retraining.
- Output Transformations: For linear transformations of the output (e.g., portfolio returns), the method constructs conformal sets directly on the transformed variables using the transformed mean and covariance, avoiding the conservativeness of projecting existing sets.

Theoretical Guarantees
The paper establishes a sufficient condition for the method to achieve asymptotic conditional coverage. Specifically, if the standardized residual $U = \Sigma(X)^{-1/2}(Y - f(X))$ is independent of $X$ , the method yields exact conditional coverage asymptotically. This condition holds for a broad class of distributions, including elliptical distributions, but does not strictly require the noise to be elliptical. Even when this independence does not hold, the method is shown to improve conditional coverage by normalizing the first two moments of the residuals.

Key Contributions

Multivariate Extension: A natural generalization of standardized residuals to multivariate regression using a learned local covariance matrix.
Closed-Form Efficiency: Unlike density-based methods requiring sampling, this approach yields conformal sets in closed form, making it computationally efficient at inference time.
Theoretical Characterization: A sufficient condition characterizing distributions for which standardized residuals yield asymptotic conditional coverage.
Practical Flexibility: The framework enables handling of missing calibration outputs, refinement of sets upon partial revelation of outputs, and construction of valid sets for output transformations.

Experimental Results
The authors evaluate the method on synthetic and real-world datasets (e.g., Bias, CASP, House, rf1, scm1d, Taxi) against baselines including HPD (density-based), C-PCP, L-CP, ECM (empirical covariance), and OT (optimal transport).

Conditional Coverage: The method achieves conditional coverage closer to the target level (e.g., 90%) than existing multivariate baselines, particularly outperforming "marginal-only" methods that over-cover in some regions to compensate for under-coverage elsewhere.
Set Size: The method produces tighter (smaller volume) conformal sets compared to density-based methods while maintaining coverage.
Computational Efficiency: The method is significantly faster than density-based approaches (HPD, PCP, C-PCP), which often require expensive Monte Carlo sampling. For instance, assessing coverage for 1,000 points takes negligible time compared to the hundreds or thousands of seconds required by baselines.
Robustness: The method performs well even when the underlying Gaussian assumption is mis-specified (e.g., with exponential noise), suggesting robustness beyond the theoretical assumptions.

Significance and Claims
The paper claims that this approach constitutes an "encouraging step" toward the goal of reliable conditional coverage under minimal assumptions. By avoiding the need for full conditional density estimation, the method offers a computationally tractable and robust alternative that balances the trade-off between the strict guarantees of marginal coverage and the adaptivity of conditional coverage. The authors emphasize that while strict conditional coverage cannot be guaranteed for all distributions, the method achieves it asymptotically for a broad class and provides robust approximate coverage in practice. The ability to handle missing values and refine predictions with partial information further extends the applicability of conformal prediction to real-world scenarios where data is often incomplete or partially observed.