📊 statistics

Multi-Fidelity Quantile Regression

This paper proposes a two-stage, model-agnostic method for multi-fidelity quantile regression that leverages low-fidelity data to estimate high-fidelity conditional quantiles more accurately by modeling their relationship through a local quantile link and a correction step, ultimately yielding superior performance in both synthetic and real-world experiments.

Original authors: Yixiang Liu, Yao Zhang

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yixiang Liu, Yao Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict the weather for a specific city. You have two types of data:

High-Fidelity (HF) Data: This is the "gold standard." It comes from a super-accurate, expensive weather station right in the city. But, because it costs a fortune to run, you only have a few dozen readings.
Low-Fidelity (LF) Data: This is the "cheap" data. It comes from a satellite or a nearby town's weather station. It's not as precise, and sometimes it gets the details wrong, but you have millions of these readings.

The Problem:
You want to know not just the average temperature, but the extremes. You want to know: "What is the temperature that will be exceeded only 5% of the time?" (a rare heatwave) or "What is the temperature that will be beaten only 95% of the time?" (a rare freeze).

With so few "gold standard" readings, guessing these extremes is like trying to guess the shape of a mountain peak by looking at only three pebbles at the bottom. Your guesses will be shaky and unreliable.

The Solution: Multi-Fidelity Quantile Regression (MFQR)
The authors propose a clever two-step trick to use the millions of "cheap" readings to help you guess the "expensive" extremes accurately.

Step 1: The "Translator" (The Wrapper)

Imagine the cheap weather station (LF) and the expensive one (HF) speak different languages. The cheap one might say "It's hot," while the expensive one says "It's a heatwave."

Usually, if you just look at the cheap data, you might think a "Level 5" storm on the cheap scale equals a "Level 5" storm on the expensive scale. But that's often wrong. Maybe a "Level 3" on the cheap scale actually matches a "Level 5" on the expensive scale because the cheap sensor is just noisier.

The authors' first idea is to find a Translator.

They ask: "At this specific location, what 'level' on the cheap scale actually matches the 'extreme' level we want on the expensive scale?"
They call this the Level Function.

The Magic Analogy:
Think of the expensive data as a jagged, bumpy mountain path. It's hard to walk (hard to estimate) because it changes direction so often.
Think of the cheap data as a smooth, rolling hill. It's easy to walk (easy to estimate).

The authors realized that if you look at the probability of the event rather than the raw temperature, the "bumpy mountain" might actually look like a "smooth hill" when viewed through the lens of the cheap data.

Instead of trying to map the bumpy mountain directly, they map the smooth hill to the mountain.
They learn the relationship: "When the cheap sensor reads a '3', the expensive sensor is actually at a '5'."
Because the "smooth hill" (the relationship between the two) is much simpler and smoother than the "bumpy mountain" (the expensive data itself), it is much easier to learn with limited data.

Step 2: The "Spot-Check" (The Correction)

Sometimes, the translator isn't perfect. Maybe the cheap sensor is broken in a specific way, or the relationship between the two is too messy. If you rely only on the translator, you might still get it wrong.

So, they add a Correction Step.

They take their "translated" guess and check it against the few expensive readings they do have.
If the guess is slightly off, they nudge it closer to the truth using a mathematical "one-step" adjustment.
If the translator is really struggling (like in a "misinformative" regime where the cheap data is misleading), they can take multiple steps, repeatedly nudging the guess until it fits the expensive data perfectly.

Why This Matters

The paper tested this on:

Fake Data: They created scenarios where the cheap data was helpful, unhelpful, or even misleading. In all cases, their method (MFQR) gave better, tighter predictions than just ignoring the cheap data or using standard tricks.
Real Science Data: They used it on:
- Molecules: Predicting the energy of molecules (where "cheap" is a rough computer simulation and "expensive" is a precise lab measurement).
- Fluid Dynamics: Predicting how fast fluid moves (where "cheap" is a low-resolution computer grid and "expensive" is a high-resolution one).
- Materials: Predicting how crystals form.

The Result:
By using this "Translator + Spot-Check" method, they were able to create narrower, more accurate prediction intervals.

Without this method: "The temperature will be between 60°F and 100°F." (Too wide to be useful).
With this method: "The temperature will be between 78°F and 82°F." (Tight and useful), while still being statistically guaranteed to be correct 90% of the time.

In Summary:
The paper teaches us how to take a massive pile of "good enough" data and a tiny pile of "perfect" data, and combine them to predict rare, extreme events much better than we could with the perfect data alone. It does this by first finding a smooth, easy-to-learn connection between the two, and then fine-tuning that connection with the precious data we have.

Technical Summary: Multi-Fidelity Quantile Regression (MFQR)

1. Problem Statement

In many scientific domains (e.g., molecular design, materials science), high-fidelity (HF) data—such as wet-lab binding assays or high-resolution simulations—are accurate but scarce and expensive to collect. Conversely, low-fidelity (LF) data—such as molecular docking scores or coarse-grid simulations—are abundant but imperfect proxies. While multi-fidelity learning has been widely applied to estimate conditional means, estimating conditional quantiles remains challenging.

Quantile regression is essential for uncertainty quantification, particularly for heteroscedastic or asymmetric data. However, with limited HF sample sizes ( $n_H$ ), conditional quantile estimators suffer from high variance, scaling inversely with the conditional density at the target quantile. This leads to unstable quantile estimates and wide, unreliable prediction intervals, even when using conformal prediction to guarantee marginal coverage. Existing multi-fidelity methods primarily target mean functions or assume simple autoregressive relationships, which do not directly translate to the complex, level-dependent nature of conditional quantiles.

2. Methodology

The authors propose Multi-Fidelity Quantile Regression (MFQR), a two-stage, model-agnostic framework. The core innovation is a Local Quantile Link that reformulates the estimation problem.

2.1 The Local Quantile Link (Assumption 1)

The method posits that for a target HF quantile level $\tau$ and covariate $x$ , the HF quantile $Q_H(\tau | x)$ can be represented as an LF quantile evaluated at a covariate-dependent probability level $u_\tau(x)$ :
$Q_H(\tau | x) = Q_L(u_\tau(x) | x)$
Here, $u_\tau(x) \in (0, 1)$ is a "level function." If the LF and HF conditional distributions share similar shapes, $u_\tau(x)$ may vary more smoothly over the covariate space than the original HF quantile function $Q_H(\tau | x)$ .

2.2 The Wrapper Estimator

The first stage constructs a "wrapper" estimator by transforming the problem into estimating $u_\tau(x)$ :

Transform HF Responses: Using an LF conditional CDF estimator $\hat{F}_L(\cdot | x)$ , the HF responses $Y^H_i$ are wrapped into pseudo-responses: $\hat{U}_i = \hat{F}_L(Y^H_i | X^H_i)$ .
Estimate Level Function: A quantile regression is performed on the HF sample $\{(X^H_i, \hat{U}_i)\}$ to estimate the conditional quantile of the wrapped responses: $\hat{u}_\tau(x) = \hat{Q}_{U|X}(\tau | x)$ .
Map Back: The final wrapper estimator is obtained by inverting the LF CDF: $\tilde{q}_\tau(x) = \hat{F}_L^{-1}(\hat{u}_\tau(x) | x)$ .

Theoretical Advantage: Under structural models where the fidelity difference is a localized scale distortion, the wrapped target $u_\tau(x)$ depends only on the relative distortion function. If this distortion is smoother than the underlying mean or scale functions driving the HF quantile, the wrapper estimator achieves a faster convergence rate (lower variance) than direct HF quantile regression.

2.3 Correction Framework

To address regimes where the wrapper is suboptimal (e.g., when the LF data is non-informative or the local quantile link is weak), the authors introduce a proximal correction framework:

One-Step Correction: A Newton-like update is applied to the wrapper estimator using the HF quantile equation. This corrects the bias by moving the estimate toward the solution of $\hat{F}_H(q | x) = \tau$ . The step size is tuned via cross-fitting to minimize pinball loss.
Multi-Step Correction: If the local quantile link fails significantly, iterative Newton updates are applied to converge to the empirical HF quantile root, provided the initialization is within the Newton basin.

The framework utilizes cross-fitting on the scarce HF data to prevent overfitting during the correction step.

3. Key Contributions

Novel Reformulation: The paper introduces the "local quantile link," transforming HF quantile estimation into the estimation of a smoother level function $u_\tau(x)$ on the probability scale.
Theoretical Characterization: The authors provide convergence rates showing that the wrapper estimator outperforms direct HF regression when the distortion function is smoother than the HF quantile surface. They also characterize when the correction step provides further improvement, specifically when the HF CDF can be estimated accurately even with small sample sizes.
Model Agnosticism: The method does not require specific parametric forms or access to raw LF training data during the HF stage; it only requires an estimator of the LF conditional CDF. This allows integration with pretrained LF models.
Robustness: The framework includes correction mechanisms (one-step and multi-step) that ensure robustness even when the LF data is "misinformative" (i.e., when the wrapper assumption holds poorly).

4. Experimental Results

The authors evaluate MFQR on synthetic data and five scientific datasets (Acrolein, Thymine, o-HBDI, Burgers PDE, and Formation Energy).

Informative Regime: When LF and HF distributions have similar shapes, the wrapper estimator significantly reduces Mean Squared Error (MSE) and prediction interval width compared to HF-only baselines. The level function $u_\tau(x)$ is nearly flat, confirming the theoretical gain in smoothness.
Non-Informative Regime: When LF data introduces heteroscedasticity not present in HF data, the wrapper alone offers modest gains. However, the one-step correction drastically reduces MSE (e.g., ~78% reduction in a synthetic example) and tightens intervals by leveraging the HF quantile equation to correct the wrapper's bias.
Misinformative Regime: When LF and HF mean functions differ non-linearly, the uncorrected wrapper performs poorly. The multi-step correction successfully recovers performance comparable to direct HF estimation by iteratively targeting the HF quantile equation.
Scientific Datasets: Across molecular and PDE datasets, MFQR and MFQR + OS consistently produce the narrowest conformal prediction intervals (20–39% reduction in width) while maintaining nominal 90% coverage.

5. Significance and Claims

The paper claims that MFQR provides a robust, model-agnostic solution for uncertainty quantification in data-scarce scientific settings. Its significance lies in:

Bridging the Gap: It extends multi-fidelity learning from conditional means to conditional quantiles, addressing a critical need for reliable prediction intervals in decision-making (e.g., drug discovery).
Efficiency: By exploiting the smoothness of the level function, it reduces the sample complexity required to estimate HF quantiles.
Adaptability: The two-stage design (wrapper + correction) ensures the method remains effective whether the LF data is highly informative or potentially misleading.
Practicality: The framework is compatible with existing LF models (including foundation models) and does not require retraining on raw LF data, making it suitable for scenarios where HF labels are the primary bottleneck.

The authors conclude that while the method is most powerful when LF and HF distributions share structural similarities, the correction steps ensure it does not degrade performance in unfavorable regimes, offering a reliable tool for scientific discovery.