📈 economics

Prediction Sets and Conformal Inference with Interval Outcomes

This paper develops consistent and finite-sample valid methods for constructing optimal prediction sets for censored or interval-valued outcomes by characterizing oracle prediction intervals under partial identification and applying tailored conformal inference to account for irreducible, partial identification, and sampling uncertainties.

Original authors: Weiguang Liu, Áureo de Paula, Elie Tamer

Published 2026-02-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Weiguang Liu, Áureo de Paula, Elie Tamer

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to guess the price of a house in a new city. In a perfect world, you would know the exact price of every house sold there. But in the real world, data is messy. Sometimes, people don't say, "This house sold for $500,000." Instead, they say, "It sold for somewhere between $450,000 and $550,000." Or perhaps they only say, "It sold for more than $500,000."

This is what economists call interval data. It's like looking at a map where some cities are marked with a precise dot, but others are marked with a fuzzy cloud.

This paper by Liu, de Paula, and Tamer is about how to make the best possible guess (a prediction) when your data is fuzzy, and how to be confident that your guess is right.

Here is the breakdown of their work using simple analogies:

1. The Goal: The "Perfect Net" (The Oracle)

Imagine you are fishing. You want to catch a fish (the true outcome), but you don't know exactly where it is.

The Point Guess: Most people try to guess the exact spot where the fish is. "It's at coordinate X." If you miss by an inch, you fail.
The Prediction Set: This paper suggests casting a net. You want a net that is big enough to catch the fish 95% of the time (so you don't miss), but small enough to be useful. If your net covers the whole ocean, you technically "caught" the fish, but you learned nothing.

The authors call the "perfect net" the Oracle Prediction Set. It is the smallest possible net that still guarantees you catch the fish 95% of the time.

2. The Problem: The "Fuzzy" Data

Usually, to build this perfect net, you need to know exactly where the fish are swimming. But here, the data is censored or interval-valued.

The Analogy: Imagine you are trying to draw a map of where fish swim, but your sonar only tells you, "There is a fish somewhere in this 10-foot square," or "There is a fish somewhere above this line." You don't know the exact spot.
The Challenge: If you try to use old methods (which assume you know the exact spot), your net will either be too loose (wasting space) or too tight (missing the fish). The authors had to invent a new way to draw the net using only these fuzzy squares.

3. The Solution: Two-Step Magic

Step A: The "Smart Estimator" (Consistency)

First, the authors built a mathematical machine that looks at all those fuzzy squares and figures out the shape of the "perfect net."

How it works: They use a technique called kernel smoothing. Imagine you have a pile of sand (your data points). You pour a little water on it, and the sand settles into a smooth hill. This hill represents the probability of where the fish are.
The Twist: Because the data is fuzzy, the "hill" might have two peaks (bimodal). Maybe fish like to hang out in deep water and shallow water, but not in the middle. A simple method might draw one big net covering both. The authors' method is smart enough to draw two separate nets (one for deep, one for shallow) if that's where the fish actually are. This makes the net much smaller and more efficient.

Step B: The "Safety Net" (Conformal Inference)

Even with a smart machine, you might make a mistake because you only have a limited amount of data (sampling error). You need a guarantee that your net works right now, not just in the long run.

The Analogy: Think of Conformal Inference as a "calibration test."
1. You build your net using 75% of your data (the training set).
2. You take the remaining 25% (the calibration set) and throw them into your net.
3. You ask: "How many of these test fish did my net catch?"
4. If the net was too small and missed some, you widen the net just enough to catch them. If the net was huge and caught everything easily, you shrink it slightly.
The Result: This guarantees that no matter what the data looks like, your final net will catch the fish at least 95% of the time. It's like a "money-back guarantee" for your prediction.

4. Real-World Applications

The authors tested their method on two real-world scenarios:

UK Job Salaries: Many job ads don't list a specific salary; they say " $50k–$ 70k." The authors used their method to predict salary ranges across different UK cities. They found that in London, the "net" was huge (salaries vary wildly from low to very high), while in smaller towns, the net was tighter. This gives a much clearer picture of the job market than just guessing an "average" salary.
US Income Data: In surveys, people often refuse to give their exact income but will say, "I make between $40k and $60k." Traditional methods often throw this data away or guess a single number (imputation), which can be wrong. The authors' method keeps the "fuzzy" range and builds a prediction net around it, showing that people with more education have a wider range of possible incomes (a bigger net), not just a higher average.

Summary

This paper is about making better guesses when you don't have perfect information.

Don't guess a single number; guess a range (a net).
Make the net as small as possible while still being safe (efficient).
Handle fuzzy data (intervals) without throwing it away or guessing the exact numbers.
Use a calibration test (conformal inference) to guarantee your net works, even with small sample sizes.

It's a toolkit for turning "I don't know exactly" into "I know it's definitely in this specific zone."

1. Problem Statement

The paper addresses the challenge of constructing prediction sets (intervals or unions of intervals) for a random variable $Y$ when the outcome is interval-censored or interval-valued.

Context: In many economic and social surveys (e.g., income, wealth, job wages), respondents provide ranges (brackets) rather than exact values due to non-response or privacy concerns. Additionally, data may be right-censored ( $Y \in [Y_L, \infty)$ ) or left-censored.
The Gap: Existing methods for prediction sets often assume point-identified outcomes or rely on quantile regression. However, when outcomes are interval-valued:
1. The underlying distribution of $Y$ is only partially identified (the true $Y$ is unobserved, only $Y_L \leq Y \leq Y_U$ is known).
2. Standard "oracle" prediction sets (based on density level sets) are not directly applicable because the conditional density of $Y$ is not identified.
3. Standard conformal inference scores designed for point outcomes cannot handle interval data without modification.
Goal: To develop a method that constructs prediction sets $C_n(x)$ $C_{n} (x)$ which are:
- Valid: Guaranteeing a coverage probability of at least $1-\alpha$ in finite samples.
- Efficient: Minimizing the volume (length) of the prediction set.
- Consistent: Converging to the "oracle" prediction set as sample size increases.

2. Methodology

The authors propose a two-stage framework combining Partial Identification theory with Conformal Inference.

A. Defining the Oracle Prediction Set under Partial Identification

In the absence of censoring, the oracle prediction set is the set of minimal volume containing $1-\alpha$ of the probability mass (related to the highest density region). Under interval censoring, the joint distribution $P_{X,Y}$ is not point-identified.

Identified Set ( $\mathcal{P}_I$ ): The set of all joint distributions of $(X, Y)$ compatible with the observed distribution of $(X, Y_L, Y_U)$ and the censoring mechanism.
Feasible Optimization: The authors show that minimizing the volume of a set $C(x)$ subject to the constraint that it covers $Y$ with probability $1-\alpha$ for all distributions in $\mathcal{P}_I$ is equivalent to a feasible optimization problem based on the observed brackets:
$C^*(x) = \arg \min_{C(x) \in \mathcal{C}} \mu(C(x)) \quad \text{s.t.} \quad P([Y_L, Y_U] \subset C(x) \mid X=x) \geq 1-\alpha$
where $\mathcal{C}$ is the class of finite unions of disjoint intervals.
Key Insight: The constraint depends only on the observed conditional distribution of the brackets $(Y_L, Y_U)$ , which is fully identified. This allows for the construction of a consistent estimator without needing to impute $Y$ .

B. Consistent Estimation (Nonparametric)

Estimator: The authors propose a kernel-based estimator for the conditional probability $P([Y_L, Y_U] \subset C(x) \mid X=x)$ .
Handling Discrete Censoring: In cases of fixed censoring (e.g., specific survey brackets like " $10k-$ 20k"), the distribution is discrete, violating standard smoothness assumptions. The authors introduce a relaxed constraint ( $\geq 1-\alpha - \psi_n$ ) to ensure consistency in these pathological cases.
Result: The estimated set $\hat{C}(x)$ is shown to be uniformly consistent with the oracle set $C^*(x)$ in terms of the symmetric difference measure $\mu(\hat{C} \triangle C^*)$ .

C. Conformal Inference for Finite-Sample Validity

To guarantee finite-sample coverage (validity) regardless of the underlying distribution, the authors adapt Split Conformal Inference:

Split Data: Divide data into a training set (to estimate $\hat{C}$ ) and a calibration set.
Conformity Score for Intervals: A novel score function $s(y_L, y_U, x; \hat{C})$ $s (y_{L}, y_{U}, x; \hat{C})$ is defined.
- If the interval $[y_L, y_U]$ is contained within the prediction set $\hat{C}(x)$ , the score is non-positive.
- If the interval is partially or fully outside, the score is positive, representing the "distance" or "gap" required to cover the interval.
- This score penalizes both under-coverage and over-coverage.
Construction: The final conformal prediction set $\tilde{C}(x)$ is constructed by expanding $\hat{C}(x)$ by the $(1-\alpha)$ -quantile of the conformity scores from the calibration set.
Theoretical Guarantee: The resulting set $\tilde{C}(x)$ satisfies $P(Y_{n+1} \in \tilde{C}(X_{n+1})) \geq 1-\alpha$ for any distribution $P$ , provided the data is exchangeable.

3. Key Contributions

Oracle Characterization under Partial Identification: The paper formally defines the oracle prediction set for interval-censored data, proving that the optimal set minimizes volume subject to covering the observed brackets with high probability.
Novel Conformity Score: They design a specific conformity score tailored for interval outcomes that allows the prediction set to be a union of disjoint intervals (crucial for multimodal distributions).
Dual Guarantees: The proposed method offers finite-sample validity (via conformal inference) and asymptotic efficiency (consistency with the oracle set).
Robustness to Censoring Types: The method handles both random censoring (continuous brackets) and fixed censoring (discrete survey brackets) without requiring assumptions about the censoring mechanism (e.g., missing at random).
Empirical Application: The framework is applied to real-world data (UK job ads and US CPS), demonstrating its utility in labor economics where interval data is prevalent.

4. Results

Numerical Experiments (Monte Carlo)

Setup: Simulations included unimodal, bimodal, and skewed distributions with both random and fixed (discrete) censoring.
Comparison: The proposed Conformal Prediction (CP) and Local CP methods were compared against Conformal Quantile Regression (CQR) with quadratic and cubic specifications.
Findings:
- Volume: The proposed method produced prediction sets with significantly smaller volume (tighter intervals) than CQR, especially in bimodal cases where CQR (forcing a single interval) failed to capture the structure.
- Coverage: The proposed methods achieved coverage rates close to the nominal level ( $1-\alpha$ ), whereas CQR often suffered from under-coverage or excessive width.
- Fixed Censoring: The relaxed constraint approach successfully handled discrete bracket data where standard estimators failed to be consistent.

Empirical Applications

UK Job Advert Data (Adzuna):
- Data: 121k job ads, ~62% of salary data reported as intervals.
- Result: The method mapped spatial variations in salary distributions across the UK. It revealed that while lower bounds were relatively uniform, upper bounds (and thus interval widths) were highly concentrated in major cities (London, Manchester), indicating high wage dispersion in these hubs.
- Advantage: Avoided bias from imputing missing salary ranges, capturing the full distribution shape.
US Current Population Survey (CPS):
- Data: Income data with ~22% interval-valued responses.
- Result: Comparing "Censored" (using intervals directly) vs. "Imputed" (filling in missing values) approaches.
- Finding: The conformal prediction sets using interval data had significantly different (wider) upper bounds and better coverage rates compared to imputed data. Imputation tended to underestimate the tail risk of income distribution. The method successfully captured the uncertainty inherent in the interval data without arbitrary assumptions.

5. Significance

This paper bridges a critical gap between partial identification econometrics and machine learning (conformal prediction).

Theoretical: It extends the concept of "oracle prediction sets" to settings where the outcome is not fully observed, providing a rigorous definition of optimality under partial identification.
Practical: It offers a robust, assumption-free tool for policymakers and researchers dealing with survey data where non-response leads to interval reporting. By avoiding imputation, it prevents the introduction of model-based bias.
Methodological: The development of interval-specific conformity scores opens the door for applying conformal inference to other forms of set-valued or censored data in fields like survival analysis, finance, and environmental science.

In summary, the authors provide a mathematically sound and empirically robust framework for predicting outcomes when data is inherently imprecise, ensuring that prediction sets are both statistically valid and as informative (narrow) as the data allows.