📈 economics

Local conformal prediction for individual causal effects

This paper proposes an Individualized Causal Prediction (ICP) framework that constructs finite-sample valid conformal prediction intervals for individual causal effects by localizing calibration via Causal Forest-weighted cosine similarity and employing doubly robust AIPW scores, thereby achieving nominal coverage and improved point accuracy in high-heterogeneity settings where standard estimators fail.

Original authors: Fernando Delbianco, Fernando Tohmé

Published 2026-08-11

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Fernando Delbianco, Fernando Tohmé

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a doctor trying to predict how a specific patient will react to a new medicine. In the world of statistics, there's a common tool called the "Conditional Average Treatment Effect" (CATE). Think of this like a weather forecast for a whole city: it tells you the average temperature for everyone in the neighborhood. If the weather is usually mild, this average is a great guess for any single person. But what if the neighborhood is a chaotic mix of people? Some are shivering in a blizzard, others are sweating in a heatwave, and the "average" temperature is a comfortable 70 degrees. That average is useless for the person freezing or burning. This is the problem of "heterogeneity": when individual reactions vary wildly, the average stops telling the truth about any single individual.

To fix this, scientists use "Conformal Prediction." Imagine you are trying to guess the weight of a mystery box. Instead of just guessing a number, you gather a group of similar boxes you've already weighed. You look at how much your guess was off for those known boxes, and you build a safety net (a range) around your guess that is wide enough to catch the mystery box's true weight 95% of the time. This method is powerful because it doesn't need to assume the weights follow a perfect bell curve; it just needs a group of comparable examples. The challenge, however, is finding the right group of examples. If you compare a mystery box to a group of boxes that are all different from it, your safety net might be huge and useless, or worse, it might miss the mark entirely.

This paper, titled "Local Conformal Prediction for Individual Causal Effects," tackles the messy reality of individual differences. The authors, Fernando Delbianco and Fernando Tohmé, propose a new framework called "Individualized Causal Prediction" (ICP). They argue that to predict what will happen to one specific person, you shouldn't look at the whole world's data. Instead, you should look only at the tiny, specific neighborhood of people who are causally similar to that person.

The authors suggest a clever three-step recipe to build this perfect neighborhood. First, they use a smart algorithm (a "Causal Forest") to figure out which features actually matter for the outcome, ignoring the noise. They then find the people in the dataset who are most similar to the target person based on these important features. Second, because this neighborhood might be too small to make a reliable prediction, they use a trick called "synthetic augmentation." They create fake, computer-generated data points that look like the real neighbors, effectively filling in the gaps so the prediction isn't based on just a handful of people. Third, they apply a strict filter to ensure these real and fake neighbors are truly comparable, discarding anyone who looks similar on the surface but has a different underlying mechanism.

The paper tests this idea using computer simulations and a well-known dataset about infant health. The results suggest that this local approach is a winner. In their simulations, the new method produced more accurate point estimates (guessing the exact effect better) and tighter prediction intervals (a smaller, more useful safety net) compared to the old "global" method that looked at everyone. Crucially, the new method maintained a 90% success rate in covering the true effect, meaning it was just as reliable as the old methods but much more precise. The authors show that by focusing on the local causal environment rather than the global average, we can finally give meaningful answers to the question: "What will happen to me?" rather than just "What happens on average?"

Technical Summary: Local Conformal Prediction for Individual Causal Effects

Problem Statement
Standard causal inference literature has largely converged on the Conditional Average Treatment Effect (CATE), $\tau(x) = E[Y(1) - Y(0) | X = x]$ , as the primary estimand. While asymptotic confidence intervals for CATE (e.g., from Causal Forests or Double Machine Learning) are valid for the conditional mean, they fail to cover the Individual Causal Effect (ICE), $\tau_q = Y_q(1) - Y_q(0)$ , when treatment effects exhibit "unbounded heterogeneity." In such settings, the conditional variance is large even within narrow covariate cells, meaning the conditional mean does not represent the likely effect for any specific individual. Existing conformal prediction methods for causal inference, such as the weighted split-CQR approach by Lei and Candès (2021), provide marginal coverage averaged over the covariate distribution but may yield coverage far below the nominal level for specific query units due to the inclusion of units with divergent treatment effect distributions in the calibration set.

Methodology: The Individualized Causal Prediction (ICP) Framework
The authors propose the ICP framework to construct finite-sample valid prediction intervals for the ICE of a specific query unit $q$ . The core innovation is localizing the calibration set to a causally relevant neighborhood rather than using the global dataset. The methodology proceeds through four distinct stages:

Feature-Weighted Relevant Set ( $R_q^{(t)}$ ):
For a query $q$ and intervention $t$ , the method selects observations from the dataset that received treatment $t$ and share the same local structural mechanism. Similarity is measured via cosine similarity on covariates re-weighted by Causal Forest variable importance ( $\omega_j$ ). This ensures the neighborhood is defined by causally relevant features rather than noise variables.
Synthetic Augmentation ( $S_q^{(t)}$ ):
To address the scarcity of observations in high-dimensional local neighborhoods (a problem noted by Meng, 2018), the framework generates synthetic controls. This involves sampling with replacement from $R_q^{(t)}$ and adding Gaussian perturbations to the covariates. Outcome values for synthetic points are imputed via a local linear model. Points with propensity scores outside the overlap region $[0.05, 0.95]$ are discarded.
Super-Relevance Filtering ( $R_q^{*(t)}$ ):
A composite score $\phi_i(q)$ is computed for all observations in $R_q^{(t)} \cup S_q^{(t)}$ to filter for structural homogeneity. This score combines:
- Shapley-based influence weights: Derived from Causal Forest leaf weights.
- Propensity-score proximity: $1 - |\hat{e}(X_i) - \hat{e}(x_q)|$ .
  Crucially, this score depends only on $(X, T)$ and not on outcomes $Y$ , preserving exchangeability. The final calibration set retains the top 50% of observations by this score.
Conformal Calibration with AIPW Scores:
The framework utilizes Augmented Inverse Propensity Weighted (AIPW) pseudo-outcomes as conformity scores. The non-conformity score is defined as $V_i = |\hat{\Gamma}_i - \hat{\tau}^{OOB}(X_i)|$ , where $\hat{\Gamma}_i$ is the doubly robust AIPW estimator. This choice ensures Neyman orthogonality, meaning first-order errors in nuisance parameter estimation (outcome model or propensity score) do not propagate to the causal effect estimate. The prediction interval is constructed using the quantile of these scores over the local set $R_q^{*(t)}$ .

Key Theoretical Contributions

Finite-Sample Validity: Under SUTVA, strong ignorability, and a Selection Measurability condition (Assumption 3), the authors prove (Theorem 1) that the resulting interval satisfies $P(\tau_q \in \hat{C}(\tau_q)) \geq 1 - \alpha$ . The validity relies on the fact that the calibration set selection depends only on covariates and treatment, not outcomes, ensuring the exchangeability of conformity scores.
Approximate Conditional Coverage: While the formal guarantee is marginal, the authors argue that by concentrating the calibration set around $x_q$ in the causal covariate subspace, the method achieves approximately conditional coverage, a strictly stronger guarantee than global marginal approaches.
Modularity: The framework is presented as a template where the specific estimators (e.g., Causal Forests) and similarity metrics can be swapped, provided the selection remains outcome-independent.

Empirical Results
The framework was evaluated on two datasets:

Synthetic Data: A dataset with $N=800$ and $P=20$ covariates, where only 3 were causal. The ICP strategies (particularly "Relevant + Synth" and "Super-Relevant + Synth") achieved 96% empirical coverage (exceeding the 90% nominal target) while reducing RMSE and MAE compared to a global baseline. Local strategies also produced narrower intervals (e.g., 12.63 vs. 13.70 for the global baseline) without sacrificing coverage.
IHDP Benchmark: On the Infant Health and Development Program dataset ( $N=747$ ), all strategies achieved 100% empirical coverage. The "Relevant + Synth" strategy dominated on all accuracy metrics (RMSE: 0.399 vs. 0.437 for global; MAE: 0.298 vs. 0.333) and produced the narrowest average interval width.
LaLonde Dataset: Applied to real observational data, the method produced wide intervals (ranging from \ $31k to \$ 35k), reflecting the documented heterogeneity in earnings effects, though no ground truth comparison was possible.

Significance and Claims
The paper claims to address a fundamental gap in causal inference: the inability of standard CATE estimators to provide valid uncertainty quantification for individual counterfactuals under high heterogeneity. The significance of the work lies in:

Shifting the Paradigm: Moving from estimating a conditional mean to providing a prediction interval for a specific individual's counterfactual realization.
Local Calibration: Demonstrating that restricting calibration to a causally relevant neighborhood improves both point accuracy and interval precision compared to global methods.
Handling Data Scarcity: Showing that synthetic augmentation can stabilize local estimation without violating the exchangeability assumptions required for conformal prediction.
Structural Alignment: Connecting conformal prediction with Structural Causal Models (SCM) and Invariant Causal Prediction by filtering for units that share the same structural mechanism, rather than just raw feature proximity.

The authors remain modest regarding limitations, noting that the method assumes strong ignorability (no unmeasured confounding) and that in very high dimensions, feature-weighted similarity may not perfectly isolate the causal subspace. They suggest future extensions could incorporate instrumental variables or continuous treatments.

Technical Summary: Local Conformal Prediction for Individual Causal Effects

More like this