📊 statistics

Estimation and Inference for Causal Explainability

This paper proposes a causal framework for explainability that introduces a semi-parametric one-step correction estimator to reduce asymptotic variance and a randomization-based inference procedure for testing zero explainability, demonstrating their effectiveness through simulations and an analysis of public opinion on immigration.

Original authors: Weihan Zhang, Zijun Gao

Published 2026-03-09

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Weihan Zhang, Zijun Gao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Why" Behind the "What"

Imagine you are a chef who just made a delicious soup. You know the soup tastes amazing (the Outcome), but you want to know exactly why. Is it the salt? The garlic? The fact that you stirred it clockwise? Or is it the magic combination of garlic and heat working together?

In the world of data science, we often have a "black box" model that predicts outcomes (like whether a customer buys a product or if a patient recovers). We want to know: How much does each ingredient (variable) actually contribute to the final result?

This paper introduces a new, more rigorous way to answer that question. Instead of just guessing which ingredients matter, the authors provide a mathematical recipe to measure exactly how much "explanatory power" each factor has, and how much they "team up" (interact) to change the outcome.

The Core Problem: The "Oracles" vs. Real Life

The Old Way (The Magic Oracle):
Imagine you have a magic crystal ball (an "Oracle") that can tell you exactly what the soup would taste like if you swapped the salt for sugar, or if you removed the garlic entirely. Most previous methods relied on having this crystal ball. They would ask the ball a million questions to figure out what matters.

The New Way (The Real Kitchen):
In real life (like in social science or genetics), we don't have a crystal ball. We only have a notebook of past recipes and how they turned out. We can't magically change the past; we only have the data we collected. The authors say, "Let's build a method that works with just our notebook, without needing a magic oracle."

The Three Big Innovations

The paper solves three tricky problems using three clever tricks:

1. The "Efficiency" Trick: Using Independence Like a Shortcut

The Concept:
Imagine you are trying to guess the weather in three different cities: New York, London, and Tokyo. If these cities were totally unrelated (independent), knowing the weather in New York tells you nothing about London.

The Metaphor:
Most statistical methods treat every city as if it might be connected to every other city, which makes the math heavy and slow. It's like trying to solve a puzzle where you assume every piece might fit next to every other piece.

The Paper's Solution:
The authors realized that in many experiments (like the immigration study they used), the factors are randomized and independent. They built a "shortcut" (called a One-Step Correction Estimator) that explicitly uses this independence.

Analogy: Instead of checking every possible connection between puzzle pieces, they say, "Hey, we know these pieces don't connect. Let's ignore those connections." This makes their calculation much more precise (lower variance) and gives them a sharper answer with less data.

2. The "Zero" Problem: When Nothing Matters

The Concept:
Sometimes, a factor really doesn't matter at all. Its "explainability" is exactly zero. In statistics, this is called a "degenerate null." It's like trying to measure the weight of a ghost. Standard math tools break down here because the usual "ruler" (the confidence interval) becomes useless when the answer is exactly zero.

The Metaphor:
Imagine you are a judge trying to decide if a suspect is guilty. Standard tests work great if the suspect is clearly guilty or clearly innocent. But if the suspect is perfectly innocent (zero guilt), standard tests might get confused and give you a weird, unreliable verdict.

The Paper's Solution:
They use Randomization Inference (a method inspired by Sir Ronald Fisher).

Analogy: Instead of using a broken ruler, they play a game of "What If?" They take their data, shuffle the cards (permute the variables), and see what happens. If the factor truly has zero effect, shuffling the cards shouldn't change the outcome. If the outcome stays the same no matter how they shuffle, they know for a fact the factor is irrelevant. This works perfectly even when the answer is exactly zero.

3. The "Sequential" Strategy: The Two-Step Detective

The Concept:
Since we don't know beforehand if a factor is important or zero, we need a plan that handles both cases.

The Metaphor:
Think of a detective solving a case.

Step 1: First, they do a quick "shuffling test" (the randomization test). If the evidence suggests the suspect is definitely innocent (zero explainability), they close the case immediately and say, "It's zero."
Step 2: If the shuffling test suggests the suspect might be guilty (non-zero explainability), they switch to the "high-precision ruler" (the confidence interval) to measure exactly how guilty they are.

The Result: This two-step process ensures you never get a wrong answer, whether the factor is important or completely useless.

The Real-World Test: The Immigration Experiment

To prove their method works, the authors applied it to a famous study about immigration.

The Setup: People were asked to choose between two immigrants based on different traits: Gender, Job Plan, Education, Language, etc.
The Question: Which traits actually change people's minds? Do they care more about a job plan or their gender? Do a "Job Plan" and "Job Experience" work together to change opinions?

The Findings:
Using their new "shortcut" method, they found:

Job Plans and Education were the biggest drivers of opinion (high explainability).
Gender didn't matter much (close to zero).
The Interaction: They discovered a specific "team-up" effect: The combination of a Job Plan and Job Experience mattered more than either one alone.

This is a crucial insight. It tells us that people aren't just looking at a resume line-by-line; they are looking at the story the resume tells when those two lines are read together.

Summary: Why This Matters

This paper is like upgrading from a blurry, guesswork-based map to a high-definition GPS.

It's Realistic: It works with real-world data where you can't magically change the past.
It's Smarter: It uses the fact that variables are independent to give you a clearer, more precise answer.
It's Robust: It has a special safety net for when the answer is "nothing matters," ensuring you don't get a false alarm.

In short, the authors gave us a better way to understand cause and effect in a complex world, helping us figure out not just what happened, but exactly why it happened.

1. Problem Formulation

The paper addresses the challenge of quantifying causal explainability: determining how much individual factors (variables) and their interactions contribute to the variability of an outcome. Unlike association-based measures (e.g., Shapley values or standard feature importance), the authors advocate for a causal view to uncover underlying mechanisms and ensure generalizability across contexts.

Key Challenges Identified:

Data Constraints: Many domains (social science, genetics) rely on collected observational or experimental data without access to "oracles" (black-box models that can generate outcomes for arbitrary inputs). Existing methods often assume oracle access.
Degenerate Nulls: A significant statistical hurdle arises when the true explainability is zero. In such cases, the parameter lies on the boundary of the parameter space, causing standard influence functions to vanish. This leads to degenerate asymptotic distributions, rendering standard confidence intervals invalid.
Efficiency: Standard semi-parametric estimators often ignore the independence structure of treatment variables, leading to suboptimal variance.

Framework:
The authors utilize Causal ANOVA, a framework generalizing functional ANOVA and Sobol indices to the counterfactual setting. They assume:

Independent Treatments: Factors $W_k$ are mutually independent (Assumption 2.1).
NPSEM-IE: The outcome follows a Non-Parametric Structural Equation Model with Independent Errors ( $Y = f_Y(W) + E_Y$ ), ensuring identifiability.

2. Methodology

The paper proposes a comprehensive semi-parametric framework for estimation and inference, consisting of three main components:

A. Efficient Influence Functions (EIF) and One-Step Estimation

The authors derive the Efficient Influence Function (EIF) for causal explainability quantities (total explainability and interaction explainability).

Baseline vs. Efficient: They first derive the influence function under a non-parametric model ( $\phi_{IF}$ ). Then, leveraging the independence structure of the factors ( $W_k \perp \perp W_{-k}$ ), they project this function onto the tangent space of the restricted model to obtain the Efficient Influence Function ( $\phi_{EIF}$ ).
Variance Reduction: By explicitly utilizing the independence structure, $\phi_{EIF}$ achieves a smaller asymptotic variance compared to $\phi_{IF}$ , which treats the joint distribution as a black box.
One-Step Correction: They construct a one-step correction estimator:
$\hat{\xi} = \xi(\hat{P}) + \mathbb{P}_n \hat{\phi}$
where $\hat{P}$ represents nuisance estimators (e.g., conditional expectations $\mathbb{E}[Y|W]$ ) and $\mathbb{P}_n$ is the empirical measure. This estimator is robust to nuisance estimation errors provided they converge at a rate of $o_P(n^{-1/4})$ .
Cross-Fitting: To avoid overfitting and ensure valid asymptotic normality, the method employs a cross-fitting procedure (splitting data into folds, estimating nuisances on training folds, and evaluating on held-out folds).

B. Handling Degenerate Nulls via Randomization Inference

When the null hypothesis is $\xi = 0$ (zero explainability), the standard asymptotic normality fails.

Equivalence to Fisher's Sharp Null: The authors prove that under the independence assumption, the null $\xi(WS) = 0$ is equivalent to Fisher's sharp null: $Y(w_S, w_{-S}) = Y(w'_S, w_{-S})$ . This implies that changing $W_S$ has no effect on the outcome.
Randomization Test: Instead of relying on sample splitting or noise injection (common in other literature), they propose a randomization-based inference procedure.
- Under the null, potential outcomes are invariant to permutations of $W_S$ .
- The test statistic is computed by permuting $W_S$ within the dataset (preserving $W_{-S}$ ) and recalculating the estimator.
- This yields a finite-sample valid p-value without requiring knowledge of the full joint distribution of $W$ .

C. Sequential Procedure

To handle the uncertainty of whether the null is degenerate ( $\xi=0$ ) or non-degenerate ( $\xi > 0$ ), they propose a sequential procedure:

Step 1: Perform the randomization test for the degenerate null.
- If $H_0$ is not rejected, output the confidence set $\{0\}$ .
- If $H_0$ is rejected, proceed to Step 2.
Step 2: Construct a standard $(1-\alpha)$ confidence interval using the asymptotic distribution of the one-step estimator (valid under non-degenerate nulls).

Validity: The procedure guarantees asymptotic coverage without needing to split the significance level $\alpha$ , as both steps target the same quantity.

3. Key Contributions

Semi-Parametric Efficiency with Independence: The derivation of the EIF that explicitly leverages the independence of factors. This results in an estimator with lower asymptotic variance than standard plug-in or non-parametric influence function estimators.
Robust Inference for Boundary Nulls: A novel randomization-based procedure for testing zero explainability that avoids the efficiency losses associated with sample splitting or noise injection, while maintaining finite-sample validity.
Sequential Inference Framework: A unified approach that seamlessly combines randomization tests (for $\xi=0$ ) and asymptotic confidence intervals (for $\xi>0$ ), providing valid inference regardless of the true value of the explainability.
Practical Applicability: The method operates on pre-collected datasets without requiring oracle access, making it suitable for social science and genetics where data is observational or from complex experiments.

4. Results

Simulation Studies

Efficiency Gain: Simulations demonstrate that the $\phi_{EIF}$ -based estimator (leveraging independence) exhibits significantly lower variance and narrower confidence intervals compared to the $\phi_{IF}$ -based estimator (ignoring independence), particularly when nuisance functions are estimated accurately.
Coverage: Both methods achieve exact coverage rates (approx. 95%) for non-degenerate nulls.
Finite Sample Robustness: The authors note that while $\phi_{EIF}$ is theoretically superior, it requires estimating more complex nuisance components. In small samples with estimated nuisances, the $\phi_{IF}$ -based estimator sometimes shows better stability. They recommend $\phi_{IF}$ for small samples with uncertain nuisance estimation and $\phi_{EIF}$ when nuisances are well-estimated.

Real Data Analysis: Immigration Experiment

The authors applied their method to a conjoint analysis dataset (Hainmueller et al., 2015) regarding public opinion on admitting immigrants.

Total Explainability: They identified several factors with significant non-zero causal explainability, including Job Plan, Education & Profession, Language, and Prior Trips to the U.S. Notably, Gender showed zero explainability (failed to reject the null).
Interaction Effects: The analysis revealed a significant interaction between Job Plan and Job Experience, suggesting that the impact of a job plan on admission decisions depends on the applicant's experience level.
Comparison: The results aligned with existing literature but provided a rigorous causal inference framework with valid confidence intervals, overcoming limitations of previous descriptive analyses.

5. Significance

This paper bridges the gap between causal inference and explainability (XAI) by providing a rigorous statistical foundation for quantifying feature importance in a causal setting.

Theoretical Advancement: It resolves the "degenerate null" problem in semi-parametric statistics, a known difficulty in feature importance testing, by integrating randomization inference with semi-parametric efficiency theory.
Methodological Innovation: The development of an efficient influence function that exploits independence structures offers a blueprint for improving efficiency in other factorial or multi-factor experimental designs.
Practical Impact: By removing the need for oracle access, the method democratizes causal explainability analysis for researchers working with standard observational or experimental datasets in fields like epidemiology, economics, and political science. The sequential procedure ensures that researchers can confidently report "zero importance" or "positive importance" without fear of statistical invalidity.