🤖 machine learning

ConfoundingSHAP: Quantifying confounding strength in causal inference

The paper introduces ConfoundingSHAP, a scalable Shapley-based method that leverages TabPFN to quantify and attribute confounding strength to individual covariates in observational studies, thereby helping researchers identify which variables act as confounders without exhaustive model refitting.

Original authors: Marie Brockschmidt, Santo M. A. R. Thies, Maresa Schröder, Dennis Frauen, Valentyn Melnychuk, Maximilian Muschalik, Eyke Hüllermeier, Stefan Feuerriegel

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Marie Brockschmidt, Santo M. A. R. Thies, Maresa Schröder, Dennis Frauen, Valentyn Melnychuk, Maximilian Muschalik, Eyke Hüllermeier, Stefan Feuerriegel

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Hidden Influencer"

Imagine you are a doctor trying to figure out if a new medicine actually works. You look at your patient records. You see that patients who took the medicine lived longer than those who didn't.

But wait! There's a catch. The doctor didn't flip a coin to decide who got the medicine. Instead, the doctor gave the medicine to the sickest patients because they needed it most.

Now, if you just compare the two groups, the medicine might look like it killed people (because the sick ones died anyway), or it might look like a miracle (if the doctor only gave it to the strongest patients). The problem is that the sickness level is a "confounder." It influenced both the decision to give the medicine and the final outcome (death or survival).

In real-world data, we don't have a list of "confounders." We just have a long list of variables (age, weight, blood pressure, income, etc.). We don't know which ones are messing up our results.

The Solution: ConfoundingSHAP

The authors created a tool called ConfoundingSHAP. Think of it as a "Bias Detective" or a "Confounding Strength Meter."

Its job is to look at a long list of variables and answer one specific question: "Which of these variables is actually causing the confusion (bias) in our results?"

How It Works: The "Team Game" Analogy

To understand how the tool works, imagine a game of Team Building where the goal is to solve a puzzle (finding the true effect of a treatment).

The Players: Every variable in your dataset (age, income, blood pressure) is a player on the team.
The Goal: The team wants to know how much "noise" or "confusion" remains in the data if they leave certain players out.
The Game: The tool tests every possible combination of players (teams).
- Team A: Includes everyone. (The "Perfect" team).
- Team B: Leaves out "Blood Pressure."
- Team C: Leaves out "Age" and "Income."
The Score: For every team, the tool calculates how much the result changes compared to the "Perfect" team.
- If leaving out "Blood Pressure" makes the result go haywire, then "Blood Pressure" is a heavy hitter. It is a strong confounder.
- If leaving out "Shoe Size" changes nothing, then "Shoe Size" is irrelevant.

The tool uses a mathematical concept called Shapley Values (named after a Nobel Prize-winning game theorist) to fairly split the "blame" for the confusion among all the players. It tells you exactly how much each variable contributed to the mess.

Why This Is Different (The "CATE" Trap)

The paper points out a common mistake people make. Usually, data scientists use tools to explain Treatment Effect Heterogeneity (CATE).

CATE Explainer: "Who makes the medicine work differently for different people?" (e.g., "It works better for men than women").
ConfoundingSHAP: "Who is messing up the average result because they influenced who got the medicine?"

The Analogy:
Imagine a race.

CATE asks: "Who runs faster on grass vs. mud?" (It looks at how the surface changes the runner's speed).
ConfoundingSHAP asks: "Who rigged the starting line?" (It looks at who decided which runner started at the front and which started at the back, skewing the final results).

The paper shows that standard tools often miss the "riggers" (confounders) because they are too focused on the "surface conditions" (effect modifiers). ConfoundingSHAP is built specifically to find the riggers.

The "Magic Engine" (TabPFN)

Calculating this for every possible team combination is incredibly slow. If you have 20 variables, there are over a million combinations. Doing this the old way would take forever.

The authors used a "magic engine" called TabPFN.

Old Way: To test a new team, you have to rebuild the whole engine from scratch.
TabPFN Way: Imagine a super-smart robot that has already seen millions of games. You just whisper the rules of the current team to it, and it instantly predicts the outcome without needing to relearn anything. This makes the process fast enough to be useful on real computers.

What They Found (The Results)

The authors tested their tool on three types of scenarios:

Fake Data (Synthetic): They created a fake world where they knew exactly who the "bad actors" (confounders) were.
- Result: ConfoundingSHAP correctly pointed a giant finger at the bad actors and ignored the innocent bystanders (like instruments or noise).
Randomized Trials (RCT): They looked at a real medical trial where treatment was assigned randomly (like flipping a coin).
- Result: Since there was no "rigging" in a fair coin flip, ConfoundingSHAP correctly said, "Zero confusion here!" The bias scores dropped to near zero.
Real Medical Data (SUPPORT Study): They looked at a real study about heart catheterization.
- Result: The tool identified variables that doctors actually know are important (like how sick the patient was upon arrival) as the main sources of confusion. It also correctly ignored irrelevant things like patient ID numbers.

The Bottom Line

ConfoundingSHAP is a new tool that helps researchers figure out which variables in their data are secretly messing up their causal conclusions. It uses a fair "team game" math system to assign blame for the bias, and it uses a smart AI engine to do the math quickly.

It doesn't tell you if the medicine works (that's up to the researcher); it tells you which variables you need to pay attention to so you can stop the confusion and get a clearer picture.

Technical Summary: ConfoundingSHAP

Problem Statement

In observational causal inference, treatment assignment is often non-random, leading to systematic differences between treated and untreated groups. Variables that influence both treatment assignment and outcomes are known as confounders. While randomized clinical trials eliminate this issue, observational studies require researchers to identify and adjust for confounders to estimate causal effects (e.g., Conditional Average Treatment Effect, CATE) accurately.

A critical challenge remains: Which observed covariates actually drive confounding strength?
Existing explanation methods for causal machine learning typically target different objects:

Propensity scores: Explaining treatment assignment mechanisms.
Outcome models: Explaining prognostic risk.
CATE estimators: Explaining treatment effect heterogeneity (effect modifiers).

The authors argue that applying standard SHAP (Shapley Additive exPlanations) to CATE estimators is ill-suited for identifying confounders. CATE explanations focus on variation in treatment effects; if a confounder shifts potential outcomes for both treated and control groups similarly, its contribution cancels out in the CATE ( $\tau(x) = \mu_1(x) - \mu_0(x)$ ), rendering it invisible to standard feature importance methods. Consequently, there is a lack of methods to specifically quantify and attribute observed confounding strength to individual covariates.

Methodology: ConfoundingSHAP

The paper introduces ConfoundingSHAP, a Shapley-based method designed to attribute confounding strength to individual covariates. The method operates by defining a cooperative game directly over a confounding bias functional, rather than a predictive model.

1. Defining the Confounding Bias Functional

The core of the method is quantifying the residual confounding bias ( $b_S$ ) when restricting the adjustment set to a subset of covariates $S$ .

Observational Contrast ( $\delta_S$ ): The difference in observed outcomes between treated and control units, conditional on the subset $X_S$ .
Causal Contrast ( $\tau_S$ ): The projection of the full-adjustment CATE onto the subset $X_S$ (i.e., $E[\tau(X) | X_S]$ ).
Residual Bias: $b_S(x_S) = \delta_S(x_S) - \tau_S(x_S)$ .

If $S$ contains all confounders, $b_S = 0$ . If $S$ is insufficient, $b_S \neq 0$ (unless biases perfectly cancel out).

2. The Cooperative Game

The authors define a cooperative game where:

Players: The individual covariates.
Coalitions: Subsets of covariates included in the adjustment set.
Value Function ( $\nu$ ): Defined as the negative expected residual bias, $\nu(S) = -E[b_S(X_S)]$ $ν (S) = - E [b_{S} (X_{S})]$ .
- The negative sign ensures that a positive Shapley value indicates a covariate's contribution to reducing bias (i.e., its importance as a confounder).
Shapley Values: The Shapley value $\phi_j$ for covariate $j$ decomposes the total bias reduction (from no adjustment to full adjustment) into individual contributions.

3. Scalable Estimation via TabPFN

Evaluating the value function for every coalition requires fitting models for each subset $S$ , which is computationally prohibitive ( $O(2^p)$ ). To address this, the authors propose a scalable estimation strategy using TabPFN (a tabular foundation model):

Precomputation: Fit a full outcome model on all covariates to generate pseudo-outcomes (full-adjustment CATE estimates).
In-Context Learning: For each queried coalition $S$ , use TabPFN to estimate the observational contrast and the projected causal contrast without retraining model parameters. TabPFN conditions on the training data provided in the context, treating the subset $S$ as the input features.
Shapley Approximation: Use efficient Shapley estimators (e.g., RegressionMSR) to approximate the values based on a budget of coalition evaluations, avoiding exhaustive enumeration.

Key Contributions

Novel Method: Introduction of ConfoundingSHAP, the first Shapley-based method specifically designed to measure and attribute observed confounding strength at the covariate level.
Theoretical Distinction: A formal demonstration that confounding strength is distinct from treatment effect heterogeneity. The paper proves that standard CATE explanations can miss confounders entirely if their effects on outcomes are symmetric across treatment arms.
Scalable Algorithm: A practical implementation using TabPFN that avoids the exhaustive refitting required by exact bias functional evaluation, enabling application to medium and high-dimensional datasets.
Empirical Validation: Comprehensive experiments showing the method's ability to recover ground-truth confounders in synthetic data, detect the disappearance of confounding in randomized trials (RCTs), and identify clinically plausible confounders in real-world observational studies.

Experimental Results

The authors evaluate ConfoundingSHAP across three main settings:

Synthetic Data with Known Ground Truth:
- In low-dimensional (4 covariates) and medium-dimensional (17 covariates) settings, ConfoundingSHAP correctly assigns high Shapley values to true confounders and near-zero values to instruments, effect modifiers, and outcome-only predictors.
- In contrast, standard SHAP applied to a CATE estimator failed to identify the confounder in specific scenarios where the confounder's effect canceled out in the treatment effect calculation.
- The method demonstrated high rank stability across multiple random seeds.
Randomized Controlled Trial (ACTG Dataset):
- When applied to a real-world RCT dataset (ACTG 175) where treatment was randomly assigned, ConfoundingSHAP correctly identified that confounding strength disappeared (values near zero), even when treatment assignment was artificially modified to introduce confounding in a semi-synthetic variant, the method successfully recovered the introduced confounders.
Real-World Observational Study (SUPPORT/RHC):
- Applied to the SUPPORT right-heart catheterization study, the method identified variables such as the Acute Physiology Score (APS1) as having high confounding strength, aligning with clinical knowledge that severity of illness drives both treatment decisions and mortality.
- Irrelevant variables (e.g., patient ID) were correctly attributed near-zero confounding strength.

Significance and Claims

The paper claims that ConfoundingSHAP fills a critical gap in causal machine learning explainability. By shifting the focus from "what drives treatment effect heterogeneity" to "what drives confounding bias," the method provides actionable insights for:

Variable Selection: Helping researchers choose minimal sufficient adjustment sets in observational studies.
Study Design: Informing the design of prospective observational studies (e.g., emulated target trials) by highlighting which covariates are critical for comparability.
Transparency: Offering a diagnostic tool to understand which measured covariates create incomparability between treatment groups.

The authors emphasize that ConfoundingSHAP does not certify causal unbiasedness (which depends on unmeasured confounding) but rather quantifies the bias arising from restricted adjustment of the observed covariates. It serves as a complementary tool to domain knowledge, causal diagrams, and sensitivity analysis.