📊 statistics

Towards a holistic understanding of Selection Bias for Causal Effect Identification

This paper establishes necessary and sufficient conditions for identifying the Average Treatment Effect (ATE) under selection bias by leveraging weak assumptions on probability classes to characterize propensity and selection probabilities, thereby extending existing graphical criteria with strictly weaker conditions for a more comprehensive understanding of causal effect identification.

Original authors: Yiwen Qiu, Filip Kovacevic, Shimeng Huang, Peter Spirtes, Francesco Locatello

Published 2026-05-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yiwen Qiu, Filip Kovacevic, Shimeng Huang, Peter Spirtes, Francesco Locatello

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to figure out if a new type of fertilizer makes plants grow taller. You go to a garden, pick a bunch of plants, measure them, and calculate the average growth.

But here's the catch: you didn't pick plants randomly. You only picked the ones that were already growing in the sunniest, most fertile part of the garden because that's where the gardener let you walk. The plants in the shady, rocky part of the garden are missing from your data.

If you calculate the average growth based only on the sunny plants, you might think the fertilizer works miracles. But if you could see the whole garden, you'd realize the fertilizer actually has a much smaller effect. The plants in the sunny spot were already doing well; the ones in the shade needed the help the most, but you never saw them.

This is Selection Bias. It happens when the data you collect isn't a fair representation of the whole group you care about.

The Problem: The "Healthy Volunteer" Trap

The paper starts with a real-world example: Biobanks. These are massive databases of health information used to study diseases. But often, the people who sign up to join these studies are "healthy volunteers." They tend to be wealthier, more educated, and generally healthier than the average person.

If researchers use this data to figure out how a new drug affects the entire population, they might get it wrong. The drug might look amazing on the healthy, wealthy volunteers, but fail miserably on the people who are sicker or poorer (who weren't in the study).

The Old Way vs. The New Way

For a long time, scientists tried to fix this by looking at a map of how variables are connected (called a Graph or DAG). They would say, "Ah, I see that the selection happens here, connected to that variable. If I block that path, I can fix the math."

The paper's critique: This is like trying to fix a leaky boat by only looking at the specific hole you can see. In the real world, you often don't know exactly where the selection bias is happening, or the map is too complicated to draw perfectly.

The paper's new approach: Instead of looking at the map, they look at the shape of the data itself.

They propose a new set of rules (mathematical conditions) that say: "Even if we don't know exactly how the data got selected, if the data follows certain smooth, predictable patterns (like a bell curve or a specific type of spread), we can mathematically 'stretch' the data we do have to guess what the missing data looks like."

The Magic Trick: Extrapolation

Think of it like this: Imagine you have a puzzle, but someone has cut out a big chunk of the corner.

Old method: "I can't solve this because I don't know what's in the missing corner."
This paper's method: "If I know the puzzle pieces are made of a specific type of smooth plastic that follows a certain curve, I can use the edge of the missing piece to mathematically predict exactly what the rest of the corner must look like, even without seeing it."

The authors call this extrapolation. They use advanced statistics (specifically "truncated statistics") to take the "truncated" or cut-off data and reconstruct the full picture.

What They Found

The paper proves two main things:

When it works: They identified specific types of data distributions (like Gaussian, Laplace, or Pareto distributions—basically, common shapes that data often takes) where this "stretching" trick is mathematically guaranteed to work. If your data fits these shapes, you can recover the true average effect of a treatment, even if your data is heavily biased.
When it fails: They also proved that if the data is too chaotic or doesn't fit these smooth patterns, you simply cannot recover the truth. No amount of math can fix it.

The Solution in Action

The paper doesn't just talk theory; they built a tool (an algorithm) to do this.

Step 1: Look at the data you have.
Step 2: Figure out the "shape" of the data (is it a bell curve? is it skewed?).
Step 3: Use a mathematical model to "fill in the blanks" of the missing, biased parts.
Step 4: Calculate the true average effect.

They tested this on fake data and real-world health data (from the "All of Us" research program). In almost every case, their method was much more accurate than the standard methods used today, which often just ignore the bias or try to guess it based on simple rules.

The Bottom Line

This paper is like giving scientists a new pair of glasses. Before, if data was biased, they often had to throw their hands up and say, "We can't trust these results." Now, they have a rigorous way to say, "Even though our data is biased, as long as it follows these specific patterns, we can mathematically reconstruct the truth and give you a reliable answer."

It moves the field from "We need to know exactly how the bias happened" to "We just need to know the general shape of the data, and we can fix it."

Technical Summary: Towards a Holistic Understanding of Selection Bias for Causal Effect Identification

1. Problem Statement

Selection bias is a pervasive challenge in observational studies, where data points are preferentially included based on observed variables, leading to a discrepancy between the selected sub-population and the target population. A classic example is "healthy volunteer bias" in biobanks, where respondents often possess higher socio-economic status and better health than the general population.

The core problem addressed is the identifiability of the Average Treatment Effect (ATE) in the presence of such bias. Specifically, the paper distinguishes between two goals:

s-ATE-sub: Estimating the ATE within the selected sub-population.
s-ATE-full: Estimating the ATE for the entire target population using only data from the selected sub-population.

The authors argue that estimating the s-ATE-full is significantly more challenging because it requires extrapolating beyond the observed data distribution. Existing methods often rely on strong assumptions, such as identifying the exact location of selection in a causal Directed Acyclic Graph (DAG) (e.g., selection-backdoor criteria) or imposing restrictive distributional assumptions (e.g., non-Gaussian noise in additive noise models). These approaches often fail in scenarios where selection depends on complex interactions or where the selection mechanism is not fully known.

2. Methodology

2.1. Theoretical Framework

The paper proposes a distribution-based framework rather than relying solely on graphical criteria. It defines the problem using three distribution classes:

Propensity Scores ( $P_{t|x}$ ): The probability of treatment given covariates.
Covariate-Outcome Distributions ( $P_{xy(t)}$ ): The joint distribution of covariates and potential outcomes.
Selection Probabilities ( $P_{s|xyt}$ ): The probability of a data point being included in the dataset.

The authors introduce Condition 1, a necessary and sufficient condition for the identifiability of s-ATE-full. This condition states that for any two compatible tuples of distributions that yield different ATEs, their observed distributions (conditioned on selection $S=1$ ) must differ. Formally, if two underlying distributions $P$ and $Q$ have different ATEs, there must exist a point $(x, y, t)$ in the support where the product of the selection probability, propensity score, and outcome distribution differs between $P$ and $Q$ .

2.2. Key Theoretical Results

The paper demonstrates that Condition 1 holds under specific, relatively weak assumptions:

Deterministic Selection: If selection is deterministic (i.e., certain regions of the data are completely censored), ATE is identifiable if the covariate-outcome distributions belong to a smooth family (e.g., polynomial parametrizations) and the propensity scores satisfy a weak overlap condition ( $c$ -overlap). This leverages results from truncated statistics to extrapolate the original distribution from the truncated (observed) one.
Non-Deterministic Selection: If selection is probabilistic but bounded away from zero (d-selection-overlap), ATE is identifiable for a wide range of common distribution families, including Gaussian, Laplace, Pareto, and Log-normal distributions. The intuition is that for these families, differences in the mean outcomes manifest as unbounded differences in the density ratios, ensuring the observed distributions differ if the ATEs differ.

2.3. Connection to Graphical Criteria

The authors show that their distributional conditions generalize existing graphical criteria. For instance:

The Selection-Backdoor criterion (Correa et al., 2019) is a special case of Condition 1.
Outcome-dependent selection results (Zhang et al., 2016), which previously required non-Gaussian additive noise models, are extended to include Gaussian and other heavy-tailed distributions under the proposed framework.
The framework does not require pinpointing the exact location of selection in the DAG, circumventing a major practical limitation of previous graphical approaches.

2.4. Estimation Algorithm

When the distribution classes satisfy the identified conditions, the paper proposes a three-stage estimation procedure (Algorithm 1):

Propensity Estimation: Estimate the propensity score $\hat{e}(x)$ and identify the overlap region $B$ where $c < \hat{e}(x) < 1-c$ .
Modeling: Estimate the conditional outcome distribution $\hat{P}(y|x, t)$ $\hat{P} (y ∣ x, t)$ and the selection bias function $\hat{\beta}(x, y, t)$ $\hat{β} (x, y, t)$ . This is achieved via:
- Maximum Likelihood Estimation (MLE): Minimizing negative log-likelihood with regularization to prevent indeterminacy.
- Score Matching: Minimizing the distance between the gradient of the log-density of the model and the observed data, which is invariant to scaling factors.
Extrapolation: Re-weight the observed samples by $1/\hat{\beta}$ to recover the population density and compute the counterfactual ATE.

3. Key Contributions

General Definition and Conditions: The paper provides a general definition of selection bias and establishes necessary and sufficient conditions (Condition 1) for ATE identifiability under selection, moving beyond specific graphical structures.
Weaker Assumptions: Unlike prior works that require non-Gaussian noise or specific functional forms, this framework allows for selection dependent on arbitrary observed variables and works with common distribution families (Gaussian, Laplace, etc.) under mild smoothness assumptions.
Extension of Identifiability: The results extend the scope of identifiable scenarios to include settings previously considered unidentifiable in the literature, particularly those involving outcome-dependent selection with Gaussian noise.
Practical Estimators: The paper introduces concrete estimation methods (MLE and Score Matching with bias correction) that leverage truncated statistics to extrapolate from selected samples to the full population.

4. Experimental Results

The authors evaluate their methods on both synthetic and semi-synthetic datasets:

Synthetic Data: Experiments were conducted with various noise distributions (Gaussian, Laplace) and functional forms (additive, multiplicative). The proposed methods (MLE+β and SM+β) consistently outperformed baselines like Naive Inverse Propensity Weighting (IPW) and Polynomial Regression. The corrected methods significantly reduced bias, whereas baselines failed to account for the selection mechanism.
Semi-Synthetic Data (All of Us): Using real-world covariate distributions from the NIH "All of Us" dataset (Type 2 Diabetes and BMI), the authors generated synthetic outcomes. While the bias was not fully eliminated due to the complexity of real-world distributions (e.g., very low propensity scores leading to weak overlap), the proposed methods substantially decreased bias compared to baselines, demonstrating robustness in challenging, realistic settings.

5. Significance and Claims

The paper claims to offer a holistic understanding of selection bias by shifting the focus from graphical topology to distributional properties. Its significance lies in:

Broadening Applicability: By relaxing the need for specific graphical structures and strong parametric assumptions (like non-Gaussian noise), the framework makes causal effect identification feasible in a wider range of practical scenarios.
Theoretical Rigor: It provides the first necessary and sufficient conditions for s-ATE-full identifiability under general selection bias, clarifying exactly when and why identification is possible.
Methodological Advancement: The integration of truncated statistics and score matching offers a viable path for estimating causal effects in the presence of complex selection mechanisms where traditional adjustment methods fail.

The authors modestly acknowledge limitations, noting that the assumptions regarding distribution classes, while weaker than previous works, are still required for the theoretical guarantees. They suggest future work could focus on relaxing these assumptions, applying the framework to causal discovery, and improving estimation techniques for highly complex real-world data distributions.