Deconfounding Scores and Representation Learning for Causal Effect Estimation with Weak Overlap

1. Problem Statement

In observational causal inference, estimating treatment effects (specifically the Average Treatment Effect on the Treated, ATT) relies on two key assumptions: ignorability (unconfoundedness) and overlap (positivity).

• The Challenge: While ignorability assumes observed covariates contain all confounders, overlap requires that the distributions of covariates in treated and control groups have common support.
• The Curse of Dimensionality: In high-dimensional settings, the overlap assumption often becomes implausible. Even when it holds, significant differences in feature distributions between groups lead to high variance in estimators (e.g., Inverse Probability Weighting, Doubly Robust estimators), making them brittle and unreliable.
• Existing Limitations: Current methods often either trim extreme propensity scores (discarding data) or change the target estimand to a population with better overlap. Representation learning offers a potential solution by mapping high-dimensional covariates to lower-dimensional spaces, but existing approaches often fail to guarantee that the new representation preserves the unbiasedness of the causal estimate (i.e., they may introduce confounding bias).

2. Methodology

The authors propose a framework centered on Deconfounding Scores and a new metric called Overlap Divergence.

A. Deconfounding Scores

The paper defines a deconfounding score as a representation $\phi(X)$ that preserves the identification of the causal effect. Formally, $\phi(X)$ is a deconfounding score if:
$$\tau = \tau_{\phi(X)}$$
where $\tau$ is the true ATT and $\tau_{\phi(X)}$ is the estimand adjusted for $\phi(X)$.

• Characterization: The authors prove that a representation is a deconfounding score if and only if it introduces zero "confounding bias." This bias is expressed as the conditional covariance between the outcome model $m_0(X)$ and the density ratio (or propensity score) given the representation:
  $$\text{Bias} \propto E_{P_0} \left[ \text{Cov}_{P_0} \left( m_0(X), \frac{dP_1}{dP_0}(X) \mid \phi(X) \right) \right] = 0$$
• Continuum: This definition creates a continuum of valid representations.
  • Balancing Scores (e.g., Propensity Score): Functions of $X$ that make treatment assignment independent of covariates.
  • Prognostic Scores: Functions of $X$ that predict the potential outcome under control.
  • General Case: Any representation that renders the outcome model and the propensity score conditionally independent given the representation.

B. Overlap Divergence

To quantify the quality of overlap in a representation, the authors introduce the Overlap Divergence, defined as the $\chi^2$-divergence (plus 1) between the control and treated distributions of the representation:
$$O(\phi(X)) := E_{P_0} \left[ \left( \frac{dP_1}{dP_0}(\phi(X)) \right)^2 \right] = \chi^2(P_1^{\phi(X)} || P_0^{\phi(X)}) + 1$$

• Theoretical Link: They prove that the overlap divergence tightly bounds the semiparametric efficiency bound (the minimum achievable asymptotic variance) of estimators adjusting for $\phi(X)$. Minimizing $O(\phi(X))$ is necessary (and often sufficient) to minimize estimator variance.

C. Optimization Strategy

The goal is to find a deconfounding score $\phi(X)$ that minimizes $O(\phi(X))$.

• Non-parametric Result: The authors prove that any representation $\phi(X)$ improves overlap compared to raw covariates $X$ (i.e., $O(\phi(X)) \leq O(X)$). The improvement is driven by excluding variables that predict treatment but not the outcome.
• Gaussian/GLM Analytical Solution: In a setting with Gaussian covariates and Generalized Linear Models (GLMs) for treatment and outcome, the authors derive a closed-form family of deconfounding scores.
  • These scores lie on a hyperbola in the space spanned by the prognostic score vector ($\alpha$) and the propensity score vector ($\beta$).
  • Key Theoretical Finding: Within this family, Prognostic Scores (which depend only on the outcome model) are overlap-optimal. They minimize the overlap divergence, whereas Balancing Scores (Propensity Scores) maximize it. This holds even when the propensity model is misspecified, provided the outcome model is correct.

3. Key Contributions

1. Definition of Deconfounding Scores: A rigorous characterization of the complete class of representations that preserve unbiased causal estimation, generalizing propensity and prognostic scores.
2. Overlap Divergence Metric: Introduction of a metric that explicitly links the degree of overlap to the asymptotic variance of causal estimators, providing a theoretical justification for why reducing overlap divergence improves estimation.
3. Analytical Optimality of Prognostic Scores: Under Gaussian/GLM assumptions, the paper proves that prognostic scores are the optimal deconfounding scores for minimizing overlap divergence, outperforming propensity scores.
4. Empirical Validation: Extensive experiments showing that using deconfounding scores (particularly prognostic ones) consistently improves Root Mean Squared Error (RMSE) over raw covariates, especially in low-overlap regimes.

4. Experimental Results

The authors evaluated their approach on simulated data and semi-synthetic datasets (IHDP, ACIC 2016, HC-MNIST).

• Simulated Data (Gaussian/GLM):

  • When the outcome model was correctly specified, estimators using prognostic scores achieved the lowest RMSE and bias, significantly outperforming raw covariates and propensity-score-based methods.
  • When the propensity model was correctly specified but the outcome model was misspecified, balancing scores performed better.
  • When both models were misspecified, deconfounding scores (specifically an "equiangular" score combining both) still generally outperformed raw covariates.
  • Variance Reduction: As predicted by theory, moving from balancing scores toward prognostic scores consistently reduced the standard deviation (variance) of the estimates.

• Semi-Synthetic Data:

  • On real-world derived datasets (IHDP, ACIC, HC-MNIST), which do not strictly satisfy Gaussian assumptions, there was always at least one deconfounding score that improved performance over raw covariates.
  • Prognostic scores were the most frequent winners, aligning with the theoretical intuition that outcome-predictive features are more robust for overlap in high dimensions.
  • The "equiangular" score (a mix of prognostic and balancing) often served as a robust compromise when model specification was uncertain.

5. Significance and Implications

• Theoretical Insight: The paper resolves a long-standing intuition in causal inference: variables that predict treatment but not the outcome should be excluded to reduce variance. The authors provide the first mathematical proof of this using the overlap divergence framework.
• Practical Utility: The proposed method offers a "drop-in" replacement for covariates in standard estimators (IPW, AIPW, TMLE). It allows researchers to leverage representation learning without sacrificing the unbiasedness guarantees required for causal inference.
• Handling Weak Overlap: By shifting the focus from raw high-dimensional covariates to lower-dimensional deconfounding scores (specifically prognostic scores), the method effectively mitigates the "curse of dimensionality" regarding overlap, enabling more stable estimation in high-dimensional observational studies.
• Future Directions: The authors suggest extending these results to non-Gaussian settings and integrating the overlap divergence constraint into deep learning architectures for causal inference, bridging the gap between theoretical optimality and flexible neural network representations.