Which Covariates to Adjust for? Specification-robust Causal Inference in Observational Studies

1. Problem Statement

In observational causal inference, estimating the Average Treatment Effect (ATE) typically requires adjusting for a set of covariates to satisfy the ignorability (unconfoundedness) assumption. However, in practice, domain knowledge often yields multiple plausible adjustment sets, but it is impossible to empirically determine which specific set (if any) satisfies ignorability.

• The Challenge: Different candidate adjustment sets can yield conflicting estimates and confidence intervals.
• Limitations of Existing Remedies:
  • Union/Intersection: Adjusting for the union or intersection of candidate sets can produce biased estimates if the chosen set is invalid (e.g., including a collider induces M-bias; omitting a confounder leaves residual bias).
  • Convex Hull: Taking the convex hull of confidence intervals from all candidate sets guarantees coverage if at least one set is valid. However, as sample size $n \to \infty$, the width of this interval does not vanish; it converges to the convex hull of the expected estimates of the different sets, leaving the ATE only partially identified up to a constant order.
  • Averaging: Simply averaging estimates conflates substantive disagreement with statistical noise.

The authors aim to develop a procedure that provides a single point estimate and a confidence interval with a width shrinking at the parametric rate ($n^{-1/2}$), valid under the assumption that at least one candidate adjustment set is valid, without requiring the researcher to know which one.

2. Methodology: Specification-Robust Reweighting

The core innovation is shifting the target of inference from the original population to a reweighted population. This approach mirrors strategies used for handling overlap violations (like trimming or overlap weighting) but applies them to specification uncertainty.

A. The Optimization Problem

Let $S_1, \dots, S_K$ be $K$ candidate adjustment sets. Let $\tau(X_{S_k})$ be the conditional average treatment effect (CATE) estimator using set $S_k$.
The goal is to find a weighting function $w^*(X_{S_\cap})$ (where $S_\cap = \bigcap_{k=1}^K S_k$ is the intersection of all sets) that minimizes the Kullback-Leibler (KL) divergence from the original distribution $P$, subject to the constraint that all candidate estimands agree on the reweighted population:

$$w^* = \arg\min_{w=w(X_{S_\cap})} D_{KL}(wP \parallel P)$$
$$\text{subject to } \quad E_P[w(X_{S_\cap})\tau(X_{S_1})] = \dots = E_P[w(X_{S_\cap})\tau(X_{S_K})].$$

B. Theoretical Solution via Exponential Tilting

Using results from information theory (I-projection), the authors show that the optimal weights take the form of exponential tilting:
$$w^*(X_{S_\cap}) = \frac{\exp(g(X_{S_\cap})^\top \lambda^*)}{E_P[\exp(g(X_{S_\cap})^\top \lambda^*)]}$$
where $g(X_{S_\cap}) = E_P[\Delta \tau(X) \mid X_{S_\cap}]$ represents the conditional mean of the differences between the contrasts of the candidate sets, and $\lambda^*$ is a finite-dimensional vector found by minimizing a convex objective function.

Key Interpretation (Lemma 1): If at least one $S_k$ is valid (satisfies ignorability), the common estimand identified by the reweighted population, $\tau^R$, is exactly the ATE of the reweighted population. Thus, the method recovers a causal quantity even if the original population's ATE is not point-identified.

C. Estimation Algorithm

The procedure involves two main steps (Algorithm 1 & 2):

1. Weight Estimation: Use flexible non-parametric estimators (e.g., Random Forests) to estimate the contrasts $\tau(X_{S_k})$ and their conditional means $g(X_{S_\cap})$. Solve the empirical convex optimization problem to obtain transfer weights $\hat{w}$.
2. Bias-Corrected Estimation: Construct a Specification-Robust Estimator ($\hat{\tau}^{SR}_{bc}$) using Augmented Inverse Probability Weighting (AIPW).
   • The estimator aggregates the reweighted estimates from all $K$ sets using affine weights $\nu$.
   • Crucially, it includes a bias-correction term to account for the uncertainty in estimating the transfer weights $\hat{w}$ and the nuisance functions. This ensures the estimator achieves the parametric convergence rate of $n^{-1/2}$.

D. Extensions

• Protecting Covariates: The method can be extended to preserve the distribution of specific functions of the covariates (e.g., keeping the mean of age constant) by adding linear constraints to the optimization problem.
• Infeasibility Handling: If no reweighted population exists where all sets agree (indicating fundamental incompatibility of assumptions), the method can identify clusters of compatible sets.

3. Key Contributions

1. Specification-Robust Procedure: A novel method that returns a single, precise estimate and confidence interval valid as long as at least one candidate adjustment set is valid.
2. Parametric Rate Convergence: Unlike the convex hull approach (which has a non-vanishing width), the proposed confidence interval width shrinks at the standard $n^{-1/2}$ rate.
3. Theoretical Guarantees:
   • Proof of asymptotic normality for the bias-corrected estimator.
   • Proof of uniform validity over a class of distributions where at least one set is valid.
   • Derivation of the asymptotic variance accounting for uncertainty in outcome models, propensity scores, and transfer weights.
4. Diagnostic Tools: Introduction of diagnostic plots (comparing covariate distributions of original vs. reweighted populations) to assess the interpretability of the new target population.

4. Results

Simulation Studies

• Example 1 (Omitted Confounder): One set omits a confounder, leading to bias. The proposed method correctly targets the ATE of the reweighted population with coverage near 95% and significantly narrower intervals than the convex hull.
• Example 2 (M-Bias): One set includes a collider, inducing bias. The proposed method maintains coverage and precision, whereas naive union/intersection approaches fail.
• Performance: In simulations, the proposed method achieved nominal coverage (95%) while producing confidence intervals substantially narrower than the convex hull (which often had coverage but massive width).

Real-Data Application: 401(k) Eligibility

• Dataset: Analyzed the effect of 401(k) eligibility on net financial assets (Abadie, 2003; Chernozhukov et al., 2018).
• Setup: Four nested adjustment sets were considered, varying in the inclusion of covariates like family size, income, and pension participation.
• Outcome: The proposed method produced a confidence interval ~77% narrower than the naive convex hull interval while maintaining nominal coverage.
• Interpretability: Diagnostic plots (Figure 1) showed that the reweighted population (using only age and education as common covariates) remained very close to the original distribution, suggesting the shift was minimal and the result scientifically meaningful.

5. Significance and Implications

• Resolving Multiverse Analysis: This paper offers a solution to the "multiverse" problem in causal inference. Instead of reporting a wide range of conflicting estimates or arbitrarily picking one, it synthesizes them into a single, credible estimate.
• Pragmatic Shift in Target: It formalizes the idea that when identification is impossible on the original population due to specification uncertainty, researchers should shift the target to the "closest" population where identification is possible. This is analogous to how overlap weighting shifts the target to the region of common support.
• Robustness without Over-Conservatism: It avoids the extreme conservativeness of the convex hull (which assumes the worst-case scenario for the width) while maintaining rigorous statistical validity.
• Applicability: The method is applicable in fields like economics, epidemiology, and social sciences where causal graphs are uncertain, and multiple adjustment strategies are plausible but untestable.

In summary, Ghosh and Rothenh¨ausler provide a mathematically rigorous and practically useful framework for causal inference under specification uncertainty, transforming a problem of "which model to choose" into a problem of "reweighting to a consensus population."