Causal Effect Estimation with TMLE: Handling Missing Data and Near-Violations of Positivity

Technical Summary: Causal Effect Estimation with TMLE: Handling Missing Data and Near-Violations of Positivity

Problem Statement
Observational studies investigating treatment effects frequently encounter missing data across multiple variables (exposure, outcome, and confounders). While Targeted Maximum Likelihood Estimation (TMLE) is a robust method for estimating Average Treatment Effects (ATE) due to its double-robustness properties, its performance in conjunction with various missing data handling strategies remains uncertain, particularly under complex missingness mechanisms and violations of the positivity assumption. Positivity violations—where certain subgroups have near-zero probability of receiving a treatment—can severely impact causal identifiability. Furthermore, the recoverability of the ATE depends heavily on the specific missingness mechanism, often conceptualized via missingness-directed acyclic graphs (m-DAGs). There is a need to systematically evaluate how different missing data methods (both non-multiple imputation and multiple imputation) perform when paired with TMLE across varying degrees of positivity violations and data complexities.

Methodology
The authors employed a dual-simulation framework to evaluate eight missing data methods combined with TMLE:

1. Model-Based Simulations:

   • Data Generating Processes (DGPs): Five distinct DGPs were constructed, progressively increasing in complexity (incorporating binary, continuous, categorical, and gamma-distributed variables, as well as Gaussian copulas for interdependencies).
   • Positivity Violations: Three levels of positivity violation were induced (Level 1: ≤1%, Level 2: ~10%, Level 3: ~30%) by manipulating interaction coefficients in the exposure model.
   • Missingness Mechanisms: Five m-DAGs (A–E) were utilized to represent Missing Completely at Random (MCAR), Missing at Random (MAR), and Missing Not at Random (MNAR) scenarios. These included mechanisms where missingness depended on the exposure, outcome, or other incomplete variables.
   • Methods Evaluated:
     • Non-MI: Complete Case (CC) analysis, Extended TMLE (incorporating an outcome-missingness model), and Extended TMLE with Missing Covariate Missing Indicator (Ext MCMI).
     • Multiple Imputation (MI): Predictive Mean Matching (PMM), PMM with interaction terms (Int), Classification and Regression Trees (CART), Random Forest (RF), and Joint Multivariate Normal (Amelia).
   • Analysis: TMLE with SuperLearner (ensemble learning) was used for estimation, with propensity score truncation applied to mitigate extreme weights.

2. Design-Based Simulations:

   • To bridge the gap between theoretical models and empirical reality, the authors utilized data from the 'WASH Benefits Bangladesh' study.
   • An undersmoothed Highly Adaptive Lasso (HAL) was employed to flexibly model the conditional distributions of exposure and outcome, capturing complex dependencies without strict parametric assumptions.
   • Synthetic datasets were generated via bootstrapping from the HAL fits, and missingness was induced according to the same m-DAG structures used in the model-based simulations.

Key Contributions and Results
The study provides a comprehensive comparison of bias, Root Mean Squared Error (RMSE), and confidence interval coverage across these settings:

• Recoverability and Bias:

  • In recoverable scenarios (m-DAGs A, B, and C under low-to-moderate positivity violations), Complete Case (CC) and Extended TMLE (Ext) generally exhibited the lowest bias.
  • Ext (incorporating an outcome-missingness model) proved the most robust non-MI method across both simulation types.
  • Ext MCMI performed poorly in model-based simulations (higher bias) but performed comparably to CC and Ext in the design-based simulation.
  • In non-recoverable scenarios (m-DAGs D and E), no method yielded unbiased estimates, though bias magnitudes varied.

• Impact of Positivity Violations:

  • As positivity violations increased, bias generally increased for all methods.
  • Non-MI methods (CC and Ext) tended to maintain lower bias than MI methods as positivity worsened, likely because MI methods rely on extrapolation beyond observed data support.
  • However, non-MI methods suffered from undercoverage (confidence intervals failing to capture the true parameter) under severe positivity violations.

• Performance of Multiple Imputation:

  • MI CART (Classification and Regression Trees) emerged as a superior MI strategy. It achieved the lowest RMSE and maintained coverage rates closest to the nominal 95% level across all scenarios, including non-recoverable settings with severe positivity violations.
  • Parametric MI (PMM, Int, Amelia) generally showed higher bias and RMSE, particularly under severe positivity violations, due to restrictive model assumptions and extrapolation issues.
  • MI RF (Random Forest) showed inconsistent performance, performing well in some complex DGPs but poorly in others, often exhibiting higher bias than MI CART.

• Trade-offs:

  • A clear trade-off exists between bias and coverage. Non-MI methods (specifically Ext) are preferred for bias reduction in recoverable settings but risk undercoverage. MI CART is preferred when accurate confidence intervals (coverage) are the priority, even in the presence of positivity violations.

Significance and Claims
The paper claims to offer practical guidance for researchers handling missing data in causal inference, particularly in epidemiological point-exposure studies. The authors emphasize that:

1. Recoverability is conditional: Unbiased estimation is possible under certain MNAR mechanisms (e.g., m-DAG C) without requiring the strict MAR assumption, provided specific conditions are met.
2. Method selection depends on the goal: If the primary objective is bias reduction in recoverable settings, Extended TMLE is recommended. If the priority is obtaining valid confidence intervals (coverage), MI with CART is recommended, as it demonstrates robustness against positivity violations and complex data structures.
3. Positivity matters: The severity of positivity violations significantly influences the choice of method; non-MI methods are more robust to bias in these scenarios, while MI methods (specifically CART) better preserve interval validity.

The authors conclude that while TMLE provides a strong analytical framework, the choice of missing data handling strategy is critical and must be aligned with the specific missingness mechanism (assessed via m-DAGs) and the severity of positivity violations. They recommend Extended TMLE for bias reduction and MI CART for accurate interval estimation.