Robust Causal Inference for EHR-based Studies of Point Exposures with Missingness in Eligibility Criteria

Technical Summary: Robust Causal Inference for EHR-based Studies of Point Exposures with Missingness in Eligibility Criteria

Problem Statement
Electronic health record (EHR) databases offer a valuable alternative to randomized clinical trials for observational research. However, unlike prospective studies where eligibility is rigorously screened, EHR data often contain missing values for variables defining study eligibility (e.g., BMI, A1c levels, medication history). Standard practice involves excluding patients with incomplete eligibility data, which introduces selection bias if the missingness is not completely at random (MCAR). Conversely, researchers often attempt to mitigate this by extending "lookback windows" to find older data, a strategy that risks inappropriately including patients who were ineligible at the study baseline (e.g., patients whose diabetes status was unknown at the time of surgery but is inferred from outdated records).

Existing methods for handling missing eligibility, such as multiple imputation or inverse probability weighting (IPW), often require the correct specification of all component models (outcome, treatment, missingness, and imputation) to yield consistent estimates. Furthermore, these methods often struggle when combined with flexible machine learning (ML) techniques, which typically suffer from slow convergence rates that invalidate asymptotic inference.

Methodology
The authors propose a framework for estimating the Average Treatment Effect on the Treated among the Study Eligible (ATTE), denoted as $\theta_{ATT}^{elig} = E[Y(1) - Y(0) \mid A=1, E=1]$. The approach relies on semiparametric theory to develop estimators that are robust to model misspecification and compatible with modern machine learning.

1. Assumptions and Identification:

   • The method assumes Consistency, Positivity, and No Unmeasured Confounding within the eligible population.
   • Crucially, it assumes Missing At Random (MAR) for eligibility-defining covariates ($L_m$), conditional on fully observed covariates ($L^*$) and treatment ($A$). Specifically, the indicator for complete eligibility data ($R$) is independent of the outcome ($Y$) and missing covariates ($L_m$) given $L^*$ and $A$.
   • The authors derive a novel factorization of the observed data likelihood that separates the treatment mechanism, the ascertainment mechanism ($\eta$), the distribution of missing covariates ($\lambda$), and the outcome model ($\mu$).

2. Estimators:
   The paper introduces two primary estimators based on efficient influence functions (EIF):

   • $\hat{\theta}_{EIF}$ (Efficient Influence Function Estimator): This estimator targets the semiparametric efficiency bound. It utilizes a complex set of "nested" nuisance functions (e.g., $\xi, \gamma, \chi$) that account for the uncertainty in eligibility status by marginalizing over the missing covariates. It requires the estimation of multiple component functions ($\mu, u, \eta, \varepsilon$, etc.) using flexible ML methods (e.g., SuperLearner). Theoretical results show that if the product of the estimation errors of these nuisance functions converges at a rate of $o_P(n^{-1/2})$, the estimator is $\sqrt{n}$-consistent and asymptotically normal.
   • $\hat{\theta}_{IF}$ (Influence Function Estimator): A computationally simpler alternative that avoids some of the deeply nested nuisance functions required by $\hat{\theta}_{EIF}$. While it may not attain the semiparametric efficiency bound, it offers a trade-off between computational complexity and bias reduction.
   • Baseline Comparators: The authors also evaluate a Complete Case (CC) estimator and an Inverse Weighted Outcome Regression (IWOR) estimator to demonstrate the necessity of their proposed approach.

3. Implementation:
   The estimation procedure employs sample splitting (cross-fitting) to ensure valid inference when using nonparametric ML algorithms. This prevents overfitting and allows the estimators to achieve the required convergence rates even when nuisance functions are estimated via complex algorithms like random forests or generalized additive models.

Key Contributions

• Formal Framework for Missing Eligibility: The paper provides the first rigorous causal inference framework specifically addressing selection bias arising from missing eligibility criteria in EHR studies, distinguishing it from standard missing data problems.
• Robustness and Efficiency: The proposed estimators are "multiply robust," meaning they remain consistent if various combinations of nuisance models are correctly specified. They also maintain valid asymptotic inference when using flexible ML methods, overcoming the slow convergence issues that typically plague such approaches.
• Novel Factorization: The derivation of a specific likelihood factorization that facilitates the identification of the ATTE without conditioning on unobserved strata of eligibility.

Application and Results
The methodology is applied to a cohort of 14,809 patients from the Kaiser Permanente DURABLE study undergoing bariatric surgery (Roux-en-Y gastric bypass [RYGB] vs. Sleeve Gastrectomy [SG]) between 2008 and 2011. The study evaluates long-term weight change and Type 2 Diabetes (T2DM) remission.

• Sensitivity to Operationalization: The authors analyzed 40 different operationalizations of eligibility criteria (varying lookback windows for BMI and T2DM).
• Bias in Standard Approaches: The Complete Case (CC) estimator showed significant bias (ranging from -7.1% to -6.2% for weight change and 3.7% to 8.2% for remission), suggesting that excluding patients with missing eligibility data systematically overestimates the treatment effect.
• Performance of Proposed Estimators: The proposed influence-function-based estimators ($\hat{\theta}_{EIF}$ and $\hat{\theta}_{IF}$) attenuated these effects toward the null.
  • For weight change, $\hat{\theta}_{EIF}$ still indicated a benefit for RYGB over SG, though the magnitude was reduced by an average of 19.5% compared to the CC estimator.
  • For T2DM remission, the results were more nuanced. While the CC estimator suggested a substantial advantage for RYGB, the robust estimators showed a much smaller effect (range 0.1% to 7.4%), with confidence intervals often including zero. This suggests that the apparent advantage of RYGB for remission in previous studies may have been driven by selection bias.
• Sensitivity Analysis: A tipping point analysis demonstrated that the conclusion regarding weight loss benefits for RYGB is robust to violations of the MAR assumption, requiring implausibly large differences in missingness probabilities to overturn the results. However, the T2DM remission findings were more sensitive to such violations.

Significance and Claims
The authors claim that their work addresses a critical, often overlooked source of bias in EHR-based research. By demonstrating that standard exclusion strategies can lead to inflated estimates of treatment effects, the paper argues for the adoption of semiparametric, robust methods.

The study concludes that while RYGB likely offers superior long-term weight loss compared to SG, the evidence for a superior T2DM remission rate is less definitive than previously suggested by studies using complete-case analyses. The authors suggest that this robust evidence base supports current clinical guidelines and argues against the necessity of new large-scale randomized trials for these specific outcomes, as sufficient clinical equipoise may not exist. The work provides a practical toolkit for researchers to handle missing eligibility data without resorting to arbitrary exclusion or potentially biased lookback strategies.