Optimal Treatment Policy Estimation for Recurrent Events with a Competing Terminal Event: An Instrumented Difference-in-Differences Approach

Technical Summary: Optimal Treatment Policy Estimation for Recurrent Events with a Competing Terminal Event

Problem Statement
The development of data-driven optimal treatment policies for chronic diseases often relies on large administrative health datasets. However, these datasets are frequently compromised by unmeasured confounding, rendering standard ignorability assumptions unverifiable. A specific and critical gap exists in the literature regarding recurrent event outcomes (e.g., hospitalizations, disease exacerbations) that are subject to a competing terminal event (e.g., death). Existing methods for optimal policy learning often fail to account for the complex interplay between recurrent events and mortality. Specifically, unconstrained optimization may yield "degenerate" policies that appear optimal by artificially reducing recurrent event counts simply by increasing the risk of death. Furthermore, while Instrumental Variable (IV) and Difference-in-Differences (DiD) methods exist to address unmeasured confounding, they are typically designed for continuous outcomes or average treatment effects, not for the constrained optimization of individualized treatment regimes in the presence of competing risks.

Methodology
The authors propose a novel framework based on Instrumented Difference-in-Differences (iDID) to estimate optimal treatment policies under unmeasured confounding for recurrent events with a terminal event.

• Framework and Assumptions: The approach utilizes a binary treatment $A$, a binary instrument $Z$, and a time period indicator $L$. It relies on standard iDID assumptions (Consistency, Positivity, Random Sampling, Trend Relevance, Independence/Exclusion Restriction, No Unmeasured Common Effect Modifier, Stable Treatment Effect, and Non-informative Censoring). Crucially, the iDID design allows the instrument to have a direct effect on the outcome level, provided this effect does not modify the outcome trend, relaxing the strict exclusion restriction of standard IVs.
• Constrained Optimization: To prevent degenerate policies that trade survival for reduced recurrent events, the objective is formulated as a constrained optimization problem. The goal is to minimize the expected cumulative number of recurrent events subject to a constraint that the marginal survival probability under the optimal policy must exceed that of a designated baseline behavioral policy (derived from a pseudo-propensity score).
• Estimation Strategies:
  1. Inverse Probability Weighted (IPW) Estimators: Two distinct IPW identification formulas are derived. The first relies on the uncensored subsample, while the second utilizes cumulative event information over time, allowing individuals at risk at time $t$ to contribute even if censored later. Both adjust for unmeasured confounding via the iDID structure and for censoring via inverse probability weighting.
  2. Multiply Robust (AIPW) Estimator: To protect against model misspecification, a multiply robust estimator is developed using von Mises expansions. This estimator achieves consistency if any one of several subsets of nuisance models (involving the instrument assignment, treatment trend, censoring survival, and outcome models) is correctly specified.
• Asymptotic Theory: The paper establishes the consistency and asymptotic normality of the proposed estimators. Under specific regularity conditions and rate assumptions for nuisance parameter estimation (achievable via cross-fitting and machine learning), the estimated policy parameters converge at a rate of $O_p(n^{-1/3})$, and the policy gain estimator is asymptotically normal.

Key Contributions

1. Novel Framework for Recurrent Events: The paper extends optimal policy learning to recurrent event processes with terminal events, a setting pervasive in chronic disease management but largely unaddressed in the policy learning literature.
2. Handling Degenerate Policies: The authors explicitly address the challenge of avoiding policies that reduce adverse events by increasing mortality. This is achieved through a constrained optimization formulation that penalizes solutions violating a survival constraint relative to a baseline policy.
3. Multiply Robust Estimation: A multiply robust estimator is developed that remains consistent even if only a subset of the complex nuisance models (required for iDID, censoring, and recurrent events) is correctly specified.
4. Theoretical Guarantees and Empirical Validation: The authors establish large-sample properties (consistency and asymptotic normality) and demonstrate through simulations that the multiply robust estimator outperforms existing IPW approaches in finite samples, particularly in terms of accuracy and inferential validity.

Results

• Simulations: In simulation studies across various sample sizes ($n=1000$ to $5000$), the proposed AIPW estimator consistently achieved the highest Percentage of Correct Decisions (PCD) compared to IPW alternatives. The AIPW estimator showed negligible root-$n$ scaled bias and empirical coverage probabilities near the nominal 95% level for policy gain estimation, validating the theoretical asymptotic properties.
• Medicare Data Application: The framework was applied to a national Medicare dataset (2016–2023) to optimize first-line treatment strategies for Type 2 Diabetes Mellitus (T2DM), comparing Metformin and Glucagon-like peptide-1 (GLP-1) receptor agonists.
  • The analysis utilized provider preference changes as an instrument to address unmeasured confounding.
  • The non-IV IPW approach suggested a drastic reallocation, recommending Metformin for only 4.78% of patients.
  • In contrast, the iDID-based AIPW estimator, which accounts for unmeasured confounding, recommended a more conservative reallocation, suggesting Metformin for 21.16% of patients.
  • The optimal policy identified by the iDID AIPW method assigned GLP-1 RAs to frailer, female, and non-Hispanic White patients, aligning with emerging clinical guidelines regarding cardiovascular and renal protection.
  • The estimated policy gain indicated an expected reduction of approximately 0.034 adverse composite events per patient relative to the observed behavioral policy, achieved without compromising survival.

Significance and Claims
The paper claims to fill a critical gap between causal effect estimation and decision-making in the context of recurrent events and competing risks. By integrating the iDID framework with constrained optimization and multiply robust estimation, the authors provide a method that is both robust to unmeasured confounding and clinically safe (preventing policies that sacrifice survival). The application to the Medicare dataset demonstrates the practical utility of the method, showing that ignoring unmeasured confounding can lead to extreme and potentially harmful treatment reallocations. The authors position this work as a step toward more reliable, generalizable, and reproducible optimal treatment policies for chronic diseases using real-world data. They acknowledge limitations, such as the restriction to linear decision boundaries and static instruments, suggesting future work should explore dynamic treatment settings and more flexible policy classes.