Non-parametric Causal Inference in Dynamic Thresholding Designs

Technical Summary: Non-parametric Causal Inference in Dynamic Thresholding Designs

Problem Statement
Regression discontinuity (RD) designs are a standard tool for non-parametric causal inference in static settings, where treatment is assigned based on whether a running variable crosses a fixed threshold. However, many consequential decisions in healthcare, education, and public policy occur in dynamic settings where the running variable evolves over time and is influenced by past treatments. For instance, a patient's blood sugar level (the running variable) in a subsequent year depends on whether they received lifestyle interventions (treatment) in the previous year.

Existing RD literature focuses on cross-sectional settings where units receive treatment once. Direct application of standard RD methods to dynamic settings fails because:

1. Past treatments alter the distribution of future running variables, creating complex feedback loops.
2. Defining causal estimands for every possible treatment history leads to a "curse of dimensionality" as the time horizon grows.
3. Reduced-form approaches (e.g., regressing future outcomes on the initial running variable) fail to capture the full dynamic treatment effects and are not directly interpretable as policy-relevant parameters.

Methodology
The authors propose a framework for non-parametric causal inference in dynamic thresholding designs that accounts for treatment dynamics without imposing restrictive parametric assumptions.

• Target Estimand (Marginal Policy Effect): The paper re-interprets the classical RD estimand not merely as the local average treatment effect (LATE) at the cutoff, but as a marginal policy effect. Specifically, it is defined as the ratio of the derivative of the expected discounted welfare (value function) with respect to the threshold $c$, to the derivative of the expected discounted treatment frequency with respect to $c$:
  $$\tau^{RD} := \frac{\partial V^Y(\pi_c)}{\partial c} \Big/ \frac{\partial V^A(\pi_c)}{\partial c}$$
  where $V^Y$ and $V^A$ represent the net-present expected outcomes and treatment counts, respectively, under policy $\pi_c$. This ratio characterizes the cost-benefit tradeoff of marginally lowering the threshold.

• Identification via Q-Functions: Drawing on reinforcement learning (RL) and Markov decision processes (MDPs), the authors utilize the Q-function (action-value function) to identify the policy gradients. They show that under the g-formula (which holds if the system follows a Markov structure, even with unobserved states), the policy gradients can be expressed as a discounted sum of Q-function differences at the threshold, weighted by the conditional density of the running variable.
  $$-\frac{\partial V^Y(\pi_c)}{\partial c} = \sum_{t=0}^T \gamma^t E\left[ (Q^Y_{c,t}(S_t, c, 1) - Q^Y_{c,t}(S_t, c, 0)) f_t(c | S_t) \right]$$
  This characterization allows the dynamic marginal policy effect to be identified using data collected at a single status-quo threshold.

• Estimation (Twice-Discounted Local Linear Regression): To estimate $\tau^{RD}$, the authors propose a tailored local linear regression (LLR) estimator. Unlike standard static RD, this estimator:

  1. Uses discounted sums of future outcomes ($\Gamma^Y_{i,t}$) and treatments ($\Gamma^A_{i,t}$) as the dependent variables, rather than immediate outcomes. This approximates the Q-functions and incorporates long-term downstream effects.
  2. Applies temporal discounting weights ($\gamma^t$) to the observations, mirroring the structure of the policy gradient theorem.
  3. Includes time fixed effects to absorb time-specific shifts common across units, reducing asymptotic variance.
  4. Takes a ratio form ($\hat{\tau}^Y / \hat{\tau}^A$), analogous to fuzzy RD estimators, to account for the uncertainty in how changing the threshold affects future treatment frequency.

Key Results

• Consistency: Under standard regularity conditions (continuity of conditional response functions and densities at the cutoff, smoothness of Q-functions), the proposed estimator is consistent for the dynamic marginal policy effect $\tau^{RD}$.
• Asymptotic Distribution: The estimator achieves the standard non-parametric convergence rate of $n^{-2/5}$ (with bandwidth $h_n \sim n^{-1/5}$). The asymptotic variance aggregates uncertainty across all time periods, weighted by $\gamma^{2t}$.
• Variance Estimation: The authors propose a unit-clustered sandwich variance estimator that consistently estimates the asymptotic variance, accounting for arbitrary dependence across time within a trajectory.
• Bandwidth Selection: A data-driven bandwidth selection rule is derived, extending the Imbens-Kalyanaraman (2012) method to the dynamic setting by using residualized outcomes and unit-clustered variance estimates.

Empirical Validation
The authors validate their approach through two numerical experiments:

1. Autoregressive Simulator: A stylized data-generating process where the state evolves based on past treatments. The proposed method correctly recovers the dynamic marginal policy effect, whereas standard LLR (targeting partial equilibrium effects) and naive long-run LLR (ignoring future threshold crossings) fail to provide valid inference or exhibit significant bias.
2. UVa/Padova Type-1 Diabetes Simulator: Using an FDA-approved simulator for continuous glucose monitoring, the authors emulate a dynamic thresholding rule for insulin administration. The proposed method maintains nominal coverage for confidence intervals across various discount factors, while competing methods either severely undercover (standard LLR) or produce uninformative, extremely wide intervals (naive long-run LLR).

Significance and Claims
The paper claims to provide the first non-parametric causal inference results for dynamic thresholding designs with generality comparable to standard cross-sectional RD results.

• Policy Relevance: By framing the estimand as a marginal policy effect, the approach directly enables cost-benefit analysis for resource allocation (e.g., determining whether to lower a health screening threshold).
• Robustness: The method avoids the restrictive parametric assumptions required by previous dynamic RD approaches (e.g., Cellini et al., 2010) and the strong conditional independence assumptions required by others (e.g., Hsu and Shen, 2024).
• Theoretical Bridge: The work bridges the literature on regression discontinuity designs with reinforcement learning and policy gradient theorems, demonstrating that policy gradient concepts can be adapted for observational causal inference in settings with temporal interference.

The authors note that while their specific local linear regression estimator is consistent, questions remain regarding the efficiency of other algorithms and the adaptation of bias-aware inference procedures (like Calonico et al., 2014) to the dynamic setting.