Assumption-Lean Differential Variance Inference for Heterogeneous Treatment Effect Detection

Technical Summary: Assumption-Lean Differential Variance Inference for Heterogeneous Treatment Effect Detection

Problem Statement
In clinical studies, the Conditional Average Treatment Effect (CATE) is the standard tool for detecting heterogeneous treatment effects (HTE). However, CATE-based inference relies critically on the inclusion of all true effect modifiers in the baseline covariate set. When effect modifiers are omitted due to resource constraints, measurement error, or incomplete mechanistic understanding, CATE-based methods fail to detect violations of the homogeneous treatment effect hypothesis. This can mislead decision-makers into assuming treatment effects are uniform when they are not. Furthermore, high-dimensional inference for CATE often requires unverifiable sparsity assumptions and has been shown to produce elevated false positive and negative rates. Existing Variable Importance Parameter (VIP) approaches for treatment effect modifiers (TEM-VIPs) face similar limitations, as they also require the effect modifiers to be present in the data. Additionally, standard CATE inference ignores residual heterogeneity where the mean treatment effect is constant, but the variance of potential outcomes differs—a scenario relevant to risk-sensitive decision-making.

Methodology
The authors propose a framework for "assumption-lean" inference that detects HTE without requiring access to effect modifiers. The core of this approach is the estimation of differential variance estimands, which contrast the variances of potential outcomes under treatment and control.

1. Estimands:

   • Absolute Differential Variance ($\Psi$): Defined as the difference between the standard deviations of potential outcomes: $\Psi(P_0) = \sigma(P_0; 1) - \sigma(P_0; 0)$.
   • Relative Differential Variance ($\Lambda$): Defined as the ratio of the variances: $\Lambda(P_0) = \sigma^2(P_0; 1) / \sigma^2(P_0; 0)$.
   • The paper establishes that if $\Psi(P_0) \neq 0$ (or $\Lambda(P_0) \neq 1$), the "second-moment homogeneous treatment effect hypothesis" is violated. This hypothesis is a relaxed version of the sharp homogeneous effect hypothesis, allowing for different residual noise terms as long as their conditional second moments are equal. Crucially, these estimands can detect heterogeneity even when effect modifiers ($V$) are missing from the data ($V = \emptyset$), provided conditional exchangeability holds.

2. Identifiability:
   Under standard causal assumptions (conditional exchangeability, consistency, and positivity), the potential outcome variances are identifiable from observed data. The authors show that these estimands remain valid for detecting heterogeneity in Randomized Controlled Trials (RCTs) without conditioning on covariates, though conditioning on prognostic variables improves efficiency.

3. Estimators:
   The authors derive causal machine learning estimators based on Efficient Influence Functions (EIFs) within a nonparametric model:

   • One-Step Estimators: Constructed using the EIF of the potential outcome variances.
   • Targeted Maximum Likelihood Estimators (TMLE): Developed by tilting nuisance parameter estimators (outcome regression and propensity score) to solve the EIF equation.
   • Cross-Fitted Versions: Variants of the above estimators using sample splitting to avoid Donsker class conditions.
   • The estimators utilize flexible machine learning algorithms (e.g., Super Learner, Random Forests) for nuisance parameter estimation.

4. Asymptotic Properties:
   The paper proves that these estimators are doubly robust (consistent if either the propensity score or the outcome regression is correctly specified) and asymptotically linear under mild conditions. Specifically, they achieve $n^{-1/2}$ convergence rates if the product of the estimation errors of the nuisance parameters converges at a rate faster than $n^{-1/2}$. This allows for the construction of valid Wald-type confidence intervals and hypothesis tests in nonparametric settings.

Key Results

• Simulation Study 1 (Double Robustness): Demonstrated that the proposed estimators exhibit negligible empirical bias when either the propensity score or the outcome regression is correctly specified, confirming double robustness.
• Simulation Study 2 (Asymptotic Linearity): Showed that the estimators achieve asymptotic linearity in finite samples, with coverage probabilities converging to nominal levels as sample size increases. Cross-fitted estimators showed superior performance in larger samples, mitigating bias-variance trade-offs associated with flexible machine learning.
• Simulation Study 3 (Detection under Measurement Error): In an RCT setting with a mismeasured effect modifier, the differential variance estimators maintained high statistical power to detect HTE. In contrast, a standard causal forest-based omnibus test for CATE heterogeneity lost power as measurement error increased, failing to detect heterogeneity when the effect modifier was completely mismeasured.
• Application (TTM and TTM2 Trials): The methodology was applied to re-analyze the Target Temperature Management (TTM) and TTM2 trials. While the TTM trial showed no significant differential variance (likely due to small sample size), the TTM2 trial rejected the second-moment homogeneous treatment effect hypothesis ($p < 0.01$). However, the estimated difference in standard deviations was less than one day, suggesting that while statistical heterogeneity exists, it may not be clinically actionable.

Significance and Claims
The paper claims to offer a practical alternative to CATE-based inference for detecting treatment effect heterogeneity in settings where effect modifiers are missing or mismeasured. By focusing on the variance of potential outcomes, the authors provide a method that:

• Does not require the identification or measurement of specific effect modifiers.
• Offers interpretable metrics (absolute or relative variance contrasts) for risk-sensitive decision-making.
• Provides formal hypothesis testing with asymptotic guarantees in both randomized and observational studies.
• Serves as a complementary sensitivity analysis to CATE and TEM-VIP workflows; a significant result in differential variance inference following a null CATE result can indicate that heterogeneity exists but is driven by unmeasured factors.

The authors emphasize that while their method detects the presence of heterogeneity, it does not identify the source or directionality of the effect, a limitation inherent to marginal variance contrasts. Nevertheless, they argue that detecting actionable departures from homogeneity is valuable even when the underlying mechanism remains unknown.