Estimating Treatment Effects Under Bounded Heterogeneity

1. Problem Statement

In causal inference, researchers frequently face a trade-off between model specification and estimation precision:

• The "Short" Regression: Commonly used specifications (e.g., $Y_i = D_i\beta + X_i'\gamma + u_i$) assume constant treatment effects. While these estimators are precise and remain feasible even when covariate distributions have limited overlap (i.e., some subgroups have no treated or untreated units), they are generally biased if treatment effects actually vary with covariates.
• The "Long" Regression / Flexible Models: Fully flexible models (e.g., including interaction terms $D_i \times X_i$) allow for heterogeneous treatment effects. However, they often suffer from high variance (imprecision) and can become infeasible (undefined) when there is a complete lack of overlap in certain covariate cells.
• The Gap: Existing methods either rely on strong overlap assumptions (making them inapplicable in many real-world settings) or assume constant effects (introducing bias). There is a lack of methods that provide valid inference for the Average Treatment Effect (ATE) when overlap is limited and treatment effects are allowed to vary, provided that the variation is not unbounded.

2. Methodology

The authors propose a framework based on bounded treatment effect heterogeneity and a new estimator called regulaTE (regularized treatment effect).

A. Theoretical Framework

• Data Generating Process (DGP): The paper assumes a linear model where the outcome $Y_i$ depends on treatment $D_i$, covariates $X_i$, and a conditional average treatment effect (CATE) $\tau(x_i)$.
  $$Y_i = D_i \tau(x_i) + X_i'\gamma + \epsilon_i$$
  The CATE is modeled as $\tau(x_i) = \beta_{ATE} + \tilde{x}_i'\delta$, where $\delta$ captures heterogeneity.
• The Constraint (VCATE Bound): Instead of assuming $\delta = 0$ (constant effects) or leaving $\delta$ unrestricted, the authors impose a bound on the Variance of the Conditional Average Treatment Effect (VCATE):
  $$VCATE = E_n[(\tau(x_i) - E_n[\tau(x_i)])^2] \leq C^2$$
  Here, $C$ is a user-specified sensitivity parameter representing the maximum standard deviation of treatment effects across the population.
• Partial Identification: Under complete lack of overlap, the ATE is not point-identified. However, the VCATE bound provides partial identification, restricting the range of possible treatment effects in non-overlapping cells, allowing for valid inference.

B. The Estimator: regulaTE

To address the bias-variance trade-off, the authors propose a generalized ridge regression estimator:
$$\min_{\beta, \gamma, \delta} \frac{1}{n} \|Y - D\beta - X\gamma - (D \circ \tilde{X})\delta\|^2 + \lambda \delta' V_x \delta$$

• Mechanism: The penalty term $\lambda \delta' V_x \delta$ shrinks the heterogeneity coefficients $\delta$ toward zero.
  • If $\lambda \to \infty$, the estimator converges to the short regression (constant effects).
  • If $\lambda \to 0$, it approaches the long regression (fully flexible), provided overlap exists.
• Optimality: The authors prove that this ridge estimator lies on the bias-variance frontier. For a given bound $C$, the penalty parameter $\lambda$ is chosen to minimize the length of the resulting confidence interval (CI) in a homoskedastic Gaussian setting.
• Inference (Heterogeneity-Aware CIs):
  • The authors construct Fixed-Length Confidence Intervals (FLCI) that account for the worst-case bias induced by the heterogeneity bound $C$.
  • The CI is given by: $\hat{\beta}_{\lambda^*} \pm \chi_{\lambda^*, C}$, where the critical value $\chi$ is adjusted based on the worst-case bias relative to the standard error.
  • This approach remains valid even under limited or complete lack of overlap, unlike standard IPW or long-regression methods.

C. Sensitivity Analysis

Since $C$ is not estimated from the data (due to impossibility results by Armstrong and Kolesár, 2018), the method functions as a sensitivity analysis:

1. Researchers vary $C$ from 0 (constant effects) to larger values.
2. They observe how the point estimate and CI width change.
3. They identify a "breakdown value" ($C^*$): the smallest $C$ at which the treatment effect becomes statistically insignificant.
4. If $C^*$ is economically plausible (based on prior literature or subgroup analysis), the original finding is considered robust.

3. Key Contributions

1. Formalization of Angrist and Pischke's Intuition: The paper rigorously formalizes the argument that "short" regressions can be preferable to unbiased but noisy estimators if heterogeneity is limited. It provides a formal mechanism to quantify "limited."
2. Handling Limited Overlap: Unlike semi-parametric estimators (e.g., IPW) that fail or become highly variable when propensity scores approach 0 or 1, regulaTE remains well-defined and provides valid inference even when overlap is completely absent in some cells.
3. Optimal Bias-Variance Trade-off: The paper demonstrates that the generalized ridge estimator is the optimal linear estimator for minimizing CI length under a VCATE constraint, outperforming simple bias-correction of the short regression (which yields excessively wide intervals).
4. Practical Implementation: The authors provide an R package (regulaTEr) that implements these estimators, allowing researchers to easily conduct sensitivity analyses and visualize breakdown frontiers.

4. Results

Simulations

• Calibration: The authors calibrated simulations using data from Angrist (1998) on military service and earnings, a setting known for treatment effect heterogeneity and limited overlap.
• Performance:
  • Coverage: The regulaTE CI achieves correct coverage (95%) across a range of heterogeneity bounds $C$, whereas the uncorrected short regression suffers from severe undercoverage.
  • Efficiency: The regulaTE CI is significantly tighter than the bias-corrected short regression CI. The bias-corrected short regression becomes excessively wide as $C$ increases, while regulaTE adapts the penalty to maintain precision.
  • Robustness: The method performs well even under heteroskedasticity when using robust standard errors.

Empirical Applications

1. Mothers' Pension (Aizer et al., 2016):
   • Context: Estimating the effect of cash transfers on child longevity.
   • Challenge: Lack of overlap in 11 counties (all applicants treated).
   • Result: The short regression showed a significant positive effect. The regulaTE sensitivity analysis showed the result remained significant up to a breakdown value of $C^* \approx 0.75\%$. This was deemed economically plausible compared to a benchmark of $1\%$, suggesting the finding is robust to moderate heterogeneity.
2. Staggered Adoption (Dobkin et al., 2018 / Sun and Abraham, 2021):
   • Context: Effect of unexpected hospitalization on medical spending.
   • Challenge: All units treated in the final period, making the long regression infeasible.
   • Result: The static (short) regression showed a significant increase in spending. regulaTE confirmed robustness up to a breakdown value of C^* = \1,583$. This was consistent with observed heterogeneity in the original study ($3,000 in year 1 vs. $1,000 in year 3), whereas bias-corrected short regression CIs suggested fragility.

5. Significance

This paper offers a pragmatic and theoretically sound solution to a pervasive problem in applied econometrics: how to estimate treatment effects when data is sparse (limited overlap) and effects are likely heterogeneous.

• For Practitioners: It moves beyond the binary choice of "assume constant effects" vs. "assume full flexibility." It allows researchers to explicitly state their tolerance for heterogeneity and see how it impacts their conclusions.
• For Policy Analysis: By providing valid inference under partial identification, it enables causal claims in settings (like policy rollouts or rare events) where traditional methods fail due to lack of overlap.
• Theoretical Advancement: It bridges the gap between ridge regression (typically a prediction tool) and optimal causal inference, showing that penalizing heterogeneity coefficients is the optimal strategy for minimizing worst-case error in treatment effect estimation.

In summary, regulaTE transforms the "short regression" from a potentially biased heuristic into a rigorous, sensitivity-aware inference tool, making it a vital addition to the causal inference toolkit for settings with limited overlap and uncertain heterogeneity.