📈 economics

A Toolkit for the Study of Treatment-Effect Discontinuities

This paper introduces a comprehensive toolkit for analyzing distributional treatment-effect discontinuities by proposing a framework combining Horizontal and Vertical Discontinuity Analyses, adapting crossing-point asymptotics for zero-crossing detection, and demonstrating the approach through synthetic and real-world PROGRESA data.

Original authors: Alessandro Baldi Antognini, Paolo Verme

Published 2026-06-29

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Alessandro Baldi Antognini, Paolo Verme

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a doctor trying to understand how a new medicine affects a group of patients. The old way of doing things was to look at the average result. You'd say, "On average, this medicine lowers blood pressure by 10 points."

But what if the medicine works wonders for some people, does nothing for others, and actually makes things worse for a few? The "average" hides this messy reality. It's like saying a room is "comfortable" because the average temperature is 70°F, even though one corner is freezing and the other is a furnace.

This paper introduces a new toolkit to stop hiding those messy details. It focuses on where and why the effects of a policy or treatment flip from "good" to "bad" (or vice versa) across a group of people.

Here is the breakdown of their new toolkit, using simple analogies:

1. The Map: The Treatment Effect Curve (TEC)

Instead of just looking at the average, the authors draw a map called the Treatment Effect Curve (TEC).

The Analogy: Imagine a hilly landscape representing all the people in your study, arranged from poorest to richest (or lowest score to highest).
The Curve: This line shows the difference between the "treated" group and the "untreated" group at every single point on that hill.
- If the line is above zero, the treatment is helping people at that spot.
- If the line is below zero, the treatment is hurting them.
- If the line crosses zero, that's the moment the treatment stops helping and starts hurting (or vice versa).

The authors argue that finding these crossing points is the most important part of the map because that's where the story changes.

2. The Two Tools: HDA and VDA

Once they find where the line crosses zero, they use two specific tools to investigate further.

A. Horizontal Discontinuity Analysis (HDA) – "The Neighborhood Comparison"

The Idea: Imagine the crossing point splits your group into two neighborhoods: one where the treatment helps, and one where it hurts.
The Tool: HDA looks at the people living in the "Helpful Neighborhood" and compares them to the people in the "Harmful Neighborhood."
The Goal: It asks, "What makes the people in the helpful group different from the harmful group?" (e.g., Do they have more education? Are they younger?). It uses a smart computer algorithm (called a "Causal Forest") to find these patterns.

B. Vertical Discontinuity Analysis (VDA) – "The Cliff Edge Check"

The Idea: This looks at the exact moment the line crosses zero. Is it a gentle slope, or is it a sharp cliff?
The Tool: It checks the people standing just on the helpful side and just on the harmful side of the crossing point.
The Goal: It tries to see if there is a sudden, sharp jump in results right at that line.
The Catch: The authors warn that unlike a standard policy rule (like "you get a bonus if you score 80%"), these crossing points are found by the data, not set by a rule. So, they treat this as a "detection" of a break in the pattern, not a guaranteed cause-and-effect law.

3. The "Ruler" Problem: How to Measure the Crossing

The paper spends a lot of time on the math of measuring these crossing points accurately.

The Problem: If the line crosses zero very gently (like a shallow ramp), it's hard to tell exactly where it crosses. If it crosses sharply (like a cliff), it's easy to spot.
The Solution: They developed a special statistical ruler that corrects for "blur."
- If the crossing is steep, they can pinpoint the location very precisely.
- If the crossing is flat, their ruler admits, "We can't be sure exactly where this is," and warns researchers not to make decisions based on a guess.

4. Testing the Toolkit

The authors tested their new toolkit in two ways:

Synthetic Data (The Simulation): They created a fake world where they knew exactly where the treatment helped and hurt. They ran their toolkit on this fake data, and it successfully found the crossing points and correctly identified the groups.
Real Data (The PROGRESA Check): They applied it to real data from a Mexican poverty program.
- Result 1: For the people directly receiving money, the treatment helped everyone (the line stayed above zero). No crossing needed.
- Result 2: For the neighbors (who didn't get money but lived nearby), the effects were tiny and messy. The line wiggled around zero, crossing it a few times, but the crossings were so "flat" and blurry that the toolkit correctly said, "We can't find a clear line here." This prevented them from making up a false story about a sharp cutoff.

Summary

This paper gives researchers a better way to look at data. Instead of just saying "The average effect is positive," it allows them to say:

"The treatment helps the middle group, hurts the very poor, and has no effect on the rich. Here is exactly where the switch happens, and here is why the groups are different."

It's a toolkit for finding the hidden stories inside the average, ensuring that policies don't accidentally help one group while hurting another without anyone noticing.

Technical Summary: A Toolkit for the Study of Treatment-Effect Discontinuities

Problem Statement
While Average Treatment Effects (ATEs) have long served as the backbone of impact evaluation, they often obscure important heterogeneities in treatment effects across the outcome distribution. Recent literature on Distributional Treatment Effects (DTEs) has advanced through quantile regression, causal forests, and stochastic dominance testing. However, a specific gap remains in the analysis of treatment-effect discontinuities, defined as points where marginal distributional effects change sign (i.e., where the Cumulative Distribution Functions (CDFs) of treated and control groups cross). Existing tools often struggle to clearly separate regions of opposite-signed effects or to provide rigorous inference on the location and nature of these crossings. The authors argue that identifying where and how distributions cross is critical for policy, as it reveals subgroups that benefit versus those that may be harmed, a nuance lost in mean effects or simple dominance tests.

Methodology
The paper proposes a toolkit centered on the Treatment Effects Curve (TEC), defined as the difference between the CDFs of the treated ( $F_B$ ) and control ( $F_A$ ) groups: $\Delta(x) = F_A(x) - F_B(x)$ . The framework introduces two complementary analytical approaches:

Horizontal Discontinuity Analysis (HDA): This method compares subgroups of observations located in regions where the TEC has opposite signs (e.g., below and above a crossing point). The authors utilize Causal Forests to estimate Conditional Average Treatment Effects (CATEs) within these distinct groups, aiming to identify covariates that explain why effects differ in sign across the distribution.
Vertical Discontinuity Analysis (VDA): This approach focuses on the specific points where the TEC crosses zero. Analogous to Regression Discontinuity Designs (RDD), it examines the local slope of the TEC at these crossing points to determine if the sign switch is abrupt (non-tangential) or gradual.

Inferential Framework
The paper adapts crossing-point asymptotics to the TEC setting, addressing two inferential objects:

Crossing Point Location: The authors propose locating zeros using the un-smoothed difference between empirical CDFs (ECDFs). They derive the asymptotic distribution of the crossing point estimator $\hat{x}_0$ , showing it is $\sqrt{n}$ -consistent with a variance inversely proportional to the square of the local slope $[\Delta'(x_0)]^2$ .
Local Slope Estimation and Testing: To assess the steepness of the crossing (testing $H_0: \Delta'(x_0) = 0$ ), the authors employ bias-corrected kernel density estimators. They derive a bias-corrected Wald statistic to test for non-tangentiality. A key technical contribution is the demonstration that using un-smoothed ECDFs for location avoids the horizontal bias induced by smoothing, while bias correction is essential for the slope estimator to ensure valid confidence intervals for the crossing location.

Key Contributions

Methodological Framework: The paper formalizes HDA and VDA as systematic tools for studying treatment effect heterogeneity. It clarifies the identifying assumptions, noting that VDA relies on endogenously determined thresholds (data-driven crossing points) rather than exogenous policy cutoffs, distinguishing it from standard RDD.
Inferential Procedure: The authors operationalize a procedure for locating crossing points and testing their non-tangentiality. They provide a Monte Carlo study documenting the finite-sample performance of these estimators, including detection rates, false-positive rates, bias, RMSE, and confidence interval coverage.
Empirical Illustration: The toolkit is demonstrated on two datasets:
- Synthetic Data: A World Bank synthetic census where treatment effects are constructed to create specific crossing patterns. This validates the workflow (TEC estimation, inference, HDA, VDA) in a controlled environment.
- Real Data (PROGRESA): A diagnostic application to Mexico's PROGRESA conditional cash transfer program. The analysis reveals that while eligible households show first-order stochastic dominance, ineligible households exhibit subtle, weak spillover effects with sign changes that are too flat to support threshold-based discontinuity analysis.

Results

Monte Carlo Evidence: The study finds that detection power for crossing points is highly dependent on the steepness of the crossing. Steep crossings are detected with high power even in moderate samples, while flat crossings require large samples for reliable detection. The false-positive rate for spurious crossings is negligible in samples of $n \geq 4,000$ .
Bias Correction: The explicit bias correction of the local slope significantly improves the recovery of the true slope and the calibration of the crossing-location confidence intervals. Without correction, intervals are overly conservative; with correction, they achieve near-nominal coverage.
Synthetic Data Application: The toolkit successfully identified two crossing points in the synthetic data, separating the population into three groups with negative, positive, and negative effects, respectively. HDA using causal forests identified pre-treatment expenditure as the primary driver of these differences.
PROGRESA Application: The analysis confirmed positive dominance for eligible households. For ineligible households, the TEC revealed weak negative spillovers at the bottom of the distribution and near-zero effects elsewhere. Crucially, the inference procedure correctly diagnosed that the sign changes were too flat (tangential) to justify a threshold-based VDA, preventing spurious claims of structural breaks.

Significance and Claims
The paper claims to provide a necessary complement to existing DTE instruments (stochastic dominance, quantile regression, causal forests). Its primary significance lies in offering a diagnostic toolkit that allows researchers to:

Visually and statistically identify where treatment effects switch sign along the outcome distribution.
Determine whether these sign switches are sharp enough to warrant subgroup analysis (HDA) or local discontinuity analysis (VDA).
Avoid over-interpreting weak or gradual distributional changes as sharp discontinuities.

The authors emphasize that the toolkit is particularly valuable when CDFs cross, a scenario where ATEs are least informative. They position the TEC not as a replacement for quantile regression or causal forests, but as a precursor that identifies where to look for heterogeneity, which can then be explained by covariate-based methods. The paper concludes modestly, acknowledging that VDA relies on estimated thresholds and that future work is needed to develop formal inference procedures that fully account for the uncertainty of these endogenous thresholds in real-world applications.