A Design-Based Approach to Testing and Inference in (Quasi-)Experiments with Spillovers

Technical Summary: A Design-Based Approach to Testing and Inference in (Quasi-)Experiments with Spillovers

1. The Problem
Economic policies often generate spillover effects that extend beyond direct targets, propagating through spatial, market, and social networks. To make these effects tractable, applied researchers typically summarize complex treatment assignment vectors using low-dimensional "exposure mappings" (e.g., the share of treated neighbors within a specific radius). However, two critical challenges remain:

1. Specification Uncertainty: Economic theory rarely dictates the precise functional form (e.g., ring vs. gravity kernel) or the parameters (e.g., the specific radius or decay rate) of these mappings. Researchers often select these based on institutional intuition or informal diagnostics, treating them as fixed rather than estimated.
2. Inference Limitations: Standard errors and confidence intervals in existing literature typically condition on a chosen specification, ignoring the uncertainty introduced by the selection of the exposure map itself. Furthermore, there is no formal design-based method to test whether a proposed mapping is consistent with the observed data under the known assignment mechanism.

2. Methodology
The paper proposes a design-based framework for testing and estimating exposure mappings, relying on the randomization (or quasi-randomization) of the treatment assignment as the sole source of randomness. Potential outcomes are treated as fixed for the experimental sample.

• Exposure Sufficiency and Orthogonality: The core theoretical insight is that a correctly specified exposure mapping $g(W; X_i, \theta_0)$ implies an exposure sufficiency property: conditional on the exposure $g(W; X_i, \theta_0)$, the remaining details of the assignment vector $W$ carry no additional information about the outcome $Y_i$. This implies a set of orthogonality conditions (moment restrictions). Specifically, for any outcome transformation $\phi$ and design function $\psi$, the "design-side residual" $R_{i,\theta}\psi(W) = \psi(W) - E[\psi(W) | g(W; X_i, \theta)]$ is orthogonal to $\phi(Y_i)$ at the true parameter $\theta_0$.
• Design-Side Residualization: The authors construct these moments by "recentering" design functions. The conditional expectation $E[\psi(W) | g(W; X_i, \theta)]$ is computed using the known assignment distribution $D_n$ (via simulation or enumeration), requiring no assumptions about the outcome model.
• Estimation and Testing:
  • Estimation: The tuning parameter $\theta$ is estimated via Generalized Method of Moments (GMM) by minimizing a quadratic form of the sample moments.
  • Specification Testing: When the system is overidentified (more moments than parameters), a design-based $J$-test assesses whether the proposed exposure mapping is consistent with the assignment design. Rejection indicates that residual assignment variation remains predictive of outcomes, suggesting the map is misspecified.
• Large-Sample Theory: The paper establishes consistency and asymptotic normality for the GMM estimator under spatial and network dependence, utilizing the affinity-set dependence framework (Chandrasekhar et al., 2023). This framework accommodates both smooth kernels (e.g., gravity) and non-smooth specifications (e.g., ring radii). The authors derive an efficiency bound for the class of unit-level product residualized moments and show that a sieve GMM estimator with a growing dictionary attains this bound.
• Two-Stage Inference: The framework propagates the uncertainty of the estimated exposure map $\hat{\theta}$ into downstream policy estimands (e.g., average treatment effects or multipliers) using a stacked Z-estimator approach, ensuring that standard errors reflect both experimental variation and specification uncertainty.

3. Key Contributions

• Formalizing Exposure Map Selection: The paper provides the first design-based methodology to formally test and estimate the parameters of exposure mappings, moving beyond informal sensitivity analyses.
• Design-Based Orthogonality: It introduces a novel "design-side residualization" strategy that generates valid moment conditions for testing exposure sufficiency without requiring a structural outcome model.
• Robust Inference under Dependence: It establishes uniform consistency and asymptotic normality for GMM estimators in the presence of complex spatial and network dependence, covering both smooth and non-smooth exposure maps.
• Efficiency Characterization: The paper characterizes the efficiency bound within the class of unit-level moments and demonstrates that feasible sieve estimators can attain it.

4. Empirical Results
The framework is applied to two large-scale anti-poverty programs:

• Muralidharan et al. (2023) (India NREGS): The study examines spillovers from a public works reform using a ring-based exposure measure with a fixed 20 km radius.

  • Result: The design-based overidentification tests do not reject the ring specification. The data-selected radius (approx. 23.7 km for total income) is close to the original 20 km choice.
  • Implication: The original institutional choice is supported by the data; the headline finding that income gains come predominantly from non-program earnings (general equilibrium effects) remains robust even when the radius is estimated.

• Egger et al. (2022) (Kenya GiveDirectly): The study analyzes cash transfer spillovers using a 2 km radius.

  • Result: The design-based tests reject the original 2 km support for all core outcomes. The smallest non-rejected supports are typically 4–6 km.
  • Implication: Replacing the 2 km ring with the design-selected index significantly alters the policy conclusions. The estimated local fiscal transfer multiplier drops from the original 2.5 to 1.57, aligning closely with the structural model prediction of Walker et al. (2024) (1.54). This suggests the original 2 km radius was too local to capture market-level ambient spillovers.

5. Significance and Claims
The paper claims to offer a rigorous alternative to treating exposure mappings as fixed researcher choices. By treating the mapping as an object of design-based inference, the framework can:

1. Validate economically motivated maps when they are consistent with the randomization.
2. Detect and guide revisions when the chosen specification is inconsistent with the data (as seen in the Kenya application).
3. Propagate uncertainty from the exposure specification into downstream policy estimates, preventing the underestimation of standard errors that occurs when $\theta$ is treated as known.

The authors emphasize that their approach does not claim to identify the "true" interference structure against all possible alternatives (noting the impossibility results of Gao et al., 2026), but rather provides a testable restriction within the parametric classes commonly used in applied work. The framework allows researchers to use the known randomization design to discipline the choice of exposure maps, ensuring that policy inferences are robust to the specific functional form and parameters chosen.