📊 statistics

Experimental Design under Network Interference

This paper proposes a statistical framework for optimizing two-wave experimental designs under network interference by leveraging pilot study data to minimize estimator variance, while formally characterizing the trade-offs involved in pilot size and providing theoretical guarantees for inference and regret.

Original authors: Davide Viviano

Published 2026-06-23

📖 5 min read🧠 Deep dive

CC0 1.0

Original authors: Davide Viviano

Original paper dedicated to the public domain under CC0 1.0 (http://creativecommons.org/publicdomain/zero/1.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a city planner trying to figure out if a new park improves the happiness of the people living nearby. You want to build the park in the best possible spot to get the clearest answer. But there's a catch: people in this city are very social. If you build a park for one family, their neighbors might also feel happier just because they can see the greenery, even if they don't live right next to it. This is called "spillover" or "interference."

In traditional experiments, you might just pick a few random houses and give them the park, then compare them to houses without one. But in a connected city, this is messy. If you pick a house, its neighbors are affected, and if you pick a neighbor, they affect the first house. It's like trying to measure the splash of a single pebble in a pond that's already full of ripples from other pebbles.

This paper, by Davide Viviano, proposes a clever two-step strategy to solve this mess and get the most precise answer possible. Think of it as a "test run" followed by the "main event."

The Two-Step Strategy

Step 1: The "Scout Team" (The Pilot Study)

Before you build the park for everyone, you send out a small scout team to a tiny, isolated part of the city.

The Goal: You want to learn how much happiness varies from house to house and how much neighbors influence each other.
The Trick: You must pick a scout team that is isolated. They shouldn't have neighbors who are not on the scout team. If a scout's neighbor is left out, that neighbor's mood might change based on the scout's park, and that "contamination" ruins your data.
The Math: The paper uses a special computer algorithm (like a smart maze solver) to find the perfect group of houses that are clustered together but cut off from the rest of the city. This ensures the data they collect is "clean."

Step 2: The "Main Event" (The Real Experiment)

Once the scouts return with their data, you know exactly how much happiness varies and how strong the neighbor effects are. Now, you can design the main experiment perfectly.

The Goal: You want to pick the best houses for the main park and decide exactly who gets the park and who doesn't, specifically to minimize the "noise" (statistical variance) in your final result.
The Optimization: Instead of picking houses randomly, you use the scout's data to run a complex calculation. It's like a chess player who looks ahead to see which moves will lead to the clearest win. You might pick houses that are very different from each other or group them in a specific way to cancel out the noise.
The Rule: The houses you pick for the main event must be far away from the scout team (and their neighbors) so that the scout's "contamination" doesn't mess up the main results.

The Big Trade-Off

The paper highlights a delicate balancing act, like walking a tightrope:

Too small of a scout team: You don't have enough data to know how the neighbors influence each other. Your main experiment might be poorly designed because you're guessing.
Too large of a scout team: You learn a lot, but you have to exclude a huge chunk of the city from the main experiment (because you can't use anyone near the scouts). This leaves you with fewer people to study in the main event, which also makes your results less precise.

The paper provides a mathematical "sweet spot" (a rule of thumb) for how big the scout team should be relative to the main team to get the best possible result.

Why This Matters

The paper shows that this two-step method is much better than the old ways of doing experiments (like just picking random clusters or saturating groups).

Real-World Proof: The author tested this on real data from villages in Africa and simulated city networks. The results showed that this method could reduce the "error" in the results by up to 40% compared to other methods.
Handling Chaos: It works especially well when the data is messy (some people are naturally happier than others, and neighbors affect each other differently).

The Bottom Line

If you want to know the true effect of a treatment (like a new policy, a drug, or a community program) in a world where people influence each other, don't just guess and check.

Send a small, isolated scout team to map the terrain.
Use that map to strategically place your main experiment, avoiding the scout's neighborhood.
This gives you the clearest, most precise answer possible, saving you time and money by needing fewer people to get a reliable result.

The paper is essentially a guidebook for running the smartest possible experiment in a connected world, ensuring that the "ripples" from one person don't drown out the signal you're trying to hear.

Technical Summary: Experimental Design under Network Interference

Problem Statement
This paper addresses the challenge of designing two-wave experiments to estimate treatment effects in the presence of network interference (spillovers). Unlike standard clustered or saturation designs that assume independent clusters, this setting involves a single, connected network where interference is local (units interact primarily with neighbors). The core difficulty lies in the fact that optimal experimental design requires knowledge of outcome variances and covariances, which are typically unknown and depend on the network structure and treatment assignments. Furthermore, standard randomization may yield imprecise estimates when interference and heteroskedasticity are present. The paper seeks to develop a protocol that utilizes a pilot study to estimate these variance components and subsequently optimizes the main experiment's participant selection and treatment assignment to minimize the variance of a pre-specified linear estimator.

Methodology
The proposed framework, termed "Experiment under Local Interference" (ELI), operates in two waves:

Pilot Study (Wave 1):
- Selection: A subset of units is selected as a pilot. Crucially, the selection algorithm (Algorithm 1) minimizes the number of edges between pilot units and non-pilot units (a min-cut optimization). This ensures that the pilot units and their immediate neighbors are excluded from the main experiment.
- Rationale: Excluding pilot units and their neighbors prevents the main experiment's design from becoming statistically dependent on the unobservables of the pilot, thereby preserving the unconfoundedness of the main experiment's estimators.
- Estimation: Treatments are randomized within the pilot. The pilot data is used to estimate conditional outcome variances ( $\sigma^2$ ) and neighbor covariances ( $\eta$ ) as functions of treatment status, exposure mappings (e.g., number of treated neighbors), and covariates.
Main Experiment (Wave 2):
- Design Optimization: Using the variance and covariance estimates from the pilot, the researcher solves an optimization problem (Algorithm 2) to select participants ( $R$ ) and assign treatments ( $D$ ) for the main experiment.
- Objective: The design minimizes the plug-in estimate of the conditional variance of a user-specified linear estimator (e.g., difference-in-means, regression-adjusted estimators) for target estimands such as direct effects, spillover effects, or overall effects.
- Constraints: The optimization is subject to budget constraints (maximum sample size) and the exclusion constraint derived from the pilot (no pilot units or their neighbors can participate).
Extensions:
- Randomization: The framework can be modified (Appendix A.1) to include a re-randomization step. This allows for design-based inference on sharp null hypotheses while maintaining near-optimal precision for model-based inference, trading a small amount of variance for increased randomization.
- Partial Network Information: The method is extended to settings where the full network is not observed (Appendix A.4), utilizing imputation for missing edges under the assumption that the pilot is drawn from a disconnected component.
- Higher-Order Dependence: The framework accommodates dependence extending beyond immediate neighbors (up to degree $M$ ) by excluding neighbors up to degree $M$ from the main experiment.

Key Theoretical Results

Unbiasedness: The paper establishes sufficient conditions (Proposition 3.1) under which the main experiment yields unbiased estimators of model-based estimands. This relies on the exclusion of pilot units and their neighbors from the main experiment.
Regret Guarantees: Theorem 4.1 provides a regret bound characterizing the difference between the variance of the proposed two-wave design and that of an "oracle" design (which knows the true variance/covariance functions and faces no pilot-induced constraints). The regret converges to zero at a rate governed by the pilot size ( $\bar{m}$ ) and the ratio of pilot to main experiment size.
Pilot Size Conditions: The paper derives sufficient conditions for the pilot size to achieve near-optimal precision. Specifically, for sparse networks, a pilot size scaling as $\bar{m} \gtrsim \bar{n}^{2/3}$ is shown to be sufficient to make the regret negligible (Corollary 4.2).
Asymptotic Inference: Theorem 4.3 establishes asymptotic normality for the estimator, conditional on the realized assignment mechanism, allowing for valid confidence intervals using a consistent variance estimator derived from the main experiment data.

Empirical Results
Numerical studies using real-world village friendship networks (Cai et al., 2015) and simulated Erdős–Rényi and Barabási–Albert networks demonstrate the efficacy of the ELI method:

Performance: The proposed method uniformly attains lower sampling variance compared to state-of-the-art competitors, including graph clustering (3- $\epsilon$ net), saturation designs, and simple random assignment.
Heteroskedasticity: The advantages of the ELI design are most pronounced under heteroskedasticity and nonzero covariances, where standard randomization or clustering fails to account for the specific variance structure.
Partial Observation: Even with partially observed networks, the ELI method outperforms random allocation.
Sample Size Efficiency: The method can achieve the same variance as the best competitor with up to a 40% reduction in the total number of units required.

Significance and Claims
The paper claims to provide the first statistical framework for variance-optimal two-wave experimental designs under network interference. Its primary contribution is demonstrating that incorporating a pilot study to estimate variance and covariance structures can substantially improve the precision of treatment effect estimates compared to designs that do not utilize such information. The work bridges the gap between experimental design and causal inference in network settings, offering a practical protocol (min-cut pilot selection followed by variance-minimizing optimization) that is theoretically grounded and empirically validated. The authors note that while the method assumes local interference and sparse networks, it offers a robust approach for settings where interference is a natural feature of the data, such as in field experiments involving social programs or online platforms.