📈 economics

Quasi-randomization tests for network interference: a random graph approach

This paper proposes a quasi-randomization test for network interference that treats the network as a random variable to construct null distributions via random graph models, thereby overcoming the non-imputability and computational challenges of existing methods while offering exact finite-sample validity and improved power, particularly in cluster-randomized trials.

Original authors: Supriya Tiwari, Pallavi Basu

Published 2026-06-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Supriya Tiwari, Pallavi Basu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to figure out if a new fertilizer makes plants grow taller. In a perfect world, you'd give the fertilizer to some plants and nothing to others, and the only thing that would change is the fertilizer. This is the standard way scientists test things.

But in the real world, plants are neighbors. If you fertilize Plant A, it might grow so big that it shades Plant B, or its roots might steal water from Plant C. Plant B and C didn't get the fertilizer, but they are still affected. In the world of data science, this is called network interference or spillover effects.

The paper by Tiwari and Basu tackles a very tricky problem: How do you prove that these "neighbor effects" are actually happening when the map of who is friends with whom (the network) is also a bit of a mystery?

Here is a simple breakdown of their solution using everyday analogies.

The Old Way: The "Frozen Map" Problem

Traditionally, when scientists test for these neighbor effects, they treat the social network (who knows whom) as a fixed, frozen map. They say, "Okay, this is the exact map of friendships we have. Now, let's pretend we can swap who got the treatment."

The problem is that in many real experiments (like giving a treatment to entire villages or schools), you can't just swap people around easily. If you treat a whole village, you can't pretend that village was a control group without breaking the rules of the experiment. This makes the math get stuck, like a car trying to drive on a road that suddenly disappears. The old methods often fail to find the answer even when the effect is there because they are too rigid.

The New Way: The "Imaginary Party" Analogy

The authors propose a clever shift in perspective. Instead of treating the network map as a fixed, unchangeable object, they treat it as a random variable.

Think of it this way:

The Old View: You are looking at a specific party where the seating chart is set in stone. You want to know if moving a guest changes the conversation, but you can't move the chairs.
The New View: You imagine that the party seating chart itself was generated by a random process (like people arriving and finding seats based on how many friends they have). You ask: "If we had generated a different seating chart using the same rules, but kept the guests in the same seats, would the conversation still look the same?"

By treating the network as something that could have been different (but followed the same rules), the authors can create thousands of "imaginary parties" (or random graphs) that look just like the real one.

How the Test Works: The "Swap" Game

Here is the step-by-step logic of their "Quasi-Randomization Test":

The Setup: You have a real experiment with a real network of friends and a real treatment (like an insurance workshop).
The Assumption: You assume the network formed naturally based on certain rules (like "people tend to have about 5 friends").
The Magic Trick: Instead of trying to shuffle the people (which might be impossible in a cluster experiment), they shuffle the connections. They generate thousands of fake networks that have the exact same number of friends for every person as the real network, but the specific friends are different.
The Comparison: They check the results on all these fake networks.
- If the "neighbor effect" is just a fluke, the results on the fake networks will look very different from the real one.
- If the "neighbor effect" is real, the results will look consistent across the fake networks.

Why This is a Big Deal

The authors show that this method is smarter and stronger than the old methods, especially in tricky situations like Cluster-Randomized Trials (where you treat whole groups, like schools or villages, rather than individuals).

The Old Method: In a cluster trial, the old method often gets "degenerate." It's like trying to flip a coin that is glued to the table; you can't get a random result, so you can't test anything.
The New Method: By shuffling the connections instead of the people, they keep the test moving. They found that this new method is much better at spotting real spillover effects that the old methods miss.

The Real-World Test: Weather Insurance in China

To prove their idea works, the authors re-analyzed a real experiment in rural China about farmers adopting weather insurance.

The Question: Did farmers who attended a detailed workshop tell their neighbors about it, causing the neighbors to buy insurance too?
The Result: Using their new "Imaginary Party" method, they found strong evidence (a p-value of 0.0126) that yes, the information was spreading.
The Lesson: If you ignore this spreading, you might think the workshop didn't work well because you only counted the people who attended. But actually, the workshop was a huge success because it rippled through the community.

Summary

The paper argues that when studying how things spread through networks, we shouldn't treat the network as a rigid, unchangeable cage. Instead, we should treat the network as a flexible, random structure that follows certain rules. By imagining "what if the network looked slightly different but followed the same rules," we can finally run fair and powerful tests to see if spillover effects are real.

Key Takeaway: They didn't invent a new way to give treatments; they invented a new way to look at the map of who is connected to whom, allowing us to see effects that were previously invisible.

Technical Summary: Quasi-randomization Tests for Network Interference

Problem Statement
Network interference occurs when the treatment status of one unit affects the potential outcomes of others, creating spillover effects that violate the Stable Unit Treatment Value Assumption (SUTVA). While the Fisher Randomization Test (FRT) is the standard for testing sharp null hypotheses (no treatment effect), testing for the absence of spillover effects presents a "non-sharp" null problem. Under the null of no spillover, counterfactual outcomes for varying treatment assignments cannot be imputed because the network structure itself influences the exposure.

Existing literature addresses this by restricting attention to a subset of "focal units" and conditioning on their treatment assignment (Aronow, 2012; Athey et al., 2018; Basse et al., 2019; Puelz et al., 2022). However, these conditional randomization tests face two critical limitations:

Low Power: They often suffer from reduced statistical power, particularly in cluster-randomized trials where the treatment assignment space is restricted.
Degeneracy: In cluster-randomized designs, conditioning on focal units can lead to a degenerate null distribution (where the support of the distribution collapses), rendering valid inference impossible.

Methodology
The authors propose a quasi-randomization test that treats the network $G$ as a random variable generated by a specific process, rather than a fixed quantity. This approach shifts the inference target from statements conditional on the realized network to statements averaged over hypothetical networks generated by the same process.

The core methodology involves:

Random Graph Framework: The network is modeled as an Exponential Random Graph Model (ERGM) or a random graph with arbitrary degree distributions. The null hypothesis of no spillover is defined such that outcomes are invariant to the network structure, conditional on direct treatment assignments and graph covariates.
Imputability via Network Randomization: By fixing the observed treatment assignment $Z_{obs}$ and randomizing over the network-generating process $P(G|Z, G_X)$ , the test statistic becomes imputable under the null. This overcomes the non-imputability issue inherent in fixed-network approaches.
Permutation Tests: To avoid the computational intractability of estimating the full network distribution $P(G)$ $P (G)$ , the authors develop conditional permutation tests based on sufficient statistics:
- Procedure 3 (Degree Sequence): For random graphs with arbitrary degree distributions, the test conditions on the observed degree sequence. It samples graphs from the equivalence class of graphs sharing the same degree sequence, assuming equiprobability.
- Procedure 4 (Block Isomorphism): For cluster-randomized experiments and general ERGMs, the test conditions on the degree sequence and the isomorphism classes of subgraphs induced by treatment and control groups. This ensures that the permutation respects the experimental design constraints (e.g., cluster structure) while generating a non-degenerate null distribution.

Key Contributions
The paper makes three primary contributions:

Theoretical Embedding: It embeds the random graph interference model within the potential outcomes framework, defining sharp and non-sharp null hypotheses for random graph models and establishing finite-sample validity for the proposed tests.
Addressing Technical Limitations: The method resolves the issue of non-imputability and degeneracy in cluster-randomized trials. By randomizing over the network distribution rather than the treatment assignment, the method avoids the restricted support that causes existing focal-unit methods to fail in clustered settings.
Computational Efficiency: The proposed permutation tests utilize fast polynomial-time algorithms (e.g., Viger and Latapy, 2005; Fosdick et al., 2018) to sample graphs with fixed degree sequences, making the approach scalable for large networks.

Results
The authors validate the method through extensive Monte Carlo simulations and an empirical application:

Simulation Studies:
- Type I Error: The proposed tests maintain correct Type I error rates (controlled at $\alpha = 0.05$ ) under both completely randomized and cluster-randomized designs across small-world and stochastic block network models.
- Power: The quasi-randomization tests demonstrate substantially improved power compared to existing conditional focal tests (Athey et al., 2018). This gain is particularly pronounced in cluster-randomized trials and settings with high clustering (e.g., stochastic block models), where existing methods often suffer from power loss or degeneracy.
- Robustness: The method retains power even when the potential outcome function includes graph-dependent covariates (e.g., degree effects).
Empirical Application:
- The authors reanalyze a weather insurance adoption experiment in rural China (Cai et al., 2015).
- Using the block isomorphism permutation test (Procedure 4) on a subset of 433 households, they found significant evidence of information spillover ( $p = 0.0126$ ).
- The result suggests that households receiving intensive information sessions transmit knowledge to neighbors, a finding that would be obscured or biased if interference were ignored.

Significance and Claims
The paper argues that treating the network as a random variable offers a computationally tractable and statistically powerful alternative to fixed-network inference. The authors claim that their approach is "exactly valid in finite samples" under mild assumptions on the network-generating process.

The significance of the work lies in its ability to test for interference across all units in the population, rather than a restricted subset, and its ability to function effectively in cluster-randomized trials, a setting where current state-of-the-art methods often fail due to degeneracy. The authors acknowledge that this power gain comes at the cost of an assumption regarding the network-generating process (e.g., the ERGM or degree distribution), presenting a trade-off between potential model misspecification bias and improved inferential power. They position this trade-off as reasonable for many practical settings where the observed network is a realization of an ongoing social process.