📈 economics

Difference-in-Differences in the Presence of Unknown Interference

This technical note investigates how unknown interference violates the SUTVA assumption in Difference-in-Differences designs, demonstrating that the standard estimand identifies a contrast of causal effects rather than specific effects unless additional assumptions are invoked, a point illustrated through a re-examination of Card and Krueger's (1994) minimum wage study.

Original authors: Fabrizia Mealli, Javier Viviens

Published 2026-07-22

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Fabrizia Mealli, Javier Viviens

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery: Did a new policy actually cause a change, or was the change just part of a natural trend? In the world of economics and social science, researchers use a clever tool called "Difference-in-Differences" (DiD) to crack this case. Think of it like a time-traveling comparison. You pick two groups of people: a "Treatment Group" that gets a new rule (like a higher minimum wage), and a "Control Group" that keeps things the same. You measure both groups before the rule changes, then measure them again after. The magic happens when you compare how much the Treatment Group changed versus how much the Control Group changed. If the Treatment Group jumped up while the Control Group stayed flat, you might conclude the rule caused the jump.

But there's a catch. This detective work relies on a hidden rule called SUTVA (Stable Unit Treatment Value Assumption). In plain English, this rule says that one person's experience shouldn't be affected by what happens to someone else. It assumes that if you raise the minimum wage in one town, it doesn't secretly change the job market in the neighboring town. If this rule holds, the math works perfectly. But what if it doesn't? What if the "Treatment" group and the "Control" group are actually whispering to each other, trading secrets, or influencing each other's outcomes? That is the puzzle this paper tackles: what happens to our detective work when the groups aren't isolated islands, but are actually connected by invisible threads of influence?

The authors, Fabrizia Mealli and Javier Viviens, dive into this problem with a mix of rigorous math and practical examples. They show that when these invisible threads (which they call "interference" or "spillovers") exist, the standard DiD calculation stops telling us the truth about a single group. Instead of giving us a clear answer like "The policy increased jobs by 5%," the math only gives us a difference between two unknowns. It's like trying to figure out how much a thief stole from a bank by only knowing that the bank's vault is $1,000 lighter than the neighbor's safe, without knowing how much money was in either one to begin with. The result tells you the gap between the two groups, but it cannot tell you if the bank lost money, if the neighbor gained money, or if both lost money but the bank lost more.

The paper proves that under these messy conditions, the standard DiD number is just a "contrast of causal effects." It reveals that the Treatment group was affected differently than the Control group, but it remains silent on the direction or size of the actual effect for either group. A positive result could mean the policy worked great for the treated group, or it could mean the policy hurt the control group even worse, or both. The authors are very clear: without extra assumptions, we cannot separate the signal from the noise.

However, the authors don't just leave us with a dead end. They offer a set of "what-if" scenarios that could help us get back on track. They suggest that if researchers are willing to make specific, reasonable guesses about how the interference works, they can start to untangle the knot. For instance, if we assume that the "spillover" effect on the Control group is zero (they are truly unaffected), or if we assume the effect on the Control group is smaller than the effect on the Treatment group, we can finally start to say something about the actual policy impact. They even show how to set up "sensitivity analyses," which are like stress tests: "If the spillover effect was this big, would our conclusion still hold?"

To make their point concrete, the authors revisit a famous study from 1994 by Card and Krueger, which claimed that raising the minimum wage in New Jersey actually increased employment in fast-food restaurants. The original study compared New Jersey to neighboring Pennsylvania. The authors of this new paper argue that if the wage hike in New Jersey caused workers to cross the border or changed the economy in Pennsylvania, then the "no interference" rule was broken. In that case, the original conclusion that "jobs went up" isn't necessarily proven. The math only shows that New Jersey did better than Pennsylvania. It's possible that jobs went down in both places, but they went down less in New Jersey. The original conclusion only survives if we assume that the spillover effect on Pennsylvania was either non-existent or smaller than the effect on New Jersey.

Ultimately, this paper is a wake-up call for anyone using these statistical tools. It doesn't say DiD is useless; it just says we need to be much more careful about how we interpret the results when the world is interconnected. The authors demonstrate that while the standard number is still useful for comparing groups, it stops being a direct measure of a policy's success unless we add extra layers of logic to explain how the groups might be influencing each other. By making these hidden assumptions visible, they hope researchers can stop guessing and start building stronger, more honest arguments about what their data really means.

Technical Summary: Difference-in-Differences in the Presence of Unknown Interference

Problem Statement
The Stable Unit Treatment Value Assumption (SUTVA) is a foundational pillar of the Difference-in-Differences (DiD) research design, specifically the component that rules out interference (spillover effects) between units. While recent influential DiD literature (e.g., de Chaisemartin and D'Haultfœuille, 2020; Callaway and Sant'Anna, 2021) has rigorously addressed issues like heterogeneous treatment effects and parallel trends violations, it has largely treated the no-interference assumption as implicit rather than explicit. Consequently, the precise identification properties of the DiD estimand in the presence of unknown interference—where the structure of spillovers is not specified and may occur both within and between groups—remains under-explored. The authors argue that without explicit assumptions regarding interference, the standard interpretation of DiD estimates is often invalid.

Methodology and Framework
The paper adopts a superpopulation approach within a canonical two-group, two-period DiD setting. It formalizes the potential outcomes framework by allowing the outcome of unit $i$ at time $t$ to depend on the treatment assignment of all units ( $W$ ), denoted as $Y_{it}(W)$ . The treatment assignment vector is decomposed into the unit's own treatment ( $W_{it}$ ), the treatment of units in the same group ( $W_{Git}$ ), and the treatment of units in the opposite group ( $W_{1-Git}$ ).

The analysis proceeds by relaxing the no-interference assumption in stages:

No Interference: The standard case where $Y_{it}$ depends only on $W_{it}$ .
Partial Interference: Interference exists within groups but not between groups.
Unknown Interference: No restrictions are placed on the interference structure; units may interfere with any other unit.

The authors define specific causal estimands for each scenario and derive identification results based on modified parallel trends assumptions. They do not impose exposure mappings or restrict analysis to finite populations, maintaining agnosticism regarding the specific form of interference.

Key Results and Identification

Under Unknown Interference (Proposition 3):
The primary finding is that under unknown interference, the standard DiD estimand does not identify a single causal effect (such as the Average Treatment Effect on the Treated, ATT). Instead, it identifies a contrast of two causal effects:
$\text{DiD} = \tau_1 - \tau_0$
Where:
- $\tau_1$ (TATT) is the Total Average Treatment Effect on the Treated (the effect of the treatment on treated units, including spillovers from all other treated units).
- $\tau_0$ (ASC) is the Average Spillover Effect on the Control (the effect of the treatment on control units, derived from general equilibrium effects or spillovers from treated units).
The authors demonstrate that while the DiD estimand allows researchers to test whether the impact differs between groups, it provides no information on the magnitude or direction of $\tau_1$ or $\tau_0$ individually. A zero DiD estimate could result from both effects being zero, or both being equal and non-zero. Similarly, a positive DiD estimate only implies $\tau_1 > \tau_0$ , leaving open scenarios where both effects are negative.
Partial Identification Strategies (Section 3):
To recover specific causal effects, the paper proposes alternative sets of assumptions:
- Assumptions on Unobserved Trends (Propositions 4 & 5): If researchers assume the potential outcome in the absence of treatment ( $Y(0,0,0)$ ) follows a specific trend (e.g., constant or bounded within a range $k$ ) within each group, they can partially identify $\tau_1$ and $\tau_0$ separately without relying on cross-group comparisons.
- Assumptions on the Sign of Spillovers (Proposition 6): If the sign of the spillover effect on the control group ( $\tau_0$ ) is known (e.g., $\tau_0 \geq 0$ ), the DiD estimand provides a bound for the total effect on the treated ( $\tau_1 \geq \text{DiD}$ ).
- Assumptions on Relative Magnitude (Proposition 7): If the magnitude of the total effect on the treated is assumed to be larger than the spillover effect on the control ( $|\tau_1| \geq |\tau_0|$ ), the sign of the DiD estimand identifies the sign of the total effect on the treated ( $\text{sgn}(\tau_1) = \text{sgn}(\text{DiD})$ ).

Application: Card and Krueger (1994)
The paper revisits the seminal study on the New Jersey minimum wage increase. The authors argue that the no-interference assumption is likely violated due to cross-border commuting and general equilibrium effects affecting Pennsylvania (the control group).

Reinterpretation: Under unknown interference, the original DiD estimate of 2.75 FTE workers represents the difference between the total effect in New Jersey and the spillover effect in Pennsylvania, not the causal effect of the wage hike on New Jersey employment alone.
Alternative Interpretations: The original conclusion (that employment increased) holds only if additional assumptions are made, such as:
1. The spillover effect in Pennsylvania is positive (Assumption 10a), making the DiD estimate a lower bound.
2. The magnitude of the effect in New Jersey exceeds the spillover in Pennsylvania (Assumption 11), allowing the sign of the DiD estimate to identify the sign of the New Jersey effect.

Significance and Contribution
The paper's primary contribution is to make the identification problem in the presence of interference explicit and transparent. It clarifies that the DiD estimand is not inherently a causal effect but a contrast of effects when interference is present. By formalizing the arguments that applied researchers often invoke implicitly (e.g., regarding the direction or magnitude of spillovers), the paper provides a structured framework for discussing the validity of DiD conclusions. It does not propose a new estimation algorithm but rather a rigorous set of conditions under which existing DiD estimates can be given a causal interpretation, bridging the gap between standard DiD practice and the complex reality of interference in observational settings.

Technical Summary: Difference-in-Differences in the Presence of Unknown Interference

More like this