📈 economics

When Do Treatment Changes Identify Causal Effects?

This paper clarifies the distinct identifying assumptions for causal inference based on treatment changes versus treatment levels, demonstrating their non-nested nature while establishing conditions for equivalence and a structural double robustness that ensures consistency for estimators like two-way fixed effects if either assumption holds.

Original authors: Martin Huber

Published 2026-06-02

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Martin Huber

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Measuring the Impact of Change vs. The State of Things

Imagine you are trying to figure out if changing a recipe makes a cake taste better.

The Old Way (Treatment Levels): You look at two different cakes. One has 2 cups of sugar, the other has 4 cups. You assume the difference in taste is because of the sugar amount, provided you control for other things like the oven temperature or the brand of flour.
The New Way (Treatment Changes): You look at the same baker over time. Last month, they used 2 cups. This month, they used 4 cups. You ask: Did the change in sugar cause the change in taste?

This paper asks: When is it safe to trust the "New Way" (looking at changes) compared to the "Old Way" (looking at levels)?

The author, Martin Huber, argues that these two methods rely on completely different rules of the road. Sometimes one works and the other fails. Sometimes they both work. And sometimes, a specific method called TWFE (Two-Way Fixed Effects) is a "superhero" that works even if only one of the rules is true.

The Two "Secret Rules" for Trusting Changes

To trust that a change in treatment (like a price hike or a new policy) caused an outcome (like sales or health), the paper says we need to believe in one of two specific "Secret Rules" (Structural Models). These rules are not nested, meaning they are like two different keys that open the same door, but neither key is a smaller version of the other.

Rule A: The "No Time Travel" Rule

The Analogy: Imagine a chef who changes the recipe every day. For the "Change" method to work, the chef must not be influenced by what happened in the kitchen yesterday. If the chef's decision to add more salt today depends on how the soup tasted yesterday, or if the salt added yesterday changes how the soup tastes today, the "Change" method breaks.
The Science: This model assumes that past treatments don't have a "dynamic effect" on future outcomes. It also assumes that the "hidden factors" (like the chef's mood or the quality of the ingredients) are constant over time and cancel out when you look at the difference between today and yesterday.

Rule B: The "Random Walk" Rule

The Analogy: Imagine the chef's decisions are like a drunk person walking down a street. Every step they take is a random stumble. They don't plan their path; they just stumble forward. If the chef's decision to change the recipe is a pure random shock (like a sudden gust of wind knocking a spice jar), then looking at the change is safe, even if the chef has a bad memory or if the soup from yesterday affects today's soup.
The Science: This model assumes the "hidden shocks" driving the treatment changes follow a Random Walk. This means the change is purely random noise that doesn't depend on the past. If this is true, the "Change" method works even if past treatments affect future outcomes.

The Catch: You usually don't know which rule is true in real life. You can't see the "hidden factors" (the chef's mood or the random wind).

The "Double Robust" Superhero: TWFE

The paper introduces a powerful tool called Two-Way Fixed Effects (TWFE). Think of this as a Swiss Army Knife or a Safety Net.

How it works: TWFE looks at both the change in the treatment (e.g., price went up) AND the change in the outcome (e.g., sales went down) at the same time.
The Superpower: The paper proves that TWFE is Double Robust. This means it will give you the correct answer if EITHER Rule A is true OR Rule B is true. You don't need both to be true.
- If the "No Time Travel" rule holds, TWFE works.
- If the "Random Walk" rule holds, TWFE works.
- It only fails if neither rule is true.

This is a big deal because it protects researchers. If they are wrong about why the treatment changed, but right about how the outcome changed (or vice versa), TWFE still saves the day.

The "Lie Detector" Test: Overidentification

What happens if you use both the "Old Way" (Levels) and the "New Way" (Changes) on the same data?

If they agree: Great! It suggests both methods are likely valid, and you can trust the result.
If they disagree: This is the "Lie Detector" moment. Since the rules for the two methods are different, if they give different answers, it means at least one of the rules is broken.

The paper suggests using a statistical test (called a Hausman test) to check this. If the test says "They are different," you know your data doesn't fit the perfect world where both rules apply. You then have to investigate which rule is the liar.

The Real-World Test: Cigarettes

To prove this works, the author tested it on cigarette demand in the US (1963–1992).

The Setup: Did raising cigarette prices (the treatment change) lower smoking (the outcome)?
The Results:
- The "Old Way" (Levels) and the "Superhero" (TWFE) all agreed: Raising prices lowers smoking by about 0.4%.
- The "New Way" (Changes) said: Raising prices lowers smoking by a huge 0.7%.
- The Verdict: The "Lie Detector" test screamed that the "New Way" was wrong. The data suggested that the "Random Walk" rule (Rule B) was likely false. The changes in cigarette prices weren't just random; they were likely driven by political or economic factors that also influenced smoking habits in ways the model couldn't see.

Summary for the Everyday Reader

Don't just look at changes blindly. Assuming that a change in a policy is "random" is a big leap. It requires specific, often unprovable, conditions.
There are two different ways to make that leap work. One requires no "time travel" effects; the other requires the changes to be purely random.
Use the "Safety Net" (TWFE). If you use a method that looks at changes in both the cause and the effect, you are safer. It works if either of the two rules is true.
Compare your methods. If you calculate the result using "Levels" and "Changes" and get different numbers, trust the "Safety Net" or the "Levels" method more, because the "Changes" method likely violated its strict rules.

The paper doesn't tell us how to fix bad data, but it gives researchers a clear map to know when their map is wrong and when they can trust their compass.

Technical Summary: When Do Treatment Changes Identify Causal Effects?

Problem and Motivation
Empirical research frequently relies on observational data to identify causal effects, typically employing either selection-on-observables strategies (conditioning on covariates to treat treatment levels as exogenous) or difference-in-differences (DiD) designs (assuming parallel trends in potential outcomes). However, a third, often implicit practice exists: assessing the effect of changes in a dynamically evolving treatment (e.g., tax rate adjustments, dosage modifications) rather than specific treatment levels. While researchers often recode treatments as binary indicators of increase versus no change, the structural assumptions required for this "treatment-change" identification strategy remain theoretically under-explored. Specifically, it is unclear how these assumptions relate to conventional strategies based on treatment levels and whether they are nested within them. This paper addresses the conditions under which variation in treatment changes identifies causal effects, characterizes the structural models supporting such identification, and investigates the relationship between treatment-change and treatment-level identification strategies.

Methodology and Structural Framework
The paper employs the potential outcomes framework to define the causal effect of interest: the average effect of a treatment increment $\nabla d$ relative to no change, denoted $\Delta_t(\nabla d)$ . The core identifying assumption is a conditional independence assumption for treatment changes (CIA- $\nabla D$ ), requiring that the treatment change $\nabla D_T$ is independent of potential outcomes conditional on the history of observed covariates $\bar{X}_{T-1}$ .

To understand the validity of CIA- $\nabla D$ , the author characterizes two distinct, non-nested structural models (Model A and Model B) involving outcome and treatment equations with time-invariant unobservables (fixed effects $U$ ) and time-varying unobservables ( $V_t, W_t$ ):

Model A: Requires that the fixed effect $U$ is additively separable in the treatment equation (canceling out in the change $\nabla D_T$ ) and rules out dynamic treatment effects (earlier treatments do not affect later outcomes or covariates). It also requires that time-varying unobservables in the treatment equation are independent of those in the outcome equation.
Model B: Also requires additive separability of $U$ in the treatment equation but imposes a stronger restriction on the treatment process: the time-varying shocks driving treatment changes must follow a random walk (serially uncorrelated conditional on covariates). In exchange for this stronger assumption, Model B permits dynamic treatment effects.

The paper formally demonstrates that the assumptions underlying treatment-change identification are generally not nested with those of treatment-level strategies (Selection-on-Observables, CIA- $D$ , and DiD, CIA- $\nabla Y$ ). Specifically, CIA- $\nabla D$ does not imply CIA- $D$ or CIA- $\nabla Y$ , and vice versa. However, under the specific conditions of Model B (random walk), conditioning on treatment changes becomes equivalent to conditioning on treatment levels given lagged treatment, rendering the three strategies jointly valid.

Key Contributions and Theoretical Results

Characterization of Non-Nested Models: The paper establishes that valid identification via treatment changes relies on structural conditions distinct from those required for level-based identification. This non-nesting implies that a strategy valid under one set of assumptions may fail under another where a level-based strategy succeeds.
Equivalence under Random Walks: A key theoretical result is that if the treatment process follows a random walk (Model B), conditioning on treatment changes is equivalent to conditioning on treatment levels given the lagged treatment. This equivalence allows for the construction of Hausman-type overidentification tests to jointly exploit methods based on levels and changes.
Structural Double Robustness (DR): Beyond overidentification tests, the non-nesting results imply a structural double robustness property for estimators that difference both outcomes and treatments, such as the classical Two-Way Fixed Effects (TWFE) estimator. The paper proves that TWFE remains consistent if either the treatment-change assumption (additive separability in the treatment equation) or the parallel-trends assumption (additive separability in the outcome equation) holds, without requiring both simultaneously. This property holds even when the other assumption fails.
Estimand Differences and Rescaling: The paper clarifies that under heterogeneous treatment effects, the estimands of different strategies differ. Specifically, the CIA- $\nabla D$ estimator targets a weighted average of marginal effects scaled by a persistence factor $\lambda$ (the degree to which treatment levels co-move with changes), whereas TWFE and level-based strategies target the unscaled marginal effect directly. Consequently, valid overidentification tests require rescaling the CIA- $\nabla D$ estimates by $1/\lambda$ before comparison.

Empirical and Simulation Results

Simulation Study: The paper conducts simulation studies confirming the non-nesting results. In scenarios where only one assumption holds, the corresponding estimator is consistent while others are biased. Crucially, the simulations validate the double robustness of TWFE: it remains consistent whenever either the treatment-change or the parallel-trends assumption holds, even if the other fails. When the treatment follows a random walk, all strategies yield consistent estimates.
Empirical Application: The methodology is applied to estimate the price elasticity of cigarette demand using state-level panel data (Baltagi and Levin, 1986). The study compares four strategies: CIA- $\nabla D$ $\nabla D$ , CIA- $\nabla Y$ $\nabla Y$ (DiD), CIA- $D$ $D$ (selection-on-observables), and TWFE.
- The results show that level-based strategies (CIA- $\nabla Y$ , CIA- $D$ , TWFE) yield consistent estimates (elasticity $\approx -0.38$ to $-0.40$).
- The treatment-change strategy (CIA- $\nabla D$ ), even after rescaling for the persistence factor, yields a significantly larger elasticity in absolute value ( $\approx -0.68$ ).
- Hausman overidentification tests strongly reject the joint validity of the treatment-change strategy against the level-based strategies.
- Given the double robustness of TWFE, the discrepancy suggests that the treatment-change assumption (CIA- $\nabla D$ ) likely fails in this context, possibly because states with large price increases differ systematically in unobserved ways not captured by covariates.

Significance
The paper provides a systematic theoretical foundation for the use of treatment changes in causal inference, clarifying that this approach relies on distinct structural assumptions from conventional methods. It highlights that while treatment-change identification can offer power by eliminating fixed effects that contaminate level-based identification, it is not universally superior. The identification of non-nested assumptions allows researchers to construct overidentification tests to falsify joint validity. Furthermore, the demonstration of the structural double robustness of TWFE offers a practical safeguard: researchers can rely on TWFE for consistent estimation if they can justify either the treatment-change or the parallel-trends assumption, even if the other fails. The paper concludes that researchers utilizing treatment changes must be explicit about the structural conditions they rely upon (e.g., absence of dynamic effects or random walk shocks) and should consider overidentification tests as a diagnostic tool for the validity of their identifying assumptions.