📊 statistics

A formal approach to variable selection in difference-in-differences

This paper proposes a formal, graph-based framework for selecting covariates to satisfy conditional parallel trends in difference-in-differences analysis, arguing that identification challenges often stem from a misalignment between the adjustment sets required for validity and those used by popular estimators rather than from the estimators themselves.

Original authors: Daniela Rodrigues, Laura A. Hatfield

Published 2026-05-05

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Daniela Rodrigues, Laura A. Hatfield

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to figure out if a new, expensive vitamin actually helps people grow taller. You have two groups: one group takes the vitamin (the "treated"), and the other doesn't (the "control").

The classic way to test this is the Difference-in-Differences (DiD) method. It's like looking at a race. You check how tall everyone was at the start of the race (before the vitamin). Then, you check how tall they are at the finish line (after the vitamin). The logic is: If the non-vitamin group kept growing at the same speed they were going before, we can assume the vitamin group would have done the same if they hadn't taken the vitamin. The difference between what actually happened and what "would have happened" is the effect of the vitamin.

However, this only works if the two groups were running on the same track to begin with. If the vitamin group was already naturally taller or had better shoes, the race isn't fair. This is the "Parallel Trends" assumption.

This paper is like a rulebook for referees on how to make sure the race is actually fair. Here is the breakdown of their new rules, explained simply:

1. The "Kitchen Sink" Problem

In the past, researchers often just threw every single piece of data they had into the mix (height, weight, shoe size, favorite color) without thinking about why. They called this the "kitchen sink" approach.

The Paper's Fix: You can't just throw everything in. You need a map (a "causal diagram") to see which variables actually matter. Some variables might look important but actually mess up the race.

2. The "Perfect Balance" Myth

The old way of thinking assumed that if you just looked at the groups as a whole, their growth trends would naturally balance out perfectly.

The Paper's Fix: This is like hoping two different cars will drive at the exact same speed just because they are both cars. It's a very strong, unrealistic assumption. The paper shows that often, the groups aren't balanced unless you account for specific differences (like engine size or tire pressure).
The Trap: Sometimes, if you do try to balance them by adding too many variables, you might accidentally break the perfect balance that was already there, making the result worse instead of better.

3. The "Boring" Variable

Usually, researchers ignore variables that don't change over time (like a person's gender or their birth city) because they think, "Well, that doesn't change, so it can't be the reason the vitamin worked."

The Paper's Fix: Sometimes, these "boring" variables are actually the secret sauce! Even if a variable doesn't change, it might be the reason the two groups were different to begin with. Adjusting for it can save the study. It's like realizing that even though the cars didn't change their color, the color actually determined which track they were on.

4. The "After-the-Fact" Variable

Standard advice says: "Never look at data collected after the treatment starts." For example, don't look at how much the vitamin group ate after taking the vitamin, because maybe the vitamin made them hungry.

The Paper's Fix: It depends on why the eating habits changed.
- If the vitamin made them hungry, don't count the eating (it's part of the effect).
- But if something else caused them to eat more (like a new restaurant opening nearby), you must count that, or you'll get the wrong answer.
- The Metaphor: It's like a detective. If a suspect's alibi changes because of the crime, you ignore it. But if the alibi changes because of a traffic jam (unrelated to the crime), you have to account for the traffic jam to solve the case.

5. The "Staggered" Start

Sometimes, different groups get the treatment at different times (like states rolling out a new law in 2020, 2021, and 2022).

The Paper's Fix: The paper distinguishes between when the treatment starts and what the treatment is. If the treatment itself changes over time based on how things are going (dynamic), it creates a feedback loop that is hard to untangle. If the treatment is just a one-time thing that happens at different times for different people (static), it's much easier to handle.

6. The "Wrong Tool" vs. "Wrong Settings"

There has been a lot of arguing in the scientific community about which "calculator" (statistical estimator) is best for this job.

The Paper's Big Discovery: It doesn't matter which calculator you use! The problem isn't the calculator; it's the settings you put into it.
The Metaphor: Imagine you are baking a cake. You can use a fancy electric mixer or a simple wooden spoon. If you use the wrong ingredients (the wrong variables), the cake will taste bad no matter which tool you use.
The Solution: The authors show you exactly how to set the "ingredients" (the adjustment set) for any calculator you want to use. If you feed the right variables into the machine, even the simplest machine will give you the right answer.

The Bottom Line

This paper tells researchers: Stop guessing which variables to use.

Draw a map of cause-and-effect.
Use that map to pick the exact right variables to balance your groups.
Don't worry about picking the most complex statistical tool; just make sure you feed the right variables into whatever tool you have.

If you do this, you get a fair race and a true answer. If you don't, you might be measuring the wrong thing entirely.

1. Problem Statement

Difference-in-Differences (DiD) is a widely used causal inference method relying on the parallel trends assumption: that, in the absence of treatment, the outcome evolution of the treated group would have mirrored that of the control group.

The Gap: In practice, researchers often relax this to conditional parallel trends (assuming trends are parallel only within strata of covariates). However, the selection of these covariates is frequently ad hoc (e.g., "kitchen sink" approaches) or based on isolated assessments of time-varying relationships.
The Misconception: The literature often frames the difficulty of incorporating covariates (especially time-varying ones) as a limitation of specific estimators (e.g., Two-Way Fixed Effects). The authors argue this is actually a misalignment between the adjustment set implicitly used by the estimator and the adjustment set required for causal identification.
Key Challenges:
- Unconditional parallel trends is a strong assumption that may conflict with conditional trends.
- The role of time-invariant covariates and post-treatment covariates is often misunderstood (e.g., assuming time-invariant variables are never confounders, or that post-treatment variables should always be excluded).
- Treatment-confounder feedback in multi-period settings is often conflated with staggered rollout strategies.

2. Methodology

The authors propose a graphical framework based on Directed Acyclic Graphs (DAGs) and the concept of Equi-confounding.

A. Theoretical Foundation: Equi-confounding

The paper reframes the parallel trends assumption as equi-confounding: the unobserved confounder affects the pre-treatment and post-treatment outcomes with equal magnitude.

Graphical Representation: In a standard DAG, an unobserved confounder $V_0$ creates backdoor paths to both $Y_0$ and $Y_1$ . The equi-confounding assumption is represented by equal coefficients on the edges from $V_0$ to $Y_0$ and $Y_1$ .
Compact Representation: The authors introduce a simplified diagram focusing on the outcome change $\Delta Y = Y_1 - Y_0$ . Under equi-confounding, the path from $V_0$ to $\Delta Y$ vanishes (coefficients cancel out). Thus, identification requires blocking all backdoor paths from Treatment ( $A_1$ ) to $\Delta Y$ in this compact graph.
Variable Selection: By applying the standard backdoor criterion to this compact graph, researchers can identify a minimally sufficient adjustment set ( $X$ ) that justifies conditional parallel trends.

B. Estimation Alignment

The authors analyze popular DiD estimators (TWFE, IPW, Doubly Robust, etc.) to determine their effective adjustment sets. They demonstrate that many estimators default to adjustment sets that do not match the requirements of the causal model (e.g., TWFE with time-invariant covariates cancels them out; standard IPW often uses only baseline values).

Proposed Solution: Instead of changing the estimator, the authors propose modifying the covariate specification (e.g., creating time-constant copies of time-varying variables, interacting covariates with time) to force the estimator to use the correct adjustment set.

C. Simulation

The authors conduct a simulation study across multiple scenarios (varying numbers of covariates, time-invariant vs. time-varying effects, post-treatment covariate mechanisms) to validate their theoretical findings. They compare the bias of various estimators when using sufficient, insufficient, or unclear adjustment sets.

3. Key Contributions and Insights

A. Unconditional vs. Conditional Parallel Trends

Conflict: Unconditional parallel trends requires perfect offsetting of all confounding. Conditioning on covariates can paradoxically break this symmetry and introduce bias if the unconditional assumption was actually holding.
Implication: Researchers should not blindly condition on covariates after a pre-trends test; the decision must be guided by the causal structure.

B. Time-Invariant Covariates

Insight: A time-invariant covariate with a time-invariant effect on the outcome (traditionally ignored in DiD) can be a sufficient adjustment variable if it blocks a backdoor path created by a time-varying confounder.
Utility: This expands the pool of valid control variables, especially when time-varying confounders are unmeasured.

C. Post-Treatment Covariates

Nuance: There is no universal rule against adjusting for post-treatment covariates. The decision depends on the mechanism driving the covariate's change:
- Auto-regressive: Adjusting for post-treatment values is harmless.
- Endogenous shock: Adjusting for both pre- and post-treatment values may be required.
- Unmeasured confounder: Adjusting for post-treatment values may be the only way to identify the effect.
- Treatment-induced: Adjusting for post-treatment values blocks the causal path (mediator), recovering only the direct effect, not the total effect.

D. Multi-Period Extensions

Distinction: The authors distinguish between Treatment Type (Static vs. Dynamic) and Rollout Strategy (Simultaneous vs. Staggered).
Treatment-Confounder Feedback: This feedback loop (where a prior treatment affects a covariate which then confounds a subsequent treatment) arises only under dynamic treatment regimes, not merely because of staggered implementation.
Strategy: For staggered static treatments, separate causal diagrams should be constructed for each cohort to account for different assignment mechanisms.

E. Estimation Alignment

The Core Issue: Bias often stems from the estimator's default handling of covariates (e.g., TWFE pooling coefficients) rather than the estimator's mathematical form.
The Fix: By modifying the input data (e.g., duplicating time-varying covariates as time-constant columns, interacting with time), any standard estimator can be aligned with the identification requirements.

4. Results

The simulation study confirms the theoretical predictions:

Sufficient Adjustment Sets: When the effective adjustment set matches the sufficient set derived from the DAG, all estimators yield low bias (near zero), regardless of the specific method used.
Insufficient Adjustment Sets: All estimators produce substantial bias when the adjustment set is insufficient.
Unclear Adjustment Sets: Estimators with ambiguous effective adjustment sets (e.g., standard TWFE with time-varying covariates) show variable bias ranging from negligible to severe, highlighting the danger of relying on default specifications.
Specific Findings:
- In scenarios requiring post-treatment covariates (e.g., unmeasured confounders driving covariate evolution), standard implementations of Heckman, IPW, and DR estimators were biased because they excluded post-treatment values by default. Correcting the input data eliminated this bias.
- In scenarios with treatment-confounder feedback, adjusting for the mediator at the wrong time point introduced massive bias.

5. Significance

This paper fundamentally shifts the DiD literature's focus from estimator selection to identification strategy and variable selection.

Formalizes Variable Selection: It provides the first rigorous, graphical criteria for selecting covariates in DiD, moving beyond ad-hoc practices.
Clarifies Misconceptions: It corrects common misconceptions about time-invariant variables and post-treatment covariates, showing they can be essential for identification.
Practical Guidance: It offers a "plug-and-play" solution for practitioners: rather than switching to complex new estimators, researchers can achieve valid inference by simply restructuring their covariate inputs to align with the causal model.
Robustness: The findings suggest that the choice of adjustment set is more critical to validity than the choice of estimator, provided the estimator is correctly specified for that set.

In summary, the authors provide a unified framework that integrates causal diagrams, identification theory, and estimation mechanics, offering a clear path to valid DiD analysis in complex, real-world settings.