Covariate-Balanced Weighted Stacked Difference-in-Differences

This paper introduces Covariate-Balanced Weighted Stacked Difference-in-Differences (CBWSDID), a design-based estimator that enhances weighted stacked DID by combining within-subexperiment covariate balancing with across-subexperiment aggregation to address conditional parallel trends in both absorbing and repeated treatment settings.

The Gist

Imagine you are a detective trying to figure out if a new "super-fertilizer" actually makes plants grow taller. You have a garden with 500 different plots of land. Some plots got the fertilizer in 2004, some in 2005, some in 2006, and some never got it at all.

Your goal is to measure the true effect of the fertilizer. To do this, you need to compare the plots that got the fertilizer (the "treated" group) with plots that didn't (the "control" group).

The Problem: The "Apples vs. Oranges" Trap

In the past, researchers used a method called Stacked Difference-in-Differences. Think of this as taking all the fertilizer plots and all the non-fertilizer plots, dumping them into one giant bucket, and comparing their average growth.

But here's the catch: The plots aren't identical.

• The plots that got fertilizer in 2004 might have been on a hill with great soil.
• The plots that got it in 2006 might have been in a swamp.
• The plots that never got fertilizer might be in a desert.

If you just mix them all together, you aren't comparing fertilizer to no-fertilizer; you're comparing "Hill Soil" to "Swamp Soil." The results will be wrong because the starting conditions were different. This is the "Apples vs. Oranges" problem.

The Old Fix: "Corrective Weights"

A few years ago, smart statisticians (Wing et al.) realized that even if you fix the "mixing" problem, you still have a second problem: Aggregation.

Imagine you have 100 plots that got fertilizer in 2004, but only 2 plots that got it in 2006. If you just average everything, the 2004 group dominates the result. But maybe the 2006 group is the one you really care about.

The old fix was to use Corrective Weights. It's like a scale: you put a heavy weight on the small groups and a light weight on the big groups so that every "batch" of fertilizer gets equal say in the final answer. This fixed the mixing of the groups, but it didn't fix the fact that the individual plots inside the groups were still different (Apples vs. Oranges).

The New Solution: CBWSDID (The "Double-Check" Detective)

This paper introduces a new method called Covariate-Balanced Weighted Stacked Difference-in-Differences (CBWSDID).

Think of this method as a two-step detective process that solves both problems at once.

Step 1: The "Matchmaker" (Inside the Group)

Before you even look at the whole garden, the detective goes into each specific year's group (e.g., the 2004 group).

• They look at the "Treated" plot (the one with fertilizer).
• They look at the "Control" plots (the ones without).
• They say: "Wait, this 2004 treated plot is on a hill. I need to find a control plot that is also on a hill, has the same amount of rain, and the same soil type."

This is Matching or Weighting. It's like a dating app for data points. The algorithm finds the perfect "twin" for every treated plot from the control group. If a control plot is a perfect match, it gets a high score (weight). If it's a terrible match (like a desert plot for a hill plot), it gets a zero score and is ignored.

Result: Now, inside every group, the treated and control plots are identical twins. The "Apples vs. Oranges" problem is gone.

Step 2: The "Fair Judge" (Across the Groups)

Now that we have perfect twins inside each group, we need to combine the results from 2004, 2005, 2006, etc.

• The detective uses the Corrective Weights from the old method.
• They make sure that the 2004 group, the 2005 group, and the 2006 group all get a fair voice in the final verdict, regardless of how many plots were in each group.

Result: You get a final answer that is both internally fair (comparing identical twins) and externally fair (giving every year a fair voice).

Why is this a Big Deal?

1. It handles "Switching": In the real world, things don't just happen once. A country might become a democracy, then become a dictatorship, then become a democracy again. Old methods struggled with this "on-off-on" switching. This new method treats every "switch" as a new episode and applies the Double-Check logic to each one.
2. It stops "Fake Trends": In the paper's examples, old methods saw a huge drop in "city whiteness" after a law passed. But when they used this new method, they realized the drop was fake! It was just because the cities that passed the law were already different from the ones that didn't. The new method flattened the fake trend and showed the real effect was much smaller.
3. It's a Bridge: It connects two worlds. One world uses "Matching" (finding twins), and the other uses "Weighted Averages" (mathematical balancing). This method says, "Why choose? Let's do both."

The Takeaway

Imagine you are trying to judge a cooking contest.

• Old Method: You taste a soup from a rich chef and a soup from a poor chef and say, "The rich chef's soup is better." (But maybe the rich chef just had better ingredients to start with).
• Weighted Method: You make sure to count the rich chef's vote and the poor chef's vote equally. (Better, but you still tasted different ingredients).
• CBWSDID: You find a poor chef who has the exact same ingredients as the rich chef. You taste both. Then, you make sure every chef in the contest gets an equal vote in the final score.

This paper gives researchers a powerful new tool to ensure that when they say "X caused Y," they aren't just seeing an illusion created by bad comparisons. It's about making sure the comparison is fair, the math is balanced, and the answer is real.

Technical Summary

1. Problem Statement

The paper addresses two distinct but related identification problems in Staggered Adoption Difference-in-Differences (DID) settings:

1. Within-Subexperiment Comparability: In standard stacked DID, treated units and "clean" control units (those not yet treated) within a specific sub-experiment often differ significantly in observed covariates and lagged outcomes. This violates the assumption of conditional parallel trends, leading to biased estimates even if the aggregation logic is correct.
2. Across-Subexperiment Aggregation: Standard stacked DID often fails to recover a well-defined causal parameter (the Aggregate Average Treatment Effect on the Treated, or ATT) because it aggregates treated and control trends using different cohort weights. As noted by Wing et al. (2024), this leads to a distorted mixture of trends unless "corrective stacked weights" are applied.

Existing methods often treat covariate adjustment (matching/weighting) and aggregation as separate, incompatible steps, or they fail to extend these designs to repeated treatment episodes (e.g., regimes switching on and off).

2. Methodology: CBWSDID

The author proposes Covariate-Balanced Weighted Stacked Difference-in-Differences (CBWSDID), a unified framework that separates design adjustment from aggregation while maintaining a single regression-based estimator.

A. Core Logic

The estimator operates in two conceptual stages, both represented through non-negative weights:

1. Stage 1: Within-Subexperiment Design Adjustment.

   • For each sub-experiment (cohort $a$), the author constructs design weights ($b_{sa}$) for control units.
   • These weights are derived via matching (e.g., nearest neighbor) or weighting (e.g., entropy balancing, IPW) to balance pre-treatment covariates ($X_{sa}$) and lagged outcomes between treated and control units.
   • Treated units retain a weight of 1.
   • This step ensures that the counterfactual trend for a specific cohort is estimated using a comparable control pool, satisfying conditional parallel trends.

2. Stage 2: Across-Subexperiment Aggregation.

   • The author integrates the corrective stacked weights from Wing et al. (2024).
   • The final sample weight ($W_{sa}$) for a control unit in sub-experiment $a$ is the product of the design weight ($b_{sa}$) and a corrective factor that aligns the control mass with the treated cohort share:
     $$W_{sa} = b_{sa} \times \frac{N^D_a / N^D_{\Omega}}{\tilde{N}^C_a / \tilde{N}^C_{\Omega}}$$
     (Where $\tilde{N}$ represents the effective mass of controls after design weighting).
   • This ensures the final estimator aggregates cohort-specific effects using the target weights ($N^D_a / N^D_{\Omega}$), recovering the trimmed aggregate ATT.

B. Extension to Repeated Treatments

The framework is extended beyond absorbing treatments (0 $\to$ 1) to repeated episodes (0 $\to$ 1 and 1 $\to$ 0) under a finite-memory assumption.

• Unit of Analysis: Shifts from "cohorts" to "episodes" defined by a switch time ($\tau$) and a recent treatment history profile ($h$) of length $L$.
• Finite Memory: Potential outcomes depend on the treatment path only through the most recent $L$ periods.
• Control Definition: Controls are "stable untreated episodes" with the same recent history $h$, rather than "never-treated" units. This allows previously treated units to serve as controls for new episodes if their recent history matches.
• The same two-stage weighting logic applies, resulting in an episode-weighted aggregate ATT.

C. Identification Assumptions

The identification relies on:

1. No Anticipation: No treatment effects prior to the switch.
2. Within-Subexperiment Weighted Parallel Trends: The weighted average of untreated trends in the control group equals the untreated trend of the treated group.
3. Overlap: Non-empty treated and control sets with finite effective mass.
4. Pre-treatment Refinement: Design weights are constructed solely from pre-treatment information.
5. Finite Memory (for repeated cases): Outcomes depend only on the recent $L$ periods of history.

3. Key Contributions

1. Unified Framework: CBWSDID unifies matching/weighting refinement with the aggregation logic of weighted stacked DID. It allows researchers to use design-based refinements without altering the target estimand defined by the aggregation weights.
2. Extension to Repeated Treatments: It generalizes the weighted stacked DID approach to settings with switching treatments (0 $\to$ 1 and 1 $\to$ 0), bridging the gap between weighted stacked DID and episode-based designs like PanelMatch (Imai et al., 2023).
3. Software Implementation: The author provides the R package cbwsdid to facilitate the implementation of this estimator.

4. Results and Evidence

A. Simulation Study

• Setup: A panel with 500 units where treatment timing is correlated with covariates and untreated slopes (violating unconditional parallel trends).
• Findings:
  • Ordinary and Weighted Stacked DID: Exhibited significant bias and spurious pre-trends (rejecting the null of no effect in pre-periods).
  • CBWSDID (Matching & Weighting): Substantially reduced pre-trend bias and recovered the true dynamic treatment effect path.
  • Performance: The weighting-based version performed slightly better in this specific simulation, but both refinement methods significantly outperformed unrefined benchmarks.

B. Empirical Application 1: Fair Housing Act (Trounstine, 2020)

• Context: Effect of Fair Housing Act adoption on city racial segregation (whiteness).
• Challenge: Treated and control cities had visibly different pre-treatment dynamics.
• Results:
  • Unrefined estimators (TWFE, Sun-Abraham, Weighted Stacked DID) showed large, significant pre-trends and a sharp post-treatment decline in whiteness.
  • CBWSDID: Flattened the pre-treatment path (coefficients near zero) and substantially attenuated the post-treatment decline, rendering it statistically indistinguishable from zero.
  • Implication: The large negative effect found in previous studies was driven by poor treated-control comparability, not the policy itself.

C. Empirical Application 2: Democracy and Growth (Acemoglu et al., 2019)

• Context: Effect of switching between autocracy and democracy on GDP per capita.
• Comparison: CBWSDID vs. PanelMatch.
• Results:
  • Both methods yielded similar substantive conclusions: weak positive effects for democratization and persistent negative effects for autocratization.
  • CBWSDID produced lower variance estimates and offered a more transparent regression-based framework compared to the matching-heavy PanelMatch approach.
  • Pre-trends remained non-zero, highlighting the difficulty of the design even with rich covariates.

5. Significance

The paper provides a critical bridge between design-based panel matching (which focuses on local comparability) and weighted stacked DID (which focuses on global aggregation).

• Practical Value: It offers a robust solution for researchers dealing with staggered or repeated treatments where unconditional parallel trends are implausible.
• Methodological Clarity: It demonstrates that covariate adjustment and aggregation are not mutually exclusive; they can be combined into a single, coherent weighting scheme.
• Policy Implications: The empirical results suggest that many previously accepted "large effects" in staggered DID literature may be artifacts of unbalanced covariates, urging a re-evaluation of causal claims in demanding settings.

The accompanying R package cbwsdid makes this advanced methodology accessible for applied researchers.