The Generalized Falsification Adaptive Set for Violations of the Exclusion Restriction and Exogeneity

Technical Summary: The Generalized Falsification Adaptive Set for Violations of the Exclusion Restriction and Exogeneity

Problem Statement
Instrumental variables (IV) estimation relies on two core identifying assumptions: the exclusion restriction (instruments affect the outcome only through the treatment) and exogeneity (instruments are uncorrelated with unobserved determinants of the outcome). When the baseline model is falsified by overidentifying restriction tests, researchers often lack a point estimate for the treatment effect, $\beta$. Masten and Poirier (2021, henceforth MP) proposed the Falsification Adaptive Set (FAS) to address this, defining an identified set for $\beta$ under the minimal relaxation of assumptions required to avoid falsification.

However, MP's approach assumes that invalid instruments violate the exclusion restriction only. This paper demonstrates that the nature of the violation is critical. An invalid instrument may act as a confounder (violating exclusion via a direct effect on the outcome, $\gamma \neq 0$) or as a collider (violating exogeneity via correlation with the error term, $\alpha \neq 0$). These two types of violations require opposite treatments: confounders should be controlled for (included as regressors), while colliders should be excluded from the analysis. The paper argues that applying MP's exclusion-based FAS ($FAS_{excl}$) when instruments actually violate exogeneity (or vice versa) can result in the FAS excluding the true parameter $\beta$, even if a valid instrument exists.

Methodology and Model
The authors consider a linear model $Y = X\beta + E$, where $E = Z\gamma + U$ and $cov(Z, U) = \alpha$.

• Exclusion Violation: $\gamma_\ell \neq 0$ (instrument $\ell$ has a direct effect on $Y$).
• Exogeneity Violation: $\alpha_\ell \neq 0$ (instrument $\ell$ is correlated with the error term).
• Validity: An instrument is valid if $\gamma_\ell = \alpha_\ell = 0$.

The authors derive specific FASs for distinct violation patterns:

1. $FAS_{excl}$: Derived under the assumption that all invalid instruments violate exclusion only ($\alpha = 0$). This set is constructed by relaxing the exclusion restriction bounds ($|\gamma| \leq \delta$).
2. $FAS_{exo}$: Derived under the assumption that all invalid instruments violate exogeneity only ($\gamma = 0$). This set is constructed by relaxing the exogeneity bounds ($|\alpha| \leq \eta$).
3. Alternative Derivation: The paper provides an alternative derivation of the falsification frontier using the population 2SLS objective function, clarifying that these sets represent minimal relaxations of moment conditions leading to non-falsification, regardless of the specific violation type.

Key Contributions
The paper makes three primary contributions:

1. Distinction of Violation Types: The authors show that incorrect classification of an invalid instrument (treating a confounder as a collider or vice versa) causes the resulting FAS to potentially exclude the true $\beta$. They demonstrate that $FAS_{excl}$ and $FAS_{exo}$ are distinct sets, and the choice between them depends on the researcher's belief regarding the violation mechanism.
2. Pattern-Specific FASs: The authors derive a pattern-specific FAS for every possible combination of exclusion and exogeneity violations across instruments. Under Assumption 6, an invalid instrument violates either exclusion or exogeneity, but not both ($\gamma_\ell \alpha_\ell = 0$). For $k_z$ instruments, there are $S = 2^{k_z}$ such patterns.
3. The Generalized FAS ($FAS_G$): To address researcher agnosticism regarding the specific violation pattern, the authors propose the Generalized Falsification Adaptive Set as the union of all pattern-specific FASs:
   $$FAS_G = \bigcup_{s=1}^{S} FAS_s$$
   They prove that if there is at least one valid and relevant instrument, and the researcher correctly transforms the instruments according to the true violation pattern of the invalid ones, the true $\beta$ is guaranteed to be contained within $FAS_G$.

Results and Empirical Illustration
The paper illustrates these results using the "roads and trade" application from Duranton et al. (2014), which uses three historical instruments to estimate the effect of interstate highways on export propensity.

• Falsification: The baseline model with all three instruments is falsified (J-test p-value 0.043).
• Divergence of Sets: The empirical analysis shows significant differences between $FAS_{excl}$, $FAS_{exo}$, and the generalized set. For instance, $FAS_{exo}$ yields a narrow interval $[0.13, 1.09]$, while $FAS_{excl}$ yields $[-0.32, 3.16]$.
• Generalized Set: The generalized FAS ($FAS_G$) is the union of all eight pattern-specific sets, resulting in $[-0.61, 3.74]$. The authors note that while this interval is wide, it is not an artifact of the union operation but reflects a specific violation pattern (where one instrument is a confounder and another is a collider) that cannot be excluded without further assumptions.
• Weak Instruments: The paper discusses the impact of the "weak instrument" cutoff (F-statistic < 10) used by MP. Removing instruments classified as weak based on this cutoff can inadvertently change the model specification (e.g., by excluding an instrument that should be a control), potentially rendering the FAS no longer motivated by the falsification of the baseline model.

Significance and Claims
The authors do not claim that one specific FAS is superior in all contexts. Instead, they argue that the choice of the FAS must match the type of violation.

• If the researcher is agnostic about the violation types, the Generalized FAS is the appropriate set, as it guarantees the inclusion of the true $\beta$ provided at least one instrument is valid and relevant.
• If domain knowledge allows the researcher to pin down the violation type for some or all instruments, the set can be "sharpened" to a specific pattern-based FAS, which is weakly narrower than the generalized set.
• The paper emphasizes that selecting a specific FAS (e.g., $FAS_{excl}$) without justification amounts to "cherry-picking" and risks excluding the true parameter. The generalized FAS is recommended for reporting when the baseline model is falsified and the violation pattern is unknown.