📈 economics

Testing identification in mediation and dynamic treatment models

This paper proposes and validates a machine learning-based statistical test for verifying the identification assumptions of causal effects in mediation and dynamic treatment models, which is demonstrated through simulations and an application to Slovak labor market data.

Original authors: Martin Huber, Kevin Kloiber, Lukas Laffers

Published 2026-06-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Martin Huber, Kevin Kloiber, Lukas Laffers

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to figure out if a specific training program actually helps people get jobs. You have a hunch that the program works, but you can't just look at the data and say, "Aha! It worked!" because maybe the people who signed up for the program were already more motivated, smarter, or had better connections than those who didn't. In the world of statistics, we call these hidden factors "confounders," and they make it very hard to prove cause and effect.

This paper introduces a new "lie detector test" for researchers. It helps them check if their assumptions about cause and effect are actually holding up, or if they are just guessing.

Here is how the paper breaks it down, using simple analogies:

The Setup: The Training Camp and the Job

Imagine a job seeker goes through a two-step process:

Step 1 (The Treatment): They attend a "Graduate Practice" training (let's call this D).
Step 2 (The Mediator): After that, they might get a second boost, like a "Hiring Incentive" or subsidized job (let's call this M).
The Result: Finally, we see if they get a job (Y).

Researchers want to know: Did the first training help directly? Did it help by getting them into the second program? Or did the second program do all the work?

The Problem: The "Hidden Hand"

Usually, researchers assume that if they control for everything they can see (like age, education, and past work history), the decision to join the program is random. But what if there's a "hidden hand" (unobserved factors) pushing both the decision to join and the chance of getting a job? If that hidden hand exists, the whole study is flawed.

The Solution: The "Spotlight" Test

The authors propose a test that uses two special tools called Instruments (let's call them Z1 and Z2).

Think of these instruments like spotlights or random switches that influence whether someone gets into the training, but only through the training itself.

Z1 is a spotlight that makes it more likely someone gets into the first training (D).
Z2 is a spotlight that makes it more likely someone gets into the second program (M).

The Golden Rule of the Test: These spotlights should never touch the final result (getting a job) directly. They can only influence the result by pushing the person into the training programs.

How the Test Works

The researchers check three things to see if the "spotlights" are behaving correctly:

Does the first spotlight (Z1) affect the job outcome directly?
- The Test: If we know who got the first training and we know their background, does the spotlight (Z1) still predict if they get a job?
- The Answer: It shouldn't. If Z1 predicts the job even after we account for the training, then Z1 is a "bad spotlight" (it's cheating).
Does the first spotlight (Z1) affect the second program (M) directly?
- The Test: If we know someone's background and the first training, does the spotlight (Z1) still predict if they get the second program?
- The Answer: It shouldn't. Z1 should only get them into the first step, not the second.
Does the second spotlight (Z2) affect the job outcome directly?
- The Test: If we know who got the first training, who got the second, and their background, does the second spotlight (Z2) still predict the job?
- The Answer: It shouldn't. Z2 should only influence the second step, not the final result.

If the data passes all three checks, it suggests that the "hidden hand" isn't messing things up, and the researchers can trust their conclusions about cause and effect. If the test fails, it's a red flag that the study's assumptions are broken.

The "Machine Learning" Engine

Because real-world data is messy and has thousands of variables (like 264 different details about a job seeker), the authors use Machine Learning to run this test. Think of Machine Learning as a super-smart assistant that can juggle all these variables at once to see if the "spotlights" are truly independent of the final result.

The Real-World Test: Slovakia

The authors took their test to the real world using data from Slovakia. They looked at young unemployed people who went through a sequence of government programs:

First, a "Graduate Practice" (training).
Then, "Employment Incentives" (subsidized jobs).

They used the local availability of these programs at different government offices as their "spotlights." (For example, if a local office had a lot of spots open last year, it was more likely a person would get in this year, purely by chance of location).

The Result: The test did not reject their assumptions. This means the data looked clean; the "spotlights" seemed to work correctly, and the researchers could confidently say that participating in both programs increased the chances of getting a job by about 8.5%.

The Bottom Line

This paper doesn't invent a new way to calculate the effect of a program. Instead, it invents a quality control check. It gives researchers a way to say, "We checked our assumptions using these special variables, and they passed the test, so our results are likely trustworthy." If the test fails, it warns them: "Stop! Your assumptions are broken, and your results might be wrong."

Technical Summary: Testing Identification in Mediation and Dynamic Treatment Models

Problem Statement
In causal mediation analysis, dynamic treatment evaluation, and sample selection models, the identification of causal effects (such as direct, indirect, or joint effects) typically relies on the assumption of sequential ignorability. This assumption posits that, conditional on observed covariates, the treatment and the mediator (or a selection indicator) are exogenous, meaning they are not confounded by unobservables. However, this assumption is untestable in standard observational data and is often controversial, as unobserved confounders cannot be definitively ruled out. While previous literature has proposed tests for the identification of single treatment effects using instruments, there is a lack of methods to jointly test the identification assumptions for models involving sequential treatments, mediators, and sample selection.

Methodology
The authors propose a testing framework that extends the work of Huber and Kueck (2022) from single treatment models to settings with sequential assignment of a treatment ( $D$ ) and a mediator ( $M$ ), or a selection indicator. The approach relies on two sets of observed variables: covariates ( $X$ ) to be controlled for and suspected instruments ( $Z_1$ for $D$ , and $Z_2$ for $M$ ).

The methodology is built on a specific causal structure (Assumption 1) that rules out reverse causality from the outcome to the treatment, mediator, covariates, or instruments, and assumes causal faithfulness. Under this structure, the authors derive testable implications that are equivalent to the joint satisfaction of:

Exogeneity: The treatment and mediator are conditionally independent of potential outcomes and potential mediators given covariates (sequential ignorability).
Instrument Validity: The instruments are conditionally independent of potential outcomes and mediators given covariates and the respective treatment/mediator status, implying they do not directly affect the outcome and are unconfounded.

Specifically, the testable implications (TIs) are conditional independence statements involving factual quantities:

$Y \perp Z_1 | D, X$
$M \perp Z_1 | D, X$
$Y \perp Z_2 | D, M, X$

The framework is flexible enough to accommodate:

Instrument Interdependence: Scenarios where $Z_1$ affects $Z_2$ , requiring $Z_2$ to be included in the conditioning set for testing $Z_1$ .
Post-treatment Confounding: Scenarios where post-treatment covariates ( $W$ ) affect the mediator-outcome relationship, requiring $W$ to be included in the conditioning sets.

To implement the test in high-dimensional settings, the authors employ a machine learning-based approach. They test the null hypothesis that the conditional mean of the outcome or mediator is independent of the instruments. This is formulated as a quadratic moment condition involving the squared differences between regression functions that include and exclude the instruments. The estimation utilizes cross-fitting and Neyman orthogonality (Chernozhukov et al., 2018), allowing for the use of flexible machine learning algorithms (e.g., Lasso) to control for high-dimensional covariates without compromising asymptotic properties. A random perturbation term is added to the score function to ensure non-degenerate variance under the null hypothesis.

Key Contributions

Extension of Identification Tests: This is the first study to propose an identification test specifically for causal mediation, dynamic treatment effects, and sample selection models, extending the single-treatment framework of Huber and Kueck (2022).
Joint Testing: The method allows for the joint testing of treatment exogeneity and instrument validity, rather than testing them separately.
Flexibility: The framework handles multivalued instruments, sequential instruments (where one instrument affects another), and dynamic confounding (post-treatment covariates).
Diagnostic Decomposition: The test structure allows researchers to localize rejections to specific parts of the causal model (e.g., distinguishing between a violation of treatment exogeneity versus instrument validity).
Machine Learning Integration: The proposal of a data-driven testing procedure capable of handling high-dimensional covariates via machine learning, addressing the practical challenge of controlling for numerous confounders.

Results

Simulation Study: The authors conduct a simulation study with sample sizes of 1,000 and 4,000. The results indicate that the test maintains correct empirical size (close to the nominal 5% level) when identification assumptions hold. When assumptions are violated (e.g., by introducing unobserved confounders or invalidating instruments), the test demonstrates high power, with rejection rates approaching 100% in larger samples. The study also shows that the proposed test performs well in detecting violations even when instruments have direct effects on the outcome.
Empirical Application: The method is applied to Slovak labor market data to evaluate the effects of sequential training programs (Graduate Practice followed by Employment Incentives) on employment. Using local program availability as instruments and controlling for a rich set of pre- and post-treatment covariates, the testable implications are not rejected at conventional significance levels. This suggests that the sequential ignorability assumption is plausible for this specific application. Consequently, the authors estimate the dynamic treatment effect, finding that participation in both programs increases employment probability by approximately 8.55 percentage points.

Significance and Claims
The paper claims to provide a necessary tool for verifying identification assumptions in complex causal models where unobserved confounding is a primary concern. By establishing sufficient testable conditions, the authors argue that researchers can move beyond relying solely on theoretical motivation or intuition for sequential ignorability. The significance lies in the ability to "verify" identification in observational data, thereby increasing confidence in causal estimates derived from mediation and dynamic treatment models. The authors remain modest, noting that while the test verifies necessary conditions for identification, it does not guarantee the identification of natural direct and indirect effects (which require stronger cross-world independence assumptions), but it does support the identification of controlled direct effects and total effects.