📈 economics

Heterogeneous Treatment Effects and Causal Mechanisms

This paper establishes that while detecting heterogeneous treatment effects with respect to pre-treatment covariates can support inferences about mechanism activation under specific exclusion assumptions, the absence of such heterogeneity is generally uninformative, thereby necessitating more rigorous research designs and interpretative caution.

Original authors: Jiawei Fu, Tara Slough

Published 2026-04-08

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Jiawei Fu, Tara Slough

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery. You know that a specific event (the Treatment) caused a change in the outcome (the Result). But you don't just want to know that it happened; you want to know how or why it happened. You suspect a specific "mechanism" was the culprit.

For example, you know that a new government program increased people's trust. You suspect the mechanism is "feeling heard." But how do you prove it?

In the world of social science research, the standard detective tool for the last few decades has been looking for Heterogeneous Treatment Effects (HTEs).

The Old Detective's Trick: "The Split"

The old way of thinking goes like this:

The Theory: "If my theory is right, the program should work much better for people who were previously angry at the government than for people who already liked it."
The Test: You split your data into two groups: "Angry People" and "Happy People."
The Conclusion: If the program worked significantly better for the Angry group, you say, "Aha! The mechanism (feeling heard) is active!" If the program worked the same for both, you say, "The mechanism is dead; it didn't do anything."

The Problem: This paper argues that this detective trick is full of holes. It's like trying to identify a specific ingredient in a soup by tasting the soup, but you don't know if the saltiness you taste is from the salt you added, or if the pot itself was salty, or if the spoon changed the flavor.

Here is the paper's new framework, explained with simple analogies.

1. The "Exclusive Ingredient" Rule (The Exclusion Assumption)

The Analogy: Imagine you are testing if a specific spice (Cinnamon) makes a cake taste better. You decide to test this by seeing if the cake tastes better when you use "Cinnamon-Lovers" vs. "Cinnamon-Haters."

The Flaw: What if "Cinnamon-Lovers" also happen to be "Sugar-Lovers"? If the cake tastes better for them, is it because of the Cinnamon (your mechanism) or the Sugar (another mechanism)?

The Paper's Point: To use this "split" method, you must be 100% sure that the group you are splitting by (the moderator) only affects the mechanism you are testing. It cannot affect any other part of the process.

In the paper: If you split by "Partisan Alignment," you have to prove that partisanship only changes how much they hate corruption, and doesn't also change how they react to the news in some other way. If it does both, your test is useless.

2. The "Binary Switch" Trap (Non-Linear Transformations)

The Analogy: Imagine you are measuring how much a student enjoys a class.

Scenario A (The Real Feeling): You ask them to rate their enjoyment from 0 to 100. This is a smooth, continuous scale.
Scenario B (The Binary Choice): You ask them, "Did you pass or fail?" (Yes/No).

Now, imagine a teacher gives a great lecture.

In Scenario A, the students who were bored (score 20) become happy (score 80). The students who were already happy (score 90) stay happy (score 95). The change is huge for the bored ones, small for the happy ones. You see a clear difference.
In Scenario B, the bored student (who was failing) passes. The happy student (who was already passing) stays passing. Both say "Yes." The change looks the same (0 to 1, or 1 to 1).

The Paper's Point: Most political science studies use Scenario B (Did you vote? Yes/No). They use "Vote Choice" instead of "Voter Satisfaction."

When you turn a smooth feeling (Utility) into a binary choice (Vote), you create fake differences.
You might see that "Partisans" react differently to corruption news, not because their mechanism is different, but simply because they were already so close to the "voting line" that a tiny nudge pushed them over, while others were too far away to move.
The Lesson: If you measure a binary outcome (Vote/No Vote), finding a difference between groups tells you nothing about why it happened. It just tells you about the math of the switch.

3. The "Silence" is Deafening (Absence of Evidence)

The Old Detective: "I looked for a difference between Group A and Group B. I found none. Therefore, the mechanism is dead."

The New Detective: "I looked for a difference. I found none. This could mean:

The mechanism is dead.
The mechanism is alive, but it affects everyone exactly the same way.
I picked the wrong group to split by (maybe I should have split by 'Age' instead of 'Gender')."

The Paper's Point: Finding a difference (HTE) is strong evidence that a mechanism is working. But finding NO difference is useless. It tells you nothing. You cannot rule out a mechanism just because you didn't see a split in the data.

4. The "Magic Detector" (Mechanism Detector Variables)

To make this work, you need a special kind of variable called a Mechanism Detector Variable (MDV).

Think of this like a metal detector.
If you walk a metal detector over a patch of sand and it beeps, you know there is metal there.
But if it doesn't beep, you don't know if there is no metal, or if the metal is buried too deep, or if the detector is broken.
The Paper's Advice: Before you run your study, you must have a very strong theory that says, "This specific variable (e.g., 'Corruption Aversion') is the only thing that changes how the mechanism works." If you can't prove that, your "metal detector" is useless.

Summary: What Should Researchers Do?

The authors aren't saying "Stop doing this research." They are saying, "Stop doing it blindly."

Don't trust the "No Difference" result: If you don't see a split in the data, don't say the mechanism is dead. You just learned nothing.
Watch out for "Yes/No" answers: If your outcome is a binary choice (Vote/Don't Vote), be very careful. The math of the choice can fake a difference where none exists. Try to measure the feeling (Utility) behind the choice if possible.
Be the "Exclusion" Police: You must explicitly argue why the group you are splitting by (e.g., Partisanship) doesn't affect anything else in the story. If you can't prove that, your conclusion is shaky.
Use Multiple Detectors: Don't rely on just one way to split your data. If you have two different "metal detectors" (two different variables) and only one beeps, you learn something. If you only have one, and it's silent, you are stuck.

The Bottom Line:
Using "Heterogeneous Treatment Effects" to find mechanisms is like trying to hear a whisper in a noisy room. Sometimes, if you listen carefully to the right person, you can hear the whisper (the mechanism). But often, the noise (other variables) or the way you are listening (binary choices) makes it impossible to tell if the whisper is there or not. The paper gives us a new manual on how to tune our ears so we don't get fooled.

1. Problem Statement

Social scientists frequently use Heterogeneous Treatment Effects (HTEs)—specifically, the variation in Conditional Average Treatment Effects (CATEs) across pre-treatment covariates—to infer the activation of causal mechanisms. For example, if a corruption information treatment reduces voting for an incumbent more among voters with high "corruption aversion," researchers often conclude that "distaste for corruption" is the active mechanism.

However, the paper argues that the theoretical foundation for this practice is weak. Current empirical practice often treats the presence of HTEs as definitive proof of a mechanism and the absence of HTEs as proof that a mechanism is inert. The authors identify three critical gaps:

Identification Failure: HTEs do not automatically isolate a specific mechanism without strong, often implicit, exclusion assumptions.
Asymmetry of Inference: The presence of HTEs may support mechanism activation, but the absence of HTEs is generally uninformative regarding whether a mechanism is active or inert.
Measurement Artifacts: Common non-linear transformations of outcome variables (e.g., converting utility to binary vote choice, binning, logging) violate the additive separability required for valid HTE-based mechanism tests, generating spurious heterogeneity.

2. Methodology and Framework

The authors develop a formal theoretical framework extending the potential outcomes (Neyman-Rubin) model and causal mediation analysis.

Core Definitions

Treatment ( $Z$ ): Randomly assigned intervention.
Mechanisms ( $M_1, M_2, \dots$ ): Unobserved mediators activated by $Z$ that affect the outcome $Y$ .
Moderator ( $X_k$ ): A pre-treatment covariate used to detect HTEs.
Mechanism Detector Variable (MDV): A covariate $X_k$ that moderates the indirect effect of a specific mechanism $M_j$ (i.e., the Average Indirect Effect (AIE) varies by $X_k$ ).

The Decomposition

The total effect ($TE$) is decomposed into a Direct Effect ($DE$) and Indirect Effects ($IE$) for multiple mechanisms. The CATE for a covariate $X_k$ is expressed as:
$\text{CATE}(X_k=x) = \text{ADE}(X_k=x) + \sum_j \text{AIE}_j(X_k=x)$
The difference in CATEs between two levels of $X_k$ ( $x$ and $x'$ ) is:
$\Delta \text{CATE} = \Delta \text{ADE} + \sum_j \Delta \text{AIE}_j$

Key Assumptions (Exclusion Assumptions)

To link $\Delta \text{CATE}$ to a specific mechanism's $\Delta \text{AIE}$ , the authors propose two exclusion assumptions:

Assumption 1 (Exclusion I): The covariate $X_k$ does not moderate the Direct Effect (or effects of other unmeasured mechanisms).
Assumption 2 (Exclusion II): The covariate $X_k$ does not moderate the Indirect Effects of other mechanisms ( $M_{j \neq 1}$ ).

These assumptions are distinct from the Sequential Ignorability required in standard mediation analysis. The authors show that neither set of assumptions logically implies the other; thus, HTE analysis is not inherently more or less "agnostic" than mediation analysis, but it offers the advantage of not requiring measurement of the mediator.

The Role of Non-Linear Transformations

The framework introduces the concept of a Transformed Outcome ( $\tilde{Y} = h(Y)$ ), where $h(\cdot)$ is a non-linear function (e.g., $Y \to \mathbb{I}[Y \ge 0]$ for binary vote choice).

Critical Insight: Even if Assumptions 1 and 2 hold for the latent outcome $Y$ , they generally fail for the transformed outcome $\tilde{Y}$ . Non-linear transformations break the additive separability of mechanisms, causing $X_k$ to appear as a moderator for mechanisms it does not actually moderate (e.g., creating HTEs for partisan alignment even when partisanship does not moderate the corruption distaste mechanism).

3. Key Results

The authors analyze four exhaustive cases based on the presence of HTEs and the nature of the outcome variable (Table 2 in the paper):

Case	Outcome Type	HTEs Observed?	Inference about Mechanism Activation
1	Latent ( $Y$ )	Yes	Valid Inference: The covariate is an MDV. The mechanism is active for at least one unit.
2	Latent ( $Y$ )	No	Uninformative: Could mean (a) the covariate is not an MDV, (b) the mechanism is active but uniform across units, or (c) the mechanism is inert. Cannot rule out activation.
3	Transformed ( $\tilde{Y}$ )	Yes	Invalid Inference: HTEs may be generated by the non-linear transformation itself. Cannot infer mechanism activation.
4	Transformed ( $\tilde{Y}$ )	No	Uninformative: Lack of HTEs provides no information about mechanism activation due to violated exclusion assumptions.

Key Findings:

Asymmetry: HTEs can confirm a mechanism is active (under strict assumptions), but the absence of HTEs cannot confirm a mechanism is inactive.
Measurement Sensitivity: In the motivating example (corruption and voting), HTEs appear for "vote choice" (binary) across partisan lines, even though partisanship does not moderate the underlying "distaste" mechanism. This is a statistical artifact of the threshold function (vote choice = utility > 0), not evidence of a new mechanism.
Power Issues: In finite samples, researchers often lack the power to detect true HTEs (false negatives), making the "absence of HTEs" even less reliable for ruling out mechanisms.

4. Contributions

Theoretical Framework: Provides the first rigorous theoretical link between HTEs and causal mechanisms, clarifying that HTEs identify differences in indirect effects, not the effects themselves.
Identification Conditions: Establishes that Exclusion Assumptions (covariates must not moderate other channels) are necessary and sufficient to link HTEs to specific mechanisms.
Measurement Critique: Demonstrates that standard practices (using binary outcomes, Likert scales, or log-transformed variables) often violate the conditions required for HTE-based mechanism tests, leading to Type I errors (false positives).
Practical Guidance: Offers a roadmap for researchers to improve mechanism detection, including:
- Prioritizing latent/continuous outcomes over transformed/binary ones for mechanism tests.
- Explicitly stating and defending exclusion assumptions.
- Using multiple candidate MDVs to triangulate results.
- Considering model-based approaches (e.g., Random Utility Models) to recover latent utility from choice data.

5. Significance

This paper fundamentally challenges a dominant practice in political science and social science. It reveals that a majority of quantitative studies claiming to identify mechanisms via HTEs are likely making invalid inferences due to untested exclusion assumptions and inappropriate outcome transformations.

By distinguishing between mechanism activation and measurement artifacts, the authors provide a corrective lens for the field. They argue that while HTEs are powerful for policy targeting and extrapolation, their use for mechanism detection requires much stricter theoretical justification and careful measurement strategies than currently practiced. The paper suggests that without these corrections, researchers risk misattributing effects to mechanisms that are merely statistical byproducts of non-linear measurement.