📊 statistics

Semiparametric Mediation Analysis with Separately Observed Mediator and Outcome under Unmeasured Confounding

This paper introduces a novel data fusion framework that enables semiparametric mediation analysis under unmeasured confounding and separately observed mediators and outcomes by leveraging shared instrumental variables and latent alignment to identify causal pathways, which is demonstrated through an application to the effect of a specific SNP on dementia risk via immune-related gene expression.

Original authors: Sijia Li, Ruoyu Wang

Published 2026-06-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sijia Li, Ruoyu Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to figure out exactly how a new medicine works. You know the medicine (let's call it the Exposure) makes people feel better (the Outcome). But you suspect it works by first changing something in the body, like a specific protein level (the Mediator).

To prove this, you usually need to look at the same person three times:

Did they take the medicine?
Did their protein level change?
Did they feel better?

The Problem: The "Missing Link" Puzzle
In the real world, this is often impossible. Imagine you have two different libraries of medical records:

Library A has records of who took the medicine and who felt better, but it forgot to record the protein levels.
Library B has records of who took the medicine and what their protein levels were, but it forgot to record who felt better.

You have the "Medicine" and "Outcome" in one pile, and the "Medicine" and "Mediator" in another. You never see the "Mediator" and "Outcome" together for the same person. Standard statistics say, "You can't solve this puzzle because the link is broken."

Furthermore, there might be hidden factors (like a secret lifestyle habit) that affect both the protein level and the feeling better, which you can't measure. This makes the puzzle even harder.

The Solution: A New "Data Fusion" Trick
The authors of this paper invented a clever new way to solve this puzzle by combining these two incomplete libraries. They call their method a data fusion framework.

Here is the analogy they use to make it work:

1. The "Secret Decoder Ring" (Instrumental Variables)

To connect the two libraries without seeing the missing link, the researchers use a "Secret Decoder Ring" called an Instrumental Variable (IV).

Think of the IV as a remote control.
The remote control (IV) changes the TV channel (the Mediator/Protein level).
Crucially, the remote control only changes the channel; it doesn't directly make the picture look better or worse (it has no direct effect on the Outcome).
Because the remote control is recorded in both libraries, it acts as a bridge. Even though Library A doesn't know the protein levels, it knows which "remote control" was used. Library B knows the protein levels and the "remote control."

By mathematically aligning the "remote control" patterns across both libraries, the researchers can infer what would have happened if the protein levels had changed, even though they never saw the protein and the health outcome together in the same person.

2. The "Ghost Alignment" (Latent Alignment)

Since the two libraries might have different types of people (different ages, different backgrounds), the researchers assume that if you look at people with the same "remote control" settings and similar backgrounds, the hidden rules of the universe are the same in both libraries. They call this latent alignment. It's like assuming that if you play a video game on two different consoles with the same controller settings, the physics of the game world are identical, even if the graphics look slightly different.

3. The "Double-Check" Safety Net (Multiple Robustness)

The authors built their math so that it is "multiply robust."

Imagine you are trying to guess a secret number. You have two different clues.
If you get Clue A wrong, but Clue B is right, you can still guess the number.
If you get Clue B wrong, but Clue A is right, you can still guess the number.
You only fail if both clues are wrong.
This makes their method very reliable, even if some of the initial guesses about the data aren't perfect.

4. Finding the Right Remote Controls (IV Selection)

Sometimes, you might have a bunch of potential "remote controls," but some of them are broken (they affect the outcome directly, not just through the mediator). The authors created a smart algorithm that acts like a detective, sifting through the list of potential remote controls to find the ones that are truly valid and ignore the broken ones.

Real-World Test: The Dementia Study

The authors tested this method on a real medical mystery: Dementia.

The Exposure: A specific gene variant (a tiny change in DNA called rs610932).
The Mediator: The activity level of two specific genes (PTK2B and CD33) in the immune system.
The Outcome: Developing dementia.

They had data from one group of patients with the gene and dementia status, but no gene activity data. They had another group with the gene and gene activity data, but no dementia status.
Using their new method, they combined these groups. They found that the gene variant likely protects against dementia by changing the activity of the PTK2B gene (which is involved in immune cells in the brain), but it didn't seem to work much through the CD33 gene.

In Summary
This paper is about a new mathematical "glue" that allows scientists to combine two broken datasets to figure out cause-and-effect pathways. It uses "remote controls" (instrumental variables) to bridge the gap between missing pieces, handles hidden confounders, and provides a safety net so the results remain trustworthy even if some assumptions are slightly off. They proved it works by solving a real puzzle about how genes might influence dementia risk.

Technical Summary: Semiparametric Mediation Analysis with Separately Observed Mediator and Outcome under Unmeasured Confounding

1. Problem Statement

Mediation analysis is a standard tool for decomposing the total causal effect of an exposure $D$ on an outcome $Y$ into natural direct effects (NDE) and natural indirect effects (NIE) operating through a mediator $M$ . However, a critical practical limitation arises when $M$ and $Y$ are never jointly observed in the same dataset. This scenario is common in integrative genomics (e.g., combining eQTL studies for gene expression with GWAS for disease status) and population health (e.g., merging large-scale exposure cohorts with clinical databases containing granular intermediate phenotypes).

In such settings, the standard identification strategy for NDE and NIE fails because it relies on the unobserved joint distribution of $(M, Y)$ . Furthermore, this problem is exacerbated by the presence of unmeasured confounding between the mediator and the outcome ( $M$ - $Y$ confounding), which is a major obstacle even in complete data settings. Existing data fusion methods generally assume that some overlap exists where both variables are observed, or they rely on parametric assumptions that may not hold. This paper addresses the challenge of identifying causal mediation effects when:

Data sources are heterogeneous: one source contains $(D, Y)$ and another contains $(D, M)$ , with no joint observation of $(M, Y)$ .
Unmeasured confounders $U$ exist between $M$ and $Y$ .

2. Methodology and Identification Strategy

2.1 Setup and Assumptions

The authors consider a structural causal model (SCM) with exogenous variables $D = (T, Z, A)$ , where $T$ is the target exposure, $Z$ represents instrumental variables (IVs) with no direct effect on $Y$ , and $A$ are auxiliary variables with direct effects on $Y$ . The data comes from two sources ( $S=0$ and $S=1$ ):

$S=0$ : Observes $(Y, D, X)$ .
$S=1$ : Observes $(M, D, X)$ .

The identification relies on the following key assumptions:

Unconfounded Exposure: $D \perp\!\!\!\perp (\epsilon_Y, \epsilon_M, U) \mid X, S$ .
Latent Unconfounded Mediator: $\epsilon_Y \perp\!\!\!\perp \epsilon_M \mid X, U, S$ .
No $M$ - $U$ Interaction: The effect of $M$ on $Y$ does not depend on the unmeasured confounder $U$ (a "no-interaction" condition).
Cross-Source Alignment: Conditional on $D$ and $X$ , the distributions of $U, M,$ and $Y$ are exchangeable across sources ( $S \perp\!\!\!\perp (U, M, Y) \mid D, X$ ). This allows for covariate and exposure shifts between sources.
Completeness Conditions:
- $M-Z$ Completeness: Variability in $M$ is captured by variability in $Z$ .
- $Z-M$ Completeness (for efficiency): Variability in $Z$ is captured by variability in $M$ .

2.2 Identification via Bridge Functions

The core innovation is the use of bridge functions ( $w$ ) to align the observed mediator distribution in one source with the counterfactual mediator distribution required for the other. Under the completeness condition, the authors prove the existence of functions $w_{tt'}^s$ that satisfy an integral equation (Equation 2 in the paper). These functions reweight the observed mediator strata to match the counterfactual strata, effectively circumventing the need to observe $(M, Y)$ jointly.

The natural direct and indirect effects are identified as functionals of the observed data distributions using these bridge functions:
$\psi^{NDE}_s = \psi^{10}_s - \psi^{00}_s, \quad \psi^{NIE}_s = \psi^{11}_s - \psi^{10}_s$
where the terms $\psi^{tt'}_s$ involve expectations of $Y$ and the bridge functions $w$ .

2.3 Handling Unknown IVs

When the set of valid IVs $Z$ is unknown but a candidate set $C$ is available, the authors propose a sparsity-based identification strategy. They assume that more than half of the candidate IVs are valid (Assumption 6). By solving a sparse regression problem to find a function $r^*(M, D, X)$ that links the conditional outcome in source 0 to the conditional mediator in source 1, the valid IVs are identified as the components of $D$ that $r^*$ does not depend on.

2.4 Estimation

The authors develop semiparametric, influence-function-based one-step estimators.

Multiply Robustness: The estimators are consistent if at least one of several combinations of nuisance parameters (e.g., the bridge functions $w$ , the outcome regression $\nu$ , or the propensity score $\lambda$ ) is correctly specified. Specifically, consistency holds if either the outcome model is correct, or the bridge function model is correct, or specific combinations of the two are correct.
Efficiency: Under mutual completeness (where $Z$ is complete in $M$ and vice versa), the proposed estimators attain the semiparametric efficiency bound. The authors derive the canonical gradients (influence functions) for the effects.
Regularization: Since solving for bridge functions involves ill-posed inverse problems (Fredholm integral equations), the authors suggest regularization strategies (e.g., Tikhonov regularization or variance-based penalties) to stabilize estimation.

3. Key Results

3.1 Theoretical Contributions

Identification: The paper establishes that NDE and NIE are identifiable under unmeasured confounding and separate observation of $M$ and $Y$ , provided valid IVs and a no-interaction condition exist.
Efficiency Bounds: The authors derive the semiparametric efficiency bounds and show that their estimators achieve these bounds under mutual completeness.
Connection to Literature: The framework generalizes two-sample instrumental variable (IV) problems, showing that average treatment effects in two-sample settings are special cases of their mediation framework.

3.2 Simulation Studies

Performance: In simulations with both single and multiple instruments, the proposed estimators exhibited negligible bias across various model misspecification scenarios, confirming the multiply robust property.
Efficiency: In settings with multiple instruments where only the mediator is complete in the instrument set (non-complete setting), the proposed method using adaptive instrument aggregation showed substantial efficiency gains (smaller standard errors) compared to using a single IV.
IV Selection: The proposed algorithm for selecting valid IVs from a candidate set performed with high accuracy (near 100% correct identification) as sample size increased, even in the presence of invalid IVs.
Inference: Bootstrap-based confidence intervals were found to be more stable and reliable than influence-function-based intervals, particularly under model misspecification.

3.3 Data Application

The framework was applied to the Alzheimer's Disease Neuroimaging Initiative (ADNI) to quantify the mediation of the SNP rs610932 on dementia risk through the gene expression of PTK2B and CD33.

Data Fusion: The study combined ADNI-1 (outcomes and SNPs, no gene expression) and ADNI-GO/2 (SNPs and gene expression, no long-term outcomes) cohorts.
Findings: The estimated natural indirect effect (NIE) through PTK2B was negative and substantial relative to the total effect, suggesting a protective mechanism. The NIE through CD33 was close to null.
Interpretation: The authors note that while confidence intervals were wide and crossed zero, the results are suggestive of a biological mechanism where the minor allele of rs610932 operates through altered PTK2B expression and downstream microglial immune pathways.

4. Significance and Claims

The paper claims to introduce a novel data fusion framework that restores the identification of causal mediation effects in settings where the mediator and outcome are never jointly observed and unmeasured confounding is present.

Overcoming Data Limitations: It addresses a pervasive issue in modern integrative genomics and health data science where data silos prevent joint observation of key variables.
Robustness to Unmeasured Confounding: Unlike standard mediation analysis which requires strong assumptions about unmeasured confounders, this method leverages IVs and a no-interaction condition to identify effects even when $U \neq \emptyset$ .
Flexibility: The method accommodates covariate and exposure shifts across data sources and handles cases where the set of valid IVs is unknown.
Efficiency: By utilizing semiparametric efficiency theory, the proposed estimators are shown to be efficient under appropriate conditions, providing a rigorous statistical foundation for inference in these complex settings.

The authors position this work as a necessary advancement for "disentangling causal pathways" in real-world studies where data completeness is unattainable, moving beyond the "does it work" question to "through what pathways does it work" even in fragmented data environments.