📊 statistics

Bounding causal effects with an unknown mixture of informative and non-informative missingness

This paper proposes a framework for bounding causal effects under an unknown mixture of informative and non-informative missingness, developing influence-function-based estimators that achieve root-n convergence and asymptotic normality while accommodating user-specified sensitivity parameters and competing outcomes like death.

Original authors: Max Rubinstein, Denis Agniel, Larry Han, Marcela Horvitz-Lennon, Sharon-Lise Normand

Published 2026-03-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Max Rubinstein, Denis Agniel, Larry Han, Marcela Horvitz-Lennon, Sharon-Lise Normand

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery: Does Drug A cure a disease better than Drug B?

You have data from thousands of patients. But there's a problem: many patients disappeared from the study before you could see if they got better or worse. Some moved away (random), but others might have stopped taking the drug because it made them feel terrible, or because their condition got so bad they went to a different hospital (not random).

This is the core problem of missing data.

The Problem: The "Silent" Dropouts

In statistics, we usually assume that missing data is like a coin flip—random and unrelated to the outcome. This is called "Missing At Random" (MAR).

The Reality: Often, missing data is not random. It's "Informative."
The Analogy: Imagine a race. If the slowest runners drop out because they are tired, the remaining runners look faster than they really are. If you only look at the finishers, you think the race was easy. But if the fastest runners dropped out because they got injured, the finishers look slower than they really are.

The authors of this paper realize that in the real world, we rarely know why people dropped out. It's usually a mixture: some left for random reasons (lost their phone), and some left because of the outcome (got too sick).

The Solution: Drawing a "Safety Net"

Instead of guessing exactly why people left (which is impossible), the authors propose a new way to think about it: Bounding.

Think of the true answer (the real effect of the drug) as a bird flying in a foggy sky. We can't see the bird, but we can build a giant, invisible cage around the sky where the bird must be.

The "Worst-Case" Cage: If we assume everyone who dropped out was actually doing the worst possible thing, how bad could the drug look?
The "Best-Case" Cage: If we assume everyone who dropped out was doing the best possible thing, how good could the drug look?

The result isn't a single number (like "Drug A is 10% better"). Instead, it's a range (e.g., "Drug A is between 5% worse and 15% better"). This range is honest. It admits we don't know everything, but it tells us the limits of what is possible.

The "Sensitivity" Knob

The authors add a clever feature: a Sensitivity Knob.
Imagine you are adjusting the volume on a radio.

Turn the knob to "Low": You assume very few people dropped out for bad reasons. Your range is narrow, and you might say, "Drug A is definitely better!"
Turn the knob to "High": You assume many people dropped out because the drug was dangerous. Your range widens. Suddenly, you might say, "Well, Drug A could be better, but it could also be worse."

This allows researchers to ask: "How bad does the missing data have to be to change my conclusion?"

If you have to turn the knob to an impossible setting (e.g., "99% of dropouts were dying") to change your mind, then your conclusion is robust.
If a tiny turn of the knob changes everything, your conclusion is fragile.

The "Competing Event" (The Death of the Outcome)

The paper also handles a tricky scenario: Death.
If a patient dies, they can't get diabetes. If you only look at the people who survived to see if they got diabetes, you are ignoring the people the drug might have killed.

The Analogy: Imagine a video game. If a player dies, they can't finish the level. If you only count the players who finished the level, you might think the game is easy. But maybe the "hard mode" killed everyone.
The authors propose a new way to measure success that counts "Death" as a failure alongside "Getting Diabetes." This gives a more honest picture of the total risk.

Real-World Example: Antipsychotics and Diabetes

The authors tested their method on real data about antipsychotic drugs (used for schizophrenia) and diabetes risk.

The Naive View: If you just ignore the missing data, it looks like some drugs (like Haloperidol) are great at preventing diabetes.
The "Safety Net" View: When they applied their bounds, they found that the "great" result only holds if the missing data isn't too suspicious.
The Tipping Point: They calculated that for the "good" result to be true, the missing data would have to be extremely unbalanced (e.g., only the healthiest people dropped out). Since that seems unlikely, they concluded that while the drug might help, the evidence isn't as strong as the simple numbers suggested.

Why This Matters

Most studies give you a single number and a "confidence interval" that assumes everything is perfect. This paper says: "Life isn't perfect. Missing data is messy."

By using bounds and sensitivity knobs, researchers can:

Stop pretending they know more than they do.
Show exactly how much their conclusions depend on their assumptions.
Give doctors and patients a clearer, more honest picture of risk, even when the data is incomplete.

In short: Instead of guessing the exact answer in the dark, this paper builds a flashlight that shows you the walls of the room, so you know exactly how big the space is, even if you can't see the center.

1. Problem Statement

In both experimental and observational studies, outcome data are frequently missing. Standard causal inference methods typically rely on the Missing At Random (MAR) assumption, which posits that missingness depends only on observed covariates. However, in practice, missingness is often Missing Not At Random (MNAR) (or informative), where the probability of missingness depends on the unobserved outcome itself (e.g., patients dropping out due to severe side effects or death).

The specific challenge addressed in this paper is a scenario where the researcher cannot distinguish between two types of missingness mechanisms:

Non-informative (Administrative) Missingness ($UNI$): Missingness independent of the outcome (e.g., moving away, study end).
Informative Missingness ($UI$): Missingness dependent on the outcome (e.g., death, severe adverse events).

Crucially, the researcher observes a composite missingness indicator $C$ but does not know which specific mechanism caused the missingness for any given unit. This "unknown mixture" prevents standard point identification of the Average Treatment Effect (ATE) because the unobserved outcomes for the informative missing group could theoretically be anything.

2. Methodology

2.1 Framework and Assumptions

The authors propose a framework where the missingness indicator $C$ is a mixture of $UNI$ and $UI$.

Target Estimand: The Average Treatment Effect (ATE), $\Psi_0 = E[Y(1) - Y(0)]$ .
Key Assumptions:
- Ignorability: Treatment assignment is independent of potential outcomes and missingness indicators given covariates $X$ .
- Consistency: Observed data equals potential outcomes under the assigned treatment.
- Positivity: Non-zero probability of treatment and non-missingness.
- Non-informative Component: $Y \perp\!\!\perp UNI \mid X, A, UI=0$ .
- No restrictions on $UI$: The informative component can be arbitrarily dependent on $Y$ .

2.2 Bounding Strategy

Since the ATE is not point-identifiable, the authors derive sharp bounds (upper and lower limits) for the ATE under various levels of assumption:

General Bounds (No additional assumptions):
Using only the observed data and the fact that outcomes are bounded (e.g., binary), they derive the widest possible bounds. These bounds are often too wide to be practically useful but serve as a baseline.
Narrower Bounds via Sensitivity Parameters:
To tighten the bounds, the authors introduce sensitivity parameters that researchers can specify based on domain knowledge or data:
- Proportion of Informative Missingness ( $\delta$ ): The fraction of missing data that is informative ( $\pi^*_a / \pi_a$ ). Assumptions range from knowing this proportion exactly to bounding it within an interval.
- Outcome Risk Ratio ( $\tau$ ): The ratio of the expected outcome in the informative missing group versus the observed group ( $\mu^*_a / \mu_a$ ). This captures how much "worse" (or better) the missing outcomes might be.
- Monotonicity: Assumptions that informative missingness is associated with higher (or lower) risks (e.g., sicker patients are more likely to drop out).
Alternative Estimands for Competing Events:
When informative missingness represents a competing event (e.g., death preventing the measurement of diabetes), the standard ATE is ill-defined. The authors propose bounds for:
- Composite ATE ( $\Psi_1$ ): The effect on a composite outcome (Death OR Disease).
- Separable Direct Effect ( $\Psi_2$ ): The effect of the treatment on the disease pathway, holding the death pathway constant (based on Stensrud et al.'s separable effects framework).

2.3 Estimation and Inference

The authors develop doubly-robust, influence-function-based estimators for these bounds.

Machine Learning Integration: The estimators allow for the use of flexible non-parametric and machine learning methods (e.g., Random Forests, SuperLearner) to estimate nuisance parameters (propensity scores, missingness probabilities, outcome models).
Convergence Rates: Under mild conditions (specifically, if nuisance estimators converge faster than $n^{-1/4}$ ), the proposed estimators achieve root- $n$ consistency and asymptotic normality.
Uniform Inference: The paper provides methods to construct simultaneous confidence bands over a grid of sensitivity parameters, allowing for "tipping point" analysis (identifying the parameter values where the causal effect changes sign).

3. Key Contributions

Novel Mixture Framework: The paper formalizes a setting where the researcher knows missingness is a mixture of informative and non-informative types but cannot observe the specific type for each unit. This is more realistic than assuming all missingness is either fully informative or fully non-informative.
Sharp Bounds with Sensitivity Analysis: It provides a comprehensive set of sharp bounds for the ATE, Composite ATE, and Separable Direct Effect, parameterized by interpretable sensitivity parameters ( $\delta$ and $\tau$ ).
Flexible Estimation: The development of influence-function-based estimators enables the use of modern machine learning techniques while maintaining valid statistical inference (asymptotic normality and root- $n$ convergence).
Practical Tools: The authors provide an R package (marbounds) and demonstrate the methodology through simulations and a real-world application.

4. Results

4.1 Simulation Studies

The authors conducted simulations with $N=2,000,000$ (population) and samples of $n=1,000$ .
Performance: When nuisance parameters were estimated with sufficient convergence rates (simulated $\alpha=0.3$ ), the estimators were unbiased, and 95% confidence intervals achieved nominal coverage.
Robustness: When convergence rates were too slow ( $\alpha=0.1$ ), bias increased and coverage dropped, confirming the theoretical necessity of fast-converging nuisance estimators.
Machine Learning: Using SuperLearner (stacking GLM and Random Forest) yielded results comparable to the ideal simulation settings, demonstrating practical utility.

4.2 Real-World Application

Study: Causal effect of five antipsychotic drugs on Type 2 Diabetes risk and psychiatric hospitalization in elderly Medicare Advantage patients ( $N=11,618$ ).
Naive Analysis: Assuming all missingness was non-informative, Haloperidol and Quetiapine appeared to significantly reduce diabetes risk and hospitalization.
Bounding Analysis:
- The general bounds were extremely wide, ruling out no effect in either direction.
- Sensitivity Analysis: Under the assumption that missingness is monotonic (sicker patients drop out), the bounds narrowed.
- Tipping Points: For Haloperidol, the finding of reduced diabetes risk remained robust unless the proportion of informative missingness was very high (>40%) or the risk among missing patients was extremely elevated (factor of >10).
- Conclusion: The evidence for Haloperidol reducing diabetes risk is robust to reasonable levels of informative missingness, whereas results for other drugs were less robust.

5. Significance and Implications

Bridging the Gap: This work bridges the gap between strict point identification (often unrealistic) and overly conservative non-parametric bounds. It offers a middle ground where researchers can quantify uncertainty using subject-matter expertise.
Handling Competing Risks: By extending the framework to Composite ATE and Separable Direct Effects, the paper addresses a critical limitation in survival analysis and causal inference where death or other competing events obscure the primary outcome.
Policy and Clinical Relevance: The application to antipsychotic drugs demonstrates how these methods can guide clinical decision-making. It shows that while naive analyses might suggest strong benefits, rigorous sensitivity analysis is required to confirm that these benefits are not artifacts of selective dropout (informative missingness).
Future Directions: The authors note that extending this to longitudinal settings with time-varying missingness and jointly modeling unobserved confounding and informative missingness are promising avenues for future research.

In summary, the paper provides a rigorous, flexible, and computationally feasible toolkit for causal inference in the presence of unknown informative missingness, moving beyond the "all or nothing" assumptions of traditional sensitivity analysis.