📊 statistics

Stress-Testing Assumptions: A Guide to Bayesian Sensitivity Analyses in Causal Inference

This paper provides a practical, unified guide to implementing Bayesian sensitivity analyses for causal inference—covering exposure misclassification, unmeasured confounding, and missing data using both parametric and nonparametric models via Stan—to help researchers overcome methodological barriers and encode subjective beliefs about assumption violations.

Original authors: Arman Oganisian

Published 2026-03-02

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Arman Oganisian

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery: Does a new medicine actually cure a disease, or did the patients just happen to get better for other reasons?

In the ideal world, you'd run a perfect experiment where you flip a coin to decide who gets the medicine and who gets a placebo. But in the real world, we often have to rely on observational data—like looking at old medical records where doctors decided who got the medicine based on their own judgment.

The problem? These records are messy. They might have missing pages, typos, or hidden factors we didn't write down. To make sense of this, statisticians use a set of rules (assumptions) to clean up the data. But here's the catch: we can never prove these rules are 100% true.

This paper is a guide on how to "stress-test" those rules. It's like putting a bridge through a wind tunnel to see if it holds up when the wind blows harder than expected.

Here is the breakdown of the paper's ideas using simple analogies:

1. The Core Idea: The "Missing Piece" Puzzle

The author, Arman Oganisian, argues that the best way to handle these messy rules is using Bayesian Statistics.

Think of a Bayesian approach like a detective with a notebook.

Traditional methods are like a detective who says, "I found the evidence, and here is my final verdict."
Bayesian methods are like a detective who says, "I have some evidence, but I also have some hunches about what might be missing. Let's write down my hunches, combine them with the evidence, and see how my final verdict changes."

In this paper, the "hunches" are about the assumptions we make. What if the data is wrong? What if we missed a key clue? The Bayesian method lets us treat these "what ifs" as actual variables in our math, rather than ignoring them.

2. The Four "What If" Scenarios

The paper walks through four specific ways data can be "broken" and shows how to fix the math to see if our conclusion still holds.

Scenario A: The Blurry Camera (Exposure Misclassification)

The Problem: Imagine you are trying to count how many people wore a seatbelt, but your camera is blurry. You think you see a seatbelt, but maybe it's just a shadow. You aren't sure if the person actually wore one.
The Stress Test: Instead of pretending your camera is perfect, you say, "Okay, let's assume my camera is only 80% accurate." You run the math with that assumption. Then you try 90%, then 95%.
The Result: If your conclusion ("Seatbelts save lives") stays the same even when you assume the camera is terrible, you can be confident. If the conclusion flips when you assume the camera is bad, you know your result is shaky.

Scenario B: The Invisible Thief (Unmeasured Confounding)

The Problem: Imagine you see that people who drink coffee live longer. But maybe it's not the coffee; maybe coffee drinkers also happen to be wealthier and eat better food. That "wealth" is an invisible thief stealing the credit for the coffee.
The Stress Test: You can't see the "wealth" variable in your data. So, you invent a "ghost variable" in your math. You ask: "How strong would this invisible ghost have to be to make the coffee look like a miracle cure when it's actually useless?"
The Result: The paper shows you can calculate exactly how much "ghost power" it would take to ruin your conclusion. If it takes a giant, mythical ghost to change the result, your coffee theory is probably safe.

Scenario C: The Vanishing Act (Missing Data)

The Problem: You are tracking patients for 3 years to see if they recover. But 50% of them quit the study halfway through. You don't know if they got better or worse.
The Twist: Usually, we assume the people who quit are just like the people who stayed (Missing At Random). But what if the people who quit were the ones who got sicker and gave up?
The Stress Test: The paper teaches you to model the "quit rate" as a variable. You can say, "Let's assume the sick people were twice as likely to quit." You then re-run the analysis. If the result changes drastically, you know your original conclusion was too optimistic.

Scenario D: The Shape-Shifter (Non-Parametric Models)

The Problem: Sometimes the rules of the world are so complex that simple math formulas (like straight lines) can't describe them.
The Solution: The paper introduces a "Shape-Shifter" model (called a Truncated Stick-Breaking mixture). Imagine you are trying to draw a curve through a cloud of dots. A simple line is too rigid. This method uses a flexible, stretchy rubber band that can bend and twist to fit the dots perfectly, while still respecting your "hunches" about the missing parts.

3. The Secret Weapon: Stan

The paper isn't just theory; it's a "how-to" manual. It uses a free software tool called Stan.

Think of Stan as a universal translator.

You write down your math and your "hunches" in a language that looks like a recipe.
Stan takes that recipe, does millions of calculations in the background (using a super-smart engine called Hamiltonian Monte Carlo), and gives you a report.
The paper provides the actual "recipes" (code) so that anyone can copy-paste them and start stress-testing their own data immediately.

The Big Takeaway

The main message of the paper is: Don't just trust your assumptions; poke them.

In the past, scientists would build a house of cards on a shaky foundation and hope it didn't fall. This paper gives you a fan to blow on the cards.

If the house stands, you have a strong result.
If the house falls, you know exactly which assumption was weak, and you can fix it or admit you don't know the answer yet.

It turns the scary, abstract world of "what if I'm wrong?" into a concrete, solvable math problem. It empowers researchers to say, "I've checked the weak spots, and here is how confident I really am."

Technical Summary: Stress-Testing Assumptions in Bayesian Causal Inference

1. Problem Statement

Observational data is frequently used to estimate causal effects of biomedical treatments, but such analyses rely on untestable assumptions regarding causal identification (e.g., unconfoundedness) and data observation mechanisms (e.g., no measurement error, missingness at random). When these assumptions are violated, standard causal estimates can be biased.

While sensitivity analysis methods exist to characterize how violations affect inferences, their adoption in practice is low due to:

Methodological Complexity: Implementation often requires custom algorithms rather than off-the-shelf software.
Lack of Guidance: Few resources provide practical, code-based guidance on implementing these analyses.
Software Barriers: While probabilistic programming languages (PPLs) like Stan exist, applying them to causal sensitivity analysis is not straightforward.

The paper addresses the gap between theoretical Bayesian sensitivity analysis and practical implementation, specifically focusing on how to "stress-test" assumptions using a unified missing-data framework.

2. Methodology

The authors propose a unified Bayesian approach where missing data (whether due to misclassification, unmeasured confounders, or non-random missingness) and sensitivity parameters are treated as additional unknown random variables. Instead of fixing assumptions, the analyst encodes beliefs about the direction and magnitude of assumption violations via prior distributions and samples from the joint posterior.

Core Framework

The methodology relies on the principle that in Bayesian inference, parameters and data are both random variables. The inference proceeds by sampling from the joint posterior:
$f(D_M, \xi, \eta, \gamma, \theta \mid D_O)$
Where:

$D_O$ : Observed data.
$D_M$ : Missing data (e.g., true exposure status, unmeasured confounders, missing outcomes).
$\xi$ : Sensitivity parameters encoding the violation (e.g., sensitivity/specificity of exposure, strength of unmeasured confounding).
$\eta, \gamma, \theta$ : Parameters for outcome, treatment, and covariate models.
Implementation: The authors utilize Stan (a PPL using Hamiltonian Monte Carlo) to sample from these complex joint posteriors.

Four Illustrative Examples

The paper demonstrates this framework through four specific scenarios, moving from parametric to nonparametric models:

Example 1: Exposure Misclassification

Scenario: The true treatment status $A$ is unobserved; only an error-prone indicator $\tilde{A}$ is available.
Approach: Treat the true $A$ as missing data. The misclassification mechanism is modeled via sensitivity parameters $\xi$ (Sensitivity and Specificity).
Technical Challenge: Since $A$ is discrete, it cannot be directly sampled in Stan's HMC.
Solution: The authors marginalize out the discrete unknowns analytically to create a continuous joint posterior for the sensitivity parameters and model parameters. This involves a mixture likelihood term computed using log_sum_exp for numerical stability.

Example 2: Unmeasured Confounding

Scenario: A confounder $U$ affects both treatment $A$ and outcome $Y$ but is unobserved.
Approach: Treat $U$ as missing continuous data. The dependence of $A$ and $Y$ on $U$ is governed by sensitivity parameters $\xi_1$ (effect on outcome) and $\xi_2$ (effect on treatment).
Implementation: Unlike Example 1, $U$ is continuous, so it is declared directly in the parameters block of Stan. The model samples $U$ alongside other parameters, allowing for "tipping point" analysis to determine how strong the unmeasured confounding must be to overturn conclusions.

Example 3: Missing Not-at-Random (MNAR) Outcomes

Scenario: Outcomes $Y$ are missing for some subjects, and the missingness depends on the unobserved outcome itself.
Approach: Model the missingness mechanism $P(\Delta=1 \mid Y, A, L)$ using sensitivity parameters (e.g., $\xi_3$ representing the interaction between treatment and outcome on missingness).
Technical Challenge: Similar to Example 1, the missing binary outcomes are discrete.
Solution: The missing outcomes are marginalized out, resulting in a mixture likelihood for the observed data that accounts for the MNAR mechanism.

Example 4: Bayesian Nonparametric Sensitivity Analysis

Scenario: Extending the MNAR framework to flexible, high-dimensional models.
Approach: Uses Truncated Stick-Breaking (TSB) Mixture Models. The joint distribution of $(Y, A, L)$ is modeled as a finite mixture of parametric components.
Advantage: This allows the model to be data-adaptive in regions with observed data while relying on prior beliefs (encoded via sensitivity parameters) in regions with missing data. It separates identifiable components (flexible nonparametric modeling) from non-identifiable components (parametric missingness mechanism).

3. Key Contributions

Unified "Missing Data" Framework: The paper demonstrates that diverse causal violations (misclassification, unmeasured confounding, MNAR) can all be conceptualized and solved as missing data problems within a single Bayesian framework.
Practical Implementation Guide: It provides the first comprehensive guide combining mathematical derivation, conceptual intuition, and executable Stan code for these analyses.
Demystifying Stan for Causal Inference: The authors show how to handle non-standard posterior densities (e.g., mixtures, discrete marginalization) using Stan's target incrementation and log_sum_exp functions, moving beyond standard textbook examples.
Tipping Point Analysis: The methodology facilitates "tipping point" analyses, identifying the specific magnitude of assumption violations required to change the sign or significance of the causal effect estimate.

4. Results

Synthetic Data Validation: The authors validated the methods using synthetic datasets.
- In the Exposure Misclassification example, they showed that posterior credible intervals for the Average Treatment Effect (ATE) shift significantly as sensitivity/specificity deviate from 100%.
- In the Unmeasured Confounding example, they demonstrated that a relatively small departure from the "no unmeasured confounding" assumption (e.g., log-odds ratios of $\approx 1$ ) could shift the posterior probability of a treatment effect from significant to null.
- In the MNAR example, they illustrated how prior beliefs about the missingness mechanism (e.g., $\xi_3 = -1$ ) directly alter the imputed values of missing outcomes and the resulting regression lines.
Nonparametric Flexibility: The TSB model (Example 4) successfully captured complex non-linear relationships in observed data while appropriately incorporating prior beliefs about missing data mechanisms, visualized through posterior regression lines that "dip" or shift based on the MNAR prior.

5. Significance

Bridging Theory and Practice: This paper is a critical resource for practitioners, proving that sophisticated Bayesian sensitivity analyses are feasible using publicly available, open-source software (Stan) without requiring custom MCMC code.
Enhancing Scientific Rigor: By providing tools to quantify the "scientific quality" of assumptions, the paper encourages researchers to move beyond simply stating assumptions to actively testing their robustness.
Domain-Specific Expertise: The authors emphasize that sensitivity analysis is an "art" requiring domain knowledge to specify realistic sensitivity parameters (e.g., plausible ranges for misclassification rates or confounding strength), preventing the analysis from becoming a purely mathematical exercise.
Future Direction: The unified approach suggests that future causal inference work should integrate sensitivity analysis as a standard step, rather than an optional add-on, to ensure the reliability of observational findings.

Code Availability: All code referenced in the paper is available in a companion GitHub repository, facilitating immediate replication and extension by the research community.