📊 statistics

Generative Synthetic Data for Causal Inference: Pitfalls, Remedies, and Opportunities

This paper demonstrates that fully generative synthetic data models often fail to preserve causal estimands like the Average Treatment Effect (ATE) and proposes a hybrid framework that separates covariate, treatment, and outcome generation to ensure causal fidelity and provide robust diagnostics for causal inference.

Original authors: Yichen Xu

Published 2026-04-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yichen Xu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a chef trying to create a "synthetic" version of a famous secret sauce. You want this fake sauce to taste almost exactly like the original, look the same, and even be able to be used in new recipes.

Most people currently use AI (like GANs or LLMs) to make this sauce. They tell the AI: "Look at this real sauce and make something that looks and tastes just like it."

The problem is that this paper reveals a massive hidden trap: Just because the fake sauce looks and tastes right doesn't mean it will react the same way when you add a new ingredient.

Here is the breakdown of the paper using that analogy.

1. The Problem: The "Flavor Illusion" (Predictive vs. Causal Fidelity)

In science, we often want to know: "If I add salt (the Treatment), how much will the flavor change (the Outcome)?" This is called the Average Treatment Effect (ATE).

Current AI models are like chefs who are obsessed with the appearance of the sauce. They focus so much on getting the color, the thickness, and the smell right (this is "predictive fidelity") that they completely forget to model how the sauce reacts to salt.

The AI might create a synthetic sauce that looks perfect, but when you add salt, it does nothing—or it explodes! In technical terms, the AI reproduces the "covariates" (the ingredients) but fails to preserve the "causal contrast" (the reaction to the treatment).

2. The Solution: The "Hybrid Kitchen" (Hybrid Generation)

The author suggests we stop asking one single AI to do everything. Instead, we should use a Hybrid Approach.

Instead of one chef making the whole sauce, we split the job:

Chef A (The Artist): Their only job is to study the ingredients (the covariates) and make sure the synthetic version has the right texture and color.
Chef B (The Scientist): Their job is to study exactly how the real ingredients react to the salt (the treatment and outcome).

By separating these roles, we ensure that the synthetic data doesn't just look real, but actually behaves real when we test it.

3. The "Safety Net": Fixing the "Missing Ingredient" Problem (Positivity)

Sometimes, in real life, we have a problem where we’ve never seen a specific combination—like a recipe that calls for "blue cheese and chocolate," but no one has ever actually tried it. In statistics, this is a "positivity" problem. It makes our math very shaky and unstable.

The paper proposes a way to use synthetic data as a "Safety Net." We can use the AI to "hallucinate" plausible versions of those rare combinations. It’s like a chef saying, "I've never seen blue cheese and chocolate together, but based on what I know about both, I can simulate a version that is scientifically plausible." This helps stabilize our estimates so we don't make wild, incorrect guesses.

4. The "Flight Simulator": Testing Before the Real Thing (Simulation Engine)

Finally, the paper introduces a "Flight Simulator" for scientists.

Before a scientist performs a massive, expensive, real-world experiment, they can use this "Hybrid Engine" to run thousands of "fake" experiments. Because this engine is built using the "Hybrid Kitchen" method, the simulations are realistic.

It allows the scientist to ask: "If I use Method A, will I get a biased result? If I use Method B, will my results be too shaky?" It’s a way to practice and pick the best tools before the real mission begins.

Summary in a Nutshell

The Pitfall: Current AI makes "fake data" that looks real but breaks when you try to use it for cause-and-effect science.
The Remedy: Don't let one AI do everything. Separate the "look" of the data from the "logic" of the data.
The Opportunity: Use this smart synthetic data to fill in the gaps where real data is missing and to build a "practice arena" to test scientific methods before they are used in the real world.

Technical Summary: Generative Synthetic Data for Causal Inference

1. The Problem: The "Causal Fidelity" Gap

The paper identifies a critical misalignment between standard generative modeling objectives and the requirements of causal inference. Current state-of-the-art tabular synthesizers (e.g., GANs, LLMs) are typically optimized for predictive fidelity (Train-on-Synthetic-Test-on-Real, or TSTR) or distributional similarity (reproducing the marginal distributions of covariates).

However, causal inference—specifically estimating the Average Treatment Effect (ATE)—depends on the conditional contrast $\Delta Q(w) = Q(1, w) - Q(0, w)$ and the covariate law $P_W$ . The author formalizes a "pitfall" where a model can achieve high TSTR scores and low privacy-distance metrics while still substantially distorting the treatment-effect contrast. This occurs because, in high-dimensional tabular data, the outcome $Y$ represents only a tiny fraction of the total reconstruction loss, leading the model to prioritize covariate reconstruction over the subtle mechanisms that determine causal effects.

2. Methodology: The Hybrid Generation Framework

To remedy this, the author proposes moving away from "fully generative" models (which attempt to model the joint distribution $P(W, A, Y)$ in one pass) toward a hybrid synthetic-data framework.

Separation of Concerns: Instead of a single joint generator, the framework decomposes the process:
1. Covariate Synthesis: A generative model (LLM or GAN) is trained solely to produce the covariate distribution $\tilde{W}$ .
2. Nuisance Modeling: Separate, high-precision models for the propensity score $\hat{g}(A|W)$ and the outcome mechanism $\hat{Q}(A, W)$ are trained on the original seed data.
3. Triplet Construction: For each synthetic covariate $\tilde{W}_i$ , the treatment $\tilde{A}_i$ is sampled from the propensity model, and the outcome $\tilde{Y}_i$ is generated via the outcome model.
Targeted Synthetic Augmentation: To address positivity violations (where certain treatment-covariate combinations are rare), the paper introduces a strategy of "pairing." It identifies extreme-propensity observations in the real data and pairs them with nearby synthetic covariates to improve balance and stabilize estimation in sparse regions of the covariate space.
Synthetic Simulation Engine: The framework is used to build a "pre-analysis" engine. By learning the data-generating environment, researchers can perform repeated finite-sample simulations to compare the bias, variance, and RMSE of different estimators (OR, IPW, AIPW, TMLE) before applying them to the real dataset.

3. Key Contributions

Theoretical Formalization: Provides sensitivity bounds and tradeoff theorems (Proposition 3.1 and Theorem 3.1) that mathematically prove why joint reconstruction loss leads to diluted outcome modeling and ATE error.
Hybrid Framework: A practical algorithmic approach to generating causal-valid synthetic data.
Augmentation Strategy: A method to use synthetic data to mitigate practical positivity/overlap problems without purely relying on extrapolation.
Diagnostic Engine: A novel application of synthetic data as a tool for estimator selection and stress-testing.

4. Key Results

Causal Fidelity: Experiments show that hybrid models reduce ATE Mean Squared Error (MSE) by an order of magnitude compared to fully generative models. For example, in LLM-based settings, TMLE MSE dropped from 0.0108 to 0.0005.
Positivity Mitigation: Targeted pairing (Hybrid LLM/GAN) significantly reduced MSE for IPW and AIPW estimators in regimes with limited overlap, outperforming "self-supervised" methods that failed to incorporate additional outcome information.
Estimator Benchmarking: In the ACTG175 (real-world HIV trial) dataset, the LLM-based hybrid simulator successfully reproduced the finite-sample behavior (bias, variance, and RMSE) of real-world estimators, whereas GAN-based simulators were less faithful and often produced incorrect bias signs.
TSTR vs. Causal Utility: The results confirm that high TSTR AUC (predictive utility) is not a sufficient proxy for causal validity.

5. Significance

This paper shifts the paradigm of synthetic data from a tool for privacy and prediction to a tool for causal discovery and robust estimation. It provides a roadmap for using Generative AI (specifically LLMs) in scientific workflows, suggesting that the value of synthetic data lies not in its ability to "mimic" a table, but in its ability to "simulate" a causal mechanism. This has profound implications for researchers working with sensitive or sparse observational data who need to validate their statistical methods before conducting high-stakes analyses.