Technical Summary: The Illusion of Intervention in LLM-Simulated Experiments

Problem Statement

Large Language Models (LLMs) are increasingly utilized to simulate human behavior for synthetic experiments, such as evaluating AI agents or conducting large-scale surveys. The prevailing assumption is that by initializing identical LLMs with the same persona (e.g., "a 30-year-old man") and exposing them to different interventions (e.g., Agent A vs. Agent B), researchers can mimic a Randomized Controlled Trial (RCT) to estimate causal effects.

However, this paper argues that such synthetic experiments suffer from user drift, rendering them more akin to observational studies than randomized trials. Because LLMs are trained on observational data and operate via abductive reasoning, they infer unspecified latent attributes ($X \setminus L$) to maintain coherence with the intervention context ($A$). Consequently, two synthetic users initialized with identical observed attributes ($L$) may drift into distinct latent populations depending on the intervention they receive. This creates selection bias (confounding), where the distribution of latent user attributes differs across treatment conditions ($P(X | A=1, L) \neq P(X | A=0, L)$), distorting the estimated treatment effect ($\tau_{obs}$) from the true causal effect ($\tau_{ATE}$).

Methodology

Formalization of Bias

The authors formalize the problem using the potential outcomes framework. In an ideal RCT, random assignment ensures that background factors are balanced. In LLM simulations, the generative process factors the joint distribution as $P(L, A, X, Y)$ rather than the causal ordering $P(X, L, A, Y)$. When an intervention $A$ is introduced, the LLM infers latent attributes $X \setminus L$ that are consistent with $A$, creating an open backdoor path from $A$ to $Y$ through unobserved user attributes. This induces a bias term $\delta_{SB}$ such that $\tau_{ATE} = \tau_{obs} - \delta_{SB}$.

Diagnostic: Negative Control Outcomes

To detect this drift without observing the full latent persona, the authors propose using negative control outcomes ($Z$). These are variables that:

1. Should be causally unaffected by the intervention $A$ (invariant under treatment).
2. Depend on the same unobserved latent attributes ($X \setminus L$) that drive the drift in the primary outcome.

Examples include demographic attributes (race, citizenship) or stable beliefs that should not change based on whether a user interacts with a "positive" or "negative" AI agent. The authors quantify drift using the Total Variation Distance (TVD) between the distributions of $Z$ under different intervention conditions ($A=0$ vs. $A=1$). A non-zero TVD indicates that the intervention has shifted the latent user population, signaling confounding.

Mitigation: Iterative Confounder Adjustment

To reduce bias, the authors propose an iterative procedure to elicit and specify additional confounders ($L'$) that proxy the unobserved latent attributes.

1. Elicitation: The system prompts the synthetic user for specific attributes (e.g., income, religion, or targeted questions related to the intervention) after the intervention.
2. Sampling: New confounders are sampled from a treatment-uniform mixture distribution to ensure the adjusted population reflects a randomized scenario.
3. Iteration: These elicited attributes are added to the persona specification ($\tilde{L} = L \cup L'$), and the experiment is re-run. This loop continues until the TVD of negative control outcomes stabilizes or drops below a threshold.

The authors emphasize that this is an iterative, targeted approach rather than a one-shot specification of all possible attributes, to avoid context window degradation and sampling bias.

Key Contributions

1. Formalization of User Drift: The paper provides a causal inference framework demonstrating that LLM-simulated experiments inherently suffer from selection bias due to intervention-dependent shifts in latent user attributes.
2. Empirical Diagnostic: It introduces the use of negative control outcomes and Total Variation Distance as a practical tool to diagnose user drift and selection bias across various LLM families.
3. Mitigation Strategy: It demonstrates that targeted, iterative elicitation of confounders during persona specification can substantially reduce selection bias and stabilize effect estimates in both survey-style and multi-turn agent evaluation settings.

Results

The authors evaluated their approach across multiple models (Qwen3, GPT-OSS, Gemma, Gemini) and datasets (OpinionQA, Book Opinions, MovieLens).

• Presence of Drift: At iteration 0 (baseline), substantial TVD was observed in negative control outcomes across most models and datasets, confirming that identical initial personas drift into different latent populations based on the intervention. Notably, models with stronger abductive reasoning capabilities (e.g., Gemma-4) exhibited more drift than smaller, less capable models.
• Effect of Adjustment: Iterative confounder adjustment generally reduced TVD, indicating a reduction in user drift.
  • In survey settings (OpinionQA), TVD decreased consistently over iterations.
  • In agent evaluation settings, results varied; some models (e.g., Gemma-4) showed clear reductions, while others (e.g., GPT-OSS-20B) showed flat or weakly changing TVD, suggesting prompt-based conditioning is not uniformly effective in multi-turn settings.
• Impact on Effect Estimates: The observed treatment effects ($\tau_{obs}$) shifted significantly during early adjustment iterations before stabilizing. This confirms that naive estimates were confounded by user drift.
• Design Nuances: The study found that adding generic demographic attributes initially could sometimes increase TVD (selection bias) before targeted, task-specific confounders reduced it. This highlights the importance of selecting confounders relevant to the specific treatment and outcome.

Significance and Claims

The paper concludes that evaluations using synthetic users should be treated as observational studies by default, not randomized experiments. The "illusion of intervention" arises because the LLM's generative nature allows the intervention to retroactively influence the simulated user's latent state.

The authors claim that:

1. Without diagnosis and mitigation, synthetic user experiments risk producing misleading causal estimates due to unobserved selection bias.
2. Negative control outcomes provide a necessary diagnostic tool to detect this bias.
3. Iterative, targeted confounder adjustment is a viable strategy to improve internal validity, though it does not guarantee the elimination of all bias, particularly if substantial latent variation remains unmeasured.

The work serves as a cautionary framework for researchers using LLMs to simulate human responses, urging a shift from assuming causal validity to actively diagnosing and correcting for user drift.