The Epi-LLM Framework: probing LLM behavioral priors through epidemiological agent-based models

Technical Summary: The Epi-LLM Framework

Problem Statement
Human behavior is a critical determinant of infectious disease dynamics, yet quantifying the impact of voluntary behavioral changes—such as quarantine, social distancing, and vaccination uptake—remains a significant challenge in epidemiological modeling. Traditional compartmental models (e.g., SIR, SEIR) rely on simplifying assumptions that fail to capture the complex, adaptive interplay between public behavior and pathogen transmission. While Agent-Based Models (ABMs) have improved this by simulating individual-level heterogeneity, they are often constrained by a lack of empirical behavioral data, relying instead on inferred parameters to represent decision-making. Furthermore, the integration of Large Language Models (LLMs) as the behavioral engine within ABMs, where agents reason and adapt dynamically over an outbreak, remains underexplored. Existing Generative Agent-Based Models (GABMs) have shown potential but are often limited by single-model architectures, small agent populations, and insufficient grounding in real-world behavioral data.

Methodology
The authors introduce the Epi-LLM framework, a novel integration of agent-based modeling, real-life epigames, and LLMs. The framework is built upon Starsim, an open-source platform for infectious disease dynamics, and utilizes a modified SEIR (Susceptible-Exposed-Infected-Recovered) transmission model.

• Simulation Environment: The simulation runs over a 15-day period, mirroring the duration of a specific epigame study conducted at the American University of Iraq–Baghdad (AUIB). The contact network is generated using a dynamic RandomNet module with a log-normal degree distribution to match empirical contact data from the AUIB study.
• Agent Architecture: Agents are instantiated using four distinct LLM architectures accessed via the OpenRouter API: DeepSeek V3, Llama 3 70B, Nemotron 120B, and GPT-OSS 120B. These models represent both dense transformer and Mixture-of-Experts (MoE) families across varying parameter scales.
• Behavioral Parameterization: To ground the simulation in reality, agents are parameterized using Health Belief Model (HBM) scores derived from actual survey responses of AUIB participants. These scores capture perceived susceptibility, severity, benefits, and barriers.
• Decision Mechanism: Daily, agents observe their health status, local symptomatic prevalence, and point balance. They then make a binary decision (quarantine or remain active) via an LLM prompt. The decision framework includes a trade-off: quarantine guarantees protection from infection but yields a lower reward (5 points), while remaining active offers higher rewards (10–15 points) but exposes the agent to transmission risk.
• Experimental Variations:
  • Architectural Comparison: Evaluating epidemic outcomes across the four LLM models.
  • Geographic Framing: Assigning agents geographic labels (China, Iraq, Kenya, UK) to test if these labels alone induce culturally differentiated behavior.
  • Incentive Structures: Testing an alternative scoring system where points are earned per contact to simulate the social benefits of interaction.
  • Statistical Analysis: Binomial and Poisson Generalized Linear Models (GLMs) were fitted to analyze quarantine behavior and compare it with human participant data.

Key Contributions

1. Unified Framework: This is the first work to integrate LLMs, agent-based epidemic modeling, and real-world behavioral epi-game data into a single simulation framework.
2. Empirical Grounding: Unlike prior GABM studies, this work parameterizes agent belief profiles using real psychosocial survey data from the AUIB epi-game, transforming static behavioral data into dynamic simulation inputs.
3. Architectural Analysis: The study provides a systematic comparison of four LLM architectures under identical conditions, demonstrating that model architecture is a non-trivial determinant of emergent population-level behavior.
4. Stochasticity Characterization: The authors introduce a self-agreement analysis to characterize agent stochasticity, providing a tool for assessing whether observed differences in epidemic trajectories reflect genuine behavioral priors or seed-level noise.

Results

• Epidemic Dynamics: All LLM architectures reduced peak active infections compared to a no-intervention SEIR baseline. However, the effectiveness varied by architecture. Llama 3 70B produced the most delayed and flattened epidemic peak with the lowest cumulative infection burden, while Nemotron 120B exhibited higher peak infections. Models within the same architectural family (e.g., DeepSeek V3 and GPT-OSS 120B, both MoE) showed convergent behaviors.
• Geographic Framing: Assigning geographic labels (China, Iraq, Kenya, UK) did not induce statistically significant differences in quarantine behavior or epidemic outcomes. The trajectories across all four conditions were structurally identical, with cumulative infections differing by only 11 agents. This suggests that geographic labels alone are insufficient to elicit culturally differentiated behavior in LLM agents without explicit attitudinal parameterization.
• Behavioral Drivers: A binomial GLM identified perceived health severity as the strongest predictor of quarantine behavior (β = 0.33, p = 0.002), yielding a pseudo-R² of 0.055. This is comparable to the 0.072 observed in the human trial, indicating that LLM agents successfully reproduce broad threat-driven compliance patterns.
• Incentive Structures: Modifying the reward structure (shifting from fixed rewards to contact-based rewards) substantially altered epidemic trajectories, reducing peak prevalence from ~0.5 to ~0.3 and prolonging compliance.
• Consistency: Llama 3 70B exhibited the highest self-agreement (consistency), while Nemotron 120B was the most stochastic. This implies that low-variance architectures offer greater internal validity for testing specific rules, while high-variance models may better represent the stochastic nature of real-world decision-making.

Significance and Claims
The paper claims to lay the methodological groundwork for deploying the Epi-LLM framework as a scalable, risk-free simulation environment for pandemic preparedness research. The authors assert that the framework is technically feasible and empirically grounded, capable of generating substantive insights into behavioral dynamics.

The study emphasizes that LLM architecture is a key design choice for synthetic societies: researchers must choose between low-variance models for controlled hypothesis testing and high-variance models for realistic heterogeneity. Furthermore, the null result regarding geographic framing serves as a critical finding, suggesting that meaningful cultural variation in GABMs requires explicit parameterization of attitudinal and normative factors rather than simple demographic labeling. Finally, the framework is presented as a prospective sandbox for testing epi-game mechanics and public health intervention strategies before real-world deployment.