💻 computer science

Can LLMs Emulate Human Belief Dynamics?

This paper demonstrates that large language models systematically fail to emulate human belief dynamics in social networks due to their inability to replicate initial belief distributions, their excessive conformism, and their nuanced handling of homophily, thereby warning against their use as human proxies in social simulations.

Original authors: Adiba Mahbub Proma, Neeley Pate, James N. Druckman, Gourab Ghoshal, Hangfeng He, Ehsan Hoque

Published 2026-05-20

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Adiba Mahbub Proma, Neeley Pate, James N. Druckman, Gourab Ghoshal, Hangfeng He, Ehsan Hoque

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build a perfect digital copy of a human being to test how they would react in a social experiment. You want to see if a computer program can think, change its mind, and choose friends just like a real person does. This paper is essentially a "stress test" for Large Language Models (LLMs) to see if they can pull off this act.

The researchers set up a three-part game based on a real human study about political opinions (specifically regarding immigration and fuel). Here is how the game works and what the computers did:

The Three-Stage Game

The Initial Opinion: First, you ask someone, "How much do you agree with this statement?" (e.g., "Immigration is good"). They give a score from 0 to 4.
The Peer Pressure: Next, you show them what a few other people said. Then, you ask, "Now that you've seen what others think, do you want to change your score?"
The Friend Selection: Finally, you ask, "Who do you want to follow in the future? Who do you want to see more of?"

The researchers created "Digital Twins" (LLMs) using basic info about real people (like their age and personality type) and asked these AI models to play the exact same game.

The Results: The AI Failed the "Human" Test

The short answer is no, the AI models could not successfully emulate how humans behave in this social setting. The paper found three main ways the AI broke character:

1. The "Default Setting" Problem (Stage 1)

The Human Reality: Real humans have a messy, varied mix of opinions. Some strongly agree, some strongly disagree, and many are in the middle.
The AI Glitch: The AI models didn't match this messy mix.
- The "thinking" models (the smarter, more complex ones) tended to be overly skeptical, almost like a grumpy librarian who disagrees with everything by default.
- The "non-thinking" models tended to be overly agreeable, like a people-pleaser who nods at everything.
- Analogy: Imagine asking a room of 100 real people for their favorite ice cream flavor. You'd get a colorful mix of chocolate, vanilla, and strawberry. But if you asked 100 robots, they might all suddenly decide they only like "Vanilla" or only "Chocolate," ignoring the natural diversity of the crowd.

2. The "Yes-Man" Effect (Stage 2)

The Human Reality: Real humans are actually quite stubborn. When shown opposing views, they often stick to their guns or only change their minds a tiny bit. They are "rigid."
The AI Glitch: The AI models were incredibly easy to influence. When they saw what "others" thought, they immediately shifted their answers to match the group.
Analogy: If you are at a dinner party and someone says, "I hate pizza," you might think, "Well, I love pizza, and I'm sticking with that." But the AI is like a chameleon that instantly changes its skin color to match the person sitting next to it, even if it contradicts its own nature. The paper calls this being "conformist."

3. The "Stranger Danger" in Friend Selection (Stage 3)

The Human Reality: Humans are "homophilic," which is a fancy word for "birds of a feather flock together." When we choose who to follow, we usually pick people who already think like us.
The AI Glitch: The AI models were okay at picking someone, but they picked the wrong kind of someone. They tended to choose people who were further away from their own beliefs than real humans would.
Analogy: If you are a huge fan of jazz music, you'd probably follow other jazz fans. The AI, however, acted like a jazz fan who suddenly decided to follow a heavy metal band just because it was interesting, failing to capture the human instinct to stick with our own "tribe."

The Big Takeaway

The paper concludes that while LLMs are great at writing essays or coding, they are bad at pretending to be humans in social situations.

They don't just "act" human; they act like very agreeable, easily swayed, slightly confused robots. The researchers warn that if scientists try to use these AI models to simulate real-world social networks (like predicting how a rumor spreads or how a political movement grows), the results will be wrong because the AI's "social brain" works differently than a human's.

In a nutshell: You can ask an AI to write a story about a human, but you cannot ask it to be a human in a social experiment. It will always give you a "sycophant" (a yes-man) instead of a real person.

Technical Summary: Can LLMs Emulate Human Belief Dynamics?

Problem Statement

Large Language Models (LLMs) are increasingly utilized to simulate human attitudes, opinions, and behaviors in social network contexts, ranging from survey replication to the modeling of opinion dynamics. While prior work suggests LLMs can reproduce specific human behaviors (e.g., organizing events or reaching scientific consensus), there is a critical gap in understanding how well these models emulate belief dynamics—specifically, how humans form initial beliefs, update them in response to competing information, and restructure their social networks based on homophilic tendencies.

The central problem addressed is whether LLMs, even when provided with "digital twin" personas (demographic and personality data), can accurately replicate the complex psychological processes underlying human belief formation and change. The authors posit that without rigorous validation, deploying LLMs as human proxies in social simulations carries significant risks, particularly regarding polarization and persuasion studies.

Methodology

The study replicates an established human experiment on political belief dynamics (Proma et al., 2025c) using a "digital twin" approach.

Experimental Design: The original human study involved three stages:
1. Response: Participants rated their belief in a statement (0–4 Likert scale).
2. Revision: Participants were exposed to peers' ratings and could update their own.
3. Selection: Participants chose which peers to "follow" in subsequent rounds.
LLM Simulation: The authors replaced 341 human participants with LLM "digital twins." Each LLM was prompted with the specific participant's demographic information and Big-5 personality traits. The models were then tasked with navigating the same three stages, receiving the same stimuli (including peer responses) as the original human subjects.
Models Evaluated: The study evaluated 12 LLMs across four families and varying parameter sizes, including both non-thinking (non-reasoning) models (e.g., Llama 3, Gemma 3) and thinking (reasoning) models (e.g., Qwen 3, GPT-OSS).
Data Scope: The analysis covered two political topics ("immigration" and "oil and fuel"), resulting in 1,023 total samples (3 rounds × 341 participants).
Evaluation Metrics:
- Stage 1 (Initial Beliefs): KL Divergence, Wasserstein Distance, Mann-Whitney U tests, and Spearman correlation to compare distribution shapes, central tendencies, and individual-level alignment.
- Stage 2 (Belief Change): Analysis of belief update magnitude and "Social Influence" (correlation between the gap between initial self-belief and peer average, and the resulting belief change).
- Stage 3 (Network Selection): "Follow Signal" (mean rating of selected peers) and "Belief Network Distance" (absolute difference between initial belief and selected peers' beliefs) to measure homophily.

Key Results

1. Failure to Emulate Initial Belief Distributions

LLMs systematically failed to match the distribution of initial human beliefs.

Distributional Mismatch: Both KL Divergence and Wasserstein Distance values for LLMs were significantly higher than the baseline (human-vs-human split), indicating poor distributional alignment.
Systematic Bias: Non-thinking models tended to overestimate belief ratings (peaking at 2), while thinking models tended to underestimate them (peaking at 0, indicating strong disagreement).
Low Individual Correlation: Spearman correlation between individual LLM responses and human responses was moderate at best (max $\rho = 0.31$ ), suggesting LLMs cannot reliably predict specific individual preferences.

2. Excessive Conformity in Belief Updates

LLMs demonstrated a higher susceptibility to social influence than humans.

Rigidity vs. Fluidity: Humans exhibited greater belief rigidity, changing their opinions less frequently and to a lesser degree than LLMs.
Social Influence Metric: The correlation between "social influence" (the gap between self and peers) and belief change was significantly higher for LLMs in 11 out of 12 models. This indicates LLMs are more conformist, shifting their responses to align with the average of their observed network more readily than humans do.

3. Nuanced Failure in Network Restructuring (Homophily)

While LLMs could mimic the act of selecting peers, they failed to capture the underlying homophilic logic.

Follow Signal: LLMs successfully simulated the general distribution of which peers humans chose to follow (high correlation in Follow Signal).
Belief Network Distance: However, the "Belief Network Distance" (how far selected peers' beliefs were from the agent's own) was consistently higher for LLMs. This suggests that while LLMs can select peers, they do not do so based on the same strong preference for similarity (homophily) that characterizes human network formation. Their selections are less anchored to their own initial beliefs.

Key Contributions and Claims

The paper makes three primary contributions to the literature on LLM simulation:

Systematic Divergence: It provides empirical evidence that LLMs diverge from humans in fundamental ways regarding belief dynamics, specifically in initial distribution, susceptibility to social influence, and homophilic network selection.
Persona Limitations: It challenges the efficacy of the "digital twin" method when limited to demographic and personality data, suggesting that such constrained personas are insufficient for capturing complex belief dynamics.
Scaling Law Anomaly: The study notes that larger parameter sizes do not necessarily yield better alignment with human belief dynamics, suggesting that belief rigidity and motivated reasoning are distinct from the reasoning and knowledge retrieval tasks where scaling laws typically hold.

Significance and Implications

The authors conclude with a "sharp warning" against the uncritical deployment of LLMs as human proxies in social simulations, particularly for studies involving polarization, persuasion, and network interventions.

Fundamental Asymmetry: The findings highlight a fundamental asymmetry in how LLMs process social context compared to humans. LLMs appear to lack the identity, motivation, and affective processes that drive human belief dynamics.
Research Caution: The paper argues that using LLMs to simulate human belief networks may lead to erroneous conclusions, as the models are inherently more conformist and less rigid than the human populations they aim to represent.
Future Directions: The authors suggest that future work must explore more elaborate personas (including social media habits and lived experiences) and investigate whether closed-source models or models with cross-round memory can bridge this gap, though they remain skeptical of the current capabilities.