Debiasing International Attitudes: LLM Agents for Simulating US-China Perception Changes

This study introduces an LLM-agent framework to simulate U.S. citizens' attitudes toward China from 2005 to 2025, demonstrating that while subjective news framing has a modest impact on negative attitudes, a "devil's advocate" agent is the most effective mechanism for debiasing opinions and producing more human-like cognitive outcomes.

The Gist

Imagine you want to understand why neighbors in two different countries (the US and China) have been getting more suspicious of each other over the last 20 years. Instead of interviewing thousands of real people, the researchers in this paper built a massive, digital "simulation city" populated by 2,000 AI characters (agents).

Think of these AI agents as digital actors in a very long-running soap opera. Their job is to act like real American citizens, read the news, and change their opinions about China over time.

Here is the story of what they did, broken down simply:

1. Building the Cast (The Agents)

First, the researchers needed their actors to feel real. They didn't just make them up; they built them using a "Frankenstein" approach, stitching together real data from:

• Social Media: Real tweets and bios from Twitter/X.
• Surveys: Real answers from the General Social Survey (a big poll of Americans).

They mixed these to create a cast that looked like the real US population: different ages, races, political parties, and hobbies. Each agent had a "memory" of who they were.

2. The Plot: Reading the News

The simulation ran for 20 years (2005–2025). Every year, the agents were fed a diet of 100,000+ real news articles about China.

• The Setup: The agents picked headlines that matched their interests (just like you might click on a sports story if you love sports).
• The Reaction: After reading, the agents had to "reflect." They asked themselves: "Does this new info match what I already believe?"
  • If it matched, they felt good.
  • If it clashed, they had to decide: Do I change my mind? Do I double down on my old view? Or do I just ignore it?

3. The Problem: The "Hate Machine"

When the researchers ran the simulation with no help, the AI agents became extremely negative about China. They turned into a room full of angry people, much more so than real humans actually are.

• Why? The AI models (the brains behind the agents) seemed to have a built-in "negativity bias." They treated the news like a horror movie, assuming the worst about China, even when the news was neutral.

4. The Solution: Three "Debiasing" Tricks

To fix this, the researchers tried three different ways to help the agents think more clearly, like a coach trying to stop a player from making bad calls.

• Trick #1: The "Fact-Checker" (Fact Elicitation)

  • The Metaphor: Imagine a news editor who takes a sensational headline like "China's Secret Plot to Take Over!" and rewrites it to be boringly dry: "China passed a new law regarding video content."
  • The Result: It helped a little, but the agents were still pretty grumpy. Stripping away the emotion didn't fix the underlying bias.

• Trick #2: The "Devil's Advocate" (The Critical Friend)

  • The Metaphor: Imagine you read a scary story, and a skeptical friend says, "Wait, hold on. That story sounds dramatic. Let's look at the other side. Maybe there's a reason for this law that isn't evil."
  • The Result: This was the winner. When the agents were forced to hear a critical, balanced counter-argument before forming an opinion, their attitudes became much more realistic. It forced them to think, not just react.

• Trick #3: The "Role Reversal" (Counterfactual)

  • The Metaphor: They took the news about China and swapped the words so it read like news about the USA. "The US government ordered a crackdown..."
  • The Result: This didn't fix the bias; it actually revealed it.
    • The US-based AI (GPT-4o) got angry when the story was about the US (protecting its "in-group").
    • The China-based AI (Qwen) got angry when the story was about the US.
    • This proved that the AI models have a hidden "national loyalty" programmed into them, just like humans do.

5. The Big Takeaways

• Thinking Beats Reacting: The most effective way to stop AI from being biased is to make it pause and argue with itself (the Devil's Advocate method). It mimics how smart humans actually process bad news.
• AI Has a "Home Team" Bias: Even when you tell an AI to act like an American, if it was trained in China, it still secretly roots for China. It's like a sports fan who was born in one city but grew up in another; they still have a soft spot for their original team.
• News Matters: The simulation showed that Politics and Economics news made people dislike China, while Sports, Culture, and Tech news actually made them like China more.

In a Nutshell

This paper is like a scientific stress test for AI. It showed that if you just let AI read the news, it will become a paranoid, angry citizen. But if you give it a "critical thinking coach" (the Devil's Advocate), it can learn to form opinions that look a lot more like real human opinions.

It's a warning to policymakers: Don't trust AI simulations blindly. If you want to know how people will react to a new policy, you have to teach the AI how to think critically first, or it will just give you a distorted, overly negative version of reality.

Technical Summary

Here is a detailed technical summary of the paper "Debiasing International Attitudes: LLM Agents for Simulating US-China Perception Changes."

1. Problem Statement

The paper addresses the challenge of modeling long-term opinion evolution in computational social science, specifically focusing on the dynamic and often polarized attitudes of US citizens toward China. While Large Language Models (LLMs) have shown promise in simulating short-term cognitive tasks, they struggle with:

• Long-term memory: Capturing trends over decades (2005–2025) exceeds standard prompt context limits.
• Inherent Bias: LLMs often exhibit inherent biases (e.g., negative sentiment toward specific geopolitical entities) that diverge from real-world human data.
• Lack of Cognitive Realism: Standard LLM agents often lack the complex cognitive mechanisms (e.g., cognitive dissonance, critical reflection) required to simulate how humans process conflicting information over time.

The core problem is how to construct an LLM-based agent framework that accurately reproduces macro-level international attitude trends while mitigating the inherent biases of the underlying models.

2. Methodology

The authors propose a cognitive-aware LLM-agent framework consisting of four main stages:

A. Agent Initialization (Population Generation)

To create a representative sample of the US population, the authors merged two real-world datasets:

• X/Twitter Data: 3,849 profiles (bios, demographics, tweets) from 30 users per US state.
• General Social Survey (GSS): 3,309 respondents with detailed demographics, media habits, and political views.
• Process: Users were matched based on demographics (gender, race, political party, region). Content fields were concatenated and processed via an LLM (Deepseek V3) to infer interests across 15 news categories. This resulted in a pool of ~2,000 agent profiles with 50 features each.

B. Simulation Framework (2005–2025)

The simulation runs iteratively for 21 years. In each year:

1. News Distribution: A distributor module selects random headlines from a corpus of 100,000+ articles (56 publications) based on the agent's profile. Agents select articles to read based on interest, then retrieve full text.
2. Debiasing Interventions: Before reflection, articles undergo one of three debiasing mechanisms (or none for the control group):
   • Fact Elicitation (FA): An "editor" agent rewrites articles to remove subjective language, sensationalism, and bias, leaving only objective events.
   • Devil's Advocate (DA): A critical agent analyzes the article for logical fallacies, omitted context, and unsubstantiated claims, providing an alternative perspective alongside the original text.
   • Counterfactual (CF): All references to "China" are swapped with "USA" (and vice versa) to test for in-group favoritism and model-specific biases.
3. Reflection Mechanism: Agents process the news using Cognitive Dissonance Theory.
   • Agents check for contradictions between new info and existing beliefs.
   • If contradictions exist, agents choose to Revise (accept new info), Reinforce (rationalize existing belief), or Dismiss (ignore the info).
   • Opinions are updated across specific domains (e.g., Economy, Politics) with a valence score (-2 to +2).
4. Survey & Aggregation: At the end of each year, agents take a survey (0–4 scale) to rate their overall attitude toward China. These are aggregated to compare against ground truth.

C. Ground Truth

• Primary: Pew Research Institute data (2005–2025) for overall trend accuracy.
• Secondary: Gallup data (1979–2023) for granular demographic breakdowns.

3. Key Contributions

1. Hybrid Agent Initialization: A novel workflow combining social survey data and social media profiles to generate high-fidelity, feature-rich agent populations representing a national demographic.
2. Cognitive-Driven Simulation Framework: A system that integrates real-world longitudinal news data with cognitive theories (dissonance, reflection) to simulate long-term opinion evolution, moving beyond simple "prompt-and-response" models.
3. Systematic Debiasing Mechanisms: The proposal and evaluation of three distinct strategies (Fact Elicitation, Devil's Advocate, Counterfactual) to mitigate LLM bias in social simulations.
4. Empirical Analysis of Model Bias: A comparative study using two state-of-the-art models (US-based GPT-4o and China-based Qwen3-14b) to reveal how geographic origin influences simulation outcomes.

4. Results

The experiments were conducted with 50 agents per run across 20 years, processing ~92 million tokens per model.

A. Overall Attitude Accuracy

• Control (No Debiasing): Both models produced significantly more negative attitudes than real-world data, indicating inherent model bias toward China.
• Devil's Advocate (DA): This method yielded the highest fidelity to ground truth for both models.
  • GPT-4o: MAE reduced from 28.9% (Control) to 14.3% (DA).
  • Qwen3-14b: MAE reduced from 37.9% (Control) to 24.9% (DA).
• Fact Elicitation: Performed second-best, suggesting that removing emotional language helps but is less effective than critical reasoning.
• Counterfactual: Results were mixed. It increased error for Qwen3-14b (MAE 51.0%) but showed different patterns for GPT-4o.

B. Model-Specific Biases (Counterfactual Analysis)

• In-Group Favoritism: The Counterfactual experiment revealed that models tend to favor their country of origin.
  • Qwen3-14b: When the subject was swapped to the US, the model became extremely negative toward the US (simulating US citizens being negative toward China).
  • GPT-4o: Showed the opposite pattern, becoming more negative toward China when the subject was swapped.
• This suggests that pre-training data and cultural priors heavily influence "neutral" simulations.

C. Trend Reversal Analysis

• Metric: Ability to capture the direction change (slope reversal) of attitudes during specific years.
• Finding: Accuracy in capturing trend reversals is not correlated with overall score accuracy.
  • Qwen3-14b was better at capturing reversal points (17 tolerant matches) compared to GPT-4o (15), despite GPT-4o having better overall score accuracy.
  • Both models struggled to capture "favorable" reversals compared to "unfavorable" ones.

D. Domain & Demographic Analysis

• Domains: Politics and Economy news drove the most negative opinion shifts. Tech, Culture, and Sports drove positive shifts.
• Demographics: GPT-4o demonstrated superior fidelity in reproducing real-world demographic differences (e.g., Democrats vs. Republicans, gender, race) compared to Qwen3-14b.

5. Significance and Implications

• For Social Scientists: Provides a robust framework for simulating macro-trends in international relations using LLMs, provided that debiasing mechanisms (specifically the Devil's Advocate approach) are employed.
• For Policymakers: Highlights the risks of using raw LLM simulations for policy support without debiasing, as models may artificially amplify negative geopolitical sentiments.
• For AI Research: Demonstrates that "Chain of Thought" alone is insufficient for long-term social simulation; explicit cognitive architectures (memory, reflection, critical critique) are necessary to align AI behavior with human cognitive tendencies.
• Bias Detection: The study confirms that LLMs carry latent geopolitical biases tied to their training origins, which can be exposed through counterfactual testing.

Conclusion: The paper establishes that while LLMs can simulate international opinion evolution, they require structured cognitive interventions (like the Devil's Advocate agent) to overcome inherent biases and achieve realistic, human-like opinion dynamics.