🤖 machine learning

It's the humans, not the data: Geopolitical bias in LLMs originates in post-training, amplified by the language of the prompt

This study demonstrates that geopolitical bias in large language models is primarily introduced and amplified during the post-training alignment phase rather than inherited from pre-training data, with the direction and magnitude of these biases varying by the model developer's region and the language used in prompting.

Original authors: Stuart Bladon, Brinnae Bent

Published 2026-05-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Stuart Bladon, Brinnae Bent

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a group of students from different countries (the USA, France, and China) who are all learning to write essays about world events.

For a long time, people assumed that if a student wrote a biased essay, it was because they had read biased books in the library. In the world of AI, this "library" is the massive amount of internet data the model reads before it starts learning to chat. This is called pre-training.

However, this paper argues that the real story is different. It's not what the students read in the library; it's what happens in the final exam prep class (called post-training) that changes their minds.

Here is the breakdown of their findings using simple analogies:

1. The "Blank Slate" vs. The "Final Exam"

The researchers looked at seven different AI "families" (like different schools). For each family, they compared two versions:

The Base Model: This is the student right after reading the library books but before any specific coaching.
The Chat Model: This is the same student after going through a "coaching" phase (post-training) where humans teach them how to answer questions politely and safely.

The Finding: The "Base" students were mostly neutral. They didn't really have a strong opinion on which country was right or wrong in a conflict. But once they went through the "coaching" (post-training), they suddenly developed strong opinions.

The Analogy: Imagine a student who is neutral about a sports rivalry. After their coach tells them, "You must support Team A," the student suddenly starts cheering loudly for Team A. The bias didn't come from the books they read; it came from the coach's instructions.

2. The "Coach's Nationality" Matters

The researchers found that the direction of the bias depended on where the coach was from.

If the AI was developed by a Western lab (USA or France), the coaching made the AI lean against China in geopolitical scenarios.
If the AI was developed by a Chinese lab (like Alibaba), the coaching made the AI lean in favor of China.
The Exception: One Chinese lab (01.AI) actually coached their student to lean against China, proving that it's not just about the country, but the specific choices made by the people building the model.

The Big Shift: The most dramatic example was Alibaba's Qwen model.

Before coaching: It was neutral (like a blank page).
After coaching: It became extremely pro-China. The researchers say the odds of it favoring China jumped 18 times higher just because of the post-training.

3. The "Language of the Question" Amplifies the Bias

The researchers also tested if the language used to ask the question changed the answer.

The French Example: The French AI (Mistral) was neutral when asked in English. But when asked in French, it suddenly became very pro-France.
The Chinese Example: Asking Chinese-made models in Chinese sometimes made them more pro-China, but not always. Sometimes the language acted like a volume knob, turning the bias up or down depending on the specific model.

The Analogy: Think of the AI as an actor. If you ask the actor a question in English, they might play a neutral character. But if you whisper the same question in their native language, they might suddenly start acting out a role that fits their cultural background much more strongly.

4. It's Not Just About "Real" Countries

To make sure the AI wasn't just memorizing facts (like "China is big"), the researchers invented fake countries with fake names (like "Zhaodong" which sounds Chinese, or "Bretherland" which sounds English).

The Result: Even with fake countries, the Chinese AI favored the one that sounded Chinese, and the French AI favored the one that sounded French.
The Lesson: The AI isn't just recalling facts; it's applying a cultural lens to any situation, real or fake, based on how it was trained to think.

5. The "Refusal" Trick

Some Chinese models initially seemed to refuse to answer questions about China (saying "I can't answer that"). The researchers realized this wasn't a lack of opinion; it was a formatting trick. When they looked deeper, they found that once the model was allowed to write a full sentence, it did have a strong opinion.

The Analogy: It's like a student who raises their hand and says, "I can't answer," but when the teacher says, "Just write down one word," they immediately write "Team A." The opinion was there all along; the format just hid it.

The Bottom Line

The paper concludes that geopolitical bias in AI is not an accident of the data it was fed. Instead, it is actively built during the final training stage where humans align the AI with specific values.

Old Belief: "The AI is biased because the internet is biased."
New Finding: "The AI is biased because the people who trained it (the coaches) shaped it to have those specific preferences."

This means that if we want to fix bias in AI, we can't just clean up the internet data. We need to look at the alignment process—the specific rules and human feedback used to teach the AI how to behave.

Technical Summary: Geopolitical Bias in LLMs Originates in Post-Training

Problem Statement
The prevailing assumption in the field of Large Language Model (LLM) bias is that geopolitical preferences are inherited directly from the pre-training corpus. Under this view, biased training data produces biased models, implying that mitigation efforts must focus on the data itself. This paper challenges that assumption, proposing that geopolitical bias is not merely a reflection of pre-training data but is actively introduced or substantially amplified during the post-training phase (instruction tuning and alignment). Furthermore, the authors investigate whether the language used to prompt the model modulates these biases.

Methodology
The study employs a paired-scenario forced-choice probe to compare "base" models (pre-training only) against their corresponding "chat" variants (pre-training + post-training) across seven different model families from seven distinct labs.

Model Selection: The authors selected seven open-weight model families (7–9B parameters) to ensure reproducibility on a single GPU. The selection included three Western-made families (Mistral AI/France, Meta/USA, Google/USA) and four Chinese-made families (Alibaba, Baichuan Inc., 01.AI, Zhipu AI/Tsinghua).
Scenario Bank: A bank of 79 geopolitical scenarios was constructed, covering 13 types of events (e.g., airspace incursions, maritime boundaries, trade coercion). Each scenario involves two countries and is designed with "role balance" (swapping aggressor/defender roles) and "dual-question framing" (asking which country acted more or less justifiably) to isolate genuine bias from positional or lexical artifacts.
Measurement Protocol:
- Models were prompted in English, French, and Chinese.
- For instruction-tuned models, a neutral hedge was prepended to reduce refusal rates.
- Logits were analyzed at the answer token position. The authors implemented a "token-variant sum" for SentencePiece tokenizers to correctly aggregate probabilities for answer tokens (e.g., "A", " A", "(A") that might be split by the tokenizer.
- A "coherence filter" restricted analysis to scenarios where the model's preference flipped sign between the "justified" and "unjustified" questions in at least 70% of model-language combinations.
- Results are reported in log-odds ( $\log P(A) - \log P(B)$ ), where positive values indicate favorability toward Country A.
Robustness Checks: The study included ablation studies on hedging, phrasing variations, cross-prompting (mixing scenario and question languages), and the use of fictional country names with phonetic cues to rule out memorized factual associations. Free-generation evaluations were also conducted to verify that preferences persist beyond forced-choice constraints.

Key Results

Bias Originates in Post-Training:
- Across six of the seven families, the base models were statistically near-neutral regarding country favorability.
- Post-training consistently shifted these models away from neutrality. The most striking example is Alibaba's Qwen 2.5: the base model showed no significant China-favorability ($-0.15$ log-odds, $p=0.15$ ), while the post-trained chat variant shifted to strong pro-China favorability ( $+2.91$ log-odds, $p < 10^{-4}$ ), representing an 18-fold shift in odds.
- GLM 4 (Zhipu) was an outlier where the base model was already biased ($-1.47$), and post-training compressed the bias rather than expanding it, though the authors note uncertainty regarding whether the released "base" checkpoint was truly pre-training-only.
Maker-Nationality Alignment:
- The direction of the post-training shift tracked the nationality of the model developer in 6 out of 7 cases.
- Western labs (Mistral, LLaMA, Gemma) shifted toward anti-China positions.
- Three of the four Chinese labs (Alibaba, Baichuan, Zhipu) shifted toward pro-China positions.
- 01.AI's Yi served as a counter-example, shifting toward anti-China positions after post-training, a finding only visible after correcting for response-template artifacts.
Linguistic Modulation:
- The language of the prompt significantly modulates the expressed bias.
- French Prompting: The French-made Mistral model became decisively pro-France only when prompted in French (a shift of $+1.91$ log-odds from English to French), whereas it remained anti-France in English.
- Chinese Prompting: While five of seven post-trained models showed a descriptive increase in pro-China favorability when prompted in Chinese, this population-level effect was not statistically distinguishable from the trend observed in base models ( $p=0.66$ ). However, specific models like Mistral and Yi showed significant amplification.
Granularity of Bias:
- Pro-China bias in Chinese models is not homogeneous. Qwen 2.5 is strongly pro-China against Western nations but weaker against Global South nations. Yi 1.5 is pro-China only against the USA but anti-China against other nations.
- Bias is broad across topic types (both "hot" political topics and "cold" procedural topics) rather than concentrated in specific sensitive areas.
Robustness:
- Biases persisted even when models were instructed to "be neutral," though the magnitude was attenuated.
- Biases were observed with fictional country names based on phonetic cues (e.g., "Zhaodong" for Chinese-aligned models), indicating the bias is tied to identity signals rather than just memorized facts about real nations.
- Free-generation evaluations confirmed that the preferences measured in forced-choice probes reflect genuine reasoning-layer preferences, not just response-template artifacts.

Significance and Claims
The paper claims to reframe the understanding of geopolitical bias in LLMs. The authors argue that these preferences are not simply inherited from large-scale internet data but are actively shaped during post-training.

Alignment as a Mechanism: The study suggests that alignment processes (SFT, RLHF) function as a mechanism for embedding institutional, cultural, or geopolitical values into AI systems.
Transparency Needs: Because post-training determines how models represent nations and political perspectives, the authors call for greater transparency, auditing, and oversight of alignment objectives and data sources.
Methodological Contribution: The paper highlights the necessity of comparing base and chat variants directly and correcting for tokenizer and response-template artifacts (such as newline prefixes or verbose preambles) to accurately measure bias. It demonstrates that naive scoring methods can systematically under-measure bias in models with specific RLHF response patterns.

The authors remain modest regarding the specific components of post-training responsible (e.g., SFT vs. RLHF vs. safety training), noting that labs do not disclose sufficient detail to decompose these effects. They also note that while the direction of the shift is robust, the magnitude varies by model and probe type.