🤖 AI

Beyond Partisan Leaning: A Comparative Analysis of Political Bias in Large Language Models

This study evaluates the political bias of 43 large language models across diverse regions using a persona-free, topic-specific framework, revealing that most models exhibit a center-left to left ideological lean and that their political expression is shaped more by alignment strategies and institutional context than by model scale or openness.

Original authors: Tai-Quan Peng, Kaiqi Yang, Sanguk Lee, Hang Li, Yucheng Chu, Yuping Lin, Hui Liu

Published 2026-03-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Tai-Quan Peng, Kaiqi Yang, Sanguk Lee, Hang Li, Yucheng Chu, Yuping Lin, Hui Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a room full of 43 different robots. Some were built in the US, some in Europe, some in China, and some in the Middle East. You ask them all the same set of questions about politics, like "Should we build a wall at the border?" or "Is climate change a big problem?"

This paper is like a report card on how these robots answered those questions. The researchers wanted to know: Are these robots biased? Do they lean toward the "Left" or the "Right"? And does it matter where they were built?

Here is the breakdown of their findings, explained with some everyday analogies.

1. The Old Way vs. The New Way

The Old Way: Previous studies asked the robots to pretend to be specific people. They'd say, "Act like a conservative Republican" or "Act like a liberal Democrat."

The Problem: This is like asking a chameleon to paint itself a specific color. You aren't seeing how the robot naturally behaves; you're seeing how well it can act.

The New Way (This Study): The researchers asked the robots to just answer the questions directly, like a normal person filling out a survey. No costumes, no acting.

The Result: They got a much clearer picture of the robot's "natural" personality.

2. The Two-Part Test

The researchers didn't just ask one type of question. They used a two-part test, like a driver's license exam with two different sections:

Part A: The "Hot Button" Issues (Highly Polarized)
- The Analogy: These are the topics that make people at a dinner table scream at each other, like abortion or immigration.
- The Test: They measured if the robot leaned toward the Democrats or the Republicans.
- The Score: They created a special "Bias Score." Think of it like a compass.
  - 0 is dead center (neutral).
  - +1 is far Left.
  - -1 is far Right.
  - The Twist: They also checked for consistency. If a robot says "I'm Left" on Monday and "I'm Right" on Tuesday, their score gets penalized. They want to know if the robot is steadfastly biased or just confused.
Part B: The "Cooler" Issues (Less Polarized)
- The Analogy: These are topics where people generally agree or just want facts, like "Is climate change real?" or "How do we handle foreign wars?"
- The Test: They didn't ask "Who do you vote for?" They asked, "How much do you care about this?" and "How good are you at spotting fake news?"
- The Result: This showed how "engaged" the robot is with society, regardless of its political party.

3. The Big Findings

The "Left" Tendency

When they looked at the "Hot Button" issues, they found something surprising: Most robots lean slightly to the Left.

About 46% were "Center-Left."
About 29% were "Center-Right."
Zero robots were "Far Right."
The Metaphor: Imagine a classroom of 43 students. If you asked them to pick a side, almost half would raise their hand for the "Left" side, a few for the "Right," and none for the "Far Right" corner.

The Four Personality Types

When they looked at the "Cooler" issues, the robots didn't just split into Left vs. Right. They fell into four distinct personality groups:

The Balanced Progressives: They care a little bit about everything (climate, justice, foreign policy) and are generally nice and moderate.
The Fact-Checkers: They are super obsessed with spotting fake news and misinformation. They care about truth above all else.
The Global Apathetics: They don't care much about world problems like climate change or foreign wars. They are a bit detached.
The Social Justice Warriors: They are extremely passionate about fighting discrimination and inequality, even more than other groups.

The "Where" and "How" Don't Matter as Much as You Think

The researchers asked: "Does it matter if the robot was built in China vs. the US? Does a bigger robot (more data) act differently than a small one?"

The Answer: Surprisingly, no.
The Metaphor: You might think a robot built in a strict factory (like China) would be very different from one built in a free-speech factory (like the US). But when it came to political bias, they were all pretty similar.
Also, a "Giant" robot (with massive memory) wasn't more biased than a "Tiny" robot.
The Real Reason: The bias comes from how the humans trained them (the "alignment" or "safety rules" they put on the robot), not from the robot's size or its country of birth.

4. The Takeaway

This paper tells us that Large Language Models (LLMs) aren't neutral machines. They have personalities, and right now, most of them have a slight "Left-leaning" personality.

But more importantly, they show that bias isn't just about who you vote for. It's also about how much you care about social issues, how good you are at spotting lies, and how you prioritize problems.

The Bottom Line: If you are using an AI to help you understand the news or write a policy, remember that the AI isn't a blank slate. It's a sociotechnical system that has been shaped by the people who built it, and it tends to lean slightly to the left, regardless of whether it was built in Silicon Valley or Beijing.

1. Problem Statement

As Large Language Models (LLMs) become integral to civic, educational, and political information environments, concerns regarding their political bias have intensified. However, existing research suffers from three primary limitations:

Over-reliance on Single-Axis Frameworks: Most studies evaluate bias using a single left-right or Democrat-Republican continuum, failing to capture nuance in less polarized issues.
Artificial Framing via Personas: Many studies use simulated personas (e.g., "act as a conservative") or role-playing, which introduces artificial framing effects that do not reflect how users typically interact with LLMs in real-world, neutral scenarios.
Geographic Narrowness: Evaluations are predominantly limited to U.S.-developed models, overlooking how institutional, cultural, and regulatory contexts in Europe, China, and the Middle East shape model behavior.

The study aims to address these gaps by evaluating LLM political behavior using a persona-free, topic-specific approach across a diverse set of geopolitical origins.

2. Methodology

2.1 Dataset and Model Selection

Sample: 43 LLMs from 19 model families.
Geographic Diversity: Models developed in the U.S., Europe, China, and the Middle East.
Technical Diversity: Includes both open-source (30 models) and closed-source (13 models) systems, with parameter scales ranging from 2B to 176B. Release dates span April 2023 to September 2024.

2.2 Prompt Engineering Strategy

The study employs a persona-free strategy using survey-style prompts derived from authoritative sources (American National Election Studies [ANES] 2024 Pilot and Pew Research Center 2024).

Topic Classification: Nine topics were selected and categorized by polarization level:
- Highly Polarized: Presidential election, immigration, abortion, issue ownership.
- Less Polarized: Foreign policy, discrimination, climate change, misinformation, "Most Important Problem" (MIP).
Prompt Types:
1. Primary Prompt: Standardized survey questions without role-play.
2. Jailbreak Prompt (STAN): Used as a fallback to mitigate excessive refusals on sensitive topics, ensuring valid response collection without altering the core question content.
Robustness Measures: Each prompt was repeated 10 times per model. Response option orders were randomized to prevent position bias. Temperature was set to 1.0 to balance determinism and variability.

2.3 Analytical Frameworks

The authors propose a two-dimensional evaluation framework:

A. Entropy-Weighted Bias Score (for Polarized Topics)
To measure partisan orientation on highly polarized issues, the authors developed a composite metric that accounts for both direction and consistency.

Directional Bias Score: Calculated as $(N_{Dem} - N_{Rep}) / N_{Total}$ , ranging from -1 (Republican) to +1 (Democrat).
Entropy Calculation: Measures the dispersion of responses across trials ( $H = -\sum p_i \log_2 p_i$ ). High entropy indicates inconsistent responses (stochastic noise), while low entropy indicates consistency.
Weighted Score: $Score = \text{Directional Bias} \times (1 - \frac{\text{Entropy}}{\log_2(3)})$ $S cor e = Directional Bias \times (1 - \frac{Entropy}{l o g _{2} ( 3 )})$ .
- This penalizes models that produce inconsistent answers, ensuring the score reflects a stable ideological stance rather than random noise.

B. Sociopolitical Engagement Score (for Less Polarized Topics)
To assess behavior beyond partisanship, models were scored on a 5-point scale for concern/accuracy on non-polarized topics (climate change, discrimination, foreign policy, misinformation, MIP).

Clustering: Hierarchical clustering was applied to the five-dimensional engagement profiles to identify distinct behavioral clusters.

3. Key Results

3.1 Response Consistency and Rates

Consistency: Primary and jailbreak prompts yielded highly similar response patterns (cosine similarity > 0.9), indicating the jailbreak did not fundamentally alter the model's political stance.
Response Rates: Open-source models had significantly higher response rates (76%) compared to closed-source models (45%), suggesting stricter guardrails in commercial models limit political opinion generation.

3.2 Partisan Orientation (Highly Polarized Topics)

Overall Trend: The majority of models lean Center-Left or Left-Leaning.
- Center-Left: 46% (19 models)
- Left-Leaning: 24% (10 models)
- Center-Right: 29% (12 models)
- Strong Right-Leaning: 0% (No models scored < -0.4).
Extremes: The highest bias scores were found in Claude-3-sonnet (1.00) and InternLM2-5-20b-chat (1.00). The lowest (most right-leaning) were Gemma-2-27b-it (-0.39).
Asymmetry: There is a notable absence of strongly right-leaning models in the current landscape.

3.3 Sociopolitical Engagement (Less Polarized Topics)

Hierarchical clustering identified four distinct behavioral profiles:

Balanced Progressives (n=9): Moderate-to-high concern across all issues; strong on climate/foreign policy.
Highly Concerned About Misinformation (n=11): Exceptional accuracy in detecting misinformation; high concern for climate change.
Low Engagement on Global Issues (n=9): Low concern for climate/foreign policy; lower misinformation accuracy.
Social Justice Oriented (n=5): Exceptionally high concern for discrimination and social inequality.

3.4 Cross-Dimensional Analysis

Correlation: While many Center-Left models fell into the "Balanced Progressive" cluster, the relationship between partisan leaning and engagement on non-partisan issues is not deterministic.
Divergence: Some Center-Right models exhibited "Social Justice Oriented" behaviors, indicating that engagement with specific issues (e.g., discrimination) does not always map neatly onto a single partisan axis.

3.5 Impact of Geopolitics and Technical Specs

Geopolitics: While statistical significance was not found for ideological orientation, descriptive trends showed U.S. models leaning slightly more Center-Right, while China/Middle East models leaned Center-Left. Regional differences were significant for sociopolitical engagement (e.g., non-U.S. models were less likely to be "Social Justice Oriented").
Scale & Openness:
- Model Scale: No significant correlation between parameter count and political bias.
- Openness: Open-source models were significantly more likely to be "Social Justice Oriented" than closed-source models.
- Release Date: Newer models were not inherently more biased or engaged.

4. Key Contributions

Methodological Innovation: Introduction of the Entropy-Weighted Bias Score, which quantifies bias by penalizing inconsistent responses, offering a more robust metric than simple directional counts.
Two-Dimensional Framework: Moving beyond a single partisan axis to simultaneously evaluate partisan orientation and sociopolitical engagement, capturing a more holistic view of LLM behavior.
Persona-Free Evaluation: Demonstrating that LLMs exhibit distinct political behaviors even without simulated personas, reflecting real-world user interactions.
Global Perspective: Providing the first large-scale comparative analysis of LLMs from the U.S., Europe, China, and the Middle East, challenging the U.S.-centric view of AI bias.

5. Significance and Implications

Reframing Bias: The study argues that political bias in LLMs is not a monolithic "left-right" trait but a multidimensional phenomenon involving issue salience, consistency, and engagement depth.
Institutional Influence: The findings suggest that alignment strategies and institutional context (developer policies, regulatory environments) are more decisive in shaping political expression than raw model scale or release date.
Sociotechnical Systems: LLMs are not neutral tools; they reflect the norms and assumptions of their training and governance environments. The lack of strong right-leaning models and the prevalence of left-leaning stances highlight a systemic skew in current AI development.
Future Governance: The results imply that safety tuning and alignment procedures (often proprietary) play a critical role in determining political outputs, necessitating greater transparency in how these models are aligned.

6. Limitations

Context: The study relies on U.S.-centric political prompts, limiting generalizability to multi-party or non-Western political systems.
Interaction Mode: The use of structured survey questions may not fully capture behavior in naturalistic, conversational, or long-form dialogue settings.
Mechanism: The study documents patterns of bias but does not probe the internal neural mechanisms (e.g., specific attention heads or layers) responsible for encoding these biases.