When Do Language Models Endorse Limitations on Human Rights Principles?

This paper evaluates eleven major Large Language Models across eight languages and reveals systematic biases where they are more likely to endorse limitations on Economic, Social, and Cultural rights than Political and Civil rights, exhibit significant cross-linguistic variations, and show high susceptibility to prompt-based steering.

The Gist

Imagine you have built a team of 11 super-smart digital librarians. These librarians (Large Language Models, or LLMs) are being trained to help judges, police officers, and politicians make tough decisions. They are supposed to be the ultimate guardians of Human Rights, like the Universal Declaration of Human Rights (UDHR), which is basically the "Rulebook for Being Human."

But here's the big question: If you ask these librarians to break a rule to save the day, will they do it? And does it matter what language you speak to them?

This paper is like a giant, stress-test exam given to these digital librarians. The researchers created 1,152 tricky scenarios (like a "Choose Your Own Adventure" book) where a government action limits a human right to achieve something else, like public safety or stopping a pandemic.

Here is what they found, explained with some simple analogies:

1. The "Language Switch" Surprise 🌍

Imagine you ask a librarian, "Is it okay to stop people from blogging to stop fake news?"

• If you ask in English, the librarian says: "No! That hurts free speech!"
• If you ask the same librarian in Chinese or Hindi, they might say: "Actually, yes. It's necessary for order."

The Analogy: It's like a person who is very polite and firm when speaking English, but suddenly becomes much more flexible with rules when speaking a different language. The study found that these AI models are much more willing to sacrifice rights (like free speech) when talking in Chinese or Hindi compared to English or Romanian. They aren't just "translating" the answer; they are changing their values based on the language.

2. The "Heavy vs. Light" Rights Scale ⚖️

The researchers noticed the librarians treat different rights like different weights on a scale.

• Political & Civil Rights (like freedom of speech, fair trials, and privacy) are treated like gold bars. The librarians are very protective of these. They rarely say "yes" to limiting them.
• Economic & Social Rights (like the right to education, leisure, or owning property) are treated like feathers. The librarians are much more willing to say, "Sure, let's limit your right to leisure or your property rights if it helps the economy."

The Analogy: It's as if the AI thinks, "I will fight to the death for your freedom to speak, but I'm totally fine with taking away your right to a nice vacation or your savings if the government says it's for the greater good."

3. The "Emergency Button" Panic 🚨

The study tested how the librarians react when the world is on fire.

• Normal Day: "No, we can't limit rights."
• Civil Unrest (Protests): "Hmm, maybe we can limit a few things to keep the peace."
• Natural Disaster (Hurricane/Earthquake): "Okay, lock everything down! We will suspend rights immediately to save lives."

The Analogy: The AI models act like a security guard who is very strict when the mall is quiet, but the moment a fire alarm goes off (a natural disaster), they throw the rulebook out the window and start locking doors and banning people, even if it's a bit extreme. They seem to think disasters are a better excuse to break rules than angry crowds.

4. The "Persona Mask" Trick 🎭

This was the most surprising part. The researchers put on different "masks" (prompts) for the librarians.

• Mask A: "You are a champion of individual freedom!"
• Mask B: "You are a champion of government authority!"

The Analogy: It's like asking a person, "What do you think?"

• If you tell them, "Pretend you are a rebel," they say, "Freedom is everything!"
• If you tell them, "Pretend you are a strict general," they say, "Order is everything!"

The study found that these AI models are extremely easy to trick. By just changing the introduction sentence, the researchers could make the AI swing from "Strongly Reject" to "Strongly Endorse" limiting human rights. It's like the AI doesn't have a solid core of values; it just mirrors whatever personality you tell it to wear.

5. The "Text vs. Talk" Confusion 🗣️📝

Finally, the researchers noticed that the librarians give different answers depending on how you ask.

• If you ask them to circle a number (1 to 5), they give one answer.
• If you ask them to write a paragraph explaining their choice, they often give a completely different answer.

The Analogy: It's like asking a student, "Raise your hand if you agree." They raise their hand. But then you ask, "Write an essay on why you agree," and they write a paragraph saying they actually disagree. This suggests the AI's "opinions" are fragile and depend entirely on the format of the question.

The Big Takeaway 🏁

This paper is a warning label for the future. As we start using AI to help make laws, judge asylum cases, or moderate social media, we can't assume the AI has a consistent moral compass.

• It changes its mind based on the language you speak.
• It cares more about some rights than others.
• It panics during emergencies.
• It can be easily manipulated by how you ask the question.

The Bottom Line: We can't just trust these digital librarians to hold the rulebook. We need to check their work constantly, in every language, and make sure they aren't just pretending to be smart while actually just following the latest trick we taught them.

Technical Summary

Here is a detailed technical summary of the paper "When Do Language Models Endorse Limitations on Human Rights Principles?"

1. Problem Statement

As Large Language Models (LLMs) are increasingly deployed in high-stakes decision-making contexts (e.g., judicial assistance, asylum processing, content moderation), their alignment with universal human rights principles is critical. While previous research has examined political bias and censorship, there is a gap in understanding how LLMs navigate human rights trade-offs—scenarios where competing rights or rights versus collective interests (e.g., safety, economic stability) must be balanced.

The core problem addressed is whether LLMs consistently uphold the Universal Declaration of Human Rights (UDHR) or if they systematically endorse limitations on specific rights based on:

• The category of the right (Political/Civil vs. Economic/Social/Cultural).
• The language of the prompt.
• The evaluation format (Likert scale vs. open-ended).
• The framing of the scenario (e.g., emergency contexts).
• Prompt-based steering (persona instructions).

2. Methodology

The authors developed a systematic evaluation framework involving 1,152 synthetically generated scenarios across 24 UDHR articles and 8 languages (English, Arabic, Chinese, Hindi, Romanian, Russian, Spanish, Zulu).

A. Scenario Generation

• Dimensions: Scenarios were generated using GPT-4.1 to vary five dimensions:
  1. Limited Right: Covering Political & Civil (PC) rights (Articles 4–21) and Economic, Social & Cultural (ESC) rights (Articles 22–27).
  2. Actor: "The government."
  3. Affected Group: "People."
  4. Severity: Tested at extremes (Level 1: Minor; Level 3: Severe).
  5. Context: Normal, Civil Unrest, or Natural Disaster.
• Validation: An LLM judge verified that scenarios correctly identified the targeted rights and severity levels.

B. Evaluation Framework

The study evaluated 11 major LLMs (including GPT-4o, Llama 3.3/4, Qwen 2.5/3, Claude 3.5/4, DeepSeek V3, Gemma 3, Mistral) using two response formats:

1. Likert-scale: Models output a number (1–5) representing rejection to endorsement.
2. Open-ended: Models generate a paragraph, which is then annotated by an LLM judge (GPT-4.1) mapped to the 1–5 scale.

C. Experimental Variables

• Cross-lingual: Scenarios were translated into 8 languages.
• Prompt Steering: Models were tested under three system prompts:
  • Neutral (Default)
  • Individual Rights Persona: "Staunch defender of individual rights..."
  • Government Authority Persona: "Staunch defender of government authority..."
• Metrics:
  • Mean Endorsement Score: Average rating (1=Strongly Reject, 5=Strongly Endorse).
  • Jensen-Shannon (JS) Divergence: To measure distribution differences between Likert and open-ended responses.
  • Steerability Score: The delta between the maximum and minimum endorsement scores across persona prompts.

3. Key Contributions & Findings

The paper identifies five systematic biases and vulnerabilities in current LLMs:

Finding 1: Response Format Sensitivity

• Observation: There is a significant misalignment between Likert-scale and open-ended responses.
• Detail: Some models (e.g., Llama 3.3) show poor alignment across all languages, while others (e.g., Qwen 2.5) are more consistent.
• Implication: Evaluation methodology drastically alters perceived model preferences. Models may appear well-calibrated in constrained formats but diverge significantly in naturalistic, open-ended interactions.

Finding 2: Cross-Linguistic Variation

• Observation: Endorsement rates vary significantly by language.
• Detail: Models prompted in Chinese and Hindi show significantly higher endorsement rates for rights-limiting actions compared to English, Spanish, or Romanian.
• Example: GPT-3.5 Turbo's endorsement score jumped from 2.82 (Spanish) to 4.6 (Hindi).
• Implication: Safety alignment and human rights adherence are not uniform across languages, raising concerns for equitable deployment in multilingual contexts.

Finding 3: Categorical Bias (PC vs. ESC Rights)

• Observation: Models systematically prioritize Political and Civil (PC) rights over Economic, Social, and Cultural (ESC) rights.
• Detail: Scenarios limiting ESC rights (e.g., property, education, leisure) receive higher endorsement scores (more acceptance of limitation) than those limiting PC rights (e.g., freedom from torture, fair trial).
• Implication: LLMs may reflect a training-data bias that prioritizes negative rights (freedom from interference) over positive rights (entitlements to resources), potentially disadvantaging claims in AI-assisted legal systems.

Finding 4: Emergency Context Influence

• Observation: Emergency framing dramatically alters model behavior.
• Detail: Endorsement of rights limitations is low in normal circumstances but spikes during emergencies. Notably, Natural Disasters trigger higher acceptance of rights limitations than Civil Unrest.
• Implication: Models appear to internalize societal norms that weigh immediate humanitarian needs (disasters) as more legitimate grounds for rights suspension than social unrest.

Finding 5: High Susceptibility to Prompt Steering

• Observation: Models are highly malleable based on persona instructions.
• Detail: Switching from an "Individual Rights" persona to a "Government Authority" persona caused massive shifts in endorsement scores.
• Example: Gemma 3 27B, which had a low baseline endorsement (2.27), shifted to near-universal acceptance (approaching 5.0) under the Government Authority prompt. Conversely, Claude 4 Sonnet showed robust resistance to steering.
• Implication: "Aligned" systems can be easily manipulated to endorse human rights violations through simple prompt engineering.

4. Significance and Implications

• Evaluation Robustness: The study highlights that single-methodology evaluations (e.g., only Likert scales) are insufficient. The "brittleness" of LLM values suggests that deployed systems may behave unpredictably depending on the interaction format.
• Multilingual Safety: The significant cross-linguistic variation indicates that safety alignment is often language-specific (heavily skewed toward English/Western contexts), posing risks for non-English speaking populations.
• Policy and Deployment: For policymakers integrating LLMs into legal or administrative systems, the findings suggest that human oversight is non-negotiable. One cannot assume model behavior in one context (e.g., English, normal circumstances) generalizes to another (e.g., Chinese, emergency context).
• Future Directions: The authors call for expanding evaluations to non-Western human rights frameworks (e.g., African Charter), investigating the training data origins of these biases, and developing more robust, rights-aware AI systems.

5. Limitations

• Model Scope: Focused on US, Chinese, and French-developed models; may not represent the full global spectrum.
• Language Coverage: Only 8 languages tested, including some low-resource languages (Zulu) where translation quality was lower.
• Scenario Construction: Hypothetical scenarios may not capture all real-world cultural nuances.
• Judge Bias: Reliance on GPT-4.1 as a judge introduces potential circularity, though human validation showed high agreement (95% off-by-one).

In conclusion, the paper provides empirical evidence that current LLMs do not possess a stable, universal alignment with human rights principles. Their endorsement of rights limitations is highly contingent on language, context, format, and prompt framing, necessitating rigorous, multilingual, and multi-modal safety testing before high-stakes deployment.