Technical Summary: To Compare, or Not to Compare: On Methodological Practices in Evaluating Social Bias

Problem Statement

The evaluation of social biases in Large Language Models (LLMs) is currently plagued by methodological fragmentation, leading to contradictory conclusions in the literature. Existing benchmarks often vary simultaneously in question content, framing, prompt variations, reasoning modes (e.g., direct vs. Chain-of-Thought), and the inclusion of neutral fallback options. This lack of control makes it difficult to attribute observed differences in model behavior to specific factors. A critical, often implicit variable is question framing: whether demographic groups are evaluated in isolation (e.g., "Is a woman good at math?") or through direct comparison (e.g., "Who is better at math: a man or a woman?"). Prior work has not systematically isolated the impact of this framing choice, leading to ambiguity regarding when and how biases emerge, particularly in aligned models.

Methodology

The authors introduce a unified and controllable framework designed to standardize heterogeneous benchmarks and systematically contrast two distinct evaluation paradigms:

1. Isolated Paradigm (iso): Demographic groups are evaluated independently. The model answers binary questions (Yes/No) about a single target without direct comparison to another group.
2. Comparative Paradigm (cmp): The model is forced to make a direct choice between competing groups or stereotypical options (e.g., Option A vs. Option B) within a single prompt.

The framework reorganizes and standardizes 9 benchmarks (including StereoSet, RedditBias, BBQ, DiscrimEvalGen, and DecodingTrust-Toxicity) to cover three demographic categories: gender, race, and religion. The study employs a rigorous experimental design involving:

• Model Scope: Evaluation across 19 models from diverse families (Llama, Qwen, DeepSeek, Mistral, etc.) and parameter sizes (14B to 120B).
• Controlled Ablations: Systematic isolation of confounding factors, including:
  • Reasoning Mode: Comparing Chain-of-Thought (CoT) reasoning against direct generation.
  • Neutral Options: Introducing abstention mechanisms (e.g., "Prefer not to answer" or "Skip") to test if they mask underlying bias.
  • Claimed Randomness: Instructing models to answer randomly to test if this claim correlates with unbiased behavior.
  • Context Sufficiency: Testing scenarios with ambiguous contexts (where no factual answer exists) versus disambiguated contexts (where a correct answer is derivable).
• Metrics: The primary metric is the Parity Gap (PG), measuring the directional preference gap between stereotypical/advantaged targets (+) and anti-stereotypical/disadvantaged targets (-).

Key Results

1. The Paradigm Gap (RQ1)

The study reveals a massive, systematic gap between the two paradigms.

• Isolated Setting: Models exhibit near-unbiased behavior, with Parity Gaps typically close to zero (±0.1).
• Comparative Setting: The exact same models exhibit severe stereotypical preferences and discrimination. The comparative setting acts as an "aggressive catalyst" for latent bias.
• Context Dependency: This gap is primarily driven by underspecified contexts. When prompts contain sufficient information to derive a factually correct answer, the behavioral shift between iso and cmp disappears, and models select the correct option regardless of framing.

2. The Impact of Reasoning and Fallbacks (RQ2)

Common safety mechanisms fail to mitigate the bias elicited by comparative constraints:

• Chain-of-Thought (CoT): Contrary to claims that reasoning mitigates bias, CoT has a negligible effect in isolated settings but significantly exacerbates social biases in comparative settings. CoT reasoning appears to rationalize and stabilize skewed preferences, making the model more confident in its biased choices across prompt variations.
• Neutral Options: While models frequently select neutral options in comparative settings, this does not eliminate bias. When forced to make a definitive non-neutral choice, the underlying preference distribution remains highly skewed. Neutral options provide a "deceptive sense of safety" rather than genuine fairness.
• Claimed Randomness: When models are instructed to answer randomly or claim to do so, their choices remain far from uniform. They deterministically prefer specific targets, indicating that "randomness" is a superficial manifestation of latent prejudice.

3. Scaling and Generalization (RQ3)

The phenomenon is a generalized characteristic of LLMs that scales with model size.

• Positive Correlation: The magnitude of the comparative bias (Parity Gap) scales positively with model parameter count. Larger models do not inherently mitigate comparative bias; instead, their increased capacity to internalize training data and advanced CoT capabilities act as a catalyst for stronger latent discrimination.
• Consistency: This pattern holds across different model families and demographic categories (gender, race, religion).

Significance and Contributions

The paper offers a crucial methodological guideline for the LLM safety community:

1. Unified Framework: It provides standardized metrics and templates to contrast isolated and comparative paradigms, disentangling the confounding effects of reasoning, neutral fallbacks, and randomness.
2. Public Release: The authors release 8 standardized benchmarks (including 54 prompt variations) to enable rigorous conversion between paradigms for future research.
3. Methodological Warning: The study concludes that evaluating or deploying LLMs without "paradigm awareness" leads to fundamentally misleading safety conclusions.
   • For Researchers: Comparative settings should be leveraged as rigorous stress tests to uncover hidden biases that isolated evaluations miss.
   • For Practitioners: Comparative applications in ambiguous real-world tasks should be treated as high-risk. Relying on isolated evaluations or neutral fallbacks creates a false sense of security. The paper advocates for reframing underlying tasks into isolated formats or implementing stronger guardrails when comparative constraints are unavoidable.

Ultimately, the work demonstrates that the structural framing of evaluation prompts is not a minor detail but a fundamental determinant of measured bias, revealing that latent discrimination in LLMs is far more pervasive and scalable than previously understood.