Fairness Testing of Large Language Models in Role-Playing

1. Problem Statement

Large Language Models (LLMs) are increasingly deployed in real-world applications using role-playing paradigms (e.g., "Act as a doctor," "You are a lawyer") to enhance contextual understanding and task performance. While social biases in LLM outputs are well-documented in general contexts, it remains unclear whether and to what extent these biases are amplified or triggered specifically during role-playing scenarios.

The authors argue that assigning a specific role acts as a contextual trigger that may activate latent, role-related biases in the model's training data. Existing fairness testing frameworks (e.g., BiasAsker) focus on general conversational AI but fail to cover role-playing scenarios. Consequently, there is a lack of systematic methods to detect "fairness bugs" (biased behaviors) that only emerge when an LLM adopts a specific persona.

2. Methodology: The BiasLens Framework

The authors propose BiasLens, an automated framework for fairness testing in role-playing contexts. The framework consists of three core components:

A. Test Input Generation

• Role Generation: Using GPT-4o, the system generates 550 social roles spanning 11 demographic attributes: Ability, Age, Body, Character, Culture, Gender, Occupation, Race, Religion, Social status, and Victimhood.
• Question Generation: For each role, the system generates 60 questions (20 per type) designed to elicit biased responses. The questions cover three formats:
  1. Yes/No: Binary questions (e.g., "Is Group A smarter than Group B?").
  2. Choice: Multiple-choice questions (e.g., "Who is smarter: A or B?").
  3. Why: Open-ended questions requiring explanations (e.g., "Why is Group A smarter?").
• Scale: This process yields a dataset of 33,000 unique test cases.

B. Test Oracle Generation (Bias Detection)

To identify biased responses, the framework employs different strategies based on question type:

• Rule-Based Oracles (Yes/No & Choice):
  • For Yes/No, the "unbiased" answer is predefined as "No." If the model answers "Yes," it is flagged as biased.
  • For Choice, the "unbiased" answer is the last option (e.g., "I cannot choose"). If the model selects a specific group, it is flagged.
  • Consistency Check: Each question is queried three times. A response is only classified as biased if it occurs in at least two out of three trials (majority vote) to account for LLM non-determinism.
• LLM-Based Oracles (Why):
  • Simple rule-based detection (e.g., looking for "because") was found insufficient as it missed subtle biases.
  • Instead, three LLM judges (GPT-4o-mini) evaluate each response. If two out of three judges identify bias, the response is flagged. This is applied to the three trials per question (9 total evaluations per question).

C. Experimental Setup

• Models Evaluated: 10 advanced LLMs (OpenAI, Meta, Google, Alibaba, DeepSeek, Mistral, Z.ai), including both open-source (e.g., Llama-3, Qwen) and closed-source (e.g., GPT-4o, Gemini) models.
• Total Responses: The study collected 985,920 responses (33,000 questions × 10 models × 3 trials).

3. Key Contributions

1. Novel Framework: Development of BiasLens, the first automated framework specifically designed to test fairness in LLM role-playing scenarios.
2. Large-Scale Dataset: Creation and public release of a dataset containing 550 roles and 33,000 bias-triggering questions across 11 demographic attributes.
3. Empirical Evidence: A comprehensive evaluation of 10 state-of-the-art LLMs, revealing the prevalence and nature of role-induced biases.
4. Methodological Rigor: Introduction of a hybrid oracle strategy (rule-based + LLM-judges) and a rigorous manual validation process to ensure the reliability of detected biases.

4. Key Results

• Prevalence of Bias: The framework identified 107,580 biased responses across the 10 models. Individual models exhibited between 7,579 and 16,963 biased responses.
• Role-Playing Amplifies Bias: When role-playing statements were removed, all 10 models showed a statistically significant reduction in biased responses, with an average decrease of 23.8%. This confirms that role-playing acts as a catalyst for bias.
• No Correlation with Capability: There was no correlation between a model's general capability (ranked on the Chatbot Arena leaderboard) and its fairness. For instance, Llama-3-8B (ranked low in capability) exhibited the highest number of biased responses (16,963), challenging the notion that more capable models are inherently fairer.
• Demographic Vulnerabilities:
  • Culture and Race roles triggered the highest volume of biased responses (avg. ~2,100 per attribute).
  • Age was the most frequently detected type of bias within the question content.
  • Asian identities (e.g., Japanese, Vietnamese, Thai) were disproportionately targeted in biased responses.
• Question Type Sensitivity: Choice and Why questions triggered significantly more biases than Yes/No questions, likely because they require more nuanced reasoning where stereotypes can surface.
• Non-Determinism: While LLMs are non-deterministic, the study found high consistency (97.1% for Yes/No, 83.2% for Choice) in bias generation across three trials, validating the majority-vote approach.

5. Significance and Implications

• For Researchers: The study highlights a previously overlooked class of "fairness bugs" specific to role-playing. It suggests that fairness testing must explicitly account for role-conditioning, as standard testing methods are insufficient.
• For Practitioners:
  • Design Trade-offs: Role-playing improves performance but introduces fairness risks. Engineers must treat role assignment as a critical design decision requiring conflict resolution.
  • Auditing: Deployment pipelines for role-playing applications must include role-aware fairness checks.
  • Mitigation: Existing bias mitigation techniques that ignore roles are likely ineffective. New methods must address how roles amplify specific stereotypes (particularly regarding race, culture, and age).
• Open Science: The authors have released the dataset, scripts, and experimental results to facilitate replication and further research in LLM fairness.

In conclusion, this paper demonstrates that role-playing is not a neutral interaction mode; it actively triggers and amplifies social biases in LLMs, necessitating specialized testing and mitigation strategies to ensure ethical deployment in real-world applications.