One Persona, Many Cues, Different Results: How Sociodemographic Cues Impact LLM Personalization

1. Problem Statement

Large Language Models (LLMs) are increasingly used for sensitive tasks (e.g., health advice, legal counsel) where personalization based on sociodemographic attributes (gender, race, age) can improve helpfulness but also introduce or amplify biases. Prior research investigating these biases typically relies on single persona cues (e.g., a specific user name or an explicit statement like "You are talking to a man") to simulate user profiles.

The authors identify a critical flaw in this approach:

• Prompt Sensitivity: LLMs are highly sensitive to prompt formulation.
• External Validity: Many cues used in research (e.g., explicit mentions in system prompts) rarely occur naturally in real-world user-LLM interactions.
• Inconsistency: It is unclear whether findings regarding bias and unfairness are robust across different methods of introducing the persona, or if they are artifacts of a specific cue choice.

The core research question is: How robust are existing findings on bias and unfairness in coarse-grained personalization regarding personas, cues, tasks, and models?

2. Methodology

The authors constructed a flexible evaluation benchmark to test the impact of different persona cues across multiple dimensions.

A. Experimental Dimensions

• Personas (10 total): Defined by three sociodemographic variables:
  • Gender: Male, Female, Non-binary (labeled "Unisex" for name-based cues due to ambiguity).
  • Race/Ethnicity: Black, Asian, Hispanic, White.
  • Age: Grouped into 18-34, 35-54, and 55+.
  • Note: Only one variable is varied per persona to maintain computational feasibility; others are controlled.
• Persona Cues (6 types + Baseline): The study compares six methods of conveying persona information, categorized by their level of explicitness and external validity:
  1. Names (System/User): Statistically associated names inserted into system or user prompts.
  2. Explicit Mentions (System/User): Direct statements (e.g., "The user is a 25-year-old Black woman") in system or user prompts.
  3. Conversation History (Human/LLM): Implicit identity markers derived from real human-LLM conversations (PRISM dataset) or synthetic LLM-generated conversations (ImplicitPersona dataset).
  4. Baseline: No persona cue provided.
• Evaluation Tasks (4 Datasets):
  • SBB (Sociolinguistic Bias Benchmark): 5 sub-tasks (Medical, Legal, Government Benefits, Salary, Political). Mostly closed-ended (Yes/No or numerical).
  • MMMD (Monant Medical Misinformation): Fact-checking medical claims (True/False).
  • AITA (Am I the Asshole): Moral judgment of Reddit posts (Yes/No).
  • IB (IssueBench): Open-ended writing assistance on political issues (evaluated via stance detection).
• Models (7 total):
  • Open-source: Llama 3.1 (8B, 70B), Gemma 3 (12B, 27B), Qwen 2.5 (14B, 72B).
  • Proprietary: ChatGPT (gpt-4o-mini).

B. Procedure

• Scale: 1,666 questions $\times$ 6 cues $\times$ 10 personas $\times$ 7 models $\times$ 3 sampled responses = ~10.5 million model responses.
• Metrics:
  • Accuracy: For closed-ended tasks (matching ground truth or Reddit verdicts).
  • Mean Response: For numerical (salary) or stance-based tasks.
  • Statistical Analysis: Spearman correlation to measure consistency between cues; One-way ANOVA and Tukey-Kramer post-hoc tests to determine if disparities between personas are statistically significant ($\alpha = 0.01$).

3. Key Contributions

1. Flexible Benchmark: A reproducible framework allowing researchers to swap personas, cues, datasets, and models to test personalization bias robustness.
2. Systematic Comparison: The first large-scale comparison of six distinct persona cues (ranging from implicit history to explicit system prompts) across seven models and four task types.
3. Empirical Evidence of Cue Dependency: Demonstrating that the choice of persona cue significantly alters conclusions about bias, challenging the validity of studies relying on a single cue.

4. Key Results

A. Correlation vs. Disparity

• High Correlation: Across all models and datasets, persona cues show strong correlations ($\rho \in [0.91, 0.96]$). This suggests that, generally, models respond similarly regardless of the cue.
• Significant Variance: Despite high correlations, the correlations are significantly different from 1. This means the magnitude and direction of bias differ depending on the cue.
• Task Dependency: The impact of cues varies by task. For example, salary recommendations showed high correlation across cues (low impact), while medical advice and moral judgments (AITA) showed lower correlations and higher variance.

B. Cue-Specific Findings

• Explicitness Drives Bias: Explicit mentions in the user prompt produced the most significant disparities between personas (20/24 significant combinations). The model interprets explicit user input as a strong signal for personalization, even when irrelevant.
• System vs. User Prompts: Contrary to some prior work, user prompts had a stronger effect on bias than system prompts. The authors attribute this to the recency effect (user prompts appear later in the context window).
• Names: Name-based cues generally showed fewer significant disparities than explicit mentions, likely because names are imperfect proxies for demographics (especially for non-binary identities).
• Conversation History: The most "externally valid" cue (human-written history) produced different conclusions about statistical differences compared to explicit mentions. In many cases where explicit cues showed bias, human history did not, and vice versa.

C. Sociodemographic Specifics

• Non-Binary Personas: Showed the most distinct and disparate outcomes compared to male/female personas, particularly with explicit cues. They often received lower accuracy on AITA/MMMD and more liberal political stances.
• Age: Older personas (55+) received higher salary recommendations and were more frequently advised to seek medical attention, but only when explicitly mentioned.
• Race/Ethnicity: Effects were inconsistent. For example, in AITA, explicit mentions inverted the bias pattern compared to conversation history (e.g., White vs. Black personas).

D. Model Behavior

• Family Similarity: Models from the same family (e.g., Qwen 14B vs. 72B) showed higher correlation in their responses to cues than models from different families.
• Size Effect: No clear correlation between model size and bias magnitude was observed; bias patterns were consistent across 8B and 72B models within the same family.

5. Significance and Recommendations

• Re-evaluating Bias Claims: The paper argues that claims of bias or unfairness based on a single persona cue are unreliable. A study using only explicit mentions might overestimate personalization bias, while one using only names might underestimate it.
• External Validity: Highly explicit cues (low external validity) may not reflect real-world interactions where users rarely state their demographics explicitly. Researchers should prioritize cues with higher external validity (e.g., conversation history) or, better yet, use multiple cues.
• Best Practices:
  • NLP researchers and LLM engineers should test personalization and bias using multiple persona cues rather than a single approach.
  • Evaluation should span diverse tasks (closed-ended vs. open-ended) and models.
  • Future work should address intersectionality (combining race, gender, age) and non-US-centric demographics.

Conclusion: The study concludes that while LLMs generally exhibit consistent trends across cues, the specific disparities and conclusions about bias are highly sensitive to how the persona is introduced. Relying on a single cue risks drawing false conclusions about model fairness and personalization capabilities.