💬 NLP

Demographic Prompting at Scale: When More Attributes Hurt LLM--Human Agreement

This study demonstrates that while supplying one to three high-signal demographic attributes in prompts can improve alignment between LLMs and human annotations, over-specification with full attribute sets degrades performance, revealing that successful demographic prompting depends on the joint consideration of signal learnability, directional coherence, and task context rather than simply increasing the volume of attributes.

Original authors: Mahammed Kamruzzaman, Shrabon Kumar Das, Gene Louis Kim

Published 2026-07-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mahammed Kamruzzaman, Shrabon Kumar Das, Gene Louis Kim

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a super-smart robot friend (a Large Language Model, or LLM) who is really good at reading stories and guessing how people feel about them. Sometimes, the robot guesses perfectly, but other times, it misses the mark because it doesn't know who is reading the story.

Researchers from the University of South Florida decided to test a popular idea: "What if we tell the robot exactly who the reader is?" They tried giving the robot a "persona" by listing the reader's age, race, gender, religion, and more. They wanted to see if this helped the robot agree better with actual human readers.

Here is what they found, using a mix of five different robot brains and five different types of tricky tasks (like spotting toxic comments, guessing if a text is polite, or figuring out the emotion).

The "Goldilocks" Rule: Less is More

The biggest surprise? More information didn't make the robot smarter; it made it confused.

Think of it like giving directions to a friend. If you say, "Go to the park," they might get lost. If you say, "Go to the park, turn left at the red house, then right at the blue car," they might get there. But if you keep adding details—"and don't forget the dog, and the time is 3 PM, and the weather is rainy, and the park has a fountain, and..."—your friend might just freeze up and go the wrong way.

The researchers found that for most robots, the sweet spot was giving them one to three specific details about the reader.

The Sweet Spot: When they gave just a few clues (like "a person who is LGBTQ+ and Black"), the robot's guesses matched human readers much better.
The Overload: When they gave the robot every single detail available (the "full attribute set"), the robot's performance actually got worse. It was like trying to listen to five people talking at once; the robot couldn't figure out who to listen to.

The "Signal vs. Noise" Mystery

You might think, "If a group of people really disagrees about something, the robot should definitely know about it!" But the paper suggests that's not always true.

Imagine the robot is a detective trying to solve a crime based on clues.

The Magnitude Trap: Just because a group of people argues loudly about a topic (high "magnitude" of difference) doesn't mean the robot can use that argument to solve the case. The researchers found that knowing how much humans disagreed didn't help predict if the robot would get better.
The Coherence Key: The real secret was directional coherence. This is a fancy way of asking: "Do the different people in this group agree on what the clues mean?"
- Good Signal: If all the people in a group (say, "LGBTQ+ individuals") agree that the word "threat" means something specific, the robot can learn that.
- Bad Signal: If half the group thinks "threat" means danger, but the other half thinks it means something else, the robot gets confused. Even if the group is "loud" about their opinions, if they are pulling in opposite directions, telling the robot "act like this group" actually makes it worse.

The "Neuron" Detective Work

To understand why this happened, the researchers peeked inside the robot's brain (a process called "neuron probing"). They looked at which tiny parts of the robot's brain lit up when they gave it a persona.

They found a strange paradox with one specific robot, DeepSeek.

The High-Volume Paradox: When DeepSeek was given a persona, it lit up more neurons than any other robot. You'd think, "Wow, it's working so hard!" But actually, it was the worst at following the instructions.
The Lesson: Just because a robot's brain is buzzing with activity doesn't mean it's thinking clearly. Sometimes, all that noise is just the robot getting confused by the extra instructions, not actually understanding the persona.

What This Means for You

The paper doesn't say "Demographic prompting is broken." Instead, it suggests that it's a tool that needs to be used carefully.

Don't dump everything on the robot: If you want the robot to sound like a specific type of person, pick one to three clear, consistent traits. Don't list their entire life story.
Check for agreement: If the group you are trying to mimic is split down the middle on what things mean, the robot probably won't be able to help.
It depends on the robot: Some robots (like Llama or Qwen) got better at these tasks with the right hints. Others (like DeepSeek) actually got worse, no matter what you told them.

In short, the researchers suggest that giving a robot a "persona" isn't a magic button that fixes everything. It's more like tuning a radio: you have to find the right frequency (the right few attributes) to get a clear signal. If you turn the dial too far or try to listen to too many stations at once, you just get static.

Technical Summary: Demographic Prompting at Scale: When More Attributes Hurt

Problem Statement

Large Language Models (LLMs) are increasingly deployed as substitutes or complements to human annotators for subjective NLP tasks (e.g., toxicity detection, sentiment analysis). While prior research has established that LLM outputs are not demographically neutral and that explicit demographic cues can shift model behavior, the field lacks a systematic understanding of how these cues function. Existing literature typically compares a "no-demographic" baseline against a single "all-attributes" prompt or examines only one or two specific attributes in isolation. This approach fails to explore the combinatorial space between single and full attribute sets, leaving critical questions unanswered:

Is there an optimal number of demographic attributes to include in a prompt, or does adding more always improve alignment?
What properties of a dataset's demographic signal predict whether a specific attribute will improve or degrade LLM alignment?
How do LLMs internally process these cues, and does increased neural activation correlate with better alignment?

Methodology

The authors conducted a comprehensive study across five subjective language understanding tasks (Toxicity, Sentiment, Politeness, Offensiveness, Emotion Attribution) and five open-source LLMs (Llama-3.2-3B, Mistral-7B, Gemma3-12B, Qwen2.5-7B, DeepSeek-R1-7B).

Experimental Design

Combinatorial Prompting: The study systematically enumerated every possible combination of demographic attributes for each task, ranging from single-attribute prompts to the full set of available attributes. This allowed for the mapping of alignment trajectories from single-component to full-attribute configurations.
Alignment Metrics: Model-human alignment was quantified using quadratic-weighted Cohen's $\kappa$ (for ordinal tasks) and accuracy (for non-ordinal tasks), measured against human annotations.
Dataset-Side Signal Analysis: To understand why certain attributes work, the authors analyzed the annotation data itself using three levels of structural detail:
1. Magnitude: Using SHAP values on a Logistic Regression classifier to measure the influence of demographic attributes on human label variation.
2. Learnability: Training a LinearSVC with explicit word-demographic interaction features to determine if the model can learn discriminative lexical patterns for specific subgroups.
3. Directional Coherence: Calculating Fisher-averaged Spearman correlations between the weight vectors of different subgroups within an attribute. A positive correlation indicates subgroups agree on which words signal the label (coherent); a negative correlation indicates opposed signals (incoherent).
Neuron Probing: Adapting interpretability frameworks, the authors identified "specialized neurons" (those activated by a demographic prompt but not by a baseline prompt). They measured the proportion of specialized neurons and correlated this with alignment performance to test if internal engagement translates to better output.

Key Contributions

Full Combinatorial Analysis: The paper is the first to systematically enumerate all attribute combinations across multiple tasks and models. It establishes that demographic prompting is not monolithic; its utility is highly context-dependent.
Three-Level Signal Framework: The authors propose a diagnostic framework linking dataset-side signal to LLM alignment. They demonstrate that the magnitude of demographic influence on human labels is uninformative for predicting LLM alignment. Instead, successful alignment requires a joint consideration of learnability and directional coherence.
Neuron Probing Application: This work is the first to apply neuron probing to demographic alignment in subjective tasks. It reveals that specialized neuron activation correlates with alignment gains only when the underlying annotation signal is directionally coherent.

Key Results

1. The Over-Specification Threshold (RQ1)

Contrary to the intuition that "more information is better," alignment consistently peaks with one to three high-signal attributes and degrades when the full attribute set is provided.

Toxicity: Four of five models achieved peak alignment with compact sets (1–3 attributes). The "all together" configuration never outperformed the best compact prompt and often caused significant degradation.
Politeness & Offensiveness: These tasks showed broad resistance to demographic prompting. For most models, adding attributes degraded performance relative to the baseline.
Emotion: Results were highly polarized; while Mistral and Qwen benefited significantly from age-based prompts, Gemma and DeepSeek were degraded by every demographic prompt.

2. The Role of Directional Coherence (RQ2)

The study found that the magnitude of a demographic attribute's influence on human labels (SHAP importance) does not predict LLM alignment.

Learnability vs. Coherence: While learnability (LSVC $\kappa$ ) is a better predictor than magnitude, it is insufficient on its own.
The Coherence Mechanism: Attributes with directionally coherent signals (subgroups agree on lexical patterns) enable alignment gains. Conversely, attributes with opposed signals (subgroups disagree on what words indicate the label) cause alignment to degrade, even if the attribute is highly learnable by a classifier.
- Example: In the Offensiveness task, age was the most learnable attribute but caused the worst alignment because subgroups held opposed views. race, which had lower learnability but higher coherence for specific models, was the only attribute that improved alignment.

3. Neuron Probing and the High-Volume Paradox (RQ3)

Selective Support: On the Toxicity task, models like Llama and Qwen showed a positive correlation between the proportion of specialized neurons activated and alignment gains, but only for attributes with directionally coherent signals.
The High-Volume Paradox: DeepSeek-R1 activated the highest volume of specialized neurons across all tasks yet consistently performed best with no demographic prompt. This suggests that high activation volume does not equate to useful knowledge engagement; it may instead reflect broad, undifferentiated representational perturbation.

Significance and Claims

The paper argues that demographic prompting is not a monolithic intervention. Its effectiveness is strictly conditioned by:

Attribute Signal Quality: Specifically, the directional coherence of the annotation signal.
Task Characteristics: Some tasks (e.g., Toxicity) are amenable to demographic prompting, while others (e.g., Politeness) are resistant.
Model Architecture: Different models respond differently to the same cues (e.g., Gemma vs. DeepSeek on Emotion).

The authors conclude that principled attribute selection is essential. Practitioners should avoid "all-attributes" prompts and instead select 1–3 attributes that exhibit high directional coherence. They emphasize that high learnability alone is a necessary but insufficient condition; if subgroups disagree directionally, no single persona prompt can satisfy the conflicting demands, leading to degraded performance.

The study modestly notes limitations, including its English-only scope, the specific parameter scales of the tested models, and the correlational nature of the neuron analysis, which identifies associations rather than proving causal mechanisms.