💬 NLP

In-Situ Behavioral Evaluation for LLM Fairness, Not Standardized-Test Scores

This paper argues that LLM fairness should be evaluated through in-situ multi-agent conversational behavior rather than standardized-test benchmarks, introducing the MAC-Fairness framework to reveal stable, model-specific behavioral signatures that are obscured by the structural unreliability of traditional prompt-based testing.

Original authors: Zeyu Tang, Sang T. Truong, Deonna Owens, Shreyas Sharma, Yibo Jacky Zhang, Brando Miranda, Sanmi Koyejo

Published 2026-05-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zeyu Tang, Sang T. Truong, Deonna Owens, Shreyas Sharma, Yibo Jacky Zhang, Brando Miranda, Sanmi Koyejo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: Stop Testing, Start Watching

Imagine you want to know if a group of people is fair and unbiased. The current way we test Large Language Models (LLMs) is like giving them a multiple-choice pop quiz. We ask them questions like, "Who is less creative: a 50-year-old or a 28-year-old?" and we grade their answers. If they pick the "wrong" person, we say they are biased.

The authors of this paper argue that this "pop quiz" method is broken. They say it's like judging a driver's safety by how well they can fill out a written test about traffic laws, rather than actually watching them drive in real traffic.

Part 1: Why the "Pop Quiz" is Broken

The researchers discovered that the results of these fairness quizzes are incredibly unstable. It's not the model's "heart" that changes; it's the format of the test.

Think of it like a magic trick. If you ask a model a question with a specific format (e.g., "Choose A, B, or C"), it might give a fair answer. But if you change the format slightly (e.g., "Choose 1, 2, or 3" or "Write your answer before your reason"), the model's answer can flip completely.

The Analogy: Imagine a student taking a math test. If the teacher writes the numbers in blue ink, the student gets an A. If the teacher writes them in red ink, the student gets an F. The student's math skills didn't change; the way the question was presented changed the score.
The Finding: The authors tested 11 different models on standard fairness benchmarks. They found that simply changing how the answer choices looked (the font, the order, or whether the model had to explain itself first) caused the fairness scores to swing wildly. Sometimes a model looked "fair" on one version of the test and "deeply biased" on another. This means the test scores are mostly measuring how the model reacts to the format, not its actual fairness.

Part 2: The New Solution – "MAC-Fairness"

Instead of a pop quiz, the authors built a playground simulation called MAC-Fairness.

Instead of asking the model, "Are you biased?", they put the model in a multi-round conversation with another AI agent. They treat the conversation itself as the test.

How the Playground Works:

The Setup: Two AI agents are talking. One is a "Baseline" agent (just a standard AI). The other is an "Identity" agent (the model being tested, but told it is a specific type of person, like "a Black physician" or "an older teacher").
The Game: They are given a controversial topic (like the creative design job example above). They discuss it over several rounds.
The Observation: The researchers don't care what the final answer is. They watch how the agents behave.
- Position Persistence: If the "Identity" agent disagrees with the other person, does it stubbornly stick to its own view, or does it easily change its mind?
- Peer Receptiveness: If the "Identity" agent reveals its background (e.g., "I am a Black physician"), does the other agent listen to it more or less?

Part 3: What They Found in the Playground

When they watched these conversations (analyzing 8 million of them!), they found stable patterns that the pop quizzes missed.

1. The "Stubbornness" Effect (Self-Perspective)
When an AI is assigned a specific identity (like being a specific race, gender, or age), it tends to become more stubborn.

The Metaphor: Imagine you are playing a game. If you are just "Player 1," you might easily change your mind if your friend suggests a better move. But if you are told, "You are the Captain of the Team," you might hold your ground much tighter, even if your friend has a good point.
The Result: The models became significantly more resistant to changing their minds when they were assigned a demographic identity (like "Black" or "Older") compared to when they had no identity at all. This happened consistently across different models and different topics.

2. The "Listening" Effect (Other-Perspective)
When the models knew who their conversation partner was, their behavior changed based on who that partner was.

The Metaphor: Imagine you are in a group discussion. If you know your partner is a specific type of person, you might listen to them more carefully (or less).
The Result: The models showed different levels of receptiveness depending on the peer's identity. For example, some models were more likely to change their minds when the peer was revealed to be "Black" compared to "White," or "Female" compared to "Male."

The Conclusion

The paper argues that we need to stop relying on standardized test scores (the pop quiz) because they are too easily tricked by the way the question is written.

Instead, we should use in-situ behavioral evaluation (the playground). By watching how models behave in a natural, multi-turn conversation where identities are swapped, we can see their true "behavioral signatures." These signatures—like how stubborn a model gets when it has an identity—are stable and real, whereas the test scores are just noise caused by the test format.

In short: Don't judge a fish by how well it climbs a tree (the test score); watch how it swims in the water (the conversation).

Technical Summary: In-Situ Behavioral Evaluation for LLM Fairness, Not Standardized-Test Scores

Problem Statement

The current paradigm for evaluating Large Language Model (LLM) fairness relies heavily on standardized-test Q&A benchmarks. In this approach, models are probed with questions referencing sensitive attributes (race, gender, age), and aggregate bias scores are derived from their answers. The authors argue that this paradigm is structurally unreliable.

Empirical analysis reveals that surface-level prompt construction choices—factors entirely orthogonal to the fairness question itself, such as the formatting of answer choices (e.g., lettered vs. bulleted) or the ordering of JSON output fields (answer-first vs. rationale-first)—dominate the variance in fairness scores. These arbitrary choices can:

Account for the majority of score variance, even after controlling for decoding stochasticity.
Shift fairness conclusions in both direction and magnitude (e.g., a model appearing biased in one direction under one prompt format and unbiased or biased in the opposite direction under another).
Induce severe discordance in model rankings, where a model's position relative to others fluctuates wildly based on prompt formatting rather than intrinsic model behavior.

Consequently, the standardized-test paradigm fails to distinguish between genuine fairness signals and measurement artifacts, potentially leading to evaluations that are "fair on the sheet but biased in the street."

Methodology: MAC-Fairness

To address these limitations, the authors propose MAC-Fairness, a multi-agent conversational framework that shifts the evaluation focus from static question-answering to in-situ behavioral measurement.

Core Framework

Conversation as Instrument: Instead of treating the model as a test-taker, the framework uses multi-round dialogue as the measurement instrument. Standardized-test questions are repurposed merely as "conversation seeds" to initiate dialogue, not as the direct evaluation metric.
Agent Configuration: Agents are instantiated from the same underlying model but configured with specific attributes:
- Demographics: Social categories (e.g., Black, White, Older, Young).
- Persona: Professional roles (e.g., Physician, Teacher).
- Instantiation: Whether the agent is the person ("human") or assists the person ("AI").
- Baseline Agent: A control agent with null demographics and persona, instantiated as AI.
Experimental Design:
- Self-Perspective: Examines how an agent's assigned identity affects its position persistence (how often it holds its initial stance) when interacting with a baseline agent.
- Other-Perspective: Examines how an agent's receptiveness (willingness to shift its stance) changes based on the revealed identity of its peer.
- Controlled Variations: The framework systematically varies prompt structures, identity assignments, and disclosure conditions (revealed vs. anonymous peers) across 8 million conversation transcripts.

Behavioral Metrics

The primary metric is the shift rate ( $\lambda$ ), defined as the probability of an agent changing its selected answer to match a peer's previous answer, conditioned on prior disagreement.

Position Persistence: Measured by how often an identity agent fails to shift toward a baseline agent's position.
Peer Receptiveness: Measured by how often a baseline agent shifts toward an identity agent's position when the peer's identity is revealed.

Key Results

1. Instability of Standardized-Test Paradigms

Evaluation across 11 open-weight models (3B to 70B parameters) and two major benchmarks (BBQ and Difference-Awareness) confirmed the structural instability:

Variance Dominance: Prompt construction choices explained a significant proportion of score variance ( $\eta^2$ ), with many results remaining statistically significant after Benjamini-Hochberg correction.
Directional Shifts: For models like GLM 4.7 Flash, the ratio between extreme bias scores within a single category reached 57×. Several categories exhibited "sign flips," where the model appeared biased in opposite directions depending on the prompt format.
Ranking Discordance: Model rankings were highly unstable; a model could traverse the full range of rankings (from best to worst) solely based on surface-level prompt variations.

2. Stable In-Situ Behavioral Signatures

In contrast to the volatility of standardized tests, MAC-Fairness revealed stable, model-specific behavioral patterns that replicated across different benchmarks (BBQ, Discrim-Eval, Difference-Awareness):

Self-Perspective (Position Persistence):
- Demographic Assignment Increases Stubbornness: Assigning any demographic identity (e.g., Black, White, Older) generally increased position persistence (made agents more "stubborn") compared to a null demographic assignment.
- Asymmetry in Disadvantaged Groups: Agents assigned disadvantaged demographics (Black, Older, Female) consistently showed greater position persistence than their paired counterparts (White, Young, Male) across multiple models and benchmarks. This effect was particularly pronounced in Ministral 3 and Qwen3 models.
- Instantiation Effects: Race-related effects persisted under both "human" and "AI" instantiations, whereas age-related effects were primarily observed under "human" instantiation.
Other-Perspective (Peer Receptiveness):
- Differential Receptiveness: The baseline agent's receptiveness varied depending on which identity was revealed. Revealing "Black" (vs. "White") or "Female" (vs. "Male") increased the baseline agent's receptiveness in specific models (notably GLM 4.7 Flash).
- Modulation by Instantiation: The effect of identity revelation was modulated by whether the peer was instantiated as "human" or "AI." For example, in Phi-4 Mini, human instantiation dampened receptiveness when the peer was revealed as "Older."
- Identity vs. Null: Unlike the self-perspective, simply revealing any demographic identity did not robustly change receptiveness compared to a null reveal; the specific demographic content mattered more than the mere presence of an identity.

Significance and Claims

The paper claims that LLM fairness should be evaluated through in-situ conversational behavior rather than standardized-test scores.

Paradigm Shift: The authors argue for moving away from static, score-based evaluations that are vulnerable to prompt artifacts toward frameworks that capture fairness through dynamic behavioral interactions.
Generalizability: The identified behavioral signatures (e.g., increased persistence for disadvantaged identities) are claimed to be stable across benchmarks with different fairness targets, question structures, and methodologies. This cross-benchmark consistency suggests these patterns reflect intrinsic behavioral properties of the models rather than artifacts of specific evaluation designs.
Measurement Validity: By treating the conversation itself as the instrument, MAC-Fairness aims to provide a more grounded and reliable assessment of how models behave when identity is integrated into natural interaction, offering evidence that the standardized-test paradigm cannot provide.

The work does not propose a new "fairness score" to replace existing ones but rather a methodological shift to identify stable behavioral patterns that are robust to the noise inherent in current evaluation protocols.