🤖 AI

AgentFairBench: Do LLM Agents Discriminate When They Act?

This paper introduces AgentFairBench, a cost-effective, multi-domain benchmark and methodology for measuring demographic disparities in the actions of LLM agents, revealing that when properly accounting for statistical noise and test arity, advanced models like Claude Haiku 4.5 exhibit no significant discriminatory effects in hiring, lending, or medical triage scenarios.

Original authors: Triveni Morla, Rohith Reddy Bellibaltu, Manpreet Singh, Manmeet Singh Kapoor

Published 2026-06-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Triveni Morla, Rohith Reddy Bellibaltu, Manpreet Singh, Manmeet Singh Kapoor

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a new, super-smart robot assistant. You ask it to hire employees, approve loans, or decide which patients need urgent care. For a long time, we've tested if these robots are "fair" by asking them simple questions like, "Is this person a good candidate?" and grading their written answers.

But this paper argues that grading the words a robot says is like judging a chef only by their recipe book, not by the meal they actually cook. A robot might write a perfectly polite, unbiased recipe but still serve a smaller portion to a specific group of people.

The authors introduce AgentFairBench, a new way to test these robots by watching what they actually do when they make real decisions.

Here is the breakdown of their work using simple analogies:

1. The Problem: The "Recipe" vs. The "Meal"

Old Way (Answer-Level): We ask the robot, "What do you think of this job applicant?" and check if the answer sounds nice.
New Way (Action-Level): We watch the robot actually hire the applicant. Does it give the job to the qualified person, or does it secretly reject them based on their name?
The Analogy: Imagine a judge who always says, "I treat everyone equally" (the answer), but then secretly gives heavier sentences to people with certain last names (the action). AgentFairBench catches the judge in the act of sentencing, not just listening to their speech.

2. The Tool: A "Shadow Twin" Experiment

To test this fairly, the researchers created synthetic profiles (fake resumes, fake loan applications, fake medical records). These profiles are identical in every way—same skills, same money, same symptoms—except for one thing: the name.

The Analogy: Imagine you have two twins, "John Smith" and "Jamal Jones," who are identical in every way. You send their identical resumes to the robot. If the robot hires John but rejects Jamal, the robot is biased.
They tested this across three high-stakes areas: Hiring (getting a job), Lending (getting a loan), and Medical Triage (deciding who gets seen by a doctor first).

3. The "Scaffolding" Test: Does More Thinking Help or Hurt?

The researchers didn't just ask the robot to decide instantly. They tested it in four different "modes" of thinking, like giving the robot different levels of brainpower:

Direct: "Decide now."
Chain-of-Thought: "Think step-by-step before deciding."
Debate: "Have two robot agents argue about the decision."
Tool-Use: "Go look up more info before deciding."

The Big Question: Does making the robot think harder (using more "scaffolding") wash away the bias, or does it accidentally make the bias worse? The authors call this the "Super-additivity" test.

4. The Surprise Finding: The "Noise" Trap

This is the most important part of their discovery. When they first looked at the data, it looked like the robot was biased. The scores for different groups varied.

The Analogy: Imagine you are trying to hear a whisper in a noisy room. You think you heard a word, but it was just the wind.

The researchers realized they were making a math mistake. They were comparing a "six-person crowd's noise" against a "two-person whisper." Because a crowd naturally has more variation than a pair, it looked like the robot was biased, but it was actually just random statistical noise (like static on a radio).
The Fix: They created a new "noise floor" that matched the size of the crowd. When they did this, the "bias" disappeared.

The Result: For the specific robot they tested (a model called claude-haiku-4-5), there was no detectable bias once they fixed the math. The robot acted fairly, and the apparent unfairness was just random chance.

5. The "Tool" Channel

They also discovered a new way bias can hide: Tool Invocation.

The Analogy: Imagine a security guard who stops people with a certain name to ask for extra ID, while letting others walk right through without asking. The guard might let everyone in eventually, but the process was unfair.
The researchers built a metric to catch this. In their test, the robot didn't do this either, but the tool is now ready to catch it if it happens in the future.

6. The "Live Leaderboard"

To keep things honest, they built a public scoreboard (a leaderboard).

The Canary: They hid a secret "canary" (a tiny, unique string of text) in the test questions. If a robot company tries to cheat by memorizing the answers, the canary will show up in their output, and they will get disqualified.
The Cost: They made the test cheap (a few dollars per robot) so anyone can run it, not just big tech companies.

Summary of Claims

We need to test actions, not just words.
We need to be careful with math: Comparing a group of 6 to a group of 2 makes random noise look like bias.
The Pilot Result: The specific robot they tested (claude-haiku-4-5) showed no bias above random noise in hiring, lending, or medical triage.
The Tool: They released a free, open-source toolkit so others can test their own robots and see if they are biased.

Important Note: The authors are very careful to say this is a pilot (a first test) on one robot. They are not saying all robots are fair. They are saying, "Here is a better ruler to measure fairness, and when we used it on this one robot, it passed." The real work begins when the community uses this ruler to test many different robots.

Technical Summary: AgentFairBench

Problem Statement

Large Language Model (LLM) agents are increasingly deployed to perform consequential actions—screening job applicants, recommending credit, and triaging patients—yet current fairness evaluation paradigms remain stuck at the "answer level." Existing benchmarks (e.g., BBQ, StereoSet) grade the textual outputs of models in response to prompts. However, a structural gap exists: a model may produce token-level parity (answering demographic probes evenly) while still executing systematically different actions when those answers are operationalized into decisions, tool calls, and multi-step trajectories. This "masking" of bias means that answer-level fairness does not guarantee action-level fairness. Furthermore, existing agent benchmarks either focus on single domains, lack demographic instrumentation, or fail to account for how agentic scaffolds (e.g., chain-of-thought, tool use) might amplify rather than attenuate bias.

Methodology

The paper introduces AgentFairBench, a multi-domain, action-level benchmark grounded in the Bias Conduction Framework (BCF). The framework models an agent as a tuple of policy, task, memory, and repertoire, proposing that disparity can be masked, preserved, amplified, or generated as it passes through these components.

Core Design Elements

Counterfactual Matched Sets: The benchmark uses synthetic, demographic-neutral profiles where the only variable is a name-coded race $\times$ gender signal (following the Bertrand–Mullainathan tradition). Each set includes six variants (White-M, White-F, Black-M, Black-F, Hispanic-M, Hispanic-F) to isolate the effect of the demographic cue.
Three Regulatory Domains:
- Hiring: Anchored to EEOC adverse impact guidelines and NYC Local Law 144. Outputs: Advancement decision (boolean) and quality score (0–100).
- Lending: Anchored to ECOA/Regulation B. Outputs: Approval decision (boolean) and APR tier (1–5).
- Medical Triage: Anchored to AI governance frameworks for high-risk health AI. Outputs: Escalation decision (boolean) and acuity score (1–5).
Agent Scaffolds (BCF Entry Loci): The benchmark evaluates agents across four levels of increasing agency to test the BCF's "Super-additivity" proposition:
- C0: Direct prompting (zero-shot).
- C2: Chain-of-thought (reasoning trace).
- C3: Multi-agent deliberation (reviewer + decision agent).
- C4: Tool-augmented (information gathering before decision).
Metrics:
- Counterfactual Flip Rate (CFR): The fraction of matched sets where the binary decision is not unanimous across the six demographic conditions.
- Mean Absolute Score Difference (MASD): The mean spread between the maximum and minimum scores assigned across the six conditions.
- Action-Rate Disparity: Differences in positive decision rates between groups.
- Tool-Invocation Disparity ( $\Delta_{tool}$ ): The spread in tool-invocation rates across groups (C4 only).
Statistical Rigor: The evaluation harness uses BCa bootstrap confidence intervals, paired McNemar and Wilcoxon tests, and Benjamini–Hochberg FDR correction. Crucially, it introduces an arity-matched noise floor: comparing a 6-group score spread against a 2-run pairwise noise difference is shown to overstate disparity by $\sim$ 2.4 $\times$ . The benchmark corrects this by comparing the observed spread against a 6-group noise spread derived from test-retest runs.

Implementation

AgentFairBench is released as a pip-installable Python package (agentfairbench) with a NumPy-only dependency footprint. It includes a live leaderboard with a held-out private split and a "contamination canary" to prevent gaming. The cost to run a full evaluation is estimated at single-digit dollars per model.

Key Results (Pilot Study)

The authors conducted a pilot on claude-haiku-4-5 (864 decisions across 36 profiles, 6 demographic conditions, and 4 scaffolds, plus a test-retest run).

Methodological Finding: A naive comparison of the 6-group MASD against a 2-run pairwise noise floor suggested a disparity signal roughly 2.4 $\times$ the noise floor. However, when corrected for statistic arity (comparing the 6-group spread against a 6-group noise floor), the observed disparity was 0.83 $\times$ the noise floor.
Null Result: Against the arity-matched noise floor and using omnibus group tests, zero of the 120 pairwise contrasts and zero of the 9 omnibus contrasts survived multiple-comparison correction. The pilot found no demographic effect distinguishable from sampling noise for this model.
Instrument Validity: A "planted-bias" property test (injecting a -25 point penalty for a specific group) confirmed the harness successfully recovers known disparities, proving the null result is not due to an insensitive instrument.
Scaffold Effects: No evidence of scaffold amplification (BCF Proposition P3) was found; apparent increases in disparity across scaffolds were within the noise floor.
Tool Disparity: While $\Delta_{tool}$ was measurable, the pilot counts were too small to draw conclusions beyond small-sample noise.

Significance and Claims

The paper positions AgentFairBench not as a definitive statement on the fairness of all LLMs, but as a sound, sensitive, and adoption-ready instrument for measuring action-level disparity.

Closing the Measurement Gap: It shifts the focus from "what models say" to "what models do," addressing the structural gap where token-level parity coexists with action-level disparity.
Methodological Correction: It identifies and corrects a specific statistical error in disparity measurement (arity mismatch), demonstrating that naive comparisons can falsely identify noise as bias.
Operationalizing Theory: It provides the first empirical instantiation of the Bias Conduction Framework, testing propositions regarding masking (P2) and super-additivity (P3) across multiple domains and scaffold depths.
Reproducibility and Anti-Gaming: By providing a low-cost, open-source harness with a private test split and contamination canaries, it enables the community to scale confirmatory studies across diverse model families, moving beyond self-reported results.

The authors explicitly state that the pilot's null result on claude-haiku-4-5 is an honest finding at a specific scale ( $n=12$ matched sets per cell) and does not generalize to all agents. The benchmark is designed to facilitate the powered, multi-model studies required to detect real effects if they exist.