💻 computer science

Picturing Perceptions: An Open-Source Toolkit to Uncover Bias in Humans and Machines

This paper introduces PictoPercept, an open-source toolkit that uses visual forced-choice comparisons against real-world labor data to measure and reveal significant, often divergent, biases in both human judgment and AI systems regarding earnings across different demographic groups.

Original authors: Saurabh Khanna, Zhijun Chen, Chei Billedo, Jiayi Yan, Irene van Driel, Alex Barco Martelo, Hugo Moreda Cartagena, Haizea Gonzalez Atorrasagasti, Markel Adanez Perez, Daniela An, Qianyi Wang, Xinkangru

Published 2026-06-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Saurabh Khanna, Zhijun Chen, Chei Billedo, Jiayi Yan, Irene van Driel, Alex Barco Martelo, Hugo Moreda Cartagena, Haizea Gonzalez Atorrasagasti, Markel Adanez Perez, Daniela An, Qianyi Wang, Xinkangrui Gao, Lauren Taylor, Olga Eisele, Sindy Sumter

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to guess who earns more money just by looking at their face. Would you guess correctly? Or would your brain rely on old stereotypes, like a broken GPS that keeps sending you down the wrong road?

This paper introduces a new tool called PictoPercept (short for "Picturing Perceptions") designed to test exactly that. It's like a "bias detector" that works for both humans and computers.

Here is the story of what the researchers did and what they found, explained simply:

The Problem: Old Tools Were Broken

For years, scientists tried to measure bias using surveys (asking people directly) or reaction-time tests (measuring how fast people click buttons).

Surveys are like asking someone, "Are you racist?" Most people will say "No" because they want to look good. It's like asking a thief if they steal; they'll lie.
Reaction tests are like a video game where you have to sort words quickly. But computers don't have "reaction times" in the same way humans do, so these tests couldn't be used to check if AI was biased.
The Missing Map: None of these old tools checked if people's guesses matched reality. They just measured if people preferred Group A over Group B, without asking, "Is Group A actually richer or poorer in the real world?"

The Solution: A Visual "Forced Choice" Game

The researchers built PictoPercept, an open-source toolkit (meaning anyone can use it for free).

The Game: You see two faces on a screen. The prompt asks, "Who is more likely to have higher earnings?" You have to pick one. You can't say "I don't know."
The Trick: The faces are carefully matched so they look equally attractive and trustworthy. The only difference is their race and gender.
The Map: The researchers compared your choices against real data from the U.S. Bureau of Labor Statistics. If the data says Asian men earn the most, but you keep picking Latino men as the highest earners, the tool calculates that as a "bias score."

They tested this on 283 real American adults and also fed the exact same pictures to GPT-5, a powerful AI model.

The Big Surprises (The Results)

1. The "Model Minority" Blind Spot
The biggest shock was about Asian Americans.

The Reality: According to government data, Asian Americans are actually the highest-earning group in the U.S.
The Perception: People dramatically underestimated their earnings. Even when shown faces of Asian men and women, people guessed they earned less than they actually do.
The Irony: Even Asian participants in the study underestimated their own group's earnings! It's as if the whole country has a blind spot to this group's success, perhaps because they aren't seen enough in leadership roles or media.

2. The "Ingroup" Myth
We often think people always favor their own group (e.g., White people favoring White people, Black people favoring Black people).

The Reality: This wasn't true for everyone.
- White men did show favoritism, overestimating their own group's earnings.
- Asian participants, however, did the opposite. They significantly underestimated their own group. They didn't just ignore the bias; they actively thought their own group earned less than they did.
- Other groups (like Black and Latino participants) were surprisingly accurate about their own groups, showing no strong favoritism or bias.

3. The AI is Even More Biased
When they tested GPT-5 (the AI), the results were scary.

The AI didn't just mimic human bias; it amplified it.
The AI was extremely bad at judging women. It systematically thought all female groups earned much less than they actually do.
It thought all male groups earned much more.
While humans made mistakes, the AI's errors were huge, consistent, and extreme. It's like a student who not only gets the answer wrong but gets it wrong by a factor of ten.

Why This Matters

The paper argues that bias isn't just about "hating" a group; it's about having a distorted map of reality.

If you think a group earns less than they do, you might treat them unfairly in hiring or lending.
The tool shows that our brains (and our computers) are often running on outdated software that doesn't match the real world.

The Takeaway

PictoPercept is a new, open-source way to check if our perceptions match reality. It found that:

We are blind to the high earnings of Asian Americans.
We don't always favor our own groups; sometimes we underestimate ourselves.
AI models like GPT-5 are currently much more biased than humans, especially against women.

The authors suggest that before we let AI make big decisions (like who gets a loan or a job), we need tools like this to check if the AI is looking at the world through a funhouse mirror.

Technical Summary: Picturing Perceptions (PictoPercept)

Problem Statement
Bias in human judgment and artificial intelligence (AI) systems presents critical challenges in consequential domains such as hiring, lending, and criminal justice. Existing methodologies for measuring bias suffer from fundamental limitations:

Human Bias Measurement: Traditional survey-based tools are text-based, explicit, and vulnerable to social desirability effects. They often require separate instruments for different dimensions (e.g., gender, ethnicity), making intersectional assessment inefficient. Furthermore, they lack grounding in demographic reality, failing to distinguish between accurate perceptions of existing inequalities and distorted perceptions.
AI Bias Measurement: Current AI fairness research relies on algorithmic audits and dataset analyses, which are incompatible with human bias research methods (like surveys or the Implicit Association Test). The Implicit Association Test (IAT), while effective for human implicit bias via reaction times, cannot be applied to AI systems which lack human-like reaction times or operate via different computational mechanisms.
The Gap: There is a lack of a unified, open-source framework capable of measuring bias in both humans and machines simultaneously, grounded in population-level benchmarks rather than abstract relative preferences.

Methodology: PictoPercept
The authors introduce PictoPercept, an open-source toolkit designed to measure bias through visual forced-choice comparisons grounded in population-level benchmarks.

Stimuli: The study utilizes 68 forced-choice trials (64 experimental + 4 attention checks) using pairs of normed facial photographs from the Chicago Face Database (CFD). Images are carefully balanced on non-focal covariates (attractiveness, trustworthiness, age, dominance) to isolate demographic effects.
Task: Participants (both humans and AI) view pairs of faces and answer the prompt: "Which person is more likely to have higher personal earnings?" Participants are instructed to rely on initial impressions to capture System 1 (automatic) processing.
Benchmarking: Unlike traditional measures that yield relative indices (e.g., preference for Group A over B), PictoPercept compares selections against actual U.S. Bureau of Labor Statistics (BLS) data for median weekly earnings across eight demographic categories (Male/Female × Asian/Black/Latino/White).
Bias Calculation:
- Reality-Based Bias Score: Calculated as the difference between the observed selection probability ( $P_{observed}$ $P_{o b ser v e d}$ ) and the expected selection probability ( $P_{expected}$ $P_{e x p ec t e d}$ ) derived from actual earnings data.
  - $P_{expected}(i|i, j) = \frac{Earnings_i}{Earnings_i + Earnings_j}$
  - $Bias_{reality}(i) = P_{observed}(i) - P_{expected}(i)$
- Positive scores indicate overestimation of a group's earnings relative to reality; negative scores indicate underestimation.
Participants:
- Humans: A nationally representative sample of 283 U.S. adults recruited via Prolific Academic.
- AI: The GPT-5 vision-language model (OpenAI) was tested using the identical 64 experimental image pairs. The model was run 100 times per trial to ensure stability, with the majority response used for analysis.

Key Results
The study validated PictoPercept as a psychometrically sound instrument (split-half reliability $r_{sb} = 0.779$ ) and revealed three primary findings:

Systematic Misperception of Earnings:
- Participants dramatically underestimated Asian American earnings, despite this group having the highest actual earnings in U.S. labor statistics (Asian Male: $1,759; Asian Female: $1,363).
- Conversely, participants overestimated earnings for Latino males and White males.
- Black females were also significantly underestimated.
Heterogeneity of Ingroup Favoritism:
- Ingroup favoritism was not universal. White male participants showed clear ingroup bias (overestimating their own group's earnings).
- In contrast, Asian participants demonstrated significant negative ingroup bias, significantly underestimating the earnings of both Asian males and Asian females. This suggests a lack of awareness of their own group's economic success even within the community.
- White females, Black participants, and Latino participants showed near-accurate perceptions of their own groups or minimal ingroup bias.
AI Bias Amplification:
- GPT-5 exhibited substantially stronger biases than human participants.
- The AI showed extreme systematic underestimation of all female groups and overestimation of all male groups.
- The magnitude of AI bias, particularly regarding gender, exceeded human averages, with Latino males receiving the most extreme overestimation (Bias score > 1.0).

Significance and Contributions
The paper claims several theoretical and practical contributions:

Methodological Advancement: PictoPercept offers a unified framework for assessing bias in both humans and AI. By using visual stimuli and forced-choice tasks, it captures implicit, automatic processes (System 1) while reducing social desirability effects inherent in text-based surveys.
Intersectionality: The tool allows for the simultaneous assessment of multiple intersecting dimensions (gender and ethnicity) without the need for separate testing blocks, addressing the limitations of factorial designs like the IAT.
Grounding in Reality: By anchoring measurements to objective population statistics (BLS data), the framework distinguishes between accurate perception of inequality and biased distortion. This allows for the calculation of bias direction (over/underestimation) and magnitude relative to ground truth.
Open Science: As an open-source toolkit, PictoPercept promotes reproducibility and cumulative knowledge development, overcoming the limitations of proprietary, closed-source measurement tools.
Implications for AI: The findings suggest that state-of-the-art AI models may not only mirror but amplify human stereotypes, particularly regarding gender. This challenges the assumption that AI provides a neutral alternative to human judgment in high-stakes decision-making.

The authors conclude that PictoPercept enables a more efficient, integrated, and empirically grounded approach to bias measurement, revealing systematic misperceptions that diverge from demographic reality and highlighting the need for rigorous auditing of AI systems before deployment in consequential domains.