Whose fairness? Structural concentration in AI bias research
This paper reveals that AI bias research is structurally concentrated within a narrow set of US-dominated institutions and authors, raising concerns that the resulting fairness definitions and mitigation methods may lack generalizability to diverse global populations and contexts.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine the world of Artificial Intelligence (AI) as a massive, bustling kitchen where chefs are constantly inventing new recipes to solve problems in healthcare, law, and daily life. But before they can cook, they need to agree on what "fair" tastes like. This paper is a behind-the-scenes look at who is actually standing in the kitchen, who gets to write the recipes, and whether those recipes work for everyone.
Here is the story of the paper, broken down into simple parts:
1. The Problem: Who is in the Kitchen?
The authors wanted to know: Whose fairness are we talking about?
For years, researchers have been trying to fix AI bias (when computers make unfair decisions). They've built tools and rules to measure this. But they never stopped to ask: Who is making these rules?
The paper acts like a headcount of the entire "AI Fairness" research community. They looked at 692 research papers published between 2015 and 2025. They treated these papers like a map to see where the researchers live, which schools or companies they work for, and who they talk to.
2. The Main Finding: A Very Small Group is Running the Show
The results show that the "AI Fairness" kitchen is heavily concentrated in a few places.
- The United States is the Head Chef: The US dominates the field. They wrote nearly half of all the papers that set the "rules" for fairness. If you look at who leads the research teams (the first authors), the US is responsible for almost 50% of them.
- The "Global South" is Missing: Countries in South Asia, Africa, and Latin America are barely in the room. Even India, which has a huge tech workforce, is surprisingly quiet in this specific conversation.
- The "Inner Circle": The researchers mostly talk to each other within a small Western club (US, UK, Canada, Germany). It's like a dinner party where everyone knows everyone, but the rest of the world isn't invited.
3. The "Foundation" Layer: The Most Important Room is the Most Exclusive
The researchers divided the papers into five different "rooms" or topics:
- General Fairness & Bias Mitigation (The rulebook)
- Health AI
- Large Language Models (LLMs)
- Recommender Systems (Like Netflix or YouTube suggestions)
- Graph-Based Fairness
Here is the critical twist: The most important room is the "General Fairness" room. This is where the definitions, benchmarks, and basic rules are invented. These rules are then copied and used by the other four rooms (like Health or LLMs).
- The Danger: The "General Fairness" room is the most concentrated of all. The US leads this room more than any other.
- The Metaphor: Imagine a company where the people designing the blueprints for all the buildings live in one specific neighborhood. They design the doors and windows based on what fits their neighborhood. Then, they send those blueprints to build houses in the jungle, the desert, and the arctic. The doors might not fit, or the windows might let in too much rain, because the original designers never visited those places.
4. The "Popularity Contest" (Citations)
The authors also looked at who gets the most "high-fives" (citations) from other scientists.
- A few papers get thousands of citations, while most get very few.
- Interestingly, just because a country writes more papers (like China or India), it doesn't mean those papers get the most "high-fives."
- However, the papers that do get the most attention are still mostly from the US and UK, and they are mostly about the "General Fairness" rules.
5. Why This Matters
The paper argues that because the "rule-makers" are mostly from the US and Western Europe, the rules they create might not work for everyone.
- Cultural Blind Spots: What looks "fair" in New York might look unfair in Kathmandu or Nairobi. For example, the paper mentions that in some cultures, avoiding eye contact is a sign of respect, while in the West, it might be seen as a lack of confidence. If an AI is trained only on Western rules, it might misjudge people from other cultures.
- The Risk: We are building AI systems for the whole world, but the "safety manuals" are being written by a tiny, unrepresentative group. This means the solutions might not work for the people who need them most.
6. What They Did to Help
The authors didn't just point out the problem; they built a live map (an interactive atlas) that anyone can use to watch this field grow. This allows the community to see in real-time if the kitchen is becoming more diverse or if it's still the same small group running the show.
Summary
The paper is a warning label on the AI industry. It says: "We are trying to build fair AI for everyone, but the people designing the concept of 'fair' are mostly from one part of the world. Until we invite more voices from different cultures and countries to the table, our definition of fairness will remain incomplete and potentially harmful to those outside the 'inner circle'."
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.