What Is the Point of Equality in Machine Learning Fairness? Beyond Equality of Opportunity

This paper argues that the prevailing focus on distributive equality in machine learning fairness is insufficient because it fails to address representational harms and structural inequality, proposing instead a comprehensive egalitarian framework that integrates both distributive and relational equality to tackle the full spectrum of harms perpetuated by ML systems.

The Gist

Imagine you are building a giant, automated robot chef to serve food to a massive city. You want this robot to be fair. But what does "fair" actually mean in this context?

This paper argues that most people trying to make AI fair are looking at the problem through the wrong lens. They are mostly focused on Distributive Equality (making sure everyone gets an equal slice of the pie), but they are ignoring Relational Equality (making sure everyone is treated with the same respect and dignity at the table).

Here is a simple breakdown of the paper's argument, using everyday analogies.

1. The Old Way: Just Counting the Slices (Distributive Equality)

For a long time, researchers have tried to fix unfair AI by looking at numbers. They ask: "Does the robot give the same number of job interviews to men and women?" or "Does it reject the same percentage of loan applications from different racial groups?"

This is called Distributive Equality. It's like a teacher handing out cookies. If the teacher gives 50 cookies to the boys and 50 to the girls, the teacher thinks, "I am fair!"

• The Good: This works well for Allocative Harms. These are situations where the AI steals resources. For example, if an AI hides housing ads from minority groups, it's stealing their "cookies" (opportunities). Fixing the numbers helps here.
• The Bad: This fails when the problem isn't about how many cookies you get, but how you are treated while getting them.

2. The Missing Piece: The Tone of the Voice (Relational Equality)

The paper argues that AI causes a second, deeper kind of harm called Representational Harm. This isn't about losing a job or a loan; it's about being insulted, erased, or mocked by the machine.

Imagine the robot chef doesn't just serve you a smaller portion; it serves you a tiny, burnt cookie while saying, "You people only deserve this because you're lazy," or it refuses to serve you at all because it thinks you don't exist.

This is Relational Equality. It asks: "Does the AI treat me as an equal human being, or does it treat me like a second-class citizen?"

The paper identifies four ways AI messes this up:

1. Stereotyping: The AI thinks all nurses are women and all doctors are men. It's not just a math error; it's reinforcing the idea that women aren't leaders.
2. Demeaning: The AI generates stories where people of a certain race are always the criminals, while others are always the heroes. It's like a bully constantly calling you names.
3. Erasing: The AI's face recognition software can't see Black women. It's like the robot chef pretending you aren't even in the room.
4. Reifying: The AI forces everyone into rigid boxes (like "Male" or "Female" only) and punishes anyone who doesn't fit, ignoring that human identity is fluid.

The Core Problem: You can't fix "being insulted" just by giving the victim a bigger cookie. You have to change the attitude of the robot. Distributive equality (counting cookies) cannot explain why it's wrong to be called a "gorilla" by an image tagger.

3. The Solution: A Two-Pronged Approach

The author proposes a Multifaceted Framework. We need both lenses to fix AI.

• Lens A (Distributive): Make sure the AI distributes opportunities fairly (no stealing jobs or loans).
• Lens B (Relational): Make sure the AI treats everyone with equal respect and doesn't reinforce social hierarchies (no insults, no erasing, no stereotypes).

Why not just use Lens B?
Some philosophers say, "If we treat everyone with equal respect, the resources will follow." The author says, "No, that's too simple." If you ignore the resource distribution, you might have a polite robot that still starves the poor. If you ignore the respect, you might have a robot that gives everyone a cookie but calls them "worthless" while doing it. We need both.

4. Why "Technical Fixes" Aren't Enough

The paper critiques current attempts to fix AI. Many developers try to fix "Representational Harm" by just redistributing the output.

• Example: If an AI generates too many pictures of white men as doctors, the fix is to force it to generate 50% women and 50% men.

The Flaw: This is like a bully who stops calling you names but still pushes you around. Just swapping the numbers doesn't fix the deep-seated prejudice.

• If you force an AI to show a Black Pope or an Asian Nazi soldier just to "balance the numbers," you aren't being fair; you're being historically inaccurate and disrespectful.
• True fairness isn't just about the ratio of images; it's about the context and the power dynamics.

5. How Do We Actually Fix It? (The Roadmap)

The paper suggests we can't just tweak the code. We have to change the whole process of building AI, involving real people:

1. Community Data Collection: Don't just scrape the internet for data. Ask the communities being harmed to help build the data. (e.g., Let African communities help train translation tools for their own languages).
2. Critical Reflection: Developers and users need to stop and think: "What biases are we bringing in? Who is being hurt?"
3. User Agency: Don't hide the AI's mistakes. Show users how the AI works so they can spot the bias. Give them tools to say, "Hey, this story is wrong."
4. Iterative Process: Fixing AI isn't a one-time patch. It's a continuous conversation. We need to keep listening to the people being harmed and adjust the system as society changes.

The Big Takeaway

Making AI fair isn't just about balancing the math. It's about human dignity.

If an AI system is "fair" only in the sense that it gives everyone an equal chance to fail, but it does so while insulting their identity, it is not truly fair. We need to build systems that not only distribute resources equally but also treat every human being as an equal partner in society, not as a stereotype or a statistic.

Technical Summary

1. Problem Statement

The field of Machine Learning (ML) fairness has traditionally focused on developing statistical metrics (e.g., demographic parity, error rate parity) to measure bias. However, there is a critical gap in the normative foundation of these efforts: Why is ML unfairness morally wrong?

• Current Limitation: Most existing research relies exclusively on Distributive Equality (DE), specifically the concept of Equality of Opportunity (EOP). This framework views unfairness as the unequal distribution of benefits (e.g., jobs, loans) or burdens (e.g., false positives).
• The Gap: While DE effectively addresses allocative harms (denial of resources), it fails to adequately explain the moral wrongness of representational harms (stereotyping, demeaning, erasing, and reifying social groups). DE treats social relations as "distributable goods," which is conceptually flawed when addressing issues of social hierarchy, status, and cultural imperialism.
• Core Question: How can we construct a normative framework that addresses both the unequal distribution of resources and the reinforcement of social hierarchies by ML systems?

2. Methodology

The paper employs a normative philosophical analysis grounded in critical social and political philosophy to re-evaluate the ethical foundations of ML fairness.

• Theoretical Frameworks:
  • Distributive Egalitarianism (DE): Analyzed through the lens of Barocas, Hardt, and Narayanan (BHN) and the concept of Equality of Opportunity.
  • Relational Egalitarianism (RE): Drawing heavily on the work of Elizabeth Anderson (on social hierarchies, esteem, standing, and domination) and Iris Marion Young (on structural injustice and the "five faces of oppression").
  • Nancy Fraser's Justice Theory: Utilized to synthesize "redistribution" (DE) and "recognition" (RE), rejecting the dichotomy that treats them as mutually exclusive.
• Analytical Approach:
  • The author categorizes ML harms into Allocative Harms (opportunity/economic loss) and Representational Harms (stereotyping, demeaning, erasing, reifying).
  • The paper critiques current technical "fixes" (e.g., prompt engineering, data augmentation) for reducing relational equality to mere distributional balancing.
  • It proposes a Multifaceted Egalitarian Framework that integrates DE and RE to address the full spectrum of structural inequality.

3. Key Contributions

A. Theoretical Contribution: The Multifaceted Egalitarian Framework

The paper proposes a novel framework that integrates Distributive Equality (DE) and Relational Equality (RE) as co-equal pillars of ML fairness.

• DE addresses Allocative Harms: It ensures the equal distribution of opportunities and resources (e.g., preventing job algorithm bias against women).
• RE addresses Representational Harms: It ensures that people relate as equals, challenging social hierarchies where groups are stigmatized (esteem), marginalized (standing), or dominated.
• Synthesis: The framework argues that structural inequality manifests in both maldistribution (DE failure) and misrecognition (RE failure). A robust ethical foundation must address both simultaneously, as they often reinforce each other.

B. Critique of Current Technical Interventions

The paper provides a critical technical analysis of current "fairness" interventions, particularly in generative AI (T2I, LLMs):

• The "Equal Representation" Fallacy: Current methods often try to solve representational harm by forcing demographic parity (e.g., 50% male/50% female outputs). The author argues this reduces Relational Equality to a Distributive problem.
• Limitations of Technical Fixes:
  • Substitution: Swapping a "Black criminal" for a "White criminal" does not address the racialization of crime.
  • Removal: Simply removing negative terms can reinforce stereotypes by creating asymmetry (e.g., refusing to generate content for Muslim names while allowing it for others).
  • Positive Bias: Increasing "positive" associations (e.g., women = excellence) can reinforce neoliberal narratives that ignore structural barriers.
  • Context Ignorance: Diversifying outputs without historical context (e.g., generating an "Asian Nazi soldier") creates disrespectful and historically inaccurate results.

C. Practical Pathways for Implementation

The paper outlines four non-technical and socio-technical pathways to implement the framework across the ML pipeline:

1. Community-Centered Participatory Data Collection: Moving beyond extraction to empower marginalized communities (e.g., the Masakhane project) to co-create datasets, ensuring lived realities are represented and fostering relational equality through agency.
2. Critical Reflection for Practitioners and Users: Implementing anticipatory harm assessment workflows (e.g., simulating stakeholder experiences) and fostering critical digital literacy among users to recognize encoded biases.
3. User Agency-Enhancing Design:
   • Seamful Design: Exposing the "seams" of the system (e.g., showing contradictory data sources) to break the illusion of neutrality.
   • Counter-Narratives: Presenting stories that challenge dominant scripts and center marginalized perspectives, rather than just removing negative ones.
4. Iterative Harm Mitigation: Treating fairness as a continuous, iterative process involving user feedback loops and acknowledging that technical fixes alone cannot solve structural societal oppression.

4. Results and Analysis

• Diagnosis: The paper demonstrates that relying solely on DE leads to an incomplete ethical justification. It cannot explain why it is wrong for an AI to depict a group as "subhuman" or "inferior" if the resource distribution remains statistically equal.
• Reframing Representational Harm: By applying Anderson's theory, the paper maps specific representational harms to social hierarchies:
  • Stereotyping/Demeaning $\rightarrow$ Esteem Hierarchies (denial of equal moral worth).
  • Erasing/Reifying $\rightarrow$ Standing Hierarchies (systematic disregard of interests).
  • Opacity/Exclusion $\rightarrow$ Domination (lack of agency).
• Validation of the Framework: The framework successfully explains complex cases (e.g., the YouTube algorithm flagging queer content) as both an allocative harm (economic loss) and a representational harm (erasure and stigmatization), requiring a dual-pronged solution.

5. Significance

• Ethical Foundation: The paper shifts the discourse from "which metric is best?" to "what kind of equality are we pursuing?" It establishes that ML fairness is not just about statistical parity but about dismantling structural oppression.
• Beyond Technical Fixes: It challenges the "techno-solutionist" mindset in AI ethics, arguing that fairness requires socio-technical interventions. It posits that technical fixes (like debiasing datasets) are insufficient without addressing the power dynamics and social contexts in which models operate.
• Policy and Practice: The proposed framework offers a roadmap for researchers and practitioners to move beyond "fairness metrics" toward participatory design, critical literacy, and iterative governance, ensuring that ML systems promote a society where individuals relate as equals rather than as superiors and inferiors.

In conclusion, the paper argues that a Multifaceted Egalitarian Framework is necessary to fully address the harms of ML. It posits that true fairness requires simultaneously correcting the maldistribution of resources (via Distributive Equality) and the misrecognition of social groups (via Relational Equality).