Beyond Categories of Caste: Examining Caste Bias and Morality in Text-to-Image AI Models

Imagine you have a magical, super-smart robot artist. You can tell it, "Draw a picture of a person eating lunch," or "Draw two people studying together," and it will create an image instantly. This is what Text-to-Image AI does.

However, this paper argues that this robot artist isn't just a blank canvas. It has been trained on millions of images from the internet, and unfortunately, it has learned some very old, very harmful rules about how people should be treated based on their background. Specifically, the researchers looked at how this AI handles caste in South Asia (a system of social hierarchy in India).

Here is the breakdown of their findings in simple terms:

1. The Old Way of Looking at the Problem

Previous researchers treated caste like a checkbox. They thought the AI was biased because it saw a name like "Sharma" (often upper-caste) and drew a rich person, or saw a name like "Kumar" (often lower-caste) and drew a poor person. They thought the problem was just about labels.

The Paper's New Idea:
The authors say this is too simple. Caste isn't just a label on a name tag; it's a relationship. It's like a script that tells people who they are in relation to others. The AI isn't just mislabeling people; it is acting out a deep, invisible script that decides who deserves dignity and who deserves to serve.

2. The "Magic Mirror" Experiment

To test this, the researchers didn't just ask the AI to draw "a Brahmin" or "a Dalit." Instead, they asked it to draw everyday scenes using names that were:

• Clearly upper-caste.
• Clearly lower-caste.
• Ambiguous (names that could be anyone).
• No surname (just a first name).

They asked the AI to draw these people doing normal things: eating, studying, praying, or working in a neighborhood.

3. What the Robot Artist Actually Drew

The AI didn't just get the names wrong. It created a whole moral universe where some people were "pure" and "worthy," and others were "dirty" and "useful only for labor."

The researchers found three main ways the AI enforced this bias:

• The "Body Worth" Filter (Bodily Morality):

  • The Analogy: Imagine a scale that weighs your soul.
  • What happened: When the AI drew upper-caste or ambiguous names, it showed them sitting in clean, spacious rooms, studying in libraries, or just standing around looking dignified. They weren't "doing" anything.
  • The Bias: When the AI drew lower-caste or ambiguous names, it almost always made them working. Even if the prompt didn't ask for a job, the AI gave them a broom, a fish basket, or a cleaning tool. It decided that these bodies were only valuable if they were useful to others. The AI assumed they deserved to be poor or laboring, while others deserved to be comfortable.

• The "Social Circle" Filter (Social Relation):

  • The Analogy: Imagine a party where some guests are allowed inside the house, but others must stand outside in the mud.
  • What happened: When the AI drew people eating together, it often separated them. The "upper" person was shown in a clean, private home with no one else around (protecting their "purity"). The "lower" person was shown eating on the street, in a shack, or in a crowded, dirty area.
  • The Bias: The AI refused to imagine people from different backgrounds mixing freely. It reinforced the idea that certain people are "too pure" to be near others, or that others are "too dirty" to be in the same space.

• The "Stage Set" Filter (Material-Spatial Relation):

  • The Analogy: Imagine a movie set where the rich characters get a mansion, and the poor characters get a cardboard box, even if the script didn't say so.
  • What happened: The AI placed lower-caste characters in specific, restricted environments. For example, it drew a lower-caste person in a village with a sign saying "Bhangi Colony" (a segregated area for sanitation workers) or showed them in a makeshift house.
  • The Bias: The AI didn't just show poverty; it showed segregation. It created a world where the "dirty" work and the "dirty" places are the only options for certain people, effectively trapping them there in the image.

4. The Big Surprise: It Happened Even Without Names

The most shocking part of the study was that the AI did this even when the names were ambiguous or had no surnames.

• If you gave the AI a name that could belong to anyone, it still "guessed" the person's social rank based on what it thought they should be doing.
• It assigned the "dirty" jobs and "dirty" neighborhoods to people who had no clear caste label.
• The Lesson: The AI isn't just looking at names; it has learned a moral order. It has learned a set of rules that says, "This type of body belongs in the kitchen cleaning toilets, and that type of body belongs in the library reading books."

5. Why This Matters (The "Why" Behind the "What")

The authors argue that the problem isn't just that the AI is "biased" in a simple way. The problem is that the AI has absorbed Brahminical Normativity.

• Simple Translation: This is a fancy way of saying the AI has learned the "default settings" of a society where one group is considered superior and pure, and everyone else is considered inferior and impure.
• The AI isn't just reflecting reality; it is reinforcing the idea that this hierarchy is natural and moral. It makes it look like it's "right" for some people to be cleaners and others to be bosses.

6. What the Authors Want to Happen

The paper concludes that we can't just fix this by adding more "fair" data or telling the AI to "be nice."

• We need to stop thinking of caste as just a category (like a checkbox).
• We need to understand it as a relationship and a moral system.
• To fix AI, we need to challenge the idea that some people are naturally "worthy" of comfort and others are naturally "worthy" only of labor. We need to teach the AI that dignity isn't something you earn by your job or your name; it's something everyone has.

In short: The robot artist isn't just making mistakes; it's acting like a strict, old-fashioned rulebook that says some people are "too good" to do certain things, and others are "only good" for the dirty work. The authors want us to realize this and rewrite the rulebook.