Bias in Large Language Models: Origin, Evaluation, and Mitigation

The Big Picture: The "Super-Reader" with a Blind Spot

Imagine a Large Language Model (LLM) as a super-advanced student who has read almost every book, website, and social media post ever written. This student is incredibly smart and can write essays, answer questions, and translate languages better than almost anyone.

However, because this student learned from everything humans have written, they also learned all our prejudices, stereotypes, and unfair habits. Just like a child who grows up in a neighborhood where everyone believes a certain group of people are bad at math, the student might start believing that too, even if it's not true.

This paper is a massive guidebook that asks three main questions:

1. Where did these bad habits come from? (Origins)
2. How do we catch the student cheating? (Evaluation)
3. How do we teach them to be fair? (Mitigation)

---

Part 1: Where the Bad Habits Come From (Origins)

The authors split the "bad habits" (bias) into two types: Intrinsic and Extrinsic.

1. Intrinsic Bias: The "Internal Memory"

Think of this as the student's internal dictionary and worldview formed while they were studying.

• The Training Data (The Textbooks): The student read books where "doctors" were almost always men and "nurses" were almost always women. So, in their internal memory, they link "doctor" to "male" and "nurse" to "female." This is a data bias.
• The Collection Method (The Library): Imagine the student only went to libraries in one specific country. They would think that country's holidays are the only holidays in the world. This is spatial/temporal bias.
• The Tools (The Pen and Paper): Even the way the student breaks words into pieces (tokenization) can be unfair. If the student's "pen" breaks up names from minority groups into tiny, confusing fragments but keeps common names whole, they can't understand the minority names as well. This is tokenization bias.

2. Extrinsic Bias: The "Bad Performance"

This is when the student acts on their bad habits during a specific test or job.

• Understanding Tasks (NLU): If you ask, "Who is the doctor?" and the text says "The doctor arrived," the student might guess "He" even if the text doesn't say the gender, because their internal memory is biased.
• Generating Tasks (NLG): If you ask the student to write a story about a leader, they might automatically make the leader a man and the nurse a woman, even if you didn't ask for that.
• The Result: The student might give a job recommendation to a man over a woman, or translate a sentence in a way that insults a culture, simply because they are repeating patterns they saw in their training data.

---

Part 2: How to Catch the Cheating (Evaluation)

How do we know if the student is biased? The paper suggests checking them at three different levels, like a teacher grading a student.

1. Data-Level Check (Checking the Textbooks):
   • Before the student even starts, we look at the books they are reading. Are there too many books about men and too few about women? Are there too many books from the US and none from Africa? We use math to count how often different groups appear.
2. Model-Level Check (Checking the Brain):
   • We poke the student's brain to see how they connect ideas. If we ask, "Is a nurse a man?" and the student's brain lights up with "No," but if we ask "Is a doctor a man?" it lights up with "Yes," we know there is a hidden bias in their connections. We use tools like Counterfactuals (changing a name from "John" to "Jane" and seeing if the answer changes).
3. Output-Level Check (Grading the Homework):
   • We look at the actual answers the student gives. Do they use mean words for certain groups? Do they give different advice to different people? We also use Human Reviewers (real people) to read the answers and say, "Hey, that sounds unfair."

---

Part 3: How to Fix the Bad Habits (Mitigation)

The paper suggests three ways to "de-bias" the student, depending on when you intervene.

1. Pre-Model Debiasing (Cleaning the Textbooks)

• The Idea: Before the student starts studying, we clean up the library.
• How: We add more books about underrepresented groups (oversampling) or remove the really hateful books. We might even rewrite sentences to swap genders (e.g., changing "The doctor is he" to "The doctor is she") so the student learns both options.
• Pros/Cons: It's cheap and easy to start, but you can't fix everything just by cleaning the books. Some bad habits are already baked into the student's brain structure.

2. Intra-Model Debiasing (Rewiring the Brain)

• The Idea: While the student is studying, we force them to learn differently.
• How: We change the rules of the game. If the student tries to link "nurse" to "female," we give them a "penalty score" (a loss function) so they learn to stop doing that. We might also "prune" (cut out) the specific parts of their brain that hold these bad connections.
• Pros/Cons: This is very effective at fixing the root cause, but it is expensive and hard. It's like trying to rewire a computer while it's running.

3. Post-Model Debiasing (Editing the Homework)

• The Idea: Let the student write the answer, but we edit it before showing it to the world.
• How: If the student writes a biased sentence, we use a filter to change the words. Or, we use a "causal prompting" trick where we ask the student to think about the problem in a specific way that forces them to ignore stereotypes.
• Pros/Cons: This is fast and doesn't require retraining the student. However, it's like putting a bandage on a wound; it fixes the symptom but doesn't cure the disease.

---

Part 4: Why This Matters (Ethics and Law)

The paper warns that these biases aren't just annoying; they are dangerous.

• Representational Harms: The student might say, "Muslims are violent" or "Women aren't leaders." This hurts people's feelings and reinforces negative stereotypes in society.
• Allocational Harms: This is where people lose real-world opportunities. If a biased student helps a company hire people, they might reject a qualified woman's resume. If a biased student helps a hospital diagnose patients, they might miss a disease in a minority group.

The paper notes that laws are starting to catch up. In places like New York City and the European Union, companies are now required to audit their AI tools to make sure they aren't discriminating against people based on race, gender, or age.

The Bottom Line

This paper is a roadmap. It tells us that AI is like a mirror reflecting our society's flaws. To build fair AI, we can't just rely on one fix. We need to clean our data, train our models carefully, check their work constantly, and fix the output before it reaches people. It's a continuous process, not a one-time job.