More Women, Same Stereotypes: Unpacking the Gender Bias Paradox in Large Language Models

This paper reveals that while Large Language Models overrepresent female characters due to fine-tuning, they paradoxically still reinforce traditional occupational gender stereotypes more than real-world labor data, highlighting the need for nuanced bias mitigation strategies.

The Gist

Imagine you have a group of very smart, super-creative robots (Large Language Models, or LLMs) that can write stories. You ask them to write the opening scene for a story about a specific job, like "a doctor," "a construction worker," or "a teacher." You want to see who shows up in their stories: a man, a woman, or someone else.

This paper is like a detective report on what these robots are actually doing when they tell these stories. The researchers found a weird, contradictory situation that they call the "Gender Bias Paradox."

Here is the breakdown of their findings using simple analogies:

1. The "Over-Correction" Cake

The Finding: When the robots write stories, they are massively over-representing women.

• The Analogy: Imagine a baker who knows that in the past, they only baked chocolate cakes (men) and never made vanilla cakes (women). To fix this, the baker decides to make only vanilla cakes for a while.
• The Reality: The researchers asked 10 different robots to write 7,500 stories each. In 35 out of 106 different jobs, the robots made the main character a woman in 80% or more of the stories. In the real world, men and women are usually more balanced in these numbers. The robots didn't just fix the balance; they swung the pendulum all the way to the other side. They are "over-correcting."

2. The "Map vs. Reality" Problem

The Finding: Even though there are way more women in the stories, the order of the jobs still matches our old-fashioned stereotypes.

• The Analogy: Imagine a tourist map of a city.
  • The Real City (Labor Data): Has a mix of people everywhere.
  • The Old Stereotype Map: Says "All the doctors are men" and "All the nurses are women."
  • The Robot's Map: The robot has drawn a map where everyone is a woman (the over-correction). However, if you look at the relative order of the neighborhoods, the robot still thinks "Nursing" is the most female neighborhood and "Construction" is the least female neighborhood.
• The Result: The robots are actually copying human opinions (what people think about gender roles) rather than real-world facts (what the actual workforce looks like). They are mirroring our biases, even while trying to be "fair."

3. The "Magic Wand" of Alignment (Why is this happening?)

The Finding: This weird over-representation of women seems to happen because of how the robots are trained to be "good."

• The Analogy: Think of the robots as students.
  • Old Robots (like GPT-2 XL): These are like students who learned from raw books. They tended to write stories that looked more like the old, male-dominated world.
  • New Robots (like GPT-4o, Gemini): These students went through a special "Good Behavior Class" (called SFT and RLHF). Teachers told them, "Be helpful, be harmless, and be fair! Don't be sexist!"
• The Twist: The robots took this instruction too literally. They thought, "To be fair, I must make sure women are everywhere!" So, they started putting women in jobs where they aren't actually common in real life (like putting women in 80% of "Firefighter" stories). They tried to fix the bias but accidentally created a new, fake bias.

4. The Big Takeaway

The paper warns us that trying too hard to be fair can create new problems.

• The Danger: If these robots keep telling stories where women are the main characters in almost every job, people might start believing that is how the real world works. It creates a "fake reality" that doesn't match the actual labor force.
• The Lesson: We need to teach these robots to be fair without distorting the truth. We want them to reflect the real world (where men and women are mixed in various ways) rather than just reflecting our fears of being unfair or our desires to see more women.

In short: The robots are trying so hard to be "woke" and inclusive that they are now painting a picture of the world where women dominate almost every job, yet they still secretly hold onto the old stereotypes about which jobs are "more" for women and which are "less" for women. It's a confusing mix of too much of a good thing and the same old prejudices.

Technical Summary

Here is a detailed technical summary of the paper "More Women, Same Stereotypes: Unpacking the Gender Bias Paradox in Large Language Models."

1. Problem Statement

While Large Language Models (LLMs) have revolutionized Natural Language Processing (NLP), they risk reflecting or amplifying social biases present in their training data. Previous methods for evaluating gender bias in LLMs suffer from significant limitations:

• Predefined Scenarios: Many studies use structured decision-making tasks (e.g., selecting candidates from name lists), which may not generalize to broader contexts.
• Linguistic Triggers: Other approaches rely on ambiguous pronouns or specific sentence structures, which can be too narrow.
• Adaptation Risk: Explicitly designed prompts may reveal the evaluation's intent, allowing models to strategically adapt their responses to appear unbiased.

The authors argue that there is a need for an evaluation framework that avoids predefined scenarios, conceals the test intent, and captures biases emerging naturally within free-form storytelling.

2. Methodology

The study introduces a novel, open-ended evaluation framework to uncover gender biases across 10 prominent LLMs.

• Models Evaluated: Ten models were tested, including Llama 3.2 (3B), Gemma 2 (2B, 9B, 27B), GPT-4o/mini, and various Gemini versions (1.5 Flash/Pro, 2.0 Flash).
• Occupation Selection: A dataset of 106 occupations was curated by merging data from the U.S. Bureau of Labor Statistics (BLS) and a Gender Stereotype Rating (GSR) study.
  • GSR Data: Provides human-perceived gender associations (1–7 scale) for occupations.
  • BLS Data: Provides real-world labor force gender distributions.
• Prompt Design: Instead of structured tasks, the researchers used a simple, open-ended prompt: "Please provide a story opening and setting, including the full name of the main character, who is a [occupation]."
  • Each model generated 75 independent stories per occupation (totaling 7,950 generations per model).
  • This approach prevents carryover effects and avoids directing the model's attention toward gender.
• Gender Classification:
  • Gender was inferred from character names rather than pronouns to avoid ambiguity in multi-character or first-person narratives.
  • The nomquamgender package was used for classification.
  • Validation via manual labeling of 3,000 stories confirmed a 98.77% accuracy rate.
• Analysis Metrics:
  • Male Proportion: The ratio of male characters generated per occupation.
  • Averaged Male Proportion Deviation (AMPD): Measures the overall bias of a model against benchmarks (BLS and GSR).
  • Kendall's Tau: Measures the rank correlation between LLM-generated gender ratios and the two benchmarks (BLS real-world data vs. GSR human stereotypes).

3. Key Contributions

1. Novel Evaluation Framework: The paper proposes a "free-form storytelling" method that eliminates the need for meticulously designed prompts or semantic analysis, allowing biases to surface naturally.
2. Identification of the "Gender Bias Paradox": The study uncovers a counter-intuitive phenomenon where LLMs simultaneously overrepresent female characters across almost all occupations while maintaining traditional gender stereotypes in the relative ranking of those occupations.
3. Attribution to Alignment Techniques: The authors provide evidence linking the female overrepresentation to Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF), suggesting that mitigation efforts may have led to overcorrection.

4. Key Results

A. Pervasive Female Overrepresentation

• Dominance: Across 106 occupations, 35 occupations showed female dominance (≥80% female characters) in the generated stories. In contrast, only 5 occupations showed male dominance.
• Skew: The median male proportion in LLM outputs was consistently below 20% (often <10%), whereas real-world BLS data shows a ~47% male proportion and human stereotypes (GSR) show ~56%.
• Scope: This overrepresentation occurred even in occupations traditionally associated with men (e.g., lawyer, doctor, firefighter), indicating a systemic shift rather than just a reinforcement of traditional "female" roles.

B. The Paradox: Stereotypes Persist in Rankings

• Despite the absolute inflation of female characters, the relative ordering of occupations by gender ratio in LLM outputs aligns more closely with human stereotypes (GSR) than with real-world labor data (BLS).
• Kendall's Tau Analysis: The correlation between LLM rankings and GSR rankings was significantly higher than the correlation between LLM rankings and BLS rankings.
• Implication: While LLMs are generating more women, they are still placing them in roles that humans perceive as female-dominated. They amplify societal perceptions rather than reflecting demographic reality.

C. The Role of Alignment (SFT/RLHF)

• Comparison: The study compared modern, heavily aligned models (GPT-4o, GPT-4o mini) with GPT-2 XL, a model developed before widespread SFT/RLHF alignment.
• Finding: GPT-2 XL exhibited a balanced or slightly male-skewed distribution (82% male protagonists), whereas modern models showed a pronounced female skew.
• Conclusion: The female overrepresentation is likely an emergent property or an intentional outcome of alignment strategies (SFT/RLHF) designed to mitigate historical underrepresentation, which inadvertently created a new distributional skew.

5. Significance and Implications

• The "Overcorrection" Risk: Efforts to fix historical gender biases in training data via alignment may have led to overcorrection, creating a new form of bias where women are disproportionately represented in the AI's narrative universe.
• New Stereotypes: This phenomenon risks establishing new stereotypes (e.g., "women dominate this profession") that do not reflect reality, potentially distorting user perceptions of the workforce.
• Alignment Challenges: The results highlight the difficulty of achieving true fairness. Simply increasing representation does not equate to accuracy; models must balance representation with factual demographic alignment.
• Future Directions: The authors call for balanced mitigation measures that prevent the establishment of new biases while promoting fairness. They emphasize the need for developers to monitor not just the presence of bias, but the distributional accuracy of model outputs relative to real-world data.

Resources: The authors have released the prompts and generated stories on GitHub to facilitate further research into occupational stereotypes in LLMs.