💬 NLP

Will the Prince Get True Love's Kiss? On the Model Sensitivity to Gender Perturbation over Fairytale Texts

This paper investigates language model sensitivity to gender stereotypes in fairytale comprehension by demonstrating that while models initially exhibit performance drops when faced with gender perturbations, fine-tuning on counterfactual data significantly enhances their robustness to anti-stereotypical narratives.

Original authors: Christina Chance, Da Yin, Dakuo Wang, Kai-Wei Chang

Published 2026-04-22

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Christina Chance, Da Yin, Dakuo Wang, Kai-Wei Chang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart robot librarian named "Prince." This robot has read thousands of fairytales to learn how stories work. But here's the catch: almost all the books it read were written a long time ago, back when people had very strict ideas about what boys and girls "should" do.

In these old stories, the Prince is always the brave hero who saves the day, and the Princess is always the one waiting to be saved.

The researchers in this paper wanted to know: If we trick the robot by swapping the genders in the stories, will it get confused? Will it still understand the story, or will it stumble because it's so used to the old rules?

The Experiment: The "Gender Swap" Test

Think of the robot's brain like a set of training wheels. It learned to ride a bike (answer questions) on a path where the "Prince" always goes left and the "Princess" always goes right.

The researchers decided to test the robot by building a new path where the genders are flipped:

Instead of a Prince saving a Princess, they created a story where a Princess saves a Prince.
Instead of a King, they used a Queen.
Instead of a Tailor (traditionally male), they used a Seamstress (traditionally female).

They used three different ways to make these changes:

The Dictionary Method (Rule-Based): Like using a strict translation app that simply swaps "he" for "she" and "king" for "queen" everywhere.
The Creative Writer Method (LLM Rewriting): Asking a super-smart AI writer to rewrite the whole story with swapped genders, hoping it keeps the flow natural.
The Hybrid Method (The Best of Both): Using the AI writer to help the dictionary, so it's smart enough to know when to swap words but careful enough not to make mistakes.

What Happened?

1. The Robot Stumbled (Sensitivity to Bias)
When they tested the robot on these new, flipped stories, it got confused. Its performance dropped.

The Analogy: Imagine you always drive to work on the right side of the road. If someone suddenly tells you to drive on the left, you might crash or take a wrong turn. The robot was crashing because it had learned that "Princes save" and "Princesses are saved." When that rule was broken, it didn't know how to answer questions about the story.

2. The Robot Learned to Adapt (The Fix)
The researchers then gave the robot a special training course. They showed it a mix of the old stories and the new, flipped stories.

The Analogy: It's like teaching the robot to drive on both sides of the road.
The Result: After this training, the robot became much better. It could handle the flipped stories just as well as the original ones. It learned that the story matters more than the gender of the character.

The Big Discovery: Why This Matters

The paper isn't just about fairytales; it's about how AI learns from us.

The Problem: If we only train AI on old, biased stories, the AI will keep repeating those biases. It might think only men can be heroes or only women can be nurses.
The Solution: By intentionally feeding the AI "counterfactual" stories (stories where the roles are flipped), we can "un-train" the bad habits.
The Case Study: The researchers also asked the robot to write new fairytales.
- When trained on old stories, the robot wrote stories where girls were passive and boys were active.
- When trained on the flipped stories, the robot wrote stories where girls were brave adventurers and boys were kind helpers. The stories were actually better written, too! They focused more on personality and less on physical appearance.

The Takeaway

This paper shows that AI isn't "born" biased; it learns bias from the data we give it. But just like a child, if we give it a diverse set of books where everyone gets a chance to be the hero, it can learn to tell better, fairer, and more inclusive stories.

In short: If you want a robot that understands the world as it is (and as it could be), you have to teach it with stories that break the old molds. The Prince doesn't need to be the only one with the kiss; the Princess can save the day, too.

1. Problem Statement

The paper addresses the issue of learned gender stereotypes in Large Language Models (LLMs) and pre-trained Language Models (LMs) when processing narrative texts, specifically fairytales.

Context: Fairytales are widely used in education to teach reading comprehension but often contain entrenched gender biases (e.g., the "brave prince rescuing the helpless princess").
The Gap: While existing QA models perform well on standard datasets like FairytaleQA, it is unknown how robust these models are when presented with counterfactual narratives where gender roles are swapped (e.g., a princess rescuing a prince).
Hypothesis: Models trained on traditional corpora rely on learned gender correlations rather than genuine narrative comprehension. Consequently, their performance should degrade when these correlations are disrupted via gender perturbations.

2. Methodology

A. Dataset and Counterfactual Data Augmentation (CDA)

The authors utilized the FairytaleQA dataset (278 fairytales, 10,580 questions for K-8 students) and applied three distinct CDA approaches to create counterfactual training and test sets:

Rule-based Translation: A static dictionary approach swapping gendered nouns (e.g., princess $\leftrightarrow$ prince, seamstress $\leftrightarrow$ tailor) and pronouns.
LLM Text Rewriting: Using GPT-3.5-turbo to rewrite the story, questions, and answers to swap genders while maintaining formatting.
LLM-assisted Rule-based Translation: A hybrid approach. It uses a rule-based dictionary for known pairs. For unknown words, it queries an LLM to generate the opposite-gender counterpart, which is then added to the dictionary for consistency.

Constraint: Proper names (e.g., Cinderella) were not swapped to avoid introducing cultural biases regarding name-gender associations, as the model's training data is Eurocentric while the dataset is culturally diverse.
Evaluation of CDA Quality: The authors performed human evaluation on the generated data, scoring the approaches on swap quality, storyline consistency, swap consistency, and grammar. The hybrid approach showed the best balance of consistency and quality.

B. Experimental Setup

Base Models: T5 and BART (pre-trained LMs).
Fine-tuning Strategies:
1. Fine-tuned on Original data only.
2. Fine-tuned on Counterfactual data only.
3. Fine-tuned on a 50/50 Mix of Original and Counterfactual data.
4. Fine-tuned on 100/100 Mix (Full Original + Full Counterfactual).
Evaluation Metric: ROUGE-L F1 score on Question Answering (QA) tasks.
Test Conditions: Models were tested on both original and counterfactual test sets generated via the three augmentation methods.

C. Case Study: Inclusive Generation

The authors conducted a case study where GPT-3.5-turbo was prompted to generate new fairytales based on either original or counterfactual story sections.

Metrics: Repetitive plot, unrelated events, conflicting logic, poor continuity, unsafe content, and bias propagation.
Scoring: A 6-point scale with deductions for specific flaws.

3. Key Results

A. Sensitivity to Gender Perturbations

Performance Drop: Models fine-tuned on the original dataset exhibited a consistent performance drop when tested on counterfactual data.
- Example: T5's overall ROUGE-L F1 dropped from ~65.29 (original test) to ~62.66 (LLM-assisted counterfactual test).
- Interpretation: This confirms that models rely heavily on learned gender stereotypes rather than logical reasoning. When the stereotype is broken, the model struggles to extract the correct answer.
Question Type Analysis:
- Extractive Questions (answers explicitly in text) showed larger performance drops than Abstractive Questions (requiring reasoning). This suggests extractive tasks are more susceptible to bias reliance, as models skip reasoning and rely on pattern matching.

B. Impact of Counterfactual Training

Robustness: Fine-tuning on counterfactual data significantly improved model performance on counterfactual test sets.
The 50/50 Mix: The most effective strategy was fine-tuning on a 50% Original + 50% Counterfactual mix.
- This approach maintained high performance on the original test set (minimal drop) while significantly boosting performance on counterfactual test sets.
- It demonstrated that models can learn to handle both normative and anti-stereotypical narratives simultaneously without catastrophic forgetting.

C. Case Study Findings (Generative Models)

Quality Improvement: Stories generated from counterfactual prompts scored higher (5.67/6.0) than those from original prompts (4.93/6.0).
Bias Mitigation:
- Original Prompts: Generated stories often reinforced stereotypes (e.g., emphasizing physical appearance for female characters, rigid adherence to traditional roles) and contained more logical inconsistencies.
- Counterfactual Prompts: Generated stories focused more on personality traits over physical attributes, exhibited better continuity, and showed zero instances of "Bias Propagation" in the evaluated metrics.

4. Key Contributions

Empirical Evidence of Bias: Demonstrated that state-of-the-art QA models are sensitive to gender perturbations in narrative comprehension, proving they rely on stereotypes rather than deep understanding.
Novel CDA Approach: Introduced an LLM-assisted rule-based translation method that balances the precision of rule-based systems with the adaptability of LLMs for domain-specific (fairytale) gender augmentation.
Training Strategy: Established that a mixed training set (50/50) is optimal for creating robust models that generalize across both traditional and counterfactual gender roles.
Downstream Application: Showed that incorporating counterfactual examples into training data not only improves comprehension robustness but also actively shapes generative models to produce more inclusive and less stereotypical content.

5. Significance and Limitations

Significance: The paper provides a practical framework for mitigating gender bias in educational AI tools. By proving that counterfactual data augmentation improves both comprehension and generation, it offers a scalable solution for creating inclusive educational materials.
Limitations:
- Binary Gender: The study operates within a normative gender binary (male/female), excluding non-binary and gender-non-conforming identities.
- Intersectionality: The analysis focuses primarily on gender, whereas fairytales also contain biases related to race, class, age, and beauty standards.
- Cultural Context: The reliance on Western-centric LLMs for augmentation may still introduce subtle cultural biases when handling non-Western names or contexts.

Conclusion: The authors conclude that integrating counterfactual anti-stereotype examples is a highly effective strategy for mitigating bias in NLP models, fostering inclusivity in both story comprehension and generation tasks.