💬 NLP

Is She Even Relevant? When BERT Ignores Explicit Gender Cues

This paper reveals that a Dutch BERT model trained from scratch develops strong, persistent male-default gender biases that override explicit contextual cues, demonstrating that its learned representations are insufficiently dynamic to correctly encode anti-stereotypical gender information despite the language's overt morphological gender marking.

Original authors: Jonas Klein, Chiara Manna, Eva Vanmassenhove

Published 2026-05-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jonas Klein, Chiara Manna, Eva Vanmassenhove

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: A Robot That Can't Unlearn Stereotypes

Imagine you are teaching a robot to understand human language by showing it millions of books. You want the robot to be smart enough to understand that when someone says, "She is a plumber," the robot should know the plumber is a woman, even if the robot has mostly seen male plumbers in its training books.

This paper asks a simple question: Does the robot actually listen to the word "She," or does it just ignore it and stick to its old habits?

The researchers built a version of a famous AI model (called BERT) from scratch, but instead of English, they taught it Dutch. Dutch is a tricky language because it has a mix of gendered words (like "he" and "she") and neutral words that used to be male but are now used for everyone.

The Experiment: The "Plumber" Test

To test the robot, the researchers set up a game with sentence templates. They gave the robot sentences like:

"He is a nurse." (This matches the stereotype: men are rarely nurses).
"She is a plumber." (This breaks the stereotype: women are rarely plumbers).

They then asked the robot: "Based on the word 'plumber' or 'nurse' in this sentence, do you think this person is male or female?"

They measured this by looking at the robot's "brain" (its internal math) to see which side of a gender scale the word landed on.

Key Findings: What They Discovered

1. The Robot Learns Gender Fast, But Stuck

The researchers watched the robot learn over time. They found that the robot figured out how to tell "male" from "female" very quickly (around the 20th day of training). Once it learned this, it got really good at it.

The Analogy: Imagine a student who learns the difference between "cats" and "dogs" on day one. By day 20, they can spot a cat or a dog instantly. But here's the problem: the robot learned to spot them based on what it usually sees, not on what is actually in front of it right now.

2. The "Male" Default is Everywhere

The researchers found that the robot's understanding of "male" is like a broad, comfortable blanket that covers almost everything. The understanding of "female" is like a specific, narrow spotlight.

Male: The robot spreads the idea of "male" across many different parts of its brain. It's the "default" setting.
Female: The robot only uses a few specific parts of its brain to identify "female."

The Result: When the robot is confused, it defaults to the "blanket" (Male). Even if you tell it "She is a plumber," the robot's internal math often still says "Plumber = Male" because that's the broad, comfortable default it learned from the books.

3. The Robot Ignores the "She"

This is the most surprising part. The robot is supposed to be "contextual," meaning it should change its mind based on the sentence.

If the sentence says "She is a plumber," the robot should think "Female."
What actually happened: The robot mostly ignored the word "She." It looked at the word "plumber," remembered that plumbers are usually men in its training data, and stuck with that answer.

The Analogy: Imagine you are at a party. You see a person wearing a chef's hat (stereotype: male chef). Your friend points and says, "That's my sister, she's the chef!" (Explicit cue: female).

A human would say, "Oh, I see, she is the chef."
This robot would say, "No, the hat says chef, and chefs are men. I'm ignoring your friend."

4. The Only Thing That Works: The "Suffix" Trick

The researchers found one thing that did force the robot to change its mind: Morphology (the shape of the word).

In Dutch, you can add a special ending to a job title to make it explicitly female (like adding "-ster" to make "nurse" into "female nurse").

If the sentence was "She is a verpleegster" (female nurse), the robot correctly identified her as female.
If the sentence was "She is a verpleger" (neutral/male nurse), the robot ignored the "She" and assumed the nurse was male.

The Analogy: The robot is like a person who only listens to the uniform someone is wearing, not the name tag they are holding. If the uniform says "Male Nurse," the robot believes it, even if the name tag says "She." But if the uniform has a special "Female" badge (the suffix), then the robot finally pays attention.

The Conclusion: Why This Matters

The paper concludes that this AI model is not as "smart" or "flexible" as we hope. We assume that because it looks at the whole sentence, it can override stereotypes. But this study shows that the robot's internal habits are stronger than the immediate context.

The "Male" default is so strong that even explicit words like "She" can't always break it.
The robot relies on surface-level clues (like word endings) rather than deep understanding (like who the sentence is talking about).

In short: The robot learned the statistics of the world (plumbers are usually men) so well that it stopped listening to the specific story being told in the sentence (this specific plumber is a woman). It's a robot that knows the rules of the game, but it can't play along when the rules change in front of it.

Technical Summary: "Is She Even Relevant? When BERT Ignores Explicit Gender Cues"

Problem Statement
While gender bias in Large Language Models (LLMs) has been extensively investigated for English, languages with grammatical or morphological gender remain comparatively understudied. This gap is critical because grammatical structures can obscure or distort the measurement of social stereotypes. The paper focuses on Dutch, a language occupying an intermediate position between fully gendered systems (like German) and largely genderless ones (like English). Dutch exhibits a tension between "neutralisation" (where historically male, morphologically unmarked forms function as generic terms) and "feminisation" (where explicitly female forms are derived via suffixes). The authors investigate how a Dutch BERT model, trained from scratch, encodes gender information and whether explicit contextual cues (e.g., the pronoun zij meaning "she") can override learned statistical associations (e.g., plumber $\rightarrow$ male) within the model's internal representations.

Methodology
The study employs a checkpoint-level analysis of a Dutch BERT model trained entirely from scratch on the SoNaR-500 corpus (books subset, $\approx$ 26M words). The methodology integrates static subspace projection techniques with controlled sentence templates to trace the evolution of bias during training.

Model Training: A BERT model is trained from scratch for 100 epochs. Intermediate checkpoints are saved to track the emergence of gender representations over time.
Gender Subspace Construction: At each checkpoint, a linear Support Vector Machine (SVM) is trained on contextual embeddings of 60 unambiguous gendered word pairs (e.g., man vs. vrouw) to derive a dynamic gender axis. The authors avoid ambiguous pronouns (like zij, which can mean "she" or "they") to ensure the subspace reflects clear gender distinctions.
Bias Evaluation: The authors project profession term embeddings from controlled sentence templates onto the learned gender subspaces. The template structure is [TARGET] is een [ATTRIBUTE] (e.g., "She is a plumber").
- Variables: The study systematically varies the target gender (male/female), the morphological marking of the attribute (neutral/unmarked vs. female-suffix), and the stereotype alignment (pro-stereotypical vs. anti-stereotypical based on labor force statistics).
- Metric: Accuracy is measured by checking if the projected profession embedding aligns with the gender specified by the sentence context (the target) rather than the model's default statistical association.

Key Results
The analysis yields four primary findings regarding the formation and behavior of gender representations:

Emergence of Linear Separability: Gender information becomes clearly linearly separable in the embedding space around epoch 20. While accuracy rises steadily, it plateaus just above 0.92, indicating that gender is a stable feature of the learned representations early in the training process.
Distribution of Gender Information: Contrary to the idea of a single "gender unit," gender information is distributed across multiple embedding dimensions. However, the distribution is asymmetric:
- Male representations are diffusely encoded across many dimensions, functioning as a robust, unmarked default.
- Female representations are concentrated in a smaller, more specific subset of dimensions.
Failure of Contextual Override: Explicit contextual cues often fail to override learned priors. In anti-stereotypical contexts (e.g., "She is a plumber"), the model's internal representation of the profession remains aligned with the stereotypical male gender significantly more often than with the contextual female cue. Stereotypical pairings are predicted with 82.5% accuracy, whereas anti-stereotypical pairings drop to 43.7%.
Dominance of Morphological Markers: The model relies heavily on surface-level morphological cues rather than integrating contextual information. When a profession has an explicit female suffix (e.g., verpleegster), the representation aligns with the female subspace with high accuracy (83% in anti-stereotypical contexts). However, when the attribute is morphologically neutral (e.g., verpleger), the model defaults to the male interpretation, even when the subject is explicitly female.

Key Contributions

Checkpoint-Level Analysis: This work provides one of the first controlled, checkpoint-level investigations of how gender information emerges in a Transformer-based architecture, extending prior work on recurrent models (LSTMs) to modern BERT models.
Contextual Subspace Projection: The authors adapt projection-based subspace analysis to a contextual setting, combining it with controlled sentence templates to directly measure the interaction between explicit gender cues and learned gender connotations.
Insights into Morphological Gender: By using Dutch, the study demonstrates that even strong, explicit female cues (pronouns) can be overridden by stereotypical defaults when the profession term is morphologically unmarked. This highlights the specific challenges of bias in languages where grammatical gender and social stereotypes interact complexly.
Mechanistic Baseline: The findings establish a baseline for understanding how structural gender bias forms and stabilizes, suggesting that "contextualization" in these models is not sufficiently dynamic along the gender direction to reliably reflect explicit cues in anti-stereotypical scenarios.

Significance and Claims
The paper claims that its results challenge the assumption that contextual embeddings robustly integrate local cues to adjust word representations. Instead, the authors argue that for the Dutch BERT model, contextual cues are frequently overshadowed by internal priors and morphological markers. The "male-default" behavior persists because the unmarked, historically male forms function as generic terms in the training data, creating a structural advantage for male-coded representations.

The authors conclude that while the model learns to distinguish gender, it does so in a way that reinforces existing asymmetries: male is the diffuse default, while female is a marked, concentrated signal that requires explicit morphological suffixation to be reliably encoded. Consequently, applications relying on these representations (such as translation or coreference resolution) may reproduce gender stereotypes even when the input explicitly contradicts them. The paper modestly notes that these conclusions are limited to the representational level captured by the probes and do not directly measure downstream task performance, but they offer critical insight into the mechanisms of bias formation in multilingual NLP.