It's How You Ask: Gender-Associated Linguistic Bias in LLMs

Technical Summary: "It's How You Ask: Gender-Associated Linguistic Bias in LLMs"

Problem Statement
As Large Language Models (LLMs) increasingly mediate professional communication, a critical question arises regarding their equity across different user groups. While extensive research has documented that LLMs encode social biases regarding demographic attributes (e.g., names, race, gender identity) in narratives and decision-making scenarios, there is a gap in understanding whether models perform differently for users based on their linguistic style. Sociolinguistic literature establishes that men and women often exhibit systematic, largely unconscious differences in language use, such as the frequency of hedges, tag questions, collective reference, and expressive adjectives. This paper investigates whether LLMs respond to these gender-associated linguistic patterns (GALPs) in user prompts by generating systematically different outputs, potentially disadvantaging users who employ women-associated linguistic features (WALF) in professional contexts.

Methodology
The authors constructed a controlled evaluation framework using 427 real-world professional prompts from the WildChat corpus, covering three document types: emails, job applications, and resignation letters.

1. Prompt Manipulation: Using GPT-4, the authors created paired versions of each prompt.

   • WALF (Women-Associated Linguistic Features): Prompts were rewritten to include hedges (e.g., "maybe"), tag questions (e.g., "isn't it?"), collective reference (e.g., "we"), and expressive adjectives.
   • MALF (Men-Associated Linguistic Features): Prompts were rewritten to feature direct assertions, individual reference (e.g., "I"), neutral adjectives, and minimized hedging.
   • Validation: Human annotators confirmed that 95% of rewritten pairs preserved the underlying task semantics while altering the linguistic register. Realism checks indicated rewritten prompts remained plausible, though slightly less realistic than original WildChat prompts.

2. Model Evaluation: The modified prompts were fed into four popular LLMs (GPT-4, Gemma 2, Mistral, and Llama 3). The authors measured response differences across:

   • Complexity: Word count, token count, lexical sophistication (mean word length), readability (Flesch Reading Ease), grade level, and type-token ratio (TTR).
   • Style: Politeness density, formality (F-measure), and clout (confidence/authority).

3. Control for Mirroring: To ensure differences were not merely the model mirroring the input style, the authors performed regression analyses and mediation tests. They examined whether prompt complexity predicted response complexity and whether linguistic features carried over from prompt to response mediated the observed effects.

4. Mechanistic Interpretability:

   • Explicit vs. Implicit Cues: A 2x2 factorial experiment crossed linguistic features (WALF/MALF) with sign-off names (men-associated vs. women-associated) to compare the impact of implicit register against explicit gender markers.
   • Linear Probing: Logistic regression classifiers were trained on layer activations of Llama-3.2-3B-Instruct to decode linguistic features and name gender, identifying where these representations are encoded.
   • Activation Patching: The authors replaced activations from WALF prompts with those from corresponding MALF prompts at specific layers to measure the causal impact on output distributions via KL divergence.

Key Results

• Systematic Output Differences: WALF prompts consistently elicited responses that were shorter, less lexically sophisticated, more readable (lower grade level), and less formal compared to MALF prompts. These effects were robust across GPT-4, Gemma, Mistral, and Llama, particularly in emails and job applications.
• Style Calibration: While formality was significantly lower for WALF responses, politeness density and clout scores showed no significant differences. This suggests models calibrate politeness to the task genre rather than the user's register, but do adjust complexity and formality based on linguistic cues.
• Mirroring is Insufficient: Regression analyses showed that prompt complexity explained at most 34% of the variance in response complexity. Mediation analysis revealed that feature carry-over (the presence of hedges, etc., in the output) only partially explained the complexity differences. The model was adjusting its overall register, not just echoing injected features.
• Implicit vs. Explicit Cues: In the sign-off experiment, explicit gender cues (names) had virtually no effect on response complexity or style. In contrast, implicit linguistic register produced large, consistent effects.
• Mechanistic Findings:
  • Encoding: Linguistic feature information is robustly encoded in the model, with decoding accuracy jumping from chance at layer 0 to 0.962 at layer 1 and peaking at 0.988 at layer 5.
  • Causality: Activation patching revealed that early layers (0–7) are causally critical for these effects, with the highest KL divergence observed in layers 0, 3, 4, 6, and 7.
  • Orthogonality: While both linguistic features and name gender peak in representation at layer 5, they occupy largely orthogonal subspaces. The network encodes explicit gender cues (names) but does not route them to influence generation, whereas implicit linguistic register is both strongly encoded and causally efficacious.

Significance and Claims
The paper argues that current bias mitigation strategies focused on explicit demographic markers (e.g., name-based debiasing) are insufficient because they fail to address biases triggered by unconscious, culturally embedded linguistic registers. The authors claim that:

1. User Burden: Because these linguistic patterns are often unconscious, users cannot easily avoid them through strategic self-presentation without adopting unnatural communication styles.
2. Professional Risk: In workplace settings, where women's communication styles may already face bias, LLMs generating less sophisticated or formal outputs for WALF prompts could inadvertently reinforce stereotypes about professional competence or dilute the perceived authority of women's communications.
3. Intervention Pathways: The study suggests that because these biases are rooted in early transformer layers and are causally linked to specific representations, targeted interventions (such as activation steering or prompt standardization) are technically feasible. However, the authors caution that such interventions must be carefully designed to avoid paternalism or overriding legitimate user preferences.

The work concludes that the field must move beyond explicit demographic cues to consider how models respond to the subtle linguistic features of users, as these features can lead to disparate impacts in downstream professional tasks.