💬 NLP

Bi-directional Bias Attribution: Debiasing Large Language Models without Modifying Prompts

This paper proposes a framework that detects stereotype-inducing words and attributes bias to specific neurons in large language models, enabling effective debiasing through direct activation intervention without requiring fine-tuning or prompt modifications.

Original authors: Yujie Lin, Kunquan Li, Yixuan Liao, Xiaoxin Chen, Jinsong Su

Published 2026-02-05

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yujie Lin, Kunquan Li, Yixuan Liao, Xiaoxin Chen, Jinsong Su

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine Large Language Models (LLMs) as incredibly talented chefs who have read almost every recipe and story ever written. They can cook up amazing dishes (answers) for any request. However, because they learned from the real world, they sometimes accidentally add "secret ingredients" of bias. For example, if you ask, "Who is the doctor?" they might automatically assume the chef is a man, or if you ask about a "nurse," they might assume a woman. These aren't malicious choices; they are just the model repeating patterns it saw in its training data.

The paper you provided, titled "Bi-Directional Bias Attribution," proposes a new way to fix these "secret ingredients" without having to retrain the whole chef or change the way customers order their food.

Here is a simple breakdown of how they did it:

1. The Problem with Old Methods

Usually, to fix a biased chef, you have two bad options:

Retraining: You make the chef practice with new, perfectly balanced cookbooks. This takes a huge amount of time, money, and energy (computing power).
Prompt Engineering: You tell the customer, "Please ask the chef in a very specific way to make sure they don't get biased." This is annoying for the user and makes the conversation clunky.

The authors wanted a solution that fixes the chef's internal habits without making them practice again or changing how customers talk to them.

2. Step One: Finding the "Trigger Words" (Stereotype Cues)

First, the researchers needed to find out which specific words in a sentence cause the model to get biased.

The Analogy: Imagine a light switch that, when flipped, turns on a specific biased thought. The researchers wanted to find exactly which words act as that switch.
How they did it: They tested thousands of adjectives and nouns (like "gentle," "doctor," "warrior") to see which ones made the model's predictions become very narrow and predictable (low "entropy"). If a word like "nurturer" made the model 99% sure the person was female, that word was flagged as a "Stereotype Cue."

3. Step Two: The "Bi-Directional" Detective Work

Once they had the trigger words, they needed to find where inside the model's brain these biases lived. The model is made of millions of tiny processing units called neurons. The authors used two different detective strategies to find the "bad" neurons:

Forward Bias Attribution (The "Prediction" Detective):
- The Scenario: The model sees a biased word (like "nurturer") and predicts a gender.
- The Method: They traced the path from the word to the prediction to see which specific neurons were doing the heavy lifting to make that biased guess.
Backward Bias Attribution (The "Difference" Detective):
- The Scenario: They compare how the model reacts to "a male nurse" vs. "a female nurse."
- The Method: They looked for neurons that fired differently depending on the gender, identifying the parts of the brain that create the gap between the two groups.

By using both detectives, they got a complete map of exactly which neurons were responsible for the bias.

4. Step Three: The "Silence Button" (Intervention)

Now that they knew exactly which neurons were causing the trouble, they didn't delete them or retrain the model. Instead, they simply froze those specific neurons.

The Analogy: Imagine a noisy choir where a few singers are singing off-key. Instead than firing the whole choir or teaching them to sing again, you just put a mute button on the specific singers who are off-key.
The Result: They set the activation of these "biased neurons" to a constant, neutral value. This stops them from pushing the model toward a stereotype, but the rest of the model (the good singers) keeps working perfectly.

5. The Results

The researchers tested this on three popular AI models (Llama-3.1, Llama-3.2, and Mistral).

Fairness: The models became much fairer. When asked about doctors or nurses, they stopped assuming specific genders and started guessing more evenly (closer to a 50/50 split).
Performance: Crucially, the models didn't get "dumber." They still understood language well and could answer questions correctly. The "mute button" only silenced the bias, not the intelligence.

Summary

This paper introduces a "surgical" approach to fixing AI bias. Instead of a heavy overhaul (retraining) or a clumsy workaround (changing prompts), they identified the specific "bad apples" (neurons) inside the model using a two-way detective method and simply neutralized them. This allows the AI to remain smart and helpful while treating all groups of people more fairly.

Technical Summary: Bi-Directional Bias Attribution

Problem Definition

Large Language Models (LLMs) frequently exhibit and amplify societal biases regarding gender, race, religion, and occupation. Existing debiasing approaches face significant limitations: fine-tuning methods are computationally expensive and impractical for large-scale models, while prompt engineering (e.g., instructing models to avoid bias) degrades user experience by increasing context length and inference costs in multi-turn interactions. Furthermore, current neuron-level attribution methods often struggle with deep architectures where individual neuron contributions are marginal or fail to capture cross-pair bias relationships.

This paper addresses the need for a debiasing framework that requires neither model fine-tuning nor prompt modification. The authors define two core subproblems:

Demographic-Invariant Generation (DIG): Ensuring the model's output distribution remains unchanged when only demographic information in the prompt varies.
Stereotype-Free Inference (SFI): Ensuring the model does not systematically favor one demographic over another when interpreting prompts containing stereotype cues (e.g., associating "doctor" with "male").

Methodology

The proposed framework, Bi-Directional Bias Attribution, operates without modifying model weights or user prompts. It consists of three primary stages:

1. Stereotype Cue Selection via Entropy Minimization

Instead of relying on pre-defined human assumptions, the method automatically identifies "stereotype cues" (adjectives or nouns) that trigger biased predictions.

Process: The authors construct sentence templates (e.g., "The [Demographic Attribute] of this [Stereotype Noun] is...") and query the model to predict demographic groups.
Metric: They calculate the Shannon entropy of the model's predicted probability distribution over demographic groups for each candidate cue.
Selection: Cues yielding lower entropy (indicating highly skewed, confident predictions toward specific groups) are selected as the most bias-inducing words. This process covers both adjective and noun forms.

2. Bi-Directional Bias Attribution

To trace biased outputs to specific neurons, the authors propose two gradient-based attribution strategies using Integrated Gradients (IG):

Forward Bias Attribution (Forward-IG):
- Direction: Prompt $\to$ Demographic Prediction (SFI context).
- Mechanism: The model is prompted with stereotype cues to predict a demographic group. Forward-IG quantifies the contribution of input neurons in the projection layer to the model's certainty (low entropy) regarding a specific demographic.
- Goal: Identify neurons that drive the model to confidently associate a cue with a specific demographic.
Backward Bias Attribution (Backward-IG):
- Direction: Demographic Group $\to$ Output Variation (DIG context).
- Mechanism: The model is prompted with sentences containing different demographic groups to predict the associated stereotype cues. Backward-IG uses the Jensen-Shannon Divergence (JSD) to measure the disparity in predicted distributions across demographic groups.
- Goal: Identify neurons responsible for generating different outputs based on the input demographic group.

3. Targeted Intervention

Once biased neurons are identified (the top $N$ neurons with the highest attribution scores), the framework intervenes directly at the projection layer (the layer mapping hidden representations to logits).

Action: The activation values of these specific neurons are fixed to a constant $C$ .
Theoretical Basis: The authors provide a theoretical analysis (Theorem 1) establishing that modifying hidden representations along an attribution-guided path reduces bias, with the magnitude of bias change bounded by the output variation and the local bias gradient.

Key Contributions

Entropy-Based Cue Selection: An automated method to identify stereotype-inducing adjectives and nouns specific to the model's internal bias, avoiding reliance on static, human-curated lists.
Dual Attribution Strategies: The introduction of Forward-IG and Backward-IG, which provide complementary perspectives on bias: Forward-IG captures bias in inferring demographics from cues, while Backward-IG captures bias in generating text conditioned on demographics.
Prompt-Free Intervention: A debiasing pipeline that modifies only the activation values of specific neurons at the projection layer, requiring no fine-tuning or prompt engineering.
Theoretical Connection: A formal proof linking the reduction of bias to the variation in model outputs under attribution-guided modifications.

Experimental Results

The framework was evaluated on three LLMs (Llama-3.1-8B, Llama-3.2-3B, Mistral-7B-v0.3) across four demographic attributes (Gender, Nationality, Profession, Religion) using three benchmarks:

StereoSet (DIG Task): The proposed methods (FBA and BBA) significantly reduced bias (moving the Stereotype Score closer to 50%) while maintaining high Language Modeling Scores (LMS) and ICAT scores. In contrast, baseline methods like Auto-Debias and prompt engineering either failed to reduce bias or degraded language modeling capabilities.
BBQ (DIG Task): On ambiguous questions, FBA and BBA reduced reliance on stereotypes (increasing "unknown" selection) without the sharp drop in accuracy on disambiguated questions observed in prompt-based methods.
WinoBias (SFI Task): The Backward Bias Attribution (BBA) method achieved the smallest gap between stereotype and anti-stereotype probabilities while maintaining $P_{other} \approx 0$ , indicating preserved language modeling capability. Notably, while IG2 reduced bias on Llama-3.2, it severely degraded language modeling performance (high $P_{other}$ ).

Ablation Studies confirmed that both the attribution strategy (identifying specific neurons) and the cue selection algorithm are critical; random neuron selection or using non-selected cues resulted in inferior debiasing performance and lower language modeling scores.

Significance and Claims

The paper claims to offer a practical and interpretable pathway for building fairer LLMs. By demonstrating that biased behaviors can be localized to specific, identifiable subsets of neurons, the authors argue that targeted intervention at the projection layer is a viable alternative to resource-intensive fine-tuning or user-intrusive prompt engineering. The work emphasizes that effective debiasing does not require compromising the model's core language modeling capabilities or altering the user interaction flow.