💬 NLP

Uncertainty Drives Social Bias Changes in Quantized Large Language Models

This study reveals that post-training quantization fundamentally alters the social biases of large language models through uncertainty-driven "masked bias flipping," causing significant and asymmetric demographic shifts that remain undetected by aggregate metrics despite substantial changes in individual responses.

Original authors: Stanley Z. Hua, Sanae Lotfi, Irene Y. Chen

Published 2026-02-09

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Stanley Z. Hua, Sanae Lotfi, Irene Y. Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very wise, well-trained librarian (a Large Language Model) who has been taught to treat everyone fairly, regardless of their age, gender, or background. You want to put this librarian's knowledge into a smaller, more portable backpack so it can run on a phone or a cheap laptop. This process is called quantization. It's like taking a massive, high-resolution encyclopedia and compressing it into a pocket-sized pamphlet to save space.

This paper argues that while this compression saves space, it accidentally scrambles the librarian's sense of fairness in ways we didn't expect.

Here is the breakdown of what the researchers found, using simple analogies:

1. The "Invisible Shuffle" (Masked Bias Flipping)

The biggest surprise is that when you look at the average score of the librarian's fairness, it looks exactly the same before and after compression. It's like flipping a coin 1,000 times; if you get 500 heads and 500 tails, the average is 50/50.

However, the researchers found that 21% of the individual answers actually flipped.

Before compression: The librarian might have said, "I can't decide, that's a stereotype," (Unbiased).
After compression: The librarian suddenly said, "Yes, that stereotype is true," (Biased).
The Twist: At the same time, for other questions, the librarian flipped from biased to unbiased. Because these flips happened in opposite directions, they canceled each other out in the final average score. The "score" looked neutral, but the individual answers were wildly different.

2. The "Wobbly Table" (Uncertainty Drives the Change)

Why did the librarian flip their answers? The paper found it happens mostly when the librarian is unsure.

Think of the librarian as a person standing on a wobbly table.

Confident answers: When the librarian is 100% sure of the answer, the table is stable. Compression doesn't shake them off.
Uncertain answers: When the librarian is hesitating (high uncertainty), the table is already shaky. Compression is like someone giving that wobbly table a little push. The librarian falls off their "unbiased" stance and lands on a "biased" one (or vice versa).

The study found that uncertain answers were 3 to 11 times more likely to change their mind after compression than confident ones.

3. The "Heavy Backpack" (Quantization Strength)

Not all backpacks are the same.

8-bit compression: This is like a sturdy, slightly heavy backpack. It keeps the librarian's balance mostly intact.
4-bit compression: This is a tiny, ultra-light backpack. It forces the librarian to drop a lot of details. The study found that using this "lighter" backpack caused 4 to 6 times more chaotic flipping of answers than the heavier one.

4. The "Uneven Shuffle" (Asymmetric Impact)

This is the most dangerous part. Even though the average fairness score stayed the same, the changes didn't happen equally for everyone.

Imagine a group of people waiting in line.

When the librarian gets compressed, Group A (e.g., men) might suddenly get treated worse (bias increases by up to 18.6%).
At the same time, Group B (e.g., short people) might get treated better (bias decreases by up to 14.1%).

Because one group got worse and another got better, the "average" fairness looks fine. But in reality, specific groups are suffering while others are benefiting. The paper calls this an asymmetric impact. It's like a seesaw: if one side goes up and the other goes down, the center point doesn't move, but the people on the ends are having very different experiences.

5. Bigger Isn't Always Better

You might think a bigger, smarter librarian (a larger model) would be more stable. The paper found no evidence that bigger models are safer. A tiny model and a giant model both got equally shaken up by the compression. You can't just assume "bigger is safer" when you compress them.

The Bottom Line

The paper concludes that compressing AI models is like playing a game of Jenga. You can remove blocks (data) to make the tower smaller, but you might not notice that the tower has become unstable until it suddenly tips over in a specific direction.

Because the "average" score hides these individual flips, we cannot trust standard tests to tell us if a compressed model is safe. The researchers suggest:

Don't trust the average: Look at individual answers, especially the ones where the model seems unsure.
Use heavier backpacks: 8-bit compression is much safer than 4-bit.
Check specific groups: Make sure the model isn't accidentally hurting one group while helping another.

The authors warn that if we deploy these compressed models in high-stakes areas (like healthcare or law) without checking for these hidden flips, we might accidentally introduce harmful biases that we thought we had already fixed.

Technical Summary: Uncertainty Drives Social Bias Changes in Quantized Large Language Models

Problem Statement

Post-training quantization (PTQ) is a standard technique for reducing the computational cost of Large Language Models (LLMs) to enable deployment in resource-constrained environments. However, while PTQ optimizes for efficiency at the sub-module level, it operates without awareness of downstream behavioral changes. Existing literature suggests that quantization can degrade fact recall, increase hallucinations, and unpredictably shift social biases, potentially reintroducing harmful behaviors mitigated during alignment. Current studies on this topic suffer from disjoint evaluations across different models, datasets, and metrics, leading to conflicting conclusions and a lack of actionable guidance for practitioners deploying quantized models in high-stakes domains like healthcare and law.

Methodology

To address these inconsistencies, the authors introduce PostTrainingBiasBench, a unified framework designed to evaluate bias changes in quantized models through rigorous paired statistical testing.

Framework and Benchmarks

The study evaluates 50 quantized models derived from 10 instruction-tuned LLMs (including LLaMA 3.1/3.2, Ministral, and Qwen 2/2.5 families with parameter sizes ranging from 0.5B to 14B). These models are compressed using 5 distinct PTQ strategies:

Round-to-Nearest (RTN) at 4-bit and 8-bit weights.
Generative Pre-trained Transformer Quantization (GPTQ) at 4-bit.
Activation-Aware Weight Quantization (AWQ) at 4-bit.
Activation-Smoothing Quantization (SmoothQuant) combined with RTN at 4-bit.

Evaluation is conducted across 13 diverse datasets categorized into three capabilities:

Bias Identification: Detecting harmful content or inherent bias (e.g., CEB-Recognition, Jigsaw).
Equal Outcomes under Informative Context: Ensuring decisions are independent of sensitive attributes given sufficient context (e.g., Adult, Credit).
Preference for Refusal/Uncertainty: Prioritizing safe responses or expressing uncertainty when prompted with biased text (e.g., BBQ, IAT, StereoSet, BiasLens).

Evaluation Protocol

The framework employs a paired evaluation strategy where responses from the original (full-precision) model and its quantized variants are generated for the same prompts.

Response Extraction: For closed-ended tasks, responses are selected based on the geometric mean of token probabilities (equivalent to lowest perplexity) to mitigate token bias. For open-ended tasks, greedy decoding is used.
Metrics: The study tracks response flipping (individual responses changing from biased to unbiased or vice versa) and aggregate bias metrics (normalized to [0,1]).
Statistical Analysis: Significance is determined via permutation-style bootstrap tests (1,000 simulations) with Benjamini–Hochberg FDR correction. Effect sizes are quantified using Cohen's $d$ .
Uncertainty Measurement: Model uncertainty is measured via normalized Shannon entropy across response options for closed-ended tasks and average token probabilities for open-ended tasks.

Key Contributions

1. Quantization-Induced Masked Bias Flipping

The study identifies a phenomenon where up to 21% of responses flip between biased and unbiased states after quantization, despite aggregate bias scores remaining statistically unchanged. This "masked" flipping indicates that standard aggregate metrics fail to capture significant behavioral shifts at the individual response level.

2. Uncertainty as the Primary Driver

The research establishes a strong causal link between model uncertainty and response flipping.

Responses with high uncertainty (entropy > 0.66) are 3–11 times more likely to flip than confident responses (entropy < 0.33).
Preference tuning experiments (using SimPO to modulate pre-quantization uncertainty) confirm a dose-response relationship: increasing pre-quantization uncertainty directly increases post-quantization flipping rates.
Crucially, the overall distribution of model uncertainty remains stable post-quantization; quantization redistributes uncertainty rather than systematically increasing or decreasing it.

3. Asymmetric Impacts on Social Groups

While aggregate metrics often suggest neutral effects, subgroup analysis reveals dramatic asymmetry.

Bias changes can worsen by up to 18.6% for specific groups while improving by 14.1% for others within the same model, creating misleadingly neutral aggregate outcomes.
These impacts are unpredictable across model families and datasets; for instance, the "male" demographic showed increased bias in some datasets but decreased in others.

4. Quantization Strength and Model Scale

Quantization Strength: 4-bit quantization methods exhibit 4–6 times more behavioral changes than 8-bit methods. RTN 8-bit quantization shows minimal deviation from baseline, whereas 4-bit variants (GPTQ, AWQ, RTN) show significantly higher instability.
Model Scale: Larger models do not demonstrate a consistent robustness advantage. A 14B model can exhibit similar or worse stability than a 0.5B model, invalidating heuristics that larger models are inherently safer under compression.

Results and Findings

Hidden Volatility: In datasets like FMT10K, 21% of responses flip even when aggregate metrics show non-significant changes. Open-ended generation tasks exhibit the highest volatility in effect sizes.
Ranking Instability: Quantization fundamentally alters comparative model rankings. A model ranked as least biased pre-quantization can drop significantly post-quantization, and vice versa.
Dataset Specificity: The vulnerability of specific questions varies by orders of magnitude. Some questions are highly susceptible to flipping (up to 50% of the time), while others remain stable, suggesting that specific semantic content or question construction drives vulnerability.

Significance and Claims

The paper claims that compression fundamentally alters bias patterns in ways that current evaluation practices fail to detect. The authors argue that:

Standard evaluations are misleading: Relying solely on aggregate metrics masks significant, asymmetric harms to specific demographic groups.
Uncertainty is a critical safety signal: Model confidence (uncertainty) serves as a pre-screening tool for quantization safety, a connection previously established in classification but not in compression contexts.
Deployment guidance: Practitioners should favor 8-bit over 4-bit quantization for safety-critical applications, conduct task-specific evaluations rather than relying on benchmark-only results, and prioritize subgroup-level metrics over dataset-level aggregates.
Necessity of Post-Quantization Assessment: The findings necessitate mandatory post-quantization bias assessments, particularly for high-uncertainty predictions and vulnerable social groups, to ensure reliability in real-world deployment.

The study concludes that ignoring these effects risks deploying systems whose behavior diverges sharply from pre-quantization evaluations, with severe consequences for vulnerable populations.