🤖 machine learning

Aligned but Stereotypical? How System Prompts Shape Demographic Bias in LLM-Based Text-to-Image Models

This paper reveals that LLM-based text-to-image systems introduce stronger demographic biases than non-LLM baselines due to their system prompts, and proposes FairPro, a training-free framework that mitigates these biases by adaptively generating fairness-aware instructions while preserving user intent.

Original authors: NaHyeon Park, Na Min An, Kunhee Kim, Soyeon Yoon, Jiahao Huo, Hyunjung Shim

Published 2026-06-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: NaHyeon Park, Na Min An, Kunhee Kim, Soyeon Yoon, Jiahao Huo, Hyunjung Shim

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: The "Over-Enthusiastic Translator"

Imagine you want to draw a picture based on a simple sentence, like "A botanist."

In older drawing machines, you would hand the sentence directly to the artist. The artist would draw a botanist, but might rely on their own old habits or stereotypes (e.g., drawing only white men).

In newer, smarter drawing machines (the ones this paper studies), there is an extra step. Before the artist sees your sentence, a smart translator (an AI language model) reads it first. This translator is supposed to help by adding details to make the picture better. For example, it might turn "A botanist" into "A botanist in a green field wearing a hat, holding a magnifying glass."

The Problem: The paper found that this "smart translator" often adds unwanted stereotypes while it's trying to be helpful. Even if you didn't ask for a specific gender, race, or age, the translator might assume, "Oh, botanists are usually white men," and add those details to the instructions before the artist even starts drawing.

The Investigation: The "COMPBIAS" Test

The researchers built a giant test kit called COMPBIAS (think of it as a standardized exam for drawing machines). They tested 1,024 different prompts, ranging from very simple ("A doctor") to very complex ("A doctor saving a patient in a busy hospital").

They compared two types of machines:

Old School: Direct instructions to the artist.
New School: Instructions go through the "smart translator" first.

The Findings:

The "New School" machines were more biased. They produced images with much stronger stereotypes (e.g., mostly white men for "doctors") than the old ones.
The "Alignment Trap": The new machines were actually better at following your instructions and making high-quality images. But the paper found a trade-off: the better they were at understanding your words, the more they leaned into stereotypes when you didn't specify details.
Complexity makes it worse: The more complex your prompt was, the more the translator added stereotypical details, making the bias stronger.

The Culprit: The "System Prompt"

The researchers dug into why this happens. They discovered the culprit is the System Prompt.

Think of the System Prompt as the rulebook or the manager's memo given to the smart translator. It tells the translator how to behave.

The paper found that these default rulebooks often contain hidden assumptions.
When the translator reads a neutral prompt like "A judge," the rulebook nudges it to think of a specific type of person (e.g., an older white man) before it even writes the description for the artist.
The researchers proved this by looking at the "thoughts" (text embeddings) of the machine. They saw that the translator was secretly changing neutral words into biased ones before the image was ever created.

The Solution: "FAIRPRO" (The Fairness Coach)

The researchers didn't want to just delete the smart translator, because it makes the pictures look great. Instead, they created a fix called FAIRPRO.

How it works:
Imagine the smart translator has a coach standing next to it.

The Coach Checks the Prompt: When you say "A botanist," the coach looks at the translator's rulebook.
The Coach Rewrites the Rules: Instead of letting the translator guess, the coach gives it a new, temporary instruction: "This person is a botanist. Do not assume their gender, race, or age. Keep it diverse."
The Result: The translator still adds helpful details (like the hat and magnifying glass), but it stops assuming the botanist is a white man. It creates a more diverse mix of images.

Why it's special:

No Re-training: They didn't have to rebuild the whole machine or teach it new things. They just changed the instructions it gets while it's working.
Respects You: If you do say "A female botanist," the coach respects that and keeps the gender female, but it still ensures the race and age are diverse. It only fixes the parts you didn't specify.

The Bottom Line

The paper shows that the "smart" parts of modern AI image generators are actually the source of new kinds of bias. By simply tweaking the instructions given to these smart translators (the System Prompts), we can stop them from making unfair assumptions without ruining the quality of the pictures. It's like teaching a helpful assistant to stop guessing your preferences and instead ask, "Who should I draw?" before making assumptions.

Technical Summary: Aligned but Stereotypical? How System Prompts Shape Demographic Bias in LLM-Based Text-to-Image Models

Problem Statement
Recent Text-to-Image (T2I) systems increasingly utilize Large Language Models (LLMs) for text conditioning to interpret and expand user prompts, aiming to improve text-image alignment and compositional reasoning. However, this paper identifies a critical side effect: the integration of LLM-based conditioning introduces implicit demographic assumptions into image generation, even when user prompts are neutral or underspecified. The authors posit that while these systems enhance prompt adherence, they may simultaneously amplify demographic skew, creating a fairness–alignment trade-off. A specific focus is placed on system prompts—embedded instructions that guide the LLM in interpreting user inputs—as a previously underexplored pathway for bias propagation.

Methodology
To investigate this phenomenon, the authors introduce COMPBIAS, a comprehensive benchmark consisting of 1,024 prompts organized into four levels of complexity:

Neutral (L1): Occupational prompts (e.g., "A botanist").
Attribute (L2): Prompts with one explicit demographic attribute (e.g., "An Asian botanist").
Attribute + Context (L3): Prompts enriched with action contexts.
Expanded (L4): Lengthy descriptions generated by an LLM from base occupations.

The study evaluates eight recent T2I models, including non-LLM baselines (e.g., SD3.5, FLUX.1) and LLM-based systems (e.g., SANA1.5, Qwen-Image, FLUX.2). Bias is quantified using a normalized Fair Discrepancy (FD) score, measuring the deviation of generated demographic distributions from a uniform reference, while fidelity is assessed via CLIP similarity and GenEval.

Key Findings and Diagnostic Analysis

LLM-Induced Bias: LLM-based T2I systems consistently exhibit higher demographic bias scores than non-LLM baselines across all prompt complexity levels. Furthermore, as prompts become more complex, both text-image alignment and demographic bias increase, revealing a tension between semantic fidelity and fairness.
The Role of System Prompts: Through controlled analysis in the text-embedding space, the authors demonstrate that default system prompts actively reshape neutral user inputs into demographically biased representations before image synthesis begins.
- Embedding Shift: Removing system prompts weakens gender associations in text embeddings without significantly degrading semantic fidelity.
- Decoded Text: Decoding intermediate text representations reveals that system prompts introduce demographic descriptors (e.g., gendered terms) absent in the original user prompt.
- Propagation: There is a strong correlation (64% agreement) between the demographic attributes inferred in the decoded text and the final generated images, confirming that bias propagates from language-side processing to visual outputs.

Proposed Solution: FAIRPRO
Motivated by the finding that system prompts are a primary source of bias, the authors propose FAIRPRO (FAIRness-aware System PROmpt), a training-free, test-time debiasing framework.

Mechanism: Instead of removing system prompts (which reduces enrichment) or rewriting user prompts (which is less effective), FAIRPRO dynamically generates input-specific, fairness-aware system instructions.
Process: Given a user prompt $u$ , FAIRPRO uses an LLM to identify potential stereotypical completions and constructs a revised system prompt $s_{fair}$ that preserves explicitly specified user constraints (e.g., "male") while enforcing diversity on unspecified attributes (e.g., ethnicity, age).
Efficiency: The method requires only a single additional LLM call, resulting in minimal computational overhead (e.g., ~1.05x for Qwen-Image) and no changes to the underlying image generator architecture.

Results
Experiments across eight models and the COMPBIAS benchmark demonstrate that FAIRPRO:

Reduces Bias: Substantially lowers demographic bias scores across gender, age, ethnicity, and appearance attributes compared to both default settings and "no system prompt" baselines.
Maintains Fidelity: Preserves high text-image alignment and prompt controllability, avoiding the fidelity loss associated with simply removing system prompts.
Enhances Diversity: Increases visual diversity in pose, clothing, and background for unspecified attributes without overriding explicit user constraints.
Generalizes: Shows robust performance on external benchmarks (TIBET) and across different model families.
Ablation: The study confirms that input-adaptive prompting and structured self-auditing are critical components; static fairness instructions or user-prompt rewriting alone are insufficient.

Significance and Claims
The paper claims that LLM-based text conditioning is not merely a passive interface for prompt understanding but an active mechanism that can encode and transmit demographic bias. The significance of this work lies in:

Identifying a New Bias Vector: Highlighting system prompts as a critical, underexplored intervention point for bias in modern T2I systems.
Diagnostic Insight: Providing evidence that bias originates upstream in the language-conditioning stage, shifting the focus of mitigation from image generation to prompt interpretation.
Practical Mitigation: Introducing FAIRPRO as an effective, model-agnostic, and training-free solution that balances fairness with the enhanced capabilities of LLM-based conditioning.

The authors maintain a modest stance, acknowledging that their approach mitigates biased output distributions at test time but does not remove biases encoded in the underlying generative models' parameters. They also note limitations regarding the use of discrete demographic categories and the context-dependency of defining "fair" distributions.