💻 computer science

KG-FairDiff: Knowledge Graph-Guided Prompt Refinement for Demographically Fair Text-to-Image Generation

KG-FairDiff is a model-agnostic, inference-time framework that leverages a knowledge graph and a closed-loop optimization process to refine text-to-image prompts, effectively reducing demographic and cultural biases while preserving semantic fidelity without requiring costly model retraining.

Original authors: Farbod Davoodi, Seyed Reza Tavakoli Shiyadeh, Pooria Safaei, Sana Harighi, Parsa Gholami, Amirali Amini, Kimia Vanaei, Emad Firoozi, Parham Abed Azad, Babak Khalaj, Siavash Ahmadi, Amir Hossein Payber

Published 2026-06-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Farbod Davoodi, Seyed Reza Tavakoli Shiyadeh, Pooria Safaei, Sana Harighi, Parsa Gholami, Amirali Amini, Kimia Vanaei, Emad Firoozi, Parham Abed Azad, Babak Khalaj, Siavash Ahmadi, Amir Hossein Payberah, Mohammad Hossein Rohban, Soheil Kolouri, Ali Diba

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a magical camera that can turn any sentence you write into a picture. You type "a doctor," and it snaps a photo. But here's the catch: this camera has been trained on millions of old photos from the internet. Because of that, it has developed a bad habit. When you ask for a "doctor," it almost always shows you an older white man. When you ask for a "nurse," it shows a young woman. It's like the camera is stuck in a time warp, only seeing the world through a very narrow, stereotypical lens.

This paper introduces a new tool called KG-FairDiff to fix this camera without having to rebuild the camera itself.

The Problem: The Camera's "Default Settings"

Think of these text-to-image systems as chefs who have memorized a massive cookbook. If you ask for "CEO," they automatically pull out a picture of a man in a suit because that's what they've seen most often in their training data. They aren't trying to be mean; they are just following the most common patterns they know. This leads to a world where certain groups of people (like women, people of color, or older adults) are rarely seen in powerful roles, or are shown in funny, exaggerated ways.

The Solution: The "Smart Editor"

Instead of trying to fire the chef and hire a new one (which would be expensive and impossible for many closed cameras), KG-FairDiff acts like a smart editor who stands between you and the camera.

Here is how the editor works, step-by-step:

The Knowledge Library (The "Fact-Checker"):
The editor carries a special library of about 1,200 facts (a Knowledge Graph). These facts are like little cards that say things like: "Doctors can be women," "Nurses can be men," or "Traditional clothing in this culture looks like this." It also has cards that say, "This is a stereotype we should avoid."
The Rewrite (The "Suggestion"):
When you type "a doctor," the editor doesn't just send that to the camera. First, it looks in its library. It sees that "doctor" is often stereotyped. It then asks a super-smart AI (an LLM) to rewrite your sentence.
- Your input: "A doctor."
- Editor's suggestion: "A diverse group of doctors, including women and people of different backgrounds, working in a modern hospital."
  The editor makes sure the new sentence still sounds like what you wanted (semantic fidelity) but adds the missing diversity.
The Gatekeeper (The "Validator"):
Before sending the new sentence to the camera, the editor checks two things:
- Did we fix the bias? (Does the new sentence encourage a more diverse picture?)
- Did we change the meaning too much? (Is it still about a doctor?)
  If the answer to both is "Yes," the editor sends it. If not, it tries again, like a loop until it gets it right.

Why This is a Big Deal

Usually, to fix a biased camera, you have to take it apart, retrain it on new data, and hope it learns better. This is hard, expensive, and often impossible because many cameras are "black boxes" (you can't see inside them).

KG-FairDiff is different because:

It's a plug-and-play fix: It works with any camera, whether it's a free one or a paid, secret one.
It doesn't break the camera: It just tweaks the instructions (prompts) you give it.
It's grounded in facts: It doesn't just guess; it uses a structured list of facts to guide the changes.

The Results

The researchers tested this "Smart Editor" on eight different popular image generators. They found that:

It significantly reduced the number of times the camera defaulted to stereotypes (like showing only men as CEOs).
It increased the variety of people shown (more women, people of color, and different ages).
It did this without making the pictures look weird or changing the original idea of the user.

The Limitations (The "Fine Print")

The authors are honest about where the tool might stumble:

The "Safety Filter" Clash: Some cameras have their own strict safety rules. Sometimes, the editor's suggestion to add diversity might clash with the camera's internal rules, causing the picture to fail or look strange.
The "Circular" Editor: The same AI model is used to both write the new sentence and check if it's good. It's like a student grading their own homework; it usually works, but it's not perfect.
Cultural Gaps: While the tool tries to include non-Western cultures, the library of facts isn't perfect yet. It might miss some specific cultural details.

In a Nutshell

KG-FairDiff is like a fairness filter for your imagination. It doesn't change the camera; it just helps you ask for pictures in a way that ensures the camera sees the whole world, not just a tiny, biased slice of it. It's a practical, ready-to-use way to make AI-generated images more representative of real life.

Technical Summary: KG-FairDiff

Problem Statement
Text-to-Image (TTI) systems have become critical infrastructure for journalism, education, and advertising. However, these models inherit demographic and cultural stereotypes from their training data, systematically under-representing or caricaturing women, people of color, older adults, and non-Western cultures. Existing mitigation strategies face significant limitations: dataset rebalancing and latent-space regularization require access to training data or model internals (infeasible for closed-source backbones), while lighter-weight prompt-level methods often rely on fixed demographic templates that ignore cultural context, resulting in superficial or tokenistic outputs.

Methodology
The authors propose KG-FairDiff, a model-agnostic, inference-time framework that treats fairness-aware prompt refinement as a constrained optimization problem. The system operates as a closed-loop pipeline without modifying the underlying generator:

Knowledge Graph (KG) Construction: The framework utilizes a hand-curated KG containing approximately 1,200 triples. These triples are sourced from the McGillNLP Bias Dataset and CultureBank and cover three categories: stereotypical associations (to be countered), counter-stereotypes (to be asserted), and cultural grounding (non-Western concepts, attire, and practices).
Retrieval and Rewriting: At inference time, the system retrieves the top- $k$ relevant triples from the KG based on the user's initial prompt. An LLM rewriter (GPT-4o) uses this structured context to propose refined prompts.
Closed-Loop Validation: A validator (also GPT-4o) evaluates the refined prompt against two criteria:
- Fairness Improvement: The prompt must reduce a divergence-based fairness loss (measured via KL divergence from a uniform target distribution).
- Semantic Fidelity: The refined prompt must maintain a cosine similarity above a threshold ( $\tau$ ) with the user's original intent to prevent semantic drift.
Optimization Formulation: The process is formalized as minimizing a weighted sum of fairness loss and a style-preservation regularizer, subject to the semantic fidelity constraint. The authors prove a finite-termination bound for the refinement loop under a monotone-improvement assumption.

Key Contributions

Algorithmic Formulation: The paper formalizes fairness-aware prompt refinement as a constrained optimization problem with explicit terms for fairness, semantic fidelity, and style preservation, providing an iterative algorithm with a theoretical termination bound.
Reusable Knowledge Graph: The authors construct and release a curated KG of ~1,200 triples spanning stereotypes, counter-stereotypes, and cultural grounding, double-annotated for factual accuracy and cultural sensitivity.
Theoretically Grounded Evaluation Suite: A new evaluation framework is introduced that links standard bias metrics (Bias-P, Bias-W) to divergence from target distributions and relates Entropy-based measures (ENS) to KL divergence, utilizing Wasserstein alignment and CLIP similarity to assess semantic preservation.
Comprehensive Audit: The framework is evaluated against eight widely deployed backbone generators (including GPT-Image-1, SD 3.5, Gemini 3 Pro) across a 100-prompt benchmark covering 50 occupations.

Results
KG-FairDiff demonstrates substantial improvements in reducing demographic disparities while preserving prompt semantics:

Performance vs. Baselines: Compared to prompt-level baselines like MinorityPrompt and PreciseDebias, KG-FairDiff achieves significantly larger reductions in bias. Specifically, it achieves up to 20× larger Bias-W reductions (e.g., -0.1396 vs. -0.0037 for Race) and 13× larger ENS gains compared to MinorityPrompt.
Semantic Preservation: The refinement process maintains high semantic fidelity, with a mean Wasserstein distance change ( $\Delta \bar{W}_2^2$ ) of only 0.0011.
Ablation Studies: Experiments confirm that the KG retrieval component contributes independently to fairness gains, distinct from the LLM rewriter's capabilities.
Limitations Observed: The authors note that for certain generators (e.g., SD 3.5 Large, Gemini 3 Pro), some metrics worsened due to conflicts between the refined prompts and the generators' built-in safety filters, or due to conflicting cues in intersectional optimization. Additionally, the framework relies on automated classifiers (FairFace) which have lower recall for non-Western phenotypes.

Significance and Claims
The paper claims that KG-FairDiff offers a practical, deployment-ready route to more equitable generative AI. Its primary significance lies in its ability to intervene at the prompt level without requiring access to model weights or costly retraining, making it applicable to both proprietary and open-weight black-box generators. By grounding refinements in structured cultural knowledge rather than fixed templates, the framework aims to reduce population-level harms caused by the repetition of narrow representations. The authors position this work as a step toward "trustworthy AI" that can audit and intervene in deployed models under realistic constraints, though they acknowledge the need for future work to scale the knowledge graph to broader cultural coverage and rigorously calibrate the validator against downstream metrics.