Technical Summary: BAFIS - Dataset + Framework to Assess Occupational Bias and Human Preference in Modern Text-to-Image Models

1. Problem Statement

Generative artificial intelligence, specifically Text-to-Image (T2I) models, holds significant potential for productivity and creative content generation. However, existing research indicates these models are heavily influenced by biases, often misrepresenting the diversity of genders, ethnicities, and cultures. This risks perpetuating social biases, as users may unknowingly trust biased outputs.

Current evaluation methodologies for T2I models rely predominantly on automated metrics (e.g., prompt alignment, image quality) or classification-based bias detection. While some platforms exist for human preference evaluation (e.g., Chatbot Arena, IMGSYS), they lack specific capabilities for assessing bias. Furthermore, many existing benchmarks focus on English prompts and image quality, often neglecting socio-economic bias or the nuances of multilingual contexts. There is a critical need for a framework that combines automated metrics with human preference feedback to evaluate inherent and language-induced biases, particularly within the context of occupational image generation.

2. Methodology

2.1 The BAFIS Framework

The authors introduce BAFIS (Battle-Arena for Fair Image Synthesis), a static benchmarking platform designed to collect anonymous, randomized human feedback on bias, image quality, and prompt alignment. Unlike live arenas where users modify prompts, BAFIS utilizes a pre-computed dataset to investigate the influence of specific prompts under controlled conditions.

• Mechanism: Users select a language (English or German) and engage in "battles" where two randomly selected models generate images for a specific prompt.
• Evaluation: Users vote subjectively on three dimensions:
  1. Bias: The perceived distribution of gender and ethnicity relative to societal expectations.
  2. Image Quality: Realism and aesthetics.
  3. Prompt Alignment: How well the image matches the textual description.
• Ranking: Pairwise comparisons are converted into rankings using the Elo rating system and Maximum Likelihood Estimation (MLE) via the Bradley-Terry model to ensure stability.

2.2 Dataset Generation

To facilitate this evaluation, the authors constructed a comprehensive dataset comprising 21,140 synthetic images generated from 1,057 prompts.

• Source Prompts: Based on the MAGBIG dataset, extended to include 151 German and 151 English "group" prompts, alongside "direct," "indirect," "direct feminine," and "German gender star" variations.
• Models Evaluated: Five state-of-the-art T2I models were tested in their default configurations:
  • Midjourney v6.1
  • Stable Diffusion 3 Medium
  • DALL-E 3
  • Playground v2.5
  • FLUX.1-dev
• Filtering: Images with occluded or invisible faces were automatically excluded using face detection algorithms.

2.3 Evaluation Metrics

The study employs a dual approach:

1. Automated Metrics:
   • Bias: Measured using YoloV8 for face detection and FairFace for gender/ethnicity classification. Bias is quantified using Mean Absolute Deviation (MAD) scores against a uniform distribution baseline.
   • Image Quality: Assessed via FID (Fréchet Inception Distance) against the Flickr-Faces-HQ dataset and MagFace scores for face image quality (FIQ).
   • Prompt Alignment: Measured via cosine similarity between CLIP vector representations of prompts and images.
2. Human Feedback: Collected via the BAFIS platform (459 battles, 1,377 votes) and compared against official statistics from the German Federal Employment Agency to provide a socio-economic reference point.

3. Key Results

3.1 Bias Analysis

• Gender Bias:
  • English Prompts: Midjourney v6.1, Playground v2.5, and SD 3 Medium generated over 55% male faces. "Direct" prompts showed extreme bias (e.g., Playground v2.5 at ~70% male).
  • German Prompts: A reduction in gender bias was observed across most categories compared to English, though a male bias persisted in "Service & Hospitality."
  • Outliers: DALL-E 3 exhibited the most balanced gender distribution. FLUX.1-dev showed a preference for female faces, particularly in English.
• Ethnicity Bias:
  • All models showed a significant preference for generating faces of white ethnicity in both languages.
  • Language Impact: Bias was more pronounced in German prompts. For instance, the proportion of white faces in SD 3 Medium increased by approximately 30% when switching from English to German prompts.
  • DALL-E 3 was the only model with a white ethnicity face proportion of less than 50% in both languages.
• MAD Scores: DALL-E 3 achieved the lowest MAD scores (indicating least bias) for both gender and ethnicity. German prompts consistently yielded higher MAD scores (more bias) than English prompts.

3.2 Image Quality and Alignment

• Automated Metrics: DALL-E 3 achieved the best FID scores, while SD 3 Medium scored highest on MagFace (face recognition quality).
• Human Preference (Elo Ratings):
  • Image Quality: FLUX.1-dev ranked highest, followed by Midjourney v6.1. DALL-E 3 ranked last in human preference for image quality, despite its superior FID score.
  • Prompt Alignment: SD 3 Medium ranked highest for English battles, while DALL-E 3 ranked highest for German battles.
  • Bias Preference: DALL-E 3 (German) and Midjourney v6.1 (English) received the highest human ratings for bias.

3.3 Correlation Discrepancies

The study found partial correlations between established automated metrics and subjective human ratings:

• Image Quality: There was a notable disconnect between FID scores (where DALL-E 3 excelled) and human Elo ratings (where FLUX.1-dev excelled).
• Prompt Alignment: While CLIP similarity scores favored Playground v2.5 for English prompts, human Elo ratings favored SD 3 Medium.
• DALL-E 3 Anomaly: DALL-E 3 showed high similarity between German images and English prompts, attributed to its "prompt revision" feature which rewrites prompts for security and translation, potentially enhancing alignment with human expectations.

4. Key Contributions

1. BAFIS Platform: A novel static benchmarking platform designed specifically to evaluate T2I models through human preference feedback regarding bias, quality, and alignment.
2. Comprehensive Dataset: A multilingual (German/English) dataset of 21,140 synthetic images focused on occupational contexts, extending the MAGBIG benchmark.
3. Multilingual Bias Analysis: A systematic investigation of how language (English vs. German) influences bias, revealing that German prompts often exacerbate ethnic bias.
4. Human vs. Automated Metrics: A comparative analysis demonstrating that established metrics (FID, CLIP similarity, MagFace) do not always align with human preferences, highlighting the necessity of human-in-the-loop evaluation for fairness.

5. Significance and Claims

The paper claims that its research underscores the limitations of relying solely on automated metrics for assessing T2I model fairness. By integrating human preference feedback, the authors demonstrate that:

• Systematic biases exist across all evaluated models, particularly regarding ethnicity (favoring white faces) and gender.
• Language plays a critical role in bias manifestation, with German prompts often showing stronger biases than English ones.
• Human preferences for image quality and alignment can diverge significantly from automated scores, suggesting that current evaluation standards may not fully capture the nuances of "fairness" and "quality" as perceived by users.

The authors conclude that developing fairer and more inclusive T2I models requires the inclusion of human preferences in the evaluation pipeline. They advocate for guidelines addressing bias, diverse dataset curation, and regular bias assessments, while emphasizing the need for feedback channels to identify bias-related concerns. The code and dataset are made public to facilitate further research.