🤖 AI

When Cultures Meet: Multicultural Text-to-Image Generation

This paper introduces the new task of multicultural text-to-image generation, presents a comprehensive benchmark and dataset to evaluate state-of-the-art models across cultural and demographic dimensions, and proposes MosAIG, a multi-agent framework that leverages culturally distinct LLM personas to improve image quality and cultural grounding.

Original authors: Parth Bhalerao, Mounika Yalamarty, Brian Trinh, Oana Ignat

Published 2026-04-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Parth Bhalerao, Mounika Yalamarty, Brian Trinh, Oana Ignat

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a magical paintbrush that can draw anything you describe in words. If you say, "Draw a dog," it draws a perfect dog. If you say, "Draw a dog at the Eiffel Tower," it draws a dog at the Eiffel Tower. This technology is called Text-to-Image Generation, and it's getting very good at it.

But here's the catch: Most of these magical paintbrushes have only ever been taught to paint scenes from one culture. They know how to draw an American at the Golden Gate Bridge or a French person at the Eiffel Tower. But what happens if you ask them to draw a Vietnamese grandmother visiting the Golden Gate Bridge? Or a German teenager at the Taj Mahal?

According to this paper, the paintbrushes get confused. They often mix up the clothes, the faces, or the buildings, because they haven't practiced "cross-cultural" painting enough.

Here is a simple breakdown of what the researchers at Santa Clara University did to fix this:

1. The Problem: The "Tourist Trap" Bias

Think of current AI image generators like a tourist guidebook that only has photos of people from New York visiting New York. It doesn't know what it looks like when someone from Tokyo visits New York.

The Reality: The world is a mix. People travel, migrate, and share spaces.
The AI Gap: When you ask the AI to mix cultures (e.g., "A Vietnamese girl at the Golden Gate Bridge"), it often fails. It might put the girl in the wrong clothes, make her look like a stereotypical cartoon, or forget the bridge entirely.

2. The Solution: The "Dream Team" of AI Agents

To fix this, the researchers didn't just tell the AI to "try harder." They invented a new way of asking for the picture, called MosAIG.

Imagine you are commissioning a painting. Instead of giving the instructions to one person, you hire a Dream Team of four specialists who talk to each other before they start painting:

The Moderator: The boss who says, "Okay, we need a 12-year-old Vietnamese girl at the Golden Gate Bridge."
The Culture Expert: "Wait, a Vietnamese girl might wear an Áo Dài (a traditional dress), not a t-shirt. Let's make sure the colors match her culture."
The Landmark Expert: "Okay, but the Golden Gate Bridge is orange and has specific towers. Let's make sure the background is accurate."
The Detail Expert: "And since she's a child, her face and posture should look like a kid, not an adult."

These "agents" chat back and forth, refining the description until they have a perfect, detailed recipe. Then, they hand this recipe to the image generator.

The Analogy:

Simple Prompt (Old Way): "Draw a girl at a bridge." (Like giving a chef a vague order: "Make me dinner.")
Multi-Agent Prompt (New Way): "Draw a 12-year-old Vietnamese girl in a blue Áo Dài, standing on the Golden Gate Bridge, with the bay water in the background and the sun setting." (Like giving the chef a specific, gourmet recipe.)

3. The Big Test: The "Cultural Mix-Up" Dataset

The researchers created a massive test bank of 9,000 images. They mixed and matched:

5 Countries: USA, India, Germany, Spain, Vietnam.
25 Famous Landmarks: From the Taj Mahal to the White House.
Different People: Boys, girls, adults, elders.
5 Languages: English, German, Hindi, Spanish, Vietnamese.

They used this to test if the "Dream Team" (Multi-Agent) approach worked better than just asking the AI directly.

4. What They Found

The Dream Team Wins: When the AI agents talked to each other first, the resulting pictures were much better. The clothes were more accurate, the landmarks looked real, and the people looked like the right age and gender.
The Language Gap: The AI still struggles more with non-English languages. If you ask in English, it does a decent job. If you ask in Hindi or Vietnamese, it sometimes gets lost. It's like the AI speaks English fluently but is still learning to speak other languages.
The "Uncanny Valley": Even with the help, the AI sometimes makes weird mistakes, like giving someone six fingers or putting a German woman in a Spanish dress when visiting a Spanish castle.

5. Why This Matters

This paper is a wake-up call. As AI becomes part of our daily lives (creating movies, ads, and art), it needs to represent everyone, not just one type of person.

If we want AI to be fair and useful for the whole world, we can't just teach it about "American tourists." We need to teach it how to paint the global village—where a child from one culture can stand next to a landmark from another, and the AI knows exactly how to draw them both with respect and accuracy.

In short: The researchers built a "cultural translator" for AI artists, proving that when AI agents collaborate like a team of experts, they can finally paint a world that looks like our real, diverse world.

1. Problem Statement

Current text-to-image (T2I) generation models excel in culturally homogeneous settings but struggle significantly with multicultural scenes, where elements from different cultural origins (e.g., a person from one culture and a landmark from another) coexist in a single visual scene.

Limitations of Existing Benchmarks: Most existing datasets and benchmarks focus on narrow demographics (predominantly Western, adult, male) and single-culture scenarios (e.g., a Chinese temple with Chinese visitors). They fail to capture real-world multicultural interactions, such as a Vietnamese girl visiting the Golden Gate Bridge.
The Gap: There is a lack of systematic study on how T2I models handle the composition of diverse cultural, demographic, and landmark-specific cues. Current models often exhibit biases, hallucinations, or failures in grounding when asked to synthesize cross-cultural elements.

2. Methodology

A. The MosAIG Benchmark

The authors introduce the first benchmark specifically designed for multicultural T2I generation, comprising 9,000 images generated across a controlled matrix of variables:

Countries: 5 (Vietnam, USA, India, Germany, Spain).
Demographics: 3 age groups (Child, Adult, Elder), 2 genders (Male, Female).
Landmarks: 25 historical landmarks (5 per country).
Languages: 5 (English, German, Hindi, Spanish, Vietnamese).
Task: Generate an image of a person from Culture A at a landmark from Culture B.

B. MosAIG Framework (Multi-Agent Image Generation)

To address the complexity of composing these prompts, the authors propose MosAIG, a multi-agent framework that leverages Large Language Models (LLMs) with distinct cultural personas to construct rich, context-aware image captions.

Agent Architecture:
1. Moderator Agent: Receives raw inputs (nationality, age, gender, landmark, language) and assigns tasks.
2. Social Agents (3 distinct personas):
  - Country Agent: Focuses on traditional attire and cultural context.
  - Landmark Agent: Focuses on architectural details and environmental context.
  - Age-Gender Agent: Focuses on physical traits and age-appropriate mannerisms.
3. Interaction Loop: Agents engage in multi-round conversations to refine descriptions (e.g., checking if attire suits the age group or if the landmark interaction is culturally authentic).
4. Summarizer Agent: Consolidates the refined descriptions into a final, concise prompt (max 77 tokens) for the image generator.
Image Generation Models: The framework was tested using two state-of-the-art models:
- AltDiffusion: A multilingual open-source model supporting 18 languages.
- FLUX (Flux.1-lite): A high-performance English-centric model.

C. Evaluation Metrics

The study employs a multi-dimensional evaluation strategy:

Alignment: Measured via CLIPScore (text-image semantic similarity).
Quality: Measured via Inception Score (IS) for realism and diversity.
Aesthetics: Measured via a SigLIP-based predictor (visual appeal).
Knowledge: Assessed by replacing the landmark in the caption to see if the model's alignment score drops significantly (testing landmark recognition).
Fairness: Measured by the variance in alignment scores when demographic attributes (e.g., gender, nationality) are swapped while keeping other elements constant.

3. Key Contributions

First Multicultural Benchmark: Release of a dataset with 9,000 images covering 5 countries, 25 landmarks, and 5 languages to study cross-cultural generation.
MosAIG Framework: A novel multi-agent prompting strategy that decomposes cultural and demographic cues to generate more accurate and culturally grounded image captions.
Comprehensive Analysis: A systematic evaluation revealing significant disparities in model performance across languages, demographics, and cultural pairings.

4. Key Results

A. Impact of Multi-Agent Prompting

Quality & Knowledge: Multi-agent prompting significantly outperformed simple prompts. It improved Image Quality (e.g., AltDiffusion: 0.77 vs. 0.48) and Knowledge scores (better landmark recognition).
Fairness: The multi-agent approach reduced performance disparities across demographic groups (lower $\Delta S$ in fairness metrics), suggesting that explicit decomposition of cultural cues mitigates bias.
Alignment & Aesthetics: While alignment scores saw modest gains, aesthetic scores remained largely unchanged, indicating that multi-agent prompting improves semantic correctness and composition rather than just surface-level visual style.

B. Demographic and Linguistic Disparities

Age: Models performed best for Adults and worst for Elders and Children in terms of quality. Fairness was lower for children (higher variance).
Gender: Males generally achieved higher quality and knowledge scores than females, though fairness was slightly better for males.
Nationality (Person): Performance varied by the person's country. For example, Indian and Vietnamese subjects showed higher alignment with US landmarks than Spanish subjects did.
Landmarks: German landmarks were the most "known" by models (highest knowledge scores), followed by US landmarks.
Language: English prompts consistently achieved the highest alignment and knowledge scores. Non-English languages (Hindi, Vietnamese) showed significantly lower alignment, likely due to training data scarcity. However, non-English models sometimes showed better fairness (lower variance across demographics).

C. Error Analysis

Common Errors: The most frequent failures involved background inaccuracies (wrong landmark features) and human rendering errors (distorted bodies, missing fingers, incorrect traditional attire).
Cross-Cultural Grounding: The lowest alignment scores occurred in specific cross-cultural pairings, such as Vietnamese people at Spanish landmarks, indicating that models struggle most with "distant" cultural combinations.
Model Differences: Flux models produced more accurate backgrounds but blurrier images, while AltDiffusion models had sharper backgrounds but more severe body distortions and artifacts.

5. Significance and Future Directions

Real-World Applicability: This work highlights that current AI models are not yet ready for the diverse, multicultural reality of global society. The ability to generate accurate cross-cultural scenes is crucial for inclusivity in AI applications.
Prompt Engineering: The study demonstrates that richer prompt composition (via multi-agent collaboration) is a viable strategy to improve factual correctness and reduce bias, outperforming simple prompt engineering.
Bottlenecks: The research identifies multilingual support and training data diversity as the primary bottlenecks. Models perform poorly in low-resource languages and for underrepresented demographics.
Evaluation Needs: The authors argue for better evaluation metrics that go beyond CLIPScore, specifically targeting compositional correctness and demographic fidelity, as current automated metrics can be fooled by visually plausible but semantically incorrect images.

In conclusion, the paper establishes a new standard for evaluating and improving T2I models in multicultural contexts, proving that structured, multi-agent reasoning can significantly enhance the cultural grounding and fairness of generated images.