💻 computer science

Where Do Images Come From? Analyzing Captions to Geographically Profile Datasets

This paper investigates the geographical bias in large-scale multimodal datasets by using LLMs to map image-caption pairs to countries, revealing that training data is heavily skewed toward high-GDP, English-speaking nations and lacks significant representation from the Global South.

Original authors: Abhipsa Basu, Yugam Bahl, Kirti Bhagat, Preethi Seshadri, R. Venkatesh Babu, Danish Pruthi

Published 2026-02-11

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Abhipsa Basu, Yugam Bahl, Kirti Bhagat, Preethi Seshadri, R. Venkatesh Babu, Danish Pruthi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The "Global Mirror" Problem: Why AI Sees the World Through a Western Lens

Imagine you are building a giant, magical photo album that is supposed to contain a picture of everything in the world. You ask the album, "Show me a picture of a house," and it shows you a suburban home with a white picket fence in America. You ask, "Show me a road," and it shows you a wide, paved highway in Europe.

You might think, "Wait, what about the colorful houses in Morocco? What about the winding dirt paths in the Andes? What about the bustling streets of Mumbai?"

This paper, "Where Do Images Come From?", is a scientific investigation into why these "magical photo albums" (which we call AI models like Stable Diffusion) have such a skewed view of our planet.

1. The "Rich Neighbor" Bias (The Core Discovery)

The researchers looked at the massive collections of images and text used to train AI. They wanted to know: Where are these images actually from?

To do this, they used a smart AI (an LLM) to read the captions of millions of pictures. If a caption said, "A beautiful sunset in Paris," the AI knew to tag that image as France.

The Finding: The data is incredibly lopsided. It’s like throwing a massive party where 80% of the guests are from the US, UK, and Canada, while people from Africa and South America are barely invited.

The researchers found a direct link between money and visibility: the wealthier a country is (measured by GDP), the more likely its images are to be in the AI's "brain." If a country is economically powerful, the AI "sees" it; if not, the AI is essentially blind to it.

2. The "Stereotype Trap" (Diversity vs. Frequency)

You might think, "Okay, so the US has more pictures. But surely, the pictures of India or Mexico are diverse and colorful!"

The researchers tested this using a "Diversity Score." They found that just because a country has more pictures doesn't mean those pictures are varied.

The Analogy: Imagine two travelers.

Traveler A (Norway) takes 1,000 photos, but they are all of the same snowy mountain from slightly different angles.
Traveler B (Mexico) takes only 100 photos, but they include a street market, a wedding, a police car, and a mountain.

Even though Traveler A took more photos, Traveler B’s collection is much more "diverse." The researchers found that AI training data often suffers from this. It might have many images of a country, but they all look the same, which leads to stereotyping.

3. The "Uncanny Valley" of AI Generations

Finally, the researchers tested the AI itself. They asked it to generate images of specific things in specific places, like "A car in India."

The Result: The AI produced images that looked "realistic" (they didn't look like fake cartoons), but they had terrible coverage.

When asked for an Indian car, the AI tended to generate old, broken-down, or dilapidated vehicles. It captured the idea of a car in India, but it missed the reality of the modern, diverse, and shiny cars actually driving on Indian roads today. It’s like an artist who has only ever seen one postcard of a city and thinks the whole city looks exactly like that one card.

Why does this matter?

If we use these AI models to design cities, write history books, or create movies, we are accidentally teaching the world to see through a very narrow, Western-centric window.

The takeaway: To make AI truly "intelligent" and "global," we can't just scrape the internet and hope for the best. We have to intentionally go out and find the stories, the colors, and the landscapes of the entire world—not just the parts that are easiest to find online.

Technical Summary: "Where Do Images Come From? Analyzing Captions to Geographically Profile Datasets"

1. Problem Statement

Recent advancements in Vision-Language Models (VLMs) and text-to-image generators (like Stable Diffusion) have revealed significant geographic biases, often producing outputs that are heavily skewed toward Western, English-speaking, and high-income regions. While the symptoms of this bias (model output) are documented, the root cause—the geographic distribution of the massive, web-scale datasets used for training—remains largely unexamined. Characterizing this distribution is difficult because web-scale datasets lack reliable metadata (URLs are noisy, and image geolocalization is often limited to street-view imagery).

2. Methodology

The authors propose a novel, data-centric framework to geographically profile large-scale datasets by leveraging the semantic information present in image captions.

Geolocalizing Captions (Extract–Retrieve–Predict): To overcome the limitations of naive string matching or standard NER (which suffer from low recall and ambiguity), the authors use a three-step LLM-based pipeline:
1. Extract: An LLM identifies potential location mentions within a caption.
2. Retrieve: These mentions are used to query the GeoNames database to retrieve the top- $k$ most likely matching locations and their corresponding countries.
3. Predict: The LLM uses the retrieved candidates as context to perform final country disambiguation.
Entity-Presence Filtering: To ensure the analysis is accurate, the authors train a binary classifier (using an SVM on CLIP image features) to verify that the visual entity mentioned in the caption (e.g., "house," "car") is actually present in the image. This prevents "noise" from captions that mention a location but depict something else.
Diversity Measurement: The authors use the Vendi Score to quantify semantic and visual diversity. This metric measures the "effective number" of distinct samples in a feature space, allowing for a fair comparison between datasets of different sizes.
Datasets Analyzed: The study profiles three major datasets: Re-LAION2B-en, DataComp1B, and Conceptual Captions (CC12M), as well as multilingual subsets of Re-LAION.

3. Key Contributions

First Large-Scale Geo-Profiling: Provides the first systematic, large-scale analysis of geographic representation in the datasets that power modern multimodal AI.
Robust Geo-Profiling Pipeline: Introduces an LLM-augmented "extract-retrieve-predict" method that outperforms traditional geoparsing tools.
Quantification of Bias: Establishes a formal link between a country's economic status (GDP) and its representation in training data.
Diversity vs. Frequency Analysis: Disentangles the relationship between how often a country appears in a dataset and how diverse its visual/semantic content actually is.

4. Key Results

Severe Geographic Skew: In English-language datasets, the US, UK, and Canada account for nearly half (48%) of all location-specified samples. Conversely, South American and African countries are drastically underrepresented (1.8% and 3.8%, respectively).
Economic Correlation: There is a very strong correlation ( $\rho = 0.82$ ) between a country's nominal GDP and its frequency in the datasets, indicating that "wealthier data" dominates the training sets.
Language-Region Alignment: In multilingual subsets, representation follows language dominance (e.g., Spanish captions significantly increase South American representation), suggesting that multilingual data is a key lever for improving diversity.
The "Diversity Gap": Higher representation does not guarantee higher diversity. For example, while Norway appears frequently, its images are semantically narrow (mostly landscapes), whereas Mexico shows higher visual and semantic diversity despite lower frequency.
Model Generation Failures: Analyzing Stable Diffusion v1.3 reveals that while generated images are "realistic" (high precision), they have near-zero recall. This means the model fails to cover the true visual variety of a country, often falling into stereotypes (e.g., generating only old, dilapidated cars when prompted for "Indian cars").

5. Significance

This paper shifts the focus of AI ethics from evaluating model outputs to auditing training data. By proving that geographic bias is a direct consequence of the composition of web-scraped datasets, the authors provide a roadmap for data curators. The work highlights that simply "collecting more data" is insufficient; curators must actively seek out diverse, non-Western, and multi-contextual data to prevent multimodal models from reinforcing global socio-economic stereotypes and "erasing" the visual reality of much of the world.