🤖 AI

IndicFairFace: Balanced Indian Face Dataset for Auditing and Mitigating Geographical Bias in Vision-Language Models

This paper introduces IndicFairFace, a novel and ethically sourced dataset of 14,400 images balanced across India's 28 states and 8 Union Territories, to quantify and mitigate geographical bias in Vision-Language Models while maintaining high retrieval accuracy.

Original authors: Aarish Shah Mohsin, Mohammed Tayyab Ilyas Khan, Mohammad Nadeem, Shahab Saquib Sohail, Erik Cambria, Jiechao Gao

Published 2026-02-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Aarish Shah Mohsin, Mohammed Tayyab Ilyas Khan, Mohammad Nadeem, Shahab Saquib Sohail, Erik Cambria, Jiechao Gao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a super-smart digital assistant, a "Vision-Language Model" (VLM), that has read almost everything on the internet and seen billions of photos. You ask it, "Show me a picture of an Indian person."

Ideally, this assistant should show you a beautiful, diverse collage: a farmer from the green fields of Punjab, a fisherman from the islands of Andaman, a student from the hills of Sikkim, and a shopkeeper from the bustling markets of Mumbai.

But in reality, the paper IndicFairFace reveals that this assistant is biased. When you ask it, it mostly shows you people from just a few places—like Rajasthan or Tamil Nadu—while completely ignoring the Northeast, the islands, and many other regions. It's as if the assistant thinks "Indian" only looks a certain way, based on the most popular photos it saw online, rather than the true, messy, wonderful diversity of the country.

Here is a simple breakdown of what the authors did to fix this:

1. The Problem: The "Monolithic" Mirror

Think of the current AI models as a funhouse mirror. Instead of reflecting the real India (which has 28 states and 8 Union Territories, each with unique faces, skin tones, and styles), the mirror squashes everyone into a single, narrow shape.

The researchers found that if you ask these AIs for "Indian faces," they pull from a tiny slice of the population. They over-represent big, urban, or tourist-heavy areas and completely erase the faces of people from the Northeast, the islands, and rural areas. This isn't just a technical glitch; it's a form of digital erasure that can cause real-world harm, like an AI hiring tool rejecting a qualified candidate just because their face doesn't match the AI's narrow idea of what an "Indian" looks like.

2. The Solution: Building a "Fairness Scale"

To fix a broken scale, you need a perfect set of weights to calibrate it. The authors created IndicFairFace, which is exactly that: a perfectly balanced dataset.

The Recipe: They gathered 14,400 photos of real people.
The Balance: They didn't just grab random photos. They ensured there were exactly the same number of men and women from every single state and territory in India.
The Source: They were careful to use only images that were legally free to use (like from Wikimedia Commons) or found through ethical web searches, ensuring they didn't steal anyone's privacy.

Think of this dataset as a perfectly balanced choir. Before, the choir was mostly tenors from one city. Now, every voice from every region of India has an equal seat in the choir.

3. The Fix: "Surgical" De-biasing

Once they had this balanced choir, they used it to "re-train" the AI's brain. But they didn't want to re-teach the whole AI from scratch (which would be slow and expensive). Instead, they used a clever technique called Iterative Nullspace Projection (INLP).

Here is an analogy for how this works:
Imagine the AI's brain is a giant library of ideas. In this library, there is a specific "corridor" where the AI keeps all its stereotypes about geography.

The Old Way: You might try to burn down the library to get rid of the bad books. But then you lose all the good books too (the AI stops working well).
The New Way (INLP): The researchers found the specific "corridor" where the bias lived. They then built a forcefield around that corridor. When the AI tries to walk down that corridor to make a biased decision, the forcefield gently pushes it back onto the main path.
The Result: The AI still knows everything about cars, animals, and math (its general intelligence is untouched), but it can no longer "walk down the bias corridor." It now sees "Indian" as a diverse concept again.

4. The Proof: Does it still work?

A common fear with fixing AI bias is that you might break the AI. The authors tested their "de-biased" models on dozens of other tasks (like identifying cars, animals, or reading text in images).

The result? The AI didn't lose its smarts.

Its accuracy on other tasks dropped by less than 1.5% (which is basically nothing).
But when asked about "Indian people," it suddenly started showing faces from all over the country, not just the usual suspects.

Why This Matters

This paper is a blueprint for making AI fairer, not just for India, but for the whole world. It shows that we can't just say "AI is neutral" because it was trained on the internet. The internet is full of imbalances. To fix the AI, we need to build our own balanced datasets that represent the real world, not just the popular parts of it.

In short: The authors built a balanced photo album of India, used it to gently nudge the AI's brain out of its narrow thinking, and proved that the AI can be fair without losing its intelligence. It's a step toward ensuring that when AI looks at the world, it sees everyone.

1. Problem Statement

Vision-Language Models (VLMs), such as CLIP and its variants, have achieved state-of-the-art performance in multimodal tasks. However, they inherit and amplify societal biases from their web-scale training data. While existing fairness research has addressed global race and gender imbalances, it largely treats "Indian" as a monolithic category. This oversimplification ignores the vast intra-national diversity across India's 28 states and 8 Union Territories (UTs).

The paper identifies a critical geographical bias where VLMs disproportionately associate the concept of "Indianness" with a narrow set of North-Western and Southern states (e.g., Rajasthan, Uttar Pradesh, Tamil Nadu), while severely under-representing or ignoring North-Eastern, Himalayan, and island regions. This leads to representational harm in applications like identity verification, recruitment, and recommendation systems, where equitable representation is crucial.

2. Methodology

The proposed work follows a three-tier architecture: Dataset Creation, Bias Auditing, and Debiasing.

A. Dataset Creation: IndicFairFace

The authors introduce IndicFairFace, the first geographically balanced face dataset for the Indian context.

Scale & Composition: Contains 14,400 images (7,200 original + 7,200 augmented), equally distributed across 36 geographical units (28 States + 8 UTs) with gender parity (Male/Female).
Data Sourcing:
- Wikimedia Commons: Used SPARQL queries on Wikidata to filter humans based on citizenship (India) and birthplace/residence/work location linked to specific states.
- Web Scraping: For states with insufficient Wikimedia coverage (e.g., Mizoram, Ladakh), images were retrieved via SerpAPI (Google Image Search) using state-specific queries.
Preprocessing: A rigorous pipeline using OpenCV and YOLOv8 was applied, including:
- Quality Control: Rejection of grayscale images, synthetic/cartoon faces (via Laplacian variance), and low-skin-pixel ratio images.
- Standardization: Faces cropped with 20% padding and resized to 512×512 RGB.
Augmentation: To ensure strict balance (100 faces per gender per state), synthetic augmentations (flips, rotations, hue jitter) were applied to under-represented regions.

B. Bias Auditing

The authors evaluated multiple CLIP-based VLMs (OpenCLIP, SigLIP, MetaCLIP, MobileCLIP, OpenVision) using text-to-image retrieval.

Prompts: Tested variations like "An Indian person," "An Indian male," and "An Indian female."
Metrics:
- Top-k Similarity: Analyzed the distribution of states in the top-500 retrieved images.
- Average Similarity: Measured mean cosine similarity between state-specific image embeddings and the target text prompt.
- Normalized Jensen-Shannon (JS) Score: Quantified the statistical distance between the retrieval distribution and a uniform distribution.

C. Debiasing Strategy

To mitigate the identified bias without retraining the massive models, the authors employed a post-hoc approach:

Iterative Nullspace Projection (INLP): Identifies linear classifiers that separate embeddings based on state identity. The embeddings are iteratively projected onto the nullspace of these classifiers to remove bias directions.
Spherical Linear Interpolation (SLERP): Controls the strength of the projection to prevent over-correction and semantic drift.
Similarity Compensation: Aligns the debiased embeddings back toward the original concept vector (e.g., "Indian person") to preserve semantic utility.

3. Key Contributions

IndicFairFace Dataset: A novel, ethically sourced, and strictly balanced dataset covering all Indian states and UTs, addressing the gap in state-level granularity for Indian facial data.
Empirical Evidence of Bias: Comprehensive auditing reveals that current VLMs collapse "Indianness" into a North-West–South corridor, with North-Eastern states appearing in <0.5% of top retrievals.
Effective Debiasing Framework: Demonstrates that INLP combined with SLERP can reduce inter-state variance in similarity scores by approximately 37% while maintaining model utility.
Preservation of Generalization: Proves that the debiasing process does not degrade performance on external benchmarks (e.g., ImageNet, CIFAR), with accuracy drops remaining below 1.5%.

4. Results

Bias Quantification: Before debiasing, models like OpenCLIP and MetaCLIP retrieved 45–50% of top-500 images from just five states (Rajasthan, UP, Tamil Nadu, Puducherry, Gujarat). North-Eastern states were virtually absent.
Debiasing Efficacy:
- Uniformity: Post-debiasing, the distribution became significantly more uniform. Previously dominant states dropped to 5–7% of retrievals, while under-represented regions (e.g., Nagaland, Mizoram) rose to 2–3%.
- Metric Improvement: The standard deviation ( $\sigma$ ) of average similarity scores across states decreased significantly (e.g., a 22.24% reduction for OpenCLIP), and JSD values dropped by up to 90% in some models, indicating a move toward a uniform distribution.
Generalization: Table 4 shows that debiased models maintained near-identical accuracy on 35 external classification benchmarks (e.g., ImageNet, Cars, FER2013). In some cases, performance slightly improved.
Downstream Task (Job Application Simulation): In a zero-shot classification task simulating recruiter decisions, debiased models achieved near-perfect accuracy (1.000) in identifying candidates from "at-risk" states (previously misclassified as "foreign"), whereas base models showed significant failure rates (e.g., 0.000 for some states in MobileCLIP).

5. Significance

This work establishes a critical foundation for region-aware fairness in multimodal AI.

Technical Impact: It challenges the "monolithic" approach to demographic grouping in fairness research, proving that intra-national diversity must be accounted for to avoid representational harm.
Practical Application: The proposed debiasing framework offers a computationally efficient, post-hoc solution to correct geographical bias in pre-trained VLMs without the need for expensive retraining.
Ethical Relevance: By ensuring equitable representation of India's diverse regions, the work mitigates risks in high-stakes applications like automated hiring, security, and content generation, ensuring that AI systems do not perpetuate the marginalization of specific Indian communities.

The dataset and code are publicly available at: https://github.com/aarishshahmohsin/IndicFairFace.