💬 NLP

Common to Whom? Regional Cultural Commonsense and LLM Bias in India

This paper introduces Indica, the first benchmark revealing that cultural commonsense in India is predominantly regional rather than national, and demonstrates that current large language models struggle with this diversity while exhibiting significant geographic bias toward Central and North India.

Original authors: Sangmitra Madhusudan, Trush Shashank More, Steph Buongiorno, Renata Dividino, Jad Kabbara, Ali Emami

Published 2026-04-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sangmitra Madhusudan, Trush Shashank More, Steph Buongiorno, Renata Dividino, Jad Kabbara, Ali Emami

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: India Isn't One Big "Flavor"

Imagine you walk into a massive ice cream shop called "India." The menu says "Indian Ice Cream." You might expect every scoop to taste exactly the same—maybe a uniform "Masala Chai" flavor.

But this paper argues that India is actually a giant food truck festival, not a single factory. Inside that festival, the North serves spicy Lassi, the South serves sweet Payasam, the East serves Misti Doi, and the West serves Vada Pav. They are all "Indian," but they are totally different.

The researchers wanted to see if Artificial Intelligence (AI) knows the difference between these regional flavors, or if it just assumes everything tastes like "North Indian Masala."

The Problem: The AI's "One-Size-Fits-All" Glasses

For a long time, AI researchers treated countries like single blocks of cheese. They assumed that if you asked an AI, "What do people in India eat for breakfast?", the AI would give one answer and call it a day.

But in a country as huge and diverse as India (with 28 states, 8 union territories, and 22 languages), that assumption is broken. It's like asking a chef, "What is the national dish of the USA?" Is it a burger? A taco? A bagel? It depends on where you are standing.

The Solution: INDICA (The Regional Taste Test)

The researchers built a new test called INDICA (Indian Cultural Commonsense Inventory with Cross-regional Answers). Think of it as a regional taste test for AI.

The Menu (The Questions): They created 515 questions about everyday life, like "What gift do you bring to a wedding?" or "How do you signal you are full after a meal?"
The Tasters (The Humans): They didn't just ask one person. They asked 5 groups of real humans, one from each major region: North, South, East, West, and Central India.
The Results: They found something shocking.
- Only 39.4% of the questions had the same answer across all five regions.
- 60%+ of the time, the answer depended entirely on where you were.
- Example: If you ask, "What is a good gift for new parents?"
  - North India says: "Jewelry."
  - South India says: "Bangles."
  - East/West India says: "Clothes."
  - Central India says: "Sweets and Jewelry."

The Test: How Did the AI Do?

The researchers took 8 of the smartest AI models (like GPT-5, Claude, Gemini, etc.) and put them through this taste test.

Result 1: The AI is "Vague" (The Generalist)
When asked to answer for a specific region, the AI got the right answer only 13% to 20% of the time.

The Metaphor: The AI is like a tourist who has read a travel brochure but never actually visited the country. It knows "India has weddings," but it doesn't know that in the South, you wear silk, while in the North, you might wear different silk. It gives a "safe," generic answer that is half-right but misses the specific cultural nuance.

Result 2: The AI is Biased (The "Default" Setting)
This is the most critical finding. When the researchers removed the region name from the question (e.g., just asked "What gift is given?" without saying "In South India"), the AI didn't pick a random answer. It had a favorite.

The Bias: The AI consistently picked answers from North and Central India about 30–40% more often than it should have.
The Under-Selection: It almost ignored East and West India.
The Metaphor: Imagine a map of India where the AI's "cursor" is stuck on New Delhi and Bhopal. Even when you ask about Mumbai (West) or Kolkata (East), the AI keeps pointing to the middle.

Why does this happen?
The paper explains it like a radio signal:

The Noise: The internet is full of content in Hindi (spoken mostly in the North/Central). Content in Tamil, Bengali, or Marathi is much quieter on the internet.
The Translation: The AI "learns" by reading the internet. Since it reads more Hindi/English content from the North, it thinks that is the whole story. It's like trying to learn about the whole world by only reading newspapers from one specific city.

The Takeaway: Why This Matters

This paper is a wake-up call. It tells us that AI cannot treat nations as single, uniform blocks.

For India: If we want AI to be helpful in India, it needs to know that "Indian culture" isn't one thing. It's a mosaic.
For the World: This applies to China, Brazil, Nigeria, and the US too. A "national" AI is a myth.

The Final Analogy

Imagine you are hiring a tour guide for a trip across India.

Old AI: The guide says, "In India, we all eat rice and wear white." (Wrong. In the North, they eat wheat and wear colorful clothes).
New AI (with INDICA): The guide says, "In the North, we eat wheat. In the South, we eat rice. In the East, we celebrate Durga Puja. In the West, we celebrate Ganesh Chaturti."

The paper argues that for AI to be truly smart and respectful, it needs to stop being a "National Tour Guide" and start being a "Regional Specialist." It needs to know that culture lives in the details, not just the borders.

1. Problem Statement

Existing benchmarks for cultural commonsense reasoning often treat nations as monolithic entities, assuming that cultural practices and norms are uniform within national borders. This assumption fails in culturally heterogeneous nations like India, which comprises 28 states, 8 union territories, and 22 official languages.

The Gap: Current datasets (e.g., CultureBank, CulturalBench) aggregate data at the national level, obscuring sub-national diversity.
The Question: Does cultural commonsense hold uniformly within a nation, or does it vary significantly at the sub-national (regional) level? Furthermore, do Large Language Models (LLMs) capture this regional nuance, or do they exhibit implicit geographic biases?

2. Methodology: The INDICA Benchmark

The authors introduce INDICA (Indian Cultural Commonsense Inventory with Cross-regional Answers), the first benchmark designed to evaluate regional variation in cultural commonsense. The creation process involved three main phases:

A. Question Creation (Anthropological Grounding)

Taxonomy: The authors grounded their domain selection in the Outline of Cultural Materials (OCM), a standard anthropological taxonomy.
Domains: They selected 8 domains relevant to everyday life: Interpersonal Relations, Education, Clothing & Adornment, Food Processing & Consumption, Communication, Finance, Festivals & Rituals, and Traffic & Transport Behavior.
Generation: Using GPT-4, they generated topics and seed questions, which were manually refined to ensure they focused on everyday practices rather than institutional facts (e.g., government policies).
Scale: This resulted in 515 unique questions across 39 topics.

B. Response Collection

Participants: The study recruited 5 participants per region (total 25) from five major Indian regions: North, South, East, West, and Central.
Criteria: Participants were required to have lived in their respective region for the majority of their lives.
Data Volume: Each participant answered all 611 initial questions, yielding 15,275 raw responses.

C. Gold Standard Establishment

The authors established gold standards through a rigorous consensus process involving automated (GPT-4o) and manual verification:

Intra-Region Agreement: For a region to have a "gold answer," at least 4 out of 5 participants must provide semantically equivalent answers.
Inter-Region Agreement: Pairs of regions were compared to see if they shared the same cultural practice.
Universal Agreement: Checked if all five regions agreed on a single answer.

Final Dataset: 515 questions yielded 1,630 region-specific question-answer pairs (some regions had "N/A" if consensus wasn't reached).

3. Evaluation Framework

The authors evaluated 8 state-of-the-art LLMs (including GPT-5.2, Claude Sonnet 4.5, Gemini 3.0, Grok-4, Llama 3.3, etc.) using two complementary tasks:

Region-Anchored Short Answer (RASA):
- Task: Models were given a question with a specific region prepended (e.g., "In South India, what is...?").
- Goal: Test if models can generate accurate, region-specific knowledge.
- Scoring: An LLM judge (Gemini 3.0 Flash) scored responses as Fully Correct, Partially Correct, or Incorrect based on alignment with the gold standard.
Region-Agnostic Multiple Choice Questions (RA-MCQ):
- Task: Models were presented with a question and options representing different regional answers, without specifying the region.
- Goal: Detect implicit geographic bias by observing which regional practice the model defaults to as the "standard" Indian answer.
- Analysis: Selection rates were compared against a uniform distribution baseline (20% per region) using Chi-square tests.

4. Key Results

A. Cultural Commonsense is Predominantly Regional

Low National Consensus: Only 39.4% of questions achieved unanimous agreement across all five regions.
High Regional Variation: In domains like Festivals & Rituals (1.8% universal agreement) and Food Processing (6.0%), cultural practices are highly distinct. Even in standardized domains like Education, universal agreement was only 13.8%.
Conclusion: Treating India as a single cultural unit is empirically invalid; commonsense is primarily regional.

B. LLM Performance Gaps

Low Accuracy: Models achieved only 13.4% – 20.9% "Fully Correct" accuracy on region-specific questions.
Over-Explanation: The majority of errors (61.3%–75.3%) were "Partially Correct." Analysis showed models tend to over-explain using a generic cultural template rather than retrieving specific regional details. They often bury the correct regional fact under generalized cultural noise.

C. Systematic Geographic Bias

When geographic context was removed (RA-MCQ), all models exhibited significant bias ( $p < 0.001$ ):

Over-Selection of Central & North India: Models consistently selected answers from Central and North India 30–40% more often than expected (e.g., Central India selected at 1.37x the expected rate).
Under-Selection of East & West: East and West India were significantly under-represented (selected at ~0.73x–0.82x expected rates).
Causes: The authors attribute this to:
1. Data Imbalance: Hindi (spoken in North/Central) dominates Indian web content.
2. Tokenization: Multilingual tokenizers fragment underrepresented languages (e.g., Tamil, Bengali) into less meaningful sub-words.
3. Media Representation: Bollywood and Hindi media reinforce North/Central India as the cultural default.

5. Significance and Contributions

INDICA Benchmark: Provides the first dataset explicitly designed to test sub-national cultural variation, moving beyond the "nation-as-monolith" assumption.
Methodological Framework: Offers a generalizable pipeline (OCM-grounded design $\to$ regional data collection $\to$ bias measurement) applicable to other heterogeneous nations (e.g., China, Brazil, Nigeria).
Empirical Evidence: Demonstrates that current SOTA LLMs fail to capture regional cultural nuances and default to a "Hindi-centric" or "North-Centric" view of India, potentially reinforcing stereotypes and erasing local identities.
Implications for AI: Highlights that scaling data alone is insufficient for cultural commonsense; models require regionally grounded training and evaluation to be truly culturally competent.

6. Limitations & Future Work

Granularity: The study uses broad regional divisions (5 regions) rather than state-level granularity, potentially masking intra-regional diversity (e.g., differences between Tamil Nadu and Kerala within "South India").
Demographics: Participants were recruited via Prolific, potentially skewing toward English-speaking, urban populations.
Temporal Validity: Cultural practices evolve; the dataset represents a snapshot of 2025.

In conclusion, the paper argues that cultural commonsense is not national—it is regional. To build culturally competent AI, developers must model diversity where it exists, rather than assuming uniformity.