💬 NLP

LLMs as Cultural Archives: Cultural Commonsense Knowledge Graph Extraction

This paper introduces an iterative framework for extracting Cultural Commonsense Knowledge Graphs from large language models to structure implicit cultural knowledge, revealing that while these graphs enhance cultural reasoning and generation, they currently exhibit significant bias toward English even when representing non-English cultures.

Original authors: Junior Cedric Tonga, Chen Cecilia Liu, Iryna Gurevych, Fajri Koto

Published 2026-01-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Junior Cedric Tonga, Chen Cecilia Liu, Iryna Gurevych, Fajri Koto

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine that Large Language Models (LLMs) are like massive, invisible libraries. These libraries are filled with books written by people all over the world, so they contain a huge amount of information about how different cultures think, act, and celebrate. However, right now, this information is messy. It's buried deep inside the library, scattered in millions of unorganized sentences, making it hard to find or understand the specific "rules" of a culture.

This paper proposes a way to clean up that library and turn it into a Cultural Commonsense Knowledge Graph (CCKG). Think of this graph not as a list of facts, but as a train track system.

The Core Idea: Turning Facts into Train Tracks

Usually, if you ask an AI about culture, it might give you a single fact, like "Indonesians eat soto ayam for breakfast." That's a static fact.

The authors wanted to map out the journey of that fact. They built a system that asks the AI: "If someone wants breakfast, what do they do next? Then what? And what happens after that?"

The Track: "If you look for breakfast $\rightarrow$ you go to a warung (small shop) $\rightarrow$ you order soto ayam $\rightarrow$ you feel warm and comforted."

By connecting these steps, they create a "train track" of cultural logic. This allows the AI to understand not just what happens, but the flow of cultural life.

How They Built the Tracks (The Process)

The researchers treated the AI (specifically GPT-4o) as a "cultural archivist." They used a two-step process to build these tracks:

Laying the First Rails (Initial Generation): They asked the AI to generate simple "If-Then" statements about daily life topics (like food, weddings, or habits) for five different countries: China, Indonesia, Japan, England, and Egypt.
Extending the Tracks (Iterative Expansion): This is the clever part. Once the AI said, "If you look for breakfast, you order soto ayam," the researchers asked the AI to fill in the gaps.
- Intermediate Expansion: "What happens between looking for breakfast and ordering? Maybe you choose a side dish first."
- Forward Expansion: "What happens after ordering? Maybe you add sambal (chili sauce)."

They repeated this process, turning a single fact into a long, logical chain of cultural behavior.

The Big Surprise: The "English Bridge"

The researchers had a hypothesis: "If we want to know about Indonesian culture, we should ask the AI in Indonesian. If we want to know about Chinese culture, we should ask in Chinese."

They were wrong.

When they tested the quality of these cultural tracks, they found a strange but consistent pattern: The tracks built in English were much better than the tracks built in the native languages.

The Analogy: Imagine trying to draw a map of a city. You might think drawing it in the local language would be most accurate. But the researchers found that the AI's "internal map" was drawn most clearly in English. Even when describing Indonesian breakfasts or Egyptian weddings, the AI produced more logical, coherent, and culturally accurate "tracks" when it was forced to speak English.
The Result: The English versions of the knowledge graph were more consistent and made more sense to human evaluators than the versions written in Chinese, Arabic, or Indonesian.

Does This Help Smaller AIs?

The researchers then took these "train tracks" (the CCKG) and gave them to smaller, weaker AI models to see if it helped them understand culture better.

The Test: They asked these smaller models to answer questions about culture or write short stories.
The Outcome: When the smaller models were allowed to "look at the tracks" (using the CCKG as a guide), they got much better at answering questions and writing stories that felt culturally authentic.
The Catch: The biggest boost came when the models used the English tracks, even for non-English cultures. It seems that the "English bridge" is currently the most reliable way to access the AI's cultural knowledge.

What They Did NOT Claim

It is important to stick to what the paper actually says:

They did not say this system is ready for use in hospitals, schools, or legal systems.
They did not claim they solved the problem of AI bias or stereotypes (in fact, they noted that bias is still a risk and they didn't deeply audit for it).
They did not say that English is the "best" language for humans to speak; they only found that, for these specific AI models, English is the most stable language for extracting cultural logic.

Summary

The paper shows that we can treat AI models as giant archives of cultural knowledge. By asking the AI to build "train tracks" of cultural logic (If this happens, then that happens), we can create a structured map of how different cultures work. However, there is a twist: currently, these AI models seem to understand and express these cultural maps most clearly when they are speaking English, even when the culture itself is not English-speaking. This "English advantage" helps smaller AI models perform better, but it also highlights that our current AI tools are still uneven in how they represent the world's diverse cultures.

Technical Summary: LLMs as Cultural Archives: Cultural Commonsense Knowledge Graph Extraction

Problem Statement
Large Language Models (LLMs) encode vast amounts of cultural knowledge derived from diverse web-scale data, offering a potential resource for modeling cultural commonsense at scale. However, this knowledge remains largely implicit, unstructured, and difficult to interpret. Existing computational approaches to commonsense reasoning have historically treated culture as neutral or Western-centric (e.g., ATOMIC), while recent efforts to build cultural knowledge bases often rely on static, isolated facts from sources like Wikipedia or social media. These approaches fail to capture the procedural, sequential, and contextual nature of cultural practices, which are often defined by ordered chains of actions rather than atomic statements. Furthermore, there is a lack of structured, inferential representations that reflect local norms and practices across different languages.

Methodology
The authors propose an iterative, prompt-based framework to construct a Cultural Commonsense Knowledge Graph (CCKG), treating LLMs as "cultural archives" to systematically extract and structure culture-specific knowledge.

Framework Construction:
- Graph Definition: The CCKG is a directed labeled graph $G = (V, E)$ where nodes represent actions and edges represent labeled relations. An assertion is a triple $(A_i, R, A_j)$ interpreted as "if action $A_i$ occurs, then action $A_j$ follows."
- Relation Types: The framework utilizes five relation types: $xNext$ (agent's next action), $xEffect$ (effect on agent), $xNeed$ (pre-requisite for agent), $oNext$ (others' next action), and $oEffect$ (effect on others).
- Extraction Process (Algorithm 1):
  - Initial Generation: Given a subtopic, country, and language, an LLM (GPT-4o) generates initial "if-then" assertions based on predefined relation types.
  - Iterative Expansion: The knowledge base is enriched through two mechanisms:
    - Intermediate Expansion: Decomposing a direct link $(A_i \to A_j)$ into a sequence of finer-grained intermediate steps.
    - Forward Expansion: Generating new continuations starting from the resulting action $A_j$ .
  - Pruning: To ensure efficiency and reduce redundancy, generated actions are matched against existing nodes using similarity scores; duplicates are merged, and the expansion list is capped at six assertions per node.
Experimental Setup:
- Scope: The framework was applied to five countries (China, Indonesia, Japan, England, Egypt) covering 11 daily-life topics and 65 subtopics.
- Languages: Knowledge graphs were generated in both English and the respective native languages (Chinese, Indonesian, Japanese, Arabic).
- Evaluation: Human annotators (native speakers) evaluated the generated graphs on three dimensions: Correctness (valid actions and relations), Cultural Relevance (specificity to the target culture), and Logical Path Coherence (consistency of inferential chains).
- Downstream Tasks: The extracted CCKG was used to augment smaller LLMs for three tasks: Multiple-Choice Question Answering (MCQA), Sentence Completion, and Story Generation. Benchmarks included IndoCulture and ArabCulture.

Key Contributions

CCKG Framework: The introduction of a method to extract multilingual, culture-specific inferential knowledge chains from LLMs, moving beyond static facts to procedural, multi-step reasoning structures.
Empirical Analysis of Cultural Encoding: A systematic evaluation revealing that while native languages capture richer cultural details, English often serves as a more coherent and consistent medium for representing cultural knowledge graphs within current LLMs.
Performance Augmentation: Demonstration that augmenting smaller or base models with CCKG (via in-context assertions or paths) significantly improves their performance on cultural reasoning and story generation tasks, particularly when using native-language assertions for specific cultural contexts.

Results

Graph Quality: In human evaluations, English-generated CCKGs consistently outperformed native-language versions across correctness, cultural relevance, and path coherence for non-English cultures (e.g., Chinese, Indonesian, Arabic). This suggests an asymmetry in current multilingual LLMs where English acts as a more stable representational medium for cultural reasoning, likely due to richer pretraining data.
Reasoning Tasks (MCQA): Integrating CCKG knowledge improved accuracy on IndoCulture and ArabCulture benchmarks. Native-language augmentation yielded the highest gains (average +1.2 points), outperforming English augmentation and existing baselines like Mango. Notably, Chain-of-Thought (CoT) prompting performed worse than the baseline, suggesting cultural commonsense relies more on intuitive context than explicit step-by-step logic.
Generation Tasks: For sentence completion and story generation, CCKG augmentation (specifically path-based) improved sentence similarity and coherence scores. In story generation, incorporating CCKG paths consistently increased Cultural Relevance scores (average +1.4 increase for Llama models), with the most significant improvements observed in English story generation, though native-language generation also benefited in specific cases (e.g., Indonesian stories with Llama3.1-8B-IT).
LLM-as-a-Judge: Automated evaluation using LLMs correlated strongly with human judgments for native-language stories but showed moderate to weak correlation for English stories, particularly regarding fluency and coherence, likely due to stylistic variations and translation nuances.

Significance and Claims
The paper posits that LLMs can serve as viable "cultural archives" for extracting structured, inferential knowledge, provided the extraction process is systematic. The authors claim that chain-structured cultural knowledge is a practical substrate for developing culturally grounded NLP systems.

The study highlights a critical limitation and insight: despite the goal of modeling local cultural reasoning, current LLMs represent cultural commonsense most effectively through English, even when the target culture is non-English. This finding challenges the assumption that native-language outputs are inherently superior for cultural tasks and suggests that English may act as a "pivot" for coherent cultural reasoning in current models.

The authors conclude that while their method shows promise, it is a research prototype. They caution against using the extracted content as a formal dataset for production systems without addressing potential risks, such as the reinforcement of stereotypes, and emphasize the need for future work to investigate bias and extend the framework to more fine-grained cultural levels.