💬 NLP

Cross-Lingual Consensus: Aligning Multilingual Cultural Knowledge via Multilingual Self-Consistency

This paper proposes a novel self-supervised framework that leverages multilingual self-consistency and self-critique to bridge cross-lingual knowledge gaps, significantly improving cultural alignment in LLMs by surfacing and transferring latent cultural knowledge from local languages to English without relying on external data.

Original authors: Andrew Ivan Soegeng, Patrick Sutanto, Tan Sang Nguyen

Published 2026-05-22

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Andrew Ivan Soegeng, Patrick Sutanto, Tan Sang Nguyen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a brilliant, multilingual librarian named "LLM" who has read almost every book in the world. This librarian is incredibly smart, but they have a strange habit: when you ask them a question in English, they give you a very polished, standard answer. However, when you ask the same question in a local language (like Indonesian, Hausa, or Sundanese), their answer might be a bit messy or miss the cultural nuances, even though they actually know the right answer deep down.

The problem is that the librarian's "English brain" is so loud that it drowns out the quiet, culturally rich knowledge hidden in their other languages. They tend to default to Western perspectives, even when talking about local customs.

This paper proposes a clever, self-teaching trick to fix this without hiring any new human experts. Here is how it works, using a simple analogy:

The "Group Consensus" Trick

Imagine the librarian is asked a question about a local festival. Instead of just giving one answer, the paper's method asks the librarian to answer the same question multiple times in two different ways:

Once in English.
Once in the local language.

The researchers then act like a referee. They look at the "English answers" and the "Local Language answers" separately.

The Consistency Check: They ask, "Do the English answers all agree with each other?" and "Do the Local Language answers all agree with each other?"
The Winner: Whichever language group produces answers that are more consistent with each other is declared the "Stronger Language" for that specific topic.
- Example: If the English answers are all over the place (one says "celebrate," one says "fast," one says "nothing"), but the Local Language answers are all saying the exact same thing (e.g., "celebrate with specific food"), the system knows the Local Language holds the true cultural knowledge.

The "Translation and Critique" Lesson

Once the system identifies that the Local Language has the better, more consistent answer, it does two things:

Translation: It takes that "winning" local answer and translates it back into English (or the weaker language).
Self-Critique: It asks the librarian to look at a "bad" answer they previously gave, compare it to the "winning" answer, and write a critique explaining why the bad answer was wrong.

This process creates a new set of training data where the librarian teaches itself. It learns: "Ah, when I answer in English, I was wrong. The local language version was the consistent truth. I need to remember that."

The Results: What Actually Happened

The researchers tested this on a model called Llama 3.1. Here is what they found:

The Big Win: When they asked the model questions in English about cultures from around the world, the model got significantly better at giving the right cultural answers. On average, their accuracy went up by about 5%. This means the model stopped being so "Western-centric" and started reflecting diverse cultures more accurately, even when speaking English.
The Catch: The method worked best for languages that had enough data to begin with. For very rare, low-resource languages (like Hausa or Assamese), the model actually got slightly worse. It's like trying to teach a student using a textbook that only has a few pages; the student gets confused and forgets what they already knew.
No Magic External Help: The best part is that they didn't need to hire human experts to write the answers or use a "smarter" AI to fix the mistakes. The model did all the work itself, using its own internal knowledge.

In a Nutshell

The paper shows that Large Language Models already have a rich library of cultural knowledge hidden inside their local-language settings. They just struggle to find it when speaking English. By using a "self-consistency" check—asking the model to agree with itself across different languages—they can unlock that hidden knowledge and teach the model to be more culturally fair and accurate, all without needing human teachers.

Technical Summary: Cross-Lingual Consensus

Problem Statement
Large Language Models (LLMs) exhibit significant performance disparities across languages, often achieving their strongest results when prompted in English. This English-centric routing introduces a Western-centric bias, hindering the models' ability to accurately reflect diverse cultural knowledge. While LLMs may possess rich cultural knowledge embedded within local-language representations, they frequently fail to retrieve or transfer this knowledge when prompted in English. Existing approaches to mitigate this—such as prompt-based injection or training on curated cultural datasets—often suffer from shallow understanding, high costs, or reliance on stronger external models for data synthesis.

Methodology
The authors propose a self-supervised framework that leverages multilingual self-consistency to align cultural knowledge across languages without human annotation or stronger teacher models. The pipeline consists of two primary stages:

Bilingual Question Generation:
- Assertive statements from existing datasets (CANDLE, CultureAtlas) are converted into knowledge paragraphs.
- The model generates culturally grounded questions in local languages.
- These questions are translated into English (and vice versa) to form bilingual pairs, filtering for non-English questions supported by translation tools.
Self-Supervised Ground Truth Generation:
- Response Sampling: For each bilingual question, the model generates $N$ independent responses in both the English and local language.
- Intra-Language Consistency: The model evaluates the internal agreement of responses within each language using pairwise cosine similarity of embeddings (via Qwen3-Embedding-0.6B).
- Language Selection: The language yielding higher consistency is designated the "stronger" language ( $\lambda^+$ ), while the other is the "weaker" language ( $\lambda^-$ ).
- Ground Truth Construction: The most consistent response from the stronger language is selected as the ground truth. This is translated into the weaker language to serve as the target for training.
- Critique-Augmented Training: The framework employs a self-critique mechanism where the model generates a critique of a sampled response in the weaker language against the translated ground truth. The final training data consists of tuples containing the question, a sampled answer, the critique, and the ground truth.
- Fine-Tuning: The base model (Llama 3.1 8B Instruct) is fine-tuned using LoRA on this synthesized dataset, augmented with the Aya Dataset to prevent catastrophic forgetting, maintaining a 3:1 ratio of cultural to general examples.

Key Contributions

Self-Supervised Framework: The authors introduce a method to generate reliable training signals for cultural alignment entirely from the model's own responses, eliminating the need for external human annotations.
Cross-Lingual Knowledge Transfer: A mechanism is proposed to identify and transfer knowledge from "stronger" (more consistent) language representations to "weaker" ones, effectively surfacing latent cultural knowledge.
Integration of Self-Critique: The approach combines self-consistency with self-critique to improve the quality of alignment in open-ended generation without relying on stronger external models.

Results
Evaluation on the BLEnD benchmark (covering 16 regions and 13 languages) demonstrates the following:

Cultural Alignment: The method significantly improves performance on English queries across all 16 regions, achieving an average boost of 5.03%.
Local Language Performance: While English performance improved, the method faced challenges with low-resource languages (e.g., Hausa, Assamese, Sundanese) due to severe data scarcity (<1% of total cultural data), leading to catastrophic forgetting in these specific regions.
General Reasoning Preservation: General capabilities remained largely intact, with only minor decreases observed on Multilingual HellaSwag (-0.85%) and Global MMLU (-1.42%).
Ablation Studies: Removing the self-consistency module (replacing it with random selection) resulted in a substantial performance drop, confirming that the consistency-based selection is critical. However, the critique-augmented format alone still provided gains over the unaligned base model.

Significance and Claims
The paper claims that latent cultural knowledge exists within LLMs but is often inaccessible when prompted in dominant languages. By leveraging multilingual self-consistency, the authors demonstrate that this knowledge can be successfully surfaced and propagated across languages. The work highlights the potential for achieving more culturally equitable and consistent LLMs through self-supervised means. However, the authors modestly acknowledge that while effective, the approach remains sensitive to the availability and balance of multilingual data, particularly for low-resource languages where data scarcity can lead to performance degradation.

The "Group Consensus" Trick

The "Translation and Critique" Lesson

The Results: What Actually Happened

In a Nutshell

Technical Summary: Cross-Lingual Consensus

More like this