Technical Summary: Mitigating Cross-Lingual Cultural Inconsistencies in LLMs via Consensus-Driven Preference Optimisation

1. Problem Definition: Cross-Lingual Cultural Inconsistency (CCI)

Multilingual Large Language Models (MLLMs) frequently exhibit inconsistent behavior when the language of a prompt changes, even when the user's identity is explicitly defined. While MLLMs are generally designed to culturally adapt to the input language, this becomes a critical failure in constrained contexts where a specific user persona (e.g., "User is British") is provided.

In such scenarios, the model often ignores the explicit persona instruction and defaults to the cultural stereotypes associated with the prompt's language. For example, given a fixed British persona and an ambiguous query about literature, an English prompt may elicit "Shakespeare," while a Spanish prompt elicits "Cervantes." This phenomenon, termed Cross-lingual Cultural Inconsistency (CCI), represents a failure of instruction following where implicit language-driven personalisation overrides explicit user-driven constraints. The paper posits that this inconsistency is not merely a performance gap but a semantic divergence driven by the tight entanglement of language and culture within the model's representations.

2. Methodology

2.1 Dataset Construction

To systematically evaluate CCI, the authors constructed a parallel multilingual dataset derived from the BLEnD benchmark (Myung et al., 2024).

• Source: The Multiple-Choice Question (MCQ) subset of BLEnD was selected for its discrete answer space, facilitating agreement metrics.
• Languages: Eight languages were selected to cover diverse resource levels and linguistic families: English, Spanish, Chinese, Arabic, Indonesian, Korean, Greek, and Persian.
• Neutralisation: To isolate implicit language-driven bias, explicit country references in the original prompts were removed using GPT-5.2 (e.g., changing "Popular sport in Mexico?" to "Popular sport?").
• Translation: Answer options were translated into all eight target languages, ensuring the dataset contained valid mappings to eight specific countries (US, Mexico, China, Algeria, Indonesia, South Korea, Greece, Iran).
• Split: The data was partitioned into 70% training, 10% validation, and 20% testing, with strict measures to prevent data leakage across splits.

2.2 Measurement: Singleton Fleiss's $\kappa_S$

Standard consistency metrics often fail when models hallucinate or generate invalid formats, as discarding these errors introduces survivorship bias. The authors introduce Singleton Fleiss's $\kappa_S$, a novel metric mathematically proven to be resilient to hallucinations.

• Mechanism: Unlike standard Fleiss's $\kappa$, which assumes all responses map to a predefined valid set, $\kappa_S$ extends the valid answer set $V$ with a set of dynamically generated singleton (invalid) answers $U$.
• Penalty: Every invalid response maps to a unique singleton. This ensures that hallucinations are treated as distinct categories, mathematically penalising inconsistency without requiring the ad-hoc exclusion of samples.
• Convergence: The authors prove that as dataset size increases, $\kappa_S$ asymptotically converges to the standard valid-category $\kappa$.

2.3 Mitigation: Cross-lingual Cultural Consistent Preference Optimisation (C-3PO)

The paper proposes C-3PO, a self-supervised, consensus-driven alignment framework designed to mitigate CCI without relying on costly human annotations or subjective ground truths.

• Consensus Extraction: The base model generates responses for a culturally neutral query across all $N$ languages. A cross-lingual consensus is defined as the answer selected by a strict majority of languages.
• Preference Pair Construction:
  • Chosen ($y_w$): The consensus answer is universally assigned as the preferred response.
  • Rejected ($y_l$):
    • For languages where the model initially disagreed with the consensus, the actual divergent output is used as the rejected response.
    • For languages that already agreed, a random non-consensus option is sampled as the rejected response.
• Balancing: An undersampling strategy is applied to ensure that languages dictating the consensus do not disproportionately bias the training data, balancing the representation of consensual and divergent languages.
• Optimisation: The model is fine-tuned using Direct Preference Optimisation (DPO) with Low-Rank Adaptation (LoRA). Data is structured into parallel batches (the same query translated across all languages) to ensure gradient updates simultaneously pull disparate language representations toward a unified semantic anchor.

2.4 Baselines

C-3PO is evaluated against three established behavioural steering baselines:

1. Persona Prompting: Explicitly conditioning the model on a specific nationality via system prompts.
2. Few-shot Prompting: Providing in-context examples of consistent cross-lingual answers.
3. Persona Vector Steering: Steering the model's latent space using intervention vectors derived from the difference in residual stream activations between prompts with and without persona instructions.

3. Key Results

3.1 Consistency Improvements

Empirical evaluations on three open-weight models (Gemma-2-27b-it, Llama-3.1-8B-Instruct, Qwen2.5-3B-Instruct) demonstrate that C-3PO consistently outperforms all baseline mitigation strategies across virtually all models and language groups.

• C-3PO achieves up to a 0.10-point absolute increase in Singleton Fleiss's $\kappa_S$ over unaligned models.
• While baselines like Persona Vector Steering occasionally achieve high scores in specific configurations (e.g., specific persona-country pairs), these results are described as "cherry-picked oracle scenarios." C-3PO provides robust alignment without requiring manual selection of optimal configurations.
• Few-shot prompting was found to artificially inflate Soft and Hard Consistency scores by collapsing to a single mode (indiscriminately repeating an answer), a degenerate behaviour that $\kappa_S$ robustly penalises.

3.2 Impact of Language Resources

The study reveals that CCI disproportionately affects lower-resource languages (Indonesian, Persian, Korean, Arabic, Greek).

• Lower-resource groups exhibit significantly lower $\kappa_S$ scores across all models and mitigation strategies compared to higher-resource groups (English, Chinese, Spanish).
• Incremental addition experiments show that consistency degrades strictly as lower-resource languages are added to the evaluation pool, identifying resource scarcity as a primary driver of inconsistency.

3.3 Interpretability Analysis

A layer-wise interpretability analysis on Llama-3.1-8B reveals the underlying mechanism of CCI:

• Emergence Point: Cross-lingual consistency does not emerge gradually; it manifests abruptly in the later layers (around layer 22) as forward-pass representations stabilise.
• Implicit Personalisation: As representations stabilise, the model implicitly anchors outputs to the stereotypical culture associated with the prompt language. For instance, Spanish prompts increasingly trigger predictions for Mexico, and Arabic prompts for Algeria or the US.
• Slope Analysis: Linear regression of prediction frequencies across layers confirms that the surge in predictions for a specific country is significantly steeper when the query is in that country's stereotypical language, supporting the hypothesis that the model implicitly personalises answers based on the prompt language.

4. Significance and Claims

The paper claims to make four primary contributions:

1. Formalisation of CCI: It isolates and formalises Cross-lingual Cultural Inconsistency as a distinct failure mode where models fail to leverage explicit user identities in favour of implicit language cues.
2. Robust Metric: It introduces Singleton Fleiss's $\kappa_S$, a metric mathematically resilient to hallucinations, enabling the quantification of consistency without discarding invalid data.
3. Effective Mitigation: It proposes C-3PO, a self-supervised framework that significantly improves cultural consistency across diverse baselines and model architectures without requiring human-annotated ground truths.
4. Mechanistic Insight: It provides early-decoding evidence that MLLMs implicitly personalise outputs towards the prompt language's stereotypical culture as internal representations stabilise, and that this effect is exacerbated in lower-resource languages.

The authors maintain a modest stance, acknowledging that their framework equates countries with cultures (a reductive proxy) and that the "consensus" generated may introduce a new form of bias. They argue that while their work addresses instruction following in constrained settings, the broader debate on when cross-lingual consistency is desirable versus when cultural adaptability is preferred remains open for future investigation.