💬 NLP

Cross-Lingual Exploration for Parametric Knowledge

This paper proposes cross-lingual exploration as an efficient inference-time strategy that significantly enhances parametric knowledge retrieval, factual recall, and cross-lingual consistency in Large Language Models across 17 diverse languages, outperforming native-language scaling.

Original authors: Elisha Diskind, Itamar Trainin, Uri Shaham, Leshem Choshen, Idan Szpektor, Omri Abend

Published 2026-06-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Elisha Diskind, Itamar Trainin, Uri Shaham, Leshem Choshen, Idan Szpektor, Omri Abend

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Core Problem: The "Locked Room" of Knowledge

Imagine a Large Language Model (LLM) is like a massive, multi-lingual library. Inside this library, all the facts about the world are stored on shelves (the model's "parameters"). However, there's a catch: some facts are only written on specific shelves in specific languages.

If you ask the librarian (the AI) a question in English about a local Japanese radio show, they might look in the "English" section, find nothing, and guess a wrong answer or say, "I don't know." They are stuck in a "locked room" because the correct information is actually sitting on a shelf labeled "Japanese," but the librarian never thought to walk over there.

This paper calls this problem Parametric Knowledge Inaccessibility. The knowledge exists inside the model, but standard methods can't reach it because they stay in one language.

The Solution: Cross-Lingual Exploration

The authors propose a strategy called Cross-Lingual Exploration. Instead of just asking the question in the language you started with, you tell the AI: "Hey, maybe the answer isn't in English. Go find the language where this fact is most likely to live, think about it there, and then bring the answer back to me."

Think of it like a detective solving a case:

Standard Method: The detective only speaks English. They ask an English-speaking witness who doesn't know the answer. Case closed (incorrectly).
Cross-Lingual Exploration: The detective realizes the clue is in a foreign country. They hire a translator, go to that country, ask the local witness in their native tongue, get the correct answer, and translate it back.

The Four Dimensions of the Strategy

The paper breaks this "detective work" down into four main tools (dimensions) they tested:

Language Selection (Choosing the Right Detective):
- The Analogy: Do you send the detective to a generic hub (like English, which is a "pivot"), or do you let them figure out the specific country the clue belongs to?
- The Finding: Letting the AI autonomously choose the best language (e.g., realizing a fact about an Argentine lawyer is best found in Spanish) worked better than forcing everything through English.
Routing (The Path Taken):
- The Analogy: Does the detective take one direct path to the right country, or do they send out a team of 13 detectives to 13 different countries at the same time?
- The Finding: Sending out a multi-path team (parallel exploration) was the most effective. It's like casting a wider net; even if 12 detectives get it wrong, the one who went to the right country saves the day.
Aggregation (Deciding the Winner):
- The Analogy: Once the team returns with 13 different answers, how do you decide which one is true? Do you just pick the most popular answer (Majority Vote), or do you look for the "expert" answer that might be the minority opinion?
- The Finding: For very specific, local facts, the "Minority Aware" strategy (looking for the unique, correct answer even if it's the only one) was surprisingly effective.
Budget (The Cost):
- The Analogy: How much money (computing power) are you willing to spend?
- The Finding: Even though asking the AI to think in multiple languages costs more "tokens" (money), it is actually more efficient than just asking the same question 13 times in English. You get better results for the money spent.

Key Results: What They Discovered

Unlocking Hidden Facts: By switching languages, the AI could retrieve facts that were previously impossible to find. In some cases, this improved accuracy by up to 44% for specific localized knowledge.
Better Consistency: When the AI is asked the same question in different languages (e.g., "Who is the DJ?" in English, Japanese, and Korean), it usually gives different, conflicting answers. Cross-lingual exploration made the AI give the same correct answer regardless of the language used. This makes the AI more reliable and less "confused."
It's Not Just "Thinking Harder": The authors proved that the improvement didn't just come from the AI "thinking longer" (reasoning). The improvement came specifically from switching languages. The act of translating and reasoning in a different tongue unlocked the memory.

The Bottom Line

This paper shows that Large Language Models have a lot of knowledge hidden in their "parameters," but they often forget to look in the right language to find it. By treating language not just as a way to talk, but as a key to unlock different parts of the model's memory, we can make AI much smarter, more accurate, and more consistent across the world.

It's not about teaching the AI new facts; it's about teaching it how to look for the facts it already knows.

Technical Summary: Cross-Lingual Exploration for Parametric Knowledge

Problem Statement

Large Language Models (LLMs) encode vast amounts of real-world, multi-lingual knowledge within their parameters. However, this parametric knowledge is not uniformly accessible across all languages. Standard inference techniques often fail to retrieve localized facts when queried in a specific language, leading to hallucinations or incorrect answers, even when the knowledge exists within the model. This phenomenon creates a gap in cross-lingual knowledge transfer and consistency, where the same underlying fact may be easily accessible in one language (e.g., Japanese) but remain latent or inaccessible in another (e.g., English). The authors posit that the mechanisms for storing and accessing this knowledge are intricately encoded in parametric superposition, and standard single-language inference paths are insufficient to unlock all latent facts.

Methodology

The paper proposes Cross-Lingual Exploration, an inference-time strategy that leverages the model's internal cross-lingual capabilities to surface hidden factual knowledge. Rather than treating language selection as a fixed task-specific prompt, the authors frame cross-lingual exploration as a structured search process defined by four inherent dimensions:

Language Selection: Determining which linguistic contexts ( $L_e$ ) to probe. Strategies range from fixed pivots (e.g., translating everything to English) to autonomous model-driven selection, where the model reasons about the cultural origin of a fact to select the most appropriate reasoning language.
Routing: Defining the search trajectory. This includes Single-path strategies (committing to one language or sequentially switching languages) and Multi-path strategies (generating independent reasoning trajectories in multiple languages simultaneously).
Aggregation: The mechanism for distilling the final answer from multiple candidate trajectories. The authors evaluate Majority Vote (selecting the most frequent answer) and Minority-Aware selection (reasoning that the correct answer for localized facts might be a minority among hallucinated responses from other paths).
Inference Budget: Tracking the computational cost (tokens and model calls) to ensure fair comparison between exploration strategies and native-language scaling.

The authors instantiate a suite of strategies based on these dimensions (summarized in Table 1 of the paper) and evaluate them on two benchmarks:

ECLeKTic: Focuses on "language-specific" facts where information exists in a source language but lacks equivalents in the target query language.
CLIKE: Evaluates general Wikidata facts across 13 typologically diverse languages to measure overall factual recall and cross-lingual consistency.

Experiments were conducted across seven state-of-the-art models (including Gemini 2.5 Flash, GPT-4o, Qwen 3, and Grok 4.1) covering 17 unique languages.

Key Results

The study demonstrates that cross-lingual exploration significantly improves performance across all evaluated models and languages:

Knowledge Transfer: Cross-lingual exploration consistently improves the ability to answer queries in a target language when the fact is localized to a different source language. Compared to native monolingual baselines, exploration strategies achieved knowledge transfer gains of up to 21% on the ECLeKTic dataset.
Factual Recall: The benefits extend beyond localized facts. Exploration also boosts general factual recall on the CLIKE benchmark, indicating that traversing alternative linguistic paths helps access parametric knowledge that is nominally present in the query language but inaccessible via standard monolingual reasoning.
Mechanism of Gain: The authors isolate the source of these gains by comparing cross-lingual exploration against monolingual reasoning (using intermediate "thinking" steps in the same language). They find that the language shift itself, rather than the reasoning process alone, is the primary driver for unlocking inaccessible knowledge. Autonomous selection strategies (where the model reasons about the fact's origin) outperformed even fixed English pivots and oracle-based source-language routing.
Efficiency (Pareto Frontier): When plotting accuracy against token cost, cross-lingual exploration establishes a more efficient compute Pareto frontier than native-language scaling. It achieves higher accuracy for the same computational budget compared to simply increasing the number of reasoning paths in the native language.
Cross-Lingual Consistency: The paper introduces a hypothesis testing framework to measure intrinsic consistency. Results show that cross-lingual exploration improves consistency between languages significantly beyond what can be explained by accuracy gains alone, providing more robust and predictable behavior for downstream applications.

Significance and Claims

The authors claim that their work establishes multilingual prompt exploration as a highly effective inference-time strategy for unlocking latent parametric knowledge. The significance of this work lies in:

Challenging the "English Pivot" Assumption: The results suggest that routing all queries through English is suboptimal. Autonomous selection of reasoning languages based on the cultural context of the fact yields superior results.
Redefining Inference-Time Scaling: The paper argues that performance gains in LLMs should not be viewed solely through the lens of compute scaling (more tokens in one language) but through the lens of search efficiency (exploring diverse linguistic paths). Cross-lingual exploration offers a more efficient path to accessing hidden knowledge.
Intrinsic Consistency: The work demonstrates that consistency is an independent desideratum from accuracy. Cross-lingual exploration not only makes models more accurate but also more consistent across languages, a critical property for reliable multilingual systems.

The authors conclude that while answer aggregation remains a bottleneck (as the potential upper bound of knowledge retrieval is often higher than the realized gain), explicitly training models to strategically perform cross-lingual exploration represents a promising direction for building more robust and factually accessible multilingual LLMs. The paper does not propose specific architectural changes but rather a structured design space for inference-time strategies.