💬 NLP

When Does Mixing Help? Analyzing Query Embedding Interpolation in Multilingual Dense Retrieval

This paper demonstrates that embedding-level interpolation of multilingual queries yields predictable retrieval gains, revealing that mixing is universally beneficial for non-English document indices but counterproductive for English-dominant ones, with optimal performance driven by English as a mixing partner and negatively correlated with typological distance.

Original authors: Tongyao Zhu, Chao-Ming Huang, Min-Yen Kan

Published 2026-06-12

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Tongyao Zhu, Chao-Ming Huang, Min-Yen Kan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to find a specific book in a massive library that contains millions of books written in many different languages. Usually, you ask the librarian (the search engine) for the book in one language, say English. But what if you are a bilingual person and you naturally ask for the book using a mix of English and Chinese in the same sentence? Does that help the librarian find the book faster, or does it confuse them?

This paper investigates exactly that question, but instead of a human librarian, it studies computer systems called "dense retrievers" that use math to find information.

Here is the breakdown of their findings using simple analogies:

1. The "Smoothie" Experiment (How they tested it)

Normally, to test mixed-language queries, researchers would have to ask an AI to write thousands of different sentences mixing words from two languages (like "What is the goddess of agriculture in 希腊神话?"). This is hard, slow, and the AI might make mistakes.

Instead, the authors used a clever shortcut. They took the "mathematical fingerprint" (embedding) of an English query and the "fingerprint" of a Chinese query. They then created a smoothie by blending these two fingerprints together in different ratios (e.g., 10% English, 90% Chinese; 50/50; etc.). They tested these "blended fingerprints" to see if they found the right books better than using just English or just Chinese alone.

2. The Main Discovery: Sometimes Mixing is Magic

The researchers found that in 88 out of 105 different scenarios, blending the two languages actually worked better than using the single best language alone.

The Sweet Spot: Just like a recipe, there is an "optimal ratio." Sometimes a 50/50 mix is best; sometimes it's 70/30.
The Result: By finding this perfect blend, the search engine could find more relevant documents than it could with a "pure" single-language query.

3. The "English Dominance" Rule

The most important finding is that English acts like a heavy anchor in the system. The results depend entirely on whether the library (the document index) contains English books.

Scenario A: The Library has NO English books.
If you are searching for books written only in Spanish, French, or Chinese, mixing English into your query is a superpower. It helps the system find the right books much better than using just the local language. English acts like a universal translator that pulls the search in the right direction.
- Analogy: Imagine trying to find a hidden island on a map. If you only have a local compass, you might get lost. But if you mix in a "Global North Star" (English), you find the island faster.
Scenario B: The Library HAS English books.
If the library contains English documents, do not mix. If you add a non-English language to an English query, it actually hurts performance. The system works best if you speak "pure English" when searching an English-heavy library.
- Analogy: If you are already standing in the English section of the library, speaking a little bit of French might just confuse the librarian. Stick to English to get the best results.

4. English is the "Best Friend" for Every Language

The study showed that for any non-English language (like Arabic, Hindi, or Vietnamese), English is the best partner to mix with. No matter which language you are searching in, adding English to the mix helps more than adding any other language.

5. The "Distance" Factor

The researchers also looked at how different the languages are from each other (like how different English is from Chinese compared to English and German).

They found that when English is not involved, mixing languages that are very different (like English and Chinese) tends to help less than mixing languages that are similar (like Spanish and Italian).
However, this effect is small compared to the massive impact of whether English is in the library or not.

Summary

Does mixing help? Yes, often! Blending two languages in a search query can find better results than using just one.
When does it help most? When you are searching a library that does not contain English documents. In this case, mixing in English is a huge boost.
When does it hurt? When you are searching a library that does contain English documents. In this case, keep your query pure English.
The Golden Rule: English is the strongest "mixing partner" for every other language.

The paper concludes that this behavior is predictable. If you are building a search system, you should check if your database has English. If it doesn't, try mixing in English to get better results. If it does, stick to pure English queries.

Technical Summary: When Does Mixing Help? Analyzing Query Embedding Interpolation in Multilingual Dense Retrieval

Problem Statement
While mixed-language querying (code-mixing) is ubiquitous in multilingual communities, the sensitivity of dense retrievers to such queries remains poorly understood. Existing multilingual information retrieval (IR) benchmarks typically assume queries are monolingual or fully translated, leaving the behavior of dense retrievers under mixed-language conditions largely untested. A critical open question is whether mixing languages in a query can improve retrieval effectiveness beyond the performance of the best monolingual endpoint, or if retrieval effectiveness is strictly bounded by the superior single-language representation. Furthermore, generating high-quality, ratio-controlled code-mixed queries via Large Language Models (LLMs) is resource-intensive and prone to generation noise and tokenization inaccuracies.

Methodology
The authors propose a novel, controlled approach to studying language mixing: embedding-level mixing. Instead of generating text-based code-mixed queries, they construct mixed query representations by interpolating the vector embeddings of two semantically equivalent monolingual query translations.

Data and Setup: The study utilizes the mMARCO dataset, covering 14 languages and 35 representative language pairs. For each pair $(L1, L2)$ , they evaluate three document-language settings: $L1$ -only, $L2$ -only, and a bilingual union ( $L1+L2$ ).
Mixing Mechanism: Given monolingual embeddings $e_{L1}$ and $e_{L2}$ , a mixed embedding $\tilde{e}(\lambda)$ is generated via convex combination:
$\tilde{e}(\lambda) = \frac{(1 - \lambda/100)e_{L1} + (\lambda/100)e_{L2}}{\|(1 - \lambda/100)e_{L1} + (\lambda/100)e_{L2}\|}$
where $\lambda \in \{0, 10, 30, 50, 70, 90, 100\}$ represents the mixing ratio.
Validation: To ensure embedding-level mixing is a valid proxy for natural code-mixing, the authors compare it against word-level mixing generated by an LLM (specifically for the English-Chinese pair). They demonstrate that while word-level mixing introduces noise, the performance trends across mixing ratios are consistent with embedding interpolation.
Models: The primary experiments use BGE-M3, with ablation studies conducted across other model families (E5, GTE, Jina) and scales (Qwen3 variants).

Key Contributions and Findings
The study reveals that language-mix sensitivity is structured and predictable, characterized by five main findings:

Optimal Mixing Often Outperforms Monolingual Endpoints: In 88 out of 105 (83.8%) experimental settings, the optimal interior mixing ratio outperformed the best monolingual query. This suggests that combining embeddings captures complementary semantic signals, yielding more robust representations.
English Dominance and Asymmetry: The presence of English in the document index fundamentally alters mixing dynamics.
- Non-English Indices: When English is absent from the document index, mixing is uniformly beneficial.
- English Indices: When English documents are present, mixing provides no benefit; pure English queries are optimal.
- Asymmetry: Adding English to a non-English query improves retrieval on non-English documents, but adding a non-English language to an English query consistently degrades retrieval on English documents.
English as the Strongest Partner: For every non-English document language tested, English acts as the strongest mixing partner, yielding the highest performance gains compared to other language pairs.
Typological Distance Correlation: When controlling for English dominance, mixing gains correlate negatively with typological distance; languages that are more distant typologically show smaller gains from mixing.
Robustness: These patterns hold consistently across different model families and scales, indicating that embedding-mix sensitivity is a predictable property of current dense retrievers.

Significance and Implications
The paper concludes that language mixing is not a binary "good or bad" property but a nuanced behavior dependent on the document language composition. The authors validate that embedding-level interpolation serves as a low-cost, controllable diagnostic for language-mix sensitivity.

Practically, the findings offer actionable guidance for multilingual IR and Retrieval-Augmented Generation (RAG) systems:

For English-inclusive indices: Systems should default to pure English queries to avoid performance degradation.
For non-English indices: Embedding mixing serves as an effective, test-time augmentation strategy to maximize retrieval recall, particularly when the optimal ratio is approximately balanced or slightly skewed toward the document language (often near 70% document language content).

The authors note that while their controlled setup isolates the effects of mixing ratios, future work is needed to extend these findings to naturally occurring, organically mixed queries and to explore multi-language interpolation simultaneously.