Technical Summary: SHIFT - Semantic Harmonization via Index-side Feature Transformation for Multilingual Information Retrieval

1. Problem Statement

Multilingual Information Retrieval (MLIR) aims to enable users to retrieve semantically relevant documents from multilingual text collections using a single-language query. However, state-of-the-art multilingual dense retrieval models suffer from severe language bias. Instead of prioritizing semantic relevance, these models exhibit a disproportionate preference for documents written in the same language as the query.

As illustrated in the paper's analysis of the Belebele dataset, an English query often results in top-ranked documents being dominated by English content, even when semantically equivalent ground-truth documents exist in other languages. This phenomenon distorts the embedding space, where "language identity" overrides "semantic signals," leading to unfair information access and suboptimal retrieval performance for non-query languages. Conventional metrics like nDCG and Recall often mask this bias by rewarding the retrieval of source-language documents, failing to capture the lack of diversity in the results.

2. Methodology: SHIFT

The authors propose SHIFT (Semantic Harmonization via Index-side Feature Transformation), a training-free method designed to mitigate language bias during the indexing stage. Unlike approaches that require computationally expensive retraining or introduce inference-time latency, SHIFT employs a linear displacement strategy to correct spatial misalignment in the embedding space.

Core Mechanism

The method operates in two primary phases:

1. Estimating Relative Language Vectors:

   • SHIFT utilizes parallel translation pairs from the mMARCO dataset (machine-translated versions of English content in 14 languages).
   • For a target language $\ell_{tgt}$ and a source language $\ell_{src}$, the model encodes parallel document pairs $(D^{\ell_{src}}_i, D^{\ell_{tgt}}_i)$ into embeddings $z^{\ell_{src}}_i$ and $z^{\ell_{tgt}}_i$.
   • The difference $\Delta^{\ell_{tgt}}_i = z^{\ell_{tgt}}_i - z^{\ell_{src}}_i$ approximates the representational offset caused solely by the language shift.
   • A relative language vector $V_{\ell_{tgt}}$ is computed by averaging these offsets across all $N$ translation pairs:
     $$V_{\ell_{tgt}} = \frac{1}{N} \sum_{i=1}^{N} (z^{\ell_{tgt}}_i - z^{\ell_{src}}_i)$$

2. Index-side Language Shift:

   • During the indexing of a document collection, each document $d$ with language label $\ell$ is processed.
   • If $\ell \neq \ell_{src}$, the corresponding relative language vector is subtracted from the document embedding:
     $$\tilde{z} = z - \alpha V_{\ell}$$
   • Here, $\alpha$ is a global scale factor ($\alpha \in [0, 1]$) controlling the magnitude of the shift.
   • Documents in the source language remain unmodified.
   • Retrieval: The query is encoded using the same model without modification. The retrieval score is calculated as the similarity (e.g., cosine similarity) between the query embedding and the shifted document embeddings $\tilde{z}$. This process introduces zero additional latency at query time.

3. Key Contributions

The paper outlines three primary contributions:

1. SHIFT Method: A universally effective, training-free solution that consistently improves MLIR performance and mitigates language bias across diverse dense retrieval models (both encoder-based and decoder-based architectures).
2. Target-Languages Recall@k (TLR@k): A novel evaluation metric introduced to explicitly quantify language bias. Unlike standard metrics, TLR@k measures the recall of relevant documents written in languages other than the query language, providing a clearer picture of cross-lingual retrieval effectiveness.
3. Empirical Evidence: Comprehensive experiments demonstrating that SHIFT successfully realigns multilingual documents based on semantics, correcting the skewed geometry of the embedding space without requiring model retraining.

4. Experimental Results

The authors evaluated SHIFT across four MLIR benchmarks (Belebele, XQuAD, MLQA, MultiEuP-v2) and six diverse dense retrieval models (including multilingual-e5-large, bge-m3, Qwen3-Embedding, and llama-nemotron-embed).

• Performance Gains: SHIFT consistently improved retrieval performance. For instance, using multilingual-e5-large, the average nDCG@20 increased from 0.633 to 0.737.
• Bias Mitigation: The most significant improvements were observed in TLR@20. For multilingual-e5-large, TLR@20 on the MultiEuP-v2 dataset jumped from 0.095 to 0.268, indicating a substantial increase in the exposure of relevant non-English documents.
• Universality: The method proved effective regardless of the source language. Experiments using Chinese, Vietnamese, and Hindi as source queries showed consistent improvements, confirming SHIFT is not limited to English-centric alignment.
• Robustness: Sensitivity analysis revealed that while optimal $\alpha$ values vary by model (e.g., $\alpha=0.6$ for multilingual-e5-large vs. $\alpha=1.0$ for others), SHIFT outperforms baselines across the entire range of $\alpha$.
• Comparison to Baselines: SHIFT outperformed other training-free debiasing methods, such as language-wise centering (Libovický et al., 2020) and Language Information Removal (Yang et al., 2021), particularly in terms of TLR@20, while maintaining or improving overall nDCG.

5. Significance and Claims

The paper claims that SHIFT offers a practical, low-cost solution to a critical flaw in modern MLIR systems. By shifting document embeddings at the index side, it forces models to rely on underlying semantic signals rather than linguistic surface forms.

• Efficiency: The method requires no additional training and imposes no inference-time overhead, making it highly deployable in latency-sensitive systems.
• Fairness: It directly addresses the "language bias" that leads to the under-exposure of relevant documents in non-query languages, thereby promoting more equitable information access.
• Scalability: The approach is model-agnostic and works across different architectures and scales.

The authors acknowledge limitations, noting that the quality of the shift vectors depends on the translation quality of the parallel corpus (mMARCO) and that the method requires pre-labeled document languages. However, they argue that the one-time cost of vector estimation and the offline nature of the transformation make it a robust tool for managing the trade-off between semantic alignment and language preference in real-world retrieval systems.