Technical Summary: Why Advanced Encoders Lag on Sparse Retrieval? The Answer and an Approach to Bridging Vocabulary Gaps

1. Problem Statement

The paper addresses a critical anomaly in neural information retrieval: while advanced foundation models (e.g., ModernBERT, RoBERTa-large) significantly outperform older architectures like BERT-base in dense retrieval, they surprisingly underperform in Learned Sparse Retrieval (LSR) tasks. In standard sparse retrieval settings (e.g., SPLADE), these modern encoders often lag behind the aging BERT-base-uncased baseline, sometimes failing catastrophically (e.g., RoBERTa-large dropping to near-zero performance).

The authors identify the root cause not as an architectural deficiency, but as a Vocabulary Gap. Modern tokenizers (e.g., BPE) utilize raw, case-sensitive vocabularies designed for lossless reconstruction. These vocabularies map single semantic units to redundant surface forms (e.g., "Token" vs. "token"), forcing the model to waste capacity bridging orthogonal dimensions. This "morphological noise" hinders lexical matching, which is the core objective of sparse retrieval. Simple preprocessing like lowercasing is insufficient, as it can fragment tokens on case-sensitive tokenizers, destroying semantic integrity.

Retraining models from scratch with a sparse-friendly vocabulary is computationally prohibitive given the scale of modern pre-training corpora (e.g., 2 trillion tokens for ModernBERT). Thus, the field faces a dilemma: the reasoning power of modern backbones is inaccessible to sparse retrieval due to vocabulary incompatibility.

2. Methodology: Vocabulary Transfer (VT)

To resolve this, the authors propose Vocabulary Transfer (VT), a model-agnostic framework that migrates advanced encoders to sparse-friendly, normalized vocabularies (e.g., bert-base-uncased) with minimal computational cost (< 0.2% of original training tokens). The method consists of three key steps:

2.1 Theoretical Foundation

The authors formalize the intuition that Representation-Compatible (RC) coarse-graining improves generalization. They derive a generalization bound showing that appropriate vocabulary normalization (merging surface variants while preserving semantic distinctions) reduces the complexity of the hypothesis class (Rademacher complexity) without sacrificing approximation power, provided the approximation error ($\varepsilon_{RC}$) is minimal.

2.2 The VT Recipe

The migration process involves:

1. Target Vocabulary Alignment: Aligning the model with a well-normalized, lowercased target vocabulary.
2. Semantic Initialization via Spatial Topology:
   • Overlapping Tokens: Parameters are preserved unchanged.
   • New Tokens: Instead of random initialization, new tokens are initialized by transferring neighborhoods from the target embedding space into the source manifold. This is achieved by computing cosine affinities between new tokens and overlapping anchors in the target space, applying sparsemax to generate sparse convex weights, and performing barycentric interpolation over the source embeddings. This ensures new units land in semantically plausible regions, minimizing representation discrepancy.
   • Sub-token Initialization: For custom vocabularies without a pre-trained target model, new tokens are initialized as the mean of their constituent sub-tokens from the source model.
3. Discrepancy-Aware Adaptation:
   • Prior-Aware Distribution Alignment (PDA): A Z-score transfer aligns the output bias (unigram priors) of the target vocabulary with the source model's dynamic range.
   • Overlap-Aware Masking Curriculum: During a short Masked Language Modeling (MLM) phase (approx. 1 epoch), masking probabilities are biased toward new tokens ($\lambda > 1$) to accelerate convergence on unaligned regions while preventing catastrophic forgetting of overlapping tokens.
   • Activation Potential Calibration (APC): A scalar shift is applied to the output bias to calibrate the activation rate. This prevents "dead neurons" (low activation) and "dense collapse" (excessive activation), ensuring the model operates in a regime suitable for sparsity regularization (e.g., ReLU in SPLADE).

3. Key Contributions

• Theoretical Analysis: The paper derives a generalization bound for sparse retrieval under vocabulary coarse-graining, introducing the concept of Representation Compatibility (RC) to explain why normalization improves learnability.
• Methodology (VT): A novel, efficient procedure to transplant regularized vocabularies onto advanced encoders using geometric initialization and discrepancy-aware adaptation, requiring only ~500 MLM steps for near-optimal performance.
• Empirical Validation: Demonstration that VT universally resolves the performance lag across diverse backbones (ModernBERT, RoBERTa-large, BERT-cased) and architectures (standard and inference-free sparse retrievers).

4. Experimental Results

The authors evaluate VT on the BEIR benchmark (zero-shot generalization) and MS MARCO (in-domain).

• State-of-the-Art Performance: The ModernBERT-VT model achieves 52.4 nDCG@10 on BEIR, surpassing the previous best sparse retriever (SPLADE-v3 at 52.0) and the standard Co-EnsembleDistil baseline (50.9). This represents a +4.7 improvement over the naive ModernBERT baseline.
• Resuscitating Failing Models: VT successfully "resuscitates" models that fail in sparse settings. For instance, RoBERTa-large improves from a near-zero score (1.4 nDCG) to 51.3 nDCG, outperforming the BERT-base baseline.
• Efficiency: The method maintains efficiency comparable to SPLADE baselines, with ModernBERT-VT showing competitive FLOPs and document lengths.
• Ablation Studies:
  • Initialization: Semantic Initialization via spatial topology significantly outperforms random or mean initialization, providing a high-quality "warm start."
  • Components: Removing Activation Potential Calibration (APC) causes a severe drop in performance (BEIR drops from 52.4 to 48.8), highlighting its critical role in preventing dense collapse.
  • Full vs. Head-Only: Full vocabulary transfer (VT) significantly outperforms head-only adaptation (ESPLADE-style), proving that aligning the input embedding space is crucial.
• Generalization: VT generalizes to inference-free architectures and specialized domains (e.g., Chemistry), where it improves performance despite limited domain-specific pre-training data.

5. Significance and Claims

The paper claims that the performance lag of advanced encoders in sparse retrieval is a solvable vocabulary mismatch, not an architectural limitation. By shifting from lossless-reconstruction-oriented tokenization to normalized, representation-compatible vocabularies, the reasoning power of next-generation backbones can be unlocked for lexical matching.

The authors position VT as a universal recipe that bridges the divide between modern foundation models and sparse retrieval objectives. It allows the community to leverage the inference efficiency and context windows of models like ModernBERT without the prohibitive cost of retraining from scratch. The work suggests that vocabulary design is a fundamental bottleneck in neural sparse retrieval and that appropriate coarse-graining is essential for achieving state-of-the-art results.