Technical Summary: Grounding Lexical Retrieval in Semantic Evidence (CE-QE)

1. Problem Statement

The paper identifies a structural limitation in hybrid retrieval systems that fuse lexical (BM25) and dense semantic search. While dense retrieval captures semantic relationships and paraphrases, and lexical retrieval captures exact term matches and corpus-wide significance, neither alone solves the semantic vocabulary gap.

The core issue is that BM25 retrieves only documents sharing surface terms with the query. If a relevant document phrases an answer differently (e.g., "adverse reactions" vs. "side effects"), BM25 fails to include it in the candidate set. Consequently, downstream processes like reranking or score fusion cannot recover these documents because they operate only on the initial candidate set. The "recall ceiling" is fixed at the first stage; if the document is not retrieved, it cannot be ranked.

2. Methodology: Cross-Encoder Query Expansion (CE-QE)

The authors propose Cross-Encoder Query Expansion (CE-QE), a method that injects semantic signals directly into the lexical query before retrieval, rather than attempting to fix the ranking afterward.

Core Mechanism

1. Semantic Seeding: The system first runs a semantic retriever (dense search) on the original query to retrieve top passages. Unlike classical pseudo-relevance feedback (e.g., RM3), which seeds expansion from the lexical retriever's own (potentially flawed) top results, CE-QE uses the semantic retriever's results. This ensures the seed set is topically relevant even when surface terms diverge.
2. Attribution Extraction: A cross-encoder (reranker) is applied to these top semantic passages. Instead of using raw term frequencies or TF-IDF, CE-QE extracts per-token relevance attributions. The model identifies which specific tokens in the passage were decisive in determining relevance to the query.
3. Query Expansion: The highest-attribution content tokens (e.g., specific symptoms, entities, or technical terms absent from the original query) are extracted and appended to the original query.
4. Lexical Retrieval: The expanded query is run against the standard, unmodified BM25 index.

Design Rationale

• Why Query, Not Ranking? Reranking cannot retrieve documents that were never selected. CE-QE intervenes at the query stage to ensure the missing document enters the candidate set.
• Why Semantic Seeding? If BM25's top results are wrong due to a vocabulary gap, feeding terms back from them reinforces the error (query drift). Semantic retrievers, being independent of surface overlap, provide a trustworthy seed set in these failure cases.
• Why Attribution? Frequency-based selection often picks generic scaffolding words. Cross-encoder attributions act as a supervised selector for discriminative terms, ensuring only relevant content terms are added.
• Why Grounding? Unlike generative query expansion (e.g., HyDE, Query2doc) which hallucinates text from an LLM's parametric knowledge, CE-QE copies terms verbatim from retrieved passages. This prevents the introduction of vocabulary not present in the corpus.

3. Semantically Enriched Score Fusion (SESF)

The paper also introduces SESF, a hybrid pipeline that combines the CE-QE enriched lexical results with semantic scores.

• Mechanism: The enriched BM25 list is fused with the semantic list, and the merged set is reranked by a cross-encoder.
• Advantage: Because the lexical list is now semantically aware, the overlap between the two lists on genuinely relevant documents increases. This provides a stronger, more concentrated candidate pool for the reranker, improving the overall ceiling of what the system can achieve.

4. Key Contributions

1. CE-QE Algorithm: A query-expansion method that uses cross-encoder token attributions from semantic search results to expand BM25 queries, directly addressing the vocabulary gap without modifying the index.
2. Grounded Expansion: A contrast to generative expansion methods, ensuring all expansion terms are grounded in the actual corpus content, avoiding hallucination.
3. SESF Pipeline: A fusion strategy that leverages the enriched lexical list to outperform standard cross-encoder score fusion and state-of-the-art learned sparse/late-interaction models.
4. Architectural Separation: The method leaves the underlying BM25 index completely unmodified, making it compatible with any standard or scaled lexical index (e.g., hierarchical or sharded) without architectural changes.

5. Experimental Results

Evaluated on seven BEIR datasets using the RUMIR pipeline:

• Recall Improvements: CE-QE significantly improves Recall@100 where query and answer vocabularies diverge.
  • Natural Questions (NQ): Recall@100 increased from 0.32 to 0.47.
  • TREC-COVID: Increased from 0.56 to 0.67.
  • Climate-FEVER: Increased from 0.19 to 0.26.
  • Note: On datasets with high vocabulary overlap (e.g., FEVER), gains are negligible or slightly negative, confirming the method targets specific failure modes.
• SESF Performance:
  • Beats standard cross-encoder score fusion by 2.5% on Recall@100.
  • Outperforms SPLADEv2 by 5.3% and ColBERTv2 by 4.6% on nDCG@10, despite using a standard flat BM25 index rather than specialized sparse or late-interaction structures.
• Latency:
  • The full SESF pipeline adds approximately 400ms of latency over a baseline RRF hybrid (total ~557ms).
  • This cost is attributed to semantic retrieval and cross-encoder attribution extraction, which are bounded operations (top 10 passages, top 10 tokens), avoiding the massive slowdown (485–2051x) associated with unbounded reranking.

6. Significance and Claims

The paper claims that CE-QE provides a query-side fix for the structural weakness of lexical retrieval. Its significance lies in:

• Grounding: It avoids the "hallucination" risk of generative expansion by sourcing terms strictly from retrieved corpus passages.
• Cost Efficiency: It utilizes components (cross-encoders) already present in hybrid pipelines for reranking, adding only attribution extraction rather than a separate generation call.
• Index Agnosticism: It improves recall without requiring index retraining or modification, making it a drop-in improvement for existing BM25-based systems.
• Targeted Efficacy: The method is specifically effective for the "vocabulary gap" failure mode, as evidenced by performance gains on datasets with divergent registers (NQ, TREC-COVID) and lack of gains on datasets with high overlap (FEVER).

The authors note that while they have not tested CE-QE on billion-scale indices end-to-end, the architectural separation of concerns (query expansion vs. index structure) suggests it should compose naturally with large-scale lexical indexes.