💬 NLP

RARe: Retrieval Augmented Retrieval with In-Context Examples

RARe is a method that enhances the performance and out-of-domain generalization of encoder-only text retrievers by fine-tuning them to utilize semantically similar query-document pairs as in-context examples.

Original authors: Atula Tejaswi, Yoonsang Lee, Sujay Sanghavi, Eunsol Choi

Published 2026-02-10

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Atula Tejaswi, Yoonsang Lee, Sujay Sanghavi, Eunsol Choi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a librarian in a massive, infinite library. A person walks up to you and asks a very vague question: "Tell me about that thing with the economic dispatch."

If you are a Standard Retriever, you look at that one sentence, search your brain, and try to find a book. You might find something, but because the question was a bit fuzzy, you might grab a book on "General Economics" instead of the specific "Direct Search Methods" the person actually wanted.

The researchers who wrote this paper have come up with a way to make the librarian much smarter. They call it RARe.

The Analogy: The "Show, Don't Just Tell" Librarian

Think of RARe as a librarian who doesn't just listen to your question, but also looks at a "cheat sheet" of similar questions asked by other people earlier that day.

1. The Old Way (Standard Retrieval):
The user asks: "How do I solve economic dispatch?"
The librarian thinks: "Okay, 'economic dispatch'... let me look..." (Result: Often misses the mark).

2. The "In-Context" Way (The RARe Method):
Before the librarian answers you, they quickly glance at a small notebook of recent successful interactions:

Example 1: Someone asked about "Direct Search Methods," and the answer was "Method X."
Example 2: Someone asked about "Dynamic Dispatch," and the answer was "Method Y."
Now, the librarian looks at YOUR question: "How do I solve economic dispatch?"

Because the librarian saw those similar examples right before your question, their brain "clicks" into the right mode. They realize, "Ah! We are talking about specific mathematical optimization methods!" They are now much more likely to hand you the exact book you need.

What did the researchers actually do?

In technical terms, they are applying In-Context Learning (ICL)—a trick that makes giant AI models (like ChatGPT) smarter by giving them examples in the prompt—to Retrieval Models (the specialized AI that finds specific documents).

Here is the breakdown of their "recipe":

The Problem: Usually, if you just throw examples at a retrieval model without training it specifically to handle them, the model gets confused and performs worse. It’s like giving a chef a recipe but also throwing random ingredients on the counter; they might get distracted.
The Solution (RARe): They fine-tuned the model. They trained the AI specifically to look at those "cheat sheet" examples (query-document pairs) and use them to understand the target question better.
The "Secret Sauce": They didn't just pick random examples. They used a simple search tool (BM25) to find examples that were semantically similar to the user's question. If you ask about math, the "cheat sheet" shows math examples.

Why does this matter? (The Results)

Better Accuracy: They saw performance boosts of up to 2.72%. In the world of massive AI benchmarks, that is a significant leap.
Better "Out-of-the-Box" Thinking: The model became better at handling topics it hadn't seen much during training. It learned the pattern of how to use examples, which helped it generalize to new, weird subjects.
Reasoning Power: It worked especially well on "reasoning" tasks—questions that require a bit of logic rather than just matching keywords.

Summary in one sentence:

RARe turns a "keyword matcher" into a "context-aware expert" by training it to learn from similar examples provided right alongside your question.

Technical Summary: RARe: Retrieval-Augmented Retrieval With In-Context Examples

Published at: COLM 2025

1. Problem Statement

While In-Context Learning (ICL) has revolutionized decoder-only Large Language Models (LLMs) by allowing them to perform new tasks via few-shot examples without parameter updates, its application to encoder-only embedding models (used for dense retrieval) has been significantly less explored.

Existing attempts to apply ICL to retrievers often involve adding randomly sampled examples to the query. However, the authors observe two major issues:

Performance Degradation: Unlike decoder models, simply prepending in-context examples to a pre-trained encoder model at inference time often leads to a significant drop in retrieval performance.
Lack of Semantic Guidance: Randomly sampled examples do not necessarily provide the task-specific or query-specific semantic context needed to refine the embedding vector effectively.

2. Methodology: RARe

The authors propose RARe (Retrieval-Augmented Retrieval with In-Context examples), a method that fine-tunes retriever models to effectively leverage semantically similar query-document pairs as context.

The RARe Workflow:

In-Context Example Selection: For a given target query $q$ , the system uses a sparse retriever (BM25) to find the $k$ most semantically similar queries from the training set.
Example Construction: For each retrieved query, the corresponding positive document $d^+$ is paired with it to form an exemplar $(q_{ic}, d^+_{ic})$ .
Query Augmentation: The target query is augmented into a structured prompt:
- Instruct: {task_instruction}; Query: {q_ic1}; Document: {d+_ic1} ... Query: {target_q}.
Contrastive Fine-Tuning: The model is trained using a standard contrastive loss, where the embedding of this augmented query is pushed closer to the target positive document and further from hard negatives.

The researchers tested this approach across two training regimes:

From Decoder-only Checkpoints: Initializing with models like Llama-3 and training them specifically for retrieval.
From Retriever Checkpoints: Continued fine-tuning of high-performing existing models (e.g., E5-Mistral-Instruct and LLM2Vec).

3. Key Contributions

New Paradigm for Encoder ICL: Establishes that in-context learning can be successfully adapted for dense retrievers if the model is specifically trained to process the augmented format.
Semantic Selection Strategy: Demonstrates that using BM25 to select semantically similar examples is superior to random sampling.
Comprehensive Design Analysis: Provides a deep dive into how the number of examples ( $k$ ), the format of the prompt (query-only vs. doc-only), and the similarity of the examples affect performance.
Open Source: The authors released their code and model checkpoints to facilitate future research.

4. Key Results

Performance Gains: RARe achieves significant improvements over standard retrieval and random ICL. On the BeIR benchmark, it shows gains of up to +1.41% nDCG@10. On the reasoning-heavy RAR-b benchmark, it achieves much larger gains of up to +2.72% nDCG@10.
Out-of-Domain (OOD) Generalization: A standout finding is that RARe exhibits stronger generalization to datasets not seen during training, mirroring the behavior seen in large decoder-only LLMs.
Sensitivity to Similarity: Performance scales with the similarity between the in-context examples and the target query; when the closest example is highly similar, gains can exceed 10% on certain datasets (e.g., NFCorpus).
Format & Content: The study found that including documents in the prompt is more beneficial than queries alone, and adding negative documents to the in-context prompt actually slightly decreases performance.

5. Significance

The paper is significant because it bridges the gap between the generative power of LLMs and the efficiency of dense retrievers. It proves that the "context" in in-context learning for retrievers acts more as task-relevant information/query expansion rather than just increasing model capacity. By providing a recipe to make embedding models "few-shot learners," RARe lays the foundation for more adaptive, query-aware retrieval systems that can handle complex reasoning and diverse domains more effectively than static embedding models.