🤖 machine learning

Retriever Portfolios: A Principled Approach to Adaptive RAG

This paper proposes "Retriever Portfolios," a principled method that automatically selects a diverse subset of retrievers to handle heterogeneous queries, achieving superior accuracy with lower latency and cost compared to single-retriever or naive multi-retriever baselines.

Original authors: Miltiadis Stouras, Vincent Cohen-Addad, Silvio Lattanzi, Ola Svensson

Published 2026-06-01

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Miltiadis Stouras, Vincent Cohen-Addad, Silvio Lattanzi, Ola Svensson

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to answer a question, and you have a giant library of books (the internet) to help you. In the world of Artificial Intelligence, this is called RAG (Retrieval-Augmented Generation). The AI asks a "librarian" (a retriever) to find the best pages, and then a "writer" (the Large Language Model) uses those pages to write the answer.

For a long time, these systems used one single librarian for every single question. The problem? Some questions are simple ("Who is the president?"), while others are complex puzzles requiring you to connect dots across many different books ("How did the invention of the printing press influence the French Revolution?").

A librarian who is great at finding simple facts might be terrible at solving complex puzzles, and vice versa. Relying on just one librarian means you often get the wrong pages, leading to wrong answers or "hallucinations" (made-up facts).

The Solution: A "Retriever Portfolio"

The authors of this paper propose a smarter way: instead of hiring one librarian, hire a small team of specialized librarians (a "portfolio").

Think of it like a medical clinic.

The Old Way: You have one general practitioner who tries to treat everything from a broken toe to a heart attack. They might miss the heart attack because they are used to looking at feet.
The New Way (The Paper's Idea): You have a clinic with a foot doctor, a heart specialist, and a neurologist. When you walk in, a smart triage nurse (the "router") quickly looks at your symptoms and sends you to the best specialist for your specific problem.

In this paper, the "specialists" are different search algorithms (retrievers) with different settings. Some are good at finding exact word matches, others are good at finding ideas that sound similar, and others are good at connecting related concepts.

How They Built the Team

The researchers didn't just guess who to hire. They created a mathematical method to build the perfect team:

The Big Pool: They started with a massive pool of 360 different "librarians" (different search strategies and settings).
The "Best-of-K" Goal: They asked a simple question: "If we pick a small team of 5 librarians, which 5 will cover the most ground?" They didn't want 5 librarians who all do the exact same thing. They wanted a diverse team where if one fails, another succeeds.
The Algorithm: They used a greedy algorithm (a step-by-step selection process) to pick the team. It picks the best librarian first, then picks the second librarian who helps with the questions the first one missed, and so on.
The Result: They ended up with a small, diverse team (e.g., 4 or 5 specialists) that covers almost all types of questions.

How It Works in Real Life

When a user asks a question:

The Router: A lightweight AI looks at the question and decides, "This looks like a complex puzzle; let's ask the graph-expert and the diversity-expert."
Parallel Search: Instead of asking one librarian, then waiting, then asking another, the system asks the selected team members at the same time (in parallel).
The Final Answer: The system gets a few different sets of pages. A "judge" AI looks at the answers generated from these pages and picks the best one.

Why This is Better (The Results)

The paper tested this on several difficult question-answering benchmarks (like HotpotQA and TriviaQA). Here is what they found:

Better Accuracy: The team of specialists found the right information more often than any single librarian could, even if that single librarian was the "best" one on average.
Faster and Cheaper: Other methods tried to "tune" the librarian while answering the question (like trying different settings one by one until it works). This is slow and expensive. The paper's method picks the team beforehand. Because the team members work in parallel, it's much faster and uses fewer computer resources (tokens) while getting the same or better results.
Not Just "More Books": They proved that the improvement wasn't just because they were reading more pages. It was because they were reading the right pages from the right specialist.

The Bottom Line

This paper introduces a principled way to stop using a "one-size-fits-all" search engine for AI. By building a small, diverse team of search specialists and using a smart router to pick the right ones for the job, the system becomes more accurate, faster, and more efficient. It's the difference between having one generalist trying to do everything and having a specialized team working together to solve the problem.

Technical Summary: Retriever Portfolios: A Principled Approach to Adaptive RAG

1. Problem Formulation

Retrieval-Augmented Generation (RAG) systems typically rely on a single retriever with a fixed set of hyperparameters to handle all user queries. However, query distributions in real-world applications are highly heterogeneous, ranging from simple factoid questions to complex multi-hop reasoning tasks. Different retrieval configurations (models, strategies, or hyperparameters) often specialize in different regimes, meaning no single retriever is optimal for all queries. While recent works like Adaptive-RAG and Vendi-RAG attempt to adapt retrieval strategies, they either operate over a small, hand-designed set of strategies or perform expensive per-query hyperparameter tuning at inference time, leading to high latency and token costs.

This paper addresses the limitation of "one-size-fits-all" retrieval by proposing a principled approach to Retriever Portfolio Optimization. The core problem is defined as selecting a small, diverse subset (a portfolio) of $k$ retrievers from a large candidate pool of $m$ candidates. The goal is to maximize the expected performance of the "best-of- $k$ " retriever for a random query drawn from the underlying distribution, rather than maximizing the average performance of a single retriever.

2. Methodology

2.1. Theoretical Framework

The authors formalize the problem using an expected best-of- $k$ objective. Let $Q$ be the set of queries with an unknown distribution $D$ , and $\mathcal{R}$ be the set of candidate retrievers. A score function $s(q, r) \in [0, 1]$ measures the suitability of retriever $r$ for query $q$ . The quality of a portfolio $S \subseteq \mathcal{R}$ (where $|S| \le k$ ) is defined as:
$F(S) = \mathbb{E}_{q \sim D} \left[ \max_{r \in S} s(q, r) \right]$
The objective is to find $S$ that maximizes $F(S)$ . The authors prove that for any fixed query $q$ , the function $f_q(S) = \max_{r \in S} s(q, r)$ is non-negative, monotone, and submodular. Consequently, the population objective $F(S)$ and its empirical estimate are also submodular set functions.

2.2. Algorithm: Greedy Portfolio Selection

To solve this optimization problem efficiently, the paper proposes Algorithm 1, a sample-based greedy procedure:

Sampling: Draw $N$ i.i.d. queries from the distribution $D$ .
Empirical Estimation: Replace the population objective with an empirical estimate $\hat{F}_Q(S)$ .
Greedy Construction: Iteratively select the retriever $r$ that maximizes the marginal gain $\hat{F}_Q(S \cup \{r\}) - \hat{F}_Q(S)$ . The algorithm maintains a memoized vector of the current best scores per query to compute gains efficiently.
Guarantees: The algorithm runs in $O(k |\mathcal{R}| N)$ time. Theoretical analysis shows it achieves a $(1 - 1/e)$ -approximation to the optimal portfolio, up to a statistical error term $\epsilon$ , provided the sample size $N$ scales appropriately with the pool size and desired confidence.

2.3. Inference Pipeline

The proposed system consists of two stages:

Offline Portfolio Construction: Using Algorithm 1, a diverse portfolio of $k$ complementary retrievers is selected from a large pool (including Dense, Diversity-based, and Graph-based retrievers) based on training data.
Adaptive Inference: A lightweight router is trained to dynamically select the best retriever(s) for a specific query.
- The router encodes the query using raw text and cached dense embeddings (MPNet/E5).
- It scores the $k$ portfolio members via similarity to learned retriever embeddings.
- The top- $\ell$ retrievers are executed in parallel to minimize latency.
- A selector LLM aggregates the answers from the parallel retrievers to produce the final output.

This approach amortizes the cost of adaptation offline, avoiding the iterative search required by inference-time tuning methods.

3. Key Contributions

Problem Formulation: Introduction of the "RAG portfolio optimization" problem, formalizing the selection of a small, diverse set of retrievers to cover heterogeneous query distributions via an expected best-of- $k$ objective.
Theoretical Guarantees: Proof that the objective is submodular, allowing for the efficient construction of near-optimal portfolios with provable $(1 - 1/e)$ approximation guarantees.
Practical Pipeline: A complete system comprising offline portfolio selection and a lightweight router that enables parallel retrieval and answering, significantly reducing inference latency compared to sequential adaptive methods.
Empirical Validation: Comprehensive evaluation across four diverse QA benchmarks (HotpotQA, 2WikiMultiHopQA, TriviaQA, MusiQue) demonstrating superior performance over single-retriever baselines and inference-time tuning approaches.

4. Experimental Results

The method was evaluated on four open-domain QA datasets using two answer models (Gemma-3-27B-IT and Llama-3.1-70B-Instruct).

Retrieval Coverage: Learned portfolios significantly outperform "average-best" retriever selection. For example, a portfolio of size $k=5$ achieved a Support Recall of 0.594 compared to 0.492 for the top-5 average retrievers. The gains were shown to stem from complementarity (covering different query subpopulations) rather than simply retrieving more documents.
End-to-End Accuracy: Routed portfolios consistently achieved higher Exact Match (EM) scores than single-retriever baselines and Vendi-RAG (an inference-time tuning baseline). The "all-pool" portfolio (combining Dense, Vendi, and GraphDense retrievers) yielded the best average EM across all datasets.
Efficiency vs. Accuracy: In controlled comparisons with Vendi-RAG (using the same Vendi search space), the offline portfolio approach achieved comparable or better accuracy with substantially lower latency and token costs. This is because the portfolio allows for parallel execution of a fixed set of retrievers, whereas Vendi-RAG requires sequential, iterative generation and judging.
Router Effectiveness: Ablation studies showed the learned router significantly outperforms random selection and consistently selects retrievers that improve upon the single best average member, effectively routing queries to the most suitable portfolio members.

5. Significance and Claims

The paper claims to advance the field of RAG by shifting the optimization burden from expensive, sequential inference-time adaptation to efficient, offline portfolio construction. The authors argue that:

No single retriever is optimal: Fixing a single strategy leaves performance on the table for diverse information needs.
Complementarity is key: A small set of diverse retrievers can cover the query distribution more effectively than a single "best" retriever or simply increasing the document budget.
Efficiency: Fixed portfolios enable parallel retrieval and LLM calls, offering a practical trade-off that achieves high accuracy with predictable, low latency and token usage, making adaptive RAG more deployable in production environments.

The work does not claim to solve all RAG challenges but provides a principled, theoretically grounded method for selecting and routing among retrieval strategies that is both provably near-optimal and empirically effective.