💬 NLP

Lightweight Query Routing for Adaptive RAG: A Baseline Study on RAGRouter-Bench

This paper presents the first systematic evaluation of lightweight classifiers for query routing on the RAGRouter-Bench, demonstrating that a simple TF-IDF with SVM approach achieves high accuracy (93.2%) and significant token savings (28.1%) by leveraging surface keyword patterns over semantic embeddings to distinguish between factual, reasoning, and summarization queries.

Original authors: Prakhar Bansal, Shivangi Agarwal

Published 2026-04-07

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Prakhar Bansal, Shivangi Agarwal

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you run a massive, high-tech library where you have three different ways to find answers for your visitors:

The Quick Scan: A librarian who glances at the index and grabs the first relevant book. (Fast, cheap, but might miss complex details).
The Detective: A librarian who reads multiple books, connects the dots, and writes a detailed report. (Slower, costs more effort, but great for hard puzzles).
The Archivist: A librarian who gathers every document on a topic and summarizes the whole collection. (Very slow, very expensive, but necessary for big-picture overviews).

In the past, most libraries just used the Archivist for every single question, just to be safe. But that's like hiring a team of PhDs to answer "What's the weather?" It's a huge waste of time and money.

The Problem:
How do you know which librarian to send to which question without asking a human expert first? You need a "Traffic Cop" (a router) to look at the question and say, "Oh, this is a simple fact, send it to the Quick Scan!" or "This is a complex mystery, send it to the Detective!"

The Study:
The authors of this paper built a new test library called RAGRouter-Bench. It has nearly 8,000 questions from four different worlds: General Knowledge, Literature, Law, and Medicine. Each question is secretly labeled with what kind of "librarian" it actually needs.

They wanted to see if they could build a simple, lightweight computer program (a classifier) to act as the Traffic Cop, just by reading the question text.

The Experiment:
They tried 15 different combinations of "Traffic Cops" (algorithms) and "Reading Glasses" (how the computer sees the words):

The Glasses:
- TF-IDF: Looks at specific keywords (e.g., if the word "summarize" appears, it knows it's a big-picture task).
- MiniLM: Tries to understand the "vibe" or deep meaning of the sentence.
- Structural: Counts things like sentence length or looks for words like "why" or "how."
The Cops: Different types of math models (like SVM, Random Forest, etc.) to make the final decision.

The Surprising Results:

Simple is Better: The most effective "Traffic Cop" wasn't the one trying to understand deep meaning. It was the one just looking for keywords (TF-IDF) combined with a solid decision-maker (SVM).
- Analogy: It's like realizing that if a question starts with "Who is the president?", you don't need to analyze the deep philosophy of the sentence; you just need to know it's a "Who" question.
- This simple method got 93% accuracy.
The Savings: By using this smart router instead of just using the expensive "Archivist" for everything, they saved 28% of the computing power (tokens). That's a massive bill reduction for a company running these systems.
The Tricky Part: The system worked great for Law and Literature questions but struggled a bit with Medical questions.
- Why? In the medical part of the test, all the questions came from one giant, dense medical textbook. It was hard to tell a simple fact from a complex diagnosis just by looking at the words, because the "vocabulary" was the same for everything. In Law, the questions followed very specific, predictable patterns, making them easy to sort.

The Big Lesson:
The paper warns us: Don't just chase the cheapest option.
If you tell the router to always pick the cheapest librarian (the Quick Scan) to save money, you'll save 60% of the cost, but you'll give terrible answers to complex questions. The goal is a balance: save money without losing quality.

In a Nutshell:
This paper proves that you don't need a super-complex AI to decide how to answer a question. Sometimes, a simple tool that just looks for specific keywords is enough to route questions to the right "expert," saving a ton of money while keeping the answers accurate. However, to get even better, future systems will need to know not just the question, but also what books are available to answer it.

1. Problem Statement

Retrieval-Augmented Generation (RAG) systems utilize diverse retrieval strategies ranging from simple, low-cost methods (e.g., NaiveRAG) to complex, high-cost iterative pipelines (e.g., IterativeRAG). While selecting the optimal strategy per query can significantly reduce token costs and latency, most deployed systems apply a single paradigm uniformly.

The core problem addressed is the lack of query-side routing classifiers trained on the recently released RAGRouter-Bench (Wang et al., 2026). This benchmark contains 7,727 queries across four domains (Medical, Legal, Literature, Wikipedia), annotated with three canonical query types:

Factual (Single-hop)
Reasoning (Multi-hop)
Summarization (Corpus-level aggregation)

Prior to this work, no study had established a baseline for routing these queries using lightweight classifiers based solely on query text, without incorporating corpus-side signals.

2. Methodology

The authors conducted a systematic evaluation of lightweight classifiers to predict the optimal RAG paradigm based strictly on query text features.

A. Label Mapping & Cost Simulation

Target Labels: The three query types (Single-hop, Multi-hop, Summary) were mapped to specific RAG paradigms based on literature (Jeong et al., 2024):
- Single-hop $\rightarrow$ NaiveRAG (1.4× cost)
- Multi-hop $\rightarrow$ HybridRAG (2.8× cost)
- Summary $\rightarrow$ IterativeRAG (3.5× cost)
Cost Metric: Token savings were simulated post-hoc by comparing the predicted routing cost against a baseline of always using the most expensive paradigm (IterativeRAG).

B. Feature Regimes

Three distinct feature sets were extracted from query text without requiring retrieval or LLM calls:

Lexical (TF-IDF): Unigram and bigram TF-IDF vectors (3,000 feature ceiling).
Semantic (MiniLM): 384-dimensional dense sentence embeddings using all-MiniLM-L6-v2.
Structural (Hand-crafted): 23 features including query length, question-word type (who/what/why), negation presence, and clause counts.

C. Classifiers

Five classical machine learning classifiers were trained on each feature set (15 total combinations):

Logistic Regression, SVM (RBF), Random Forest, KNN, and MLP.
Evaluation was performed using 5-fold stratified cross-validation.
Primary Metrics: Macro-averaged F1 (to handle class imbalance) and Accuracy.

3. Key Contributions

First Baselines on RAGRouter-Bench: Established the first query-side classification baselines for this benchmark, quantifying the routing signal available from query text alone.
Feature Regime Comparison: Provided a comprehensive comparison of lexical, semantic, and structural features, revealing that surface-level keywords outperform semantic embeddings for this specific task.
Domain Analysis: Identified significant performance disparities across domains, highlighting that routing difficulty is heavily influenced by corpus structure (e.g., Medical vs. Legal).
Cost-Accuracy Trade-off Analysis: Demonstrated that maximizing token savings in isolation leads to poor answer quality, establishing the need for a balanced approach.

4. Key Results

Performance Metrics

Best Configuration: TF-IDF features + SVM classifier.
- Accuracy: 93.2%
- Macro-F1: 0.928
Feature Comparison: Lexical TF-IDF features outperformed Semantic MiniLM embeddings by 3.1 macro-F1 points (0.928 vs. 0.897) and Structural features by 14.0 points.
- Insight: Surface keyword patterns (e.g., "summarize," "compare," specific question words) are stronger predictors of query complexity than dense semantic vectors for this dataset.

Cost Savings

The best configuration (TF-IDF + SVM) achieved 28.1% token savings relative to always using the most expensive paradigm.
This represents 79% of the theoretical maximum savings (35.2%) achievable if the routing were perfect (ground-truth labels).
The "Majority Class" Trap: A baseline that always predicts the most common class (Single-hop) achieves 60.0% savings but with a macro-F1 of only 0.231. This highlights that optimizing for cost alone collapses the system to the cheapest paradigm, degrading answer quality.

Domain-Specific Findings

Easiest Domain: Legal (UltraDomain) – Macro-F1 of 0.967 (TF-IDF).
Hardest Domain: Medical (GraphRAG-Bench) – Macro-F1 of 0.803 (TF-IDF).
Reasoning: Medical queries are harder to route because the corpus consists of a single long document, making query types less distinguishable by surface vocabulary. Legal queries follow more formulaic patterns across diverse documents.

5. Significance and Limitations

Significance:

Reproducibility: Provides a solid, reproducible baseline for future research in adaptive RAG.
Efficiency: Proves that lightweight, non-LLM classifiers can achieve high routing accuracy, enabling significant cost reductions (nearly 30%) without complex inference.
Design Insight: The finding that TF-IDF outperforms embeddings suggests that for query-type classification, specific lexical triggers are more robust than general semantic similarity, challenging the assumption that dense embeddings are always superior.

Limitations:

Query-Only Focus: The models ignore corpus-side signals (e.g., document connectivity, density), which the original RAGRouter-Bench paper identifies as crucial for optimal paradigm selection. These baselines represent a "floor" for performance, not a ceiling.
Simplified Mapping: The mapping from query type to RAG paradigm is a simplification; in reality, the optimal paradigm depends on the specific query-corpus interaction.
Generalization: Models were evaluated only within the four domains of the benchmark; out-of-distribution generalization is untested.

Conclusion

The paper establishes that lightweight classifiers using simple lexical features (TF-IDF) can effectively route RAG queries, achieving ~93% accuracy and ~28% token savings. However, the gap between these query-only baselines and the theoretical optimum underscores the necessity of future work that integrates corpus-aware signals to fully realize the potential of adaptive RAG systems.