⚡ electrical engineering

Natural Language Query to Configuration for Retrieval Agents

The paper introduces **BRANE**, a system that dynamically optimizes retrieval agent configurations per query by using an LLM to extract workload characteristics and a lightweight predictor to select the most cost-effective pipeline setting, thereby achieving superior cost-quality tradeoffs compared to static tuning and existing baselines.

Original authors: Melissa Z. Pan, Negar Arabzadeh, Mathew Jacob, Fiodar Kazhamiaka, Esha Choukse, Matei Zaharia

Published 2026-05-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Melissa Z. Pan, Negar Arabzadeh, Mathew Jacob, Fiodar Kazhamiaka, Esha Choukse, Matei Zaharia

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are running a high-end restaurant kitchen. You have a massive menu of ingredients (the internet), a team of chefs with different skill levels (AI models ranging from fast but simple to slow but brilliant), and various cooking techniques (searching, summarizing, or asking follow-up questions).

Every time a customer orders a dish (a user asks a question), you have to decide: Which chef should cook it? How much time should they spend? Do they need to look up a recipe, or can they just guess?

Currently, most restaurants pick one standard recipe for the whole day. They might say, "We always use our best, most expensive chef for every order." This works, but it's wasteful. If a customer just wants a simple glass of water, using a world-class chef is a huge waste of money and time. If they want a complex 10-course meal, a simple chef might fail.

This paper introduces a new system called BRANE (Building Retrieval Agents via Natural language Expressions) that acts like a super-smart, instant sous-chef who looks at every single order and picks the perfect cooking strategy for that specific moment.

Here is how it works, broken down into simple concepts:

1. The Problem: One Size Does Not Fit All

The authors noticed that different questions need different "pipelines" (combinations of tools and models).

The "Water" Question: "What is the capital of France?" needs a quick, cheap, simple answer.
The "Complex" Question: "Analyze the financial trends of Company X compared to Y over the last decade, considering the impact of the 2022 regulations." needs a deep, expensive, multi-step investigation.

If you use the "Complex" setup for the "Water" question, you waste money. If you use the "Water" setup for the "Complex" question, you get a bad answer. The paper shows that the best setup changes from question to question, even within the same topic.

2. The Solution: BRANE's "Smart Menu"

Instead of picking one chef for the whole day, BRANE looks at the order and instantly decides the best path. It does this in three steps:

Step 1: The "Taste Test" (Profiling)
Before the restaurant opens, the team tries every possible combination of chefs and cooking methods on a sample of 150–600 orders. They record: "If we used Chef A with Method X, did it cost $0.01 and get the answer right? What about Chef B with Method Y?" This creates a massive map of what works and what doesn't.
Step 2: The "Translator" (Characterization)
When a new order comes in, BRANE doesn't just read the words; it asks a smart AI (an LLM) to translate the order into a list of binary traits (Yes/No questions).
- Example: "Does this question need to connect three different facts?" (Yes/No)
- Example: "Does this involve a specific person's name?" (Yes/No)
- Example: "Is this about a specific year?" (Yes/No)
Think of this as the sous-chef looking at the order and saying, "Ah, this is a 'multi-hop' order," or "This is a 'financial calculation' order." These traits are specific to the type of restaurant (the dataset) they are working in.
Step 3: The "Instant Decision" (Prediction)
Using the traits from Step 2, BRANE checks its map from Step 1. It asks: "For an order with these traits, which cooking method is the cheapest one that still guarantees a correct answer?"
It then picks that specific configuration instantly.

3. The Results: Saving Money Without Losing Quality

The paper tested BRANE on three different "restaurants" (datasets):

MuSiQue: Complex trivia questions.
BrowseComp-Plus: Deep research questions.
FinanceBench: Financial data questions.

The Magic Numbers:

Up to 89% Cheaper: On some tasks, BRANE achieved the exact same accuracy as the best fixed system but cost 89% less.
The Pareto Frontier: Imagine a graph where the top-left corner is the "perfect" spot (High Accuracy, Low Cost). Most systems are stuck in the middle. BRANE pushes the line further up and to the left, finding combinations that no one else found.
Beating the Competition: It outperformed other smart routing systems that only changed the "chef" (the AI model) but kept the rest of the kitchen the same. BRANE changes the whole kitchen setup (the model, the search depth, the summary style) based on the order.

4. Why This Matters

The paper argues that we don't need to build a "super-duper" AI that is perfect at everything. Instead, we should build a system that is smart enough to know when to be simple and when to be complex.

By treating every question as unique and selecting the most efficient "recipe" for it, we can save a massive amount of money and computing power while still getting the right answers. It's like having a waiter who knows exactly which waiter to call for your table, rather than having the head chef serve every single table.

In short: BRANE is a system that reads your question, figures out exactly how hard it is, and instantly picks the cheapest, most effective way to answer it, saving money without sacrificing quality.

Technical Summary: BRANE – Building Retrieval Agents via Natural Language Expressions

Problem Formulation: Query2Conf

Modern retrieval agents (e.g., RAG systems, research assistants) expose a vast configuration space including the choice of Large Language Model (LLM), retriever, number of retrieved documents ( $k$ ), number of reasoning hops, and synthesis strategy. Currently, these pipelines are typically hand-tuned once per workload, resulting in a static configuration that is suboptimal for individual queries.

The authors formulate Query2Conf: given a natural-language query and a target accuracy (or cost budget), dynamically select the pipeline configuration from a predefined catalog that minimizes cost while meeting the accuracy target. This problem is motivated by three empirical observations:

Per-query variance: Within a single workload, different queries have vastly different optimal configurations; no single static configuration is optimal across the entire workload.
Full-pipeline impact: The cost-quality trade-off is driven more by the interaction of the full pipeline (retriever, synthesis, depth) than by the LLM choice alone.
Workload-specific signals: The characteristics that distinguish queries and dictate configuration choices are workload-specific (e.g., financial metrics in FinanceBench vs. multi-hop reasoning in MuSiQue) and cannot be captured by generic, fixed query features or standard embeddings.

Methodology: The BRANE Framework

The paper proposes BRANE (Building Retrieval Agents via Natural language Expressions), a framework that solves Query2Conf through three stages:

1. Configuration Profiling (Offline)

The system samples a set of queries from the workload and executes each against every candidate configuration in the design space. This generates a dataset of correctness outcomes ( $y_c(q) \in \{0,1\}$ ) and dollar costs ($cost(q, c)$) for every query-configuration pair.

2. Predictor Training (Offline)

Instead of modeling the complex, non-monotonic interactions of pipeline knobs directly, BRANE trains a lightweight predictor for each configuration.

Workload-Specific Characterization: A frontier LLM (e.g., GPT-5-mini) is prompted with a small sample of queries to propose a set of binary, workload-specific characteristics (e.g., involves_regional_cuisine, requires_multi_hop). These characteristics act as a translation layer between query semantics and the configuration space. A cheaper LLM then labels all profiled queries with these binary features, creating a feature vector $F(q)$ .
Per-Configuration Prediction: For each configuration $c$ that survives "fuzzy Pareto pruning" (retaining configurations near the cost-quality frontier), a lightweight classifier (e.g., Logistic Regression, XGBoost) is trained to estimate the probability of correctness: $\hat{p}_c(F(q)) \approx P(y_c(q)=1 | F(q))$ .

3. Inference-Time Selection

At runtime, for a new query $q$ :

The system computes the binary feature vector $F(q)$ using the characterizer LLM.
It evaluates the predicted correctness $\hat{p}_c(F(q))$ for all Pareto-optimal configurations.
It selects the configuration $c^*$ that maximizes a Lagrangian score:
$c^* = \arg\max_{c} \left( \hat{p}_c(F(q)) - \lambda \cdot \text{cost}(c) \right)$
where $\lambda$ is a tunable parameter mapping the user's accuracy target to a specific point on the cost-quality Pareto frontier.

Key Contributions

Query2Formulation: The authors formalize the problem of per-query pipeline configuration, motivated by the observation that static workload-level tuning leaves significant cost-quality optimization untapped.
Workload-Specific Characterization: The paper demonstrates that LLM-proposed binary characteristics specific to a workload outperform generic embeddings and fixed rule-based features as inputs for configuration predictors.
BRANE Framework: A complete system solving Query2Conf via offline profiling, per-configuration predictor training, and Lagrangian routing. It avoids the need for end-to-end fine-tuning of large models for routing.

Experimental Results

The authors evaluate BRANE on three benchmarks: MuSiQue, BrowseComp-Plus, and FinanceBench, comparing it against static configurations, LLM routers (Carrot), rule-based routers (METIS, Adaptive-RAG), and end-to-end fine-tuned models (BERT, Qwen3-4B).

Cost-Quality Pareto Frontier: BRANE consistently pushes the Pareto frontier beyond the best static configuration.
Cost Savings: At the strict goal of matching the accuracy of the best static pipeline, BRANE achieves cost savings of up to 89.4% on MuSiQue, 81.7% on BrowseComp-Plus, and 8.1% on FinanceBench.
Accuracy Goals: BRANE is the only method capable of reaching the strict 100% accuracy goal across all three benchmarks while simultaneously reducing costs.
Baselines: BRANE outperforms extended versions of LLM-only routers (which fail to reach high accuracy goals) and rule-based systems (which do not generalize well to new workloads).
Model Efficiency: Classical per-configuration predictors trained on the binary characteristics outperform end-to-end fine-tuned BERT and Qwen3-4B models, particularly in reaching strict accuracy goals.

Significance and Claims

The paper claims that per-query configuration of the full retrieval pipeline is a practical alternative to static workload-level tuning. By decoupling the semantic understanding of the query (via LLM-generated characteristics) from the prediction of pipeline performance (via lightweight classifiers), BRANE achieves significant cost reductions without sacrificing accuracy. The authors emphasize that this approach leverages the "cost-quality headroom" inherent in modern AI pipelines, which is currently inaccessible due to the combinatorial complexity of the configuration space and the lack of per-query optimization strategies. The framework is designed to be robust, requiring only a one-time offline profiling pass and allowing engineers to retarget accuracy or budget constraints at inference time without retraining.