💬 NLP

Select-then-Solve: Paradigm Routing as Inference-Time Optimization for LLM Agents

This paper demonstrates that no single reasoning paradigm dominates across all tasks for LLM agents and proposes a "Select-then-Solve" approach, where a lightweight learned router dynamically selects the optimal paradigm per task to significantly outperform fixed strategies and recover a substantial portion of the performance gap to an oracle selector.

Original authors: Heng Zhou, Zelin Tan, Zhemeng Zhang, Yutao Fan, Yibing Lin, Li Kang, Xiufeng Song, Rui Li, Songtao Huang, Ao Yu, Yuchen Fan, Yanxu Chen, Kaixin Xu, Xiaohong Liu, Yiran Qin, Philip Torr, Chen Zhang, Zh

Published 2026-04-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Heng Zhou, Zelin Tan, Zhemeng Zhang, Yutao Fan, Yibing Lin, Li Kang, Xiufeng Song, Rui Li, Songtao Huang, Ao Yu, Yuchen Fan, Yanxu Chen, Kaixin Xu, Xiaohong Liu, Yiran Qin, Philip Torr, Chen Zhang, Zhenfei Yin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a brilliant but sometimes stubborn assistant (the AI) and a massive pile of different jobs to do. Some jobs are like writing a poem, others are like debugging a computer program, and some require you to go out and ask neighbors for information.

For a long time, researchers tried to solve a big question: Is it better to tell the assistant to "just do it," or should we give them a strict step-by-step recipe to follow?

This paper, titled "Select-then-Solve," is like a massive experiment where the researchers tried every possible recipe on every possible job to see what works best. Here is the breakdown in simple terms.

1. The Experiment: Trying Every Cooking Method

The researchers took four of the smartest AI models available and gave them 10 different types of challenges (like coding, math, trivia, and web searches). They tested six different "thinking styles" (paradigms):

Direct: "Just give me the answer." (Like ordering takeout).
Chain-of-Thought (CoT): "Think step-by-step before answering." (Like writing a grocery list before cooking).
ReAct: "Think, then act, then think again." (Like a detective who thinks of a clue, goes to the library to check it, then thinks again).
Plan-Execute: "Make a plan first, then do it." (Like an architect drawing blueprints before building a house).
Reflection: "Answer, check your work, and fix mistakes." (Like a teacher grading their own test).
ReCode: "Solve it by writing and running computer code." (Like using a calculator instead of doing math in your head).

2. The Big Surprise: One Size Does Not Fit All

The most important discovery was that no single thinking style is the best for everything. In fact, using the "right" style for the wrong job can actually make the AI dumber.

The "Over-Thinker" Problem: On a simple coding task, forcing the AI to "think step-by-step" (CoT) actually made it worse. It's like asking a professional chef to write a 10-page essay about how to chop an onion before they actually chop it. The extra chatter confused them, and the code broke.
The "Need a Map" Problem: On a task requiring web searches (like "Who played Ray in the Polish version of a TV show?"), the "Just do it" approach failed miserably. The AI hallucinated (made things up). But when they used the "Detective" style (ReAct), the AI went out, found the real answer, and succeeded.

The Analogy: Imagine you have a Swiss Army Knife. It has a blade, a screwdriver, and a corkscrew.

If you try to open a wine bottle with the blade, you'll fail.
If you try to tighten a screw with the corkscrew, you'll break the screw.
The paper found that AI models are the same. You need the right tool for the specific job.

3. The "Oracle" Gap: The Dream Scenario

The researchers calculated a "Dream Scenario" (called the Oracle). This is a magical supervisor who looks at a task, instantly knows exactly which thinking style is perfect for it, and assigns that style.

Result: This magical supervisor was 17% more accurate than the best single thinking style used for all tasks.
The Problem: We don't have a magical supervisor. We need a way to figure out the right tool before we start the job.

4. The Solution: The "Smart Dispatcher" (Select-then-Solve)

Instead of forcing the AI to use one style, the authors built a Lightweight Router. Think of this router as a traffic cop or a smart dispatcher at a busy airport.

How it works: Before the AI starts working on a task, the Router looks at the question.
- Is it a coding question? -> "Send to the Code Specialist."
- Is it a simple fact? -> "Send to the Direct Answerer (don't waste time)."
- Is it a complex web search? -> "Send to the Detective."
The Result: This simple switch improved the average accuracy of the AI significantly. It recovered about 37% of the gap between the "real AI" and the "magical supervisor."

5. Why the AI Can't Do This Itself

The researchers tried something else: they asked the AI to choose its own thinking style (Self-Routing).

The Strongest AI (GPT-5): It was okay at this. It knew when to be direct and when to think hard.
The Weaker AIs: They failed completely. They tended to pick the "Detective" style for everything, even simple questions. It's like a nervous student who, when asked a simple math question, decides to write a thesis, research the history of numbers, and interview a professor, only to get the answer wrong because they overcomplicated it.

The Takeaway

The paper concludes that the future of smart AI agents isn't about building a single "super-brain" that does everything perfectly. Instead, it's about building a smart manager that knows:

When to let the AI fly free (Direct).
When to give it a checklist (CoT).
When to send it to the library (ReAct).

In short: The smartest agent isn't the one with the most complex thinking process; it's the one that knows when to stop thinking and just do the work.

1. Problem Statement

The paper addresses a fundamental question in the development of Large Language Model (LLM) agents: When an agent improves on a task, is the gain derived from the underlying model or the reasoning paradigm (e.g., Chain-of-Thought, ReAct) wrapped around it?

Current research often introduces new paradigms and evaluates them on tasks tailored to their strengths, leading to fragmented evidence. There is a lack of controlled, large-scale comparisons to determine:

When does additional reasoning structure actually help?
When does it hurt performance?
Can a single paradigm dominate across all tasks and models?

The authors hypothesize that reasoning paradigms are complementary rather than universally superior, and that the optimal choice is task-dependent.

2. Methodology

A. The PARADIGM Framework

The authors built PARADIGM, a unified evaluation framework to ensure fair comparison.

Models: Four frontier LLMs: GPT-5, Gemini-3-Flash, Qwen3-Max, and Qwen3-30B.
Benchmarks: Ten diverse datasets covering code generation (HumanEval), mathematics (MATH500, AIME), QA (HotpotQA, NQ), knowledge (MMLU), and tool-heavy tasks (GAIA, SEAL, $\tau$ -bench).
Scale: Approximately 18,000 completed runs (761 tasks per model-paradigm pair).
Paradigms Studied: Six distinct inference-time strategies sharing the same base model and tool access (web search, code execution):
1. Direct: Free-form answer (no scaffolding).
2. CoT (Chain-of-Thought): Step-by-step verbal reasoning.
3. ReAct: Interleaved thought-action loops with tools.
4. Plan-Execute: Decompose into a plan, then execute.
5. Reflection: Generate, critique, and revise.
6. ReCode: Recursive code generation and execution as the reasoning substrate.

B. The Select-then-Solve Approach

Motivated by the finding that no single paradigm dominates, the authors propose a Paradigm Router:

Mechanism: Before answering a task, a lightweight router analyzes the task input and selects the most suitable paradigm from the six options (including Direct).
Router Architectures:
1. Handcrafted Features: Logistic Regression/MLP using dataset identity and text statistics.
2. Embedding-based: Encodes task text using text-embedding-3-small (1536 dims).
3. Combined: Concatenation of embeddings and handcrafted features.
4. Zero-Shot Self-Routing: The LLM itself selects its paradigm via prompting (no training).
Training: Routers are trained on 70% of tasks (labels = the most token-efficient successful paradigm) and evaluated on the held-out 30%.

3. Key Findings & Results

A. Task-Dependent Performance (The "Oracle" Gap)

Complementarity: Reasoning structure helps dramatically on some tasks but degrades performance on others.
- Example: ReAct improves over Direct by 44 percentage points (pp) on GAIA (information-seeking) but CoT degrades performance by 15pp on HumanEval (code generation) because step-by-step reasoning disrupts code quality.
Oracle Performance: An "Oracle" that selects the best paradigm for each specific task outperforms the best fixed paradigm by an average of 17.1pp across all models.
Model Capability: Reasoning structure acts as capability compensation. Weaker models (e.g., Qwen3-30B) gain significantly from structure (e.g., +46pp on HumanEval with CoT), while stronger models (GPT-5) sometimes suffer from unnecessary structure.

B. The Select-then-Solve Router Results

Performance: The embedding-based router (Combined LR) achieves an average accuracy of 53.1% across four models.
- This is a 5.5pp improvement over the Direct baseline (47.6%).
- It outperforms the best fixed paradigm (50.3%) by 2.8pp.
- It recovers up to 37% of the gap between the fixed baseline and the Oracle.
Cost Efficiency: The router achieves higher accuracy than always using ReAct while consuming only half the token cost (approx. 7.0k tokens vs. 14.0k for ReAct), as it frequently selects the cheaper "Direct" paradigm for knowledge-centric tasks.
Failure of Self-Routing: Zero-shot self-routing (where the model chooses its own paradigm) only works for the strongest model (GPT-5, 67.1%). Weaker models fail to calibrate their choices, often over-selecting tool-heavy paradigms even when unnecessary, resulting in performance below their Direct baselines.

C. Cost Analysis

Token Tiers: Direct/CoT use ~1.0x tokens; ReCode ~2.8x; ReAct ~4.0x; Plan-Execute ~6.9x; Reflection ~9.4x.
Trade-off: The router optimizes the cost-accuracy trade-off, avoiding expensive paradigms (like Reflection) when they offer negligible gains (e.g., +2pp for 9.4x cost).

4. Key Contributions

Controlled Large-Scale Comparison: A rigorous evaluation of six paradigms across four models and ten benchmarks, isolating the variable of "reasoning structure" from model and tool differences.
Evidence of Task-Conditional Value: Demonstrated that reasoning structure is helpful, neutral, or harmful depending on the task type (e.g., helpful for tool-use, harmful for code generation).
Select-then-Solve Framework: Introduced a lightweight, embedding-based router that dynamically selects paradigms, improving average accuracy by 2.8pp over the best fixed strategy.
Meta-Reasoning Insight: Showed that paradigm selection is a distinct meta-reasoning capability that current LLMs (except the strongest) cannot reliably perform via zero-shot prompting, necessitating an external learned router.

5. Significance

This paper shifts the paradigm of agent design from "finding the best single reasoning scaffold" to "learning when to use which scaffold."

Architectural Shift: It argues that the most effective agents are not those with the most elaborate scaffolds, but those that know when to step aside and answer directly.
Efficiency: By routing tasks to the most appropriate paradigm, systems can achieve near-oracle performance at a fraction of the computational cost.
Future Direction: The work suggests that inference-time optimization should focus on adaptive routing rather than static architectural choices, and that "self-routing" remains a challenging capability for current models.