Technical Summary: HRBench: Benchmarking and Understanding Thinking-Mode Switch Strategies in Hybrid-Reasoning LLMs

Problem Statement

Recent reasoning-oriented Large Language Models (LLMs) such as OpenAI o1 and DeepSeek-R1 achieve high performance on complex tasks through extended Chain-of-Thought (CoT) reasoning but incur substantial token overhead. To address this, a new generation of hybrid-reasoning LLMs (e.g., Qwen3.5, gpt-oss, Seed-OSS) has emerged, exposing explicit controls for users to trade off answer quality against inference cost. These models offer binary switches (think/no_think), discrete effort levels (High/Medium/Low), or numeric token budgets.

However, the field lacks a unified framework to evaluate adaptive thinking-mode switching strategies. Existing methods are evaluated under incomparable settings involving different models, datasets, metrics, and decoding configurations. Consequently, it remains unclear which strategy truly optimizes the efficiency-effectiveness trade-off, how training regimes influence these strategies, and how optimal choices vary with model scale and task domain.

Methodology: HRBench Framework

The authors introduce HRBench, a unified evaluation framework designed to systematically study thinking-mode switching. The framework is structured along two orthogonal axes:

1. Three Switching Strategy Families:

   • Prompt-Tuning (PT): The model implicitly selects a thinking mode during a single inference pass via carefully designed prompts (e.g., S1, TALE, ACPO).
   • Routing (RT): A two-stage "classify-then-generate" approach where a router evaluates query difficulty before dispatching the query to the appropriate mode (e.g., AdaptThink, HDFlow).
   • Speculative (SPEC): The model initiates in a fast mode (e.g., no_think) and dynamically switches to deep reasoning upon detecting uncertainty signals (e.g., MixReasoning, ADR).

2. Four Training Regimes:

   • Training-Free (TF): Methods relying solely on prompt engineering or rule-based heuristics without parameter updates.
   • Supervised Fine-Tuning (SFT): Training on optimal (correct, token-minimal) samples.
   • Offline RL (DPO): Direct Preference Optimization using preference pairs derived from rejection fine-tuning.
   • Online RL (GRPO): Group Relative Policy Optimization optimizing autonomous mode selection and routing policies.

This design yields 12 controlled evaluation configurations (3 strategies × 4 regimes). The framework evaluates these across 6 hybrid-reasoning LLMs (ranging from 2B to 1.1T parameters, including Qwen3.5, gpt-oss, Seed-OSS, DeepSeek-V3.1, and Kimi-K2.5) and 5 reasoning benchmarks covering Mathematics (MATH500, AIME 2025), Science (GPQA-Diamond), and Code (LiveCodeBench, Codeforces). The authors re-implemented 12+ representative prior methods within this unified pipeline to ensure fair comparison.

Key Contributions

1. Unified Evaluation Framework: HRBench is the first benchmark to systematically cover 12 configurations for thinking-mode switching, enabling controlled cross-strategy comparison under identical conditions.
2. Systematic Empirical Analysis: The paper provides a comprehensive characterization of how different strategies navigate the efficiency-effectiveness trade-off, how training signals interact with strategy choice, and how optimal combinations shift with model scale and task domain.
3. Open-Source Baselines: The authors release reference implementations for all 12 configurations and the integrated external methods, forming a plug-and-play platform for the community.

Key Results and Findings

1. Strategy Trade-off Profiles

• Prompt-Tuning (PT): Consistently achieves Pareto-optimal trade-offs. It often provides a "win-win" scenario, improving accuracy while substantially reducing token costs (e.g., +24% token reduction on Qwen3.5-9B) by guiding the model to allocate effort proportionally to difficulty.
• Routing (RT): Offers moderate token savings (e.g., ~13-18%) while preserving accuracy comparable to full reasoning. It functions by conservatively routing easy problems to low-effort modes while keeping hard benchmarks in full reasoning.
• Speculative (SPEC): Functions primarily as an accuracy-enhancing strategy rather than an efficiency one. It often increases token costs (due to re-thinking overhead) but yields notable accuracy improvements, particularly on code tasks where a "try-then-verify" mechanism excels.

2. Model Scale and Task Domain Effects

• Scale Dependency: No single strategy dominates universally across model sizes.
  • At smaller scales (2B), all strategies perform similarly.
  • At 9B and 1.1T, Prompt-Tuning leads in effectiveness.
  • At 20B and 671B, Speculative methods overtake PT in accuracy.
  • Routing is the most consistent in reducing token costs across scales but generally ranks lower in raw effectiveness compared to PT at larger scales.
• Domain Dependency:
  • Math and Science: PT is the clear winner, balancing accuracy and efficiency.
  • Code: SPEC achieves the largest accuracy boosts via its verification mechanism, though PT and RT also yield efficient gains.

3. Impact of Training

• Training Universally Improves Efficiency: Across all strategies, training (SFT, DPO, GRPO) maintains or slightly improves accuracy while achieving substantial token reductions (12% to 65%). Training primarily teaches the model when to skip reasoning rather than improving the reasoning capability itself.
• Strategy-Specific Gains: Routing benefits most from training in terms of efficiency (GRPO achieves 65% token reduction for RT). This is because the binary routing decision aligns well with training signals, whereas PT and SPEC involve finer-grained decisions (depth and trigger timing) that are harder to optimize with coarse rewards.
• Training Regime Roles: DPO yields the largest accuracy improvements; GRPO offers the greatest efficiency gains; SFT provides a balanced middle ground.

4. Fair Comparison of External Methods

When re-implemented under the unified pipeline, existing methods (e.g., RASC, AdaptThink, ADR) exhibit the same strategy-level trade-off patterns identified by the authors. No single external method dominates across all domains; performance varies significantly by task (e.g., RASC excels on AIME, Budget-Guidance on MATH500).

Significance and Claims

The paper claims that HRBench addresses a critical gap in the literature by providing the first controlled, unified platform to evaluate thinking-mode switching. The authors assert that their findings challenge the notion of a "one-size-fits-all" solution, demonstrating that the optimal strategy is highly dependent on the specific model scale and task domain.

By revealing that training gains are strategy-dependent and that different strategies occupy distinct regions of the trade-off space, the work provides a reference for practitioners to select appropriate switching mechanisms. The release of the framework and baselines is intended to facilitate more controlled research into efficient reasoning in hybrid-reasoning LLMs, moving the field beyond isolated evaluations toward systematic understanding.

The authors remain modest regarding limitations, noting that training-based evaluations were limited to the 9B scale due to computational constraints, the focus is on single-turn reasoning, and the evaluation covers only math, science, and code domains.