StyleBench: Evaluating thinking styles in Large Language Models

1. Problem Statement

Large Language Models (LLMs) have demonstrated significant reasoning capabilities through structured prompting techniques (e.g., Chain-of-Thought, Tree-of-Thought). However, the community lacks a systematic understanding of when and why specific reasoning structures improve performance versus when they introduce inefficiency or failure.

• The Gap: Existing literature often evaluates a single reasoning style on a narrow set of tasks or models, assuming a universal "best" strategy. It fails to account for the trade-off between reasoning depth/complexity and computational efficiency, nor does it adequately address how these strategies interact with varying model capacities (parameter sizes).
• The Core Question: How do different reasoning structures (depth, branching, representation density) interact with task complexity and model capacity to determine inference success, and can models learn to adaptively select the optimal strategy?

2. Methodology: StyleBench

The authors propose StyleBench, a systematic empirical framework that treats reasoning strategies not as fixed recipes, but as capacity-constrained design choices.

A. Reasoning Strategies Evaluated

The study evaluates five representative reasoning styles spanning a spectrum of structural complexity:

1. Chain-of-Thought (CoT): Linear, step-by-step natural language decomposition.
2. Chain-of-Draft (CoD): Concise, symbolic intermediate representations to reduce verbosity.
3. Sketch-of-Thought (SoT): Highly compressed, task-specific symbolic sketches (minimal text).
4. Tree-of-Thought (ToT): Parallel exploration of multiple reasoning paths with a voting mechanism.
5. Algorithm-of-Thought (AoT): Search-based strategies involving backtracking and iterative exploration.

B. Experimental Setup

• Models: 15 open-source LLMs ranging from 270M to 120B parameters, categorized into three capacity regimes: Small (<5B), Medium (5B–15B), and Large (>15B).
• Tasks: Five diverse reasoning tasks:
  • Mathematical: GSM8K, AIME.
  • Logical: LogiQA.
  • Combinatorial/Puzzle: Game24.
  • Knowledge Retrieval: CommonsenseQA.
• Adaptive Control Experiments: To test if strategy selection can be learned, the authors trained selectors on Qwen-7B-Instruct using two methods:
  • Supervised Fine-Tuning (SFT): Trained to predict the best style based on ground-truth labels.
  • Group Relative Policy Optimization (GRPO): A reinforcement learning approach where the model learns to select (or synthesize) styles based on outcome rewards (accuracy) and penalties (token cost).

3. Key Contributions

1. Systematic View of Reasoning Structure: Conceptualizes reasoning strategies as design choices involving trade-offs between accuracy and efficiency, rather than a hierarchy of "better" methods.
2. Capacity-Dependent Trade-offs: Identifies specific regimes where structured reasoning helps or harms, revealing that "more structure" is not universally better.
3. Failure Mode Characterization: Documents systematic failures in smaller models, including premature guessing and instruction-following breakdowns under complex constraints.
4. Adaptive Control Insights: Demonstrates that supervised learning for strategy selection collapses to superficial preferences, whereas reinforcement learning (GRPO) enables effective adaptive control and style synthesis.

4. Key Results

A. Scaling Effects and Model Capacity

• Search-Based Styles (ToT, AoT): These complex, branching strategies only improve performance on large-scale models (>15B) and only on open-ended, combinatorial problems (e.g., Game24, AIME). On small and medium models, they often degrade performance due to error propagation and instruction-following failures.
• Concise Styles (CoD, SoT): These perform robustly across all model sizes. They achieve significant efficiency gains (lower token usage) on structured tasks without sacrificing accuracy, making them ideal for resource-constrained scenarios.
• Knowledge-Intensive Tasks: On tasks like CommonsenseQA, model capacity (internal knowledge) dominates performance. Structured reasoning provides little benefit for small models and converges for large models, regardless of the style used.

B. Task Dependence

• Mathematical Reasoning (GSM8K): Linear CoT is consistently the most effective and sufficient strategy; adding branching (ToT/AoT) introduces unnecessary overhead.
• Logical Reasoning (LogiQA): Symbolic/Concise styles (SoT) outperform verbose CoT, as they align better with the relational constraints of logic puzzles.
• Combinatorial Problems (Game24): Search-based methods (AoT/ToT) are necessary for large models to explore solution spaces, but they fail on smaller models which cannot maintain the search state.

C. Failure Modes in Small Models

• Premature Commitment: Small models often terminate reasoning early with plausible but incorrect answers before exhausting their token budget.
• Instruction Violation: Smaller models struggle to adhere to strict formatting or control instructions required by complex styles (e.g., ToT's voting mechanism), leading to robustness issues.

D. Adaptive Control: SFT vs. GRPO

• SFT Limitations: Supervised fine-tuning for style selection collapses to "shallow style preferences," often defaulting to a single strategy regardless of the input.
• GRPO Success: The GRPO-based selector learns to adaptively choose strategies based on task difficulty. Crucially, it can synthesize new hybrid styles (e.g., "Pair-and-Target Search") when predefined styles are insufficient, leading to superior downstream performance on Qwen-7B.

5. Significance and Implications

• Redefining Reasoning: The paper shifts the paradigm from "finding the best reasoning style" to "matching the reasoning structure to the model's capacity and the task's demands."
• Efficiency vs. Accuracy: It provides empirical evidence that for many tasks, concise reasoning (CoD/SoT) offers the best Pareto frontier, providing high accuracy with minimal computational cost, whereas complex search methods are only justified for large models on specific hard problems.
• Learning to Reason: The results suggest that online credit assignment (RL/GRPO) is a more effective path for learning adaptive reasoning control than offline supervised learning, as it allows models to discover novel strategies and manage the trade-off between exploration and efficiency dynamically.
• Resource Allocation: Practitioners can use these findings to optimize inference costs by avoiding expensive search-based prompts for small models or simple tasks, while reserving them for large models tackling combinatorial challenges.

In conclusion, StyleBench establishes that structured reasoning is a double-edged sword: it is a powerful tool for large models on complex tasks but can be a liability for smaller models or knowledge-heavy tasks. The ability to learn when to apply these structures is itself a critical inference problem.