Mid-Think: Training-Free Intermediate-Budget Reasoning via Token-Level Triggers

` tag) attracts the highest attention from subsequent tokens, effectively acting as a switch to activate reasoning.

• The Newline Trigger: In "No-Think" mode, the newline pattern (\n\n) following the </think> tag absorbs the majority of attention, suppressing reasoning behavior.
• Overfitting Phenomenon: The authors observe that reasoning models and their training datasets (e.g., OpenR1-Math) exhibit "overfitting" to these unified opening patterns (e.g., starting thoughts with "Okay"). This suggests the models rely on these tokens as hard switches rather than interpreting semantic instructions.

2.2 Mid-Think: A Training-Free Prompting Format

Leveraging these observations, the authors propose Mid-Think, a prompting format that combines the activating and suppressing cues to induce an intermediate reasoning regime without any model retraining.

The proposed format is:
...</think>\n\n<reason>\nOkay...

• Mechanism: The format explicitly integrates the </think>\n\n pattern (which suppresses reasoning) with the Okay token (which activates reasoning). The <reason> tag (or alternatives like <begin>, <less think>) serves as a structural delimiter for the thinking content.
• Effect: This combination creates a "conflict" or balance between the two signals, inducing the model to generate a reasoning trace that is shorter than the standard "Think" mode but more robust than the "No-Think" mode.

2.3 Application to RL Training

The authors further apply the Mid-Think format as a training objective in Reinforcement Learning (RL) following Supervised Fine-Tuning (SFT). By training models to generate responses under the Mid-Think format, they aim to:

• Reduce the length of reasoning sequences during the RL optimization phase.
• Increase policy entropy during training (preventing early collapse into fixed patterns).
• Improve the final model's ability to balance reasoning depth and efficiency.

3. Key Results

3.1 Inference-Time Control (Training-Free)

Evaluated on MATH500, AIME, and GPQA across various models (Qwen3-8B/14B/32B, DeepSeek-R1-Distill, Phi-4):

• Pareto Optimality: Mid-Think consistently achieves performance lying on or beyond the Pareto frontier of the accuracy-length trade-off. It outperforms fixed-token budget baselines and prompt-based methods.
• Performance: On MATH500, Mid-Think achieves accuracy comparable to high-budget "Think" modes (e.g., 92% vs 94% for Qwen3-14B) but with significantly reduced generation lengths (2,589 tokens vs ~4,904 tokens).
• Generalization: The method generalizes across different model families (Llama, Qwen, Phi) and architectures (Dense, MoE), provided the trigger token matches the model's training template (e.g., using "We" instead of "Okay" for models trained on "We need...").
• Task Versatility: The approach extends beyond mathematics to coding tasks (LiveCodeBench), maintaining superior accuracy-length trade-offs.

3.2 RL Training Efficiency and Performance

When applied to RL training (using GRPO) on Qwen3-4B and Qwen3-8B:

• Training Efficiency: Mid-Think reduces RL training time by approximately 15% (e.g., from 54 hours to 46 hours for Qwen3-8B) compared to standard "Think" mode training, due to shorter reasoning sequences generated during optimization.
• Entropy Dynamics: Mid-Think training maintains higher entropy during the training process compared to standard "Think" mode, which tends to collapse into low-entropy regimes quickly.
• Final Performance: Models trained with Mid-Think achieve higher accuracy in "Think" mode evaluations than those trained with standard "Think" or "No-Think" modes.
  • AIME: Qwen3-8B improved from 69.8% (standard RL) to 72.4%.
  • GPQA: Qwen3-8B improved from 58.5% to 61.1%.
• Behavioral Balance: Unlike "No-Think" training which often degrades "Think" mode performance, Mid-Think-trained models preserve efficiency while retaining strong reasoning capabilities.

4. Significance and Claims

The paper claims that hybrid reasoning behaviors in large language models are largely governed by a few token-level triggers rather than complex semantic instructions. Based on this, the significance of the work is threefold:

1. Mechanism Discovery: It reveals that reasoning modes can be manipulated via specific lexical cues ("Okay" vs. \n\n), offering a lightweight lens for understanding model behavior.
2. Efficient Inference: Mid-Think provides a training-free mechanism to achieve intermediate-budget reasoning, offering a more reliable and fine-grained control over the accuracy-efficiency trade-off than existing baselines.
3. Improved RL Training: By integrating these triggers into the RL objective, the method accelerates training and yields models with better-balanced reasoning behaviors, suggesting that controlling the "budget" during training is a viable strategy for improving final model performance.

The authors note limitations, specifically that Mid-Think does not offer fully dynamic control over arbitrary budget levels (e.g., precise 0.1 or 0.7 ratios) and relies on the prior existence of overfitted token-level behaviors in the model's training data.