Technical Summary: HEAVYSKILL – Heavy Thinking as the Inner Skill in Agentic Harness

Problem Statement

Recent advances in agentic harnesses have achieved success in complex reasoning tasks by orchestrating multiple agents with memory, skills, and tool use. However, the underlying mechanisms driving this performance remain obscured by intricate system designs. Current approaches often rely on external orchestration loops that activate sub-agents in parallel and summarize results, or employ specialized architectural modifications and post-training recipes to elicit parallel reasoning. The paper argues that the pursuit of reasoning capability has focused heavily on test-time scaling (TTS) strategies that amplify computational resources, yet the specific contribution of "heavy thinking" as an internalizable skill within the model's parameters remains under-explored. Furthermore, existing methods often lack a unified, portable framework that can be decoupled from specific infrastructure to function as a native capability of the Large Language Model (LLM).

Methodology

The authors propose HEAVYSKILL, a perspective and framework that treats "heavy thinking" not merely as an execution unit in an orchestration harness, but as an inner skill internalized within the model. This skill is defined as a two-stage pipeline: parallel reasoning followed by sequential deliberation.

1. The Heavy Thinking Framework

The inference pipeline is decomposed into two distinct phases:

• Parallel Reasoning: Given a problem $q$, the model generates $K$ independent reasoning trajectories ($T_{\pi_\theta}(q, K) = \{y_1, \dots, y_K\}$). These trajectories are generated without access to each other's outputs to ensure diversity.
• Sequential Deliberation: A second model (or the same model) aggregates the generated trajectories. This stage involves a "summary" process where the model critically evaluates the reasoning paths, identifies errors, and synthesizes a final answer.
• Serialized Memory Cache: To bridge these phases, the framework introduces a memory cache mechanism. Since full trajectories often exceed context limits, they are pruned, shuffled to prevent positional bias, and serialized into a context ($C(x_c)$) for the deliberation stage.
• Iterative Deliberation (Optional): The framework supports iterative refinement where the output of the deliberation stage is fed back into the memory cache as a new input for subsequent rounds of analysis.

2. HEAVYSKILL as a Readable Skill

A key innovation is the distillation of this workflow into a readable skill file (e.g., HeavySkill.md). Unlike traditional workflows that rely on external Python pipelines to manage API calls and state, HEAVYSKILL is a human-readable, model-interpretable document.

• Structure: The skill file contains four components:
  1. Activation Conditions: Declarative rules for when to trigger heavy thinking (e.g., complex reasoning vs. simple queries).
  2. Parallel Reasoning Protocol: Instructions to spawn $K$ independent agents with diverse strategies.
  3. Deliberation Prompt: A core template instructing the model to critically evaluate, not just vote, on the trajectories. It explicitly prohibits superficial concatenation and demands genuine synthesis.
  4. Output Constraints: Rules for formatting the final answer (e.g., \boxed{} for math).
• Portability: Because it is a plain-text document, it can be injected into any agentic harness (e.g., Claude Code, CodeX, Hermes) that supports skill loading, allowing the orchestrator to autonomously execute the protocol without code modification.

3. Optimization via RLVR

The paper explores adapting Reinforcement Learning from Verifiable Rewards (RLVR) to optimize heavy thinking. The hypothesis is that RLVR can simultaneously improve reasoning breadth (via parallel generation) and depth (via deliberation), potentially pushing the model beyond its intrinsic per-trajectory constraints.

Key Contributions

1. Framework Proposal: Introduction of a simple, training-free framework for reproducing heavy thinking via parallel reasoning and sequential deliberation.
2. Comprehensive Empirical Study: The first systematic evaluation of heavy thinking across diverse model scales (from 7B to frontier models) and task domains (STEM, coding, general reasoning).
3. Skill Distillation: The conceptualization of heavy thinking as a portable, readable "inner skill" that decouples reasoning capability from specific orchestration infrastructure.
4. Systematic Analysis: Detailed insights into the interplay between framework components, including the roles of trajectory diversity, model selection for deliberation, and the effects of iterative refinement.

Experimental Results

Performance on STEM Tasks

Evaluated on benchmarks including AIME25, BeyondAIME, HMMT25-Feb, and GPQA-Diamond:

• Consistent Outperformance: Heavy thinking (measured by Heavy-Mean@4 or HM@4) consistently outperforms single-trajectory attempts (Mean@K) and traditional Best-of-N strategies like Majority Voting (Vote@K).
• Scaling Laws: Increasing the number of parallel trajectories ($K$) and employing sequential deliberation leads to continuous performance improvement, validating heavy thinking as a practical realization of Test-Time Scaling.
• Approaching Theoretical Limits: Stronger models (e.g., DeepSeek V3.2, GPT-5 Thinking) under heavy thinking approach the theoretical Pass@K upper bound. In some cases, the deliberation process (Heavy-Pass@K) exceeds the raw Pass@K, suggesting the model can synthesize correct solutions not present in any single raw trajectory.
• Hierarchy: A consistent performance hierarchy was observed: Heavy-Pass@k ≥ Heavy-Mean@K ≥ Vote@K ≥ Mean@K.

Performance on General Reasoning Tasks

• Objective vs. Subjective: The framework shows substantial gains on objective, verifiable tasks (e.g., LiveCodeBench, IFEval). However, gains are marginal or occasionally negative on subjective, preference-based tasks (e.g., Arena-Hard), indicating that sequential deliberation excels at "correctness-oriented" tasks but may struggle with "preference-oriented" nuances.
• Model Selection: The deliberation stage does not strictly require a model with superior intrinsic reasoning power. Larger, instruction-following models (e.g., Qwen2.5-32B) used as deliberators can significantly boost performance even if they are weaker at solving the problems independently, highlighting the importance of synthesis capabilities over raw reasoning power in the second stage.

Iterative Deliberation and Tool Use

• Iteration: While increasing iterations generally improves accuracy, it introduces a trade-off where Heavy-Pass@K may degrade due to cumulative noise or bias from earlier stages.
• Tool Integration: The framework successfully integrates with external tools (e.g., Python interpreters). Heavy thinking outperforms voting baselines in tool-interleaved scenarios, effectively leveraging execution feedback to refine trajectories.

RLVR Adaptation

Preliminary RLVR experiments show that applying reinforcement learning to heavy thinking trajectories can further improve Heavy-Mean@4 by approximately 10%. However, stability issues were observed with larger trajectory counts ($K=16$) due to sequence length limitations, suggesting a need for careful configuration.

Significance and Claims

The paper claims that HEAVYSKILL offers a promising path toward self-evolving LLMs that internalize complex reasoning without relying on brittle orchestration layers. By abstracting the agentic harness into an inherent "heavy thinking" capability, the authors demonstrate that:

1. Reasoning is an Internal Skill: Complex reasoning can be distilled into a transferable skill that operates beneath any agentic harness.
2. Deliberation is Key: Sequential deliberation is more effective than simple statistical consensus (voting) for synthesizing correct answers from multiple reasoning paths.
3. Scalability: The depth and width of heavy thinking are learnable and scalable via reinforcement learning, offering a mechanism to push LLMs beyond their current reasoning boundaries.

The authors maintain a modest stance, noting that while heavy thinking significantly outperforms traditional strategies, it is not a panacea for all task types (particularly subjective preferences) and that the dynamics of RLVR in this context require further exploration. The work primarily establishes the effectiveness and limitations of heavy thinking as a foundational component for next-generation agentic systems.