Technical Summary: Toward Skill-Native LLMs: Skill Entropy for Benchmarking and Training Long-Horizon Reasoning

1. Problem Statement

Recent Large Language Models (LLMs) have demonstrated strong capabilities in long-horizon reasoning, driving progress in deep research, agentic coding, and multi-step planning. However, real-world long-horizon tasks often require cross-skill reasoning: a single chain of thought must fluidly switch between distinct reasoning skills (e.g., from mathematical derivation to schedule planning, then to information extraction).

Existing benchmarks typically evaluate individual skills in isolation or chain steps within a single domain. They lack a principled mechanism to measure or address the difficulty of switching between different skills within a reasoning chain. Empirical observations suggest that while frontier models perform well on single-skill benchmarks, they exhibit brittleness in compositional, cross-skill tasks. This "skill-switching gap" persists even when the model handles each component skill well in isolation, indicating that the ability to switch skills is an orthogonal capability to domain-specific competence.

2. Methodology

2.1 Skill Entropy (SkE)

The authors introduce Skill Entropy, a directed pairwise measure quantifying the difficulty of switching from one skill $s_a$ to another $s_b$.

• Definition: Skill Entropy is defined as a smoothed ratio between the average single-skill accuracy and the cross-skill accuracy under a fixed reference model.
  $$\text{SkE}(s_a, s_b) = \frac{\frac{1}{2}(\text{Accuracy}(s_a) + \text{Accuracy}(s_b)) + \alpha}{\text{Accuracy}(s_a, s_b) + \alpha}$$
  where $\alpha$ is a Laplace smoothing constant.
• Interpretation: A value $>1$ indicates that chaining the two skills adds significant difficulty compared to handling them in isolation (hard to switch). A value $\le 1$ suggests the switch is easy.
• Task-Level Metric: For a cross-skill task $\tau$ with a skill sequence $(s_1, \dots, s_L)$, the task-level skill entropy is the average of the pairwise entropies along the sequence:
  $$\text{SkE}(\tau) = \frac{1}{L-1} \sum_{i=1}^{L-1} \text{SkE}(s_i, s_{i+1})$$

2.2 Skill2-Bench

Based on the Skill Entropy framework, the authors construct Skill2-Bench, a benchmark for cross-skill long-horizon reasoning.

• Scope: It covers 558 skills across 9 domains: Math, Science, Coding, Logic, Information Extraction, Planning, Creative Writing, Context Retrieval, and Instruction Following.
• Construction: Tasks are synthesized by sampling skill sequences at specific entropy levels (Low, Medium, High) and rewriting seed questions into a coherent scenario where each step depends on the previous output.
• Evaluation Protocol: Models are evaluated in two modes:
  1. Single-Skill Mode: Steps are answered in isolation.
  2. Cross-Skill Mode: The full long-horizon task is presented, requiring sequential switching.

2.3 Skill-Entropy RL

To address the identified gap, the authors propose Skill-Entropy RL, a Reinforcement Learning framework that uses skill entropy as a training signal.

• Output Format: The model is trained to emit a structured response for each step, explicitly predicting both the skill used and the answer:
  <skill> Domain_Skill </skill><answer> Step Answer </answer>
• Reward Function: The total reward $r$ combines step-level correctness ($r_{ans}$) with a skill-entropy reward ($r_{ent}$):
  $$r = \lambda_{ans} r_{ans} + \lambda_{ent} r_{ent}$$
  • $r_{ans}$: Mean per-step accuracy based on domain-specific scorers.
  • $r_{ent}$: Measures the alignment between the predicted skill sequence and the gold skill sequence by comparing their task-level skill entropy ranks. This encourages the model to predict a skill chain that matches the difficulty profile of the ground truth.
• Training Pipeline: The method utilizes a two-stage process: Supervised Fine-Tuning (SFT) on skill-annotated traces followed by Group Relative Policy Optimization (GRPO) using the composite reward.

3. Key Results

3.1 Benchmarking Findings (Skill2-Bench)

• Skill-Switching Gap: Evaluation of 8 frontier and 4 open-source models reveals a consistent performance drop as task-level skill entropy increases. Accuracy decreases nearly monotonically from low to high entropy tasks.
• Cross-Skill Penalty: When the same skills are exercised within a cross-skill task rather than in isolation, accuracy drops by 4% to 13% across models. This penalty persists even for skills where the model is highly proficient in isolation (e.g., Logic).
• Failure Mode Analysis: The dominant failure mode is skill inertia. In later steps of a task, models tend to reuse the skill and answer modality of the previous step rather than switching to the required skill. For instance, a model might continue using a "Math" skill with a numeric answer format when the next step requires "Creative Writing" with a text passage.

3.2 Training Performance (Skill-Entropy RL)

• Significant Improvements: On Qwen3-4B-Instruct, Skill-Entropy RL improved the Skill2-Bench score from 34.4% to 68.4%. On Qwen3-1.7B, the score improved from 14.6% to 40.1%.
• Comparison: The method outperforms competitive baselines, including standard GRPO (without the entropy reward), SFT, and other skill-aware post-training methods (Skill-Distill, SkillRL, STAT).
• Generalization:
  • Open-Ended Domains: Despite training only on verifiable domains, the model showed gains in open-ended domains (Creative Writing, Context Retrieval), suggesting the skill-entropy signal transfers to unseen domains.
  • Off-the-Shelf Data: The pipeline was successfully applied to OpenR1-Math, a dataset not originally structured with explicit skill sequences. By annotating existing traces with skills, the skill-entropy reward continued to improve performance, demonstrating its reusability.

4. Significance and Claims

The paper claims to address a critical gap in evaluating and training LLMs for complex reasoning:

1. New Metric: It introduces Skill Entropy as a principled, directed pairwise measure to quantify the difficulty of skill switching, moving beyond single-domain evaluation.
2. Benchmark: Skill2-Bench provides the first large-scale benchmark (558 skills, 9 domains) calibrated by this metric, exposing a structural "skill-switching gap" that single-domain evaluations miss.
3. Training Signal: It demonstrates that skill entropy is not just a diagnostic tool but a reusable training signal. By incorporating it into the RL reward function, models learn to explicitly manage skill transitions, significantly improving long-horizon reasoning capabilities.
4. Failure Mode Identification: The work concretely identifies "skill inertia" (reusing previous step's skill/modality) as a primary cause of failure in cross-skill tasks and provides a mechanism to mitigate it.

The authors conclude that handling skill switches is an orthogonal capability to domain competence and that "Skill-Native" LLMs—models trained to explicitly predict and manage their own skill sequences—are a viable path toward robust long-horizon reasoning.