Technical Summary: Rethinking Self-Evolution with SkillBoost

Problem Statement

The paper addresses a critical challenge in deploying Large Language Model (LLM) agents for real-world, sequential tasks: enabling them to accumulate and reuse experience without modifying their underlying parameters. While "skill self-evolution"—treating skills as trainable external states—offers a lightweight alternative to full model fine-tuning, existing data-driven optimization methods suffer from skill overfitting.

Current approaches typically minimize loss on the current batch of interaction trajectories. Because these batches are small, the learned skills tend to fit specific, narrow patterns (case-specific biases) rather than generalizable procedures. This leads to a large generalization gap where performance improves on observed cases but degrades on held-out tasks or previously solved cases. The authors argue that skill self-evolution is not merely a data-fitting problem but a constrained search problem governed by an exploration–exploitation trade-off. Unconstrained exploitation leads to overfitting, while unconstrained exploration risks breaking stable behaviors (regression).

Methodology: SkillBoost

The authors propose SkillBoost, a three-stage framework that formalizes skill self-evolution as a Markov Decision Process (MDP) to balance failure-driven exploitation with prior-guided exploration. The process iterates through the following stages:

1. Structured Exploitation (Attribution)

Instead of updating the entire skill text based on raw failure data, SkillBoost localizes failures to specific, editable skill components (e.g., Metadata, Workflow, Constraints, References).

• Process: The system analyzes execution trajectories of failed cases to perform a fine-grained diagnosis.
• Mechanisms:
  • Workflow Compliance: Checks if steps follow the defined workflow, identifying skipped steps (weak constraints) or unsupported conclusions (lack of evidence).
  • Reasoning Chain Reconstruction: Traces execution backward to find the first deviation from the intended rule.
  • Root Cause Clustering: Groups failures by defect type (e.g., strategy defects vs. capability gaps) to ensure a single edit can repair a whole cluster of errors.
• Output: A diagnosis $g_t$ mapping trajectory-level failures to specific skill components, confining subsequent edits to these targeted dimensions.

2. Prior-Guided Exploration (Candidate Generation)

Once the failure targets are identified, the system generates diverse repair candidates.

• Process: The diagnosis $g_t$ remains fixed, but the system employs $N$ different repair strategies ($\pi^{(n)}$).
• Mechanism: These strategies vary in edit scope (narrow vs. broad) and priority (which failure cluster to fix first). They leverage the LLM's prior knowledge to propose diverse edits (additions, deletions, replacements) rather than relying solely on batch-specific patterns.
• Output: A "Best-of-N" set of candidate skills, each representing a different hypothesis for repairing the diagnosed defects.

3. Verified Acceptance (Constrained Commitment)

The framework commits to a new skill version only if it satisfies strict criteria to prevent overfitting and regression.

• Process: Candidates are evaluated on the full task set (not just the failure set).
• Criteria: A candidate $s'$ is accepted only if:
  1. Net Improvement: It improves the overall performance score ($r(s') > 0$).
  2. Regression Bound: The number of previously correct cases broken by the edit ($Regress(s')$) is below a threshold $\epsilon$.
• Selection: The candidate with the highest gain satisfying these constraints is selected. If no candidate passes, the skill remains unchanged.

Key Contributions

1. Reformulation of Self-Evolution: The paper identifies skill overfitting as a central failure mode and recasts self-evolution as a heuristic search problem governed by an exploration–exploitation trade-off, rather than simple trajectory fitting.
2. SkillBoost Framework: A novel framework that mitigates overfitting through:
   • Structured Exploitation: Localizing failures to specific components to avoid global over-correction.
   • Prior-Guided Exploration: Generating diverse candidates using LLM priors to ensure generalizability.
   • Verified Acceptance: A gating mechanism that rejects edits causing excessive regression on previously solved tasks.
3. Empirical Validation: Extensive experiments across 23 model–benchmark configurations (including SpreadsheetBench, BFCL-v4, LiveMath, ALFWorld, and DocVQA) demonstrate that SkillBoost achieves state-of-the-art performance.
4. Transferability: The learned skills are shown to transfer effectively across different agents and related benchmarks, suggesting the discovery of reusable procedural disciplines rather than memorized patterns.

Experimental Results

• Performance Gains: SkillBoost outperforms both human-crafted skills and LLM-generated baselines (including Trace2Skill and SkillOpt) across all tested configurations. For instance, on the LiveMath benchmark with Claude-opus-4-6, it achieved a 47.4% absolute gain over the no-skill baseline, significantly outperforming other methods.
• Overfitting Mitigation: Unlike baselines that show large negative generalization gaps (Test − Train), SkillBoost maintains a near-zero or positive gap, indicating that improvements on the training set transfer to held-out test sets.
• Efficiency: The method reduces inference tokens per case by an average of 13.9% compared to SkillOpt, particularly in sequential decision-making tasks where it reduces redundant exploration.
• Ablation Studies: Removing any of the three core components (Structured Exploitation, Prior-Guided Exploration, or Verified Acceptance) consistently degrades performance, confirming the necessity of the full pipeline.
• Transfer Experiments: Skills optimized on one model (e.g., Claude) successfully improved performance on other models (e.g., DeepSeek, Kimi) and related benchmarks (e.g., transferring LiveMath skills to OlympiadBench).

Significance and Claims

The paper claims that SkillBoost provides a robust solution to the "skill overfitting" problem, which has hindered the practical deployment of self-evolving agents. By treating skill optimization as a constrained search rather than a data-fitting exercise, the framework enables agents to:

• Iteratively refine reusable competencies without modifying base model parameters.
• Discover procedural disciplines (e.g., exact function naming, bidirectional verification, static-value discipline) that bridge the gap between a model's latent capabilities and its zero-shot application consistency.
• Maintain stability by ensuring that new skills fix current failures without regressing on previously mastered tasks.

The authors emphasize that the learned rules are not new domain knowledge but rather systematic application protocols that frontier models possess but fail to deploy reliably without explicit, structured instruction. The work suggests that effective self-evolution requires a balance between exploiting observed failures and exploring diverse, prior-guided solutions, validated by rigorous acceptance criteria.