Technical Summary: MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluation

1. Problem Statement

Large Language Model (LLM) agents are increasingly tasked with solving complex, multi-step real-world problems involving external tools, code, and data. While raw model reasoning is insufficient for such tasks, existing approaches to equipping agents with "skills" (reusable units of capability) suffer from four critical limitations:

1. Creation–Usage Mismatch: Skills are often generated in isolation without access to the agent's runtime context.
2. Lack of Structured Memory: There is no mechanism to accumulate free-form experience or lessons learned about specific skills across different tasks.
3. Static and Unvalidated: Skills are typically treated as static artifacts without unit-test-driven evaluation or automatic refinement loops.
4. Poor Context Handling: Flat conversation histories often truncate or overflow during long-horizon tasks, preventing effective reuse of past knowledge.

Current methods often treat skills as one-off generation outputs rather than long-lived, evolving assets, limiting their reusability, reliability, and ability to improve over time.

2. Methodology: The MUSE-Autoskill Framework

The authors propose MUSE-Autoskill (Memory-Utilizing Skill Evolution), a skill-centric agent framework that treats skills as managed, testable, and transferable infrastructure. The framework operates within a unified skill lifecycle consisting of five stages: Creation, Memory, Management, Evaluation, and Refinement.

Core Architecture

The agent operates in an iterative ReAct loop (Planning, Action, Observation) enhanced by a dedicated skill subsystem:

• Skill Creation: Instead of pre-defining all capabilities, the agent uses a built-in skill_create tool. When existing skills are insufficient, the agent specifies the intent, and the system generates a structured skill package (containing SKILL.md, optional scripts/, resources/, and tests/).
• Skill Execution: Skills are executed in isolated sandboxes. The agent reads the SKILL.md interface to determine whether to run code, load resources, or both. Execution is mediated by sandbox lifecycle tools (create_sandbox, sandbox_run, etc.), ensuring side effects are contained.
• Skill Evaluation: Before a new skill is registered, it undergoes unit testing within a sandbox. If tests fail, the agent inspects the error trace and invokes an update_skill tool to patch the package, creating a create → evaluate → register loop that ensures only reliable skills enter the bank.
• Skill Management: The system maintains a skill bank indexed by metadata. It supports merging overlapping skills, pruning consistently failing or unused skills, and refining skills based on runtime feedback.

Memory Mechanisms

MUSE introduces a multi-level memory architecture to support long-term accumulation:

1. Skill-Level Memory (Novel): Each skill carries a .memory.md file where the agent appends notes, failure modes, and usage observations specific to that skill. This allows the agent to recall "lessons learned" about a specific tool without re-deriving them.
2. Short-Term Memory: Maintains the current task context (reasoning steps, observations). It employs adaptive context compression (Level-1 for oversized nodes, Level-2 for span merging) to handle long-horizon tasks without exceeding token budgets, while preserving the full history in a DAG structure for replayability.
3. Long-Term Memory: Stores persistent, cross-session notes (e.g., environment quirks, general best practices) that are not subject to compression.

Context Management

To prevent context-window blowup, MUSE uses a DAG of conversation nodes. It applies two-level adaptive compression:

• Level-1: Summarizes individual oversized turns (e.g., verbose tool outputs) in place.
• Level-2: Merges a contiguous range of intermediate nodes into a single synthetic summary node if Level-1 is insufficient.
  Crucially, original nodes remain in the immutable history, allowing the agent to resume from intermediate states or replay trajectories without losing information.

3. Key Contributions

The paper outlines four primary contributions:

1. Skill Lifecycle Reframing: The authors reframe skills from disposable generation outputs to long-lived, lifecycle-managed assets, identifying five necessary stages (creation, memory, management, evaluation, refinement) for practical skill-centric systems.
2. MUSE-Autoskill Agent: A framework that integrates skill creation with runtime execution, evaluates skills via unit tests, and automatically refines them upon failure.
3. Infrastructure Innovations:
   • Skill-Level Memory: A novel mechanism accumulating per-skill experience across tasks.
   • Adaptive Context Compression: Supports long-horizon tasks with cross-session state persistence.
   • Cross-Agent Transfer: Demonstrates that generated skills are externalized knowledge assets usable by different agent architectures without modification.
4. Validation: Comprehensive evaluation on SkillsBench (51 real-world tasks) showing that lifecycle-managed skills improve success rates, efficiency, and transferability.

4. Experimental Results

Experiments were conducted on SkillsBench, a benchmark of 51 tasks across four super-domains (Science & Engineering, Data Analysis, Document Processing, Ops & Planning), using GPT-5.5 as the backbone for three agents: MUSE-Autoskill, Codex, and Hermes.

Performance with Human Skills

• MUSE-Autoskill achieved the highest accuracy with human-injected skills (68.40%), outperforming Codex (67.28%) and Hermes (61.21%).
• This represents a +15.21 percentage point (pp) lift over its no-skill baseline, consistent across agents.
• MUSE-Autoskill led in 3 of 4 domains, demonstrating superior ability to read, interpret, and apply skill content within its reasoning loop.

Automatic Skill Generation

• MUSE-Autoskill successfully generated skills for 35 out of 51 tasks (68.6%) based on its own successful trajectories.
• On these 35 tasks, the agent's accuracy with self-generated skills reached 87.94%, surpassing the human-skill ceiling (68.40%).
• The overall accuracy across all 51 tasks with self-generated skills was 60.35%. The authors note the bottleneck is Phase 1 coverage (the agent's ability to solve tasks without skills to generate a trajectory) rather than the quality of the generated skills.

Cross-Agent Transfer

• Generated skills were injected into Hermes without modification.
• Hermes's accuracy improved from 47.89% (no skills) to 58.40% with MUSE-generated skills, closing 79% of the gap to Hermes with human skills.
• This confirms that MUSE produces externalized knowledge assets rather than agent-specific behaviors tied to a single runtime.

Efficiency and Cost

• Pareto Optimality: On the 35 tasks with generated skills, MUSE-generated skills offered higher reward, lower latency, and fewer tokens compared to both no-skill and human-skill baselines.
• Latency: MUSE-Autoskill reduced median latency by 273 seconds (from 684s to 411s) when using generated skills compared to no skills.
• Tokens: Generated skills reduced median token usage by 85K tokens compared to the no-skill baseline.
• Cost-Benefit: Although generating a skill costs ~383K tokens, the savings per reuse (amortized over ~3 reuses) and the immediate latency reduction make the approach cost-effective.

5. Significance and Claims

The paper claims that treating skills as long-lived, experience-aware, and testable assets is essential for scalable agent systems.

• Self-Evolution: MUSE demonstrates that agents can continuously improve their capabilities by creating, refining, and reusing skills without human authoring at every step.
• Transferability: The framework breaks the silo of agent-specific behavior, allowing skills to be shared across different agent architectures (e.g., MUSE to Hermes), effectively creating a "common unit of capability."
• Production Readiness: The authors note that the skill-centric design is already being adopted in production systems (SkillMarket, ArkClaw, SkillHub) to manage skill creation, discovery, and lifecycle governance.

The authors remain modest about limitations, acknowledging that skill generation currently succeeds on only ~68% of tasks (limited by Phase 1 coverage), that cross-agent transfer is currently validated primarily in one direction (MUSE to Hermes), and that the evaluation is restricted to a subset of the full SkillsBench benchmark. They identify future work in extracting diagnostic skills from failed trajectories and validating the framework on a broader range of models and independent benchmarks.