Technical Summary: MIND-Skill

Problem Statement

Large Language Model (LLM) powered AI agents have demonstrated success in autonomous problem-solving but struggle with complex, multi-step real-world tasks requiring domain-specific procedural knowledge (e.g., API usage, multi-step tool calls). While reusable "agent skills"—structured documents encapsulating successful strategies—offer a remedy, curating them remains a largely manual, expert-driven process. Existing automatic skill generation methods suffer from three critical limitations:

1. Lack of Quality Guarantees: Many methods generate skills directly from task specifications or trajectories without a closed-loop validation pipeline to correct errors based on execution outcomes.
2. Poor Documentation Quality: Generated skills often fail to adhere to technical writing standards (logical flow, troubleshooting guidance), reducing their portability and reusability.
3. Unverified Abstraction Faithfulness: Trajectory-distillation methods often over-generalize or fail to preserve essential edge-case handling and prerequisite checks, leading to skills that do not faithfully reconstruct the source behavior.

Methodology: MIND-Skill

The authors propose MIND-Skill (Multi-agent INduction and Deduction for Skills), a framework that automatically synthesizes generalizable skills with robust quality guarantees. The framework operates via a closed-loop induction-deduction process optimized using TextGrad.

Core Architecture

1. Induction Agent ($A_I$): Tasked with abstracting a successful trajectory ($\tau$) and task specification ($t$) into a structured skill document ($s$). The agent's system prompt ($P_I$) is the sole variable optimized during the process.
2. Deduction Agent ($A_D$): A "frozen" agent that attempts to reconstruct the original trajectory ($\tau$) solely using the task specification ($t$) and the induced skill ($s$). It receives no access to the source trajectory or external guidance. This ensures that any reconstruction failure is directly attributable to deficiencies in the skill itself.

Optimization via Textual Losses

The framework jointly optimizes the induction prompt ($P_I$) using three complementary textual loss functions, evaluated by LLM judges and environment execution:

• Reconstruction Loss ($L_{recon}$): Measures the procedural alignment between the source trajectory ($\tau$) and the reconstructed trajectory ($\hat{\tau}$). An LLM judge evaluates tactic-level equivalence (e.g., same retrieval-aggregation pattern) rather than literal step-by-step similarity.
• Outcome Loss ($L_{outcome}$): Enforces execution correctness by running the reconstructed trajectory in a live environment. It provides a ground-truth signal based on whether the task goal is achieved.
• Rubric Loss ($L_{rubric}$): Assesses the documentation quality and regularizes the abstraction level. It penalizes "ground-truth leakage" (memorizing specific API paths or entity IDs) and ensures the skill is actionable, transferable, complete, and concise.

Optimization Procedure

The system employs lexicographic minimization where Outcome Loss is the primary objective, with Reconstruction and Rubric losses serving as tiebreakers. Using TextGrad, a gradient LLM analyzes the textual feedback from all three losses to diagnose failure patterns and propose revisions to the induction prompt ($P_I$). An optimizer LLM then updates the prompt. This iterative process continues for a fixed number of steps, retaining the best skill found so far.

Key Contributions

1. Novel Framework: Introduction of MIND-Skill, a multi-agent induction and deduction framework that synthesizes generalizable skills from successful trajectories without human intervention.
2. Quality Assurance Mechanism: Proposal of a closed-loop optimization strategy using three textual losses (Reconstruction, Outcome, Rubric) to ensure skills are faithful, correct, and well-documented. Crucially, the deduction agent is kept frozen to isolate skill quality as the sole variable.
3. Empirical Validation: Demonstration that MIND-Skill improves agent performance on both source tasks and held-out tasks unseen during generation, outperforming concurrent state-of-the-art methods.

Experimental Results

The authors evaluated MIND-Skill on two benchmarks: AppWorld (interactive coding agent benchmark) and BFCL-v3 (multi-turn function-calling benchmark).

• Performance: MIND-Skill consistently outperformed baselines, including ReAct, In-Context Learning (ICL), Skill-extract (one-shot induction), ACE (lifelong evolving), and Trace2Skill (trajectory distillation).
  • On AppWorld-Challenge, MIND-Skill achieved a Task Goal Completion (TGC) of 51.8% and Scenario Goal Completion (SGC) of 39.6%, significantly surpassing the next best method (ACE at 51.1% TGC and 34.5% SGC).
  • On BFCL-v3, MIND-Skill achieved the highest accuracy (77.3% with Qwen3.5 base, 78.7% with GPT-5.4 base).
• Ablation Studies:
  • Removing any of the three loss functions resulted in performance drops, confirming their indispensability. The Reconstruction loss was most critical for Challenge tasks, while the Rubric loss was vital for preventing overfitting on Normal tasks.
  • The closed-loop optimization significantly outperformed one-shot skill extraction (Skill-extract), highlighting the value of iterative refinement.
• Model Agnosticism: MIND-Skill enabled a weaker base model (Qwen3.5-122B) to achieve performance comparable to a stronger model (GPT-5.4) when used as the skill generator, suggesting the framework effectively compensates for capability gaps through alignment and optimization.
• Efficiency: MIND-Skill generates compact skills, injecting 3–6 times fewer tokens at inference time compared to monolithic playbooks from baselines like ACE.

Significance

The paper claims that MIND-Skill addresses the critical gap in automated skill generation by providing quality guarantees through a principled, closed-loop pipeline. Unlike prior methods that rely on post-hoc error diagnosis or unverified abstraction, MIND-Skill explicitly validates skills by attempting to reconstruct the source behavior. The framework demonstrates that high-quality, generalizable procedural knowledge can be automatically distilled from successful trajectories, reducing the reliance on manual curation and enabling agents to perform complex, long-horizon tasks more effectively. The results suggest that the induction-deduction paradigm, coupled with textual loss optimization, is a robust approach for scaling agent capabilities.