🤖 AI

SkillGrad: Optimizing Agent Skills Like Gradient Descent

SkillGrad is a novel framework that optimizes agent skills by treating them as structured parameters in a gradient-descent-like process, utilizing trajectory-level loss evidence, text-based diagnostic gradients, and a momentum agent to iteratively refine skills, thereby outperforming existing training-based baselines on benchmark tasks.

Original authors: Hanyu Wang, Yifan Lan, Bochuan Cao, Lu Lin, Jinghui Chen

Published 2026-05-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Hanyu Wang, Yifan Lan, Bochuan Cao, Lu Lin, Jinghui Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: Teaching a Robot by Editing its "Playbook"

Imagine you have a very smart robot assistant (an AI Agent) that can do complex tasks, like organizing a spreadsheet or navigating a website. To make this robot good at a specific job, you give it a Skill Package. Think of this package as a physical Playbook or a User Manual that the robot reads before it starts working.

The Problem:
Sometimes, these Playbooks are messy. They might be written by a novice, downloaded from the internet with errors, or just missing the specific tricks needed for a tricky situation. If the robot follows a bad Playbook, it fails.

Existing methods try to fix this by asking the robot, "What went wrong?" and then rewriting the Playbook based on that single mistake. But this is like trying to fix a car engine by only looking at the one time it stalled, without remembering that it also stalled three times last week. It's reactive and often misses the bigger picture.

The Solution: SkillGrad
The authors propose SkillGrad, a new way to improve these Playbooks. They treat the Playbook not just as a document, but as a living set of instructions that can be "trained" using a method inspired by how math problems are solved (specifically, something called Gradient Descent).

Here is how SkillGrad works, broken down into four simple steps:

1. The "Test Drive" (Loss Evidence)

Instead of just looking at one mistake, the robot tries to solve a whole batch of tasks (like 4 different spreadsheet problems) using its current Playbook.

If it fails: The system notes exactly how it failed.
If it succeeds: The system looks at how it succeeded, especially if it succeeded on a task it previously failed. This is a crucial trick: it learns what new behaviors are working, not just what to stop doing.

2. The "Coach's Diagnosis" (Gradients)

In math, a "gradient" is a direction arrow telling you which way to move to get a better result. Since a Playbook is made of words, not numbers, SkillGrad uses an AI "Coach" to write a text-based diagnosis.

The Coach reads the test results and writes a note: "The robot failed because it didn't check the data first. It needs to add a 'Check Data' step."
This note is the "gradient"—it points the direction for improvement.

3. The "Memory Bank" (Momentum)

This is the secret sauce. In many systems, if a robot makes a mistake today, it gets fixed. But if it makes the same mistake next week, the system might forget it happened.
SkillGrad has a Memory Agent (like a coach with a clipboard).

If the robot makes the same mistake three times in a row, the Coach doesn't just fix it once; they write it down in a Persistent Memory.
This ensures that recurring patterns aren't ignored. It's like a coach saying, "We've talked about this three times. We need to make a permanent rule about it, not just a quick fix."

4. The "Editor" (The Patch)

Finally, a "Patcher" agent takes all the diagnoses and the memory of recurring patterns and edits the Playbook.

It doesn't just throw new text at the bottom of the page. It is smart about where to put things.
General Rules (like "Always check the data first") go into the main chapter that is always read.
Specific Tricks (like "How to handle a weird formula error") go into a separate "Reference" section that is only opened when needed.
This keeps the Playbook organized and prevents it from becoming a messy, unreadable wall of text.

The Results: Does It Work?

The authors tested this on SpreadsheetBench (a benchmark for Excel tasks) and WikiTableQuestions (a database question-answering task).

The Setup: They started with Playbooks that were either generated by an AI (and often imperfect) or downloaded from third parties.
The Outcome: SkillGrad consistently made the robots smarter.
- On average, it improved the robots' success rate by 6.7% compared to other methods that try to learn skills.
- It worked even when the starting Playbook was terrible, turning a failing robot into a successful one.
- It also worked on tasks it hadn't seen before (Out-of-Domain), proving the robot learned general rules, not just memorized answers.

Why This Matters (In Simple Terms)

Think of SkillGrad as the difference between telling a student to "do better" versus giving them a structured, evolving textbook.

Old Way: "You got this math problem wrong. Here is the answer. Try again." (The student might forget the lesson next time).
SkillGrad Way: "You got this wrong because you skipped a step. You also got the last three wrong for the same reason. Let's update your textbook to have a bold warning about skipping steps, and let's add a specific example for this type of problem in the appendix."

By treating the "Skill" as something that can be systematically optimized—just like training a neural network, but using text and logic instead of math—SkillGrad creates more reliable, reusable, and intelligent agents.

Technical Summary: SkillGrad

Problem Statement

Large Language Model (LLM) agents have demonstrated proficiency in long-horizon decision-making tasks such as reasoning, planning, and web navigation. However, adapting these agents to specialized, procedure-heavy domains (e.g., spreadsheet manipulation, document editing, codebase maintenance) remains challenging. Traditional adaptation methods like fine-tuning or retrieval pipelines can be costly or cumbersome, particularly when the required knowledge is procedural rather than factual.

While "Agent Skills"—structured file packages containing metadata, procedural workflows, and resources—offer a lightweight alternative for domain adaptation, their effectiveness is often compromised by quality issues. Automatically generated or third-party downloaded skills are frequently unreliable, incomplete, or outdated. Existing methods for skill evolution typically rely on heuristic reflections without an explicit optimization formulation, often addressing deficiencies only through failure analysis and lacking a systematic approach to preserve successful behaviors or stabilize updates across iterations.

Methodology: SkillGrad

The authors propose SkillGrad, a framework inspired by gradient descent to systematically optimize agent skills. Rather than treating skills as static prompts, SkillGrad treats the structured skill package as an optimizable parameter. The framework draws a conceptual, not numeric, correspondence between gradient descent components and skill optimization modules (Table 1).

1. Parameterization

The trainable parameter is the structured skill package $S_t = (H, B, Q)$ , consisting of three layers:

L1 (Metadata): YAML description for activation logic.
L2 (SKILL.md): The always-loaded body containing principles, workflows, and code examples.
L3 (Resources): Conditionally loaded files for edge cases and detailed procedures.
This structure necessitates "layer-aware" updates, where the system must decide not only what knowledge to add but where (L2 vs. L3) to place it to avoid context bloat or retrieval failures.

2. Loss Evidence

Instead of a scalar loss, SkillGrad utilizes trajectory-level evidence.

Failed Executions: Provide negative evidence ( $\tau^-_t$ ) indicating missing or incorrect guidance.
Contrastive Successful Executions: When a task that initially failed with the base skill is solved by the current skill, the system pairs the current success ( $\tau^+_t$ ) with the initial failure ( $\tau^-_0$ ). This contrastive evidence identifies behaviors worth preserving that distinguish success from failure.

3. Gradient Signals (Diagnosis)

Since skills are discrete text artifacts, numeric derivatives are unavailable. SkillGrad employs a Diagnoser agent to convert trajectory evidence into textual update signals ( $d_{t,i}$ ).

For failures, the diagnosis explains the deviation from ground truth and suggests general behavioral corrections.
For contrastive successes, it identifies the specific changes (e.g., robust coding strategies, verification steps) that enabled success.
These diagnoses serve as the textual equivalent of per-example gradients.

4. Momentum

To stabilize optimization and prevent "update churn," a Momentum Agent accumulates recurring diagnostic patterns into a persistent memory ( $M_t$ ) and a current overlay ( $O_t$ ).

It tracks whether a diagnosed mechanism is new, recurring, already covered, or unresolved.
This allows the system to reinforce directions that recur across iterations and carry successful contrastive behaviors forward, rather than chasing isolated failures.

5. Parameter Updates (Patching)

A Patcher agent executes the update by applying layer-aware edits to the skill package ( $S_{t+1} = \text{Patch}(S_t, D_t, M_t, O_t)$ ).

The patcher operates on patterns rather than individual tasks, generalizing multiple diagnoses into a single edit.
It decides whether to update the always-loaded L2 body or create/modify conditional L3 resources, ensuring the skill remains organized and retrievable.

Key Contributions

Formulation: The paper formulates agent skill improvement as an optimization problem over a structured artifact, explicitly mapping the concepts of parameter, loss evidence, gradient, momentum, and update to the skill domain.
Framework: It introduces SkillGrad, a multi-agent system that diagnoses executions, accumulates recurring patterns via textual momentum, and applies layer-aware patches.
Empirical Validation: Experiments demonstrate that SkillGrad improves agent performance on spreadsheet tasks using both LLM-generated and third-party initial skills, outperforming training-based baselines in both in-domain and out-of-domain settings.

Experimental Results

The framework was evaluated on SpreadsheetBench Verified and WikiTableQuestions using two backbone LLMs (GPT-5.4 and GPT-4.1).

Performance Gains: SkillGrad consistently outperformed training-free settings and training-based baselines (EvoSkill and Trace2Skill).
- On SpreadsheetBench with GPT-5.4, SkillGrad achieved 71.11% accuracy, improving over the average of the two training-based baselines by 4.44 percentage points.
- With GPT-4.1, the improvement was more pronounced, reaching 54.17% compared to 37.22% for the baselines.
Robustness: The framework successfully optimized both LLM-generated skills (which initially degraded performance compared to "no skill") and third-party skills.
Generalization: On the out-of-domain WikiTableQuestions benchmark, SkillGrad achieved the highest accuracy across all initialization blocks, suggesting the optimized skills contain reusable procedural guidance beyond the specific training tasks.
Ablation Studies: Removing the momentum mechanism reduced accuracy by 6.67 percentage points, and removing contrastive diagnosis reduced it by 4.17 percentage points, confirming the necessity of both components for high-quality skill evolution.

Significance and Limitations

The paper claims that SkillGrad provides a principled, optimization-inspired path for improving agent skills without requiring model fine-tuning or extensive validation sweeps. By treating skills as optimizable artifacts, the framework addresses the brittleness of static, automatically generated skills.

The authors acknowledge limitations, noting that the evaluation is primarily focused on spreadsheet tasks, with WikiTableQuestions serving as the sole out-of-domain transfer setting. They suggest future work could test the framework on other domains like web automation and codebase maintenance. Furthermore, the current analysis is empirical and qualitative; a more formal account of when textual diagnoses yield stable updates remains an open direction for strengthening the connection between agent skill optimization and classical optimization theory.