💻 computer science

SkillMentor: LLM Agent Self-Evolution via Learning Blind-Spot Diagnosis

SkillMentor is a novel framework that enables LLM agents to self-evolve by learning a separate diagnostic capability to identify and curate blind spots into reusable skills, thereby improving executor performance by 44.2% without updating the executor's weights or relying on human supervision.

Original authors: Xiaoyi Bao, Yuanzhen Xie, Yunzhi Tan, Jinghang Gu, Zhongqing Wang, Chu-Ren Huang, Bo Hu, Zang Li

Published 2026-07-31

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Xiaoyi Bao, Yuanzhen Xie, Yunzhi Tan, Jinghang Gu, Zhongqing Wang, Chu-Ren Huang, Bo Hu, Zang Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where computers are like incredibly fast, super-smart apprentices. For a long time, scientists have been trying to teach these apprentices how to do things better by constantly rewriting their brains—adding new layers of neurons, tweaking their internal wiring, and feeding them endless examples of how to act. This is the world of "agent self-evolution," where the goal is usually to make the apprentice faster and stronger at doing the task. But there's a catch: what if the apprentice doesn't just need to know how to act, but needs to realize what it doesn't know? It's like a student who keeps failing a math test because they don't know they forgot to carry the one, rather than because they can't do the multiplication. The big question this paper asks is: Can we teach an apprentice to become a detective for its own mistakes, spotting the hidden gaps in its knowledge without needing a human teacher to point them out?

Enter SkillMentor, a clever new system that flips the script on how we usually train AI. Instead of trying to upgrade the apprentice's brain directly, the researchers created a tiny, specialized "Mentor" agent. Think of the main AI (the "Executor") as a robot trying to navigate a giant, complex video game world full of apps and tools. The Executor is frozen in place; its brain cannot be changed or updated. Usually, if the robot fails, a human would step in, say, "Hey, you forgot to log in," and fix the code. But in this experiment, humans are banned from the room. No labels, no hints, no help.

So, how does the robot get better? Enter the Mentor. This little detective doesn't try to play the game itself. Instead, it watches the Executor struggle, then says, "Aha! I see what's wrong." The Mentor's job is to figure out why the robot failed (diagnosis) and then write a tiny, reusable "cheat sheet" (a skill) to help the robot avoid that specific mistake next time. It's like the Mentor noticing the robot keeps tripping over a rug, so it writes a note saying, "Watch out for the rug!" and sticks it on the robot's wall. The robot doesn't get a new brain; it just gets a better set of instructions based on what it learned from its own failures.

The researchers tested this setup in two challenging digital worlds: AppWorld, where the robot has to manage complex tasks across different apps (like booking a flight or organizing files), and BFCLv3, a test of how well the robot can call specific computer functions correctly. They pitted their new SkillMentor against other methods that rely on huge, powerful models just guessing the right answer or using fixed rules. The results were surprising. Even though the Mentor was a much smaller, simpler model than the giant ones used by other methods, it helped the frozen Executor improve its performance by an average of 44.2%.

Here is the magic trick: The Mentor didn't just guess; it learned how to diagnose. It figured out that the best way to get better wasn't to try harder, but to find the "blind spots"—the specific, recurring things the robot didn't know how to do. The paper suggests that this ability to diagnose is a skill in itself, one that can be trained. The Mentor learned to generate tricky test cases to expose the robot's weaknesses, evaluate how bad those weaknesses were, and then curate the perfect "fix" to add to the robot's toolkit.

One of the coolest findings is that this system creates a positive feedback loop. As the Mentor fixes one blind spot, the robot gets better, which means the old tricks don't work anymore, and new blind spots appear. The Mentor then spots these new gaps and fixes those too. It's a self-improving cycle where the robot gets smarter not by changing its brain, but by constantly updating its external "cheat sheets" based on what it learns from its own mistakes.

The paper also shows that these "cheat sheets" are portable. If you train the Mentor on a weaker robot, the skills it learns can actually help a stronger robot, because the weaker one exposes more mistakes to learn from. Conversely, skills learned from a super-smart robot don't help a weaker one as much because the smart robot didn't make enough mistakes to teach the basics. This proves that the knowledge gained is about the process of fixing errors, not just memorizing answers.

In short, SkillMentor suggests that we don't always need to make AI bigger or more expensive to make it smarter. We might just need to teach it how to be a better detective of its own blind spots. By separating the act of "doing" from the act of "figuring out what went wrong," the researchers found a way to let AI evolve on its own, discovering its own limitations and building its own solutions, all without a single human hand guiding the process. It's a step toward machines that don't just follow orders, but learn to understand where they fall short and how to fix it.

Technical Summary: SkillMentor

Problem Statement

Current approaches to agent self-evolution predominantly focus on execution: learning how to act by updating executor parameters or refining external knowledge bases. These methods largely overlook diagnosis: the capability to discover an agent's "blind spots" (recurrent failure modes) without external supervision. Existing diagnostic procedures are typically engineered, fixed, or rely on human-labeled data to specify what needs repair. Consequently, the ability to autonomously identify what an agent does not know remains an unlearned capability.

The authors posit that diagnosis is conceptually distinct from execution. To study this in isolation, they propose a setting where:

The Executor (the agent performing tasks) is frozen; its weights are never updated.
No human-labeled data is available; the system cannot rely on annotated examples to specify failures.
All performance improvements must originate solely from a learned diagnostic capability.

Methodology: SkillMentor

The paper proposes SkillMentor, a framework that trains a small Mentor policy ( $\pi_\theta$ ) via Reinforcement Learning (RL) to diagnose and repair a frozen Executor ( $\pi_E$ ). The system operates through a self-evolving feedback loop consisting of three stages:

1. Blind-Spot Discovery

The Mentor interacts with the environment to generate diagnostic tasks.

Exploration: The Mentor explores the environment to synthesize candidate tasks that challenge the Executor.
Diversity: A diversity buffer prevents the exploration from collapsing into narrow behavior patterns.
Gap Evaluation: For each candidate task, a strong reference model ( $M$ ) generates a high-quality trajectory and score ( $s^M$ ). The frozen Executor attempts the same task using its current skill repository, yielding a score ( $s^E$ ).
Reward: The discovery reward ( $r_{disc}$ ) is defined as the diagnostic gap: $\delta = \max(0, s^M - s^E)$ . Tasks with larger gaps are prioritized for curation.

2. Skill Curation

The Mentor transforms identified blind spots into reusable corrective skills stored in an external repository.

Operations: The Mentor can ADD new skills, UPDATE existing ones, or MERGE them.
Validation: A candidate skill is only accepted if it is syntactically valid and, crucially, improves the Executor's performance on the diagnosed task by a margin $\Delta > 0.5$ .
Reward: The curation reward ( $r_{curate}$ ) combines format compliance and the performance improvement ( $\Delta$ ).
Maintenance: A lightweight DELETE mechanism evicts skills with low success rates (< 0.05) to prevent repository bloat.

3. Joint Optimization

The Mentor is trained using GRPO (Group Relative Policy Optimization) to maximize a combined objective:
$J_{GRPO}(\theta) = J_{disc} + J_{curate} - \beta D_{KL}(\pi_\theta \parallel \pi_{ref})$
The paper argues that discovery and curation form a positive feedback loop: discovering blind spots leads to better skills, while better skills shift the Executor's capability boundary, exposing new, previously hidden blind spots. Joint optimization is essential to maintain this loop.

Key Contributions

Diagnosis as a Learnable Capability: The paper demonstrates that blind-spot diagnosis can be learned as an adaptive policy rather than being a fixed, engineered procedure or a byproduct of model scale.
Self-Evolution without Weight Updates: SkillMentor achieves significant performance gains on a frozen Executor without updating its parameters, relying entirely on an external, evolving skill repository.
Zero-Human-Data Constraint: The framework operates without human supervision or labeled data, forcing the system to autonomously discover and repair recurrent failures.
Interpretability: Unlike methods that internalize knowledge into weights, SkillMentor stores skills as human-readable Markdown files, providing an interpretable map of the Executor's deficiencies.

Experimental Results

The authors evaluated SkillMentor on AppWorld (long-horizon planning) and BFCLv3 (precise function calling) using three different frozen Executors (Qwen3.5-9B, Qwen3.5-4B, and DeepSeek-R1-Distill-Qwen-7B).

Performance Gains: SkillMentor achieved an average relative improvement of 44.2% over the "No Skill" baseline across all executors and datasets.
Comparison with Strong Models: The RL-trained 4B Mentor consistently outperformed significantly larger, prompt-based models (e.g., DeepSeek-V4-Flash) used as static mentors, suggesting that targeted RL training is more effective than mere model scaling for diagnosis.
Transferability: Skills learned from weaker Executors (which expose more blind spots) transferred effectively to stronger Executors, often approaching or exceeding the performance of the stronger model's self-trained skills. Conversely, skills from stronger models were less effective on weaker models.
Ablation Studies: Removing either the discovery or curation reward, or decoupling the policies, resulted in significant performance drops, confirming the necessity of joint optimization.
Robustness: The system's performance was robust to the choice of the "LLM Judge" (the strong model used for evaluation), with low-cost Flash-tier models achieving near-optimal results.

Significance and Claims

The paper claims that blind-spot diagnosis is a learnable capability that can be optimized independently of execution. By treating diagnosis as a policy optimization problem, SkillMentor enables agent self-evolution without the computational cost of retraining large executor weights or the privacy/cost burden of human supervision.

The authors emphasize that this approach shifts the paradigm from "learning how to act better" to "learning what the agent does not know." The resulting skill repository serves not only as a product of diagnosis but as an engine for continuous self-evolution, where the discovery of new blind spots and the curation of corrective skills reinforce each other. This work suggests that future self-evolving agents should incorporate dedicated, learned diagnostic policies rather than relying solely on fixed prompting or parameter updates.