Evo-Harness: Context-to-Harness Skill Compilation for Self-Evolving Agents
Evo-Harness introduces a self-evolving agent framework that addresses the challenge of learning from noisy, one-shot interactions in novel real-world tasks by employing context-to-harness skill compilation to distill experiences into reusable, structured skill harnesses for continuous cross-domain improvement.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the rapidly evolving world of artificial intelligence, a specific kind of software known as a large language model agent has emerged as a powerful tool for solving complex problems. These agents can read instructions, plan steps, use digital tools, and interact with websites or computer systems much like a human would. However, a persistent challenge has limited their potential: when these agents fail at a task, they often treat that failure as a dead end. They do not naturally learn from the mistake to avoid repeating it later. Traditional methods for teaching these systems usually involve gathering thousands of past attempts and analyzing them offline to find patterns, a process that is slow and often disconnected from the real-time flow of work. In many practical situations, an agent encounters a new, difficult task only once, with a single chance to succeed or fail, leaving little room for the slow accumulation of data that traditional learning requires.
Researchers have now proposed a new way for these digital workers to improve on the fly, a method they call online harness learning. Instead of trying to retrain the core brain of the agent, which is often fixed and unchangeable, they attach an external guidebook that evolves with every new experience. This guidebook, or "harness," acts as a structured collection of lessons learned from previous attempts. When the agent fails, the system does not just store the error; it actively compiles the messy details of that failure into a clear, reusable rule. This rule is then added to the guidebook, ready to help the agent navigate similar challenges in the future. The goal is to transform a noisy, one-time mistake into a clean, actionable piece of wisdom that can be applied across different types of tasks.
To test this idea, the researchers built a system named Evo-Harness and put it through its paces on five distinct and demanding benchmarks. These tests ranged from navigating complex websites and managing software code repositories to executing precise command-line instructions and using various digital tools. In every scenario, the agent was given a stream of tasks, one after another, with no opportunity to pause and retrain its underlying model. The system was designed to take the raw context of each attempt—the instructions given, the actions taken, the outcome, and any feedback received—and distill it into a structured lesson. If the agent failed, a specialized process would reflect on what went wrong, proposing a new rule or a modification to an existing one. Another process would then decide whether to add this new rule, merge it with existing knowledge, or discard it if it was too specific to that single failed attempt.
The results of this experiment were striking. The agents using the evolving guidebook consistently outperformed those that did not use it, and they also beat other methods that relied on simply retrieving past examples or storing unstructured memories. On a benchmark focused on command-line tasks, the improvement was particularly dramatic, with success rates jumping from roughly 63 percent to over 73 percent. This suggests that the ability to compile noisy, single-shot experiences into structured guidance is especially valuable when tasks require a sequence of precise operations, such as inspecting files or recovering from errors. The study also found that the type of guidance the system generated varied depending on the task. For web navigation, the guidebook filled up with workflow procedures; for software engineering, it focused on verification and recovery steps; and for reasoning tasks, it captured domain-specific logic. This adaptability indicates that the system is not just memorizing answers but is learning the underlying structure of how to solve problems.
A critical insight from the research is that not all feedback is created equal. The system performed best when it received clear, grounded feedback from the environment, such as a specific error message or a test result, rather than when it was asked to judge its own success. When the agent tried to generate its own feedback without external evidence, its performance actually dropped below the baseline, suggesting that self-judgment can introduce confusion. Furthermore, the study revealed that the size and capability of the agent's brain mattered. Stronger models were better at interpreting and following the evolved guidance, gaining significantly more from the system than weaker models. Interestingly, a smaller model could sometimes write a guidebook that helped a larger, more capable model, but the reverse was not always true. If the agent solving the task was not sophisticated enough to understand the guidance, the extra information did not help and could even hinder performance.
The researchers also explored how these learned skills could be transferred. They found that a guidebook created by one model could be useful for another, and that skills learned on a set of training tasks could improve performance on unseen test tasks. However, the most effective approach was to update the guidebook continuously as the agent worked through the stream of tasks. This real-time adaptation allowed the system to adjust to the specific quirks and failure modes of the current environment, outperforming even the best pre-trained guidebooks. The study concludes that the key to self-improving agents lies not in constantly retraining their core models, but in building a robust, external system that can turn the chaos of real-world failures into a clear, organized set of instructions. By treating the guidebook as a living document that grows and refines itself with every interaction, these digital agents can learn to handle the unpredictable nature of complex, real-world work.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.