Beyond Scaling: Self-Evolving LLM Agents for Hardware Kernel Optimization via an Experience-Driven Workflow and Experience Graph Memory
The paper introduces KOPE, an experience-driven framework that utilizes Experience Graph Memory and Active Context Management to enable LLM agents to learn from past hardware kernel optimization trajectories, achieving significant performance improvements and token efficiency over existing baselines without requiring updates to the underlying foundation model.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Making a computer chip run faster often feels like trying to tune a race car engine while it is still moving. The process involves writing a tiny piece of code, known as a kernel, that tells the hardware exactly how to handle data. This code must be compiled, tested to ensure it is correct, and then measured to see how fast it runs. If the result is not perfect, the code must be rewritten and the cycle begins again. For decades, this has been a job for highly skilled human experts who understand the specific quirks of every new chip design. Recently, artificial intelligence systems have begun to help with this work, using powerful language models to suggest code changes. However, these systems usually treat each attempt as a fresh start. They do not remember what worked or what failed in previous tries, and they cannot easily learn from the specific hardware they are trying to optimize, especially when that hardware is new and lacks a large library of existing examples.
A team of researchers has developed a new approach that allows an artificial intelligence agent to learn from its own exploration, turning every trial and error into a permanent lesson. They created a system called KOPE, which stands for a framework designed to optimize these hardware kernels by building a memory of its own experiences. Instead of relying solely on the knowledge already stored inside the AI model, KOPE records every decision it makes, the result of that decision, and the path it took to get there. It stores this information in a structured map that connects a specific choice to its outcome, much like a traveler keeping a detailed journal of a journey, noting which paths led to dead ends and which opened new doors. When the system faces a new problem, it does not start from scratch. Instead, it consults this map to find relevant past experiences that match the current situation, injecting that specific knowledge into its thinking process before it makes a new suggestion.
The researchers tested this system on a set of fifty-three different mathematical operations used in modern computing, running them on a specific type of hardware chip from Huawei. They compared their new method against two other approaches: one that used a standard AI agent without this special memory system, and another that used a powerful AI designed for a different type of computer chip. The results showed that the system with the experience memory was significantly more successful. It managed to produce working code for almost all of the operations, whereas the standard agent failed to solve many of them. More importantly, the system that learned from its own history produced code that was, on average, 1.54 times faster than the code produced by the best competing method. This improvement happened even though the underlying AI model itself did not change; the system simply became better by remembering what it had learned.
A key part of this success was how the system managed its memory. The researchers found that simply dumping all past experiences into the AI's mind at once was counterproductive, as it overwhelmed the system with too much information. Instead, they built a mechanism that actively selected only the most relevant past lessons for the specific task at hand. This selective approach allowed the system to solve problems more often, raising its success rate from sixty percent to nearly eighty-five percent, while using far fewer computing resources to do so. The study also explored whether knowledge gained on one type of hardware could help optimize a completely different type of hardware. They found that while experience from the original hardware helped the system find more working solutions on the new hardware, it did not automatically make the code faster. The system still needed to learn the specific details of the new hardware through its own trials to achieve the best performance.
The findings suggest that for hardware that is new or lacks a large collection of existing examples, the ability to retain and reuse specific experiences is more valuable than simply having a larger or smarter AI model. By keeping a fixed record of what worked and what did not, and by carefully choosing which lessons to apply to new problems, the system can continuously improve its optimization skills. This approach does not require the AI to be retrained or to have its internal brain changed; it simply needs a better way to remember and apply what it has learned. The researchers demonstrated that this method can be applied across different types of computer chips and programming languages, showing that the workflow itself is adaptable. While the system did not solve every single problem perfectly, it proved that an AI agent can evolve its own expertise through a cycle of action, observation, and memory, turning the messy process of trial and error into a reliable path toward faster and more efficient computing.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.