💻 computer science

EvoRepair: Enhancing Vulnerability Repair Agents Through Experience-Based Self-Evolution

EvoRepair introduces the first experience-based self-evolving agent framework for automated vulnerability repair that accumulates and reuses cross-vulnerability knowledge to significantly outperform existing LLM-based baselines on multiple benchmarks.

Original authors: Haichuan Hu, Guoqing Xie, Quanjun Zhang, Jiawei Liu, Shengcheng Yu, Chunrong Fang, Zhenyu Chen, Liang Xiao

Published 2026-05-29

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Haichuan Hu, Guoqing Xie, Quanjun Zhang, Jiawei Liu, Shengcheng Yu, Chunrong Fang, Zhenyu Chen, Liang Xiao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to fix a broken machine in a dark room. You have a very smart, but inexperienced, apprentice (the AI) who is trying to figure out how to fix it.

The Problem: The "Trial-and-Error" Trap
In the past, when we asked AI to fix software security holes (vulnerabilities), it worked like this: The AI would guess a fix, try it, see if it broke, and then guess again.

The Issue: If the AI made a mistake, it often forgot about it. In the next attempt, it would make the exact same mistake again. It was like a student who fails a math test, doesn't study the errors, and then fails the next test with the same wrong answer.
The Second Issue: Even if the AI learned how to fix a specific type of leak in a pipe, it wouldn't remember that lesson when it encountered a different pipe with a similar leak. It treated every new problem as if it were the very first time it had ever seen one.

The Solution: EVOREPAIR (The "Experienced Master" System)
The authors of this paper created a system called EVOREPAIR. Think of it as giving the apprentice a personal mentor and a massive library of past case studies.

Here is how it works, using a simple analogy:

1. The "Experience Bank" (The Library)

Instead of just guessing, EVOREPAIR keeps a detailed diary of every repair attempt.

What's in the diary? It doesn't just write "I fixed it." It writes: "We tried using a hammer, but it cracked the glass. We learned that for glass, we must use a screwdriver. Also, we found that if the screw is rusty, we need to oil it first."
The "Score": Not every lesson is good. Some lessons are wrong or only work in one specific situation. EVOREPAIR uses a "judge" to rate these lessons. Only the high-quality, useful lessons are kept in the library.

2. The "Search" (The Librarian)

When a new vulnerability appears (a new broken machine), EVOREPAIR doesn't start from scratch.

It acts like a librarian. It looks at the new problem and says, "Hey, this looks a lot like that broken toaster we fixed last week!"
It pulls out the specific lessons from the library that match the current problem. It tells the AI: "Don't use the hammer this time; use the screwdriver, and remember to oil the screw."

3. The "Self-Evolution" (Getting Smarter)

This is the magic part. The system is cyclic.

Repair: The AI tries to fix the problem using the library's advice.
Learn: After the attempt (whether it succeeded or failed), the system analyzes what happened.
Update: It writes a new lesson into the library based on this new attempt. If the AI found a better way to use the screwdriver, that new method is added to the book.
Repeat: The next time a similar problem comes up, the library is even better than before. The AI is literally "evolving" by learning from its own history.

The Results: How Well Did It Work?

The researchers tested this system against 12 other top methods for fixing software bugs.

The Score: EVOREPAIR fixed 93.5% of the problems on one major test and 87% on another.
The Comparison: It was significantly better than the previous best methods. For example, it fixed about 40% more problems than the next best AI system.
The "Transfer" Test: They even took the lessons learned from fixing problems in one programming language (like JavaScript) and used them to fix problems in a completely different language (like Java). It still worked, proving that the "lessons" were general enough to be useful across different types of code.

In a Nutshell

Previous AI repair tools were like students who forgot their mistakes. EVOREPAIR is like a master craftsman who keeps a detailed journal of every job, learns from every failure, and uses that accumulated wisdom to fix new problems faster and more accurately than ever before. It turns "trial and error" into "trial, learn, and succeed."

Technical Summary: EVOREPAIR

Problem Statement

Automated Vulnerability Repair (AVR) using Large Language Models (LLMs) has shown promise but faces two critical limitations that hinder scalability and effectiveness:

Lack of Intra-Vulnerability Experience Accumulation: Existing methods often treat each repair attempt as a stateless process. Even when multiple repair trajectories are explored for a single vulnerability, intermediate successes, failures, and diagnostic signals are rarely distilled into reusable knowledge. Consequently, LLMs may repeatedly follow unproductive paths or make similar mistakes across iterations.
Lack of Cross-Vulnerability Experience Reuse: Current AVR approaches largely treat each vulnerability as an isolated task. While retrieval-augmented methods exist, they typically rely on static, instance-specific reference patches rather than dynamic, generalizable repair knowledge. As the volume of reported vulnerabilities (CVEs) grows, this one-by-one isolated repair model becomes increasingly unscalable.

The core challenge is how to enable AVR systems to continuously accumulate, refine, and reuse repair experience both within a single vulnerability's repair process and across different vulnerabilities.

Methodology: EVOREPAIR

To address these challenges, the authors propose EVOREPAIR, the first experience-based self-evolving agent framework for AVR. The system operates on a cyclic learn-and-repair paradigm consisting of four main components:

1. Domain-Specific Experience Definition

EVOREPAIR defines a standardized schema for "experience" to ensure reusability and transferability. Unlike raw interaction logs, an experience entry is a structured abstraction comprising five dimensions:

Vulnerability Introduction & Analysis: Concise description of the type, location, root cause, and impact.
Repair Strategy: The chosen remediation approach and its justification.
Trajectory Analysis: A distilled account of the repair process, highlighting successful actions, failure patterns, and key commands/logs.
Experience Summary: Compact, prescriptive rules (preconditions, goals, guidance) with minimal examples.
Reflection & Improvement: Evaluation of the outcome, limitations, and suggestions for future strategies.

2. Experience Retrieval and Warm-up

Before each repair turn, the agent queries an Experience Bank.

Retrieval Mechanism: The system retrieves experiences based on a consolidated score combining similarity (CVE/CWE matching) and quality (a score derived from an LLM-as-a-Judge evaluating quality and generalizability).
Warm-up Strategies: To address the "cold-start" problem (empty experience bank), EVOREPAIR employs either a Standard Patch strategy (using historical official patches) or a Pre-repair strategy (initializing with pre-generated experiences from earlier-disclosed vulnerabilities to avoid self-instance leakage).

3. Vulnerability Repair (The Vanilla Agent)

The core repair engine is a "vanilla agent" built on a general-purpose framework (inspired by Mini-SWE-Agent).

Context Injection: Retrieved experiences are injected into the agent's context to guide the repair trajectory.
Process: The agent interacts with a Docker environment using a minimal Bash toolkit to reproduce the vulnerability, locate the flaw, generate a patch, and verify it.
Early Stopping: To ensure cost-efficiency, the system calculates a turn-level yield rate ( $\alpha$ ), defined as the ratio of the marginal gain in repair performance to the marginal increase in cost. If $\alpha$ falls below a threshold, the repair process terminates early.

4. Experience Construction and Updating

After a repair attempt, the system constructs new experiences from the trajectory:

Compression: Trajectories are compressed into the structured schema to prevent context window bloat.
Scoring: An LLM-as-a-Judge scores the new experience based on Quality (actionability) and Generalizability (transferability).
Updating: The experience bank is updated using a strategy that discards lower-quality entries, retains higher-quality ones, or "polishes" entries with identical scores by fusing complementary details.

Key Contributions

New Research Dimension: The paper introduces experience-based self-evolution as a novel dimension for AVR, focusing on the continuous accumulation and reuse of repair knowledge rather than just static retrieval.
Novel Framework (EVOREPAIR): A cyclic framework that enables both cross-vulnerability experience reuse and intra-vulnerability experience accumulation through a structured learn-and-repair loop.
Extensive Evaluation: Comprehensive experiments across multiple datasets (PATCHEVAL, SEC-bench, VUL4J) and models (GPT-5-mini, Qwen3.5-Plus, DeepSeek-v3.1, Devstral variants) demonstrate the framework's effectiveness and transferability.

Experimental Results

The authors evaluated EVOREPAIR against 12 representative baselines (including learning-based, LLM-based, and agent-based methods) using GPT-5-mini as the primary backbone.

Overall Performance: EVOREPAIR achieved the best overall performance, repairing 93.47% of vulnerabilities on PATCHEVAL and 87.00% on SEC-bench, for an overall rate of 90.46%.
Comparison with Baselines:
- Outperformed the strongest LLM-based baseline, LoopRepair, by 39.56% (PATCHEVAL) and 33.50% (SEC-bench).
- Surpassed IntentFix by 70.86% and 50.50% respectively.
- Exceeded the recent self-evolving agent Live-SWE-Agent by 6.98% overall.
Self-Evolution Impact: Across 15 repair turns, EVOREPAIR consistently broke the performance ceiling of vanilla agents, showing faster convergence and higher efficiency. For instance, on PATCHEVAL with GPT-5-mini, fixes increased from 194 (vanilla) to 215 (EVOREPAIR).
Transferability: Experiments on VUL4J (Java) using experiences distilled from PATCHEVAL (JavaScript/Python/Go) showed significant performance gains (e.g., +9.67% for Qwen3.5-Plus), confirming cross-language and cross-dataset transferability.
Cost Analysis: While prompt overhead increased for some models, the total cost remained nearly unchanged or decreased for stronger models (e.g., Qwen3.5-Plus saw a 41.58% cost reduction) due to faster convergence and fewer redundant turns.

Significance and Claims

The paper claims that EVOREPAIR substantially strengthens agentic AVR by moving beyond static retrieval and simple iterative prompting. Its significance lies in demonstrating that:

Experience is a critical resource: Accumulating and refining repair knowledge allows agents to avoid repetitive mistakes and generalize across heterogeneous vulnerabilities.
Self-evolution is effective: The cyclic learn-and-repair process enables agents to improve their own repair capabilities over time without human intervention.
Generalizability: The distilled experiences are robust enough to transfer across different programming languages, datasets, and model backbones, suggesting a path toward scalable, large-scale vulnerability repair systems.

The authors conclude that experience-based self-evolution goes beyond existing self-evolving techniques by explicitly modeling the lifecycle of repair knowledge, offering a promising direction for addressing the growing volume of software security vulnerabilities.