💻 computer science

Software Self-Extension with SelfEvolve: an Agentic Architecture for Runtime Code Generation

This paper introduces SelfEvolve, an agentic architecture that enables software systems to autonomously generate and integrate new functionalities at runtime, demonstrating a 92.7% success rate and significantly outperforming existing code generation baselines to pave the way for self-evolving software.

Original authors: Md Asif Iqbal Fahim, Oluwadamilola Adebayo, Alessio Ferrari

Published 2026-04-21

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Md Asif Iqbal Fahim, Oluwadamilola Adebayo, Alessio Ferrari

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart, but slightly rigid, personal assistant. Today, if you ask them to "calculate the best route for my road trip," they can do it instantly because that's a skill they were taught before you hired them. But if you suddenly ask, "Can you also predict how much my car will cost to maintain based on local weather patterns?" they will likely say, "I don't know how to do that. I need to call my boss (the developer) to teach me."

In the world of software, this is the norm. To add a new feature, a human developer has to write code, test it, and update the app. This takes days or weeks.

SelfEvolve is a new kind of software architecture that changes the rules. It's like giving that personal assistant a magic toolbox and the ability to teach themselves new skills while they are working with you, without ever needing to call their boss or stop the conversation.

Here is how it works, broken down into simple concepts:

1. The "Self-Teaching" Assistant

Think of SelfEvolve not as a static program, but as a team of specialized robots working together in a factory.

The Dispatcher: This is the receptionist. When you ask for something new (e.g., "Show me what my neighbors are watching"), the receptionist checks the library. If the skill exists, they do it. If not, they say, "We need to build a new tool for this."
The Architect (Code Generator): This robot takes your request and starts building the new tool.
The Safety Inspector (Test Generator): Before the tool is even finished, this robot writes a checklist of rules to make sure the tool works correctly.
The Judge (Adjudicator): This robot runs the new tool against the checklist. If it fails, the Judge doesn't just throw it away; they send it back to the Architect with a note saying, "You forgot to handle this specific error. Try again."

2. The "Test-First" Safety Net

The secret sauce of SelfEvolve is something called Test-Driven Development (TDD).
Imagine you are baking a cake.

Old Way: You bake the cake, taste it, realize it's too salty, throw it away, and start over.
SelfEvolve Way: Before you even mix the flour, you write down exactly what the cake should taste like (not too salty, sweet enough, fluffy). Then, you bake the cake and immediately check it against your list. If it fails the "not salty" test, you fix the recipe before you serve it to the customer.

In SelfEvolve, the system writes the "taste test" (the code tests) before it writes the final code. This ensures that when the new feature is added, it actually works.

3. The "Hot-Plug" Upgrade

Usually, when you update a phone app, you have to close it, download the update, and restart it.
SelfEvolve uses a trick called hot reloading. Imagine a car that can swap its engine while driving down the highway without the driver ever feeling a bump.

The system builds the new feature in a safe, isolated "sandbox" (like a test kitchen).
Once the new feature passes all the safety tests, it is instantly "plugged in" to the main system.
You, the user, get the result immediately. The system has evolved in real-time.

4. Why This Matters (The Results)

The researchers tested this system on 11 different tasks, ranging from analyzing medical data to calculating complex math.

The Competition: Other AI coding tools (like AutoGen or MetaGPT) are great at helping humans write code, but they struggle to build and install new features on their own while the system is running. They got the new features right only about 30% of the time.
SelfEvolve: It succeeded 92.7% of the time. It was able to learn, build, test, and install new skills autonomously, outperforming the best existing tools by a huge margin.

The Big Picture

Currently, software is like a frozen statue. To change it, you have to chip away at it with a hammer (human developers).
SelfEvolve turns software into a living organism. It can sense a need, grow a new limb (feature) to meet that need, and keep moving forward, all without stopping.

The Future Vision:
Imagine a Netflix that doesn't just recommend movies, but if you ask, "I want to see a movie where the main character is a time-traveling baker," it instantly writes the code to build that specific recommendation engine for you, tests it, and shows you the results. It's the beginning of software that truly evolves to fit your unique needs, instantly.

1. Problem Statement

Traditional self-adaptive systems are limited to reconfiguring existing components in response to changing requirements; they cannot generate entirely new functionalities. Current software development relies on human developers to write, test, and deploy code, a process that can take days or weeks. While Large Language Models (LLMs) have accelerated development-time code generation, they generally operate offline to assist developers.

The paper identifies a gap in runtime self-evolution: the ability of a software system to autonomously generate, validate, and integrate new capabilities during execution based on high-level user requests, without requiring a system restart or developer intervention. The authors aim to move from "self-adaptation" (reconfiguring the known) to "self-extension" (creating the unknown).

2. Methodology: The SelfEvolve Architecture

SelfEvolve is an orchestrated agentic pipeline designed to enable runtime self-extension. It treats the software system as a dynamic entity that can grow its own codebase while running.

Core Components

The architecture consists of specialized LLM agents coordinated by a deterministic control flow:

Knowledge Base: A persistent repository containing:
- Prompts: System instructions and few-shot examples.
- Functions: Executable Python code.
- Tool Descriptors: JSON specifications pairing functions with invocation metadata (enabling OpenAI function-calling).
Chat & Tool Dispatcher: The entry point that receives natural language user requests. It checks if existing tools satisfy the request. If not, it transforms the request into a structured requirement specification ( $R_i$ ).
Test-Driven Code & Function Generator: A dual-purpose agent that:
- Generates TDD test cases ( $T_i$ ) based on the requirement and previous feedback.
- Generates the function implementation ( $F_i$ ) to satisfy these tests. It leverages the Knowledge Base to reuse existing code (e.g., importing a matrix multiplication function to build an eigenvalue calculator).
Adjudicators (Intermediate & Final):
- Intermediate Adjudicator: Validates the generated code against the initial TDD tests. If it fails, it provides specific feedback ( $E_i$ ) to the generator for refinement.
- Unit Test Generator: Once TDD tests pass, this agent generates comprehensive unit tests ( $U_i$ ) to cover edge cases and error handling specific to the implementation.
- Final Adjudicator: Validates the code against the unit tests.
Executor Layer: Executes code in a sandboxed subprocess with timeouts (30s) to ensure safety and isolation from the core system.
System Integration: Upon successful validation, the new function is:
- Written to the Functions repository.
- Registered in the Tool Descriptors.
- Immediately available via importlib.reload() (Python feature) without restarting the application.

Key Technical Mechanisms

Test-Driven Development (TDD) Pipeline: The system generates tests before code to guide implementation.
Iterative Refinement: A feedback loop where failed tests generate specific error feedback ( $E_i$ ) that constrains subsequent generation attempts. The system iterates until success or a limit (set to 6 iterations).
Hot Code Reloading: Utilizes Python's dynamic import capabilities to integrate new modules instantly.

3. Key Contributions

Runtime Self-Extension Paradigm: Proposes a preliminary form of self-evolution where systems autonomously add new capabilities based on user needs, removing the developer from the loop for specific feature requests.
Agentic Pipeline Architecture: Introduces a novel orchestration of LLM agents (Dispatcher, Generator, Adjudicators) that separates code generation from execution and ensures safety via sandboxing.
TDD-Driven Validation: Embeds a rigorous two-stage validation process (TDD tests + Unit tests) with an iterative feedback mechanism to ensure code correctness before integration.
Persistent Knowledge & Reuse: Implements a mechanism where generated functions are stored and can be reused as building blocks for future requests, enabling compositional growth.

4. Experimental Results

The authors evaluated SelfEvolve on 11 distinct problems across three categories:

Integration Tasks: (4 problems) Integrating with existing multi-file codebases (577–783 LOC).
Compositional Tasks: (3 problems) Building new functions on top of previously generated capabilities.
Data Processing Tasks: (4 problems) Manipulating external datasets (50–100 records).

Performance Metrics:

Pass@1: SelfEvolve achieved an average 92.7% (51/55 successful runs).
Baseline Comparison: It significantly outperformed state-of-the-art developer-focused frameworks:
- AutoGen: 30.9% (SelfEvolve is 61.8% higher).
- MetaGPT: 27.3%.
- AgentCoder: 0.0%.
- Statistical Significance: All comparisons showed statistically significant differences (adjusted p-values < 0.05).
Convergence Efficiency: The system converged in an average of 2.2 iterations.

Ablation Study (Impact of TDD):

With TDD: 92.7% Pass@1, 2.2 average iterations.
Without TDD: 72.7% Pass@1, 4.7 average iterations.
Conclusion: The TDD pipeline provided a 20% improvement in success rate and 2.1x faster convergence, proving its critical role in guiding the LLM.

5. Significance and Future Directions

Significance:

Individualized Software: Demonstrates the feasibility of systems that evolve to meet unique, on-the-fly user needs (e.g., a Netflix user requesting a specific local analytics feature that doesn't exist yet).
Developer Displacement: Moves toward a paradigm where the "user" is the only human on the loop, minimizing the need for human developers for routine feature additions.
Feasibility Proof: Provides preliminary evidence that autonomous runtime code generation is viable for non-safety-critical consumer applications.

Limitations & Future Work:

Safety: Currently focused on functional correctness; future work must address security, scalability, and non-functional requirements.
MAPE-K Integration: The system is currently reactive (user-request driven). Future iterations aim to implement a full MAPE-K (Monitor-Analyze-Plan-Execute-Knowledge) loop for proactive self-improvement.
Human-in-the-Loop: Introducing guardrails and human verification for mission-critical contexts.
Requirements Engineering: Enhancing the system's ability to clarify ambiguous user intent and perform change-impact analysis.

In summary, SelfEvolve represents a significant step toward autonomous software evolution, proving that LLM agents can not only write code but also safely integrate it into running systems to extend functionality dynamically.