💬 NLP

SePO: Self-Evolving Prompt Agent for System Prompt Optimization

The paper proposes SePO, a self-referential framework that optimizes both task agents' and the prompt agent's own system prompts through a two-stage evolutionary search, demonstrating superior generalization and performance across diverse benchmarks compared to existing methods.

Original authors: Wangcheng Tao, Han Wu, Weng-Fai Wong

Published 2026-06-04

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Wangcheng Tao, Han Wu, Weng-Fai Wong

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a brilliant but stubborn robot assistant (the Task Agent) that needs to solve difficult puzzles, write code, or do math. To make this robot work better, you give it a set of instructions called a "system prompt."

For a long time, humans have been the ones writing these instructions. If the robot fails, a human rewrites the instructions, tries again, and hopes for the best. Some newer methods use a "Coach" (the Prompt Agent) to automatically rewrite these instructions for the robot. But here's the catch: the Coach's own instructions were still written by a human and never changed. The Coach was stuck with a fixed, hand-written manual, no matter how many times it tried to help the robot.

SePO (Self-Evolving Prompt Optimization) changes the game by letting the Coach coach itself.

The Big Idea: The Coach Coaches the Coach

Think of SePO as a gym for AI instruction manuals.

The Old Way: You hire a personal trainer (the Coach) to help a client (the Robot) get fit. But the trainer's own workout plan was written by a human and never updated. The trainer gets better at helping the client, but the trainer never gets fitter.
The SePO Way: The trainer is also a client. The trainer helps the client get fit, but the trainer also uses the same training methods to improve their own workout plan. The trainer gets smarter and stronger over time, not just the client.

How It Works: Two Stages of Training

The paper describes a two-step process, similar to how humans learn a new skill:

Stage 1: The "Boot Camp" (Pre-training)
Before the Coach helps any specific robot, it goes to a "boot camp" with a huge variety of challenges (math, logic puzzles, coding, science).

In this stage, the Coach tries to solve these problems.
When it fails, it critiques its own instructions, rewrites them, and tries again.
It keeps a "scrapbook" (an archive) of its best instructions. If a new instruction works better than an old one, it keeps the new one.
The Result: The Coach evolves into a master instructor with a general "skill" for fixing instructions, rather than just memorizing one specific trick.

Stage 2: The "Specialist Job" (Fine-tuning)
Now, the Coach is hired to help a specific robot with a specific job (e.g., solving Sudoku).

The Coach uses its evolved, super-smart instructions to help the robot.
It tweaks the robot's instructions to fit the specific puzzle.
Because the Coach learned how to learn in Stage 1, it can adapt quickly to new jobs without needing a human to rewrite its own manual first.

Why This Matters (The Results)

The researchers tested this on five very different types of challenges:

Math (Hard competition problems)
Abstract Reasoning (Visual pattern puzzles)
Science (Graduate-level questions)
Coding (Writing Python programs)
Logic (Sudoku puzzles)

They compared SePO against:

Manual-CoT: A human writing the instructions.
TextGrad: A method that tweaks instructions but uses a fixed, human-written "critic."
MetaSPO: A method that learns a global rule but still relies on a fixed human-written optimizer.

The Outcome:
SePO won on every single task. On average, it improved accuracy by 4.49 points compared to the human-written baseline.

It didn't just memorize answers; it learned a general "skill" of how to write better instructions.
Even when tested on a puzzle (Sudoku) that it never saw during its boot camp, it still performed better than the other methods. This proves it learned a transferable skill, not just a specific trick.

The Analogy of the "Evolutionary Garden"

Imagine the instructions are plants in a garden.

Old methods: You plant seeds (instructions) and water them. If a plant dies, you pick a new seed from a bag and try again. The gardener (the Coach) never changes.
SePO: The gardener is also a plant. The gardener grows, evolves, and gets better at gardening while tending to the other plants. The garden keeps a record of the best plants (the archive) and uses them as stepping stones to grow even better plants. Eventually, the garden produces a gardener who is so skilled they can grow anything, even plants they've never seen before.

Summary

SePO closes the loop. Instead of a human writing a fixed manual for an AI coach, SePO lets the AI coach write and improve its own manual. This turns the coach from a static tool into a learning partner that gets smarter with experience, leading to better performance across math, science, coding, and logic puzzles.

Technical Summary: SePO (Self-Evolving Prompt Optimization)

1. Problem Statement

Current system prompt optimization methods aim to improve agent behavior without modifying underlying model weights, producing human-readable, model-agnostic instructions. However, a critical asymmetry exists in existing frameworks: while the task agent's system prompt is treated as an optimization target, the prompt agent (the entity responsible for refining the task agent's prompt) remains hand-engineered and fixed.

Existing approaches, such as evolutionary search or textual-gradient frameworks, rely on a prompt agent that does not improve as it encounters more tasks. Consequently, the optimization process is bounded by the initial human design of the prompt agent and cannot accumulate experience or generalize optimization skills across different domains. The core issue is that the "optimizer" itself is not optimized.

2. Methodology: SePO

The authors propose Self-Evolving Prompt Optimization (SePO), a framework that treats the prompt agent's own system prompt as an optimization target alongside the task agent's prompt. This is achieved through a self-referential design and a two-stage training pipeline.

2.1 Self-Referential Design

SePO defines a "prompt task" where the prompt agent acts as a special task agent. Its objective is to improve the prompts of other task agents. By applying the same optimization procedure to the prompt agent's own system prompt ( $\tilde{p}$ ) as it does to the task agent's prompt ( $p$ ), SePO creates a "self-referential closure."

Mechanism: The prompt agent refines a copy of itself.
Evolutionary Search: The process utilizes an open-ended evolutionary search over a population of candidate prompts. It maintains an archive of candidate prompts, allowing earlier, successful prompts to serve as "stepping stones" for later improvements, preventing the search from getting stuck in local optima.

2.2 Two-Stage Training Pipeline

Mirroring the standard pre-training and fine-tuning paradigm, SePO organizes the optimization into two distinct stages:

Stage 1: Pre-training (Self-Evolving Prompt Agent)
- Goal: Evolve the prompt agent's system prompt ( $\tilde{p}$ ) to acquire broad, generalizable prompt optimization skills.
- Process: The prompt agent undergoes evolutionary search on a multi-task pool.
- Configurations:
  - SePO-Generalist (Default): Trains on a diverse mixture of tasks to maximize robustness and generalization.
  - SePO-Specialist: Trains on a single task identical to the target fine-tuning task.
- Outcome: A single, evolved prompt agent ( $\tilde{p}^*$ ) capable of optimizing prompts across various scenarios.
Stage 2: Fine-tuning (Optimize Target Task Agent)
- Goal: Optimize the system prompt ( $p$ ) for a specific target task agent.
- Process: The evolved prompt agent (from Stage 1) is fixed and used to guide the evolutionary search for the target task agent's prompt.
- Efficiency: The cost of self-evolving the prompt agent is amortized across many fine-tuning tasks, as the pre-trained prompt agent is reused.

3. Key Contributions

Closing the Optimization Loop: SePO is the first method to treat the prompt agent itself as a learnable component, removing the reliance on a fixed, hand-engineered optimizer.
Self-Referential Evolution: It introduces a mechanism where the same procedure refines both the task agent and the prompt agent, enabling the accumulation of optimization skills.
Two-Stage Training: By separating pre-training (skill acquisition) from fine-tuning (task application), the method decouples the cost of developing the optimizer from its application to specific tasks.
Archive-Based Search: The use of an archive allows the system to leverage historical successes as stepping stones, facilitating open-ended evolution rather than simple iterative refinement.

4. Experimental Results

The authors evaluated SePO on five diverse benchmarks: AIME'25 (math), ARC-AGI-1 (abstract reasoning), GPQA (graduate-level science), MBPP (code generation), and Sudoku (logic puzzles).

Performance: SePO consistently outperformed three baselines:
- Manual-CoT: Hand-crafted prompts with Chain-of-Thought.
- TextGrad: A textual-gradient framework with a fixed prompt optimizer.
- MetaSPO: A meta-learning approach with a fixed meta-optimizer.
Accuracy Gains: SePO-Generalist achieved the highest accuracy on all five tasks, improving the average accuracy by 4.49 points compared to Manual-CoT.
Generalization:
- The prompt optimization skills learned during pre-training generalized to tasks outside the pre-training mixture (e.g., Sudoku was not in the pre-training set, yet accuracy improved significantly).
- The method demonstrated robustness across different model pairs (swapping DeepSeek-V3.2 and Gemini 3.1 Pro Preview for Gemini 3.1 Flash-Lite and Claude Opus 4.6).
Cost Efficiency: While pre-training incurs a cost, SePO-Generalist amortizes this across multiple tasks, resulting in a lower per-task cost compared to training a specialist for each task or using methods like TextGrad that do not amortize optimizer training.

5. Significance and Claims

The paper claims that SePO fundamentally shifts system prompt optimization from a fixed tool to a learnable skill. By allowing the prompt agent to self-improve, the framework breaks the ceiling imposed by human hand-engineering.

Modest Claims: The authors explicitly state that SePO operates only on the natural language system prompt, leaving model weights, tools, and workflows untouched. They acknowledge that while the current work focuses on prompts, the self-referential design could theoretically extend to tools and architectures in future work, but they do not claim to have implemented this yet.
Core Insight: The primary significance lies in demonstrating that prompt optimization is not merely a search problem for a single task, but a skill that can be acquired and transferred. The "prompt agent" is no longer a static fixture but an evolving component that strengthens with experience.

The authors conclude that SePO encourages further research into self-evolving agents that can continuously broaden the scope of their own self-improvement.