🤖 AI

The Optimizer Is the Agent: Reasoning-Driven Search across Prompts, Programs, and ML Workflows

The paper introduces ReASearch, a unified framework where a single reasoning-driven agent autonomously manages the entire optimization process for prompts, programs, and ML workflows, effectively internalizing complex search policies and outperforming specialized systems across diverse tasks.

Original authors: Junbo Li, Boyi Liu, Canwen Xu, Yite Wang, Yuxiong He, Zhangyang Wang, Qiang Liu, Zhewei Yao

Published 2026-08-10

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Junbo Li, Boyi Liu, Canwen Xu, Yite Wang, Yuxiong He, Zhangyang Wang, Qiang Liu, Zhewei Yao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to solve a puzzle, write a poem, or train a video game character to win. In the world of artificial intelligence, this is called "optimization." Usually, humans act like the strict coaches on the sidelines. They design a rigid game plan: "Try this, then try that, then if it fails, go back and try something else." The robot (a large language model) is just the player, blindly following orders to make small changes while the human coach decides the big strategy. But what if the robot could be the coach too? What if it could look at the scoreboard, figure out why it lost, decide to change the rules of the game, or even realize it's stuck and need a completely new approach—all on its own? This paper explores that exact question: Can a single, smart AI agent learn to manage its own search for the best solution, without a human-written script telling it how to think?

The researchers behind this study, published at the COLM 2026 conference, introduce a new system called ReASearch. Think of it as handing the robot a Swiss Army knife instead of just a hammer. In the past, AI optimizers were like a factory assembly line with a fixed conveyor belt: a human-designed algorithm decided which candidate solution to test next, how much time to spend, and when to give up. The AI's only job was to whisper a small edit to the current solution. ReASearch flips the script. It gives the AI agent a set of tools—like a Python interpreter to run code, a memory file to remember past mistakes, and the ability to run experiments—and then lets the agent decide everything. The agent asks itself: "Should I test this idea on a small sample first? Did I just make the same mistake twice? Should I throw away my current plan and go back to a better one I tried earlier?" The agent isn't just guessing; it's reasoning through the process, building a strategy as it goes.

The team tested this "self-coaching" agent on 14 different challenges, ranging from tweaking text prompts to make them smarter, to evolving computer programs to solve math puzzles, and even optimizing machine learning training workflows. The results were surprisingly strong. In many cases, ReASearch beat specialized systems that had been built by human experts with specific, hard-coded rules. For example, in a task called "Circle Packing" (fitting circles into a square as tightly as possible), the agent discovered solutions that were better than the best-known results humans had found before. In other tasks, it improved performance by anywhere from 2% to a massive 40%.

What's truly fascinating is how the agent did it. The researchers found that the agent naturally started acting like a seasoned scientist or a grandmaster chess player, even though no one told it to. It began "double-checking" promising ideas before committing to them. It learned to "revert" to earlier, safer versions of a solution when a new path led to a dead end. It started keeping a "lessons learned" journal to avoid repeating the same errors. Most impressively, it figured out that sometimes you need to combine two different techniques to solve a problem, a discovery that usually requires deep human insight. The paper suggests that when you give an AI the right tools and let it reason through the process, it can internalize complex search strategies that we previously thought needed a human-designed algorithm to manage. It turns out, the optimizer doesn't need to be a separate, rigid controller; the optimizer can be the agent itself.

Technical Summary: The Optimizer Is the Agent (ReASearch)

Problem Statement

Optimizing text-based artifacts—specifically system prompts, programs, and machine learning (ML) workflows—remains a central challenge in modern AI. Traditional approaches typically rely on external, hand-designed algorithmic controllers (e.g., evolutionary search, bandits, Bayesian optimization, or textual gradients) to manage the search policy. In these frameworks, Large Language Models (LLMs) function merely as stateless "mutators" or "proposal generators" that execute local edits within a rigid, pre-defined loop. The authors posit that this separation leaves a fundamental question unaddressed: How much of the search policy can be internalized by a single reasoning agent?

Methodology: ReASearch

The paper introduces ReASearch (Reasoning-driven Agentic Search), a unified, "controller-light" framework that places a tool-using LLM agent fully in charge of the optimization loop. Rather than relying on an external mathematical meta-heuristic to dictate exploration, budget allocation, and candidate selection, ReASearch exposes the entire optimization process as a set of tools that the agent invokes autonomously.

Core Architecture

Unified Agent Loop: The system employs a single, task-agnostic agent scaffold. The agent receives a system prompt containing domain guidance and the current optimization state, along with its interaction history.
Tool-Based Interface: All optimization operations are packaged as tools. The agent decides which tool to invoke next based on its reasoning. Key tools include:
- Execution & Analysis: python_exec allows the agent to write analysis scripts, compute statistics, and diagnose failure patterns programmatically.
- Evaluation: Domain-specific tools (e.g., validate_candidate, run_experiment, evaluate) allow the agent to test hypotheses.
- Editing: Tools like edit_code or edit_train_file route changes to sub-agents to keep the main agent's context clean.
- Memory: A persistent lessons.md file and context compression mechanisms allow the agent to retain long-horizon insights (what worked, what failed, and what to try next) across context windows.
Domain Adaptation: The same agent scaffold is applied to three distinct domains by changing only the available tools and the system prompt:
1. Prompt Optimization: The agent adaptively samples training minibatches, evaluates candidates on validation sets (with aggregate metrics to prevent overfitting), and diagnoses failure modes.
2. Program Evolution: The agent iteratively modifies code for single-instance hard problems (e.g., ARC-AGI-2, circle packing), using sub-agents for code editing and verification while maintaining a persistent reasoning context for hypothesis formation.
3. ML Workflow Optimization: The agent modifies training scripts, runs time-bounded experiments, and analyzes training curves to diagnose bottlenecks (e.g., learning rate schedules, architecture choices, regularization).

Emergent Behaviors

The paper observes that complex search behaviors, typically implemented by explicit controllers, emerge naturally from the agent's reasoning process without hard-coding:

Double-Verification: Re-checking promising gains on additional samples before committing to expensive validation.
Hypothesis-Driven Revision: Forming structural hypotheses based on accumulated evidence and testing them.
Revert-Based Recovery: Deliberately reverting to earlier, known-good branches when a path fails, rather than blindly continuing.
Adaptive Exploration: Adjusting search strategies based on budget constraints and observed variance.

Key Contributions

Unified Framework: ReASearch casts text-based optimization as a sequential reasoning problem, demonstrating that a single agent design can optimize prompts, programs, and ML workflows.
Performance Gains: Across 14 diverse tasks, ReASearch consistently matches or outperforms specialized, state-of-the-art systems (e.g., GEPA, AdaEvolve, Claude Code) within the same computational budgets. Gains range from 2% to 40% over strong domain-specific baselines.
Human-Level Discovery: In specific tasks like Circle Packing and Heilbronn Triangle problems, ReASearch discovered solutions that improved upon prior human best-known results.
Internalized Search Logic: The work demonstrates that frontier tool-using agents can internalize a substantial fraction of search logic previously delegated to external mathematical meta-heuristics, including candidate verification, causal diagnosis, and adaptive exploration.

Experimental Results

The authors evaluated ReASearch against specialized baselines across three categories:

Prompt Optimization: On AIME, GSM8K, HotpotQA, and Terminal-Bench 2.0, ReASearch outperformed GEPA. For instance, on Terminal-Bench 2.0, it achieved 53.33% accuracy compared to GEPA's 42.22%. The agent successfully built structured task understandings and selected final prompts based on full optimization history rather than just peak validation scores.
Program Evolution:
- Circle Packing: ReASearch achieved results (e.g., 2.940 for $n=32$ ) that matched or slightly exceeded the best-known human results (2.939), outperforming AdaEvolve significantly.
- Heilbronn Triangle: The agent identified mathematical symmetries (D4 symmetry) to collapse the search space, achieving near-optimal results (0.03260 for $n=12$ ) where baselines failed.
- ARC-AGI-2: ReASearch achieved 50.0% test accuracy (Sonnet 4.6), a 4x improvement over AdaEvolve's 12.5%, by using iterative analysis to deduce transformation rules rather than brute-force search.
ML Workflow Optimization: Compared to Claude Code (AutoResearch setup), ReASearch showed clear advantages in IMG-100 (83.99% vs 78.59%), Atari Q*bert (4500 vs 1250 reward), and MuJoCo (5267 vs 3986 reward). Notably, in a Kaggle Crypto prediction task, ReASearch improved the leaderboard ranking from 36th to 6th within 15 experiments, whereas the baseline stagnated.

Significance and Claims

The paper argues that reasoning is not just for solving tasks but can itself be a powerful engine for open-ended search. The significance of ReASearch lies in its demonstration that the "optimizer" does not need to be a separate, hand-designed algorithmic controller. Instead, a single, reasoning-driven agent equipped with appropriate tools can internalize complex search policies—such as budget management, failure diagnosis, and strategy adaptation—that were previously the domain of external meta-heuristics.

The authors claim that this approach offers a more flexible and potentially more powerful paradigm for optimization, where the agent's ability to accumulate lessons, reason about causal failures, and adapt its strategy over long horizons leads to superior performance across diverse modalities. The work suggests a future where the distinction between the "searcher" and the "solver" dissolves, with the agent acting as both.