🤖 machine learning

Adaptive Auto-Harness: Sustained Self-Improvement for Agentic System Deployment on Open-Ended Task Streams

The paper introduces Adaptive Auto-Harness, a framework and system designed to enable sustained self-improvement for LLM agents in open-ended task streams by decomposing performance gaps into evolution and adaptation losses, utilizing a stateful multi-agent evolver, a harness tree with solve-time routing, and human-steering hooks to outperform existing baselines on dynamic, heterogeneous tasks.

Original authors: Zewen Liu, Zhan Shi, Yisi Sang, Bing He, Minhua Lin, Tianxin Wei, Dakuo Wang, Benoit Dumoulin, Wei Jin, Hanqing Lu

Published 2026-06-02

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Zewen Liu, Zhan Shi, Yisi Sang, Bing He, Minhua Lin, Tianxin Wei, Dakuo Wang, Benoit Dumoulin, Wei Jin, Hanqing Lu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Forever-Running" Robot

Imagine you have a very smart robot assistant (an AI agent) that you want to use for a job that never ends. Instead of giving it a fixed list of rules and then letting it go, you want it to learn and get better every single day as new, weird, and different tasks arrive.

The problem is that most current AI systems are like students who only study for one specific exam. They get really good at that one test, but if you give them a different subject the next day, they forget everything or get confused. If you try to keep adding new rules to their brain to cover every possible future task, their brain gets too cluttered, they get slow, and they start making mistakes on the old tasks they used to be good at.

This paper introduces a new system called Adaptive Auto-Harness. Think of it as a smart workshop manager that builds and organizes a toolbox for the robot, ensuring it always has the right tool for the current job, without getting overwhelmed by tools it doesn't need.

The Three Big Problems (The "Why")

The authors say that real-world AI jobs have three tricky features that old systems can't handle:

The Never-Ending Stream (Unbounded): The job doesn't stop. New tasks keep coming forever. Old systems try to memorize everything that ever happened, which eventually fills up their memory and makes them slow.
- Analogy: Imagine a chef who tries to remember every single recipe they've ever cooked in the last 10 years. Eventually, they can't remember the new dish they need to make right now because their brain is full of old recipes.
The Mix of Different Jobs (Heterogeneity): The tasks are all over the place. One minute the robot is doing math, the next it's writing a poem, and the next it's hacking a security system.
- Analogy: Imagine a Swiss Army knife that tries to be a screwdriver, a corkscrew, and a saw all at the same time, permanently attached to your hand. It's heavy and clumsy. You'd be better off having a drawer of specific tools and picking the right one for the job.
The Moving Target (Non-Stationarity): The rules of the game change over time. What worked last month might be useless today.
- Analogy: Imagine playing a video game where the map changes every day. If you memorize the path to the treasure from Day 1, you'll get lost on Day 10.

The Solution: The "Adaptive Auto-Harness"

The authors built a system that solves these problems using three main tricks.

1. The "Team of Experts" (Multi-Agent Evolution)

Instead of one AI trying to figure out how to improve itself (which is hard and slow), they use a team of specialized AI agents working together in a cycle.

The Analyst: Looks at where the robot failed and says, "Hey, we need a better way to do this."
The Researchers: Go out and find new ideas or tools to fix the problem.
The Builders: Actually write the code or update the instructions.
The Verifiers: Test the new instructions to make sure they work before letting the robot use them.
Analogy: Instead of one person trying to build a house alone, you have an architect, a construction crew, and an inspector. They work in shifts, so the house gets built better and faster without anyone getting tired or confused.

2. The "Smart Branching System" (Solve-Time Routing)

This is the most important part. The system doesn't keep one giant, messy instruction manual. Instead, it builds a Tree of Specialized Workspaces.

Imagine a library. Instead of one giant book with every topic mixed together, you have different sections: "Sports," "Politics," "Finance."
When a new task arrives, a Router Agent looks at the task and says, "This is a sports question. Let's go to the Sports Branch."
The robot then "checks out" that specific branch, uses the tools and rules made just for sports, and ignores the politics rules.
Analogy: It's like a hospital. If you have a broken leg, you go to the Orthopedic wing. You don't go to the Cardiology wing and try to read heart surgery manuals. The system keeps the "wings" separate so they don't get contaminated with the wrong advice.

3. The "Human Safety Net" (Human-in-the-Loop)

Sometimes, the robot runs into a problem it has never seen before, like needing a secret password or a specific website it doesn't know about. The system can't invent this out of thin air.

In these rare cases, the system has a "panic button" that asks a human for a quick hint.
Analogy: If a robot is trying to open a locked door and doesn't have the key, it doesn't keep banging on the door forever. It asks a human, "Do you have the key?" Once the human gives the key, the robot learns to look for keys next time.

What Did They Find? (The Results)

The team tested this system on three very different, long-running streams of tasks:

Prediction Markets: Guessing the outcome of sports and political events.
Security Challenges: Solving computer hacking puzzles (CTF).
Event Forecasting: Predicting future news events.

The Results:

Old Systems: They got good at first, but then their performance dropped. They got "cluttered" and confused.
Adaptive Auto-Harness: It kept getting better over time. By using the "Team of Experts" to build better tools and the "Smart Branching" to pick the right tools, it solved way more problems than the old systems.
The "Human" Factor: When the system got stuck because it lacked a specific piece of information (like a website link), a quick human nudge helped it recover and learn, proving that sometimes a little human help is better than waiting for the AI to figure it out alone.

The Bottom Line

The paper argues that to make AI truly useful in the real world, we can't just give it a static set of rules. We need a system that:

Builds new skills over time using a team of AI workers.
Organizes those skills into separate "rooms" so the AI doesn't get confused.
Switches between those rooms instantly depending on the task.

This allows the AI to keep improving forever without getting "brain fog" from trying to remember everything at once.

Technical Summary: Adaptive Auto-Harness

Problem Definition

The paper addresses the limitations of existing Auto-Harness systems (e.g., A-Evolve, GEPA, Meta-Harness) when deployed in open-ended task streams. While current systems successfully optimize prompts, skills, tools, and memories from execution feedback on fixed offline benchmarks, they fail in dynamic deployment regimes characterized by three specific dimensions:

Unbounded Streams: Tasks arrive continuously without a fixed endpoint, causing history to grow indefinitely. Existing single-agent evolvers struggle to compress this expanding history into finite context windows, creating a bottleneck for effective harness construction.
Task Heterogeneity: Streams contain mixed task types (e.g., politics, sports, finance in prediction markets) requiring distinct sources and tools. A single, static, dense harness optimized for the aggregate stream fails to adapt to specific task contexts at solve time.
Distributional Non-Stationarity: The distribution of incoming tasks shifts over time. A harness optimized for recent cycles drifts out of alignment with new tasks, leading to performance degradation even with rich historical data.

The authors argue that a single, repeatedly updated harness becomes brittle, causing accuracy to peak early and then decline as specialized skills for earlier tasks interfere with later ones (overfitting).

Methodology: Adaptive Auto-Harness

The authors propose Adaptive Auto-Harness, a framework and system designed for sustained self-improvement on open-ended streams. The approach is grounded in an analytical framework that decomposes the performance gap to an "oracle harness" into two loss terms:

Evolution Loss ( $L_{evo}$ ): The gap caused by the evolver's inability to construct necessary capabilities from history (e.g., lacking specific infrastructure or multi-file reasoning).
Adaptation Loss ( $L_{adapt}$ ): The gap caused by committing to a single harness before observing the specific task context.

To minimize these losses, the system introduces three core mechanisms:

1. Sustained Auto-Harness via Multi-Agent Evolution

To reduce $L_{evo}$ , the system replaces stateless, single-agent evolution with a stateful multi-agent evolver operating in a persistent workspace across cycles. This evolver follows a four-phase cycle:

Analyze: An Analyst agent reviews failure patterns and prioritizes gaps, explicitly auditing for "toxic artifacts" (skills/tools that help one regime but hurt others).
Research: Parallel Researcher agents investigate solutions for specific regimes, utilizing a temporal-reveal gate that only exposes outcome labels after task resolution to prevent data leakage.
Build: A Builder agent implements verified solutions (tools, skills, prompts) into the workspace.
Verify: A Verifier agent tests the new code before it is committed to the next cycle.
This architecture includes cross-cycle memory (task boards, research logs, architecture docs) to allow the system to refine construction capabilities over time rather than restarting from scratch.

2. Solve-Time Adaptation via Harness-Tree Routing

To reduce $L_{adapt}$ , the system shifts heavy adaptation to evolution time and uses a lightweight operator at solve time.

Harness Tree: Instead of a single monolithic harness, the evolver constructs a Git-based repository with specialized branches (e.g., branch/crypto, branch/sports). Each branch contains regime-specific prompts, skills, and tools, isolated to prevent cross-contamination.
Agentic Routing: At solve time, a Router agent reads the metadata of each branch (via git show) and routes the incoming task $x_t$ to the most suitable branch based on context. The solver then checks out that branch to execute.

3. Human-in-the-Loop (HITL) Steering

Recognizing that historical experience ( $H_t$ ) may lack critical signals (e.g., API credentials, novel data sources), the system includes a third axis: Human-in-the-Loop.

Task-Board Steering: Humans can review the Analyst's failure board to add entries, adjust priorities, or supply domain guidance.
Interactive Assistance: If a Researcher agent encounters a barrier (e.g., an authentication wall), a hook prompts a human to provide credentials or source directions in real-time.

Key Contributions

Deployment-Regime Analysis: The paper formalizes why static auto-harness evaluation is insufficient for unbounded, heterogeneous, and non-stationary streams, introducing the decomposition of regret into evolution and adaptation losses.
System Architecture: The introduction of a stateful multi-agent evolver, a harness-tree structure for regime isolation, and structurally triggered HITL hooks.
Empirical Validation: Comprehensive evaluation across three distinct streaming benchmarks:
- PolyBench: Prediction markets (politics, sports, finance).
- CTF-Dojo: Security challenges (binary exploitation, cryptography, etc.).
- FutureX: Event forecasting (heterogeneous sources, temporal constraints).

Results

The Adaptive Auto-Harness system outperforms five existing auto-harness baselines (A-Evolve, GEPA, Meta-Harness, Continual Harness, SkillOS) and a human-designed system (OctoTools) across all three benchmarks.

PolyBench: The Full System achieved 80.9% Accuracy and +330% Return (coverage-scaled), significantly outperforming the best baseline (Meta-Harness at 50.8% Accuracy).
CTF-Dojo: The Full System achieved 50.2% Pass@1, surpassing the best baseline (A-Evolve at 45.2%).
FutureX: The Full System achieved 47.3% Pass@1, outperforming A-Evolve (47.5% was close, but the Full System showed better consistency across slices).

Ablation Studies confirmed the necessity of each component:

Multi-Agent vs. Single-Agent: Removing the multi-agent structure or cross-cycle memory significantly degraded performance, particularly in handling complex infrastructure (CTF-Dojo) and confidence calibration (PolyBench).
Routing Effectiveness: Experiments showed that while specialized branches exist (Oracle performance is high), the router captures a substantial fraction of this potential, though some headroom remains.
HITL Impact: Human steering provided significant gains (+20% lift) specifically on slices where historical data lacked necessary external signals (e.g., specific API keys or Chinese-language sources), confirming its role as a supplement to autonomous evolution.

Significance and Claims

The paper claims that Adaptive Auto-Harness provides a viable path for deploying agentic systems in real-world, open-ended environments. Its significance lies in:

Moving beyond static benchmarks: It demonstrates that sustained improvement requires separating the construction of capabilities (evolution) from the selection of strategies for specific tasks (adaptation).
Addressing the "brittleness" of dense harnesses: By using a harness tree, the system prevents the overfitting and interference seen in single-harness approaches where skills for one domain degrade performance in another.
Complementarity of mechanisms: The authors argue that multi-agent evolution, routing, and human steering are complementary, not interchangeable; each addresses a distinct failure mode (capability gaps, task heterogeneity, and missing external signals, respectively).

The work suggests that for long-term deployment, agents must not only learn from history but also maintain a structured, adaptable infrastructure that can evolve without collapsing under the weight of its own accumulated knowledge.