💬 NLP

HarnessForge: Joint Harness and Policy Evolution for Adaptive Agent Systems

HarnessForge is a meta-adaptive framework that jointly evolves the external execution harness and internal reasoning policy of LLM agents through fault-guided tailoring and harness-conditioned alignment, significantly outperforming isolated adaptation methods by optimizing their executable compatibility across diverse task regimes.

Original authors: Mingju Chen, Can Lv, Guibin Zhang, Heng Chang, Shiji Zhou

Published 2026-06-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mingju Chen, Can Lv, Guibin Zhang, Heng Chang, Shiji Zhou

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Rigid Robot"

Imagine you hire a brilliant but inexperienced robot assistant (an LLM Agent) to do different jobs.

Job A: You need it to book a flight. It needs a strict checklist and a calendar.
Job B: You need it to write a mystery story. It needs a creative flow and a memory of plot points.
Job C: You need it to fix a leaky pipe in a simulation. It needs to know exactly which tools to grab and in what order.

The problem is that most robot assistants come with a fixed instruction manual (called a Harness). This manual tells the robot how to think and act.

If the manual is too rigid, the robot fails at creative tasks.
If the manual is too loose, the robot gets confused during complex, step-by-step tasks.

Traditionally, researchers tried to fix this in two separate ways:

Rewrite the Manual: Keep the robot the same, but try to write a better instruction manual for every new job. (This is slow and often misses the mark).
Train the Robot: Keep the manual the same, but try to "teach" the robot to be smarter through trial and error. (This is expensive and the robot might still get confused by a bad manual).

HarnessForge says: "Why choose one? Let's evolve both together."

The Solution: A "Tandem Dance"

The authors propose a system called HarnessForge. Think of the Agent System as a Dance Pair:

The Harness (The Choreographer): This is the external structure. It decides the steps, the music, the rules, and the tools the dancer can use.
The Policy (The Dancer): This is the robot's brain. It decides how to actually move its feet to follow the choreography.

In the past, people tried to fix the Choreographer or train the Dancer separately. HarnessForge realizes they are coupled. A great choreography is useless if the dancer can't do the moves. A talented dancer is useless if the choreography is nonsensical.

HarnessForge evolves them together in a loop:

Step 1: The "Fault Diagnosis" (Finding the Stumble)

The system runs the dance pair through a series of tasks. When they stumble (fail), a "Meta-Agent" (a super-smart referee) looks at the footage.

Did the dancer trip because they were tired? (Policy issue)
Did the dancer trip because the choreographer gave them a step that was physically impossible? (Harness issue)
Did they trip because the music stopped at the wrong time? (Memory/Structure issue)

Step 2: Tailoring the Harness (Rewriting the Choreography)

Based on the diagnosis, the system automatically rewrites the Harness.

If the robot kept forgetting the plan, the Harness is updated to add a "sticky note" system (Memory).
If the robot kept picking the wrong tool, the Harness is updated to add a "safety guard" that checks the tool before use.
Analogy: It's like a coach watching a soccer team lose a game and immediately changing the formation or the playbook to fix the specific weakness.

Step 3: Aligning the Policy (Training the Dancer)

Once the new Harness (choreography) is ready, the system doesn't just throw the old robot at it. It fine-tunes the robot specifically for this new set of rules.

It takes the successful attempts from the previous round and teaches the robot: "Hey, when the Harness says 'Check the tool,' you should do this specific movement."
Analogy: It's like a dance instructor teaching a specific routine to a dancer, ensuring their muscle memory matches the new steps perfectly.

Step 4: The "Survival of the Fittest"

The system keeps the best pairs (Harness + Robot) and discards the ones that still fail. It repeats this process over several rounds. With every round, the choreography gets smarter, and the dancer gets better at following that specific choreography.

Why This Works (The Results)

The paper tested this on five different "dance floors" (benchmarks) involving things like:

Using tools to solve puzzles.
Searching the web for answers.
Calling APIs (digital tools) to get movie data.

The Findings:

Better Together: Systems that evolved both the Harness and the Policy together performed significantly better than those that only changed one or the other.
Efficiency: It didn't require millions of extra tries (rollouts) to work. Because the system learns from the structure of the failure, it learns faster.
Compatibility is Key: The biggest takeaway is that a "perfect" robot doesn't exist in a vacuum. A robot is only as good as the system it operates in. By making the robot and its system evolve together, they become perfectly compatible.

Summary Analogy

Imagine you are trying to build the ultimate Swiss Army Knife.

Old Way: You either try to make the handle (Harness) perfect, or you try to make the blade (Policy) sharper, but you never change both at the same time.
HarnessForge Way: You realize that if you make the handle ergonomic for a left-handed carpenter, you must also adjust the angle of the blade. You build a workshop where the handle and the blade are forged together, tested, and refined in a loop until they fit each other perfectly.

The result is a tool that is perfectly adapted to the specific job it needs to do.

Technical Summary: HarnessForge

Problem Statement

Large Language Model (LLM) agents are increasingly deployed in heterogeneous task regimes (e.g., multi-step tool use, retrieval-heavy reasoning, web interaction) that demand distinct execution paradigms. Existing adaptation strategies typically suffer from a "compatibility gap" because they optimize either the external execution structure (the harness) or the internal reasoning behavior (the policy) in isolation.

Search-style methods optimize external workflows, tool-use procedures, and memory management but often leave the compatibility with the model-side executor implicit.
Training-style methods adapt the model policy via supervised learning or reinforcement learning but typically assume a fixed, externally specified interaction loop.

This component-level view is insufficient for system-level meta-adaptation. A more expressive harness may fail if the reasoner cannot execute its protocols, while a stronger policy may be constrained by a harness exposing unsuitable states. The paper argues that effective adaptation requires treating the agent system as a coupled harness–policy pair and optimizing their joint evolution to ensure executable compatibility.

Methodology: HarnessForge

HarnessForge is a meta-adaptive framework that formulates an LLM agent system $G$ as a pair $(H, R_\delta)$ , where $H$ is the editable execution harness and $R_\delta$ is an adapted reasoner (a base model $R_{\theta_0}$ with a lightweight adapter $\delta$ ). The framework operates through iterative rounds of harness–policy co-evolution.

1. System Formulation

The harness $H$ is decomposed into three execution-layer components:

Planning ( $P$ ): Task decomposition, replanning, and termination.
Action ( $A$ ): Tool interfaces, role assignment, and orchestration rules.
Memory ( $M$ ): What is written, retrieved, summarized, and exposed to future decisions.

The reasoner $R_\delta$ executes within the constraints of $H$ . The goal is to evolve both $H$ and $\delta$ jointly to maximize a fitness indicator $J$ based on task performance, token cost, and latency.

2. Co-Evolution Workflow

Each evolution round $r$ alternates between two mutually reinforcing processes:

A. Fault-Guided Harness Tailoring

Instead of random search, HarnessForge uses a meta-agent to diagnose and refine the harness structure:

Fault Attribution: The system evaluates current harness–policy pairs on a batch of tasks. A meta-agent analyzes rollout traces and execution statistics to attribute failures specifically to Planning, Action, or Memory components.
Archive-Guided Improvement: A historical archive $Z$ of harness designs and their evaluation vectors is maintained. The meta-agent retrieves Pareto-optimal exemplars relevant to the current faults and generates an improvement report.
Refine-and-Filter: Based on the report, the meta-agent generates revised harness candidates (editing only $P, A, M$ ). These candidates undergo a staged, budgeted Pareto filtering process (half-selection) to retain the most promising survivors for the next stage.

B. Harness-Conditioned Policy Alignment

Once a set of survivor harnesses is selected, the framework aligns the reasoning policy to them:

Parent Initialization: For each survivor harness, the policy is initialized from its parent lineage (folding the previous adapter).
Trajectory Curation: The framework reuses the successful rollout traces generated during the harness filtering stage. It does not require a separate data collection phase.
Adapter Training: A new, harness-specific LoRA adapter is trained via supervised fine-tuning (SFT) on the curated successful trajectories. The input includes the task instruction, the active harness interface, accumulated observations, and memory state, ensuring the policy learns to execute specifically under the evolved harness.

Key Contributions

Reformulation of Adaptation: The paper shifts the unit of optimization from isolated components to the harness–policy pair, explicitly addressing the compatibility gap between external structures and internal reasoning.
HarnessForge Framework: A meta-adaptive co-evolution mechanism that coordinates fault-guided harness tailoring with harness-conditioned policy alignment. This ensures that structural improvements in the harness are matched by corresponding adaptations in the policy.
Empirical Validation: Extensive evaluation across five benchmarks (ToolHop, SearchQA, RestBench-TMDB, API-Bank) and two backbone models (Qwen3-4B, Qwen3-8B).

Experimental Results

Performance Gains: HarnessForge consistently outperforms both search-style baselines (e.g., AFlow, AgentSquare, MaAS) and training-style baselines (e.g., SFT, GRPO, RLOO).
- It achieves an average improvement of 3.56% over the strongest baseline across all metrics.
- It reaches up to 12.0% improvement on specific benchmarks (e.g., TMDB with Qwen3-4B).
Ablation Studies: Removing either harness tailoring or policy alignment consistently degrades performance. The gap widens over evolution rounds, confirming that the gains stem from the co-evolution of the pair rather than independent optimization.
Efficiency: HarnessForge demonstrates favorable rollout–performance tradeoffs. Unlike policy-only RL methods that require massive rollout budgets to explore behaviors under a fixed interface, HarnessForge reuses rollout evidence to improve the harness and then aligns the policy, achieving strong results with significantly fewer rollouts (e.g., 12K rollouts for SFT vs. 45.6K for RL-based baselines in Round 3).
Compatibility: Analysis of cross-combinations of evolved harnesses and policies shows that matched pairs perform significantly better than mismatched ones, validating that the framework induces pair-specific compatibility gains rather than just producing generally stronger components.

Significance and Claims

The paper claims that executable compatibility between the harness and the reasoning policy is the critical factor for effective agent-system adaptation. By treating the agent as a coupled system and evolving the interface and the executor together, HarnessForge enables agents to meta-adapt to diverse task regimes more effectively than methods that optimize structure or policy in isolation.

The authors note that while the framework is currently evaluated on resource-constrained backbones (Qwen3-4B/8B), where the coupling is most salient, the magnitude of these gains for frontier-scale models remains an open direction. Additionally, the current implementation uses structured edit operators for harness evolution to ensure auditability and executability, though future work could explore broader code-level rewrites.