Technical Summary: Self-Evolving Agent Harnesses via Gated Semantic Quality-Diversity

1. Problem Statement

The performance of a Large Language Model (LLM) agent in real-world tasks is determined not only by the model weights but significantly by its "harness"—the surrounding runtime scaffolding including system prompts, injected knowledge, control loops, tool definitions, and configuration. In deployment, model weights are often frozen (due to cost, licensing, or being rented), leaving the harness as the only adjustable lever.

Current approaches to auto-evolving agent harnesses face two primary challenges:

1. Unreliable Feedback: Self-generated feedback is noisy. Apparent gains may be measurement artifacts (e.g., flaky sandboxes, verifier timeouts) or statistical fluctuations rather than genuine improvements.
2. Overfitting and Collapse: Without structured search, optimization loops tend to overfit to specific training tasks or collapse into a narrow set of "safe" edits (typically minor prompt tweaks), failing to address deeper failure modes.

Existing methods often lack statistical rigor, relying on peak pass rates over noisy rounds without held-out validation, making reported gains un-auditable and prone to overfitting.

2. Methodology

The paper proposes a framework that separates the proposal of changes from the crediting of those changes, utilizing a frozen task agent and a separate, stronger "evolver" model.

2.1 System Architecture

• Frozen Task Agent: The model being optimized ($M$) remains frozen. It executes tasks within a mutable harness ($H$).
• Evolver Agent: A separate, stronger model ($D \neq M$) diagnoses failures in the task agent's trajectories and proposes semantic patches.
• Deterministic Measurement Layer: All sampling, scoring, and significance testing are handled by deterministic code, not the LLM, ensuring reproducibility.

2.2 The Self-Evolution Loop

The process follows a funnel structure (Figure 1):

1. Diagnosis: The evolver reads failure traces and assigns a (where $\times$ why) coordinate to each failure.
   • Where: The mutable surface touched (Prompt, Knowledge, Runtime, Config).
   • Why: The specific failure pathology (e.g., "thinking-runaway," "premature finalization").
2. Proposal: The evolver designs patches (git commits) targeting specific pathologies.
3. Preflight Pruning: Zero-inference checks drop candidates that declare no activation or have triggers that never fired.
4. Screening & Validation:
   • Anchor Screen: A cheap subset ($K=1$) filters egregious losers.
   • Full Evaluation: Survivors are run on the full training set ($K=3$).
5. Three-Gate Crediting: A patch is only credited if it passes:
   • Validity Gate: Ensures the environment is healthy (no sandbox crashes/timeouts). Infrastructure failures are re-run, not counted as agent failures.
   • Activation Gate: The patch must have actually fired during the trial.
   • Significance Gate: Requires a paired 2$\sigma$ test ($z \ge 1.96$) on the same task set to distinguish real gains from noise.

2.3 Gated Semantic MAP-Elites (GSME)

To prevent overfitting and search collapse, edits are organized into a categorical Quality-Diversity (QD) archive:

• Indexing: Cells are keyed by the (where $\times$ why) pathology, not the specific task. This acts as an anti-overfitting inductive bias.
• Structure: One "gated elite" (best patch) is kept per cell.
• Selection & Variation: The system uses quality-biased selection (extending the global best) combined with cross-cell recombination (stacking elites from different pathology cells). This allows the system to discover synergistic combinations (e.g., a runtime fix + a prompt fix) without the cost of uniform exploration.

2.4 Evaluation Protocol

• Sealed Test: A held-out test set is completely withheld from the evolver during the entire process. It is scored once after evolution terminates.
• Termination: The loop stops if no candidate beats the vanilla baseline on the training set for $P$ rounds or a round cap $R$ is reached.
• Metric: Performance is measured by Retention (sealed lift / training lift) and Credited Gain (sealed lift passing the 2$\sigma$ threshold).

3. Key Contributions

1. Auditable Self-Evolution Loop: A reproducible system where the LLM owns semantics (diagnosis/patches) and deterministic code owns quantitative judgment (sampling/significance). This ensures every credited improvement is attributable and reproducible.
2. Gated Semantic MAP-Elites (GSME): A novel archive structure keyed on failure pathologies rather than tasks. The paper provides evidence that this structure enables cross-cell recombination, allowing the system to accept diverse edits (runtime, config, knowledge) rather than collapsing to prompt-only tweaks.
3. Generalization Evidence: The framework demonstrates that gains generalize rather than overfit. Across six domains, the sealed-test gains retained 86–147% of the training gains, with all gains clearing the paired 2$\sigma$ significance bar.
4. Model-Specific Pathology Matching: The research shows that the "winning patch" tracks the specific model's dominant failure mode (e.g., "careless $\to$ submit-verify" for some models, "engagement $\to$ verify-finalize" for others), rather than being a universal solution. The transferable contribution is the diagnose-and-credit loop, not a specific harness configuration.

4. Experimental Results

The framework was evaluated across seven domains from three sources (Terminal-Bench-2, EvoAgentBench suite, AppWorld) using a frozen open-weight model (Qwen3.6-27B).

• Performance Gains: The train-selected harness achieved credited gains of +9 to +15.5 percentage points (pp) on the sealed test.
• Generalization:
  • TB2: +9.3 pp (Retention: 147%)
  • LiveCode: +13.7 pp (Retention: 86%)
  • Omni-MATH: +11.7 pp (Retention: 115%)
  • BrowseComp+: +13.9 pp (Retention: 88%)
  • GDPval: +9.2 pp (Retention: 109%)
  • AppWorld: +15.5 pp (Retention: 87%)
  • SWE-bench: +5.1 pp (Not credited due to low sample size $n=26$ failing the 2$\sigma$ bar, though directionally positive).
• Diversity of Edits: Accepted edits spanned all four levers (Prompt, Knowledge, Runtime, Config), avoiding the collapse to prompt-only edits observed in prior work.
• Cross-Model Validation: Experiments on AppWorld with Qwen3.5 and Gemini 3 Flash confirmed that the loop adapts to different model families by identifying their specific dominant pathologies.

5. Significance and Claims

The paper claims that the primary bottleneck in improving frozen LLM agents is not generating changes, but knowing which changes truly helped. By decoupling proposal from crediting and enforcing strict statistical significance on a sealed test, the authors demonstrate that:

• Self-evolution can be made trustworthy by construction, avoiding the overfitting and noise inflation common in prior auto-optimization literature.
• Generalization is achievable in a frozen-weight regime, with evolved harnesses retaining the majority of their training gains on unseen data.
• The diagnose-and-credit loop is the transferable mechanism. The specific patches found are model-specific, but the methodology for finding them applies across different model families and task domains.

The work reframes harness self-evolution not as discovering a "universally good" scaffold, but as fitting corrective patches to a specific model's failure distribution, with the loop itself serving as the robust, generalizable engine for this adaptation.