Technical Summary: Test-Time Harness Evolution (TTHE)

Problem Statement

Modern Large Language Model (LLM) agents are increasingly constructed as executable systems where behavior is governed not only by the underlying frozen model but also by the surrounding "harness"—the control program that manages context construction, tool invocation, intermediate verification, and failure recovery. Current deployment practices typically involve optimizing this harness on labeled development or training data before freezing it for evaluation. This static approach limits adaptability when test distributions, failure modes, or tool interactions differ from those seen during development.

Existing adaptation mechanisms, such as fine-tuning or test-time training, modify model weights, while self-debugging or reflection methods often revise single responses or store textual experiences. Automated workflow searches usually occur pre-deployment. A critical open question remains: Can an LLM agent adapt during evaluation solely from its own unlabeled execution traces by modifying its surrounding executable harness, without updating model weights, requiring gold labels, or training a separate adaptation model?

Methodology: Test-Time Harness Evolution (TTHE)

TTHE addresses this by treating the executable harness as the state of test-time adaptation. Instead of searching for a fixed workflow on labeled data, TTHE moves harness optimization into the evaluation process itself, operating on unlabeled test batches.

Core Mechanism

TTHE maintains a population of candidate harnesses and refines them through an agentic loop consisting of three roles instantiated around the same frozen LLM:

1. Solver: The agent executing the task within the current harness.
2. Proposers: Agentic coding agents that reason over execution traces to rewrite the harness code.
3. Judge: An agentic selector that commits an improved harness based on execution-derived signals.

The Evolution Loop

For each unlabeled batch of inputs $X_t$:

1. Observation: The current committed harness $H_t$ is executed on $X_t$, generating detailed execution traces (prompts, tool calls, stdout/stderr, errors, runtime states) and lightweight proxy signals.
2. Population Evolution: TTHE evolves $G$ parallel branches over $R$ rounds. In each round, $G$ proposers edit their respective parent harnesses into child harnesses. Proposers read traces from other branches for context but edit their own lineage. This ensures both lineage (persistence of useful strategies) and diversity (exploration of different edit directions, e.g., conservative repair, exploration, adversarial).
3. Selection: A judge inspects the final set of candidate harnesses, their traces, and proxy signals to commit a single harness $H_{t+1}$ for the next batch.
4. Measurement: Gold labels are consulted only after selection to measure accuracy; they never enter the adaptation loop.

Optimization Signals

Since gold labels are unavailable during adaptation, TTHE relies on execution-derived proxy signals, which are imperfect evidence rather than correctness oracles:

• Execution Health: Whether the output runs without error and returns a well-formed result.
• Round-Trip Consistency: Whether the output can be inverted into a description that matches the input (detecting semantic drift).
• Public-Test Pass Rate: The fraction of public examples or repository tests passed (incomplete evidence as hidden tests may still fail).

The system avoids collapsing these into a single scalar reward, instead exposing raw traces and signals to the agentic proposer and judge to prevent gaming and systematic error repetition.

Key Contributions

1. Formulation of Test-Time Harness Evolution: The paper defines a new adaptation paradigm where the persistent executable harness is adapted during evaluation from unlabeled traces, with model weights fixed and gold labels isolated to post-selection measurement.
2. Population-Based Generate-and-Judge Loop: The authors realize this as a system where the solver, proposers, and judge are different roles around the same frozen LLM, enabling adaptation entirely through changes to the surrounding program.
3. Empirical Validation: The method is demonstrated across five execution-grounded domains (Text-to-SQL, competitive programming, software engineering, data-science coding, and agentic tool-use), producing inspectable policies for grounding, verification, and repair.
4. Diagnostic Analysis: The paper provides ablation studies on search budget, batch size, and model backbones, and performs trace audits to characterize failure modes, specifically identifying "selection regret" (good candidates not committed) and limited candidate coverage.

Results

TTHE was evaluated on hard slices of five benchmarks: BIRD (Text-to-SQL), LiveCodeBench, SWE-bench Verified, DS-1000, and claw-eval (agentic tool-use).

• Performance Gains: Using a DeepSeek-v4-flash backbone, TTHE improved the baseline ReAct harness significantly:
  • BIRD: 12.0% $\to$ 50.0%
  • LiveCodeBench: 30.0% $\to$ 38.3%
  • SWE-bench Verified: 20.0% $\to$ 35.0%
  • DS-1000: 38.0% $\to$ 44.0%
  • claw-eval: 48.9% $\to$ 69.8% (mean graded score)
• Model Independence: Gains were consistent across different backbones (MiMo V2.5, Kimi K2.5), indicating the loop is not tied to a specific model.
• Emergent Policies: The evolved harnesses introduced inspectable behaviors not present in the baseline, such as strict reproduce-first workflows, execute-and-self-debug loops for SQL, and conditional verification-and-repair policies for coding.
• Ablation Insights:
  • Search Budget: Performance is non-monotonic with search budget; parallel lineages ($G=3$) outperform single lineages ($G=1$) because they prevent the loss of good early candidates.
  • Batch Size: Performance peaks at a moderate batch size ($B=10$), balancing evidence per decision against the number of adaptation steps.
  • Accumulation: Carrying the committed harness across batches yields higher accuracy than resetting to the baseline for each batch.

Significance and Limitations

The paper posits that TTHE recasts test-time adaptation for LLM agents as evolution over executable control programs. It demonstrates that persistent, inspectable improvements can be achieved without weight updates or gold labels.

However, the authors identify central technical bottlenecks:

• Proxy Reliability: The quality of execution-derived signals is critical. The system must optimize under incomplete evidence where a query may execute successfully but answer the wrong question.
• Judge Reliability: A significant portion of potential gains is lost to "selection regret," where the agentic judge commits a plausible but incorrect program or fails to select a better candidate from the pool. The judge is measurably miscalibrated, often preferring benchmark-incompatible output shapes.
• Coverage Limits: Even with an oracle over all generated candidates, some tasks remain unsolved, indicating intrinsic limits in the current search space coverage.

The authors conclude that while TTHE offers a practical mechanism for adapting agents without changing model weights, the robustness of such evolution depends heavily on the reliability of proxy signals and the selection mechanism. Future work is needed to establish forward generalization via prequential scoring and to address the selection bottleneck.