AutoHarness: improving LLM agents by automatically synthesizing a code harness

This paper demonstrates that a smaller language model (Gemini-2.5-Flash) can automatically synthesize code harnesses or complete policies through iterative refinement, effectively preventing illegal actions and outperforming significantly larger models across various TextArena games while offering greater cost efficiency.

The Gist

Here is an explanation of the AutoHarness paper, translated into simple language with creative analogies.

The Big Idea: Teaching a Genius to Follow the Rules

Imagine you have a brilliant, hyper-intelligent chess player (the LLM, or Large Language Model). This player knows every strategy in the book, can calculate complex future moves, and understands the deep philosophy of the game.

However, there is a catch: This genius is terrible at following the basic rules.

In a recent competition, this "genius" lost 78% of its games not because it made a bad strategic choice, but because it tried to move a Knight like a Bishop, or moved a piece off the board entirely. It was like a grandmaster trying to play chess but accidentally knocking the pieces off the table because they forgot how a piece moves.

Usually, to fix this, humans have to write a "rulebook" (called a harness) that acts as a referee. This referee checks every move the genius makes. If the move is illegal, the referee says, "Nope, try again." But writing these rulebooks by hand is slow, boring, and you have to do it from scratch for every new game.

AutoHarness changes the game. Instead of a human writing the rulebook, they ask the genius itself to write its own rulebook.

---

How It Works: The "Code as a Harness" Concept

Think of the AI agent as a Driver (the LLM) and a Car (the game environment).

• The Problem: The Driver is great at navigating, but they keep trying to drive through walls or onto the sidewalk because they don't know the car's physical limits.
• The Old Solution: A human mechanic builds a custom bumper guard for every single car model.
• The AutoHarness Solution: You give the Driver a piece of paper and a pen and say, "Write a set of instructions that tells you exactly how to check if a move is safe before you make it. If you make a mistake, I'll tell you, and you'll rewrite your instructions."

The Process: A Tree of Guesses

The researchers didn't just ask the AI to "try harder." They used a clever search method called Tree Search (think of it like a "Choose Your Own Adventure" book, but for code).

1. The First Draft: The AI writes a piece of code (a "harness") that tries to check if a move is legal.
2. The Test Drive: They run the game. The AI tries to make a move.
3. The Critic: If the AI tries to move a piece illegally, the environment yells, "Illegal move!"
4. The Revision: The AI looks at the error, says, "Ah, I forgot to check if the path is clear," and rewrites the code to fix that specific mistake.
5. Repeat: They do this over and over, branching out different ideas (like exploring different paths in a maze) until the code is perfect.

Eventually, the AI synthesizes a perfect "filter" or "harness" that catches every illegal move before it happens.

---

The Results: Small Brain, Big Win

The most surprising part of the paper is the outcome.

• The Setup: They used a smaller, cheaper AI model (Gemini-2.5-Flash) to write the harness.
• The Result: Once this small model wrote its own "rulebook," it became better at playing games than a much larger, more expensive, and "smarter" model (Gemini-2.5-Pro) that didn't have a custom harness.

Analogy:
Imagine a Junior Mechanic (the small model) who builds a perfect, custom safety harness for a race car. Once the car is equipped with this harness, it drives perfectly.
Meanwhile, a World-Famous Racing Legend (the large model) is driving a car with no safety harness. The legend is faster and smarter, but they keep crashing into walls because they aren't checking their mirrors.
Result: The Junior Mechanic's car wins the race because it never crashes, while the Legend keeps hitting the walls.

The "Super" Version: The Self-Driving Car

The researchers pushed this idea even further. Instead of just writing a rulebook to check moves, they asked the AI to write the entire strategy in code.

• Normal Agent: The AI looks at the board, thinks, "I should move here," and then asks the harness, "Is this legal?"
• AutoHarness Agent: The AI writes a Python script that is the strategy. The script calculates the best move and executes it instantly.
• The Benefit: Once the code is written, you don't need the expensive AI brain anymore! You just run the code. It's like teaching a robot to walk, and then letting the robot walk on its own without a human holding its hand.

In tests, this "code-only" strategy beat the world's most advanced AI models (including GPT-5.2) in single-player games, and it did so for almost zero cost because the expensive AI wasn't needed during the actual game.

Why This Matters

1. Cost Efficiency: You can use a cheap, small AI to build a custom tool that makes it perform like a super-AI.
2. Reliability: It solves the "hallucination" problem where AI makes up rules. The code acts as a strict, mathematical gatekeeper.
3. Scalability: Instead of humans writing rules for 1,000 different games, the AI can learn to write the rules for all of them automatically.

Summary

AutoHarness is like giving a brilliant but clumsy artist a set of self-correcting tools. The artist (the AI) uses those tools to build a safety net for themselves. Once the net is built, the artist can perform at a world-class level without ever making a basic mistake, proving that sometimes, the best way to make an AI smarter is to let it build its own guardrails.

Technical Summary

Here is a detailed technical summary of the paper "AutoHarness: improving LLM agents by automatically synthesizing a code harness."

1. Problem Statement

Large Language Models (LLMs) have shown remarkable capabilities in code synthesis and reasoning, yet their performance as agents in interactive environments remains brittle. A primary failure mode is the generation of illegal actions (actions prohibited by the environment's rules) rather than suboptimal strategic moves.

• Evidence: In the Kaggle GameArena chess competition, 78% of losses for the Gemini-2.5-Flash model were due to illegal moves, not strategic errors.
• Limitations of Current Solutions:
  • Fine-tuning: Costly, slow, and risks degrading performance on other tasks (catastrophic forgetting).
  • Manual Harnesses: Writing custom "harnesses" (code wrappers that validate moves) for every new game is labor-intensive and does not scale.
• Core Challenge: How to bridge the gap between an LLM's apparent understanding of rules and its ability to strictly adhere to them in a scalable, automated manner without relying on massive model sizes.

2. Methodology: AutoHarness

The authors propose "Code as Harness," a framework where the LLM itself synthesizes the code wrapper (harness) that governs its interaction with the environment. This transforms the agent into a hybrid system of an LLM and a generated program.

A. Core Framework

The system treats harness generation as a search problem over the space of programs.

1. Tree Search with Thompson Sampling: Instead of simple iterative prompting, the method employs a tree search guided by Thompson sampling to explore the landscape of potential code harnesses.
2. Iterative Refinement Loop:
   • Base LLM: Acts as a "mutation operator," proposing code refinements based on feedback.
   • Environment (Critic): Executes the code and provides feedback on whether moves were legal and what reward was obtained.
   • Heuristic: The search prioritizes nodes based on the average legal move accuracy.
3. Refinement Logic:
   • If is_legal_action() returns True but the move is invalid: Refine both the action proposer and the legality checker.
   • If is_legal_action() returns False and the move is invalid: Refine only the action proposer (propose_action).

B. Harness Architectures

The paper explores three variations of the harness:

1. Harness-as-Action-Filter: The LLM generates a set of legal moves, and the harness filters/ranks them (using Chain-of-Thought).
2. Harness-as-Action-Verifier (Primary Focus): The LLM proposes a move; the generated code (is_legal_action) verifies it. If invalid, the system retries with a prompt containing an "illegal action" warning.
3. Harness-as-Policy: The LLM synthesizes a complete policy in code. Once generated, the policy runs as pure Python (using standard libraries like numpy) without further LLM calls at inference time.

3. Experimental Setup

• Dataset: 145 games from TextArena (including Chess, Checkers, Sudoku, 2048, etc.).
• Constraint: The authors removed "Available Moves" hints from the observation strings to force the agent to deduce legal actions from environmental feedback, simulating real-world scenarios.
• Models:
  • Training: Gemini-2.5-Flash.
  • Evaluation: Compared against Gemini-2.5-Flash (vanilla), Gemini-2.5-Pro, and GPT-5.2 variants.
• Metrics:
  • 1-Player (1P): Average reward.
  • 2-Player (2P): Win/Draw/Loss rates.
  • Legal Action Rate: Percentage of moves that do not violate environment rules.

4. Key Results

A. Elimination of Illegal Moves

• The synthesized harness achieved a 100% legal action success rate across all 145 TextArena games.
• Training converged quickly: On average, only 14.5 tree search iterations were required. 19 out of 32 evaluated games converged in under 10 iterations.

B. Performance vs. Larger Models

• 2-Player Games: The smaller Gemini-2.5-Flash + Harness outperformed the much larger Gemini-2.5-Pro.
  • Win rate against Pro: 56.3% (vs. Pro's 38.2%).
  • Win rate against vanilla Flash: 64.8%.
• 1-Player Games: The harness approach achieved a higher average reward (0.745) than Gemini-2.5-Pro (0.707) and vanilla Flash (0.673).

C. Harness-as-Policy (Code-Only Inference)

• In the extreme case where the entire policy is synthesized as code (no LLM at inference):
  • Reward: Achieved 0.870 average reward, outperforming GPT-5.2-High (0.844) and Gemini-2.5-Pro (0.707).
  • Cost Efficiency: The inference cost was near zero (pure Python execution), whereas running GPT-5.2-High for the same experiments cost approximately $640.

5. Significance and Contributions

1. Scalability: Demonstrates that a smaller, cheaper model (Flash) can outperform larger, more expensive models (Pro, GPT-5.2) when augmented with an automatically synthesized code harness.
2. Robustness: Solves the "action applicability" problem by offloading rule enforcement to verifiable code rather than relying on the LLM's internal (and often hallucinated) world model.
3. Cost Efficiency: The "Harness-as-Policy" approach eliminates the need for LLM inference during decision-making, drastically reducing latency and cost while improving performance.
4. Generalization: The method is domain-agnostic, successfully applied to 145 diverse text-based games without manual intervention for specific game rules.

Conclusion

AutoHarness represents a paradigm shift from relying solely on LLM reasoning to LLM-guided code synthesis. By using the LLM to write its own "guardrails" (the harness), the system ensures strict adherence to environment rules, enabling smaller models to achieve state-of-the-art performance in complex game environments at a fraction of the computational cost.