💻 computer science

Better Harnesses, Smaller Models: Building 90% Cheaper Agents via Automated Harness Adaptation

This paper demonstrates that automatically adapting agent harnesses to compensate for the limitations of small language models can enable them to match frontier LLM performance on routine business tasks while reducing inference costs by up to 90%.

Original authors: Chenyang Yang, Xinran Zhao, Tongshuang Wu, Christian Kästner

Published 2026-07-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Chenyang Yang, Xinran Zhao, Tongshuang Wu, Christian Kästner

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a super-smart, expensive robot chef (a "Frontier LLM") that can cook a five-star meal for a company. It's amazing, but it costs a fortune to run—like $0.22 just to ask it to chop an onion. Then, you find a tiny, budget-friendly robot (a "Small Language Model" or SLM) that costs almost nothing to run. You swap them out, expecting the same delicious results. But disaster strikes! The tiny robot gets confused, burns the toast, and sends the wrong invoice. It only gets the job right 75% of the time, while the big robot was hitting 97%.

This paper asks: Can we make the tiny robot as good as the big one without spending the big money?

The answer is a resounding yes, but with a twist. You can't just swap the robots; you have to build a better kitchen around the tiny one. The authors call this a "harness." Think of the harness as a set of training wheels, a recipe book, and a safety net all rolled into one.

The Big Discovery: The "90% Cheaper" Magic

The researchers found that by automatically redesigning the kitchen (the harness) specifically for the tiny robot, they could make it perform just as well as the expensive giant. In their best experiment, the tiny robot recovered 89.7% of the big robot's performance while costing only 4% as much. That's a 90% cost reduction for nearly the same result!

They didn't guess how to fix the kitchen. They built a "Meta-Agent"—a super-smart robot supervisor that watched the tiny robot fail, figured out why it failed, and then automatically rewrote the kitchen rules.

How the "Kitchen Fix" Works

The paper breaks down why the tiny robot fails and how the harness fixes it, using three main tools:

The Recipe Book (Context): Sometimes the tiny robot just doesn't know the rules or the ingredients. The harness adds detailed instructions, examples, and "cheat sheets" directly into its brain.
The Specialized Tools (Tools): If the tiny robot gets overwhelmed by a drawer full of 40 different knives, the harness locks away the ones it doesn't need and gives it just the one perfect knife it knows how to use.
The Safety Net (Hooks): If the tiny robot starts spinning in circles (like sending the same email twice), the harness has a "stop button" that catches the mistake before it happens.

What Makes a Fix Work? (The Rules of the Game)

The paper suggests that this magic trick doesn't work for every job. It found two big rules:

Repetition is Key: The harness works best on jobs where the steps are the same every time, like auditing attendance records or approving budget requests. If the job is chaotic and changes every time (like refactoring a messy codebase), the harness struggles to cover all the bases. The paper found that for the most repetitive tasks, the tiny robot's accuracy jumped by 21.1% more than for the most chaotic ones.
The Robot Needs a Brain: The tiny robot still needs to be smart enough to start with. If the robot is too weak, no amount of training wheels will help. The paper suggests that models with stronger base capabilities benefit the most, seeing a 48.8% boost in performance compared to only 15.5% for weaker models.

What the Paper Says "No" To

The authors are careful to point out what doesn't work or isn't the answer:

No "One-Size-Fits-All": You can't just take a harness built for one tiny robot and slap it onto another. Different tiny robots fail in different ways (one might hate JSON, another might hate file editing), so the harness must be custom-built for each specific model.
No "Magic Sub-Robots": The researchers tried to see if creating a team of tiny robots (sub-agents) would help, but the automated optimizer didn't find this strategy successful. The paper suggests this is because the tiny robots aren't good at managing other robots yet.
No "Free Lunch": While the running cost is tiny, there is a one-time cost to build the harness. However, the paper notes that this cost pays for itself after just 13 runs on average.

The Bottom Line

This paper proves that we don't need to rely on the most expensive, powerful AI to get business tasks done. By using a smart, automated system to build a custom "harness" around a cheaper, smaller model, we can get 90% of the performance for 10% of the price.

It's not a magic wand that fixes everything instantly; it's a systematic way to take a budget-friendly robot, give it the right tools and instructions, and turn it into a reliable worker for routine business tasks. The paper suggests that for repetitive jobs, this approach is a game-changer for making AI affordable.

Technical Summary: Better Harnesses, Smaller Models

Problem Statement

Frontier Large Language Models (LLMs) power many agentic systems but incur substantial inference costs, high latency, and data privacy concerns, making large-scale deployment unsustainable. Small Language Models (SLMs) offer a cheaper, faster, and more private alternative. However, naively swapping an SLM into a harness designed for a frontier LLM typically results in significant performance degradation. The core challenge is that SLMs often lack the general reasoning or tool-use capabilities required to execute complex, open-ended workflows without assistance. This paper investigates whether automated harness adaptation can bridge the performance gap, allowing SLMs to match frontier LLM performance at a fraction of the cost for routine business tasks.

Methodology

1. Theoretical Framework: Failure Modes and Adaptations

The authors first establish a framework mapping agent failure modes to harness adaptation strategies. They categorize failures by the underlying model capability they implicate:

Tool-use: Selecting wrong tools or malformed calls.
Instruction-following: Violating system/user constraints.
Knowledge: Lacking domain or commonsense assumptions.
Long-context: Forgetting instructions or losing track in long trajectories.
Planning/Reasoning: Failing to decompose goals or replan.

Corresponding adaptation strategies are grouped into three categories:

Context Adaptations: Modifying prompts, adding examples, externalizing knowledge, or compressing context.
Tool Adaptations: Wrapping complex sequences into simpler tools, filtering tool sets, or encoding constraints programmatically.
Agent Loop Adaptations: Adding deterministic hooks (e.g., anti-loop checks), orchestrating sub-agents, or instrumenting code to enforce constraints.

2. Automated Harness Optimization

To avoid laborious manual engineering, the authors developed a harness optimizer driven by a meta-agent.

Search Space: Defined via the software-agent-sdk, allowing modifications to system prompts, tools (primitive and custom), hooks, context management, and sub-agents.
Optimization Loop:
1. Sample & Evaluate: The system samples a harness from a pool (using a Pareto-front genetic search) and evaluates it on a batch of training instances, logging full trajectories.
2. Diagnose & Propose: The meta-agent analyzes failure trajectories, the current harness code, search memory (to avoid redundant fixes), and design-space documentation. It proposes targeted edits (e.g., adding a tool, rewriting a prompt).
3. Validate & Save: Proposed harnesses undergo a sanity check, are re-evaluated, and added to the pool if they improve validation scores.

3. Experimental Setup

Tasks: Seven diverse, business-oriented agentic tasks (e.g., budget approval, attendance auditing, code refactoring) with varying levels of workflow diversity.
Models: Three SLM families (gemma-4-26b-a4b, qwen3-coder-30b-a3b, ministral-3-8b) compared against a frontier LLM baseline (gemini-3.1-pro-preview).
Metrics: Accuracy, inference cost (USD), and latency.

Key Contributions

Empirical Evidence: A systematic study demonstrating that optimized harnesses can recover 89.7% of frontier LLM performance using SLMs at 4% of the cost (a 90%+ cost reduction).
Framework: A mapping of agent failure modes to specific harness adaptation strategies, providing a diagnostic tool for understanding why an SLM fails and how to fix it.
Automation: A meta-agent-powered harness optimizer that automatically discovers effective adaptations without manual intervention.
Task Suitability Analysis: Identification that adaptation is most effective for tasks with repetitive workflows (low diversity) and for SLMs with stronger base capabilities.

Results

Performance and Cost

Gap Closure: Optimized harnesses significantly improved performance on 16 out of 21 task-SLM pairs.
Parity: Seven task-SLM pairs achieved performance parity with the frontier LLM baseline.
Cost Efficiency: The best adapted SLM agent (gemma-4-26b-a4b) recovered 89.7% of LLM performance at 4% of the cost. The one-time offline optimization cost ($20 per task) is amortized after an average of 13 runs.

Factors Influencing Effectiveness

Task Diversity: There is a strong negative correlation (Spearman $\rho = -0.96$ ) between task diversity and adaptation success. Tasks with stable, repetitive workflows (e.g., budget approval) benefit significantly, while highly diverse tasks (e.g., code refactoring) are harder to adapt.
Model Capability: SLMs with stronger base capabilities benefit more from adaptation (+48.8% improvement) compared to weaker models (+15.5%). Harnesses can offload difficulty but cannot fully replace missing core capabilities (e.g., complex coding or long-context reasoning).
Dominant Strategies: The most common successful adaptations addressed instruction-following (81%) and knowledge failures (81%), primarily through adding contexts (86%) and creating/managing tools (43%/29%). Notably, the optimizer rarely succeeded in creating sub-agent systems, likely due to current SLM limitations in managing agent coordination.

Model-Specific Nuances

Different SLMs exhibit distinct failure patterns (e.g., ministral-3-8b struggling with file editing vs. qwen3-coder emitting raw XML instead of JSON). This necessitates tailored harnesses for each model; a "one-size-fits-all" harness does not exist.

Significance and Claims

The paper claims that harness adaptation is a viable strategy for expanding the practical deployment range of SLM agents in routine business contexts. It argues that for tasks where difficulty is shared across instances, this difficulty can be "lifted" from the model into the harness via tailored instructions, tools, and orchestration.

The authors emphasize that this approach allows businesses to move away from expensive frontier LLMs for routine workflows without sacrificing reliability. They suggest that future SLM development should prioritize harness compatibility (robustness to various prompts and tools) alongside raw benchmark scores. Furthermore, they posit that automated harness optimization is an emerging field where the quality of diagnosis (via a strong meta-agent) is more critical than the quantity of search iterations.

The paper concludes modestly, noting that while the results are promising for repetitive tasks, the approach may not yet fully solve highly diverse or open-ended agentic challenges, and that the "bitter lesson" of automation suggests search-based methods will continue to outperform manual engineering.