🤖 AI

Harness-1: Reinforcement Learning for Search Agents with State-Externalizing Harnesses

Harness-1 is a 20B reinforcement learning-based search agent that offloads routine state management to an external stateful harness while retaining semantic decision-making, achieving superior retrieval performance and strong generalization across diverse benchmarks compared to existing open and frontier models.

Original authors: Pengcheng Jiang, Zhiyi Shi, Kelly Hong, Xueqiang Xu, Jiashuo Sun, Jimeng Sun, Hammad Bashir, Jiawei Han

Published 2026-06-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Pengcheng Jiang, Zhiyi Shi, Kelly Hong, Xueqiang Xu, Jiashuo Sun, Jimeng Sun, Hammad Bashir, Jiawei Han

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a massive, complex mystery. You have a brilliant detective (the AI model), but they have a terrible habit: they forget everything they just read, get confused by too much paperwork, and often waste time re-reading the same clues over and over.

The paper introduces a new system called Harness-1 that solves this problem by giving the detective a super-organized assistant (the "Harness").

Here is the breakdown of how it works, using simple analogies:

1. The Problem: The "Brilliant but Forgetful" Detective

Usually, when an AI tries to search for answers, it acts like a detective who has to remember everything in their head. They have to remember:

Which documents they already read.
Which ones were useful and which were trash.
Which facts they still need to find.
How much time they have left.

The paper argues that forcing the AI to do all this "bookkeeping" (organizing the files) is a waste of its brainpower. It's like asking a genius mathematician to also act as the filing clerk. They get tired, make mistakes, and stop searching effectively.

2. The Solution: The "Stateful Harness" (The Super Assistant)

Harness-1 splits the job into two distinct roles:

The Policy (The Detective): This is the AI model (a 20-billion parameter model). Its only job is to make smart decisions: "What should I search for next?" "Is this document important?" "Do I have enough evidence to stop?"
The Harness (The Assistant): This is a specialized software program that runs around the AI. It does all the boring, mechanical work. It keeps a perfect list of every document found, tags them with importance levels (like "Very High" or "Low"), connects the dots between different documents, and remembers exactly what has been verified.

The Analogy: Think of the AI as a chef and the Harness as the sous-chef.

The Chef (AI) decides: "I need to chop onions and check if the sauce is ready."
The Sous-chef (Harness) actually does: "Here is the bowl of chopped onions, I've already sorted the good tomatoes from the bad ones, and I've written down that we are out of salt."
The Chef doesn't have to worry about where the onions are or how many tomatoes are in the bin; they just focus on cooking.

3. How They Learn Together (Reinforcement Learning)

The team trained this system using a method called Reinforcement Learning.

The Training: They let the AI play the "detective" thousands of times. Every time the AI made a good move (finding the right clue, organizing the files well), it got a reward.
The Twist: Because the Harness handled the messy details, the AI learned much faster. It didn't have to waste energy trying to remember where it put a file; it just had to learn which files mattered.
The Result: The AI learned to be a master strategist. It learned to search broadly, then narrow down, verify facts, and stop exactly when it had the answer.

4. The "Magic" Features of the Assistant

The Harness isn't just a notepad; it has special tools that make the detective smarter:

The "Evidence Graph": Imagine a detective's corkboard with red string connecting clues. The Harness automatically draws these strings. If Document A mentions "Brussels" and Document B mentions "Brussels," the Harness connects them. This helps the AI see the big picture without reading every single word again.
Auto-Seeding: When the search starts, the Harness doesn't leave the detective staring at an empty desk. It immediately puts the top 8 most likely documents on the desk as a "starting point." This prevents the AI from getting stuck at the beginning.
Compression: If a search returns a 50-page report, the Harness summarizes it into the 4 most important sentences before showing it to the AI. This keeps the AI's "working memory" from getting clogged.

5. The Results: Small Model, Big Wins

The paper tested Harness-1 on eight different difficult search challenges (like finding financial data, patent info, and multi-step trivia).

The Surprise: A relatively small AI model (20 billion parameters) using this Harness performed better than much larger, more expensive "frontier" models (some with 120+ billion parameters) that didn't have this special assistant.
The Generalization: Even when the AI was tested on topics it had never seen during training (like moving from finance questions to history questions), it still performed incredibly well. This proves that the AI learned the skill of searching, not just memorized answers.

Summary

Harness-1 is a new way of building AI search agents. Instead of asking the AI to do everything (think, search, and organize), it gives the AI a powerful, stateful assistant to handle the organization. This allows a smaller, cheaper AI to outperform massive, expensive models by focusing its brainpower on making the right decisions rather than remembering the wrong details.

The paper's claim: By offloading the "bookkeeping" to the environment (the Harness), the AI learns to search more effectively, generalizes to new topics better, and achieves higher accuracy than current state-of-the-art methods, all while using a smaller model.

Technical Summary: Harness-1

Problem Statement

Search agents, typically implemented as language models that interact with retrieval tools, are often trained to operate over growing transcripts. In this formulation, the policy must simultaneously manage semantic search decisions (e.g., what to query, which documents to read) and routine state management (e.g., remembering seen documents, tracking constraints, maintaining a curated set of evidence). The authors argue that forcing the policy to reconstruct useful state from raw, append-only observations is inefficient and leads to poorly conditioned reinforcement learning (RL). As transcripts grow, hard queries may yield nearly identical empty-set rewards, tool vocabularies may collapse into repetitive search calls, and the distinction between search failure and curation failure becomes obscured.

Methodology: Harness-1

The paper introduces Harness-1, a 20B-parameter search agent trained with reinforcement learning inside a stateful search harness. The core principle is stateful cognitive offloading: the environment (harness) maintains the recoverable state, while the policy retains only the semantic decisions.

The Stateful Harness

The harness maintains a WORKINGMEMORY with two tiers:

Inner Tier (Prompt-facing): A compact, rendered summary of the search state including:
- Candidate Pool ( $P_t$ ): Compressed and deduplicated search results.
- Curated Set ( $C_t$ ) & Importance Tags ( $I_t$ ): A capped set (max 30 docs) of selected documents with tags (very high, high, fair, low).
- Evidence Graph ( $G_t$ ): A summary of cross-document links based on entities, dates, and bridge documents.
- Verification Records ( $V_t$ ): Logs of claims checked against source text.
- Search History ( $H_t$ ) & Budget Markers ( $B_t$ ): Summaries of progress and context limits.
Outer Tier (Full Text): A full-text store ( $D_t$ ) of retrieved chunks, accessible via review_docs without new corpus calls.

Policy Actions

The policy operates by emitting structured actions that edit this state rather than extending a transcript:

Retrieval: fan_out_search, search_corpus, grep_corpus, read_document.
Curation: curate (add/remove documents and assign importance tags).
Verification: verify (check policy-written claims against stored documents).
Termination: end_search.

Training Pipeline

Supervised Fine-Tuning (SFT): A teacher agent (GPT-5.4) runs inside the full harness to generate trajectories. The policy learns the interface mechanics: tool-call formats, the search-to-curate rhythm, importance tagging, and verification norms.
Reinforcement Learning (RL): The policy is trained on-policy using CISPO over full search episodes. The reward function combines:
- Set Quality: Weighted recall of the final curated set.
- Trajectory Coverage: Credit for finding evidence anywhere in the pool.
- Answer Evidence: Rewards for including answer-bearing documents and penalties for missing them.
- Tool Diversity: A bonus for using a diverse set of tools to prevent collapse into repetitive searching.
- Turn Penalty: To encourage efficiency.

Key Design Requirements

The authors identify three requirements for a stateful search harness to be trainable:

Warm-started Curation: Auto-seeding the curated set with the top results from the first successful search to provide an initial learning signal.
Compact Derived-State Rendering: Summarizing state via importance tags, evidence graphs, and verification records to avoid context bloat.
Diversity-Preserving Incentives: Reward structures that encourage a rhythm of searching, curating, reviewing, and verifying.

Key Contributions

Formulation: The paper frames harness design as a central problem in retrieval-agent RL, proposing "stateful cognitive offloading" to separate semantic decisions from bookkeeping.
Harness-1 System: A 20B search agent featuring an importance-tagged curated set, compact evidence graph, verification records, and budget-aware context rendering.
Trainability Requirements: Identification of warm-starting, compact rendering, and diversity incentives as necessary conditions for effective RL in stateful search.
Empirical Validation: Demonstration that stateful harnessing improves retrieval quality and downstream answer accuracy across eight diverse benchmarks.

Results

Harness-1 was evaluated on eight benchmarks spanning web, finance, patents, and multi-hop QA (e.g., BrowseComp+, SEC, HotpotQA).

Performance: Harness-1 achieved an average curated recall of 0.730, outperforming the next strongest open search subagent (Tongyi DeepResearch 30B) by +11.4 points. It remains competitive with much larger frontier models (e.g., GPT-5.4, Opus-4.6), surpassing several in average curated recall.
Transfer Generalization: The model showed particularly strong gains on held-out transfer benchmarks (LongSealQA, Seal0QA, FRAMES, HotpotQA) which were not used in SFT or RL training. The average gain on these held-out tasks (+17.0 points over Context-1) was 2.2x larger than the gain on source-family benchmarks (+7.9 points), suggesting the learned operations over explicit state generalize better than domain-specific patterns.
Ablation Studies: Disabling individual harness mechanisms (e.g., importance tags, evidence graph, auto-seeding) led to significant drops in Final-Answer Recall (up to -7.9%). Disabling all mechanisms simultaneously caused a -12.2% drop in Recall, confirming that the performance gains stem from the harness design rather than inherent model capability alone.
Training Dynamics: Experiments showed that without a tool-diversity reward, the policy collapsed into a search-heavy strategy with low curation quality. The diversity bonus was essential for stabilizing tool use and improving final recall.

Significance and Claims

The paper claims that stateful harness design is a critical, previously underexplored axis for retrieval-agent RL. By offloading mechanical bookkeeping to the environment, the policy can focus on high-level semantic decisions, leading to more stable training and better generalization.

The authors assert that Harness-1 demonstrates that a smaller model (20B) with a well-designed stateful interface can outperform larger open models and compete with frontier models in complex search tasks. The results suggest that the "interface itself" is a central part of what the policy learns to use, rather than merely a tool wrapper. The work points toward a future where retrieval agents are defined by the synergy between a trainable policy and a stateful, cognitive-offloading harness.

The paper concludes by releasing the model weights, harness code, data-generation pipeline, and RL recipe to facilitate further research in this direction.