💻 computer science

Harnesses for Inference-Time Alignment over Execution Trajectories

This paper introduces an inference-time trajectory alignment framework to analyze harness engineering for LLM agents, revealing that optimal performance often requires partial rather than fully structured workflows by balancing task decomposition and guided execution to avoid failure modes like over-decomposition and hallucination.

Original authors: Boyuan Wang, Bochao Li, Minghan Wang, Yuxin Tao, Fang Kong

Published 2026-05-22

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Boyuan Wang, Bochao Li, Minghan Wang, Yuxin Tao, Fang Kong

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: The "Over-Parenting" Problem

Imagine you are teaching a very smart but inexperienced robot to bake a complex cake. You have two ways to help it:

The "Full Manual" Approach: You write a 50-step recipe. Step 1: "Preheat oven." Step 2: "Open fridge." Step 3: "Take out eggs." Step 4: "Crack egg." And so on, all the way to the frosting.
The "Partial Guide" Approach: You give the robot the first three steps ("Preheat, get ingredients, crack eggs") and then say, "Okay, you know the rest. Go finish the cake."

The paper argues that more instructions do not always mean a better cake. In fact, if you give the robot too many tiny, specific steps, it might get confused, make small mistakes that add up, or get stuck trying to follow a rule that doesn't fit the situation. Sometimes, giving the robot less structure and letting it figure out the middle part on its own actually leads to a better result.

The authors call this "Harness Engineering." A "harness" is just a fancy word for the set of rules, sub-tasks, and guidance you give an AI to help it solve a big problem.

The Two Main Levers

The paper breaks down how we help AI into two distinct tools:

1. The Map (Task Decomposition)

This is breaking a big job into smaller chunks.

The Analogy: Imagine hiking a mountain.
- Too Coarse: You tell the hiker, "Climb the mountain." (Too vague; they might get lost).
- Too Fine: You tell the hiker, "Take one step forward. Now take another. Now lift your left foot. Now put it down." (Too micromanaged; the hiker gets confused and trips).
- Just Right: You give them checkpoints: "Walk to the tree, then to the rock, then to the summit."
The Finding: The paper proves that the size of these checkpoints matters. If the checkpoints are too big, the AI can't reach them. If they are too small, the AI accumulates tiny errors (like rounding errors in math) that ruin the final result. The "sweet spot" depends on how capable the specific AI is.

2. The Compass (Guidance)

This is the advice you give the AI while it is working on a specific step.

The Analogy: Imagine you are driving a car in fog.
- Aligned Guidance: You say, "Stay in the lane, but watch out for the fog." This helps you stay safe.
- Misaligned Guidance: You say, "Drive fast and ignore the fog because the map says you're fine." This might make the car crash even faster because the AI is following your rule instead of the reality of the road.
The Finding: Guidance only helps if it matches the evidence available. If you tell an AI to "be confident" when it doesn't have enough information, it will start hallucinating (making things up). If you tell it to "check the facts," it performs better. The strength of the guidance doesn't matter as much as whether the guidance is correct for the situation.

The Surprising Discovery: "Partial Harnessing"

This is the paper's most counter-intuitive finding.

Usually, people think: "If a little bit of help is good, a lot of help must be great."
The paper says: No.

They found that the best strategy is often Partial Harnessing.

The Analogy: Think of a GPS.
- Full Harness: The GPS tells you every single turn for the entire 3-hour drive. "Turn left in 500 feet. Merge right in 200 feet." If the GPS gets a signal glitch in the middle, or if there's a sudden road closure, the whole plan breaks because the AI is too busy following the rigid script.
- Partial Harness: The GPS says, "Drive to the highway entrance, then merge onto I-95." Once you are on the highway, the AI takes over and figures out the rest of the drive based on real-time traffic.
The Result: In their experiments, AI agents that were given only the first few steps and then left to finish the job on their own often succeeded more often than agents given a full, detailed plan. The AI needed just enough structure to get started in the right direction, but then needed the freedom to adapt to the rest of the journey.

Why Does This Happen?

The authors use a concept called "Recoverability."

Imagine the AI is walking a tightrope.
If you give it a rigid script (too many steps), and it makes one tiny mistake, it falls off the rope because the script doesn't allow for correction.
If you give it a partial guide, it can stumble, realize it's off-track, and use its own brain to fix the mistake and get back on the rope.

Summary of the Rules

Based on their math and experiments, the paper suggests three rules for building AI helpers:

Match the Step Size to the AI: Don't break a task into steps that are too big for the AI to handle, or so small that it gets lost in the details. Find the "Goldilocks" size for that specific AI.
Match the Advice to the Facts: Only give advice that is supported by what the AI can actually see. If you tell it to guess, it will lie.
Stop Helping When It's Time: Don't write the whole script. Give the AI the starting point, let it run for a bit, and then let it finish the job. The most effective harness is often the shortest one that keeps the AI from falling off the cliff.

In short: Don't over-engineer the AI's path. Give it a good start, a good compass, and then let it drive.

Technical Summary: Harnesses for Inference-Time Alignment over Execution Trajectories

Problem Statement

Large Language Model (LLM) agents increasingly rely on "harnesses"—inference-time scaffolding strategies that inject human priors into execution via task decomposition and guided execution. While the prevailing intuition suggests that more elaborate harnesses (finer decomposition, stricter guidance) yield better performance, empirical observations and historical AI lessons suggest otherwise: excessive structure can limit an agent's ability to search, adapt, and scale, sometimes reducing final task success.

The core problem addressed is determining what a harness should specify and what should be left for the agent to resolve autonomously. Current design practices often treat structure addition as a linear optimization problem, lacking a theoretical framework to predict when additional scaffolding becomes counterproductive (e.g., causing over-decomposition, over-pruning, or hallucinated execution).

Methodology

The authors propose a theoretical framework modeling harness design as an inference-time trajectory alignment problem. They decompose a harness into two distinct mechanisms operating on different timescales:

Workflow (Outer Timescale): Controls task decomposition, structuring a task into a sequence of sub-goals ( $g_1 \to g_2 \to \dots \to g_T$ ). This defines the granularity of the plan.
Guidance (Inner Timescale): Reshapes the local action distribution during the execution of each sub-goal. This is modeled as a reweighting of the agent's base trajectory distribution based on a guidance rule $\psi$ and strength $\lambda$ .

Theoretical Framework

The paper introduces a recoverability framework where final success is equivalent to the prefix remaining "recoverable" (capable of reaching the final answer) at every stage. Using this, they derive three alignment principles:

Granularity–Capability Alignment: The scale of sub-goals must match the agent's controllable progress scale within a given retry budget. If sub-goals are too large (coarse), the agent cannot reach them; if too small (fine), rounding errors and coordination loss accumulate.
Guidance–Evidence Alignment: Guidance improves performance only if it biases the agent toward trajectories supported by current task evidence. Misaligned guidance (e.g., rewarding instruction compliance over evidence) creates a negative "retention gap," amplifying hallucinations.
Partial Harnessing (Marginal Stopping): A harness need not cover the entire execution path. The optimal design stops scaffolding when the marginal reduction in tail risk (from adding a stage) no longer exceeds the reliability cost of that stage.

Experimental Validation

The authors validate these principles through:

Synthetic Experiments: A controlled cumulative-progress task where agents must reach a target sum using truncated Gaussian action distributions. They vary decomposition granularity ( $K$ ), guidance quality (aligned vs. misaligned pruning), and partial harness length.
Real-World Benchmarks:
- Terminal-Bench v2: Evaluating workflow granularity on software engineering tasks.
- Plotly Chart Analysis: Measuring hallucination rates under varying guidance conditions.
- Case Study: A specific task (llm-inference-batching-scheduler) comparing full vs. partial workflows.

Key Contributions

Formalization of Harness Design as Alignment: The paper decomposes harnesses into workflow (decomposition) and guidance (action reweighting), establishing two stage-level principles: sub-goal scale must match agent capability, and guidance must match task evidence.
Theoretical Bounds on Failure Modes:
- Theorem 1: Derives a bound showing that final success probability drops when the granularity penalty (mismatch between requested progress and reachable scales) exceeds the retry budget's capacity.
- Theorem 2: Proves that guidance improves stage recoverability if and only if the "retention gap" (log-ratio of weights on recoverable vs. non-recoverable trajectories) is positive. Negative gaps lead to hallucination.
Partial Harnessing Strategy: The authors introduce Partial Harnessing, a design strategy where the harness specifies only an initial prefix of the trajectory. They formalize a marginal stopping rule: extend the harness only while the next scaffolded stage saves more residual risk than it introduces.
Empirical Evidence of Non-Monotonicity: Experiments demonstrate that pass rates are unimodal with respect to both decomposition granularity and harness length. Over-specification (too many steps or too much guidance) consistently degrades performance compared to optimal partial configurations.

Results

Granularity: In synthetic and real-world tasks, pass rates peak at a specific number of sub-goals. Finer decomposition (beyond the peak) leads to accumulated terminal bias and lower pass rates due to the agent's inability to stop precisely at arbitrary milestones.
Guidance: Aligned guidance (prioritizing evidence-based actions) improves pass rates and reduces hallucination as guidance strength increases. Conversely, misaligned guidance (prioritizing instruction compliance without evidence) causes hallucination rates to rise monotonically with guidance strength.
Partial Harnessing: On the llm-inference-batching-scheduler task, a partial harness (specifying only the first 3 steps) outperformed a fully specified 10-step workflow. The full harness caused the agent to get "stuck" in repeated intermediate revisions, whereas the partial harness allowed the agent to enter a productive iterative refinement loop for the remaining steps.
Agent Capability: Stronger agents (with lower autonomous tail costs) reach the optimal partial harness length earlier (at fewer scaffolded steps) than weaker agents.

Significance and Claims

The paper argues that harness design should shift from the problem of "adding more structure" to the problem of "choosing what to specify, how strongly to specify it, and when to stop."

The authors claim that:

Stronger is not always better: More elaborate harnesses can act as reliability bottlenecks rather than aids if they misalign with the agent's capabilities or the available evidence.
Partial specification is optimal: Effective harnesses do not need to cover the full execution path. Once the remaining task falls within the agent's autonomous capability, continued control adds cost without benefit.
Design as a Stopping Problem: The "right" harness is the smallest one that keeps the agent on a recoverable trajectory. This reframes scaffolding as a marginal trade-off between tail-risk reduction and the reliability cost of imposed structure.

The paper concludes by noting that while their simplified settings isolate core mechanisms, real-world agent behavior is more complex, and extending these principles to richer execution dynamics remains a direction for future work.