🤖 AI

Structured Feedback Improves Repair in an LLM Agent Loop

The paper introduces VeriHarness, a code-controlled agent loop demonstrating that providing LLMs with structured feedback containing failure location, observed values, and admissible alternatives significantly improves repair success rates in iterative tasks compared to raw diagnostics or incomplete feedback.

Original authors: Jaideep Ray, Ankit Goyal

Published 2026-07-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jaideep Ray, Ankit Goyal

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a very smart, but slightly literal, robot how to play a video game. You give the robot a goal, like "find the treasure," and it tries to move around the world. Sometimes, the robot makes a mistake—maybe it tries to open a door that's already open, or it tries to pick up an item that isn't there. In the world of Artificial Intelligence, this robot is called an "LLM agent," and the part that checks if the robot's move is legal is called a "validator."

For a long time, when the robot messed up, the validator would just say, "Nope, that's wrong," and let the robot try again. It's like a teacher saying, "Try again," without telling the student why they got the answer wrong or what the right answer might be. The robot would just guess again, hoping to get lucky. But what if the validator could be more helpful? What if it could point exactly to the mistake, show what the robot actually did, and even suggest a list of things the robot could have done instead? This paper asks a simple but powerful question: Does giving the robot a detailed "repair kit" of information help it fix its mistakes faster and better than just getting a simple "try again" message?

The researchers behind this study built a special testing ground called VeriHarness. Think of it as a video game level where a strict referee (the validator) watches the robot (the AI model) play. If the robot makes a bad move, the referee stops the game and hands the robot a note. The team wanted to see what kind of note works best. They tested four different types of notes:

The Raw Note: Just the error message, like "Command 1 is invalid."
The Prose Note: A friendly paragraph explaining where the mistake happened, what the robot saw, and what it should have done.
The Structured Note: The same info as the Prose Note, but formatted like a technical list with labels (like a JSON file).
The "Location Only" Note: Telling the robot where it messed up and what it saw, but without suggesting any alternatives.

They ran this experiment 50 times with two different AI models (one called Qwen and one called Llama) playing text-based adventure games. The goal was to see how often the robot could reach the finish line (win the game) within a limit of four tries.

The results were a huge surprise for the "Raw Note" crowd. When the robot got the detailed notes that included where the mistake happened, what it observed, and a list of admissible alternatives (valid moves it could make instead), it became a champion.

For the Qwen model, the success rate jumped from 28% (with the raw error message) to 72% (with the detailed notes). That's a 44 percentage point improvement.
For the Llama model, it jumped from 16% to 58%, a 42 percentage point boost.

The most exciting part of the discovery is why this worked. The researchers suspected that maybe the robot just liked the neat, organized look of the "Structured Note" (the one with the technical labels). But they tested this by comparing the "Prose Note" (plain English) with the "Structured Note." The result? They performed almost exactly the same. Whether the advice was written in a friendly paragraph or a technical list didn't matter much. The magic ingredient wasn't the formatting; it was the content.

Specifically, the "Location Only" note (which told the robot where it failed but didn't give it new ideas) didn't help much at all. It stayed close to the low success rate of the raw error messages. This proves that simply pointing out the mistake isn't enough; the robot needs to be handed a list of valid alternatives to choose from. It's the difference between a teacher saying, "You spelled 'cat' wrong," versus "You spelled 'cat' wrong; you could have written 'bat,' 'hat,' or 'mat'."

The study also checked if this trick works for coding tasks (using a set of problems called HumanEval). They found that the method only works if the validator can actually see the mistake. If the robot passes a visible test but fails a hidden one that the validator can't see, no amount of feedback can fix it. This suggests that while detailed feedback is a superpower, it's not magic—it can only fix what it can see.

In the end, this paper suggests that if we want our AI agents to be better at fixing their own mistakes, we shouldn't just let them guess again. We need to build validators that act like helpful coaches: pointing out the specific error, showing what actually happened, and offering a menu of correct options to try next. It turns out that for AI, a little bit of specific, actionable advice is worth a thousand "try agains."

Technical Summary: Structured Feedback Improves Repair in an LLM Agent Loop

Problem Statement

While Large Language Model (LLM) agents frequently utilize iterative loops where a model generates a candidate and an external validator rejects it, the interface between the validator and the subsequent model call remains underspecified. Current systems often rely on "raw diagnostics"—simple rejection messages (e.g., "Command 1 is not admissible")—which may lack the specific context required for the model to generate a successful repair. The core question is: What specific information should a deterministic validator return to the next model call to maximize repair success?

Methodology

System Architecture: VeriHarness

The authors introduce VeriHarness, a code-controlled agent loop that separates generation from acceptance.

Orchestration: A principal orchestrator manages state, budgets, and traces outside the model's context window.
Workflow: The LLM worker generates a structured candidate (e.g., a JSON plan). External gates (validators) check the candidate against schema, artifacts, and environment behavior.
Feedback Loop: If a candidate fails, a "failure encoder" maps gate-specific details to a common interface, returning a repair payload to the model. The model receives only the compact task state and the current repair feedback; it cannot self-validate.

Experimental Design

The study evaluates feedback policies across 50 paired TextWorld games (3 rooms, 4 objects, 2-step quests) using two models: Qwen2.5-Coder-14B and Llama-3.1-8B. The primary constraint is a four-call cap per game.

The authors compare four feedback policies (Table 1):

RawDiag: Returns the original validation error string without explicit repair fields.
SameNL: Provides the failure location, observed value, and admissible alternatives in natural language prose, without typed labels.
LocObs: Provides a failure label and named JSON fields for location and observed value, but omits admissible alternatives.
TypedFields: Provides a failure label, named JSON fields for location and observed value, plus a field containing the list of admissible alternatives.

Robustness Checks

Budget Sensitivity: Tested call budgets of 2, 4, 6, and 8 on a 15-game subset.
Decoding Variance: Tested Qwen with sampled decoding (temperature 0.3, top-p 0.9) across three seeds.
Scope Check: A HumanEval experiment where the validator only sees one public assertion, ensuring the mechanism is tested only when a failure is actually exposed.

Key Results

1. Structured Feedback Significantly Improves Success

Under a four-call cap, feedback containing location, observed value, and admissible alternatives drastically outperforms raw diagnostics.

Qwen2.5-Coder-14B: Success rate increased from 28% (RawDiag) to 72% (TypedFields) (+44 percentage points).
Llama-3.1-8B: Success rate increased from 16% (RawDiag) to 58% (TypedFields) (+42 percentage points).
Statistical Significance: These improvements are statistically significant (Holm-adjusted $p < 0.001$ ) with 95% bootstrap intervals of [28–60] for Qwen and [28–56] for Llama.

2. The Source of Improvement: Repair Values vs. Format

Ablation studies isolate the cause of the performance gain:

Admissible Alternatives are Critical: The LocObs policy (location + observation, but no alternatives) performed near the RawDiag baseline (36% vs. 28% for Qwen; 18% vs. 16% for Llama). This indicates that simply knowing where and what failed is insufficient without knowing what to do instead.
Format is Secondary: SameNL (prose) and TypedFields (keyed JSON) performed nearly identically.
- Qwen: SameNL (70%) vs. TypedFields (72%).
- Llama: SameNL (58%) vs. TypedFields (58%).
- Statistical tests show no significant difference ( $p=1.0$ ) between prose and JSON formats when the repair values are matched. The JSON format did not provide a reasoning advantage, though it slightly reduced the total number of calls required.

3. Budget and Decoding Robustness

Budget Sensitivity: The performance gap between structured feedback and raw diagnostics widens as the call budget increases from 2 to 4. RawDiag performance plateaus after 4 calls, while structured feedback continues to improve with additional calls.
Decoding: The ordering of results (TypedFields/SameNL > LocObs > RawDiag) persisted under sampled decoding settings.

4. Limitations Exposed by HumanEval

In the HumanEval scope check, the first answer passed the visible test for all 15 tasks, meaning the validator never returned a repair opportunity. Consequently, all policies stopped after one call with identical success rates (14/15). This confirms that structured feedback cannot repair failures that the validator does not detect or expose.

Significance and Claims

The paper claims that the primary bottleneck in LLM agent repair loops is not the model's ability to reason, but the informational content of the feedback.

Validator Design: Validators should return specific repair information—specifically the failure location, the observed value, and a list of admissible alternatives—rather than generic rejection messages.
Format Agnosticism: The specific syntax (JSON vs. prose) is less important than the presence of the repair values themselves. While JSON aids in logging and routing, it does not inherently improve model reasoning capabilities in this context.
Conditional Utility: The mechanism is effective only when the validator can expose the failure. If a validator cannot enumerate alternatives or detect the specific error, the repair loop cannot function.

The authors conclude that for agent design, the focus should shift from optimizing prompt orchestration to engineering deterministic validators that return actionable, enumerated alternatives upon failure.