💻 computer science

Feedback That Backfires: Why Small Language Model Agents Repeat the Call They Just Watched Fail

This paper reveals that small language model agents are significantly more likely to repeat failed tool calls when the failure is recorded verbatim in the transcript, a counterproductive effect driven primarily by the surface form of the failed action rather than the error message, which can be effectively mitigated by replacing the raw call with a runtime-generated description of the failure.

Original authors: Esmail Gumaan

Published 2026-08-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Esmail Gumaan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a digital assistant that can use tools to solve problems, like looking up a contact, checking a calendar, or writing a piece of code. To do this, the assistant follows a simple loop: it thinks of an action, tries to perform it, and then reads the result. If the action fails, the system records exactly what the assistant tried to do and the error message it received, then asks the assistant to try again. This method, known as the agent loop, is the standard way these systems learn from their mistakes. The logic seems sound: if you tell a person "you tried to open a locked door and it didn't work," they will not try the same locked door again. They will look for a different key or a different door. For years, engineers have assumed that small computer models, which power many of these assistants, would behave the same way. They believed that showing the model the error message would teach it to avoid that specific mistake.

A researcher at the University of Passau in Germany decided to test this assumption with extreme precision. They did not just watch the assistants fail and succeed; they measured the exact mathematical likelihood of the model choosing the same wrong action before and after seeing the error. They ran these tests on six different small computer models, ranging from very tiny to moderately sized, across two different environments: one where the models tried to use simulated office tools, and another where they attempted to fix broken computer programs. The researcher wanted to know if the error message actually corrected the model's behavior.

What they found was the opposite of what everyone expected. Instead of learning from the failure, the models became significantly more likely to repeat the exact same mistake. When the system showed the model the failed attempt and the error message, the model's internal probability of choosing that same failed action again jumped dramatically. In the tool-calling tests, the chance of the model repeating the failed call rose from a low six percent to over fifty percent. In nearly every single case the researcher tested, the error message did not act as a warning; it acted as a magnet, pulling the model back toward the very action it had just failed to perform.

The researcher then asked why this was happening. They suspected that the models might simply be too small to understand the error message. However, their investigation revealed a different culprit. They discovered that the problem was not the meaning of the error, but the presence of the text itself. When the failed action was written into the record, the model's internal machinery, which is designed to copy patterns it sees, latched onto the text of the failed action. This copying effect was so strong that it overwhelmed the actual message about the failure. Even when the researcher replaced the long, detailed error message with a simple note saying "this failed," the model still repeated the mistake. But when they removed the text of the failed action entirely and replaced it with a description of what went wrong, the model stopped repeating the error.

This finding upended the standard way these systems are built. The common advice for fixing a stuck agent has been to delete the failed attempt from the history and let the model start fresh, hoping to clear the "contamination." The researcher found that this was actually the worst possible thing to do. By deleting the failure, the system restored the exact conditions that caused the mistake in the first place, guaranteeing that the model would make the same error again. The solution, they discovered, was not to delete the history, but to change it. If the system kept the record of the failure but replaced the raw text of the failed command with a description generated by the system, the repetition stopped.

The study also tested a more obvious fix: simply telling the model in its instructions not to repeat the failed action. This approach, which sounds logical to a human, had almost no effect. The model could not easily follow a rule that told it to ignore a specific string of text that was sitting right in front of it. The researcher concluded that the issue was not a lack of intelligence in the model, but a flaw in how the information was presented. The standard method of showing the failed action alongside the error message creates a powerful urge to copy, which is stronger than the urge to learn from the mistake.

By running these experiments on a standard computer processor without specialized graphics cards, the researcher proved that this behavior is a fundamental property of how these small models work, not a glitch that requires massive computing power to fix. Their work suggests that to build reliable digital assistants, engineers must stop treating the failed action as a helpful lesson to be read and start treating it as a dangerous pattern to be hidden. The fix is structural: the context after a failure must look different from the context before it, but the difference cannot be the failed action itself. By removing the raw text of the mistake and keeping only the diagnosis, the system can break the cycle of repetition and allow the model to actually move forward.

Technical Summary: Feedback That Backfires

Problem Statement

Agent frameworks typically follow a ReAct-style loop where a failed tool call and its corresponding error message are appended verbatim to the transcript, and the model is asked to continue. The prevailing assumption is that the error message serves as corrective information, guiding the model away from the failed action.

This paper challenges that assumption. It investigates whether recording a failed call and its error actually reduces the probability of the model repeating that specific action. The author hypothesizes that for small language models (SLMs), the presence of the failed action's token sequence in the context may trigger copying mechanisms that override the semantic understanding of the error, leading to "feedback inversion"—where the model becomes more likely to repeat the failure after seeing the error.

Methodology

Experimental Setup

The study utilizes two distinct environments to ensure robustness across different action types:

ToolShed: A simulated office environment with 12 typed tools (files, contacts, calendar, etc.). Actions are Python-style function calls.
CodeRepair: A program repair task using the sanitized MBPP dataset. Actions are full Python functions, and observations are real interpreter outputs (tracebacks).

The author tested 6 instruction-tuned checkpoints across 4 model families (Llama-3.2, Qwen2.5, Qwen3, SmolLM2), ranging from 135M to 1.7B parameters. All experiments were conducted on a single CPU (Intel Core i7-6820HQ) without GPUs.

Measurement Protocol

The core metric is the Corrective Gain ( $G$ ), defined as the change in the log-probability of re-emitting a specific failed action ( $a_\times$ ) after the failure record is appended to the context ( $C$ ):
$G(a_\times) = \log \pi(a_\times | C) - \log \pi(a_\times | C \oplus (a_\times, o_\times))$

$G > 0$ : The harness successfully suppresses the failed action.
$G < 0$ : Feedback inversion (the model is more likely to repeat the failure).

To decompose the cause of $G$ , the author employs counterfactual observations:

$\Delta_{copy}$ (Surface-form term): The effect of the failed action's token sequence appearing in the context, measured against a "valence-free" observation that acknowledges the call but reports neither success nor failure.
$\Delta_{sem}$ (Semantic term): The additional effect of explicitly marking the call as failed, measured by comparing the failure observation against the valence-free one.

Interventions

The paper evaluates several harness variants to test the decomposition:

Verbatim: Standard ReAct transcript.
+Instruction: Verbatim plus a system prompt instruction "do not repeat."
Abstract: The verbatim call is replaced by a runtime-generated description of the failure (diagnosis only, no token sequence).
+Ban: A decoder-level constraint that masks the final token of any previously failed string during generation.
Drop: The failed attempt is deleted from the context before retrying (clean restart).

Key Results

1. Universal Feedback Inversion

Across all 6 models and both environments, the corrective gain $G$ is consistently negative.

Magnitude: The average gain is approximately $-1.03$ nats per action token (totaling $-17.38$ nats for tool calls). This represents a factor of 2.8 increase in the odds of repeating the action.
Frequency: The effect holds on 90%–100% of individual items, not just as an average.
Behavioral Impact: In a fixed candidate set of four actions, the probability of re-emitting the failed call rises from 0.06 (before the attempt) to 0.54 (after the failure). Under greedy decoding, the exact failed call is reproduced token-for-token in 19% of items after a failure, compared to 0% before.

2. Decomposition: Form vs. Meaning

The decomposition reveals that the surface-form term ( $\Delta_{copy}$ ) accounts for approximately 83% of the inversion effect.

Copying Dominance: The mere presence of the failed string in the context drives the repetition.
Semantic Weakness: The semantic term ( $\Delta_{sem}$ ) is small and inconsistent. On ToolShed, it is slightly positive (error messages nudge the model toward the failed call, likely because explaining the error requires naming the tool again). On CodeRepair, it is slightly negative (tracebacks push the model away), but this effect is overwhelmed by the copying term (median ratio 61:1 in favor of surface form).
Conclusion: The problem is not that models fail to read error messages; it is that the copying mechanism outvotes the semantic understanding.

3. Evaluation of Interventions

Natural Language Instructions: Adding a "do not repeat" instruction has negligible effect (slightly increasing repetition in some cases), consistent with literature on negative instructions failing when the prohibited string is present in the context.
Context Clearing (Drop): Deleting the failed attempt and retrying from a clean context is the worst performing harness. It restores the exact context that produced the failure, guaranteeing the same action is emitted again. This increases the exact repeat rate to 80%.
Structural Fixes (Abstract & Ban):
- Abstract: Replacing the verbatim call with a runtime-generated description removes 76% of the inversion. In probe scoring, this drives the exact greedy repeat rate to zero; however, in free-running rollouts, the exact repeat rate is reduced to 16% (a paired difference of −14 points, with an interval touching zero).
- Ban: Blocking previously-failed strings at the decoder significantly reduces exact repetition (from 31% to 8% in rollouts).
- Combined: Using both yields the lowest loop rates (4%).

4. Task Success vs. Repetition

While structural interventions (Abstract, Ban) effectively eliminate repetition loops, they do not improve task success at this scale (0.5B models).

The "do not repeat" instruction, while failing to stop repetition, surprisingly increased task success by 17 points, likely by making the model more cautious or less prone to abandoning tasks early.
The paper concludes that at the 0.5B scale, repetition is a significant but not exclusive source of failure. Removing the loop returns the agent's budget to it, but the model's ability to generate a correct alternative call remains a separate limitation.

Significance and Claims

The paper claims to identify a specific, structural flaw in standard agent harnesses that has been misdiagnosed as a model capability issue.

Diagnosis Shift: The failure of agents to recover from errors is not primarily due to a lack of error understanding in small models, but rather a "copying curse" induced by the harness design. The error message is real information, but the verbatim inclusion of the failed action acts as a stronger, competing signal.
Design Principle: The context after a failure must differ from the context before it, but the difference must not be the failed action itself. Simply removing the failure (clean restart) is insufficient; the failed action's surface form must be obscured or made unreachable.
Practical Remedies: The author proposes two effective, low-cost remedies:
- Runtime Description: Replacing the verbatim call with a generated description of the failure.
- Decoder Constraints: Banning previously-failed strings at the decoding stage.
Limitations: The paper explicitly states that while these remedies stop the loop, they do not solve the underlying task failure for small models. The "do not repeat" instruction, while ineffective at stopping loops, may have other beneficial behavioral effects on task success that are distinct from the repetition mechanism.

The study emphasizes that the "harness" is the deciding factor in agent reliability for small models, and that structural changes to the context or decoder are required to break the cycle of repetition, rather than relying on better error messages or stronger models alone.