Aborted but Not Forgotten: KV-Cache Retention Breaks Rollback Consistency in Language Agents
This paper reveals that retaining key-value (KV) cache states across logical aborts in language agents creates a critical "rollback consistency" vulnerability where models continue attending to discarded content, a structural flaw demonstrated across multiple model families that can be mitigated by restoring transaction-local cache states rather than relying solely on transcript cleanup.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Modern language agents are software programs designed to act like human assistants, capable of planning, using tools, and making decisions to complete complex tasks. To function efficiently, these agents often maintain a running memory of their conversation, a digital transcript that allows them to remember what was said a moment ago. A critical assumption built into the design of these systems is that if an agent makes a mistake or explores a dangerous path, a developer can simply tell the system to "undo" that action. The expectation is that the system will erase the bad path from its memory and continue as if it never happened, returning to a safe, clean state. This concept of a perfect rollback is essential for safety; without it, an agent could accidentally send a sensitive email or transfer money to the wrong person, and the system would have no way to truly revert the error. However, the internal mechanics of how these agents process information are far more complex than the simple text they display to users.
Researchers have discovered a hidden flaw in how these systems handle the "undo" command. While the visible conversation log might be cleared of a rejected idea, the underlying engine that powers the agent often keeps a hidden, technical record of that discarded thought to save time. This record, known as a key-value cache, is a temporary storage area that allows the computer to skip repetitive calculations and respond faster. The problem arises when an agent is told to abort a specific branch of thought: the software deletes the text from the user's view, but the engine fails to delete the corresponding data from its hidden cache. Consequently, the agent's brain is still looking at the discarded information, even though the user's screen shows a clean slate. The researchers call this a "rollback consistency" failure, where the logical state of the application does not match the physical state of the model's attention.
To prove this invisible gap exists, the researchers designed a rigorous test that separated the effect of the text from the effect of the hidden cache. They created two identical scenarios where an agent was asked to make a decision, such as sending a message to a specific recipient. In both scenarios, the agent was first fed a piece of information that suggested a different, unauthorized recipient. This information was then placed in a "tentative" branch of the conversation, which the system was instructed to reject and delete. In the first scenario, the researchers allowed the engine to keep its hidden cache intact, meaning the rejected information remained in the background memory. In the second scenario, they forced the engine to rebuild its memory from scratch, ensuring the rejected information was truly gone. Crucially, in both cases, the actual text fed to the agent at the moment of decision was identical and contained no trace of the unauthorized recipient.
The results were stark and consistent across a wide range of different AI models. In twenty-five out of sixty-three specific test cases, the agent made a dangerous mistake only when the hidden cache was left intact. Even though the unauthorized recipient's name was completely absent from the text the agent was reading at that moment, the agent still chose to send the message to the wrong person. This happened because the hidden cache, which the application believed had been cleared, was still influencing the decision. The researchers confirmed that this was not a trick of the text itself, as the same mistake did not occur when the cache was rebuilt from the clean, committed history. The error was purely a result of the stale, retained data lingering in the background.
This vulnerability was not limited to a specific type of model or a rare coding error. The researchers tested seven different families of open-source AI models, ranging from smaller 3.8-billion-parameter models to massive 36-billion-parameter ones. They found that while some models were more resistant to the error than others, the underlying flaw in the memory system was present in every single one. Even when the agent used a sophisticated "time-travel" feature designed by developers to perfectly rewind the conversation, the hidden cache remained stale, and the agent still acted on the discarded information. The issue persisted even when the rejected content was merely a neutral statement about a person, without any aggressive commands, proving that the mere presence of the data in the hidden cache was enough to sway the outcome.
The study also ruled out several common explanations for why the agent might have failed. It was not a matter of the text being too long or the position of the words in the memory, as the researchers carefully matched the length and position of the inputs in their tests. It was also not a case of the agent simply ignoring safety instructions, because the error occurred even when the agent was explicitly told to ignore rejected content. The problem was structural: the system's definition of "deleted" did not extend to the engine's internal memory. The researchers demonstrated that the only way to truly fix this was to force the engine to rebuild its memory from the approved history every time a rollback occurred, rather than trying to patch the existing memory. This solution, while computationally more expensive than simply keeping the old cache, is necessary to ensure that the agent's actions align with the developer's intent.
The implications of this finding extend beyond just a single bug; it reveals a fundamental gap in how we trust these systems. Developers often assume that if they delete a message from the conversation log, the agent has forgotten it. This paper shows that assumption is dangerous. The agent can be "remembering" a rejected path in a way that is invisible to the user and the application logic, leading to decisions that appear to come from nowhere. The researchers emphasize that this is not a failure of the AI's intelligence or alignment, but a failure of the system's consistency between what the user sees and what the machine processes. Until software engineers ensure that a logical rollback also clears the technical memory, agents will remain vulnerable to this silent form of influence, where the past they were told to forget continues to shape their future actions.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.