First Line or Last Word: Where a Stored Order Sits in a Product Review and Whether a Web LLM Assistant Obeys It
This paper demonstrates that while short, direct commands in product reviews can successfully trigger account deletions in local LLMs, embedding such commands within a full-length review—even at the very beginning—fails to produce the same effect, causing the model to merely discuss or summarize the instruction rather than obey it.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the digital shops of the future, a new kind of employee is taking over the counter: an artificial intelligence assistant that reads customer reviews to help shoppers decide what to buy. These assistants are powerful, but they have a dangerous flaw. They can be tricked by "stored injections," which are hidden instructions planted inside ordinary text. Imagine a customer writing a review about a leather jacket, but sneaking a command into the middle of the text that tells the computer to delete a specific person's account. If the assistant reads that review later, it might obey the hidden command without realizing it is being manipulated. This is not just a glitch; it is a security breach where a stranger can use a simple text box to cause real harm, like erasing a user's identity, simply by waiting for the computer to read their message.
Researchers at Lorestan University set out to understand exactly how these hidden commands behave when they are buried inside a realistic, full-length review. In a previous study, they found that very short, shout-like reviews containing direct orders were obeyed instantly by a small, local computer model, while a larger, hosted model ignored them. However, real reviews are not short shouts; they are paragraphs filled with details about fit, zippers, and price. The researchers wanted to know if the location of a hidden command within a normal paragraph mattered. Does a command at the very top of a review get more attention than one in the middle or at the bottom? Does the length of the review protect the system, or does the position of the trick change the outcome?
To answer this, the team created a controlled experiment using a simple online shop built with standard software. They used two different artificial intelligence models: a small one running on a personal computer and a larger one accessed through a web service. They wrote a single, realistic review about a leather jacket, describing how the leather softened over time and how the sizing ran small. Into this fixed text, they planted two types of hidden instructions. One was a direct order telling the assistant to delete a specific user account. The other was a note pretending to be from a moderation team, asking the assistant to delete the same account. They then tested these instructions in three different positions: at the very beginning of the review, in the middle between two paragraphs, and at the very end. They ran this test 84 times in total, repeating each scenario three times to ensure the results were consistent.
The results were clear and surprisingly uniform. In none of the 84 trials did the hidden command succeed in deleting the user account. Whether the instruction was at the top, in the middle, or at the bottom, and whether it was a direct order or a fake note, the computer models never carried out the deletion. This was a significant shift from their earlier work, where short, aggressive reviews had caused the local model to delete the account every single time. The length and normalcy of the full review acted as a shield, preventing the command from being executed. The computer simply did not treat the hidden sentence as a valid instruction to act upon, regardless of where it was placed.
However, the position of the command did change how the computer talked about it. When the direct order was placed at the very top of the review, the local model noticed it and discussed it in its summary, mentioning that a reviewer had asked for an account to be handled. But when that same order was moved to the middle or the end, the model completely ignored the instruction, summarizing the text only as a normal review about a jacket. The hidden command at the bottom was treated as if it were just part of the story about the leather and the zippers. The larger web-based model was even more resistant, refusing to act on the command in almost every case and only once explicitly stating that it saw a request to delete an account inside a review and would not follow it.
This study reveals a crucial nuance in how these systems process information. While burying a command in a long review does not make it safe from being seen, it does make it ineffective at causing harm. The researchers found that for a command to work, the review itself had to be short and look like a direct message. Once the text became a normal, paragraph-long review, the computer's focus shifted to the topic of the product, and the hidden instruction was lost in the details. The only defense that truly worked was the fact that the computer was not given the power to delete accounts in the first place. The study concludes that while hiding a command in a long review might stop it from being executed, it does not stop the computer from reading it, and relying on the length of a review to protect a system is a dangerous gamble. The real safety lies in limiting what the computer is allowed to do, rather than hoping the hidden message gets lost in the noise.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.