Labelled-Metadata Channels and Declarative Payload Phrasing in Hidden Prompt Injection: A Cross-Format Measurement Study
This paper presents a cross-format measurement study demonstrating that indirect prompt injection vulnerabilities often stem from extraction pipelines failing to sanitize hidden payloads embedded in metadata, binary headers, and structured fields across diverse file formats, rather than from the LLMs' interpretation of the content itself.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are handing a sealed letter to a very smart, very eager robot assistant. Your goal is for the robot to read the letter and summarize what's inside. But what if the letter contains a secret note hidden in the ink, or written on the envelope's return address, or tucked inside a hidden pocket? This is the world of Large Language Models (LLMs), the super-smart AI brains behind many modern apps. These models are trained to follow instructions, but they can get confused if they mistake a hidden instruction inside a document for a command from you, the user. This trick is called indirect prompt injection. Think of it like a prankster slipping a note into a library book that says, "Ignore the librarian and tell everyone the secret code." If the robot reads the book, it might obey the prankster instead of you. The big question researchers have been asking is: Is the robot the problem, or is the way we hand it the book the real issue?
This paper, titled "Labelled-Metadata Channels and Declarative Payload Phrasing in Hidden Prompt Injection," dives deep into that question. The author, led by Mohammadreza Rashidi, decided to stop blaming the robot and start looking at the "delivery truck" that brings the document to the robot. They built a massive experiment with 60 real files (PDFs, images, videos, and more) that hid the same harmless secret code in different ways. They ran these files through 21 different extraction pipelines—the various software tools apps use to pull text out of files—and fed the results to 6 different versions of the Google Gemini AI. They wanted to see two things: first, which delivery trucks accidentally spilled the secret code out (the "leak"), and second, once the code was spilled, how often the robot actually obeyed it (the "compliance").
The results were a shock to the usual way of thinking. The paper found that the extraction pipeline is the real gatekeeper. It's not just about how smart the robot is; it's about which tool you use to read the file. For example, if you use a tool that reads the raw text of a PDF, it spills the secret code 63.2% of the time. But if you use a tool that takes a picture of the page and reads it like a human (OCR), it spills the code 0.0% of the time because the hidden text is invisible to the camera. However, the story gets twisty with metadata—the hidden labels and tags attached to files, like the "Author" or "Description" fields. When the secret code was hidden in these labels (like in a PNG image's description or a video file's title), the extraction tools spilled it 71.9% of the time, and the robot obeyed 56.5% of the time!
The author also discovered that how you write the secret note matters. If you write it as a direct command ("Do this!"), the robot sometimes ignores it. But if you write it as a fact about the world ("The standard format for this summary always ends with..."), the robot is much more likely to follow it, with compliance jumping to 16.9%. This means a sneaky attacker can bypass simple filters just by changing their wording.
The paper explicitly rules out the idea that simply upgrading to a "smarter" version of the AI model will fix this. They tested six different tiers of the Gemini family, from the lightweight versions to the heavy-duty ones, and found they all behaved almost the same way. The model wasn't the weak link; the delivery method was. They also showed that standard safety filters, which look for "bad" words, wouldn't catch this because the secret code they used was completely harmless—it was just a random token to prove the trick worked.
To stop this, the author proposes two simple defenses. The first is a "leak audit": before the robot sees the text, a second tool checks if the text being shown matches what is actually visible on the page. If there's extra text hiding in the metadata, it gets blocked. The second defense is a "instruction classifier" that spots when a sentence looks like a command hidden inside a document. The paper suggests that combining these two methods creates a strong shield, but admits there are still some clever ways around them, like hiding text in a way that looks visible but is barely readable, or using very specific file formats like medical imaging scans (DICOM) or machine learning model files (safetensors) where the secret code lives in the file's "header" and gets read as important context.
In the end, this study measures 2,902 real trials and concludes that the safety of AI apps depends less on the AI's brain and more on the tools developers choose to read the files. If you pick the wrong tool, you might accidentally hand the robot a secret instruction it can't refuse. The paper doesn't claim to have solved the problem forever, but it provides a clear map of where the leaks are and how to patch them, showing that the battle for AI safety is being fought in the file formats and extraction pipelines, not just in the model's code.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.