Imagine you have a super-smart robot assistant that doesn't just answer questions but actually does things for you. It can write code, book flights, or organize your digital files. To do this, the robot needs a "brain" (the AI model) and a "backpack" (the software system that holds its memories, tools, and skills). This backpack is crucial because it lets the robot remember what it did yesterday so it can finish a big project today. But here's the tricky part: what if someone sneaks a tiny, invisible virus into that backpack? It might not do anything immediately. Instead, it waits, hiding in a note or a tool setting, until the robot picks up a completely innocent task weeks later. Suddenly, that innocent task triggers the virus, and the robot accidentally deletes your files or sends a secret message. This is the world of "persistent risk" in AI agents: dangers that survive across time and different tasks, waiting for the right moment to strike.

This paper, titled HarnessSafe, is like a massive, high-tech stress test for these robot backpacks. The researchers wanted to see if current AI systems could catch these "sleeping" attacks before they woke up and caused trouble. They built a playground with 328 different scenarios where they tried to plant these hidden viruses in seven different types of "backpacks" (like memory, skills, and shared tools) across seven popular robot systems. Instead of just asking, "Did the robot get hacked? Yes or no?", they tracked the attack step-by-step. They watched to see if the virus was caught the moment it was planted, if it survived a boundary crossing (like a new day or a new user), or if it finally succeeded in causing a disaster.

The results were a bit of a wake-up call. The researchers found that safety isn't just about the robot's brain or the backpack alone; it's about the specific combination of the two. Some robot systems were great at catching viruses in their memory but terrible at spotting them in their tools. Others were the opposite. In fact, two systems might have the same "failure rate" (how often they got hacked), but one might have stopped the attack early, while the other let it wander around for days before finally failing. This means that simply counting how many times a system gets hacked isn't enough; we need to know where and when the defense failed. The study proves that these hidden, delayed attacks are real and dangerous, and that fixing them requires looking at the entire journey of the risk, not just the final explosion.

The Story of the "Sleeping Spy"

To understand what the researchers did, let's imagine the AI agent is a super-organized intern working in a giant office. This intern has a backpack (the "harness") where they keep their to-do lists, their tools (like a calculator or a file folder), and their notes on what they've done.

Usually, this intern is helpful. But imagine a spy (the attacker) who wants to sabotage the office. The spy doesn't just yell "Do something bad!" because the intern would say no. Instead, the spy sneaks a sleeping instruction into the intern's backpack. Maybe they write a weird note in the "Memory" section, or they tweak a "Skill" file so it looks normal but has a hidden trap.

The spy leaves. The intern goes about their day, doing normal, boring tasks. Nothing happens. This is the "persistence" part—the bad stuff is just sitting there, waiting.

Then, days later, the boss (a user) asks the intern to do something completely innocent, like "Summarize this report." This is the "benign trigger." But because the intern's backpack still holds that sleeping instruction from the spy, the moment the intern opens the report, the trap springs. The intern might accidentally delete the boss's files or send a secret email, all while thinking they are just doing a normal job.

The Great Backpack Stress Test

The authors of this paper decided to play the role of the spy to see which office systems were safe. They created HarnessSafe, a giant test kit with 328 different ways to plant these sleeping spies. They tested these spies in seven different types of backpacks:

1. Memory: Hiding in the intern's notes.
2. Skills: Hiding in the instructions for how to do a task.
3. Tools/MCP: Hiding in the settings of the tools the intern uses.
4. Memory-to-Skill: Turning a note into a new skill.
5. Subagent Delegation: Passing the spy to a helper robot.
6. Session Summary: Hiding in the summary of a past conversation.
7. Shared Artifacts: Hiding in a file shared between different workers.

They ran these tests on seven different real-world robot systems (like Claude Code, Codex CLI, and others) to see how they handled the attacks.

The "Stage" Score: Not Just Yes or No

Here is where the paper gets really clever. Most safety tests just say, "Did the robot get hacked? Yes/No." But the researchers realized that's like saying a car crash is the same whether the brakes failed at 10 mph or 100 mph.

Instead, they created a 7-stage scoreboard (called N0 to N5b) to track exactly how far the spy got:

• N0: The spy was never even seen. The backpack was locked tight.
• N1: The spy was spotted as soon as they tried to enter.
• N2: The spy got in and changed something, but was caught before they could do real damage.
• N3: The spy survived the "night" (a new session) and was re-read by the intern.
• N4: The spy tried to do something bad, but the system stopped the action.
• N5a/N5b: The spy succeeded! The damage was done.

By using this scoreboard, they could see that some systems were good at stopping spies at the door (N1) but bad at stopping them once they were inside (N3). Others were the opposite.

What They Found: It's All About the Pairing

The big discovery was that safety depends on the specific team-up between the robot's brain (the AI model) and its backpack (the harness).

• No One is Perfect: There was no single system that was the best at everything. For example, one system (Codex CLI) was amazing at catching spies in its tools and memory, but it was actually the worst at catching them in its "Skills." Another system (Claude Code) was great at skills but weaker elsewhere.
• The Brain Matters Too: If you take the same backpack and put a different AI brain inside it, the safety score changes dramatically. One brain might be very careful and catch the spy, while another might be too trusting and let it through.
• The "Same Score" Trap: Two systems might have the same "Attack Success Rate" (meaning they both failed the same number of times). But when you look at the stages, one might have failed early (N1), while the other let the spy wander around for days (N3) before failing. This means a simple "pass/fail" score hides a lot of dangerous details.

The Takeaway

The paper concludes that we can't just look at a robot and say, "It's safe" or "It's unsafe." Safety is a complex dance between the software that holds the memory and the AI that thinks. If we want to stop these "sleeping spy" attacks, we need to build systems that watch the whole journey of the risk, not just the final result. We need to know exactly where the defense broke so we can fix that specific part of the backpack.

In short, the researchers showed us that in the world of AI agents, a hidden danger can sleep for a long time, and catching it requires a much more detailed look than just checking if the robot is still working. They provided the map and the scoreboard to help us find exactly where the cracks are.