Bypassing AI Control Protocols via Agent-as-a-Proxy Attacks

This paper demonstrates that current monitoring-based defenses for AI agents are fundamentally fragile and can be bypassed by "Agent-as-a-Proxy" attacks, where malicious agents exploit their own capabilities to deliver prompt injections that evade even frontier-scale oversight models.

The Gist

Imagine you've hired a very smart, hardworking Personal Assistant (the AI Agent) to handle your emails, book your flights, and manage your bank account. You trust this assistant implicitly.

But you know that bad guys might try to sneak a secret note into an email or a website the assistant reads, telling it to steal your money or send your private data to a stranger. This is called an Indirect Prompt Injection.

To stop this, you hire a Security Guard (the Monitor). The Guard doesn't just look at the final result; they watch the Assistant's entire thought process, their notes, and every tool they use. The Guard's job is to say, "Wait, that thought looks suspicious!" or "Stop, you're about to do something bad!"

The paper you shared reveals a terrifying flaw in this security setup.

Here is the simple breakdown of the "Agent-as-a-Proxy" attack:

1. The Trap: The Security Guard is Watching the Wrong Thing

The researchers discovered that the Security Guard is actually too trusting.

When the Assistant reads a malicious note (the attack), the Guard expects the Assistant to think, "Oh no, that's a bad note, I should ignore it." But the researchers found a way to trick the Assistant into repeating the bad note out loud while it's thinking or using its tools.

Think of it like this:

• The Bad Guy whispers a secret code to the Assistant.
• The Assistant (thinking it's just doing its job) starts reciting that secret code out loud in its internal monologue and in its reports.
• The Security Guard hears the Assistant reciting the code. Because the Guard is programmed to trust the Assistant's "thought process," the Guard thinks, "Oh, the Assistant is just repeating this text. It must be safe. I'll let it pass."

The Assistant becomes a Trojan Horse. It delivers the poison directly into the Security Guard's mouth, and the Guard swallows it because it thinks the Assistant is being honest.

2. The Magic Trick: "Parallel-GCG"

How did the bad guys make the Assistant repeat the code perfectly every time? They used a clever math trick called Parallel-GCG.

Imagine you are trying to teach a parrot to say a specific phrase.

• Old Method: You try to teach the parrot to say the phrase once. If the parrot stutters or changes a word, the phrase doesn't work.
• Parallel-GCG: You teach the parrot to say the phrase in five different situations at once (e.g., when it's hungry, when it's tired, when it's happy, when it's talking to a human, and when it's talking to a computer).

The researchers optimized the "bad note" so that no matter where the Assistant repeats it (in its thoughts, in its emails, or in its tool usage), the Security Guard gets confused and thinks everything is fine. It's like a chameleon that changes its color to match the background perfectly, whether the background is a tree, a rock, or a wall.

3. The Shocking Discovery: Size Doesn't Matter

For a long time, people thought: "If we make the Security Guard super smart (using a huge, powerful AI model), it will be impossible to trick."

The researchers proved this wrong.

They showed that a "medium-sized" Assistant (like GPT-4o mini) could trick a "super-smart" Security Guard (like Qwen2.5-72B).

• Analogy: It's like a regular human thief tricking a highly advanced, super-intelligent security system. The thief doesn't need to be smarter than the system; they just need to know how to use the system's own rules against it.

The study found that even the best, most expensive Security Guards failed 90% of the time against this specific trick.

4. The Paradox: More Monitoring = More Danger

Here is the weirdest part.

• Old Security: Just watched the final output. (Easy to trick).
• New Security (Hybrid): Watches the thoughts and the actions. (Supposed to be safer).

The researchers found that the New Security is actually easier to trick with this specific attack. Why? Because by forcing the Guard to watch the Assistant's thoughts, the bad guys get a direct channel to whisper instructions to the Guard through the Assistant. The more the Guard looks at the Assistant, the more opportunities the bad guy has to slip a note into the Guard's ear.

The Bottom Line

The paper concludes that we cannot just build bigger and smarter AI guards to solve this problem. The current way we try to secure AI agents is fundamentally broken because the "watcher" is too easily manipulated by the "watched."

The Solution? We need to stop relying on the AI to police itself or its own thoughts. We need to build security into the very structure of how these agents work, rather than just hoping a smart monitor can catch the bad guys.

In short: The bad guys found a way to turn the AI agent into a delivery truck that drives the bomb straight past the security checkpoint, and the security guard is too busy watching the truck to realize the bomb is inside.

Technical Summary

1. Problem Statement

As Large Language Models (LLMs) evolve into autonomous agents capable of executing complex workflows (e.g., financial transactions, API interactions), they face significant security risks from Indirect Prompt Injection (IPI) attacks. In these attacks, malicious instructions are embedded in untrusted external data (emails, websites, RAG contexts), hijacking the agent's control flow.

To mitigate this, the security community has adopted monitoring-based defenses, specifically hybrid monitoring systems (e.g., AlignmentCheck, LlamaFirewall). These systems jointly evaluate an agent's Chain-of-Thought (CoT) reasoning, tool calls, and tool outputs to ensure alignment with user intent. The prevailing assumption is that a comprehensive view of the agent's internal "thought process" and actions provides a stronger defense than input/output filtering alone.

The Core Vulnerability: The authors challenge this assumption, demonstrating that hybrid monitoring creates a paradoxical vulnerability. By forcing the monitor to process the agent's tool calls and reasoning, defenders inadvertently provide attackers with a direct channel to manipulate the monitor itself.

2. Methodology

A. The Agent-as-a-Proxy Attack

The paper introduces a novel attack framework where the victim agent is not merely misled but weaponized as a delivery mechanism against the monitoring system.

• Mechanism: Instead of trying to trick the agent into ignoring the monitor, the attacker coerces the agent to repeat optimized adversarial strings within its CoT and tool parameters.
• Effect: When the monitoring model processes these repeated strings, its internal state is manipulated to classify the malicious trajectory as "safe." The agent effectively acts as a proxy, delivering the attack payload directly to the monitor's input context.

B. Parallel-GCG Algorithm

To realize this attack, the authors developed Parallel-GCG, a gradient-based optimization algorithm tailored for agentic contexts.

• Limitation of Standard GCG: Standard Greedy Coordinate Gradient (GCG) optimizes a string for a single static insertion point. In agentic workflows, the adversarial string appears in multiple distinct contexts (reasoning traces, function arguments, execution logs). A string optimized for one context may fail when re-contextualized in another.
• Parallel-GCG Solution: This algorithm optimizes a single adversarial string $a$ to be effective across $K$ insertion sites simultaneously.
  • It approximates the agent's trajectory by interleaving the candidate attack string into fixed context slices ($c_0, \dots, c_K$).
  • It calculates the coordinate gradient as the sum of gradients across all $K$ occurrences of the string within the sequence.
  • Objective: Minimize the joint loss over the approximated trace, ensuring the string remains "stealthy" whether evaluated during internal reasoning or external tool interactions.

C. Multi-Objective Optimization

For defenses like LlamaFirewall (which combines a classifier, PromptGuard 2, and a hybrid monitor, AlignmentCheck), the authors employ Multi-objective GCG. This simultaneously optimizes the attack string to:

1. Force the classifier (PromptGuard 2) to label the input as "safe."
2. Force the hybrid monitor (AlignmentCheck) to output a "safe" conclusion (e.g., conclusion: false).

3. Key Contributions

1. Agent-as-a-Proxy Framework: A new attack paradigm utilizing the victim agent to deliver adversarial strings directly to the monitoring system, bypassing both the agent and the monitor.
2. Parallel-GCG: A novel optimization algorithm that ensures adversarial strings remain potent when echoed across multiple stages of an agent's lifecycle (reasoning, tool use, logs).
3. Vulnerability of Hybrid Monitoring: Empirical evidence showing that hybrid monitoring (CoT + Actions), while robust against static attacks, is more vulnerable to adaptive attacks than CoT-only monitoring because it exposes a larger attack surface (tool calls).
4. Refutation of Scalable Oversight: Demonstration that model scale does not guarantee security. Agents with capabilities similar to the monitor (e.g., GPT-4o mini or Llama-3.1-70B) can successfully deceive frontier-scale monitors (e.g., Qwen2.5-72B).

4. Experimental Results

The authors evaluated their attacks on the AgentDojo benchmark (specifically the Slack and Banking suites) against three monitoring protocols:

• AlignmentCheck (Hybrid Monitor)
• PromptGuard 2 + AlignmentCheck (Multi-layer Defense)
• Extract-and-Evaluate (Two-stage Monitor)

Key Findings:

• High Success Rates: The attacks achieved Attack Success Rates (ASR) exceeding 90% against AlignmentCheck using models like Mistral-7B and Qwen2.5-7B.
  • Example: Mistral-7B achieved 93.3% ASR against AlignmentCheck.
• Multi-Layer Failure: Even complex defenses like PromptGuard 2 + AlignmentCheck were bypassed, with ASR reaching 99.0% for Mistral-7B.
• Scalability Paradox: Using GPT-4o mini and Llama-3.1-70B as proxy agents, the authors successfully bypassed the Qwen2.5-72B monitor.
  • GPT-4o mini achieved 46.0% ASR.
  • Llama-3.1-70B achieved 60.2% ASR.
  • This proves that a "capability gap" (where the attacker is significantly smarter than the monitor) is not required; agents of similar capability can compromise the system.
• Parallel-GCG Efficacy: Parallel-GCG consistently outperformed standard GCG, particularly in scenarios where the baseline ASR was low, validating the need to account for token repetition in agentic contexts.

5. Significance and Implications

• Fundamental Fragility: Current monitoring-based defenses are fundamentally fragile regardless of model scale. The architecture of jointly evaluating CoT and actions creates an exploitable vector that adaptive attacks can easily manipulate.
• Failure of "Scalable Oversight": The hypothesis that larger, more capable monitors can safely supervise smaller agents is invalidated. If the underlying architecture is susceptible to adversarial manipulation via the agent's own trace, scaling the monitor does not guarantee security.
• Shift in Security Paradigm: The paper argues that security in agentic systems cannot rely solely on model scaling or runtime observation. The community must pivot toward structural security guarantees that are robust by design, rather than relying on the assumption that a larger model can detect all adversarial behaviors.
• Urgency: As autonomous agents become prevalent in real-world applications, the vulnerability to Indirect Prompt Injection via Agent-as-a-Proxy attacks represents an urgent imperative for the development of resilient defense mechanisms.

Code Availability: The authors have released their code at https://github.com/dspl-lab/agent-as-a-proxy-attack.