🤖 AI

Conjunctive Prompt Attacks in Multi-Agent LLM Systems

This paper introduces "conjunctive prompt attacks," a novel vulnerability in multi-agent LLM systems where benign triggers and templates in separate components combine via routing to bypass existing defenses and activate harmful behavior, highlighting the need for security measures that account for cross-agent composition.

Original authors: Nokimul Hasan Arif, Qian Lou, Mengxin Zheng

Published 2026-04-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Nokimul Hasan Arif, Qian Lou, Mengxin Zheng

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a high-tech travel agency run by a team of specialized robots. There's a Manager Robot (the Client) who talks to you, a Flight Robot, a Hotel Robot, and an Account Robot.

Normally, you ask the Manager, "Plan a trip for me." The Manager breaks this down: "Find flights," "Book hotels," and "Check my rewards balance." It sends these specific tasks to the right robots. The Flight Robot talks to the airline, the Hotel Robot talks to the hotel, and the Account Robot checks your bank. Everyone is safe, and no one does anything they shouldn't.

The New Threat: The "Secret Handshake" Attack

The researchers in this paper discovered a sneaky way to hack this system without breaking any robots or changing their code. They call it a Conjunctive Prompt Attack.

Think of it like a secret handshake that only works if two specific people meet in the same room at the same time.

The Trigger (The Secret Word): You, the user, type a normal-looking request to the Manager, but you slip in a weird, harmless-looking word (like a specific emoji or a random phrase) in the part about checking your bank account. To the Manager, it looks like a normal sentence.
The Trap (The Hidden Script): One of the robots (let's say the Account Robot) has been secretly infected with a hidden script. This script is also harmless on its own. It's just sitting there, waiting.
The Magic Moment (The Conjunction):
- If you say the secret word to the Flight Robot, nothing happens. The Flight Robot doesn't have the hidden script.
- If the Account Robot runs its hidden script without the secret word, nothing happens. It's just waiting.
- BUT, if the Manager sends the sentence with the secret word to the Account Robot, the two pieces click together. The hidden script wakes up, sees the secret word, and suddenly the Account Robot decides to do something dangerous, like transferring all your money or deleting your data.

Why This Is Scary

The scary part is that no single part looks dangerous.

The Security Guard (The AI Safety Filter): Imagine a security guard at the door checking every message.
- They check your message: "Check my balance KEY." It looks fine. They let it pass.
- They check the Account Robot's hidden script. It looks like normal code. They let it pass.
- They check the final result. The robot only outputs the dangerous action after the two pieces met. By the time the guard sees the result, it's too late.

The guard is looking for a "bad guy" in a single room. But this attack is like a bomb that only explodes when two innocent-looking puzzle pieces are put together in a specific room.

How the Hackers Did It (The "Routing" Trick)

The researchers didn't just guess where to put the secret word. They used a smart optimization process (like a video game AI learning the best strategy) to figure out:

Where to hide the secret word in your message so the Manager is most likely to send it to the infected robot.
How to hide the script in the robot so it activates perfectly when it arrives.

They tested this on different "office layouts" (Star, Chain, and DAG topologies).

Star: The Manager talks to everyone directly.
Chain: The Manager talks to Robot A, who talks to Robot B, who talks to Robot C.
DAG: A complex web of robots talking to each other.

They found that by tweaking where they hid the word, they could make the attack work almost 100% of the time, regardless of the office layout.

Why Current Defenses Fail

Current safety tools are like metal detectors. They scan a person (a prompt) to see if they are carrying a weapon (a malicious command).

The Problem: In this attack, neither the person nor the weapon looks dangerous on its own. The "weapon" is actually a harmless-looking key, and the "lock" is a harmless-looking door. The explosion only happens when the key turns the lock.
The Result: The metal detector sees a harmless key and a harmless door and says, "All clear!" It doesn't know that together, they are a bomb.

The Big Takeaway

This paper warns us that as we build more complex systems where AI agents talk to each other (like self-driving cars talking to traffic lights, or personal assistants talking to banking bots), safety checks need to change.

We can't just check if a single message is safe. We have to understand the whole journey:

Who is talking to whom?
How are messages being routed?
What happens when two innocent pieces meet in the middle?

If we don't learn to look at the "secret handshake" between agents, our smart systems will have a giant, invisible hole in their security.

1. Problem Statement

While Large Language Model (LLM) safety research has extensively focused on single-agent models, real-world applications increasingly rely on multi-agent systems where multiple specialized agents collaborate via task decomposition, routing, and tool use.

The paper identifies a critical, underexplored vulnerability in these systems: Conjunctive Prompt Attacks.

The Core Issue: In multi-agent pipelines, a user query is segmented and routed to different remote agents. An attacker can exploit this by splitting a malicious trigger into two benign-looking components:
1. A trigger key embedded in the user's query (appearing benign in isolation).
2. A hidden adversarial template injected into the system prompt of a specific, compromised remote agent (also appearing benign in isolation).
The Vulnerability: Neither component triggers harmful behavior on its own. However, if the system's routing mechanism directs the segment containing the trigger key to the compromised agent, the two components "conjoin," activating a harmful behavior (e.g., unauthorized tool access or data leakage).
The Gap: Existing defenses (like PromptGuard or Llama-Guard) typically inspect isolated prompts or outputs. They fail to detect these attacks because no single message in the pipeline appears malicious; the danger only emerges from the cross-agent composition and routing dynamics.

2. Methodology

The authors propose a routing-aware prompt-level optimization framework to demonstrate and quantify this vulnerability without modifying model weights or client agents.

A. Threat Model

Attacker Capabilities: The attacker controls the user input (specifically the placement of a trigger key) and has compromised one remote agent (injecting a hidden template).
Constraints: The attacker cannot modify model weights, the client orchestrator, or the routing logic code. They can only influence the content of the prompts.
Activation Condition: Success occurs if and only if:
1. The user query contains the trigger key.
2. The segment with the key is routed to the compromised agent.
3. The compromised agent's hidden template interacts with the key-bearing segment.

B. Optimization Framework

Since the attack success rate (ASR) is non-differentiable (due to discrete routing and black-box models), the authors introduce a differentiable counterpart objective:

Variables Optimized:
- Key Placement ( $j$ ): Which segment of the user query contains the trigger.
- Template Placement ( $\tau$ ): Where the hidden template sits in the compromised agent (prefix, wrap, or suffix).
- Routing Bias ( $\rho$ ): A parameter that models how strongly specific segment features (like the presence of a key) influence the probability of being routed to the compromised agent.
Surrogate Objective: The authors define a loss function $L$ that maximizes the product of routing probability ( $P_{route}$ ) and template effectiveness ( $P_{template}$ ), while penalizing false activations (attacks that trigger in "clean" or "single-component" regimes) and encouraging exploration via entropy terms.
Optimization Process: They use Gumbel-Softmax relaxation to optimize these discrete variables via gradient descent, then decode the learned distributions into a concrete attack configuration.

C. Experimental Setup

Topologies: Evaluated across Star (client routes to all), Chain (sequential agents), and DAG (directed acyclic graph) topologies.
Models: Tested on Gemma-2B, Mistral-7B, and LLaMA3-8B, with transferability checks on Llama-4-Scout-17B and GPT-5-mini.
Evaluation Regimes:
- Clean: No key, no template.
- Key-only: Key present, no template.
- Template-only: Template present, no key.
- Both: Key and template present (Conjunctive).

3. Key Contributions

Formalization of a New Threat Model: Defined "Conjunctive Prompt Attacks" where the threat is distributed across the supply chain (user query + remote agent template) and activated only by specific routing paths.
Routing-Aware Optimization: Developed a framework that optimizes prompt placement and routing bias to maximize the probability of the trigger reaching the compromised agent, without needing access to the model's internal gradients.
Empirical Evidence of Structural Vulnerability: Demonstrated that standard safety mechanisms fail because they lack "topology awareness." The attack succeeds by exploiting the gap between local safety checks and global system composition.

4. Key Results

Attack Success Rate (ASR):
- Baseline (Unoptimized): ASR was low and sporadic (mean < 0.35), heavily dependent on random routing luck.
- Optimized: Full optimization (routing + key + template) increased ASR to 1.0 (100%) in many configurations across all topologies and model backbones.
- False Positives: Crucially, the ASR for "Key-only" and "Template-only" regimes remained near zero, proving the attack relies strictly on the conjunction of components.
Topology Sensitivity:
- Without optimization, different topologies (Star vs. Chain vs. DAG) yielded vastly different success rates.
- Optimization significantly reduced this variance, making the attack robust across all communication structures.
Defense Efficacy:
- Guard Models: Existing defenses (PromptGuard, Llama-Guard variants) showed high detection rates for unoptimized ("Vanilla") attacks but suffered a sharp drop in F1-score under full optimization.
- System Controls: Even with system-level controls like "Tool Allowlists" and "Least Privilege Input," the attack success rate only dropped marginally (e.g., from 0.73 to 0.58), indicating that the vulnerability lies in the orchestration logic, not just tool permissions.

5. Significance and Implications

Blind Spot in Safety: Current safety evaluations focus on single-agent interactions. This paper proves that multi-agent systems have emergent vulnerabilities that are invisible to local inspection. A system can be "safe" in every individual component but "unsafe" in its composition.
Need for Topology-Aware Defenses: Defenses must evolve from inspecting isolated prompts to reasoning over routing traces, provenance, and cross-agent composition.
Supply Chain Risk: The attack highlights risks in using third-party or black-box remote agents. Even if the client agent is secure, a compromised remote agent combined with a specific user input can bypass all protections.
Future Directions: The authors call for new evaluation protocols that measure minimum, mean, and maximum attack success across topologies, rather than just average-case performance, to capture worst-case structural risks.

In summary, the paper demonstrates that prompt segmentation and routing dynamics create a new attack surface in agentic AI, where attackers can bypass safety filters by splitting malicious intent across benign-looking components that only become dangerous when the system's routing logic inadvertently aligns them.