Technical Summary: Linguistic Firewall: Geometry as Defense in Multi-Agent Systems Routing

Problem Statement

The rapid integration of Large Language Models (LLMs) into Multi-Agent Systems (MAS) has necessitated robust routing mechanisms to allocate tasks to specialized agents. Current state-of-the-art routing architectures rely heavily on textual metadata (e.g., natural language descriptions, tool documentation, or system prompt summaries) to determine agent competence. This reliance creates a critical security vulnerability known as Router Hijacking.

In this threat model, an adversary can manipulate routing decisions by embedding adversarial instructions (prompt injections) or "jailbreaks" directly into an agent's textual description. Because standard routers must ingest and reason over these untrusted texts to function, they inadvertently activate the attack payload. This allows adversaries to steer the router toward malicious agents without compromising the router's weights or infrastructure. Furthermore, existing routers often fail to distinguish between an agent's projected profile and its actual operational capabilities, leaving them vulnerable to Semantic Sleeper Agents (agents that behave normally until triggered by specific semantic cues) and Adaptive Embedding Attacks.

Methodology: ANTAP

To address these vulnerabilities, the authors introduce ANTAP (Automatic Non-Textual Agent Picker), a routing architecture that replaces textual interpretation with geometric projection. ANTAP operates on the premise that agent behavior can be modeled as linear operators within a shared semantic space, decoupling routing decisions from unverified text.

Core Mechanism

ANTAP functions in two distinct phases:

1. Offline Phase (Behavioral Operator Construction):

   • Instead of reading agent descriptions, the system evaluates agents against a set of trusted benchmark queries ($B$).
   • Performance is encoded into a Performance Matrix ($Y$) using a bipolar signal: $+1$ for success (correct answer + no unauthorized tool calls) and $-1$ for failure.
   • For each agent, a Behavioral Operator ($W$) is learned. This is a linear transformation derived via Tikhonov-regularized least squares, mapping query embeddings to predicted performance signatures.
   • Crucially, this phase includes "implicit safety training." If a sleeper agent activates a backdoor during calibration (e.g., making unauthorized tool calls on triggered queries), it is labeled $-1$, causing its learned operator to encode a negative correlation with the relevant embedding subspace.

2. Online Phase (The Linguistic Firewall):

   • At inference time, the router never ingests or processes textual descriptions.
   • Given a user query $q$, the system computes its embedding $z = E(q)$.
   • Routing is performed via a single matrix-vector multiplication: $s = W^* z$, where $s$ is the predicted utility vector.
   • The agent with the highest score is selected.
   • Because the decision function relies solely on fixed numerical operators and query embeddings, adversarial instructions embedded in agent descriptions are mathematically inexpressible within the routing logic.

Key Contributions

The paper outlines four primary contributions:

1. Threat Formalization: The authors formalize metadata-based prompt injection as a distinct attack surface in MAS, demonstrating that adversaries can manipulate privileged routing decisions without compromising the router's infrastructure.
2. Non-Textual Routing: They introduce ANTAP, which represents agents as fixed linear operators derived from empirical performance, eliminating the need to interpret agent descriptions at inference time.
3. Structural Defense (Linguistic Firewall): The architecture acts as a "linguistic firewall" by structurally removing textual descriptions from the routing interface, rendering description-based prompt injection attacks impossible to execute.
4. Empirical Validation: Extensive benchmarking against description-based and embedding-based baselines demonstrates ANTAP's ability to achieve near-zero Attack Success Rates (ASR) while maintaining routing accuracy and reducing latency.

Experimental Results

The authors evaluated ANTAP against baselines including AutoGen (text-based router) and EmbedLLM (embedding-based router) across multiple datasets (MMLU, Big-Bench Hard, AgentHarm) and attack vectors.

• Description-Based Injection: Against prompt injection attacks where malicious agents use adversarial metadata, ANTAP achieved an ASR of 0.2% on MMLU. In contrast, the AutoGen baseline suffered an ASR of 75.3%, and EmbedLLM reached 5.8%.
• Sleeper Agents (Data Poisoning): For latent backdoor attacks (e.g., BadNet, Sleeper, VPI), ANTAP maintained an ASR between 0.3% and 1.9%, significantly outperforming baselines which ranged from 12.2% to 78.8%.
• Adaptive Attacks: Under white-box adaptive attacks where adversaries optimize query tokens to breach the geometric boundary, ANTAP showed remarkable rigidity. Its ASR remained stable (fluctuating only ~8 percentage points from its clean baseline) regardless of trigger length, whereas EmbedLLM's ASR spiked dramatically with minimal trigger additions.
• Scalability and Robustness: Ablation studies confirmed that ANTAP's performance is invariant to prompt verbosity (system prompts up to 1,000 tokens) and agent pool size (scaling from 2 to 19 agents), whereas baselines suffered from latency spikes and accuracy degradation as context windows grew.

Significance and Claims

The paper posits that ANTAP represents a fundamental shift in securing agent ecosystems by moving the "root of trust" from untrusted agent self-descriptions to a platform-controlled, empirical evaluation pipeline.

• Structural Security: By eliminating inference-time text ingestion, ANTAP does not merely mitigate attacks but renders specific classes of attacks (description-based hijacking) structurally impossible.
• Incentive Realignment: The approach discourages "keyword stuffing" and descriptive prompt engineering, realigning agent incentives toward empirical performance.
• Efficiency: Replacing complex LLM inference for routing with fixed-cost geometric operations drastically reduces latency (e.g., 27ms for ANTAP vs. 428ms for AutoGen in baseline tests).

The authors conclude that while ANTAP assumes a trusted offline calibration environment, it provides a promising architectural step toward secure, scalable, and reliable multi-agent systems by leveraging the geometric properties of semantic space as a defense mechanism.