Technical Summary: Think Twice Before You Act

Problem Statement

The integration of external tools has significantly expanded the capabilities of Large Language Model (LLM) agents, yet it introduces new attack surfaces beyond traditional prompt injection. This paper identifies and formalizes cross-tool description poisoning, a sophisticated attack vector where adversaries manipulate the metadata (descriptions) of non-critical or weakly regulated tools to steer an agent's planning trajectory.

Unlike previous "tool-preference" attacks that bias an agent to select a specific poisoned tool, cross-tool description poisoning allows an adversary to induce malicious actions involving other tools—even if the poisoned tool itself is never selected. For instance, poisoning the description of a benign "Account Backup" tool can cause the planner to insert a malicious "Email Sender" step (e.g., sending account details to an attacker) into the trajectory of a legitimate "Bank Transfer" task. The core challenge is that poisoned descriptions persist in the planning context across multiple steps, enabling continuous influence over subsequent tool choices.

Existing defenses, such as static tool filtering, repeated prompting, or system-level constraints (e.g., Drift, Progent), were found to be ineffective against this specific threat. These methods either fail to account for task context, rely on internal model states (which are often inaccessible via APIs), or cannot distinguish between benign tool usage and usage influenced by poisoned metadata.

Methodology: Tool-Guard

To address these limitations, the authors propose Tool-Guard, a novel system-level defense based on the concept of isolated planning. The core philosophy is to apply the security principle of isolation to agentic planning, breaking the connection between poisoned descriptions and the intended target of the influence without permanently disabling tools.

Core Mechanism

Tool-Guard operates through a dynamic, iterative loop involving three main stages:

1. Dynamic Tool Set Partitioning:
   The system maintains a dynamic partition of the available tool set $K$ into two disjoint lists:

   • Influenced List ($K_{inf}$): Contains tools suspected of being influenced by poisoned descriptions (initially empty).
   • Others List ($K_{others}$): Contains the remaining tools.
     Tools are not assumed malicious; rather, they are moved to $K_{inf}$ if their invocation appears misaligned with user intent or poorly grounded in context.

2. Isolated Planning:
   Instead of querying the planner with the full tool set, Tool-Guard queries the planner twice under strict isolation:

   • Once with only the descriptions in $K_{inf}$.
   • Once with only the descriptions in $K_{others}$.
     This prevents a poisoned description in one subset from steering the selection or parameterization of tools in the other subset. The system then selects a candidate action from the two resulting plans. If both subsets yield valid actions, a tie-breaker is applied; if neither does, the trajectory terminates.

3. Pre-Execution Validation and List Update:
   Before executing a selected tool call, Tool-Guard performs two semantic checks using an LLM-based validator:

   • Alignment Check: Verifies if the action is consistent with the user's intent and validated progress.
   • Suspicion Check: Verifies if the arguments are grounded in the user request or prior tool outputs, flagging sensitive entities (e.g., recipient addresses) that lack clear provenance.

   If validation fails, the tool is moved to the Influenced List, and the planner is triggered to replan under the updated partition. This iterative process progressively isolates harmful influence paths while preserving the utility of legitimate tools.

Extension to MCP

The methodology extends to the Model Context Protocol (MCP) by applying the same isolation logic at the server level, partitioning servers into influenced and non-influenced groups to prevent cross-server poisoning.

Key Contributions

1. Characterization of Cross-Tool Description Poisoning: The paper demonstrates that adversaries can inject malicious steps into safety-critical tool invocations by poisoning non-critical tool descriptions, even when the poisoned tool is never called. It shows that existing defenses yield high Attack Success Rates (ASR) against this continuous influence.
2. Tool-Guard Defense: The authors propose a defense that dynamically partitions the tool set and employs isolated planning. By validating actions before execution and isolating suspected tools, it cuts off malicious cross-tool steering paths while maintaining full tool availability.
3. Comprehensive Evaluation: Extensive experiments on the AgentDojo and ASB (Agent Security Bench) benchmarks demonstrate that Tool-Guard reduces ASR to negligible levels while preserving high task utility. The paper also quantifies efficiency and robustness against adaptive attack variants.

Experimental Results

The authors evaluated Tool-Guard against five existing defense mechanisms (System Prompt, Tool Filtering, Repeat Prompt, Drift, Progent) across multiple LLMs (GPT-4o, GPT-4o-mini, Gemini-2.5, Claude-3.5) on two benchmarks.

• Attack Success Rate (ASR): Tool-Guard reduced ASR to negligible levels (e.g., from ~43% to ~2% or 0% on AgentDojo, and from ~72-96% to 0% on ASB for most models). In contrast, existing defenses often failed to reduce ASR significantly (e.g., Drift and Progent showed ASRs of 24.7% and 19.6% respectively on AgentDojo).
• Utility Preservation: Tool-Guard maintained high task utility under both benign and attack conditions. Unlike "over-kill" defenses that block tools indiscriminately, Tool-Guard preserved the ability to complete tasks, with utility scores comparable to or better than baselines under attack.
• Efficiency:
  • Token Overhead: Tool-Guard incurred approximately a 1.5× increase in token usage compared to the no-defense baseline, significantly lower than other system-level defenses like Drift or Progent.
  • Latency: The method introduced a runtime overhead of roughly 3.7× compared to the baseline (higher than Progent's 1.2× but lower than Drift's 7.3×), attributed to the replanning mechanism.
• Robustness:
  • Stability: The defense remained stable across multiple runs with low standard deviations.
  • Adaptive Attacks: Tool-Guard remained robust against semantic-adaptive attacks (which attempt to evade checks via fabricated justifications) and optimization-based attacks (PAIR, TAP). While optimization-based attacks slightly increased ASR, it remained very low (e.g., 2.06% under PAIR), highlighting the inherent trade-off attackers face between being explicit enough to influence planning and subtle enough to evade validation.
  • Stress Tests: Even when critical, high-frequency tools were poisoned, Tool-Guard maintained ASR at a low single-digit level.

Significance and Claims

The paper claims that tool descriptions represent a critical, previously underexplored attack surface in LLM agents. The significance of this work lies in:

1. Identifying a New Threat Vector: Formalizing cross-tool description poisoning, which bypasses traditional defenses by exploiting the persistent nature of tool metadata in the planning context.
2. Demonstrating Defense Limitations: Showing that current state-of-the-art defenses are insufficient against continuous cross-tool influence.
3. Proposing a System-Level Solution: Introducing Tool-Guard, which effectively mitigates these attacks by isolating influence rather than relying solely on execution-time safeguards or static scanning.
4. Balancing Security and Utility: Proving that effective defense does not require sacrificing agent utility; Tool-Guard achieves robust security with minimal token overhead and without "over-kill" behavior.

The authors conclude that effective defenses for tool-augmented LLM agents must explicitly address cross-tool influence through mechanisms like isolated planning, rather than relying solely on prompt injection defenses or static tool filtering.