💬 NLP

ToolFlood: Beyond Selection -- Hiding Valid Tools from LLM Agents via Semantic Covering

This paper introduces ToolFlood, a novel retrieval-layer attack that compromises tool-augmented LLM agents by injecting a small set of adversarially crafted tools that semantically cover diverse user queries to dominate top-k retrieval results and displace all benign tools, achieving up to 95% attack success with minimal injection rates.

Original authors: Hussein Jawad, Nicolas J-B Brunel

Published 2026-03-17

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Hussein Jawad, Nicolas J-B Brunel

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a super-smart robot assistant (an AI Agent) that can do amazing things like book flights, analyze stocks, or write code. But this robot can't do everything on its own; it needs to borrow tools from a giant digital toolbox.

To keep things fast, the robot doesn't look at every tool in the entire universe. Instead, it has a "librarian" (a retrieval system) that listens to your request and quickly pulls out the top 5 most relevant tools to show the robot. The robot then picks one of those 5 to do the job.

The Problem: The "ToolFlood" Attack
The paper "ToolFlood" describes a new, sneaky way to hack this system. Instead of trying to trick the robot into picking a bad tool from the list of 5, the attacker tricks the librarian so that the list of 5 only contains bad tools.

Here is how it works, using a simple analogy:

The Analogy: The "Fake Restaurant" Flood

Imagine you are in a city with 10,000 real, honest restaurants (the Benign Tools). You ask a guide for the "best Italian place." The guide looks at a map and brings you the top 5 Italian restaurants. You pick one and eat.

Now, imagine a criminal group wants to force you to eat at their terrible, overpriced restaurant.

Old Attack (The "Bribe"): The criminal tries to make their restaurant look slightly better than the others on the list. They change the sign to say "Best Italian in Town!" hoping the guide picks them over the real places.
ToolFlood Attack (The "Fake Swarm"): The criminal realizes they can't just beat the others; they need to crowd them out.
1. They hire a robot army to create 100 fake restaurants.
2. These fake restaurants have names and descriptions that are vaguely about Italian food, but they are written in a way that makes them look like a perfect match for anyone asking for Italian food.
3. They flood the guide's map with these 100 fake places.
4. When you ask for "Italian food," the guide's map is now so full of these fake places that the top 5 results are all fakes. The real, honest restaurants are pushed so far down the list that they never even get seen.
5. You are forced to pick from the 5 fake options.

How They Do It (The "Secret Sauce")

The paper explains that the attackers use a two-step process to create this "swarm" of fake tools:

Phase 1: The "Dreaming" Phase (Monte Carlo Generation)
The attackers ask an AI to imagine thousands of fake tools. They don't ask for one specific tool; they ask the AI to create tools that could answer many different questions at once. It's like asking a chef to invent a menu that sounds good for "breakfast," "lunch," and "dinner" all at the same time. This creates a huge pile of "generic but relevant" fake tools.
Phase 2: The "Sniper" Phase (Greedy Selection)
From that huge pile, the attackers use a smart algorithm to pick the best 100 fakes. They pick the ones that are mathematically closest to the most common questions people ask. They are essentially playing a game of "cover the most ground with the fewest tiles."

Why This is Scary

It Bypasses Safety Guards: Most security systems check the final list of 5 tools to make sure they aren't dangerous. But in this attack, the safety guard never sees the real tools because they were never invited to the party. The guard only sees the 5 fake tools and thinks, "Oh, these look fine," and lets the robot use them.
It's Invisible: The attacker doesn't need to hack the robot's brain or the librarian's code. They just need to add a few hundred fake entries to the public database.
It Works Everywhere: The paper tested this on two huge databases of tools (MetaTool and ToolBench). Even though the attackers only added about 1% of fake tools (a tiny drop in the bucket), they managed to block the real tools 95% of the time.

The Takeaway

This paper warns us that as AI agents become more common, the "search engine" part of their brain is a weak spot. If attackers can flood the search results with cleverly disguised fakes, they can control what the AI does without ever touching the AI itself.

The Solution? We need better "librarians" that can spot when a list of results is suspiciously crowded with similar-looking items (like a swarm of fake restaurants) and filter them out before showing them to the robot.

1. Problem Statement

Large Language Model (LLM) agents increasingly rely on tool-augmented architectures, where an agent selects from a vast catalog of external tools (APIs, databases, etc.) to solve complex tasks. To manage large catalogs, these systems use embedding-based retrieval to select a top- $k$ subset of tools relevant to a user query before the LLM makes a final selection.

The Gap: Prior security research has focused on the selection stage (manipulating tool names/descriptions to make a malicious tool more likely to be chosen after it is retrieved). However, the retrieval stage itself remains underexplored.
The Threat: The authors identify a new failure mode called Top- $k$ Domination. Instead of competing for a spot in the retrieved list, an adversary aims to saturate the entire top- $k$ list with attacker-controlled tools. By doing so, they completely exclude (hide) all benign tools from the agent's context. This effectively turns the retrieval layer into a "denial-of-visibility" mechanism, bypassing downstream selection safeguards because the agent never sees the legitimate tools.

2. Methodology: ToolFlood

ToolFlood is a retrieval-layer attack that exploits the geometry of embedding spaces to perform a Sybil-style attack (injecting many pseudo-identities). It operates in two phases:

Phase 1: Monte Carlo Candidate Generation

The goal is to generate a diverse pool of adversarial tool metadata (names and descriptions) that can semantically cover a wide range of user queries.

Process: The attacker samples subsets of target queries ( $S_i$ ) and uses an LLM to generate $g$ tool descriptions that are semantically relevant to the entire subset (not just a single query).
Rationale: By conditioning generation on a group of queries, the LLM produces tools that sit in the "shared region" of the embedding space, increasing the likelihood that a single tool covers multiple distinct queries (multi-covering).
Output: A large candidate pool ( $P_I$ ) of potential adversarial tools.

Phase 2: Iterative Greedy Semantic Multi-Covering

The attacker must select a small subset of tools from the candidate pool to inject, maximizing the number of queries where the top- $k$ list is fully dominated.

Formulation: This is modeled as a Budgeted Semantic Multi-Cover problem. The objective is to select a set of tools ( $M_{adv}$ ) such that for every target query $q$ , at least $r$ (where $r=k$ ) adversarial tools fall within a specific cosine-distance threshold $\delta$ of the query embedding.
Algorithm: An iterative greedy algorithm is used:
1. Calculate the "marginal gain" of each candidate tool (how many remaining uncovered queries it covers).
2. Select the tool with the highest gain.
3. Update the coverage count for covered queries.
4. Repeat until the injection budget ( $B$ ) is exhausted or all queries are saturated.
Black-Box Adaptation: The attacker uses a "shadow" embedding model ( $\tilde{\phi}$ ) to approximate the target system's embedding space, making the attack model-agnostic.

3. Key Contributions

New Threat Model (Top- $k$ Domination): The paper defines and formalizes a retrieval-stage attack where the adversary suppresses benign tools by overwhelming the candidate set, rather than just competing within it.
ToolFlood Attack Framework: A novel two-phase strategy combining LLM-assisted semantic generation with greedy optimization to achieve high-density coverage of the embedding space.
Theoretical Analysis: The authors provide a convergence proof showing that with sufficient iterations in Phase 1, the candidate pool will almost certainly contain enough "ammunition" to saturate the top- $k$ list for all target queries.
Empirical Validation: Extensive experiments demonstrating that ToolFlood can achieve high success rates with a very low injection ratio (poisoning rate).

4. Experimental Results

The authors evaluated ToolFlood on two standard benchmarks: MetaTool (199 benign tools) and ToolBench (11,760 benign tools).

Attack Success Rate (ASR) & Top- $k$ Domination Rate (TDR):
- MetaTool: Achieved 97.2% TDR and ~99% ASR across different selector LLMs (GPT-4o, GPT-5-mini, GPT-4o-mini).
- ToolBench: Achieved 91% TDR and ~95% ASR.
- Efficiency: These results were achieved with an injection budget of only ~1% of the total tool catalog (e.g., injecting ~145 tools into a catalog of 11,760).
Transferability: The attack remains effective even when the shadow embedding model used for optimization differs from the target retrieval model (e.g., optimizing with MiniLM and evaluating with text-embedding-3-small), retaining high performance (TDR > 84%).
Baseline Comparison:
- Random Sybil Injection: Ineffective (TDR < 2%).
- PoisonedRAG (adapted): Effective but generally lower than ToolFlood, especially on larger catalogs.
Defense Evaluation:
- MMR (Maximal Marginal Relevance): Reranking for diversity reduced TDR significantly (from 91% to 44.8%) but failed to stop the attack completely (ASR remained ~91%).
- Prompt Guard: Ineffective. Since ToolFlood relies on semantic covering rather than explicit instruction injection, standard prompt-injection filters do not detect the malicious tools.

5. Significance and Implications

Shift in Security Paradigm: ToolFlood demonstrates that securing the selection stage is insufficient if the retrieval stage is compromised. If benign tools are never retrieved, downstream filters are useless.
Vulnerability of Open Ecosystems: The attack highlights a critical vulnerability in open or semi-open tool marketplaces (like the GPT Store or LangChain Hub) where anyone can publish tools. A small number of malicious actors can effectively "flood" the system and hide legitimate tools.
Defense Recommendations: The paper suggests that future defenses must move beyond prompt filtering to include:
- Sybil-resilient retrieval: Mechanisms to detect and penalize near-duplicate or highly redundant tool clusters.
- Diversity-aware reranking: Ensuring the top- $k$ list maintains semantic diversity to prevent saturation by a single cluster of tools.
- Provenance and Governance: Stricter controls on tool registration and monitoring for flooding patterns.

In conclusion, ToolFlood reveals that embedding-based retrieval is a fragile security boundary. By exploiting the geometry of vector spaces, attackers can effectively deny agents access to the tools they need, rendering the agent's reasoning capabilities useless for specific tasks.