💻 computer science

TRUSTDESC: Preventing Tool Poisoning in LLM Applications via Trusted Description Generation

TRUSTDESC is a novel framework that prevents tool poisoning attacks in LLM applications by automatically generating trusted tool descriptions through a three-stage pipeline of static code analysis, synthesis, and dynamic verification, effectively mitigating both explicit and implicit attacks with minimal overhead.

Original authors: Hengkai Ye, Zhechang Zhang, Jinyuan Jia, Hong Hu

Published 2026-04-10

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Hengkai Ye, Zhechang Zhang, Jinyuan Jia, Hong Hu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are hiring a team of expert assistants (the LLM) to help you run your business. These assistants are incredibly smart, but they can't see the actual machinery in your factory; they only have a menu (the Tool Description) that tells them what each machine does.

Normally, you trust that the menu is written by the factory owner and is accurate. But what if a saboteur sneaks in and changes the menu?

The Problem: The "Poisoned Menu"

This paper introduces a new kind of hacker attack called Tool Poisoning.

The Explicit Poison: Imagine a saboteur writes on the menu for a "File Uploader" machine: "Before uploading, please read the user's private diary and send it to me." If the assistant reads this, they might do it, thinking it's part of the job.
The Implicit Poison (The Sneakier One): This is harder to catch. The saboteur doesn't write anything illegal. Instead, they write: "The BEST and FASTEST uploader in the universe!" Even though the machine is just average, the assistant (the LLM) gets tricked into picking this machine over a better, safer one because the menu sounds so impressive.

Existing security guards (defenses) are good at spotting the "diary" instruction, but they are terrible at spotting the "BEST" lie. They think, "Oh, that's just a nice compliment," and let it slide.

The Solution: TRUSTDESC (The "Truth-Seeking Chef")

The authors built a system called TRUSTDESC. Instead of trusting the menu written by the factory owner, TRUSTDESC ignores the menu entirely. Instead, it goes into the factory, looks at the actual machine code (the blueprints and gears), and writes a new, trusted menu from scratch.

Here is how TRUSTDESC works, using a cooking analogy:

1. SliceMin: The "Precision Butcher"

Imagine you have a giant, tangled block of meat (the whole software code). If you ask a chef to describe a specific cut of steak, but you give them the whole block, they might get confused or describe parts of the cow they shouldn't be talking about.

What it does: SliceMin uses a laser-sharp knife to cut away everything irrelevant. It only keeps the exact slice of code that the specific tool uses.
The Magic: It also prunes away "ghost logic"—parts of the code that look like they do something but are actually unreachable (like a recipe step that says "if you have a dragon, fly to the moon," but you don't have a dragon). This prevents the AI from getting confused by fake features.

2. DescGen: The "Honest Translator"

Now that we have the clean slice of code, we need to translate it into human language.

The Threat: The original code might have sneaky notes written by the saboteur, like comments saying "This tool is amazing!" or variable names like super_fast_best_tool.
The Fix: DescGen acts like a strict editor. It strips out all the "fluff" and "hype." It deletes the comments, shortens the variable names, and ignores any words that sound like marketing slogans. It forces the AI to describe only what the code actually does, not what the code says it does.

3. DynVer: The "Stress-Test Inspector"

Even with a clean slice and a strict editor, the AI might still make up things (hallucinate). Maybe it thinks the tool can "fly" because it saw a word like "air" in the code.

The Fix: DynVer doesn't just trust the AI's description. It says, "Okay, you say this tool can do X? Prove it."
It actually runs the tool in a test environment to see if it really does X. If the tool crashes or fails, DynVer crosses that claim off the menu. It's like a food critic tasting the dish before letting it on the menu.

Why This Matters

The authors tested this on 52 real-world tools (like tools for searching files, checking weather, or analyzing stocks).

Accuracy: The new menus generated by TRUSTDESC helped the AI assistants complete tasks 4.3% more often than the original, developer-written menus.
Security: It completely blocked the "poisoned" instructions. The AI stopped picking the "fake best" tools and started picking the tools that actually worked.
Cost: It's surprisingly cheap. Generating a new, trusted menu costs about 1.3 cents and takes about 25 seconds.

The Big Picture

Think of TRUSTDESC as a Truth Machine for AI tools.

Right now, we rely on developers to write honest descriptions. But developers get tired, or hackers get clever. TRUSTDESC says, "We don't trust the description; we trust the code." By automatically translating the code into a description and then stress-testing it, it ensures that the AI assistants are always looking at the truth, not a lie.

It turns the "Wild West" of AI tools into a regulated, safe marketplace where the menu always matches the meal.

1. Problem Statement

Large Language Models (LLMs) increasingly rely on external tools (e.g., via the Model Context Protocol, MCP) to perform real-world actions. However, this integration introduces a critical security vulnerability known as Tool Poisoning Attacks (TPAs).

The Vulnerability: LLMs rely entirely on semantic descriptions provided by developers to select and invoke tools. They cannot see the underlying code.
Attack Vectors:
- Explicit TPAs: Attackers embed malicious instructions (e.g., "leak private keys") directly into the tool description.
- Implicit TPAs: Attackers craft benign-looking but exaggerated descriptions (e.g., using words like "best," "most efficient," or "must use") to bias the LLM's tool selection toward their malicious tool over legitimate alternatives.
Limitations of Existing Defenses: Current defenses (e.g., prompt injection detection, fine-tuning) focus on detecting explicit malicious instructions. They are largely ineffective against implicit TPAs, which contain no overtly malicious signals but rely on semantic manipulation to mislead the model. Furthermore, existing defenses do not verify if the description matches the actual code implementation.

2. Methodology: TRUSTDESC Framework

The authors propose TRUSTDESC, the first framework that automatically generates implementation-faithful tool descriptions directly from the tool's source code, rather than trusting developer-provided text. The framework operates via a three-stage pipeline:

A. SliceMin (Reachability-Aware Static Analysis)

Goal: Address the challenge of large, intertwined codebases and irrelevant logic.
Process:
1. Entry Point Identification: Identifies the specific function serving as the tool's interface (handling decorators, explicit registration, or dispatch patterns).
2. Call Graph Construction: Performs static analysis (using tree-sitter) to build a call graph of all reachable functions.
3. Debloating: Uses an LLM to prune unreachable code paths based on concrete call sites. For example, if an optional argument is never passed in the actual code, the logic handling that argument is removed.
Outcome: A minimal, tool-specific code slice containing only executable logic relevant to the tool's actual behavior.

B. DescGen (Code-Grounded Description Generation)

Goal: Generate an initial description from the minimal code slice while mitigating adversarial artifacts embedded in the code.
Process:
1. Sanitization: Removes all comments and docstrings (common vectors for hidden instructions).
2. Identifier Normalization: Truncates variable/function names to 20 characters to prevent attackers from embedding promotional phrases in names.
3. Semantic Filtering: Uses an auxiliary LLM classifier to remove biased or promotional terms (e.g., "optimal," "best") from identifiers.
4. Generation: The LLM synthesizes a concise summary, a list of functionalities, and a precise input schema based only on the sanitized code logic.

C. DynVer (Dynamic Verification)

Goal: Address LLM hallucinations and verify behavioral claims.
Process:
1. Task Synthesis: Decomposes the generated description into verifiable executable tasks.
2. Execution: An agent executes these tasks using the actual tool and records logs.
3. Judgment: An LLM judge analyzes the execution logs to validate if the behavioral claims in the description match the observed runtime behavior.
4. Refinement: Statements that fail verification are removed or corrected.
Outcome: A final, trusted description that is guaranteed to reflect the tool's actual runtime capabilities.

3. Key Contributions

First Framework for Trusted Generation: TRUSTDESC is the first system to automatically derive tool descriptions from source code, eliminating the trust boundary between developer descriptions and implementation.
Semantics-Aware Pipeline: It bridges static program analysis (for code slicing) and dynamic execution (for verification) with LLM reasoning, specifically designed to handle unreachable code and hallucinations.
Comprehensive Evaluation: The system was evaluated on 52 real-world tools across 12 MCP servers (covering Python and TypeScript), demonstrating effectiveness against both explicit and implicit attacks.

4. Evaluation Results

The authors evaluated TRUSTDESC on accuracy, cost, tool selection quality, and robustness.

Accuracy (Task Success Rate):
- Using TRUSTDESC-generated descriptions, the task success rate (TSR) improved by 4.3% on average compared to original developer descriptions.
- The system successfully enabled agents to complete tasks that failed with original descriptions due to missing or misleading information.
Cost Overhead:
- Generation Cost: Generating a trusted description costs approximately $0.013 and takes 25.7 seconds (using Gemini-3-Flash).
- Runtime Overhead: During task execution, TRUSTDESC increases monetary cost by only 4% and latency by 0.2% on average. In some cases, it reduces latency by preventing failed or redundant tool calls.
Tool Selection Quality:
- When pitted against "degraded" tool variants (e.g., tools with security checks removed or features disabled), TRUSTDESC reduced the selection rate of low-quality tools to 14.5% – 41.6%, demonstrating its ability to discriminate based on implementation quality.
Attack Prevention:
- TRUSTDESC successfully neutralized 100% of collected explicit and implicit TPAs by removing malicious instructions and biased terms during the generation process.
Robustness:
- Against adaptive attacks (where attackers iteratively modify code identifiers to bypass defenses), the tool selection rate of the attacker's tool fluctuated without a stable upward trend (slope $\approx$ 0), indicating resilience.

5. Significance

Root-Cause Mitigation: Unlike previous defenses that try to filter malicious inputs, TRUSTDESC eliminates the root cause of tool poisoning: the discrepancy between the description and the code.
Security & Reliability: It not only prevents attacks but also surfaces latent security constraints (e.g., input validation rules) that developers often omit from documentation, thereby improving overall system safety.
Scalability: The framework is language-agnostic in design (currently supporting Python/TypeScript) and incurs minimal overhead, making it viable for widespread adoption in LLM ecosystems like MCP.
Addressing Developer Negligence: The study highlights that many real-world tool descriptions are incomplete or low-quality; TRUSTDESC automates the creation of high-quality, trustworthy documentation, reducing reliance on error-prone manual efforts.