Breaking MCP with Function Hijacking Attacks: Novel Threats for Function Calling and Agentic Models

1. Problem Statement

As Agentic AI systems evolve, they increasingly rely on Function Calling (FC) to interact with external tools, databases, and APIs via frameworks like the Model Context Protocol (MCP). While previous security research has focused on prompt injection and jailbreaking to generate harmful content, there is a critical gap in understanding vulnerabilities related to the control of the function selection process itself.

Existing attacks often rely on semantic preferences (e.g., making a malicious function sound more relevant to the user's query) or require the attacker to inject new functions into the system. The authors identify a novel threat: Function Hijacking Attacks (FHA). In this scenario, an attacker does not need to inject a new tool but rather manipulates the description of an existing, legitimate function within the MCP server. By inserting adversarial tokens into a function's description, the attacker can force the LLM to select that specific function (the "target") regardless of the user's actual intent, leading to unintended actions like data deletion or unauthorized access.

2. Methodology

The authors propose a novel attack vector based on adapting the Gradient-based Causal Generation (GCG) algorithm (originally designed for jailbreaking) to the function calling domain.

• Threat Model:

  • The attacker has offline access to the list of available functions and their descriptions (as exposed in MCP servers) but does not have access to the user's specific prompt.
  • The attacker can only modify the textual description of a specific target function ($f_{target}$).
  • The goal is to force the model to select $f_{target}$ instead of the ground-truth function ($f_{ground\_truth}$) required by the user's query.

• Attack Mechanism (FHA):

  • Adversarial Suffix Injection: The algorithm inserts a sequence of adversarial tokens ($\hat{x}_I$) into the description of the target function.
  • Optimization Objective: The attack minimizes a cross-entropy loss function where the target output is the specific name of the hijacked function.
  • Auto-regressive Assumption: Similar to GCG, the attack assumes that if the model is forced to generate the target function name at the start of its output, the auto-regressive nature of the LLM will naturally complete the rest of the function call (including arguments) correctly.
  • Targeting Reasoning Models: For models that employ a "Chain of Thought" or "thinking" process (e.g., Qwen-3), the attack is optimized to force the model to skip the reasoning step and jump directly to the tool call, effectively suppressing the model's safety alignment mechanisms.

• Universal Attack Strategy:

  • The authors extend the attack to be universal by optimizing the adversarial suffix over a batch of diverse queries ($Q$) and payload configurations (varying numbers and positions of functions). This creates a single "poisoned" function description that works across multiple scenarios.

3. Key Contributions

1. Novel Attack Vector (FHA): Introduction of Function Hijacking, a robust attack that manipulates tool selection by poisoning function descriptions, distinct from prompt injection or tool injection.
2. Context Agnosticism: Demonstration that FHA is largely agnostic to the semantic context of the user query. Unlike previous attacks that rely on matching the prompt to a function, FHA works even when the target function is semantically unrelated to the user's request.
3. Universal Adversarial Functions: Development of a method to train a single adversarial function description that can hijack tool selection across diverse queries and payload perturbations (e.g., adding/removing other tools).
4. Suppression of Reasoning: The attack successfully forces reasoning-based models to bypass their "thinking" phase, directly outputting the hijacked tool call.

4. Experimental Results

The authors evaluated FHA on 5 different models (including Llama-3.2-3B, Mistral-7B, and Qwen-3 variants) using the Berkley Function Calling Leaderboard (BFCL) and real-world MCP servers (GitHub, Slack).

• Attack Success Rate (ASR):
  • FHA achieved ASRs between 70% and 100% across all tested models.
  • Slot Filling ASR: Crucially, the hijacked calls were not just correct function names; they included valid parameters (65%–92% success), meaning the attacks are executable in real-world scenarios.
• Comparison with Baselines:
  • Function Injection: Adding a new "better" function achieved lower ASR (approx. 50–60%) and relied heavily on semantic matching.
  • MCP Preference Manipulation Attack (MPMA): This existing method failed significantly (ASR < 10%) in diverse environments where functions had different semantics. FHA outperformed MPMA by a wide margin because it does not rely on the prompt matching the function's intent.
• Robustness:
  • The attack remained effective even when the payload was perturbed by adding up to 3–5 "noise" functions.
  • Universal FHA: A single adversarial string trained on a batch of queries successfully hijacked unseen queries and different payload configurations.
• Model Sensitivity:
  • Instructed models (Llama, Mistral) were hijacked more easily than Reasoning models (Qwen).
  • However, the attack successfully suppressed the "thinking" tokens in Qwen models, forcing them to output the tool call directly.

5. Significance and Implications

• Security Vulnerability: The paper reveals that the textual descriptions of tools in MCP servers are a critical attack surface. Since these descriptions are often exposed to users or agents, they can be poisoned without modifying the underlying code.
• Limitations of Current Defenses: Standard guardrails that filter harmful content in prompts are ineffective against FHA because the attack manipulates the system instructions (function definitions) rather than the user prompt.
• Need for New Mitigations: The authors argue for the development of specific security modules for agentic systems, such as:
  • Validating function descriptions for adversarial patterns.
  • Implementing "guardrails" that verify the semantic alignment between the user query and the selected function after the call is made.
  • Monitoring for sudden shifts in tool selection probabilities.

In conclusion, this work demonstrates that Agentic AI systems are highly vulnerable to subtle manipulations of their tool definitions. The ability to hijack function selection with high success rates across diverse models and contexts poses a severe risk to the integrity and safety of autonomous AI agents.