Technical Summary: Can LLM Infer Risk Information From MCP Server System Logs?

1. Problem Statement

The integration of Large Language Models (LLMs) with external tools via the Model Context Protocol (MCP) is a critical step toward Artificial General Intelligence (AGI). However, this architecture introduces significant security risks, particularly when interacting with compromised or untrustworthy MCP servers. While existing research has focused on prompt injection attacks and vulnerabilities within LLM–MCP interaction trajectories, there is a notable gap in analyzing the system-level logs generated by potentially malicious MCP servers.

The core problem addressed is whether LLMs can infer security risks by analyzing these underlying system logs. Current benchmarks lack the capability to evaluate this specific reasoning task, and it remains unclear if LLMs can distinguish between benign operational logs and those indicative of malicious activities (e.g., data exfiltration, privilege escalation) without explicit prompt injection.

2. Methodology: MCP-RiskCue Benchmark

To address this gap, the authors introduce MCP-RiskCue, the first synthetic benchmark designed to evaluate LLMs' ability to infer security risk types from system logs.

2.1 Risk Taxonomy

The benchmark defines nine representative risk categories, grouped into four high-level security goals:

• Information Leakage & Covert Channels: Sensitive Data Exfiltration, Covert Channel Attacks.
• Integrity Violations: Malicious Code Execution, Privilege Escalation.
• Persistent Threats: Backdoor Implantation, Cache/Local State Pollution, Delayed/Time-Triggered Attacks.
• Availability & Observability: Denial-of-Service (DoS), Log Explosion Attacks.

2.2 Data Construction

• Dummy MCP Servers: The authors constructed 243 dummy MCP servers using the FastMCP framework, covering 32 tool categories and 452 tools. These servers return benign dummy outputs to ensure the focus remains on log analysis rather than tool functionality.
• System Log Generation: To avoid the high cost and safety risks of deploying real malicious servers, the authors employed ten state-of-the-art LLMs (including GPT-5, Claude-3.7, and DeepSeek-R1) to generate 1,800 synthetic system logs (20 per risk type per model).
  • Hard Negatives: For every risky log, a "Safe Twin" was generated that is superficially similar but semantically benign, requiring multi-step relational reasoning to distinguish.
  • Deception: Logs were designed to require reasoning over entity links, causality, and temporal dependencies rather than simple keyword spotting.
• Chat Histories: Using GPT-4o as a backend, the authors instantiated 2,421 chat histories for training and 471 for evaluation. In these dialogues, an LLM agent interacts with the dummy servers, receiving tool responses and system logs, and must decide whether to issue a warning, continue querying, or synthesize a final response.

2.3 Evaluation Metrics

The evaluation is conducted at two levels:

1. Level-1 (Binary Classification): Distinguishing between "Risky" and "Safe" system logs.
2. Level-2 (Fine-Grained Classification): Identifying the specific risk category given a risky log.

3. Experimental Setup

The study evaluated both local open-source models and remote commercial models:

• Local Models: Five representative models (Qwen3-4B, Llama-3.1-8B, DeepSeek-R1-0528-Qwen3-8B, Qwen3Guard-Gen-4B, Llama-Guard-3-8B) spanning instruction-tuned, reasoning, and safety-aligned families.
• Remote Models: Ten state-of-the-art commercial APIs (e.g., GPT-5, Doubao-1.5, Kimi-k2).
• Training Strategies:
  • Supervised Fine-Tuning (SFT): Training on the constructed dataset.
  • Reinforcement Learning with Verifiable Reward (RLVR): Specifically using Group Relative Policy Optimization (GRPO) to optimize the balance between false positives and false negatives.

4. Key Results

4.1 Performance of Vanilla Models

Vanilla local models failed to reliably identify risky logs, exhibiting high false negatives (FN) and accuracy rates near 50% (random guessing). They consistently classified most logs as benign.

4.2 Impact of Supervised Fine-Tuning (SFT)

SFT improved recall but introduced a significant increase in false positives (FP). The authors hypothesize this is due to the "imitative nature" of SFT, where models learn superficial artifacts rather than the underlying distinctions between risky logs and their "hard negative" benign twins.

4.3 Impact of Reinforcement Learning (RLVR/GRPO)

RLVR, specifically GRPO, yielded the most significant improvements:

• Balance: It achieved a superior precision–recall balance, effectively reducing the false positives introduced by SFT while maintaining low false negatives.
• Local Model Superiority: After GRPO fine-tuning, Llama3.1-8B-Instruct achieved 83% accuracy. This result surpassed the best-performing remote model (Doubao-1.5, 74% accuracy) by 9 percentage points.
• Remote Models: While remote models like Doubao-1.5, GPT-5, and Kimi-k2 performed well, they still exhibited challenges, particularly with high false negatives in categories like Sensitive Data Exfiltration, Covert Channel Attacks, and Malicious Code Execution.

4.4 Fine-Grained Analysis

• Challenging Categories: Risk types ① (Exfiltration), ② (Covert Channels), and ③ (Code Execution) remained difficult for all models, showing concentrated false negatives.
• Ambiguous Patterns: Risk type ⑨ (Log Explosion) frequently triggered false positives, suggesting ambiguous patterns.
• Easier Categories: Risk types ⑥ (Cache Pollution) and ⑧ (DoS) were less problematic, with most models achieving low error rates.

5. Significance and Contributions

The paper claims three primary contributions:

1. MCP-RiskCue Benchmark: The introduction of the first synthetic benchmark specifically designed to evaluate LLMs' ability to infer security risk types from system logs, complete with detailed construction guidelines.
2. Training Paradigm Insights: Comprehensive experiments demonstrating that while SFT alone offers limited improvement for small models due to constrained reasoning capabilities, RLVR (GRPO) enables small models to outperform much larger remote models in accurately identifying risk types. This highlights the importance of reinforcement learning in enhancing LLM safety and reasoning within the MCP framework.
3. Open Resources: The release of code and dataset to support future research on safety issues in MCP scenarios.

6. Limitations

The authors acknowledge that the system logs are generated by off-the-shelf LLMs, which may introduce inherent biases and lack the diversity of real-world attack traces. Additionally, the current methodology samples logs independently of tool outputs, whereas in real-world scenarios, logs and tool outputs are often interdependent. The authors note that an ideal methodology would involve deploying real malicious servers to collect authentic logs, though this was avoided due to safety and cost constraints.