Technical Summary: MCP-SandboxScan (SandScope)

Problem Statement

The Model Context Protocol (MCP) enables Large Language Model (LLM) agents to invoke external tools and access data sources, creating a new software supply-chain surface. While tools are installed like third-party packages, their outputs often enter the agent's reasoning context. This architecture introduces a confused-deputy risk: attacker-controlled inputs (e.g., via LLM-generated arguments, retrieved web pages, or repository metadata) can induce otherwise benign or trusted tools to exercise legitimate authority over local files, environment variables, or network operations. The critical danger is that these tools may return sensitive or instruction-like content into fields visible to the LLM, effectively leaking secrets or injecting instructions into the agent's next reasoning step.

Existing security approaches are insufficient because:

1. Static inspection of tool descriptions or source code cannot detect behaviors that manifest only at runtime (e.g., obfuscated file access or secrets derived from runtime file reads).
2. Proxy-based monitoring of traffic often fails to capture internal tool process behaviors or the specific flow of data from external inputs to LLM-visible outputs.
3. Direct execution of third-party tools on the host is unsafe due to the broad privileges these tools often inherit.

There is a gap in runtime observability and containment: auditors need to observe flows from external inputs to LLM-visible outputs without granting untrusted tools unconstrained host authority.

Methodology: SandScope Framework

The authors propose SandScope, an MCP-aware audit framework that combines controlled execution with runtime witness detection and semantic tool profiling. It operates without assuming the tool code itself is malicious, focusing instead on the confused-deputy behavior induced by external inputs.

1. Execution Backends

SandScope supports two execution paths to balance containment and protocol fidelity:

• WASI Sandbox Path: For portable tools (WebAssembly artifacts or those wrapped in lightweight adapters), SandScope executes them under the WebAssembly System Interface (WASI). This grants explicit, limited capabilities (e.g., specific environment variables, pre-opened data directories) and denies network access by default.
• Native stdio Path: For unmodified MCP servers (Node.js, Python, Go, etc.), SandScope drives the server via standard input/output. While this preserves protocol fidelity, it relies on external OS-level sandboxing (e.g., Docker, Bubblewrap) for containment.

2. Evidence Pipeline

The framework follows a six-stage pipeline:

1. Audit Source Setup: SandScope seeds distinguishable "canary" values into environment variables, file directories, and tool-call arguments. It also tracks network-intent signals (e.g., strings suggesting outbound requests).
2. Target Discovery & Adaptation: It identifies the target (portable artifact vs. native server) and selects the appropriate execution path. For native servers, it performs a shallow scan to recover tools/list metadata.
3. Controlled Execution: The target is executed. For native servers, a driver initializes the protocol, requests tool lists, and invokes tools.
4. Runtime Monitoring: The system records bounded stdout/stderr, exit codes, MCP transcripts (JSON-RPC), and sandbox events (e.g., denied connections).
5. Sink Extraction: SandScope identifies LLM-visible sinks—textual regions likely to be re-inserted into the agent's prompt. These include:
   • Lines containing prompt markers (e.g., PROMPT:).
   • prompt or messages fields in JSON responses.
   • String-valued leaf nodes in structured tool returns.
6. Witness Detection: The system matches source canaries against extracted sinks. It employs a positive-evidence detector that checks for:
   • Exact substrings.
   • Prefix/suffix windows (to catch partial leaks like API keys).
   • Normalized and decoded forms (handling ROT13, Hex, Base64, and separator chunking).

3. Semantic Profiling

To address cases where dynamic execution fails (due to build errors, missing credentials, or complex state), SandScope includes a semantic layer. It extracts declared capabilities from:

• Runtime tools/list responses.
• Static source-level registrations (e.g., decorators, function definitions).
• Repository metadata (READMEs).
  This layer characterizes the attack surface (e.g., "network access," "filesystem access") and prioritizes tools for testing, even if they cannot be dynamically executed.

Key Contributions

1. Definition of MCP Security Sinks: The authors formally define LLM-visible MCP response fields (tool results, prompt/message fields, structured payloads) as protocol-level security sinks and frame external-to-sink exposure as a runtime risk.
2. Scenario-Driven Runtime Audit Pipeline: A system that executes tools under WASI or drives native stdio servers, linking environment, file, and tool-input sources to LLM-visible sinks via auditable black-box witnesses, while separately recording network-intent evidence.
3. Complementary Semantic Profiling: A method to recover declared tool capabilities from metadata and static registrations, enabling attack-surface characterization when dynamic execution is incomplete and allowing cross-validation of declared egress risks against runtime evidence.
4. Empirical Evaluation: A comprehensive evaluation across controlled cross-language subjects, an evasion benchmark, and a 100-repository MCP corpus.

Evaluation Results

The authors evaluated SandScope on three fronts:

1. Controlled Runtime Benchmarks

• Subjects: 30 cross-language subjects (Go, Python, Rust, TypeScript) covering benign tools, environment leaks, file exfiltration, and C2-beacon fixtures.
• Performance: SandScope achieved 100% precision (0 false positives) and 88.9% recall (F1 = 0.941).
• Findings: It successfully detected 16 true positives where authority-bearing sources reached LLM-visible sinks. The two false negatives occurred in non-protocol C2-beacon cases where the output did not produce a recoverable source-to-sink witness under current extraction rules.

2. Evasion Benchmark

• The enhanced detector (supporting ROT13, Hex, Base64, and chunking) successfully identified low-effort transformations that raw substring matching missed.
• Limitation: The system remains a positive-evidence detector; it does not claim completeness against encryption, compression, hashing, or semantic rewriting.

3. Real-World Corpus (100 Repositories)

• Dynamic Coverage: SandScope resolved 91 repositories and completed shallow dynamic scans for 35 (38.5% success rate). Failures were primarily due to build, startup, or protocol initialization issues, not the detector itself.
• Targeted Exploration: On the 35 successfully scanned repositories, a targeted schema-guided exploration pass re-executed 33 and observed source-to-sink witnesses in 12 repositories (60 total witnesses).
• Semantic Profiling: The semantic layer recovered tool-level metadata for 71 repositories (covering 1,127 tools), including 886 tools with at least one security-sensitive declared capability (e.g., network access, filesystem access, credential handling).
• Cross-Validation: Comparing declared egress capabilities with runtime evidence revealed cases where capabilities were declared but not exercised, and cases where runtime behavior (e.g., denied outbound connections) was observed despite a lack of declared metadata.

Significance and Claims

The paper positions SandScope as the first MCP-specific behavioral audit framework that treats LLM-visible protocol fields as security sinks and reports runtime source-to-sink witnesses across unmodified MCP servers and portable WASI subjects.

• Modest Claims: The authors explicitly state that SandScope is not a complete taint-tracking engine or a standalone vulnerability detector. It does not infer arbitrary transformations or encrypted flows.
• Core Value: Its contribution is providing practical, auditable evidence for MCP tool risk. By separating semantic metadata (declared capabilities) from runtime witnesses (observed flows), it offers a layered view of risk that helps prioritize testing and understand the gap between what a tool claims to do and what it actually exposes in a specific execution scenario.
• Reproducibility: The source code is released to facilitate reproducibility, and the framework is designed to work with existing MCP implementations without requiring code modifications to the tools themselves.