Technical Summary: ADR – An Agentic Detection System for Enterprise Agentic AI Security

1. Problem Statement

The rapid adoption of AI agents operating through the Model Context Protocol (MCP) has introduced a fundamentally new attack surface for enterprises. While MCP enables agents to interact dynamically with external tools and data, it also exposes organizations to novel threats such as prompt injection, tool manipulation, and credential exfiltration. The paper identifies three persistent challenges in securing these systems at an enterprise scale:

1. Limited Observability: Traditional Endpoint Detection and Response (EDR) tools capture file writes and network calls but lack visibility into the semantic context of agent actions. They cannot see the user prompts, agent reasoning steps, or the causal chain linking intent to execution, making it impossible to distinguish between benign automation and malicious exfiltration.
2. Insufficient Robustness: Static, rule-based defenses fail to generalize across the diverse landscape of agentic attacks. Furthermore, enterprise environments exhibit severe class imbalance (malicious events are rare), causing static defenses to either miss sophisticated attacks or generate unmanageable false positives.
3. High Detection Costs: Applying Large Language Model (LLM)-based reasoning to every agent event is computationally prohibitive at scale (e.g., processing 10,000+ daily sessions).

2. Methodology: The ADR System

The Agentic AI Detection and Response (ADR) system is an end-to-end framework designed to address these challenges through three integrated components, mirroring human security operations workflows:

A. ADR Sensor (Observability)

To close the observability gap, the ADR Sensor is a lightweight endpoint agent that reconstructs the full causal chain of agentic activity. Unlike network gateways that may miss context, the Sensor parses local caches (SQLite/JSONL) of agentic tools (e.g., Cursor, Cline, Claude Code) to capture:

• User Prompts: The original intent.
• Agent Reasoning: Intermediate thought processes and decision steps.
• MCP Tool Invocations: The sequence of tools called, arguments, and results.
• Environmental Context: MCP server configurations and installed packages.
  This high-fidelity telemetry allows the system to understand why an action occurred, not just what occurred.

B. ADR Detector (Online Detection)

To balance cost and accuracy, the Detector employs a two-tier architecture:

• Tier 1 (Triage): A lightweight, fast LLM-based screening layer designed for high recall. It flags suspicious signals (e.g., prompt injection phrases, credential access requests) and escalates uncertain events to Tier 2. Clearly benign traffic is short-circuited to minimize cost.
• Tier 2 (Reasoning): A deep, context-aware analysis layer. When an event is escalated, a reasoning agent dynamically queries three enterprise context providers via MCP:
  1. Source Code Inspection: Examines the implementation of the invoked tool to verify its actual behavior.
  2. Threat Intelligence: Checks against known attack tactics and techniques.
  3. Policy Verification: Validates actions against organizational security rules.
     This tier ensures high precision by grounding decisions in specific enterprise context.

C. ADR Explorer (Offline Robustness)

To address the lack of robustness and the scarcity of real-world attack data, the Offline Explorer functions as an automated red team. It uses an evolutionary algorithm with three collaborative agents:

• Red-Teaming Agent: Generates attack variants by mutating parameters and combining techniques.
• Eval Agent: Executes candidates in sandboxed environments to measure success and evasion.
• Threat Intelligence Agent: Curates successful, hard-to-detect attacks into a repository.
  This repository feeds back into Tier 2, continuously strengthening the detector's ability to recognize emerging attack patterns before production deployment.

3. Key Contributions

A. ADR-Bench

The authors introduce ADR-Bench, the first enterprise-grade benchmark for agentic security.

• Composition: 302 tasks (42 malicious, 260 benign) derived from real enterprise telemetry across 133 MCP servers.
• Coverage: Covers 17 distinct attack techniques across 5 threat tactics (e.g., Initial Access, Permission Abuse, Control Bypass), addressing the fragmented coverage of prior benchmarks like AgentDojo or AgentSafetyBench.
• Realism: Includes enterprise policy stores and highly imbalanced datasets to reflect production conditions.

B. Production Deployment

ADR was deployed at Uber for over ten months, scaling to over 7,200 unique hosts and processing 10,000+ agent sessions daily.

• Detection: Identified hundreds of credential exposures across 26 categories that had been inadvertently shared outside the network.
• Prevention: Enabled a "shift-left" prevention layer using hooks in development environments, achieving 97.2% precision in blocking credential leaks.

4. Results

Performance on Benchmarks

• ADR-Bench: ADR achieved zero false positives (Precision = 1.000) while detecting 67% of attacks (Recall = 0.667), resulting in an F1-score of 0.800.
  • Comparison: This significantly outperformed three state-of-the-art baselines (ALRPHFS, GuardAgent, LlamaFirewall), which suffered from high false positive rates (up to 40 FPs on 260 benign tasks) and F1-scores 2–4× lower than ADR.
• AgentDojo (Prompt Injection): ADR detected all 38 malicious tasks (Recall = 1.000) with only 3 false alarms, achieving an F1-score of 0.962.

Efficiency

• Cost: ADR costs $0.024 per task on ADR-Bench. While slightly more expensive than GuardAgent ($0.012), it is 19× cheaper than LlamaFirewall ($0.459).
• Latency: Average latency is 18.5 seconds. The hierarchical design ensures that 40.7% of tasks are handled by the low-cost triage layer (2.3s latency), reserving expensive reasoning only for suspicious events.

Ablation Studies

• Removing the Triage layer increased recall but introduced false positives and significantly raised costs/latency.
• Removing Source Code MCP context caused the largest drop in performance (15.5% decrease in F1), highlighting the necessity of understanding tool implementation to detect manipulation.

5. Significance and Claims

The paper claims ADR is the first large-scale, production-proven enterprise framework for securing AI agents operating through MCP. Its significance lies in:

1. Bridging the Observability Gap: By reconstructing the causal chain of prompts, reasoning, and tool execution, ADR enables semantic threat detection where traditional EDR fails.
2. Operational Viability: The two-tier architecture successfully balances the trade-off between the high cost of LLM reasoning and the need for high precision in enterprise environments, avoiding the "false alarm fatigue" that plagues other systems.
3. Systematic Robustness: The integration of an offline red-teaming engine (Explorer) allows the system to proactively discover and adapt to hard attack variants, addressing the dynamic nature of agentic threats.
4. Community Enablement: By releasing ADR-Bench, the ADR Sensor, and the detection framework, the authors provide the community with a standardized, reproducible method to evaluate agentic security systems under realistic enterprise constraints.

The authors conclude that while ADR demonstrates strong performance, future work involves extending the framework to multi-agent coordination protocols and tighter integration with evolving MCP standards to further close the gap between detection and response.