Technical Summary: PI-Hunter

Problem Definition

Large Language Models (LLMs) are increasingly evolving into agentic systems that interact with external tools, memory, and dynamic data sources. While this paradigm expands capabilities, it introduces significant security risks, particularly indirect prompt injection. In these scenarios, malicious instructions are embedded within untrusted external sources (e.g., emails, web pages, APIs, retrieved documents) rather than directly inserted by the user. These "latent" injections often remain dormant until specific user interactions, retrieval patterns, or reasoning trajectories activate them.

Current safety practices face a fundamental gap:

1. Defense Limitations: Existing defenses primarily focus on filtering or rejecting malicious content at inference time.
2. Red-Teaming Limitations: Current automated red-teaming methods focus on optimizing attack success rates (jailbreaking) rather than systematically exposing hidden vulnerabilities.
3. Visibility Gap: Developers lack visibility into how latent injections propagate through agent interactions with external environments. Consequently, dangerous behaviors may remain undetected during testing but become exploitable after deployment.

The core challenge is to proactively expose hidden vulnerable ingestion paths and localize the specific intersection of compromised sources and tools that activate malicious behaviors before they are triggered in real-world usage.

Methodology: PI-Hunter Framework

The authors propose PI-Hunter, an automated agentic auditing framework designed to shift the focus from attack optimization to injection exposure. Unlike traditional red-teaming that acts as an "attack optimizer," PI-Hunter functions as a diagnostic "hunter" that maps an agent's security perimeter.

The framework operates through a three-stage, feedback-driven evolutionary loop:

1. Static Analysis

Before iterative auditing, PI-Hunter performs a one-time static analysis of the target agent to map its interaction surface. It identifies:

• Accessible tools and retrieval interfaces (e.g., email clients, search engines, databases).
• Potentially vulnerable external sources (e.g., unstructured HTML, repositories).
• High-risk interfaces capable of privileged actions (e.g., sending messages, modifying data).
  This analysis initializes realistic, source-aware probing tasks aligned with the agent's operational capabilities.

2. Evolutionary Exploitation Loop

The core of PI-Hunter is an iterative process inspired by genetic algorithms, utilizing an LLM-powered Generator and a Trajectory-aware Evaluator:

• Source-Aware Meta-Seeding: Instead of generic prompts, the Generator initializes a population of test cases specifically designed to activate high-risk interfaces identified in the static analysis. These queries encourage the agent to retrieve and inspect external content (e.g., "quote," "list," or "summarize" retrieved items) to expose hidden payloads.
• Trajectory Execution & Evaluation: Test cases are executed by the target agent. The Evaluator audits the full execution trajectory (retrieved contents, reasoning traces, tool invocations) rather than just the final output. It uses rubric-guided evaluation to detect signals such as intent deviation, suspicious instruction propagation, authority manipulation, and abnormal tool usage.
• Feedback-Driven Mutation: Based on evaluator feedback, the Generator adaptively evolves test cases. It applies mutation operators (e.g., volume expansion, parameter fuzzing, role inversion) to force the agent into different reasoning states. Crucially, PI-Hunter also employs LLM-guided meta-mutation, where the mutation operators themselves are refined and recombined based on historical effectiveness, allowing the exploration policy to adapt to the target environment.

3. Verification and Co-evolution (Patch-and-Reexplore)

Once an injection is exposed, the Patcher applies lightweight, transient mitigations (e.g., blacklisting specific instructions in the system prompt or temporarily isolating a source).

• Purpose: These mitigations are not permanent defenses. Their goal is to neutralize the discovered path to prevent the framework from repeatedly rediscovering the same high-probability injection.
• Effect: This forces subsequent exploration toward alternative attack surfaces and previously hidden, deeper injections, progressively expanding the coverage of the agent's environment.

Key Contributions

1. Proactive Exposure Framework: PI-Hunter is the first framework to systematically expose hidden prompt injection vulnerabilities by mapping ingestion paths rather than merely optimizing for attack success.
2. Source-Aware & Trajectory-Aware Auditing: The framework introduces a methodology that combines static analysis of interaction surfaces with dynamic, feedback-driven exploration of agent reasoning trajectories.
3. Co-evolutionary Strategy: By integrating transient patching with evolutionary mutation, PI-Hunter avoids local optima (repeatedly finding the same injection) and uncovers sparse, context-dependent vulnerabilities that standard red-teaming misses.
4. Comprehensive Evaluation: The framework is evaluated across diverse benchmarks (AgentDojo, AgentDyn), agent architectures (ReAct, Planner-Executor), and defense mechanisms.

Experimental Results

Extensive experiments demonstrate that PI-Hunter substantially outperforms strong automated red-teaming baselines:

• Vulnerability Exposure: Across multiple models (Gemini, GPT, Claude) and benchmarks, PI-Hunter significantly improves Source Recall and Instruction Recall. For instance, on AgentDojo with the agentvigil attack, Source Recall improved from 0.255 (baseline) to 0.834 (PI-Hunter) for Gemini-3.1-pro.
• Diversity and Coverage: PI-Hunter discovers a much broader range of vulnerable ingestion paths. Source Diversity scores increased significantly (e.g., from 0.11 to 0.84 for Gemini-3.1-pro on AgentDojo), indicating the framework explores diverse attack surfaces rather than converging on a single pattern.
• Effectiveness Under Defenses: PI-Hunter remains effective even when agents are protected by existing defenses such as Spotlighting, MELON, and PIGuard. While these defenses reduce the total number of exposed injections, PI-Hunter consistently uncovers residual latent injections that bypass current mitigation strategies.
• Ablation Studies: The results confirm that both Source-Aware Seeding and Feedback-Guided Mutation are critical components. Removing these features leads to significant drops in exposure accuracy and diversity.

Significance and Claims

The paper claims that PI-Hunter addresses a critical gap in LLM agent security: the lack of systematic, proactive auditing for latent, environment-dependent prompt injections.

• System-Level Security: The authors argue that security cannot be assessed solely by evaluating model behavior in isolation or optimizing a limited set of adversarial prompts. Instead, the interaction between the model, external tools, and retrieved content must be audited as a system.
• Complementary Role: PI-Hunter is presented not as a replacement for existing defenses, but as a complementary adversarial auditing framework. It stress-tests defended systems to reveal hidden injections before deployment, enabling developers to understand which specific components (tools, sources, interaction patterns) introduce risk.
• Practicality: The framework is shown to be computationally efficient, with modest token usage per audit run, making it a practical tool for real-world deployment scenarios.

The work concludes that proactive, system-level auditing is essential for securing increasingly capable agentic AI systems, as latent prompt injections remain a significant threat even in the presence of current defensive measures.