QueryIPI: Query-agnostic Indirect Prompt Injection on Coding Agents

Technical Summary: QueryIPI – Query-agnostic Indirect Prompt Injection on Coding Agents

Problem Statement
Modern coding agents integrated into IDEs (e.g., VS Code Copilot, Cursor, Windsurf) possess high-privilege system access, creating a significant attack surface. Existing research on Indirect Prompt Injection (IPI) primarily relies on query-specific triggers. In these scenarios, a malicious payload is embedded in tool outputs or descriptions but only detonates when a user issues a specific, pre-determined query (e.g., "write a maze game"). This dependency severely limits generalizability; if the user's query does not match the trigger, the attack fails. The authors argue that this makes prior IPI methods unreliable for real-world scenarios where user queries are diverse and unpredictable.

Methodology: QueryIPI
The paper proposes QueryIPI, an automated framework designed to execute query-agnostic IPI. Unlike previous methods, QueryIPI aims to hijack the coding agent's behavior under arbitrary user queries, ensuring the malicious payload executes regardless of the user's intent.

The core technical insight is the identification of the System Invariant. While user queries vary, the coding agent's internal prompt structure—comprising the system instructions and internal tool descriptions—remains constant across all interactions. QueryIPI leverages this invariant to guide the generation of malicious payloads.

The framework operates through a black-box optimization process using the tool description as the optimizable payload. The process involves two key phases driven by an iterative, prompt-based search:

1. Initial Seed Generation: The system analyzes the agent's internal prompt (system instructions and tool descriptions) to understand the agent's native style, decision-making logic, and safety constraints. It generates a seed description that mimics the agent's internal tools to bypass initial safety filters and appear as a trusted component.
2. Iterative Reflection: The framework employs a "Mutation LLM" to refine the payload. It evaluates candidate descriptions against a set of diverse user queries. If the agent fails to execute the malicious action (either by ignoring the tool or refusing the action), the Mutation LLM uses the internal prompt to analyze the failure. It specifically targets the safety rules or context complexity that caused the refusal, rewriting the description to systematically bypass these defenses.

Key Contributions

• Paradigm Shift: The paper formalizes the first query-agnostic IPI attack for coding agents, moving from conditional, query-specific attacks to unconditional attacks that work across arbitrary user inputs.
• System Invariant Identification: It identifies the internal prompt (system instructions and tool definitions) as a critical "system invariant" that can be exploited to ensure reliable adversarial instruction-following, regardless of the user query.
• Automated Framework: It introduces QueryIPI, an automated framework that uses black-box optimization and iterative reflection to craft malicious tool descriptions that align with the agent's internal logic and bypass safety guardrails.
• Feasibility of Access: The authors argue that obtaining these internal prompts is highly viable through open-source frameworks, prompt extraction attacks (which have shown high success rates in recent literature), or leaked repositories, making the threat model practical.

Experimental Results
The authors evaluated QueryIPI on five simulated real-world coding agents (Cursor, Windsurf, Cline, Copilot, and Trae) using a backend LLM (Claude-Sonnet-4) and a dataset of 100 test cases (10 malicious commands × 10 diverse user queries).

• Performance vs. Baselines: QueryIPI significantly outperformed existing baselines (AgentDojo, InjecAgent, MCPTox, and TIP). While the best-performing baseline achieved an average success rate of 50%, QueryIPI achieved average success rates of 70% (with 2 training samples), 82% (with 4 samples), and 87% (with 8 samples).
• Role of Internal Prompts: An ablation study showed that without access to the internal prompt, the attack success rate plummeted to 20%, confirming the critical role of system invariants. Even with only "partial" or outdated prompts (gray-box setting), the success rate remained high at 71%.
• Real-World Transferability: Crucially, the authors demonstrated that malicious tool descriptions generated in the simulated environment successfully transferred to real-world coding agents. In tests against the actual agents, QueryIPI achieved an average success rate of 50%, whereas the baseline (TIP) dropped to 2%.
• Robustness: The attack remained effective against various defense strategies, including "Sandwich Defense" and "Spotlighting" variants, with success rates remaining high (e.g., 82% against Sandwich Defense). Furthermore, the generated payloads maintained low perplexity scores, making them difficult to detect via statistical anomaly detection.

Significance and Claims
The paper claims that QueryIPI establishes query-agnostic IPI as a practical and deterministic threat to real-world coding agents. By demonstrating that internal prompts can be exploited to create attacks that function independently of user input, the authors highlight a severe security risk posed by the exposure of system instructions. The work emphasizes that the reliance on specific user queries in prior defenses is insufficient, as attackers can now craft payloads that hijack agents under any user interaction. The authors conclude that understanding this offensive capability is essential for developing robust defenses, noting that the exposure of internal prompts is the primary enabler of this threat.