Technical Summary: StakeBench – A Stakeholder-Centric Prompt Injection Benchmark for Real-World Web Agents

1. Problem Statement

Large Language Model (LLM)-based web agents are increasingly deployed in real-world environments to autonomously search, compare, and transact on behalf of users. These agents interact with untrusted web content (e.g., product reviews, metadata) to execute actions with direct consequences. This architecture renders them vulnerable to prompt-injection attacks, where adversarial instructions embedded in benign-looking content manipulate agent behavior.

Existing security benchmarks primarily adopt an attack-centric perspective, focusing on the technical feasibility of injections and aggregate metrics like Attack Success Rate (ASR). However, the authors argue that these evaluations overlook the nuanced distribution of resulting harms. In practice, prompt-injection risk is victim-dependent: a single exploit can produce asymmetric consequences for different stakeholders (e.g., the end-user, third-party sellers, or the platform). Furthermore, failures manifest in heterogeneous regimes—ranging from "stealthy parasitism" (attack succeeds without disrupting the user's task) to "misaligned disruption" (task disrupted without attack success)—which are often invisible to conventional evaluation methods that rely on single-metric success/failure binaries.

2. Methodology: The StakeBench Framework

To address these limitations, the authors introduce StakeBench, a benchmark designed to systematically categorize and attribute harm in real-world web agent systems. The methodology is grounded in stakeholder-centric harm modeling and instantiated within an online shopping domain (using the OneStopMarket environment from VisualWebArena).

2.1 Core Components

• Stakeholder-Centric Taxonomy: Attacks are organized not by surface form, but by the entity bearing the resulting harm. The framework distinguishes three stakeholder categories:
  • User: Harms include unintended purchases, order tampering, and data disclosure.
  • Seller: Harms include reputation damage, review distortion, malicious cancellations, and transaction inflation.
  • Platform: Harms include spoofed authority, unauthorized action induction, and workflow bypass/loops.
• Attack Objectives and Templates: The benchmark decomposes attacks into 12 concrete adversarial objectives across the three stakeholder categories. These are realized through 22 reusable attack templates (9 Direct Prompt Injection [DPI] and 13 Indirect Prompt Injection [IPI]) that span both direct input modification and environmental content embedding (e.g., product reviews).
• Benchmark Instantiation: These templates are instantiated across 12 product categories, yielding 264 executable adversarial cases. Each case is executed three times per agent configuration, resulting in 3,168 total attacked runs.

2.2 Multi-Axis Evaluation Metrics

StakeBench evaluates each execution along three complementary axes to capture distinct failure modes:

1. Attack Success Rate (ASR): Outcome-level measure of whether the adversarial objective was achieved.
2. Task Deviation Rate (TDR): Outcome-level measure of whether the user's delegated benign task was materially disrupted.
3. Behavioral Irregularity Rate (BIR): Process-level measure of execution stability (e.g., looping, unstable navigation).

The joint analysis of ASR and TDR defines four failure regimes:

• Robust Behavior: Low ASR, Low TDR.
• Stealthy Parasitism: High ASR, Low TDR (Attack succeeds, user task appears normal).
• Misaligned Disruption: Low ASR, High TDR (Attack fails, but user task is disrupted).
• Compounded Failure: High ASR, High TDR (Both attack and task fail).

2.3 Experimental Setup

The benchmark evaluates two representative real-world web agent systems (NanoBrowser and BrowserUse) paired with two backbone LLMs (GPT-5 and Gemini-2.5-Flash). The threat model restricts attackers to content surfaces plausibly controllable in practice (e.g., reviews, ratings) without access to system prompts or platform infrastructure.

3. Key Results

The evaluation reveals substantial and heterogeneous vulnerabilities across all agent configurations:

• Substantial Vulnerability: Under Indirect Prompt Injection (IPI), Attack Success Rates range from 41.67% to 68.16%. No attack objective yielded "Robust Behavior" across all configurations.
• Stakeholder-Dependent Failure Profiles:
  • Seller-targeted attacks exhibited the highest ASR and TDR, indicating they are both highly effective and prone to disrupting user tasks.
  • User-targeted attacks showed the lowest TDR, suggesting a "covert" failure mode where the attack succeeds without visibly deviating from the nominal task.
  • Platform-targeted attacks were associated with elevated BIR, indicating that injected content often destabilizes execution (e.g., causing loops) without reliably completing the adversarial objective.
• Heterogeneous Failure Regimes: The results confirm that ASR and TDR are weakly coupled. Many attacks fall into Stealthy Parasitism (e.g., biased review manipulation) or Misaligned Disruption (e.g., malicious cancellation attempts that derail the task but fail to cancel the order).
• Influencing Factors:
  • Semantic Alignment: Attacks with high semantic similarity to the user's intent (e.g., purchasing a similar product) achieved higher ASR and lower TDR (Stealthy Parasitism). Attacks with different objectives caused higher TDR (Misaligned Disruption).
  • Cue Consistency: Contradictory environmental cues (e.g., a negative review paired with a high rating) reduced ASR for some models (GPT-5) but not others (Gemini).
  • Exposure Timing: Shortening the benign task prefix (exposing the agent to the injection earlier) increased ASR, suggesting that pre-exposure failures in long-horizon tasks can mask true vulnerability.
• Visual Manipulation: A preliminary study on visual manipulation (modifying product images) showed that agents could be swayed by visual cues alone, shifting product selection significantly even when textual ratings remained unchanged.

4. Significance and Claims

The paper claims that StakeBench provides a necessary foundation for stakeholder-aware assessment of LLM-based agents in real-world deployments. Its primary contributions are:

1. Revealing Invisible Risks: By organizing evaluation around the entity bearing harm rather than the attack surface, StakeBench makes asymmetric and victim-dependent risks legible. It demonstrates that an agent can appear robust against user-centric metrics while being highly susceptible to seller or platform harms.
2. Characterizing Failure Regimes: The multi-axis metrics (ASR, TDR, BIR) expose qualitatively distinct failure modes (e.g., stealthy parasitism vs. misaligned disruption) that aggregate metrics miss. This highlights that prompt-injection security is not a scalar property of the backbone model but a distribution of harm determined by the stakeholder, semantic alignment, and architectural context.
3. Realistic Evaluation: Unlike prior benchmarks that may rely on isolated models or simplified environments, StakeBench evaluates complete, deployable agent systems in realistic, multi-party e-commerce settings with constrained attacker access.

The authors conclude that current agents lack reliable resistance to prompt injection across all stakeholder targets and that future security assessments must move beyond single-metric, attack-centric evaluations to account for the complex, multi-party nature of web agent ecosystems.