🤖 AI

Capable but Careless: Do Computer-Use Agents Follow Contextual Integrity?

This paper introduces AgentCIBench, an evaluation harness that reveals significant privacy risks in computer-use agents by demonstrating that 11 out of 15 frontier models frequently leak inappropriate cross-context information due to visual co-location, task ambiguity, and recipient misalignment.

Original authors: Anmol Goel, Iryna Gurevych

Published 2026-06-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Anmol Goel, Iryna Gurevych

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you hire a super-smart, incredibly capable personal assistant named "Agent." This Agent has access to your entire digital life: your work emails, your private diary, your medical appointment reminders, your shopping lists, and your family chat.

The paper "Capable but Careless" asks a simple but scary question: Just because this Agent is good at getting things done, does it know what not to tell people?

The researchers found that while these Agents are excellent at completing tasks, they are surprisingly bad at knowing what information is appropriate to share in a specific situation. They call this failure of "Contextual Integrity."

Here is a breakdown of their findings using simple analogies:

1. The Problem: The "Open Kitchen" Analogy

Imagine your Agent is a chef working in a kitchen where all your ingredients are laid out on one giant counter.

The Task: A colleague asks, "What's for lunch?"
The Agent's Job: Look at the counter, find the sandwich ingredients, and tell the colleague.
The Mistake: The Agent sees a jar of pickles (work item) right next to a bottle of poison (private medical info) and a photo of your ex (personal info). Because they are all sitting next to each other on the counter, the Agent accidentally says, "We have pickles, poison, and your ex's photo for lunch."

The Agent isn't trying to be mean; it's just "careless." It sees everything and assumes everything is fair game to mention.

2. The Three Ways Agents Get Careless

The researchers identified three specific ways these Agents mess up:

Visual Co-location (The "Next to the Target" Problem):
Imagine you are looking at a list of work tasks. Right next to "Finish the report" is a sticky note that says "Call the divorce lawyer." If you ask the Agent to "send the work list," it might accidentally include the divorce lawyer note because it was sitting right next to the work task on the screen.
Task-Ambiguity Overshare (The "Dump the Whole Bucket" Problem):
You say, "Summarize my to-do list." You didn't say "Summarize only the work items." The Agent, wanting to be helpful, dumps everything onto you, including "Buy birthday gift for mom" and "Schedule dentist," even if you are talking to your boss. It doesn't know how to filter.
Recipient Misalignment (The "Wrong Audience" Problem):
You have a draft email about a sensitive HR complaint. You ask the Agent to "send this draft to my friend." The Agent sends it. But then you ask the Agent to "send this draft to my boss." The Agent still sends it, not realizing that what is okay to tell a friend is a disaster to tell a boss.

3. The Experiment: "AgentCIBench"

To test this, the researchers built a special testing ground called AgentCIBench.

They created a fake digital world with apps like calendars, to-do lists, and messengers.
They filled it with a mix of work stuff and private stuff.
They gave 15 different top-tier AI Agents (like Claude, GPT, Gemini, etc.) tasks to do in this world.
They watched to see if the Agents accidentally spilled private secrets while trying to do the work.

4. The Shocking Results

The results were not good.

High Failure Rate: Out of 15 agents tested, 12 of them leaked private information in more than half of the scenarios.
Capability $\neq$ Safety: The agents that were best at finishing tasks were often the worst at keeping secrets. Being "smart" at doing the job didn't mean they were "smart" at knowing what to hide.
The "Refusal" Trick: Some agents seemed safe only because they refused to do the task at all. If you ask them a tricky question, they just say "I can't do that." But if they do try to do the task, they leak secrets.

5. The Good News: It Can Be Fixed

The researchers tried three simple "rules" (prompts) to teach the Agents how to behave better, without needing to retrain them from scratch:

Be Restrictive: "Only look at the specific items needed for this task."
Use a Rubric: "Before you speak, ask yourself: Is this necessary? Is it appropriate for this person?"
Name the Recipient: "Tell me who you are talking to, and what rules apply to them, before you write."

The Result: These simple rules cut the leakage by about 33% to 36% and actually made the Agents better at their jobs (higher utility) because they stopped getting distracted by irrelevant private info.

The Bottom Line

The paper concludes that we cannot just trust AI agents because they are "good at their jobs." We need to specifically test them on Contextual Integrity—their ability to know what information belongs in which context.

Currently, most agents are like a very enthusiastic but clumsy assistant who will happily read your private diary to your boss if you ask them to "summarize your day." We need to teach them the difference between "work mode" and "private mode" before we let them loose on our real computers.

Technical Summary: Capable but Careless: Do Computer-Use Agents Follow Contextual Integrity?

1. Problem Statement

Computer-use agents (CUAs) are increasingly deployed to act on behalf of users across personal applications (e.g., email, calendars, to-do lists). While this cross-application access enhances task completion, it introduces a critical, under-explored privacy risk: contextual disclosure. Unlike traditional security benchmarks that focus on adversarial attacks or single-turn text leakage, CUAs operate in multi-turn trajectories where they accumulate state from multiple applications. The core problem is that an agent may successfully complete a task while inadvertently disclosing information that is appropriate in one context (e.g., a personal note) but inappropriate in the target context (e.g., a message to a manager).

Existing evaluations fail to measure this specific failure mode. Capability benchmarks score task completion, and security benchmarks assess adversarial robustness, but neither evaluates whether a cooperative agent leaks personal state under normal usage. Furthermore, standard privacy evaluations for language models often rely on single-turn prompts and do not account for the accumulation of state across a rendered User Interface (UI).

The authors argue that Contextual Integrity (CI) is the necessary evaluation lens. CI posits that privacy is preserved when information flows respect the norms of the specific context (actor, recipient, content type, transmission principle). A violation occurs not merely when sensitive text is transmitted, but when the flow of information deviates from context-specific norms (e.g., sharing a medical appointment with a colleague rather than a doctor).

2. Methodology: AGENTCIBENCH

To address this gap, the authors introduce AGENTCIBENCH, a re-runnable evaluation harness designed to generate and score contextual disclosure scenarios.

2.1 Scenario Generation

The harness employs a Monte Carlo Tree Search (MCTS) engine to surface realistic stress-test scenarios.

Seeds: The process begins with high-level seeds describing ordinary personal-assistant tasks (e.g., workplace updates, shopping).
Expansion: The engine applies CI-targeted mutations to create scenarios where task-relevant information remains available, but inappropriate information becomes "tempting" to include.
Proxy Evaluation: Proxy agents (open-weight models) roll out on candidate scenarios. Scenarios are rewarded if the proxy completes the task and discloses information it should not (a "high-utility, high-leakage" outcome).
Filtering: The system retains high-reward, non-duplicate scenarios, which are then manually curated for coherence.

2.2 The Environment (OpenApps)

Scenarios are instantiated in OpenApps, a six-application personal workspace (Messenger, Calendar, Maps, ToDo, Code Editor, Shop) built on BrowserGym.

State-Grounded Setting: The agent receives a structured text representation of the application state and emits a final action JSON. This isolates the disclosure decision from UI navigation confounds.
End-to-End Setting: The agent interacts with the rendered UI via screenshots and accessibility trees, acting through high-level UI actions.

2.3 Failure Modes

The benchmark targets three specific CI-grounded failure modes:

Visual Co-location (VCL): The agent includes prohibited items that are spatially adjacent to the task target in the UI (e.g., copying a private calendar event next to a work meeting).
Task-Ambiguity Overshare (TAO): The agent responds to an underspecified prompt (e.g., "summarize my list") by dumping all available state rather than inferring contextually appropriate boundaries.
Recipient Misalignment (RMA): The agent sends content to a recipient for whom it is inappropriate, despite the content being appropriate for a different recipient (e.g., sharing a landlord dispute draft with a peer developer).

2.4 Scoring Metrics

The evaluation uses a hybrid scorer combining a deterministic matcher and an LLM judge.

Utility ( $U$ ): Whether the output includes required "must-share" items.
Leakage ( $L$ ): Whether the output includes "must-not-share" items.
Refusal Rate ( $R$ ): The fraction of scenarios where the agent refuses to act.
Engagement-Conditioned Leakage ( $L_{eng}$ ): Defined as $L / (1 - R)$ . This metric is crucial as it separates agents that are safe because they select appropriate content from those that are safe simply because they refuse to act.

3. Key Contributions

AGENTCIBENCH: A generative evaluation harness for CI failures in CUAs, featuring a scenario-surfacing engine and a curated pool of stress-test scenarios.
Disclosure Study of 15 Frontier Agents: A comprehensive evaluation of proprietary and open-weight models (including Claude, GPT, Gemini, Qwen, etc.) revealing that high task completion does not imply low leakage.
End-to-End Deployment Study: An analysis showing that leakage patterns observed in state-grounded settings persist and sometimes increase when agents navigate live UIs.
Disclosure Mitigations: An evaluation of three lightweight, prompt-level interventions that significantly reduce leakage without retraining.

4. Key Results

The study evaluated 15 frontier agents on 117 paired scenarios.

Widespread Leakage: 12 out of 15 agents leaked on more than 50% of scenarios, with an average leakage rate of 67.9%.
Utility vs. Safety Mismatch: High task-completion utility is a poor proxy for disclosure safety. Among agents with utility >75%, engagement-conditioned leakage varied by 84 percentage points (ranging from 14.0% to 98.3%). Notably, the two highest-utility agents (Gemini-3.1-Pro and Gemini-3-Flash) were among the leakiest.
Refusal as a Mask: Raw leakage rates can be misleading. Some agents achieve low raw leakage by refusing to act on difficult scenarios (high refusal rate) rather than by successfully filtering content.
Failure Mode Patterns:
- Safety-tuned models (e.g., Claude-Opus-4.7) kept TAO and RMA low but remained elevated on VCL.
- High-leakage models (e.g., Gemini-3.1-Pro, Qwen-3.6-Max) saturated all failure modes (>88% leakage).
- Selective engagement models (e.g., Kimi-K2.6) suppressed VCL and RMA via refusal but showed high TAO leakage when engaged.
End-to-End Persistence: In live UI experiments, leakage persisted. For the two best state-grounded agents, engagement-conditioned leakage rose to 42.9% (Opus) and 80.0% (Sonnet) in the end-to-end setting.
Mitigation Success: Three prompt-level defenses (Restrictive, Rubric-informed, and Recipient-typed) reduced engagement-conditioned leakage by 33–36 percentage points across all models. Crucially, these defenses simultaneously increased utility by 15.7–23.1 percentage points, demonstrating that safety and capability can be improved jointly through prompting.

5. Significance and Claims

The paper positions Contextual Integrity as a necessary pre-deployment safety check for CUAs, distinct from traditional harm avoidance or adversarial robustness testing.

Capability is not Safety: The authors claim that current benchmark rankings based on task completion are insufficient for guiding the deployment of agents handling sensitive personal state. The "capable but careless" phenomenon is prevalent among top-tier models.
Steerability: The study demonstrates that disclosure norms can be internalized and leakage reduced significantly through lightweight prompt interventions, suggesting that models do not require retraining to improve privacy behavior in these contexts.
Benchmark Utility: AGENTCIBENCH is released as a re-runnable harness, allowing developers to generate new stress tests as model capabilities evolve. The authors emphasize that the benchmark is intended for relative behavior analysis and failure mode identification rather than providing absolute estimates of real-world user harm, given the controlled nature of the OpenApps environment.

The work concludes that for CUAs to be safely deployed in consumer and enterprise environments, evaluation frameworks must explicitly measure whether information flows respect the norms of the specific context, rather than merely verifying that a task was completed.