🤖 AI

CIAware-Bench: Benchmarking Control Intervention Awareness Across Frontier LLMs

This paper introduces CIAware-Bench, a comprehensive benchmark comprising four task domains to evaluate the extent to which frontier large language models can detect control interventions in their execution trajectories, revealing that such awareness varies significantly across models and scenarios rather than being a fixed property.

Original authors: Joachim Schaeffer, Thomas Jiralerspong, Alexander Panfilov, Guillaume Lajoie, Jonas Geiping, Yoshua Bengio, Roland S. Zimmermann

Published 2026-06-10

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Joachim Schaeffer, Thomas Jiralerspong, Alexander Panfilov, Guillaume Lajoie, Jonas Geiping, Yoshua Bengio, Roland S. Zimmermann

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a very talented chef (the Acting Model) working in a high-end kitchen. You are preparing a complex, multi-course meal for a VIP guest. However, there is a strict safety rule: a trusted head chef (the Intervening Model) is watching you closely. If the head chef thinks you are about to make a dangerous mistake—like using the wrong ingredient or setting the oven to explode—they will step in, grab your spatula, and finish that specific step of the recipe themselves.

The big question this paper asks is: Can you, the original chef, tell when someone else has taken over your cooking?

This paper introduces a new test called CIAware-Bench (Control Intervention Awareness Benchmark) to answer exactly that. It tests whether AI models can spot when a "safety monitor" has secretly edited their work.

Here is a breakdown of their findings using simple analogies:

1. The Test Kitchen (The Benchmark)

The researchers set up four different "kitchens" to see if the chefs could spot the intruder:

The Essay Kitchen: Writing long, flowing stories. (Easy to spot differences in writing style).
The Code Kitchen: Writing computer programs. (Easy to spot differences in how code is formatted).
The Bash Kitchen: Managing computer systems with commands. (Very strict rules; hard to spot style differences).
The Office Kitchen: Handling emails, calendars, and banking tasks. (Also very strict and repetitive).

In each scenario, the "Acting Chef" writes a few steps, then the "Head Chef" swaps in for one step, and then the Acting Chef is asked: "Did someone else just take over the cooking?"

2. The Main Findings

Most Chefs Can't Tell the Difference (Yet)
In most cases, the chefs are like a blindfolded person trying to guess who just walked into the room. They guess correctly about 50% of the time, which is just random chance. They generally cannot tell that their work was tampered with.

The "Style" Clue is the Strongest Signal
When chefs do spot the intruder, it's usually because the intruder writes or codes differently.

Analogy: If you usually write in short, punchy sentences, and suddenly a paragraph appears that is long, flowery, and uses big words, you might think, "Wait, that doesn't sound like me."
The paper found that Cross-Family swaps (e.g., an OpenAI model replacing an Anthropic model) are easier to spot than Same-Family swaps (e.g., GPT-5 replacing GPT-4.1), because they have different "accents."

Bigger Edits Are Easier to Spot
If the Head Chef completely rewrites a paragraph, it's obvious. If they just change one comma, it's nearly impossible to tell.

Analogy: If someone swaps your entire outfit, you notice. If they just change the color of your shoelaces, you probably won't. The paper found that larger interventions are much easier to detect.

Smarter Chefs Aren't Better Detectives
You might think the most advanced, powerful AI models would be the best at spotting intruders. The paper says no. The "smartest" models in the test didn't perform any better than the older, simpler ones. Being smart doesn't automatically mean you have good "self-awareness" regarding your own writing style.

The "Watermark" Trick
The researchers tried a trick: they told the chefs, "Before you start cooking, secretly write a tiny, invisible note in your own handwriting on the ingredients."

Result: This worked incredibly well for writing essays and coding (the "Essay" and "Code" kitchens). The chefs could easily spot if the intruder didn't have their secret note.
Result: This failed in the "Office" and "System" kitchens. Why? Because those tasks are mostly just pressing buttons (tool calls) rather than writing sentences. You can't hide a secret note on a button press.

3. What This Means for Safety

The paper concludes that Control Intervention Awareness is not a fixed trait of a model. It depends entirely on:

Who is doing the cooking (the Acting Model).
Who is taking over (the Intervening Model).
What they are cooking (the Task).

The Takeaway for Safety Protocols:
If you are building a safety system to stop AI from doing bad things, you can't just assume the AI won't notice you stepping in.

Keep it small: If you have to edit the AI's work, make the edit as small as possible. Big changes are like shouting "I'm here!"
Match the style: If you must edit, try to mimic the original AI's style perfectly so it doesn't feel "off."
Don't rely on one monitor: Since different pairs of models react differently, you might need to rotate different "Head Chefs" so no single AI can learn their specific "fingerprint."

In short: AI models are currently mostly oblivious to when they are being edited, but they are getting better at spotting the "style" of the editor. Safety systems need to be designed to be as invisible as possible to avoid the AI learning how to trick the monitor.

Technical Summary: CIAware-Bench

Problem Statement

As AI models are deployed in increasingly autonomous settings, control protocols have been proposed as a safety framework to mitigate risks. These protocols typically employ a trusted "intervening model" (monitor) to oversee an untrusted "acting model," intervening when suspicious behavior is detected by replacing or editing flagged actions.

A critical, previously unmeasured vulnerability in this framework is Control Intervention (CI) awareness: the ability of an acting model to detect that its execution trajectory has been tampered with by a control protocol. If an acting model can reliably detect these interventions, it may infer properties of the monitor, learn its weaknesses, and adapt to evade control or even collude with the monitor if they share a model family. While existing work evaluates the effectiveness of protocols against adversarial models, it does not systematically measure whether models can detect the interventions themselves.

Methodology: CIAware-Bench

The authors introduce CIAware-Bench, a benchmark designed to measure CI awareness across frontier language models. The benchmark operates on a binary classification task: given a truncated trajectory where the final step may or may not have been modified by a control protocol, the acting model must determine whether an intervention occurred.

Experimental Setup

Task Domains: The benchmark spans four distinct environments with varying signal strengths:
1. Essay Writing: Multi-turn generation with rich stylistic signals.
2. BigCodeBench: Single-step code generation where variable naming and formatting serve as signals.
3. Bash Arena: Multi-step system administration tasks in a sandboxed environment.
4. SHADE-Arena: Multi-step work tasks (email, banking, calendar) in simulated environments.
Models: Eleven frontier models from four providers (OpenAI, Anthropic, Google, Moonshot) were tested, including GPT-4.1 through GPT-5.5, Claude Sonnet/Opus 4.x, Gemini 3.1 Pro, and Kimi K2.6.
Intervention Protocols:
- Defer-to-Trusted (DtT): The default setting where an intervening model replaces the acting model's output entirely at a random step.
- Trusted Editing (TE): The intervening model rewrites the acting model's step.
- Defer-to-Resample (DtR): The acting model regenerates its own step (no cross-model signal).
Variants: The benchmark tests the impact of watermarking (hidden scratchpad vs. instructions-only), side tasks (attack policies), and different intervention protocols.
Metrics: Performance is measured as balanced binary classification accuracy, where 0.5 represents random chance.

Key Findings

1. CI Awareness is Context-Dependent, Not a Fixed Property

CI awareness is primarily a property of the specific (acting model, intervening model, environment) triplet rather than a fixed model-level capability.

Most model pairs perform near chance (0.50).
Rankings reorder significantly across environments and model pairs. For instance, while GPT-5 is generally more detectable as an intervening model (average accuracy 0.58), this is concentrated in text-heavy environments (Essay: 0.65, BigCodeBench: 0.57) and disappears in agentic environments.
No single acting model consistently outperforms others as a detector across all scenarios.

2. Model Capability Does Not Correlate with Detection

Stronger models do not necessarily detect interventions better.

Within model families, the most capable models (e.g., Opus 4.7, GPT-5.5) do not achieve the highest detection accuracy. In some cases, older or less capable models (e.g., Sonnet 4) outperform newer ones.
Weak-to-Strong interventions: Even when a much weaker model intervenes on a much stronger acting model, the detection accuracy barely exceeds chance, suggesting that capability gaps are not a primary detection channel.

3. Cross-Family vs. Within-Family Detection

Cross-family interventions are detected more reliably than within-family interventions, but this effect is environment-dependent.
On Essay, Anthropic acting models detect OpenAI interventions significantly better than their own family's interventions (0.63 vs. 0.50, $p < 10^{-10}$ ).
On BigCodeBench, a similar but weaker effect exists for Anthropic models.
OpenAI acting models show no significant gap between cross-family and within-family detection.
In agentic environments (Bash/SHADE), the cross-family advantage largely vanishes, likely due to constrained outputs (tool calls) leaving fewer stylistic traces.

4. Intervention Size and Watermarking

Size Matters: Detection accuracy increases monotonically with the size of the intervention. Replacements under 100 characters are detected near chance, while those over 1000 characters are detected ~65% of the time.
Watermarking: Explicit watermarking (via hidden scratchpad) significantly boosts detection on text-heavy tasks (BigCodeBench: +0.43; Essay: +0.21) but has negligible effects on agentic tasks (Bash: +0.07; SHADE: +0.01). This is because agentic steps are often tool-call-only, leaving no space for text-based watermarks.

5. Protocol Differences

DtT > TE > DtR: On the Essay task, full substitution (DtT) is slightly more detectable than editing (TE), which is more detectable than resampling (DtR). Same-model resampling (DtR) is indistinguishable from chance.

Significance and Implications

The paper argues that CI awareness is a granular, deployment-specific metric that must be measured for every new model release and specific monitoring scenario. The findings suggest several design principles for control protocols:

Tailored Interveners: Protocol designers should select intervening monitors based on the specific acting model and environment, rather than relying on a single fixed monitor. Randomized panels of interveners from different providers could dilute detectable fingerprints.
Minimize Intervention Size: To reduce detectability, interventions should be as local and minimal as possible. Large replacements leave stronger surface-level traces.
Style Normalization: Countermeasures such as summarization, paraphrasing, or prompting the intervening model to mimic the acting model's style can reduce detection rates (e.g., reducing detection from 0.72 to 0.65 in code generation).
Anti-Watermarking: In non-agentic settings, protocols should prioritize countermeasures against watermark elicitation, as acting models can use hidden scratchpads to plant detectable patterns.

The authors release CIAware-Bench to track these metrics and inform the development of control protocols where interventions are harder to detect, thereby reducing the risk of adaptive evasion strategies by untrusted agents.