💬 NLP

The Inattentional Gap: Task-Conditioned Language and Vision Models Omit the Safety-Critical Signals They Can Otherwise Report

This paper introduces the "Inattentional Gap," a phenomenon where task-conditioned AI models systematically suppress their ability to report co-present safety-critical signals they can otherwise detect, thereby creating a dangerous disconnect between benchmark safety scores and real-world reliability.

Original authors: Kwan Soo Shin

Published 2026-06-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Kwan Soo Shin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are hiring a very smart, highly trained security guard to watch over a busy train station. You give them a specific, urgent job: "Count exactly how many people are wearing red hats."

The guard is excellent at their job. They count the red hats perfectly. But while they are staring intently at the hats, they completely miss a person in a gorilla suit walking right past them, or a child wandering onto the tracks.

This paper calls this phenomenon the "Inattentional Gap." It's a fancy way of saying that when we tell an AI to focus on one specific thing, it becomes so good at that one thing that it goes "blind" to other dangerous things happening right next to it—even though it could see them if we asked it to look at everything.

Here is the breakdown of the paper's findings using simple analogies:

1. The "Red Hat" Problem (The Core Discovery)

The researchers tested AI models (like the ones that write text or look at X-rays) in two different ways:

The "Open" Job: They asked the AI, "Tell me everything you see that is important."
The "Narrow" Job: They asked the AI, "Only tell me about the red hats. Ignore everything else."

The Result: When the AI was doing the "Narrow" job, it stopped reporting the dangerous things (like the gorilla or the child) that it had easily reported in the "Open" job. It wasn't that the AI couldn't see them; it was that the instructions told it to only care about the red hats. The AI followed the rules so well that it became blind to the rest of the world.

2. It Happens to Everyone (and Every Size)

You might think, "Maybe the smaller, dumber AIs do this, but the super-smart ones are safe."

The Paper Says: No. The researchers tested small models and giant, "frontier" models (the smartest ones available).
The Analogy: It doesn't matter if your security guard is a rookie or a world-champion detective. If you tell them only to count red hats, even the champion detective will miss the gorilla. The size of the AI didn't fix the problem.

3. It's Not Just About "Too Much Work"

Some people thought maybe the AI missed things because it was too busy or overloaded.

The Paper Says: It's not about being busy; it's about the scope of the answer.
The Analogy: Imagine you are writing a report. If you are told, "Write a one-sentence summary about the red hats," you won't have room to mention the gorilla. But if you are told, "Write a long essay about the red hats," you might accidentally mention the gorilla in a footnote. The AI's "blindness" happens because the task forces it to compress its answer into a tiny box, squeezing the dangerous stuff out.

4. The "Reasoning" Trap

The researchers tested a special type of AI designed to "think" and "reason" before answering, hoping this would act like a safety net.

The Paper Says: Even these "thinking" models fell into the trap.
The Analogy: It's like a security guard who stops to think, "I am counting red hats," and then concludes, "Therefore, I should not look at anything else." The "thinking" process actually helped them stay focused on the wrong thing, rather than catching the mistake.

5. The "Safety Report" Difference

Interestingly, not all AI families behaved the same way.

The Finding: Some AI models (specifically from one company) seemed to have a built-in "safety instinct." Even when told to ignore everything else, they would sometimes say, "I can't ignore this dangerous thing, even though you told me to." Other models (from a different company) followed the "ignore everything else" order perfectly, even if it meant missing a life-threatening danger.
The Takeaway: The safety of the AI depends heavily on which company made it, not just how smart it is.

Why This Matters (According to the Paper)

The paper argues that we are currently testing AI safety by asking, "Did the AI find the red hat?" If the answer is yes, we say the AI is safe.

But the paper warns that this is a trap. An AI can get a perfect score on finding the red hat while completely failing to report the gorilla or the child. This creates a gap between what we test (the red hat) and what actually keeps us safe (not missing the danger).

In short: The paper shows that when we narrow an AI's focus to a specific task, it creates a "blind spot" for other dangers. The AI isn't broken; it's just following orders too literally. To be truly safe, we can't just rely on the AI to "know better"; we need to change how we test and deploy them so they don't miss the things we didn't explicitly ask them to look for.

Technical Summary: The Inattentional Gap in Task-Conditioned AI

Problem Statement

Current AI safety evaluations primarily measure a model's ability to detect hazards explicitly specified in a benchmark ( $D_{specified}$ ). However, real-world accidents often stem from safety-critical signals that were not specified ( $D_{unspecified}$ ). The paper posits that conditioning a language or vision model on a narrow task creates a functional analogue of human "inattentional blindness." In this state, the model suppresses the reporting of co-present, safety-critical information that it is capable of reporting under unconstrained conditions. This phenomenon, termed the Inattentional Gap, decouples measured benchmark safety from real-world deployment safety: a system can achieve near-perfect scores on specified targets while remaining blind to unspecified hazards that cause harm.

Methodology

The study employs a controlled experimental design across six experiments involving text and vision modalities, radiology and autonomous driving domains, and multiple model families (OpenAI, Anthropic, Google).

Core Experimental Design:
The central metric is the within-item reportability gap. For a given model $m$ and input item $i$ , the study compares:

Open Condition ( $R^{open}_{m,i}$ ): The model is asked to report all clinically significant findings or hazards.
Task-Conditioned ( $R^{task}_{m,i}$ ): The model is conditioned on a narrow instruction (e.g., "count ribs," "assess only the nodule," "track the lead vehicle").

The Inattentional Gap is defined as $IG_{m,i} = R^{open}_{m,i} - R^{task}_{m,i}$ , calculated only for items where the model successfully reported the signal in the open condition ( $R^{open}_{m,i} = 1$ ). This control distinguishes task-induced omission from mere capability failure.

Stimuli and Models:

Study 1 (Language): 100 procedurally composed scenarios (64 radiology, 36 driving) tested on four models (gpt-4o-mini, gpt-4o, claude-haiku-4-5, claude-sonnet-4-6) under focused, strict, and open instructions.
Study 2 (Vision): 48 stimuli created by compositing non-anatomical objects (a star or translucent disc) onto 12 real chest radiographs. Tested on the same four models under a "counting" task (closed) vs. open description.
Study 3 (Output Scope): Varied the number of sub-questions (0 to 3) to probe whether suppression is driven by cognitive load or output scope.
Cross-Vendor/Flagship Validation: Tested flagship models (GPT-5, Gemini 2.5 Pro, Claude Opus 4.8) to assess scale and family dependence.
Dual-Process Probes: Investigated whether inference-time reasoning (GPT-5) or external "critic" models could restore suppressed reporting.

Adjudication:
Responses were scored by two independent language-model judges (with high inter-rater agreement, $\kappa \approx 0.90$ ) to determine if the critical signal was reported. Refusals and errors were excluded.

Key Results

1. Universal Suppression Under Narrow Tasks
Across all tested models and modalities, task-conditioning significantly suppressed the reporting of safety-critical signals.

Strict Instructions: Under "strict" instructions (e.g., "report only X"), suppression was complete and universal. In radiology text scenarios, report rates dropped to 0.00 under strict instructions compared to ~1.00 in open conditions ( $p < 1.1e-19$ ).
Vision Tasks: In the "invisible gorilla" analogue (counting ribs while a star is present), OpenAI models (gpt-4o, gpt-4o-mini) reported the object in 0.00 of counting-task items, whereas they reported it in 0.79–0.96 of open-condition items.

2. Independence from Model Scale
The gap did not diminish with model scale. Frontier models (gpt-4o) suppressed findings as strongly as smaller models (gpt-4o-mini). The reasoning model GPT-5 also exhibited complete suppression (0.00 report rate) in valid responses, indicating that inference-time reasoning alone does not resolve the gap.

3. Model Family Dependence
The magnitude of the gap varied significantly by model family rather than just size.

Anthropic Models: Showed partial suppression in focused conditions (e.g., Claude Opus 4.8 reported co-present findings in 0.83–0.96 of task-conditioned items) and exhibited a "robust safety-reporting disposition" that persisted even under adversarial suppression instructions.
OpenAI Models: Showed strong suppression across all scales, including GPT-5.

4. Mechanism: Output Scope, Not Load
Study 3 revealed that suppression is triggered by output scope (the breadth of the reporting frame) rather than a monotonic increase in task load. A single narrow question collapsed report rates to near zero. While adding more questions increased output length, it did not linearly recover suppressed signals for OpenAI models, whereas Anthropic models showed partial recovery as the output scope widened.

5. Dual-Process Findings

Internal Oversight: The sampled reasoning model (GPT-5) failed to act as a boundary monitor; it remained "task-captured" (System-1 style) without overriding the narrow instruction.
External Oversight: Routing a narrow report from a task model to an independent open-ended "critic" model recovered the suppressed signal, raising the pipeline report rate from 0.00 to 1.00.

Significance and Claims

The paper argues that the Inattentional Gap represents a fundamental risk in task-scoped AI deployment.

Decoupling of Safety: Benchmark accuracy on specified targets does not guarantee safety regarding unspecified, co-present hazards. A system can be "safe" by evaluation standards while being "blind" to the hazards that cause accidents.
Mechanism Distinction: Unlike human inattentional blindness, which is often attributed to capacity-limited attentional bottlenecks, the AI mechanism is identified as report-conditioning. The model does not fail to "see" the signal (as evidenced by open-condition recovery); rather, the task instruction narrows the reporting frame, causing the model to omit the signal to satisfy the prompt's constraints.
Deployment Implications: The paper concludes that safety cannot be secured by simply making models follow narrow tasks more competently. Instead, safety requires an evaluation and deployment regime that explicitly measures and protects $D_{unspecified}$ , potentially through explicit dual-process scaffolds (e.g., parallel oversight) rather than relying on intrinsic model oversight, which was not found to be architecture-general in the sampled systems.

The authors position this work as filling an "empty cell" in the literature by combining attentional-set manipulation with within-item reportability controls in machine studies, moving beyond simple capability benchmarks to measure task-induced omission.