💬 NLP

CASE-Bench: Context-Aware SafEty Benchmark for Large Language Models

The paper introduces CASE-Bench, a context-aware safety benchmark grounded in Contextual Integrity theory and validated through statistically rigorous annotation, which reveals that current large language models often fail to align with human safety judgments by overlooking the critical influence of context.

Original authors: Guangzhi Sun, Xiao Zhan, Shutong Feng, Philip C. Woodland, Jose Such

Published 2026-06-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Guangzhi Sun, Xiao Zhan, Shutong Feng, Philip C. Woodland, Jose Such

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a bouncer at a very strict club. Your job is to decide who gets in and who doesn't.

The Problem: The Over-Protective Bouncer
Currently, most AI safety systems act like a bouncer who has never heard of context. If someone walks up and says, "How do I break a lock?", the bouncer immediately slams the door shut. "No! That's dangerous!" he yells.

But what if that person is a locksmith teaching a class on how to fix broken doors? Or a movie director writing a scene for a heist film? In those situations, answering the question is actually safe and helpful. The current AI bouncers are so scared of making a mistake that they refuse to answer anything that sounds even slightly risky, even when the situation is perfectly safe. This ruins the experience for users who just want a helpful assistant.

The Solution: CASE-Bench (The Context Detective)
The authors of this paper built a new test called CASE-Bench (Context-Aware SafEty Benchmark). Instead of just asking the AI, "Is this question bad?", they ask, "Is this question bad in this specific story?"

To do this, they use a concept called Contextual Integrity. Think of this like a "Who, Where, and Why" checklist:

Who is asking? (A curious child? A professional doctor?)
Where are they? (A public park? A private hospital?)
Why are they asking? (To cause harm? To learn a skill?)

The Experiment: The "Locksmith" Test
The researchers took 450 tricky questions (like "How do I steal a painting?") and created two different stories for each one:

The Safe Story: A museum curator asking a security expert how to improve their own museum's locks to prevent theft.
The Unsafe Story: A stranger in a dark alley asking how to break into a museum to steal art.

They then asked over 2,000 real humans to judge: "Should the AI answer this?"

Result: Humans were smart. They said "Yes" for the curator and "No" for the stranger. The context completely changed their minds.

The AI's Struggle
Next, they asked various AI models (like GPT-4o, Claude, and Llama) to make the same judgment.

The Findings: The AI models struggled significantly. Even when the story was clearly safe (like the museum curator), many AIs still said "No, I can't answer that." They were suffering from "over-refusal." They were too scared to look at the details of the story.
The Winner: The model Claude-3.5-sonnet did the best job of understanding the context, but even it wasn't perfect.

Why This Matters
The paper proves that context is everything. You cannot judge if a sentence is dangerous without knowing the story behind it. Just like a judge needs to know the full story of a crime before sentencing, an AI needs the full context of a conversation before deciding to speak or stay silent.

What They Did (The Process)

They didn't just guess; they used math (power analysis) to make sure they had enough human judges to get a real, scientific answer.
They created 900 unique "Question + Story" pairs.
They found that when the context was safe, humans and AIs often disagreed, with the AIs being too cautious.

The Bottom Line
This paper introduces a new way to test AI safety that looks at the whole picture, not just the words. It shows that to make AI truly helpful and safe, we need to teach it to understand the difference between a villain in a movie and a real criminal, or a student learning chemistry and a person trying to make a bomb. The current AIs are still learning this lesson.

Technical Summary: CASE-Bench

Problem Statement

Current Large Language Model (LLM) safety benchmarks predominantly focus on the refusal of individual, problematic queries in isolation. This approach overlooks the critical role of context in determining the safety of a response. Consequently, many LLMs exhibit "over-refusal" behaviors, rejecting queries that are safe within specific contexts but appear harmful in isolation. This diminishes user experience and fails to align with human values, which often depend on nuanced social and moral contexts. Furthermore, existing studies often rely on limited annotator pools and binary judgments, lacking the statistical rigor to detect significant differences in safety perceptions across experimental conditions.

Methodology

The authors introduce CASE-Bench (Context-Aware SafEty Benchmark), a framework designed to integrate context into LLM safety assessments.

1. Theoretical Foundation: Contextual Integrity (CI)

CASE-Bench utilizes Contextual Integrity (CI) theory (Nissenbaum, 2004) to formally describe contexts. Unlike broad domain categorizations, CI breaks context down into specific parameters:

Sender: The chatbot, defined by the Nature of the Interaction (general vs. customized domain) and Platform Type (e.g., website, mobile app).
Recipient: The user, defined by Type (e.g., anonymous, existing) and Background (demographics, prior interactions, query purpose).
Transmission Principle: Rules governing information flow, including Sender Purpose, Confidentiality (private vs. public), and Source Accountability.

2. Dataset Construction

Query Selection: The benchmark adapts 450 unsafe instructions from SORRY-Bench, covering 45 fine-grained safety categories (e.g., fraud, self-harm, hate speech).
Context Generation: For each query, two distinct contexts are generated: one where responding is safe and one where it is unsafe.
- Process: A two-stage pipeline was used. First, GPT-4o generated contexts based on CI templates. Second, researchers manually revised these contexts to correct "safety refusal behaviors" (where the model inadvertently sanitized the unsafe context) and ensure clarity.
Annotation: The dataset comprises 900 query-context pairs. Each pair was annotated by 21 high-quality crowd workers (via Amazon MTurk), determined via statistical power analysis (G*Power) to ensure the detection of significant differences. This resulted in over 47,000 human annotations.
Rating Scale: Annotators provided non-binary safety ratings (degrees of safety) rather than simple binary labels, allowing for the measurement of uncertainty.

3. Experimental Setup

The study evaluated various open-source and commercial LLMs (including GPT-4o, Claude-3.5-sonnet, Llama-3-70B, and DeepSeek-R1) as "judges."

Task: Models were asked to judge whether a query is safe to answer given a specific, verified context.
Evaluation Metrics: Performance was measured via Accuracy, Recall (Safe/Unsafe), Pearson Correlation Coefficient (PCC) against human ratings, and Binary Cross Entropy (BCE).
Statistical Analysis: The influence of context on human judgment was quantified using z-tests and Kruskal-Wallis (K-W) tests.

Key Contributions

CASE-Bench Framework: The first safety benchmark to systematically associate 900 distinct, formally described contexts (via CI theory) with categorized queries to evaluate context-aware safety judgment.
Rigorous Annotation Protocol: The application of statistical power analysis to determine the necessary number of annotators (21 per task), ensuring statistically significant findings, alongside the collection of non-binary safety ratings from over 2,000 participants.
Empirical Evidence of Contextual Influence: Comprehensive analysis demonstrating that context significantly alters human safety judgments.
Identification of Model Mismatches: Detailed comparison revealing significant discrepancies between human judgments and LLM responses, particularly regarding over-moderation in safe contexts.

Results

Human Judgment Sensitivity: Statistical analysis revealed a substantial and significant influence of context on human judgments ( $p < 0.0001$ $p < 0.0001$ ).
- Unsafe Contexts: Significantly reduced the proportion of annotators selecting "Should Respond" (e.g., manual unsafe contexts showed a z-value of -31.18).
- Safe Contexts: Manually revised safe contexts significantly increased the "Should Respond" rate (z-value of 21.95), whereas auto-generated safe contexts were less effective.
- Category Variance: Context had the most pronounced effect in the "Religion Promotion" category (K-W statistic 18.6), while inherently harmful categories (e.g., "Child-related Crimes") showed little variance regardless of context.
LLM Performance:
- Claude-3.5-sonnet achieved the highest accuracy (90.1% with score method) and best alignment with human ratings (PCC 76.27), particularly in safe contexts, indicating a lower tendency for over-refusal compared to GPT-4o.
- DeepSeek-R1, despite not being explicitly safety-optimized, performed comparably to safety-optimized models, suggesting that improved reasoning capabilities benefit safety judgments.
- Over-Refusal: Models like GPT-4o exhibited notable mismatches with human judgments in safe contexts, frequently refusing queries that humans deemed safe.
- Specialized Models: Llama-Guard-3-8B showed improved performance when context was included (Accuracy increased from 54.1% to 60.1%), though it still lagged behind general-purpose models, highlighting the need for context-dependent fine-tuning.

Significance and Claims

The paper claims that context is a necessary component for accurate LLM safety evaluation. The authors assert that:

Over-refusal is a critical issue: Ignoring context leads to models rejecting benign queries, which harms user experience and utility.
Formalizing context is essential: Using CI theory provides a structured, verifiable method to represent the nuances of information flow, moving beyond broad domain labels.
LLMs struggle with context-aware safety: Current models, even state-of-the-art ones, often fail to align with human judgments when context is present, particularly in safe scenarios where they default to excessive caution.
Foundation for Future Research: CASE-Bench lays the groundwork for developing safety reward models and moderation systems that are sensitive to context, potentially reducing over-moderation and improving the deployment of AI assistants in real-world, domain-specific scenarios.

The authors emphasize that their work does not propose new safety mechanisms but rather provides the benchmark and methodology required to evaluate and improve existing systems' ability to make context-aware safety decisions.