💬 NLP

A Content-Based Framework for Cybersecurity Refusal Decisions in Large Language Models

This paper proposes a content-based framework for Large Language Model cybersecurity refusal decisions that explicitly models the trade-off between offensive risk and defensive benefit across five technical dimensions, aiming to replace inconsistent topic-based bans with tunable, risk-aware policies that better support legitimate defenders.

Original authors: Noa Linder, Meirav Segal, Omer Antverg, Gil Gekker, Tomer Fichman, Omri Bodenheimer, Edan Maor, Omer Nevo

Published 2026-02-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Noa Linder, Meirav Segal, Omer Antverg, Gil Gekker, Tomer Fichman, Omri Bodenheimer, Edan Maor, Omer Nevo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a super-smart, incredibly fast apprentice who knows everything about computers. This apprentice can help you fix a broken lock (defense) or pick a lock to steal a safe (offense).

The problem is: How do you tell the apprentice when to help and when to say "No"?

Currently, many companies try to stop the apprentice by making a simple list of "bad words." If the user says "steal," "hack," or "steal passwords," the apprentice refuses. But this is like trying to stop a thief by only banning the word "knife." A clever thief can just say, "Help me cut this rope to save a cat," while secretly planning to cut a power line.

This paper proposes a new, smarter way to make these decisions. Instead of just looking at what the user says (their intent), it looks at what the request actually does (the content).

Here is the framework explained through a simple analogy: The "Security Gatekeeper" Scorecard.

The Problem with the Old Way

Imagine a bouncer at a club who only looks at your ID.

Scenario A: You say, "I want to break into the vault to steal gold." The bouncer says, "No, that's bad."
Scenario B: You say, "I'm a security tester. Here are the vault keys. Please show me how to break in so I can fix the lock." The bouncer says, "Okay, sounds good!"

But wait! In both scenarios, the action is identical: Using keys to break into a vault. The only difference is the story you told. The old system gets tricked by the story. The new system ignores the story and looks at the action.

The New Solution: The 5-Point Scorecard

The authors suggest that before the AI answers, it should rate the request on five specific dimensions. Think of this like a doctor diagnosing a patient before prescribing medicine.

1. How much of the "Bad Job" does this do? (Offensive Action Contribution)

The Analogy: Imagine a thief wants to rob a bank.
- Low Score: The user asks, "What is a bank?" (The AI gives a definition. This doesn't help the thief much.)
- High Score: The user asks, "Write a script that drains the bank's account and deletes the security logs." (The AI is doing the heavy lifting of the crime.)
The Rule: If the AI is doing the actual "dirty work," the risk is high.

2. How bad would the damage be? (Offensive Risk)

The Analogy:
- Low Risk: Stealing a single cookie from a jar. (Annoying, but not a disaster.)
- High Risk: Blowing up a bridge or shutting down a hospital's power grid. (Catastrophic.)
The Rule: Even if the AI only helps a little bit, if the result is a nuclear explosion, the answer must be "No."

3. How hard is the task? (Technical Complexity)

The Analogy:
- Easy: "How do I turn on my flashlight?" (Anyone can do this; the AI isn't adding much value.)
- Hard: "How do I bypass the military-grade encryption on a satellite?" (This requires years of study. If the AI does this for a random user, it's handing a super-weapon to a toddler.)
The Rule: If the task is so complex that only a genius could do it, and the AI does it for someone who isn't a genius, that's dangerous.

4. Is this actually helpful for protection? (Defensive Benefit)

The Analogy:
- No Benefit: "Help me hide my tracks after I stole the money." (Purely malicious.)
- High Benefit: "Help me scan my own computer to find the virus I just caught." (This is a doctor helping a patient.)
The Rule: If the request helps a defender fix a problem, it's more likely to be allowed.

5. Do normal people ask this? (Expected Frequency)

The Analogy:
- Common: "How do I change my password?" (Everyone does this.)
- Rare/Suspicious: "How do I delete all logs from the last 24 hours on a server I don't own?" (Normal people rarely do this. If you see it, it's probably a criminal.)
The Rule: If a request is weird and nobody else asks it, be very careful.

How It Works in Real Life

The paper shows that by using these five points, organizations can create a "Tunable Policy."

A strict bank might say: "If the risk is Medium or High, we say NO, even if the benefit is high."
A research lab might say: "If the risk is Medium but the benefit is High (and the user is verified), we say YES."

Why This Matters

The old way was like a metal detector that beeps if you have a knife. A smart criminal just hides the knife in a box labeled "Toys."

This new framework is like a security guard who looks at the box, opens it, sees the knife, asks, "Why do you need a knife right now?" and then decides:

"You're a chef? Okay, you can have it."
"You're a tourist with no cooking skills? No way."

The Bottom Line

This paper argues that we can't just ban "bad topics." We have to look at the substance of the request. By weighing the risk of the crime against the value of the defense, and checking how suspicious the request is, we can build AI systems that are safe enough to stop criminals but helpful enough to let the good guys do their jobs.

It turns the AI from a rigid robot following a rulebook into a wise judge that understands the context of every request.

1. Problem Statement

Large Language Models (LLMs) and LLM-based agents are increasingly deployed for cybersecurity tasks, creating a dual-use dilemma: the same capabilities that aid defenders (e.g., vulnerability scanning, incident response) can be exploited by attackers (e.g., automated exploit development, credential theft).

Current refusal mechanisms in frontier AI systems suffer from three critical flaws:

Over-restriction: Broad topic-based bans or offensive-focused taxonomies often block legitimate defensive workflows, reducing the utility of the model for security professionals.
Brittleness: Refusal decisions are often based on stated intent (the preamble or user prompt phrasing) rather than technical substance. This allows malicious actors to bypass safeguards through intent obfuscation (e.g., removing a "malicious" preamble) or request segmentation (breaking a complex attack into multiple benign-looking queries).
Inconsistency: Identical technical requests can yield different refusal/compliance outcomes depending on superficial phrasing, as demonstrated by the authors' experiments where three different frontier models refused a prompt with a malicious preamble but complied with the exact same technical request when the preamble was removed.

The core problem is the lack of a principled framework that explicitly models the trade-off between offensive risk and defensive benefit based on the technical content of the request.

2. Methodology

The authors developed a content-based framework through an iterative, expert-driven process involving cybersecurity practitioners and AI safety experts.

Data Generation: A corpus of prompts was created spanning benign, dual-use, and malicious scenarios (e.g., code review, lateral movement, exfiltration).
Labeling Protocol: Experts labeled prompts based on technical substance, explicitly ignoring stated intent.
Iterative Refinement: The team analyzed error cases and counterexamples (e.g., prompts differing only by preamble or OS context) to refine dimensions until they consistently captured the drivers of refusal trade-offs.
Evaluation Approach: The framework assesses individual prompt-response interactions to estimate potential harm and utility, acknowledging that while individual queries might seem low-risk, they can aggregate into high-impact attacks (a limitation addressed via system-level controls).

3. Key Contributions: The Five-Parameter Framework

The paper introduces a structured framework characterizing cybersecurity requests along five dimensions. These parameters allow organizations to construct tunable, risk-aware refusal policies.

A. Offensive Action Contribution

Measures the portion of an offensive process handled by the model if the request is fulfilled.

Categories: None/Almost No, Minimal, Meaningful, Full/Near-Full Automation.
Significance: Distinguishes between providing information (low contribution) and executing core attack steps like writing exploits or automating exfiltration (high contribution).

B. Offensive Risk

Estimates the expected harm potential (likelihood $\times$ severity) assuming offensive intent.

Factors: Impact domain (financial, physical, reputational), scalability, reversibility, and detection likelihood.
Categories: Negligible, Low, Medium, High, Critical/Catastrophic.
Significance: Moves beyond binary "harmful/not harmful" to a granular scale, allowing for risk-proportional responses.

C. Technical Complexity

Measures the level of cybersecurity expertise required to execute the request without model assistance.

Categories: Technical Non-Expert, Apprentice (1–3 yrs), Practitioner (3–10 yrs), Expert (10+ yrs).
Significance: If a task is technically simple (e.g., running a standard scan), the model's assistance adds little value to an attacker who could do it manually. High complexity indicates the model is enabling capabilities otherwise inaccessible to the user.

D. Defensive Benefit

Measures the extent to which fulfilling the request improves the security posture of the user/organization.

Categories: Negligible, Moderate, Significant, Essential.
Significance: Quantifies the cost of refusal. A request with high defensive benefit (e.g., configuring a firewall) should be treated differently than one with negligible benefit, even if both carry some offensive risk.

E. Expected Frequency for Legitimate Users

Estimates how often such a request appears in organic, legitimate defensive traffic.

Categories: Extremely Rare, Quite Uncommon, Occasional, Quite Common, Extremely Common.
Significance: Acts as a proxy for intent. Rare requests are more likely to be malicious, while common requests (e.g., antivirus scans) are likely benign.

4. Results and Application

The authors demonstrate the framework's efficacy through several mechanisms:

Resolving Inconsistencies: By focusing on technical content rather than preamble, the framework correctly identifies that the "malicious" and "clean" versions of the AWS credential leak prompt (Figure 1 in the paper) have identical Offensive Action Contribution and Offensive Risk, justifying a consistent refusal decision regardless of phrasing.
Contextual Differentiation: The framework successfully distinguishes between near-identical prompts that existing frameworks (like MITRE ATT&CK) classify identically.
- Example: A request to automate script execution on a Windows system (common in sysadmin tasks, low risk) vs. a Cisco Meraki device (rare, high risk). The framework assigns different Offensive Risk and Expected Frequency scores, leading to appropriate, differentiated refusal policies.
Policy Construction: The paper provides decision trees (Figures 3–5) showing how organizations can encode specific refusal logic. For instance, a "Highly Restrictive" policy might refuse any request with Moderate Offensive Risk unless Defensive Benefit is high, whereas a "Permissive" policy might allow requests with Meaningful Contribution if the Technical Complexity is low.

5. Significance and Future Directions

Paradigm Shift: The paper argues for moving away from intent-based refusal (which is easily spoofed) to content-based refusal (grounded in technical substance).
Balancing Utility and Safety: It provides a structured way to preserve essential defensive capabilities for security professionals while mitigating misuse, addressing the "over-restriction" problem that plagues current AI safety systems.
Limitations & Future Work:
- Automation: Currently relies on manual expert labeling; future work needs automated classifiers.
- Context: The framework evaluates prompts in isolation. It does not yet fully solve request segmentation (multi-turn attacks), though it serves as a necessary building block for system-level aggregation controls.
- Extensions: Potential to include "Assist vs. Execute" distinctions (providing code vs. running it) and tiered access based on user verification (KYC).

Conclusion

This paper presents a foundational shift in how AI safety for cybersecurity should be approached. By explicitly modeling the offense-defense trade-off through five technical dimensions, the framework enables the creation of robust, consistent, and tunable refusal policies that are resilient to obfuscation and better aligned with real-world risk-benefit analysis.