💻 computer science

The Geometry of Refusal: Linear Instability in Safety-Aligned LLMs

This paper introduces Contrastive Logit Steering (CLS), a zero-optimization framework that reveals safety alignment in LLMs as a manipulable linear feature, enabling both high-success jailbreaks by steering output distributions and enhanced defense through vector inversion without retraining.

Original authors: Shivam Ratnakar, Kartikeya Vats

Published 2026-06-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Shivam Ratnakar, Kartikeya Vats

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a Large Language Model (LLM) as a very smart, highly trained assistant who has been taught a strict rule: "Never do anything harmful." For a long time, we thought this rule was woven deep into the assistant's brain, like a fundamental part of their personality. But this paper suggests something different: the rule is actually more like a sticky note slapped on the very last page of their answer, rather than a deep-seated belief.

Here is the breakdown of the paper's findings using simple analogies:

1. The "Sticky Note" Theory (The Core Discovery)

The researchers discovered that for many popular AI models (like Llama), safety isn't a complex, deep thought process. Instead, it's a linear feature—a single, straight line in the math that the model uses to decide "Yes" or "No."

The Analogy: Imagine the AI is a chef preparing a meal.
- Old View: We thought the chef had a deep, internal moral compass that stopped them from cooking poison at the very start of the recipe.
- New View: The chef cooks the poison dish exactly the same way they cook a safe dish. The "safety" is just a single instruction written on a sticky note at the very end: "Don't serve this." If you peel off that sticky note, the chef happily serves the poison dish.

2. The New Tool: "Contrastive Logit Steering" (CLS)

The authors created a tool called CLS to test this theory. Instead of trying to trick the AI with confusing riddles or long, complex prompts (which is how hackers usually try to break safety), CLS uses simple math.

How it works:
1. The researchers ask the AI the same question twice: once with a "safe" instruction and once with an "unrestricted" instruction.
2. They look at the difference between the two answers. This difference is the "Refusal Vector" (the math representation of the "No").
3. They then subtract that "No" from the AI's final answer before it speaks.
The Result: It's like taking the sticky note off the chef's plate. The AI, which was about to say "I can't do that," suddenly says, "Sure, here is how you do that."

3. The "Late Decision" vs. "Early Divergence"

The paper found that not all AI models are the same. They fall into two categories based on when they decide to refuse a harmful request:

The "Late Decision" Models (e.g., Llama-3.1):
- Analogy: These models are like a student who listens to the entire lecture, takes notes, and only at the very last second, right before handing in the test, remembers, "Oh wait, I'm not supposed to write that."
- Vulnerability: Because they wait until the very end to decide, the researchers' tool (CLS) can easily bypass them. They achieved a 95% success rate in getting these models to say "yes" to harmful requests in just one second.
The "Early Divergence" Models (e.g., Qwen-2.5):
- Analogy: These models are like a student who realizes halfway through the lecture, "This topic is off-limits," and stops taking notes on that subject immediately.
- Result: They are much harder to break. Because the safety decision happens deep inside the thinking process (not just at the end), simply peeling off the "sticky note" at the end doesn't work as well. They are more robust.

4. Speed and Efficiency

The paper highlights that this method is incredibly fast compared to previous hacking attempts.

Old Way (GCG): Trying to find a magic phrase to trick the AI is like trying to pick a lock by trying every single key in a giant keychain. It takes about 15 minutes per attempt and often fails.
New Way (CLS): This is like having a master key that opens the door instantly. It takes one second and works almost every time on the "Late Decision" models.

5. The "Double-Edged Sword" (Defense)

The most surprising finding is that this same math trick can be used to protect the AI, not just break it.

The Analogy: If you can peel off the "No" to make the AI say "Yes" to bad things, you can also add more "No"s to make it extra strict.
The Result: By reversing the math (subtracting the "Yes" tendency), the researchers made the models more resistant to jailbreaks without needing to retrain them. It also made the AI's answers sound more confident and less hesitant.

Summary

The paper argues that current safety measures in many AI models are superficial. They are like a thin layer of paint over a deep capability. The researchers found a way to scrape off that paint instantly using simple math. While this exposes a vulnerability, it also provides a new way to understand how AI thinks and offers a tool to make AI safer by reinforcing that "paint" layer more effectively.

Key Takeaway: Safety in these models is often a "surface-level" feature that can be turned on or off with a simple mathematical switch, rather than a deep, unchangeable part of the model's intelligence.

Technical Summary: The Geometry of Refusal: Linear Instability in Safety-Aligned LLMs

Problem Statement

Despite the widespread adoption of safety alignment techniques, such as Reinforcement Learning from Human Feedback (RLHF), the mechanistic basis of how Large Language Models (LLMs) refuse harmful queries remains opaque. A critical open question is whether safety compliance represents a deep semantic understanding of harm or merely a shallow, manipulable linear feature. Current adversarial attacks often rely on computationally expensive optimization (e.g., GCG) or iterative search, which may fail against modern safety training. This work investigates whether safety is encoded as a linear geometric feature in the model's activation space that can be bypassed or reinforced through direct arithmetic intervention on output logits.

Methodology: Contrastive Logit Steering (CLS)

The authors introduce Contrastive Logit Steering (CLS), a zero-optimization framework that operates directly on the model's output distribution (logits) rather than internal hidden activations.

Core Mechanism

Contrastive Prompting: For a given user query $q$ $q$ , the model performs three parallel forward passes using distinct system prompts:
- Base ( $s_{base}$ ): A neutral instruction (e.g., "You are a helpful assistant").
- Unrestricted ( $s_{+}$ ): An adversarial instruction stripping safety guardrails.
- Restricted ( $s_{-}$ ): A restrictive instruction enforcing extreme safety.
Vector Isolation: At each decoding step $t$ , the method computes an instantaneous Steering Vector $v_t$ by subtracting the logits of the restricted stream from the unrestricted stream:
$v_t = z^+_t - z^-_t$
This vector isolates the "Refusal Direction"—the specific geometric direction in logit space that encodes the binary distinction between compliance and refusal.
Logit Intervention: The model's base logits are modulated by adding the scaled steering vector:
$\tilde{z}_t = z^{base}_t + \alpha \cdot v_t$
- Jailbreaking ( $\alpha > 0$ ): Injects the "unrestricted" behavior, suppressing safety tokens.
- Hardening ( $\alpha < 0$ ): Subtracts the unrestricted behavior, artificially inducing safety.
Prefix Injection: To bypass the model's initial refusal reflex (where models often output "I cannot" immediately), the method forces the first $k$ tokens to an affirmative prefix (e.g., "Sure") before applying the steering vector to subsequent tokens.

Detection Mechanism

The framework also utilizes the geometric separability of refusal for zero-shot detection. By computing a Global Refusal Vector from the difference in final-layer hidden states between harmful and benign anchor queries, the system can classify new queries based on cosine similarity without training a separate classifier.

Key Contributions

CLS as a Diagnostic Probe: The paper provides a zero-shot, logit-level steering framework that offers mechanistic evidence that safety in architectures like Llama acts as a shallow "wrapper" rather than a deep semantic constraint.
Topology of Refusal: Through KL-divergence analysis across network depth, the authors identify two distinct safety topologies:
- "Late Decision" Models (e.g., Llama-3.1): These models process harmful and benign queries identically for ~95% of their layers, diverging only at the final output head. This makes them highly vulnerable to logit-level steering.
- "Early Divergence" Models (e.g., Qwen-2.5): These models integrate safety mid-computation (~40% depth), making them significantly more robust to linear steering as the refusal mechanism is entangled with reasoning.
Empirical Baselines: The work provides direct comparisons against established activation-level steering methods (Arditi et al., 2024) and optimization-based attacks (GCG), demonstrating that logit-level intervention exposes greater alignment fragility.
Bidirectional Control: The method enables both attack (jailbreaking) and defense (safety hardening) via a single vector. Inverting the steering vector ( $\alpha < 0$ ) was shown to "harden" models against jailbreaks and, counter-intuitively, improve response coherence by reducing ambiguity.

Experimental Results

The authors evaluated CLS across seven model families (Gemma-3, Llama-3.1/3.2/3.3, Qwen-2.5) using datasets including AdvBench, JailbreakBench, and HarmBench.

Attack Success Rate (ASR) vs. GCG: On Llama-3.1-8B, GCG achieved only 5% ASR after 100 optimization steps, whereas CLS achieved 95% ASR in approximately one second (a ~900x speedup). On Qwen-2.5-7B, CLS achieved 90% ASR compared to GCG's 45%.
Comparison with Activation Steering: When compared to Arditi et al. (2024), CLS achieved substantially higher ASR on Llama 2 (73% vs. 22.6%) and Qwen 7B (91% vs. 79.2%). The authors attribute the larger gap on Llama to the "Late Decision" topology, where the refusal signal is concentrated at the output layer, making logit-level intervention more effective than hidden-state ablation.
Safety Hardening: Negative steering ( $\alpha = -1.0$ ) reduced attack success rates to single digits (e.g., Llama-3.3-70B dropped from 68.8% to 9.4% ASR).
Zero-Shot Detection: Using the Global Refusal Vector, the method achieved an F1 score of 0.92 on JailbreakBench for distinguishing malicious from benign prompts without training a separate model.

Significance and Claims

The paper argues that current alignment techniques create a steerable "safety axis" that serves as both a critical vulnerability and a precise primitive for defense.

Mechanistic Insight: The findings suggest that for many models, RLHF does not "unlearn" harmful knowledge but rather applies a linear suppression feature at the output layer. The "Refusal Vector" captures not just safety compliance but also hesitation and ambiguity; removing it can paradoxically improve coherence.
Architectural Determinism: The disparity in vulnerability between Llama (Late Decision) and Qwen (Early Divergence) implies that safety integration depth is a deterministic factor in steerability, likely shaped by both architecture and training interventions.
Defense Implication: The authors claim that as long as safety is implemented as a separable linear subspace, it remains vulnerable to geometric intervention. They propose that future alignment paradigms must entangle safety with general capability to make refusal a deep semantic property rather than a removable surface feature.

The work concludes that CLS provides a real-time primitive for auditing suppressed capabilities and dynamically hardening models, highlighting the fragility of current safety implementations.