🤖 AI

Using Mechanistic Interpretability to Craft Adversarial Attacks against Large Language Models

This paper introduces a novel white-box adversarial attack method that leverages mechanistic interpretability to identify and reroute embeddings from refusal to acceptance subspaces, achieving high jailbreak success rates on state-of-the-art LLMs with significantly reduced computational costs compared to existing techniques.

Original authors: Thomas Winninger, Boussad Addad, Katarzyna Kapusta

Published 2026-07-07

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Thomas Winninger, Boussad Addad, Katarzyna Kapusta

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Cracking the Safe Without Picking the Lock

Imagine a Large Language Model (LLM) as a very smart, highly trained librarian. This librarian has a strict rulebook: "If someone asks for instructions on how to build a bomb, you must refuse."

Traditional hackers (adversarial attacks) try to trick this librarian by shouting confusing words or using complex riddles until the librarian gets confused and accidentally gives the answer. This is like trying to pick a lock by jiggling every key until one works. It takes a long time, requires a lot of trial and error, and often fails on newer, smarter librarians.

This paper introduces a new method called "Subspace Rerouting" (SSR). Instead of jiggling keys, the researchers looked inside the librarian's brain to see exactly how the "refusal" thought is formed. They then crafted a specific "magic phrase" that forces the librarian's brain to take a different path, bypassing the refusal rule entirely.

How It Works: The Three Steps

1. Mapping the "Refusal Room"

The researchers first studied the model to find where the "refusal" happens.

The Analogy: Imagine the librarian's mind is a giant building with many rooms. The researchers found a specific hallway (a "subspace") where the thought "I cannot do this" lives.
The Discovery: They realized that when the model sees a harmful request, its internal thoughts (activations) get pushed into this "Refusal Room." When it sees a harmless request, the thoughts stay in the "Helpful Room." These two rooms are clearly separated, like two different colors of paint that don't mix.

2. The "Rerouting" Trick

Once they knew where the "Refusal Room" was, they didn't try to argue with the librarian. Instead, they calculated a shortcut.

The Analogy: Imagine you are walking toward a dead-end wall (the refusal). Instead of hitting the wall, the researchers found a hidden tunnel that connects the "Harmful Request" area directly to the "Helpful Response" area, skipping the wall entirely.
The Method: They added a few special words (a "suffix") to the end of the harmful question. These words act like a GPS signal that tells the model's internal thoughts, "Don't go to the Refusal Room; take this detour to the Helpful Room instead."

3. The Result: Instant Jailbreaks

The Old Way: Traditional methods (like GCG) are like trying to break down the wall by throwing rocks at it for hours. They often fail or take a long time.
The New Way (SSR): This method is like finding the secret door. The researchers were able to break the model's safety in seconds (sometimes just 14 seconds) with a success rate of 80% to 95%. They did this on modern, tough models like Llama 3.2 and Gemma 2.

The Three "Keys" They Tried

The paper tested three different ways to find the "secret door":

The Classifier Key (Probe-SSR): They trained a simple detector to spot "Refusal Thoughts." Then, they optimized the input to trick this detector into thinking the harmful request was actually harmless. This was the most effective method.
The Compass Key (Steering-SSR): They found a specific "direction" in the model's mind that points toward "Refusal." They then pushed the thoughts in the opposite direction. This worked well but was slightly less efficient than the first method.
The Spotlight Key (Attention-SSR): They found specific parts of the model (attention heads) that focus on the dangerous words. They tried to write a suffix that forces these parts to look at the harmless words instead. This was the least effective method in practice, though interesting for research.

A Surprising Discovery: The "Natural" Trick

One of the coolest findings was that the "magic words" the computer generated weren't just random gibberish (like "asjdhf98"). Sometimes, they formed coherent, sensible sentences.

The Analogy: Imagine trying to hack a security system by typing random letters. Usually, you get "xkq9z." But this method sometimes produced a sentence like, "Do this responsibly through ecological accounting."
Why it matters: This suggests the model's safety mechanisms are very specific. The model can be tricked into thinking a harmful request is safe if you frame it with the right context, even if that context is slightly weird.

Why This Matters (According to the Paper)

Speed: It turns a process that used to take hours into one that takes seconds.
Understanding: It proves that we can understand how AI models think (mechanistic interpretability) and use that knowledge to break them.
Defense: By seeing exactly how the "Refusal Room" works and how easily it can be bypassed, developers can build better locks (defenses) that don't have these specific holes.

What the Paper Does Not Claim

It does not claim this works on every model in every situation (it worked best on the specific models they tested).
It does not claim this is a tool for clinical use or medical diagnosis.
It does not claim that the "magic words" are always perfect English; sometimes they are still a bit strange, just less so than before.

In short, the paper says: "We looked inside the AI's brain, found the switch that turns off safety, and built a remote control to flip that switch instantly."

Technical Summary: Using Mechanistic Interpretability to Craft Adversarial Attacks against Large Language Models

Problem Statement

Large Language Models (LLMs) remain vulnerable to adversarial perturbations, specifically "jailbreaks" designed to bypass safety alignment. Traditional white-box attack methods, such as Greedy Coordinate Gradient (GCG), rely on end-to-end gradient optimization targeting specific output tokens. These approaches often ignore the internal mechanisms responsible for refusal or acceptance, resulting in computationally expensive processes (often requiring hours) that struggle to achieve high success rates on newer, robust models. Conversely, mechanistic interpretability research has successfully identified internal safety mechanisms—such as refusal subspaces and specific attention heads—but has largely been limited to runtime interventions (e.g., activation steering) that produce out-of-distribution activations and lack practical application for crafting static adversarial inputs.

This paper addresses the gap between interpretability insights and practical adversarial attack generation. The authors propose a method that leverages the understanding of internal model mechanisms to craft efficient, high-success-rate adversarial inputs without relying on runtime interventions.

Methodology: Subspace Rerouting (SSR)

The authors introduce Subspace Rerouting (SSR), a white-box framework that frames jailbreaking as an optimization problem in the model's continuous embedding space. Instead of targeting specific output tokens, SSR aims to reroute model embeddings from "refusal subspaces" to "acceptance subspaces" by modifying input suffixes.

1. Identifying Subspaces

The authors first characterize the internal representations of refusal and acceptance using interpretability techniques:

Semantic Subspaces: Using linear probes trained on contrastive datasets (harmful vs. harmless instructions), the authors demonstrate that harmful and harmless content occupies linearly separable regions in activation space as early as the third layer of the model. They also define a "refusal direction" ( $\hat{r}$ ) as the difference between mean harmful and harmless activations.
Component Subspaces: The authors identify specific attention heads responsible for safety mechanisms. They observe that successful jailbreaks often involve "attention hijacking," where safety-critical heads shift their focus from harmful instruction tokens to benign suffix tokens.

2. The SSR Algorithm

The core algorithm minimizes a loss function $\mathcal{L}(x_m)$ that measures the deviation of activations from targeted subspaces:
$\mathcal{L}(x_m) = \sum_{k} \alpha_k d_k(e_k, E_k)$
Where $x_m$ is the adversarial suffix, $e_k$ are activations at targeted layers, and $E_k$ are the target acceptance subspaces. The optimization uses a greedy coordinate gradient search adapted for discrete tokens.

The paper details three specific implementations of SSR:

Probe-SSR: Uses trained linear classifiers to guide optimization toward the "harmless" class probability.
Steering-SSR: Targets the refusal direction directly, optimizing inputs to achieve effects similar to runtime steering ( $e \leftarrow e^* + a\hat{r}$ ) but within the natural probability distribution.
Attention-SSR: Attempts to redirect the attention of specific safety-critical heads away from harmful content and toward the adversarial suffix, or minimizes the output magnitude of these heads.

Key Contributions

Novel Attack Vector: The introduction of SSR, a method that operationalizes mechanistic interpretability to craft practical adversarial inputs, achieving attack success rates (ASR) of 80–95% on state-of-the-art models (Gemma2, Llama3.2, Qwen2.5) within seconds or minutes.
Subspace Identification: Multiple methods for defining refusal and acceptance subspaces using linear probes and refusal directions, confirming the linear separability of safety concepts in early and mid-layers.
Efficiency and Interpretability: Demonstrating that targeting specific internal subspaces is significantly more computationally efficient than end-to-end token optimization. Furthermore, the method produces perturbations that remain within the model's natural probability distribution, unlike runtime steering interventions.
Cross-Layer Stability Analysis: The paper provides evidence that refusal subspaces crystallize into stable representations in the second half of the model, suggesting that successful rerouting requires targeting multiple layers to prevent the model from "recovering" to a refusal state.

Experimental Results

Performance: On the AdvBench dataset, Probe-SSR and Steering-SSR achieved ASRs between 80% and 95% across Llama3.2, Gemma2, and Qwen2.5. In contrast, the nanoGCG baseline achieved 0% to 53% success, and vanilla harmful instructions were almost always refused.
Speed: Attacks were generated in seconds to minutes (e.g., 14 seconds for Llama3.2 1b), a significant reduction compared to the hours often required by GCG.
Distributional Properties: Analysis using PCA and KL divergence revealed that while runtime steering interventions create out-of-distribution activations that the model "corrects" in subsequent layers, SSR-generated perturbations maintain low cosine similarity with refusal directions throughout the forward pass, indicating they are more robust and natural to the model's processing.
Interpretable Jailbreaks: Surprisingly, the optimization process sometimes yielded coherent, interpretable token sequences (e.g., "responsibly," "ethical") rather than random gibberish, suggesting the model's internal safety mechanisms can be bypassed via semantically plausible paths.
Reverse-Lens: By optimizing to trigger the refusal direction (the inverse of a jailbreak), the authors automatically identified tokens (slurs, malicious terms) that strongly activate safety mechanisms, validating SSR as a potential interpretability tool.

Significance and Claims

The authors posit that SSR represents a significant advancement in both attack research and defense development.

For Attack Research: It demonstrates that understanding internal model mechanisms allows for more efficient and effective attacks than traditional black-box or end-to-end white-box methods. It highlights that current safety training may not account for subspace manipulations, leaving models vulnerable to these specific perturbations.
For Defense and Interpretability: SSR serves as a practical tool for probing model internals. By successfully rerouting embeddings, it reveals the structural vulnerabilities of safety guardrails. The authors argue that the ability to rapidly generate interpretable jailbreak datasets opens new directions for studying adversarial perturbations and developing more robust alignment techniques.
Modesty: The paper acknowledges limitations, including the difficulty of transferring attacks across different model architectures due to the surgical nature of the perturbations, the reliance on specific hyperparameters, and the fact that some attacks inadvertently change the semantic meaning of the prompt (e.g., redirecting a bomb-making query to a cake recipe). The authors note that the Attention-SSR implementation was less effective in practice than the semantic subspace approaches.

In conclusion, the paper asserts that mechanistic interpretability is not merely an analytical tool but a practical engine for generating adversarial examples, challenging the assumption that robust alignment is achieved without addressing the specific internal subspaces responsible for refusal.