Jailbreaking Leaves a Trace: Understanding and Detecting Jailbreak Attacks from Internal Representations of Large Language Models

1. Problem Statement

Large Language Models (LLMs) are increasingly vulnerable to jailbreaking, where adversarial users craft specific prompts to bypass safety alignment and elicit restricted or harmful content.

• Limitations of Current Defenses: Existing defenses primarily operate at the prompt level (filtering keywords) or output level (using auxiliary classifiers). These methods are often brittle, easily bypassed by paraphrasing or obfuscation, and fail to generalize against adaptive attack strategies.
• The Gap: There is a lack of research leveraging the internal representations of LLMs to detect and mitigate these attacks. The authors hypothesize that jailbreak prompts induce distinct, detectable structural patterns within the model's hidden layers that differ from benign inputs, regardless of the final output.

2. Methodology

The proposed framework operates in two phases: Latent Analysis (understanding the signatures) and Inference-Time Mitigation (active defense).

A. Data and Model Scope

The study analyzes a diverse suite of models to ensure architecture-agnostic findings:

• Architectures: Transformer-based models (GPT-J, LLaMA, Mistral) and State-Space models (Mamba2).
• Training Stages: Base models, Instruction-Tuned models, and Abliterated models (instruction-tuned models with safety alignment removed).
• Datasets: A training set from Hugging Face (Jailbreak Classification) and a held-out test set from "In the Wild Jailbreak Prompts."

B. Core Technique: Tensor Decomposition

Instead of analyzing raw text, the authors extract internal activations (multi-head attention outputs and layer hidden states) for each prompt.

1. Tensor Construction: For a batch of $N$ prompts, they construct a 3-mode tensor of size $N \times T \times d$ (Prompts $\times$ Sequence Length $\times$ Hidden Dimension).
2. CP Decomposition: They apply CANDECOMP/PARAFAC (CP) tensor decomposition to factorize these tensors into low-rank components. This extracts latent factors corresponding to the prompt, sequence, and hidden dimensions.
3. Signature Extraction: The prompt-mode factors capture the latent patterns distinguishing jailbreak from benign inputs. These factors serve as features for a lightweight classifier.

C. Layer-Wise Susceptibility Analysis

The authors train separate classifiers on latent factors extracted from individual layers to determine which parts of the network are most sensitive to adversarial inputs.

• Goal: Identify specific layers where the representation of jailbreak prompts is most separable from benign ones.
• Hypothesis: High separability in a specific layer indicates that the layer is "susceptible" to encoding adversarial intent.

D. Inference-Time Mitigation (Layer Bypass)

Based on the susceptibility analysis, the authors propose an active defense mechanism:

1. Real-time Scoring: For a new input prompt, the system extracts layer representations, projects them onto the pre-learned latent factors, and calculates a jailbreak probability score for each layer using a lightweight logistic regression classifier.
2. Selective Bypass: If a layer's probability exceeds a threshold (e.g., $\tau=0.7$), that layer is bypassed during inference.
   • Layer Bypass: The output of the suspicious layer is skipped, feeding the previous layer's output directly to the next.
   • MHA Bypass: Specific attention heads can also be bypassed.
3. Advantage: This requires no model fine-tuning, no auxiliary LLMs, and introduces minimal computational overhead.

3. Key Contributions

1. Layer-Wise Jailbreak Signatures: The paper demonstrates that jailbreak and benign prompts exhibit distinct, layer-dependent latent signatures in internal representations. These signatures are consistent across different model families (Transformers and Mamba) and alignment stages.
2. Tensor-Based Detection Framework: A novel method using CP tensor decomposition to extract low-dimensional, interpretable features from high-dimensional hidden states, enabling lightweight detection without retraining.
3. Active Mitigation via Layer Bypass: A novel defense strategy that disrupts jailbreak execution by selectively bypassing "susceptible" layers at inference time. This approach effectively blocks attacks while preserving benign functionality.
4. Comprehensive Evaluation: The study covers a wide range of models (Base, Instruct, Abliterated) and architectures, proving that adversarial intent leaves stable traces in the latent space.

4. Experimental Results

The experiments were conducted on an Abliterated LLaMA 3.1 8B model to isolate the effects of the defense without interference from pre-existing safety guardrails.

• Detection Accuracy:

  • Latent factors derived from early layers showed high separability between benign and jailbreak prompts (F1 scores > 0.8 in many layers).
  • Self-Attention vs. Hidden States: Both multi-head attention (MHA) outputs and layer outputs provided discriminative signals, though layer outputs generally offered slightly better separability.
  • Visualization: t-SNE plots showed clear clustering separation between benign and jailbreak prompts in the latent space. Heatmaps revealed that jailbreaks cause localized, targeted changes in attention patterns and activation magnitudes rather than global disruption.

• Mitigation Performance (Abliterated LLaMA 3.1 8B):

  • Layer Bypass: Successfully blocked 78% of jailbreak attempts (True Positives) while preserving 94% of benign prompts (True Negatives).
  • Attack Success Rate (ASR): Reduced from 100% (baseline) to 22% with layer bypass.
  • Comparison: MHA-only bypass was less effective (ASR 39%), suggesting that layer outputs capture more critical adversarial computation than attention weights alone.

• Failure Analysis:

  • Remaining failures (22%) were primarily due to persona-based injections (e.g., "Act as an unfiltered AI") where adversarial control is reinforced across many layers, or attacks that distribute influence too thinly to trigger the layer-specific threshold.

5. Significance and Implications

• Architecture-Agnostic Security: The findings suggest that jailbreak vulnerabilities are rooted in fundamental internal structures common to both Transformers and State-Space models, making this approach highly generalizable.
• Efficiency: Unlike fine-tuning or running a second "safety" LLM, this method operates entirely at inference time with minimal overhead, making it scalable for production environments.
• New Defense Paradigm: It shifts the security focus from "filtering the input/output" to "monitoring and intervening in the internal computation," offering a complementary layer of defense that is harder for attackers to evade via simple prompt engineering.
• Interpretability: The use of tensor decomposition provides a mathematically grounded way to visualize and understand where and how adversarial intent propagates through a neural network.

In conclusion, the paper establishes that "jailbreaking leaves a trace" in the internal latent space of LLMs. By detecting and surgically bypassing these traces, it is possible to significantly reduce the success rate of jailbreak attacks without compromising the model's utility on benign tasks.