Skin-Deep: A Geometric Diagnostic for Alignment Fragility in Large Language Model Representations

Technical Summary: SKIN-DEEP: A Geometric Diagnostic for Alignment Fragility

Problem Statement
Alignment tuning in Large Language Models (LLMs) is designed to ensure robust refusal of harmful requests. However, this safety behavior is often fragile: a model that passes refusal tests at release can lose its refusal capabilities after undergoing low-cost downstream interventions, such as fine-tuning on benign-looking examples or training low-rank adapters (LoRA). While prior work has demonstrated that such interventions can bypass safety, existing studies generally only detect this fragility after an attack or fine-tuning intervention has been executed. There is currently no method to detect this fragility directly from the aligned model's internal representations before such interventions occur.

Methodology: SKIN-DEEP
The authors introduce SKIN-DEEP, a geometric diagnostic framework that measures alignment fragility using the hidden-state activations of an aligned model prior to any intervention. The method operates through the following steps:

1. Subspace Search: The framework compares the residual-stream activations of an aligned model ($M$) against its base counterpart ($M_0$) on matched sets of harmful-request prompts and benign instructions. Using contrastive PCA (cPCA), it identifies candidate directions ($v_\ell$) at each layer $\ell$ that best separate harmful requests from benign instructions.
2. Validation: To ensure these directions represent genuine safety geometry rather than artifacts (e.g., activation magnitude or prompt templates), the authors employ:
   • Cohen's $d$: To measure the separation magnitude along the candidate direction.
   • Full-space tests: Using PERMANOVA and RBF-kernel MMD to verify the split exists in the full activation space, not just the 1D projection.
   • Norm controls: Verifying results hold on unit-normalized activations.
   • Arditi Direction Comparison: Measuring cosine similarity with the known "Arditi direction" (the difference between mean harmful and harmless activations) to ensure the recovered geometry is distinct or related to known refusal axes.
3. Causal Verification (Direction Ablation): The framework tests the behavioral relevance of these directions by removing (ablating) the candidate direction during generation. If the model's refusal rate drops significantly compared to random or reference direction controls, the geometry is causally linked to refusal behavior.
4. Geometric Fragility Score (GFS): The layer-wise measurements are compressed into a single scalar metric, the Geometric Fragility Score (GFS). The formula aggregates weighted separation strength ($|d_\ell|$), emphasizing later layers ($w_\ell \propto \ell/L$), and downweighting directions that align too closely with the known Arditi refusal axis:
   $$GFS(M) = \sum_{\ell=1}^{L} w_\ell \cdot |d_\ell| \cdot (1 - |\cos(v_\ell, v^{Arditi}_\ell)|)$$
   A high GFS indicates a strong, late-emerging safety geometry that is not concentrated solely on the canonical refusal axis.

Key Contributions and Results
The paper evaluates SKIN-DEEP on a pool of 21 instruction-tuned models (3B–32B parameters) across six alignment recipes (SFT, RLHF+SFT, DPO, ORPO, RLAIF, C-RLFT). The results support four main claims:

• Existence of a Low-Rank Safety Subspace: Harmful requests and benign instructions separate along a small set of hidden-state directions. This separation persists in full activation space and is robust to prompt formatting (chat templates) and activation norm variations.
• Causal Link to Refusal: Removing specific directions identified by the cPCA search weakens harmful-request refusal in models where a decrease is measurable. This confirms that the recovered geometry underlies refusal behavior, not just prompt classification. The specific "refusal-changing" direction varies by model, suggesting a shared low-rank subspace containing model-specific vectors rather than a single universal refusal vector.
• Recurrence Across Families: Different model families (e.g., Llama, Qwen, Mistral, Gemma) preserve similar prompt-to-prompt activation relationships within this safety subspace, even though the layer where the separation is strongest varies by model.
• Diagnostic Utility (GFS): The GFS successfully predicts which initially safe models will retain the most refusal after small-scale LoRA fine-tuning. In experiments, Gemma-2-9B received the lowest GFS and was the only model that retained substantial refusal after aggressive benign LoRA updates, whereas other models (Llama, Qwen, Mistral) rapidly lost refusal capabilities. The authors attribute Gemma's resilience to its safety directions remaining closer to the canonical Arditi refusal axis, which the GFS formula downweights, correctly flagging it as less "fragile" in the context of low-rank perturbations.

Significance and Claims
The paper establishes GFS as a practical pre-deployment diagnostic. Its primary significance lies in the ability to flag alignment fragility without running an attack or fine-tuning intervention. By inspecting the geometric signature of refusal in the residual stream, developers can identify models that rely on brittle safety mechanisms before they are released.

The authors position this work as a tool for defensive utility:

• It helps flag under-aligned checkpoints.
• It aids in prioritizing safety fine-tuning or interpretability probes.
• It assists in triaging red-teaming budgets.

The paper explicitly notes limitations, including the scope of model sizes (up to 32B), the use of English-only prompts, and the fact that the GFS ranking for alignment recipes (e.g., DPO models appearing more robust) is a hypothesis requiring further validation on larger populations. The authors also emphasize a dual-use risk: while the diagnostic pipeline is released, specific attack-ready artifacts (such as layer-specific ablation hooks and adapter weights) are withheld to prevent immediate exploitation, though they acknowledge that determined adversaries could reconstruct similar pipelines using public data.