Technical Summary: Analysing Self-Harm Representations in Language Models

Problem Statement

The detection of self-harm content on social media is a high-stakes task requiring high accuracy to facilitate timely intervention and flagging of at-risk users. While automated detection systems using LSTMs, BERT-based transformers, and LLMs have been developed for content moderation and mental health monitoring, a significant gap remains in understanding the internal mechanics of these models. Specifically, there is limited knowledge regarding:

• Where in the network self-harm concepts "crystallize."
• The geometric stability of these representations across layers.
• Whether the structural representation of self-harm is consistent across different architectures and scales.

The authors argue that understanding these internal representations is crucial for building safer systems that move beyond prompt engineering to include explicit, testable components for intervention and governance.

Methodology

The study employs a cross-architecture analysis using two datasets and four instruction-tuned transformer models.

Datasets:

1. X-Sensitive (X-S): A multi-label corpus of English social media posts annotated for six categories. The authors extracted 200 self-harm positive posts and 200 control posts (all categories zero).
2. SH-Detection (SH-D): A binary dataset where self-harm posts are labeled "self-harm" and all others serve as controls. A balanced 600-post dataset was sampled.
   Preprocessing included removing posts under 20 characters, stripping URLs, collapsing whitespace, and anonymizing mentions.

Models:
Four instruction-tuned transformers were analyzed:

• Qwen3-0.6B (0.6B parameters, 28 layers)
• Llama 3.2-1B-Instruct (1B parameters, 16 layers)
• Llama 3.2-3B-Instruct (3B parameters, 28 layers)
• Gemma-3-4B-it (4B parameters, 34 layers)

Experimental Setup:

1. Layer-wise Probing: For each layer $\ell$, the authors extracted last-token residual stream activations ($a_\ell$). They trained an $\ell_2$-regularized logistic regression probe (linear classifier) using 5-fold stratified cross-validation. The primary metric was mean ROC-AUC, reported without prior normalization of activations.
2. Contrastive Direction Analysis: Following Zou et al. (2023), the authors computed a contrastive self-harm direction vector $d_\ell$ at each layer as the normalized difference between the mean activations of self-harm posts ($\bar{a}^+$) and control posts ($\bar{a}^-$).
   • Separability: Measured using Cohen's $d$ on the scalar projections of posts onto $d_\ell$ at the best-separation layer.
   • Stability: Assessed via an $L \times L$ pairwise cosine similarity matrix of direction vectors across layers to determine if the direction remains consistent or rotates through the network depth.

Key Results

1. Crystallization of Self-Harm Concepts
Across all four models and both datasets, self-harm information consistently crystallizes in the final 3–7% of network layers (93% to 97% depth).

• Performance: On the SH-Detection dataset, all models achieved high performance (AUC 0.923 to 0.972) with lower variance compared to X-Sensitive. On X-Sensitive, performance was lower (AUC 0.703 to 0.817), likely due to the presence of figurative language and subtler negative examples in that corpus.
• Error Analysis: False negatives often involved figurative/metaphorical language or third-person contexts. False positives were frequently mental health advocacy posts or content warnings that used crisis-related vocabulary without conveying first-person distress.

2. Geometric Separability vs. Classification Accuracy
A critical finding is that the most accurate linear probes are not necessarily the most linearly separable in terms of mean-difference directions.

• Gemma-3-4B Anomaly: Gemma-3-4B achieved the highest probe AUC on X-Sensitive (0.817) but the lowest Cohen's $d$ (0.71) among the models.
• Interpretation: The authors suggest that Gemma's self-harm signal is not concentrated in a single direction but is distributed across the activation space. A linear probe can find a hyperplane exploiting this distributed signal, whereas a simple mean-difference direction (Cohen's $d$) fails to capture the full separation.
• Divergence: In Llama 3.2-3B, the contrastive direction peaked in separability (Cohen's $d$) at layer 14, well before the final layers where probe AUC continued to rise, indicating that mid-network directions may be potent but insufficient for accurate classification on their own.

3. Directional Stability
The self-harm contrastive direction is not stable across network depth.

• Block-Diagonal Structure: Adjacent layers show high alignment (cosine similarity 0.76–0.92), but directions from the first and last thirds of the network are nearly orthogonal (cosine similarity 0.14–0.38).
• Implication: The self-harm concept is dynamically re-represented as it passes through the network; a direction extracted at an early layer does not transfer to late layers. Gemma-3-4B exhibited the lowest cross-layer cosine similarity (0.135) on SH-D.

Significance and Claims

The paper claims that its findings pave the way for more effective intervention strategies by revealing the internal geometry of safety-relevant embeddings. Specifically:

• Representation Engineering: The study supports the use of representation engineering to understand model internals, moving beyond black-box detection to explicit, testable components.
• Architecture Differences: The results highlight that different models (e.g., Gemma vs. Llama) may represent the same safety concepts (self-harm) in structurally different ways (concentrated vs. distributed signals), which has implications for how safety mechanisms might need to be tailored to specific architectures.
• Data Quality Impact: The systematic performance gap between the X-Sensitive and SH-Detection datasets suggests that labeling quality and corpus composition significantly affect representational quality, a factor that must be considered when training or evaluating safety models.

The authors conclude that while self-harm concepts consistently crystallize in late layers, the geometric nature of these representations is complex and model-specific, challenging the assumption that a single, stable "safety direction" exists across all layers and architectures.