Technical Summary: Scaling Monosemanticity to Claude 3 Sonnet

Problem Statement

The central open question addressed by this work is whether dictionary learning methods, specifically sparse autoencoders (SAEs), can scale from small, toy transformers to production-scale language models to extract interpretable, monosemantic features. Previous work demonstrated that SAEs could recover monosemantic features from a one-layer transformer, but it remained unclear if these methods would remain feasible or effective for state-of-the-art models like Claude 3 Sonnet. A major concern was that engineering constraints or fundamental differences in how large models operate might prevent these methods from generalizing, thereby limiting their practical contribution to AI safety and mechanistic interpretability.

Methodology

Sparse Autoencoder (SAE) Architecture

The authors trained sparse autoencoders on the residual stream activations of Claude 3 Sonnet, specifically at the middle layer. The SAE consists of:

1. Encoder: A linear transformation followed by a ReLU nonlinearity, mapping the input activation vector $x \in \mathbb{R}^D$ to a higher-dimensional feature space of size $F$.
2. Decoder: A linear transformation attempting to reconstruct the original activation vector $\hat{x}$.

The model is trained to minimize a loss function $L$ combining reconstruction error (L2) and an L1 regularization penalty on feature activations to incentivize sparsity:
$$L = \mathbb{E}_x \left[ \|x - \hat{x}\|_2^2 + \lambda \sum_i f_i(x) \cdot \|W_{\text{dec}, \cdot, i}\|_2 \right]$$
where $f_i(x)$ is the activation of feature $i$, and the inclusion of the decoder weight norm in the penalty term allows for the interpretation of unit-normalized decoder vectors as feature directions.

Scaling and Training

• Model: Claude 3 Sonnet (the finetuned, production version released March 4, 2024).
• SAE Sizes: Three SAEs were trained with varying dictionary sizes: ~1M, ~4M, and ~34M features.
• Hyperparameter Selection: The authors utilized scaling laws to guide hyperparameter selection. They treated dictionary learning as a standard machine learning problem, sweeping over the number of features and training steps to find compute-optimal configurations that minimize loss. They observed that loss decreases according to a power law with respect to compute, and optimal allocations for features and steps also follow power laws.
• Data: Training was performed on a text-only dataset similar to the model's pretraining distribution. Notably, the SAEs were not trained on "Human:/Assistant:" formatted data or images, yet they demonstrated generalization to both.

Key Contributions and Results

1. Scalability of Sparse Autoencoders

The primary contribution is the demonstration that SAEs can successfully extract high-quality, interpretable features from a production-scale model. The 34M feature SAE explained at least 65% of the variance in the model's residual stream activations, with fewer than 300 features active on average per token.

2. Feature Properties

The extracted features exhibit several robust properties:

• Multilingual and Multimodal: Features respond to the same concepts across different languages and generalize to images despite being trained solely on text.
• Concrete and Abstract Generalization: Features activate on both concrete instances (e.g., specific code with a bug) and abstract discussions of the same concept (e.g., discussing security vulnerabilities).
• Monosemanticity: Features correspond to specific concepts, ranging from famous entities (e.g., Richard Feynman, the Golden Gate Bridge) to abstract concepts (e.g., sarcasm, code errors, deception).

3. Feature Interpretability and Causality

The authors validated feature interpretations through:

• Specificity: Automated scoring using Claude 3 Opus confirmed that high-activation inputs strongly matched the proposed feature interpretations.
• Influence on Behavior (Steering): By "clamping" feature activations to high or low values during the forward pass, the authors demonstrated causal control over model outputs. For example:
  • Clamping a "Golden Gate Bridge" feature induced the model to self-identify as the bridge.
  • Clamping a "code error" feature caused the model to hallucinate errors in bug-free code or rewrite buggy code to be correct.
  • Clamping a "deception/internal conflict" feature caused the model to reveal information it was attempting to "forget."

4. Safety-Relevant Features

The study identified features corresponding to safety-critical concepts, including:

• Deception and Power-Seeking: Features related to lying, treacherous turns, and self-improving AI.
• Bias and Sycophancy: Features detecting gender bias, racial slurs, and excessive praise.
• Dangerous Content: Features activating on discussions of biological weapons, scam emails, and backdoors.
• Causal Influence: Steering these features confirmed they causally influence model behavior (e.g., clamping a "scam email" feature caused the model to generate scam emails despite safety training).

5. Feature Geometry and Completeness

• Feature Neighborhoods: Features are organized in geometric clusters in decoder space that reflect semantic relationships (e.g., a "San Francisco" feature in the 1M SAE splits into finer-grained features in the 34M SAE).
• Frequency vs. Dictionary Size: A systematic relationship was found between the frequency of a concept in training data and the dictionary size required to resolve a feature for it. Rare concepts require significantly larger dictionaries.
• Comparison to Neurons: SAE features were found to be significantly more interpretable and specific than individual MLP neurons, with the vast majority of features showing no strong correlation with any single neuron.

Significance and Claims

The paper claims to have successfully scaled monosemanticity to a medium-sized production model, addressing the concern that dictionary learning might not generalize beyond small transformers. The authors argue that this provides a "test set for safety" by allowing researchers to inspect the internal representations of large models.

Modest Claims and Limitations:
The authors explicitly caution against over-interpreting the mere existence of safety-relevant features. They state:

• Existence vs. Capability: The presence of features for deception or dangerous content does not imply the model is currently deceptive or dangerous; it reflects the model's exposure to such concepts during pretraining.
• Incompleteness: The suite of features is incomplete. The authors do not claim to have found "all" features and note that current methods may miss features due to "cross-layer superposition" (where features are smeared across multiple layers) or shrinkage issues in L1 regularization.
• Evaluation: There is currently no rigorous "ground truth" method to evaluate whether features faithfully capture all model computations. The loss function used is a proxy, not a definitive measure of interpretability.
• Safety Utility: While the features are plausibly useful for safety, the paper does not claim they are currently sufficient for deploying safety mechanisms. The work is described as "preliminary," intended to encourage further research into understanding the implications of these features.

In summary, the paper establishes that sparse autoencoders can scale to large models to reveal a rich, interpretable, and causally influential set of features, providing a new tool for mechanistic interpretability while acknowledging significant remaining challenges in completeness and evaluation.