🤖 machine learning

Descriptive Collision in Sparse Autoencoder Auto-Interpretability: When One Explanation Describes Many Features

This paper identifies and formalizes "descriptive collision," a previously overlooked failure mode in sparse autoencoder interpretability where distinct features share identical natural-language explanations, and proposes new metrics to penalize such non-discriminatory annotations that artificially inflate reported interpretability.

Original authors: Jordan F. McCann

Published 2026-05-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jordan F. McCann

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a massive library with millions of unique books (these are the "features" inside a smart computer program). To make sense of this library, you hire a team of librarians to write a short, one-sentence summary for every single book.

The goal is that if you read the summary, you should be able to find that exact book and no other.

However, this paper discovers a major problem with how these summaries are currently written. The author calls this "Descriptive Collision."

Here is the breakdown of the problem and the proposed solution, using simple analogies:

1. The Problem: Too Many Books, Too Few Labels

Imagine you have 1,000 different books in your library. The librarians are asked to write a short label for each one.

The Reality: Instead of writing 1,000 unique descriptions, the librarians keep reusing the same few phrases.
The Example: The phrase "Plural Nouns" is used as the label for 101 different books.
The Result: If you see a book labeled "Plural Nouns," you have no idea which of the 101 specific books you are actually looking at. It's like having 101 different people in a room all wearing the exact same name tag that says "John." If you call out "John," all 101 people turn around. You haven't identified a specific person; you've just identified a group.

The paper analyzed a real dataset of 722 features and found that 82% of them share their label with at least one other feature. In fact, the most common label ("plural nouns") was shared by 101 distinct features across different parts of the computer model.

2. Why Current Tests Miss This

Currently, researchers test if a label is "good" by asking: "Does this label accurately describe what happens when the feature turns on?"

The Flaw: The paper argues this test is blind to the collision problem.
The Analogy: Imagine you are testing if the name tag "John" is a good description for a specific person. You ask, "Does this person respond to the name John?" The answer is "Yes." So, the test says the label is perfect.
The Reality: The test doesn't ask, "Does only this person respond to the name John?" Because 100 other people also respond to "John," the label fails to identify the specific individual, even though it is technically "true" for everyone.

The paper proves mathematically that as long as a label describes the feature correctly, current scoring systems will give it a high score, even if that label is shared by dozens of other features.

3. The Proposed Solution: The "Distinguisher" Test

The author suggests we need a new way to grade these labels. Instead of just asking, "Is this label true?" we should ask, "Can this label tell this feature apart from its neighbors?"

They propose two new methods:

Discrimination Scoring: This is like a "spot the difference" game. You take a feature and its label, and you compare it to a "neighbor" feature that is very similar.
- The Test: "Here is a sentence. Does the label 'Plural Nouns' tell us if Feature A is active, or if Feature B (which is also about nouns) is active?"
- The Goal: A good label should be able to say, "This sentence activates Feature A, but not Feature B." If the label is just a generic "Plural Nouns" that fits both, it gets a low score because it fails to distinguish them.
Collision-Adjusted Detection: This is a simpler, cheaper fix. It takes the current score and divides it by how many other features share that label.
- The Math: If a label is shared by 100 features, its "goodness" score gets divided by 100. A label shared by 1 feature keeps its full score. This punishes generic labels that are too broad.

4. Why This Happens (The Root Cause)

The paper uses Goodhart's Law to explain this. This law says: "When a measure becomes a target, it ceases to be a good measure."

The Trap: Researchers are trying to get high scores on "detection" (does the label predict the feature?).
The Result: The system (or the AI writing the labels) learns that the easiest way to get a high score is to use broad, safe, generic phrases like "plural nouns" or "he" that are true for many things.
The Space Problem: There are millions of features (the "input space"), but only a limited number of short, simple English phrases (the "description space"). You simply cannot give a unique, short name to millions of things without running out of words and reusing them.

Summary

The paper claims that current methods for explaining AI features are overconfident. They tell us a label is "accurate" when it is actually just vague.

Old Way: "This label is 95% accurate!" (Because it describes the feature correctly).
New Way: "This label is 95% accurate, but it is shared by 100 other features, so it only identifies this specific feature 1% of the time."

The author concludes that we need to stop just checking if a label is true, and start checking if a label is unique enough to find the specific feature among millions of others.

Technical Summary: Descriptive Collision in Sparse Autoencoder Auto-Interpretability

1. Problem Definition

The paper identifies a structural failure in current Sparse Autoencoder (SAE) auto-interpretability pipelines termed descriptive collision. While existing critiques focus on polysemanticity (one feature having multiple meanings) or the predictive accuracy of explanations, this work highlights the inverse problem: many distinct SAE features admit the same natural-language explanation.

The core issue arises because the space of possible short natural-language descriptions (the "descriptive space") is structurally smaller than the space of SAE features (the "feature space"). Consequently, a single annotation string often maps to a large cluster of distinct features. The authors argue that current evaluation paradigms fail to detect this because they treat the feature as fixed and ask if the explanation describes it, rather than asking if the explanation uniquely identifies the feature among its peers.

2. Methodology and Definitions

Formal Framework

The authors formalize the problem using information theory:

Collision Cluster ( $C_E(f)$ ): For a feature $f$ and explanation function $E$ , the set of all features $g$ such that $E(g) = E(f)$ .
Descriptive Resolution: Defined as the mutual information $I(F; A)$ $I (F; A)$ between the random variable $F$ $F$ (selecting a feature uniformly) and $A = E(F)$ $A = E (F)$ (the annotation).
- Maximum resolution ( $\log_2 N$ bits) occurs when the explanation is injective (unique per feature).
- Resolution is reduced by the entropy of the collision clusters.

Empirical Analysis

The authors reanalyzed the largest publicly available dataset of human-annotated SAE features (Marks et al. [21]), comprising 722 features across Gemma 2 2B and Pythia 70M models.

Data Source: Human-annotated features with short natural-language strings, treated as a high-quality proxy for automated explanations.
Metric: Calculation of cluster sizes, mutual information, and the distribution of annotation reuse.

Theoretical Proof

The paper proves Proposition 1: Detection-style auto-interpretability scores are invariant under the substitution of any feature within a collision cluster. If two features share an explanation and have similar activation distributions, a standard detection score (measuring if the explanation predicts the feature's firing) will be identical for both. Therefore, high scores cannot certify that an explanation identifies a specific feature rather than just describing a property shared by the cluster.

3. Key Results

Empirical Findings

Pervasiveness: 82.1% of the 722 features share their annotation with at least one other feature.
Cluster Size: The mean annotation string is reused across 3.07 features. The largest cluster, labeled "plural nouns," contains 101 distinct features spanning 18 layers and four model components.
Information Loss: The average annotation resolves only 70% of feature identity. The residual entropy ( $H(F|A)$ ) is 2.85 bits, meaning that even after reading an annotation, the expected feature identity remains uncertain over an effective alphabet of $\approx 7.2$ features.
Cross-Layer Collision: 56.6% of reused annotation strings span at least two distinct layers, confirming that these collisions represent distinct mechanistic entities, not just redundant representations of the same concept.
Human Hedging: 55.7% of features with author notes used hedging language (e.g., "mostly," "sometimes"), indicating that even expert human annotators recognized their short labels as under-specifying the feature.

Scoring Invariance

The study demonstrates that standard detection scores (accuracy, F1, AUC) are blind to collision. An explanation like "plural nouns" may achieve a high score (e.g., 0.95) for a specific feature, but this score does not distinguish that feature from the other 100 features sharing the label. The scoring mechanism optimizes for generic truth within the cluster rather than specific identification.

4. Proposed Solutions

The paper proposes two complementary corrective metrics to explicitly penalize explanations that fail to distinguish a feature from its neighbors:

Discrimination Scoring ( $S_{disc}$ ):
- Mechanism: For a feature $f$ , a set of $k$ "discrimination features" ( $D(f)$ ) is selected based on activation similarity (e.g., Jaccard similarity). The score measures the ability of the explanation to help a judge distinguish texts where $f$ fires from texts where a neighbor $g \in D(f)$ fires.
- Behavior: An explanation true of both $f$ and $g$ (like "plural nouns") will yield a score near 0.5 (random guessing) when paired against a similar neighbor, whereas a unique explanation yields a high score.
- Cost: Requires $O(km)$ judge calls per feature, significantly higher than standard detection.
Collision-Adjusted Detection ( $S_{adj}$ ):
- Mechanism: A computationally cheaper heuristic that penalizes the standard detection score by the size of the collision cluster: $S_{adj} = S_{det} \cdot \frac{1}{|C_E(f)|}$ .
- Effect: This reduces the apparent quality of broad explanations (e.g., "plural nouns") by a factor of 101 in the observed dataset, without requiring new judge calls.

5. Significance and Claims

The paper positions descriptive collision as a generic failure mode of any explanation-scoring scheme where the target space (explanations) is much smaller than the feature space, invoking Goodhart's Law.

Distinctness: The authors emphasize that collision is structurally distinct from polysemanticity, recall bias, and random-baseline problems. It is a property of the label space, not the feature dictionary.
Impact on Interpretability: Ignoring collision inflates reported feature interpretability by roughly one-third of the bits required to identify a feature.
Downstream Implications: For scientific applications (e.g., feature circuits, steering vectors), conclusions drawn from a label must be read as applying to the entire collision cluster ( $|C_E|$ features), not a single feature. For the "plural nouns" label, this expands the scope of claims by two orders of magnitude.
Conservative Estimate: The authors argue that the observed collision in human-annotated data is a lower bound. Automated pipelines, operating with tighter budgets and narrower descriptive registers, are expected to suffer from even more severe collision.

The paper concludes by calling for the field to routinely report collision statistics alongside detection scores and to develop labeling schemes evaluated by their power to distinguish features, not merely describe them.