🤖 machine learning

Toward Identifiable Sparse Autoencoders

This paper addresses the theoretical instability of sparse autoencoders by proposing minimal architectural and training modifications that yield identifiable sparse autoencoders (iSAEs) with lower reconstruction error and near-identifiable sparse codes, supported by a connection to dictionary learning and the restricted isometry property.

Original authors: Walter Nelson, Theofanis Karaletsos, Francesco Locatello

Published 2026-06-01

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Walter Nelson, Theofanis Karaletsos, Francesco Locatello

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a giant, complex machine (like a modern AI) that speaks in a secret, high-dimensional code. You want to understand what it's thinking, so you build a translator called a Sparse Autoencoder (SAE).

Think of the SAE as a dictionary and a translator.

The Dictionary is a massive list of "concepts" (like "cat," "politics," or "math").
The Translator looks at the AI's secret code and says, "Okay, this specific moment is made of 5% 'cat' and 95% 'math'."

The problem, as this paper points out, is that the current translators are unreliable. If you build two translators from the same instructions and the same data, they might come up with completely different dictionaries. One might call a concept "cat," while the other calls the exact same thing "feline." This makes it impossible to trust what the translator is telling you.

The authors of this paper wanted to fix this. They wanted to build a "Identifiable" SAE (iSAE)—a translator that, no matter how many times you build it, always comes up with the same dictionary and the same translation.

Here is how they did it, using three simple fixes:

1. The "Two-Way Street" Fix (Bidirectional Features)

The Problem: Imagine your dictionary only has words for "Happy" and "Sad," but not "Not Happy." If the AI is feeling "Not Happy," your translator has to awkwardly try to combine "Happy" and "Sad" to describe it, or just ignore it. This wastes half your dictionary's potential.
The Fix: The authors changed the translator so it can handle both positive and negative concepts. Now, instead of just "Happy," it can have "Happy" and "Not Happy" (or "Sad") as distinct entries.
The Result: This doubled the dictionary's capacity without making it bigger. It allowed the translator to describe things much more accurately, like a painter who suddenly got to use the whole color wheel instead of just half of it.

2. The "Tidy Room" Fix (Dictionary Conditioning)

The Problem: Imagine your dictionary is a messy room where "Cat" and "Dog" are so similar that they look almost identical. If you see a picture of a cat, your translator can't decide: "Is this Cat? Or is it Dog?" Because they are so mixed up, the translator might pick "Cat" today and "Dog" tomorrow, even though the picture hasn't changed. In math terms, the dictionary is "ill-conditioned."
The Fix: The authors added a special rule (a "regularizer") during training that forces the dictionary entries to stay distinct. They made sure that any combination of concepts the AI actually uses looks like a unique, non-overlapping pattern.
The Result: This is like organizing the messy room so every item has its own clear spot. Now, when the AI sees a "Cat," the translator knows exactly which box to pick, every single time.

3. The "Smart Translator" Fix (Encoder Expressiveness)

The Problem: Even with a perfect dictionary, the translator might be too "dumb" to figure out how to use it. Imagine you have a perfect map (the dictionary), but your guide (the encoder) only knows how to walk in a straight line. If the path to the destination requires a turn, the guide gets lost.
The Fix: The authors upgraded the guide to be a multi-step thinker. Instead of just looking at the input once and guessing, the guide takes a few "steps" of thought, refining its guess each time, similar to how a human might solve a puzzle by checking and re-checking their work.
The Result: In their computer simulations (synthetic data), this made the translator nearly perfect. It could find the exact right combination of concepts every time. However, in real-world AI data, this smart guide sometimes got confused by the complexity of the real world, suggesting that while the idea is great, the training needs more work.

The Big Picture

The authors combined these three fixes to create two new versions of the translator (called iSAE and iSAE-ME).

What they proved: They showed mathematically that if your dictionary is "tidy" (distinct concepts) and your translator is "smart" enough, the system becomes stable. If you run the experiment 100 times, you get the same dictionary and the same translations 100 times.
What they found in practice:
- On fake/synthetic data, their new model was almost perfectly stable and accurate.
- On real AI data (like language models), the new models were much more stable than the old ones, though not yet perfect. They also reconstructed the AI's thoughts with less error.

In short: The paper argues that to trust an AI translator, it must be stable. By making the dictionary distinct, allowing for "negative" concepts, and using a smarter translation process, they built a translator that is far more reliable and consistent than the ones we used to have.

Technical Summary: Toward Identifiable Sparse Autoencoders

Problem Statement

Sparse Autoencoders (SAEs) have become a standard tool for interpreting and intervening in the internal representations of neural networks, particularly Large Language Models (LLMs). They decompose high-dimensional activations into sparse linear combinations of learned "concepts" (dictionary atoms). However, the paper identifies a critical flaw in current SAE deployments: instability. Empirical folklore and recent theoretical work suggest that different training runs on the same data often yield different concept dictionaries and sparse codes. This non-identifiability undermines the reliability of SAEs for mechanistic interpretability, steering, and scientific analysis, as the "features" extracted may be artifacts of the training process rather than stable structures in the data.

The authors argue that while dictionary learning theory offers conditions for identifiability, standard SAE architectures fail to satisfy them due to three specific characteristics, termed the "dark triad":

Bidirectional Features: Standard activation functions (like ReLU or TopK) restrict coefficients to be positive, forcing the dictionary to learn opposing concepts (e.g., $d$ and $-d$ ) to represent bidirectional signals, which violates conditions for unique recovery.
Dictionary Conditioning: Standard SAEs do not enforce conditions (like Restricted Isometry) on the learned dictionary, leading to scenarios where multiple sparse codes can reconstruct the same input equally well.
Encoder Expressiveness: The near-linear encoders typically used in SAEs may lack the capacity to map observations to the correct sparse codes, even if the dictionary is well-conditioned.

Methodology

The paper proposes a unified approach to bridge the gap between the scalability of SAEs and the theoretical guarantees of dictionary learning. The authors introduce the Identifiable Sparse Autoencoder (iSAE), which modifies the standard TopK SAE architecture and training procedure through three specific interventions:

1. Bidirectional Features (AbsTopK)

To address the restriction to positive coefficients, the authors replace the standard TopK activation with AbsTopK. This function selects the $k$ coefficients with the largest absolute values, allowing for both positive and negative loadings. This effectively doubles the representation capacity of the dictionary without increasing parameter count and aligns the SAE with classical sparse coding formulations that allow negative coefficients.

2. Dictionary Conditioning (aRIP Regularization)

The authors introduce a data-driven relaxation of the Restricted Isometry Property (RIP), termed approximate RIP (aRIP).

Theory: Classical RIP requires that any combination of $k$ atoms preserves norms, a condition that is combinatorially hard to verify for overcomplete, learned dictionaries. The authors propose enforcing this property only on the union of sparsity patterns actually observed in the data.
Implementation: They derive a regularizer based on the eigenvalues of the Gram submatrix of observed supports. This regularizer is added to the standard reconstruction loss to encourage the learned dictionary to satisfy the aRIP condition, thereby ensuring that sparse codes are uniquely recoverable (up to permutation and sign).

3. Encoder Expressiveness (Multi-Step Encoding)

To address the limitation of near-linear encoders, the authors incorporate a multi-step encoding scheme inspired by LISTA (Gregor & LeCun, 2010). Instead of a single forward pass, the encoder iteratively refines the sparse code $z$ over $T$ steps (typically $T=5$ ) using a learned step-size matrix. This increases the expressivity of the mapping from input $x$ to code $z$ , allowing the amortized encoder to better approximate the solution to the sparse coding optimization problem.

Key Contributions

Theoretical Contributions

Definition of Statistical Identifiability: The paper defines SAE identifiability not as recovering a "ground truth" mechanism (which is often unknown), but as run-to-run stability. An SAE is identifiable if independent training runs on the same distribution yield approximately the same dictionary and sparse codes (up to trivial equivariances like permutation and sign).
Impossibility Theorem (Theorem 3.2): The authors prove that learning a stable dictionary is not sufficient for learning stable codes. Even with a perfectly stable dictionary, if the dictionary does not satisfy specific conditioning properties, the sparse coding problem remains non-identifiable.
Near-Identifiability Theorem (Theorem 3.6): They prove that if an SAE achieves low reconstruction error and its dictionary satisfies the aRIP condition (with sufficient richness and diversity in the data distribution), the learned dictionary and sparse codes are near-identifiable.

Empirical Contributions

New Metrics: The authors propose measuring code identifiability directly, using Intersection-over-Union (IoU) and $\ell_2$ error on sparse patterns, in addition to the standard Dictionary Cosine Similarity (DCS). They demonstrate that high DCS does not guarantee high code identifiability.
Model Variants: They present two variants:
- iSAE: Uses AbsTopK + aRIP regularization.
- iSAE-ME: Uses AbsTopK + aRIP regularization + Multi-Step Encoding.

Results

Synthetic Data

Reconstruction: Switching from TopK to AbsTopK reduced reconstruction error by 30–40%.
Identifiability: The iSAE-ME variant achieved near-perfect identifiability in synthetic settings. The multi-step encoder was crucial here, allowing the amortized encoder to match the performance of an "oracle" solver (Orthogonal Matching Pursuit) that has access to the true dictionary.
Conditioning: The aRIP regularizer successfully improved the conditioning of the dictionary, which was necessary for the oracle to achieve high identifiability.

Real-World Data (LLM and Vision)

Reconstruction: On Pythia-160M (LLM) and DINOv2 (Vision) activations, the multi-step encoder (iSAE-ME) achieved the lowest reconstruction error (MSE), significantly outperforming baselines.
Identifiability:
- Dictionary: AbsTopK improved dictionary stability (DCS) compared to TopK.
- Codes: The results were mixed. While the oracle solver showed improved code identifiability with the conditioned dictionary, the amortized SAE encoder did not always fully exploit this. In some real-world cases, the multi-step encoder actually worsened code identifiability despite better reconstruction, suggesting that optimization dynamics in real-world settings are complex and the encoder may not yet be converging to the optimal solution.
Downstream Tasks (SAEBench): On LLM activations, iSAE-ME showed the best performance in minimizing cross-entropy loss (reconstruction fidelity). However, for "Spurious Correlation Removal" (SCR), the TopK baseline performed best, likely because zeroing a TopK feature has a clearer "absence" interpretation. iSAE-ME performed best on Targeted Probe Perturbation (TPP).

Significance and Claims

The paper claims that identifiability is a "bare minimum" requirement for SAEs used in high-stakes applications like model steering and oversight. If two SAEs trained on the same data produce different features, the scientific validity of those features is compromised.

The authors position their work as a step toward making SAEs statistically reliable. They do not claim to have solved the problem of recovering the "true" ground-truth concepts, nor do they claim their models generalize identifiability to distributions outside the training data. Instead, they demonstrate that:

Standard TopK SAEs are inherently unstable and non-identifiable.
By explicitly addressing dictionary conditioning (via aRIP) and encoder expressiveness, one can construct SAEs that are statistically identifiable (stable across runs) and achieve superior reconstruction.
The gap between synthetic and real-world performance highlights that while the theoretical framework holds, optimization challenges in real-world neural network activations remain a significant barrier to perfect identifiability.

The paper concludes that these architectural and regularization changes yield models that are more robust and theoretically grounded, providing a stronger foundation for the interpretability of large-scale neural networks.