🤖 machine learning

From Geometric Recovery to Causal Validation: A Reproducible Audit of Sparse Autoencoder Features, from Superposition Geometry to Causal Inertness

This paper introduces a causal audit framework demonstrating that standard correlational recovery metrics for Sparse Autoencoder features are insufficient, as they fail to detect that a significant portion of "recovered" features are causally inert due to geometric artifacts and competitive pathologies, thereby necessitating a shift from geometric alignment validation to causal verification.

Original authors: Mohamed Abdessalem Bal

Published 2026-07-15✓ Author reviewed ⓘ

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mohamed Abdessalem Bal

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you've built a giant, magical library where every book is a thought, but the shelves are too small to hold them all separately. To make everything fit, you start stacking books on top of each other, mixing their spines together. This is what neural networks do: they pack thousands of ideas into a limited space, a trick called superposition.

To read the library, scientists invented a special decoder tool called a Sparse Autoencoder (SAE). Think of it as a high-tech librarian who tries to un-mix the stacked books and pull out the original titles. For years, the standard way to check if the librarian did a good job was to look at the spines. If the spine of the pulled-out book looked 90% similar to the original title, everyone cheered, "Success! We found the feature!"

But this paper, written by an independent researcher named Mohamed, says: "Wait a minute. Just because the spine looks right doesn't mean the book is actually open."

The Great Spine Illusion

The author set up a tiny, perfect library in a computer simulation where they knew exactly which books were hidden and where. They built a "perfect" librarian (a well-trained SAE) and a "messy" one (a degraded SAE).

They found a shocking truth: Looking at the spine (correlation) is not the same as checking if the book is actually being read (causation).

In their simulation, they tested 22 specific books.

In the messy library, they found that 77% of the books that looked like a perfect match (spines matched with a score of 0.90 or higher) were actually dead. The librarian pulled them out, but the book never actually opened. The "feature" was there, but the librarian's tool never fired the switch to read it.
Even in the perfect library, 9% of the matches were dead. And get this: some of these dead matches had spines that were 99.98% identical to the original. The geometry was perfect, but the mechanism was broken.

The paper argues that the field has been fooled by "pretty spines." A tool can point in the exact right direction without ever actually doing the work.

The Two Ways a Librarian Can Fail

The author discovered that this "dead book" problem happens in two very different ways, like two different types of magic tricks gone wrong:

The "Antipodal" Trap (Structural Inertness): Imagine two books, "Fire" and "Ice," that are exact opposites. The library stacks them on the same shelf, but one is upside down. The librarian's tool is designed to only pick up books that are right-side up. So, when the "Fire" book is needed, the tool picks it up perfectly. But when "Ice" is needed, the tool sees the upside-down spine, matches it perfectly, but refuses to pick it up because it's upside down. The tool is geometrically perfect but causally useless for the upside-down book. This happens even in the best libraries.
The "Crowded Shelf" Trap (Competitive Inertness): In the messy library, the shelves are so crowded that when "Fire" is needed, a different, louder book always jumps in front of it and gets picked instead. The tool could pick "Fire," but it never gets the chance. This is a sign the library is poorly organized.

The "Read" vs. "Write" Surprise

Here is the wildest part. The author found that for those upside-down "Ice" books, the librarian was blind but powerful.

Read-Inert: If you ask the librarian to find the "Ice" book by turning off the tool, nothing happens. The tool never turned on for that book in the first place, so turning it off changes nothing. The librarian is blind to the book's presence.
Write-Active: But if you force the tool to grab the upside-down book, it works! You can actually steer the library's output using that upside-down book.

So, a feature can be unmonitorable (you can't see it working) but steerable (you can use it to control the system). The paper calls this a "read-write dissociation." It's like having a remote control that doesn't turn the TV on, but if you hold it down, it can change the channel.

The Real-World Test

The author didn't just stop at the toy library. They took their new "Causal Audit" tool and tested it on a real, published library (a model called GPT-2-small) with 83 different concepts like "beekeeping," "astronomy," and "law."

They found that 14% of the "successful" matches were dead (causally inert).
They also found a strange glitch: a handful of "decoder atoms" (the librarian's tools) kept popping up as the best match for dozens of totally unrelated concepts. One tool was the "best match" for astronomy, law, and cryptography all at once. This suggests the library is trying to use the same shelf for too many different things, just like in the toy simulation.

The Takeaway

The paper concludes that we cannot trust the "spine similarity" score alone. Just because a feature looks like a match doesn't mean it's actually doing the work.

The Rule: If you want to use these tools to monitor or control AI, you have to do a "causal audit." You have to check if the tool actually fires when the concept is present, and if you can actually steer the system with it.
The Warning: In a messy setup, up to 77% of what looks like a success might be a failure. Even in a great setup, 9% might be a failure.

The author built a free, open-source tool called sae-causal-audit to help others check their own libraries. They proved that you can't just look at the geometry; you have to poke the system to see if it really works. It's a reminder that in the world of AI, looking right isn't the same as being right.

Technical Summary: From Geometric Recovery to Causal Validation

Problem Statement
The field of mechanistic interpretability relies heavily on Sparse Autoencoders (SAEs) to decompose superposed neural representations into interpretable features. Current evaluation practices predominantly use correlational recovery metrics, specifically the unsigned cosine similarity between a ground-truth (or probed) feature direction and the best-matching decoder atom. This paper identifies a critical evaluation gap: high cosine similarity measures decoder-geometry alignment (how well the atom reconstructs the feature) but does not guarantee encoder-activation behavior (whether the atom actually fires when the feature is present). The authors argue that these are distinct empirical claims, and a feature can be geometrically perfect yet causally inert.

Methodology
The authors conduct a fully controlled audit using a small, synthetic toy model where ground truth is known and fully controllable. The methodology proceeds in three stages:

Reproduction of Phase Diagrams: They reproduce the superposition phase diagram of Elhage et al. (2022) and the TopK-versus-L1 comparison of Gao et al. (2024). This involves training SAEs on synthetic inputs with varying sparsity and overcompleteness ratios.
Causal Validation Battery: For every feature that passes a standard correlational recovery bar (cosine $\ge$ $\geq$ 0.90), the authors apply two interventions propagated through the model's actual output stage:
- Ablation: Zeroing the matched atom's code entry when the feature is present to measure the drop in the feature's reconstructed output (Read-direction).
- Steering: Forcing the matched atom to a fixed value with the correct sign when the feature is absent to measure the rise in the feature's output (Write-direction).
Instrument Development: The authors package this methodology into sae-causal-audit, a model-agnostic tool that enforces strict protocol typing (requiring encode, decode, and dictionary exposure) and implements a deterministic, hash-verified reproduction pipeline to ensure reproducibility.

Key Contributions and Results

Phase Diagram Artifacts and New Regimes:
- Identified a convergence artifact at high sparsity where under-training masquerades as superposition; resolved by adjusting step budgets.
- Discovered a previously under-described diffuse sharing regime at extreme overcompleteness (ratio 8, sparsity 0.99), where features are substantially represented ( $\|W_i\|_2 \approx 1.40$ ) but exhibit low feature dimensionality ( $D_i \approx 0.08$ ) due to symmetric, multi-way overlap.
L1 vs. TopK and Shrinkage:
- Reproduced the Pareto dominance of TopK SAEs over L1 SAEs.
- Provided direct quantitative evidence of L1 shrinkage: Refining the magnitudes of L1-selected atoms (freezing support) recovered 91.2% of the reconstruction gap and increased mean magnitudes by 22.5%, whereas TopK showed negligible magnitude change. This confirms L1 penalties systematically bias activation strengths toward zero.
Causal Inertness:
- Primary Finding: Correlational recovery does not imply causal efficacy. In a degraded TopK SAE, 77% of features passing the cosine $\ge$ 0.90 bar were causally inert (the matched atom never fired). Even in a well-trained SAE, 9% of matched features were inert, including matches with cosine $\approx$ 1.000.
- Mechanism: Inertness arises because decoder geometry (used for reconstruction) can align perfectly with a feature direction while the encoder's selection nonlinearity (TopK/ReLU) never selects that atom for the feature.
Taxonomy of Causal Inertness:
The authors decompose inertness by cause and direction:
- By Cause:
  - Structural Inertness: Caused by antipodal-pair superposition geometry (two features sharing a direction with opposite signs) under a positive-pass encoder. This occurs in both good and bad SAEs and is stable/predictable.
  - Competitive Inertness: Caused by TopK-selection pathology in degraded dictionaries where an atom barely loses the competition. This is unstable and linked to numerical boundary sensitivity.
- By Direction:
  - Read-Inertness: The atom never fires, making ablation-based monitoring blind.
  - Write-Inertness: Interventions along the atom fail to move the feature.
  - Dissociation: The authors found five antipodal pairs that are read-inert but write-steerable. An atom can be unmonitorable (never fires) yet highly steerable (specificity 143–310) when forced.
Reproducibility as a Stack of Claims:
The paper documents that byte-exact cross-platform reproducibility is unavailable by construction for floating-point training workloads due to compiler and BLAS differences. They propose a two-tier guarantee: SHA-256 byte-exactness within a pinned environment, and semantic equality (tolerance-based) across platforms.
Real-Model Census:
Applying the instrument to a production SAE (GPT-2-small, 83 concepts) reproduced the qualitative pattern: 14% of recovered features were causally inert. Crucially, the audit surfaced atom collisions, where a single decoder atom served as the nearest match for dozens of semantically unrelated concepts across independent batches, suggesting dictionary under-splitting in real models.

Significance and Claims
The paper claims that the field's reliance on correlational metrics creates a "geometrically plausible, causally misleading" space of solutions. The significance lies in:

Redefining Validation: Establishing that a feature is not "recovered" until it passes causal intervention tests (ablation and steering), not just geometric alignment.
Instrumentalization: Providing sae-causal-audit as a tool to detect these failures, specifically highlighting that "perfect" cosine matches can be causally useless.
Structural Insight: Demonstrating that antipodal superposition creates a systematic failure mode (structural inertness) that persists even in high-quality SAEs, necessitating sign-aware matching and direction-specific validation (read vs. write).
Reproducibility Standards: Arguing that reproducibility must be stated as a stack of claims with explicit scopes rather than a binary "fully reproducible" badge.

The authors maintain modesty regarding absolute numbers, noting that the toy setting serves as a calibration point for the "easy case" and that real-model rates are not directly transferable, though the qualitative divergence between correlation and causation is robust.