💬 NLP

Sparse Autoencoders Encode Both Concepts and Functions: The Downstream Geometry of Feature Effects

This paper introduces Feature-Effect Geometry Analysis (FEGA) to demonstrate that while sparse autoencoder features can be causally relevant, they rarely function as stable steering directions, instead exhibiting distinct geometric patterns where "value-like" features produce structured effects and "pointer-like" features generate diffuse, context-dependent changes.

Original authors: Phu Gia Hoang, Anwoy Chatterjee, Tanmoy Chakraborty, Iryna Gurevych, Subhabrata Dutta

Published 2026-07-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Phu Gia Hoang, Anwoy Chatterjee, Tanmoy Chakraborty, Iryna Gurevych, Subhabrata Dutta

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to understand a giant, magical library where books write themselves. This library is a "Large Language Model," a type of artificial intelligence that has read almost everything on the internet and learned to predict the next word in a sentence. Inside this library, there are millions of tiny, invisible switches called "neurons" that light up when the model thinks about something. For a long time, scientists tried to understand the model by just watching which switches lit up. They thought, "If a switch lights up when the model talks about 'Paris,' then that switch must be the 'Paris switch.'"

But here's the tricky part: just because a switch lights up doesn't mean it's actually doing the work. It might just be watching. To really know what a switch does, you have to reach in, flip it off, and see if the story changes. This is called "intervention." Recently, scientists built a special tool called a "Sparse Autoencoder" (or SAE) to help them find these switches. Think of an SAE as a super-organized filing cabinet that sorts the messy, glowing switches into neat, labeled folders. The hope was that if we find a folder labeled "Paris," we could just turn that folder on or off to make the model talk about Paris or stop talking about it. This idea is called "steering."

However, there was a nagging doubt. Sometimes, when scientists turned a "Paris" folder on, the model didn't talk about Paris at all. Sometimes it talked about France, sometimes it got confused, and sometimes it did the exact opposite of what was expected. It was like having a remote control where the "Volume Up" button sometimes turned the TV off, or changed the channel instead. The big question was: Why does this happen? Is the remote broken, or is our understanding of how the buttons work wrong?

This paper, titled "Sparse Autoencoders Encode Both Concepts and Functions," goes on a detective mission to solve this mystery. The authors, a team of researchers from Germany and India, decided to stop just looking at the labels on the folders and start watching what happens to the story after they flip a switch. They invented a new method called "Feature-Effect Geometry Analysis" (FEGA). Imagine FEGA as a high-tech camera that takes a snapshot of the library every time they flip a switch, not just once, but in thousands of different stories. They then look at the "cloud" of all these snapshots to see if the switch always pushes the story in the same direction.

What they found was a surprise. They discovered that the switches inside the model fall into two very different categories, which they named "Value-like" and "Pointer-like."

First, there are the Value-like switches. These are like the "facts" in the library. If a switch is tied to the fact that "Paris is in France," it acts like a static piece of information. When you flip this switch, the model's output changes in a somewhat organized way, like a group of people walking in a loose formation. It's not a single straight line, but it's not a total mess either. These switches are reliable for holding specific facts.

Then, there are the Pointer-like switches. These are the real troublemakers for the "remote control" idea. These switches don't hold a specific fact; instead, they hold a job or a function. Think of them like a "Copy" button on a keyboard. A "Copy" button doesn't care what you are copying; it just does the copying. In the model, a pointer-like switch might be responsible for "copying the word from earlier in the sentence." If the sentence says "The cat is big," the switch copies "big." If the sentence says "The cat is small," the switch copies "small." Because the job is to copy whatever is there, flipping this switch doesn't push the story in one single direction. Instead, the effect scatters everywhere, like a cloud of dust.

The paper shows that for these "Pointer-like" switches, the old idea of "steering" is mostly wrong. You can't just turn a "Copy" switch on and expect the model to always say "Copy." The effect depends entirely on what the model is looking at at that exact moment. The authors found that while some switches (the Value-like ones) have a somewhat structured effect, most of the interesting, functional switches (the Pointer-like ones) create a "diffuse" effect. They are essential for the model to work—they are the reason the model can follow rules or copy words—but they don't point in a single, predictable direction.

In short, the paper suggests that we need to change how we think about these AI switches. We can't treat them all like simple on/off buttons for specific topics. Some are like filing cabinets for facts, but others are like dynamic tools that change their effect based on the situation. If we want to control these AI models in the future, we can't just look for a single "Paris" button or a single "Copy" button and hope it works the same way every time. We have to understand that for many of these tools, the effect is a complex cloud that changes shape depending on the story being told. The authors don't claim to have fixed the problem yet, but they have provided a new map that shows us exactly where the confusion lies, proving that the "remote control" for AI is much more complicated than we thought.

Technical Summary: Sparse Autoencoders Encode Both Concepts and Functions: The Downstream Geometry of Feature Effects

1. Problem Statement

The widespread adoption of Sparse Autoencoders (SAEs) for mechanistic interpretability is currently limited by an inconsistent link between SAE features and model behavior. While features often possess clear activation descriptions (e.g., "Paris" or "copying"), their causal effects are frequently unstable. Prior work has demonstrated that steering effects can vary across prompts, oppose intended directions, or fail to produce desired output changes despite strong activation.

Existing interpretability workflows rely on activation evidence (high-activation examples, automated descriptions, decoder readouts) to associate features with concepts. However, activation alone does not establish causal contribution. A critical gap exists in understanding the geometry of changes in model logits caused by feature interventions. Specifically, it remains unclear whether a feature's downstream effect constitutes a stable, reusable direction (a "steering vector") or if it exhibits complex, context-dependent geometric structures. This paper addresses the question: How comparable are a feature's downstream effects when that feature is intervened upon across different contexts?

2. Methodology

2.1 Feature-Effect Geometry Analysis (FEGA)

The authors introduce Feature-Effect Geometry Analysis (FEGA), an unsupervised, offline framework designed to measure and classify the geometry of downstream logit changes resulting from feature ablation.

Intervention Protocol:

Reconstruction-Relative Baseline: To isolate the effect of a specific feature $j$ $j$ from SAE reconstruction error, the method compares two paths:
- Baseline: The SAE reconstruction $\bar{h}^{(c)}_\ell = D(E(h^{(c)}_\ell))$ .
- Intervention: The reconstruction with feature $j$ zeroed: $\hat{h}^{(c)}_{\ell,j\downarrow} = D(z^{(c)} - z^{(c)}_j e_j)$ .
Downstream Propagation: Both paths are propagated through the frozen model tail (subsequent layers) to the final logit readout $\phi(\cdot)$ .
Effect Cloud: The logit difference $\Delta^{(c)}_j = \phi(\hat{h}^{(c)}_{\ell,j\downarrow}) - \phi(\bar{h}^{(c)}_\ell)$ is computed for a set of valid contexts $C$ . The collection of these differences forms the feature-effect cloud $E_j$ .

Geometric Diagnostics:
FEGA analyzes the shape of $E_j$ in logit space using a series of diagnostics based on the normalized direction vectors $v^{(i)}_j$ :

Directed-Ray Concentration ( $C_{ray}$ ): Tests if all effects point in the same signed direction.
Axis or Antipodal Structure ( $B_{axis}$ ): Tests if effects align with a single axis but split between positive and negative signs (e.g., $u$ and $-u$ ).
Directional Mixture: Uses a von Mises-Fisher mixture model to detect multiple distinct, coherent directional modes.
Global Span Spectrum: Uses PCA on the kernel matrix to determine if effects lie within a low-dimensional subspace (e.g., 2D or 3D).
Centered Residual Spectrum: Analyzes the variation around the mean direction to detect low-dimensional deviations.

The framework assigns a primary geometry label (e.g., "Directed Ray," "Axis," "Diffuse") and secondary flags (e.g., magnitude heterogeneity) based on these diagnostics.

2.2 Experimental Setup

Model: Gemma-2-2B.
SAE Variants: ReLU, TopK, and Matryoshka Batch TopK (width $\approx$ 65k) applied to the post-layer-12 residual stream.
Task Spectrum:
- Value-Like (Factual): RAVEL (City-Country attribute disentanglement). Features here represent static information (e.g., "Paris is in France").
- Pointer-Like (Operational): Literal Sequence Copying (LSC), Word Content (WC), PrOntoQA (rule following), and Token Translation (TT). These tasks require context-dependent operations (copying, binding, rule application) without fixed token meanings.
Feature Isolation:
- Value-like: Selected via SAEBench's differential binary masking for attribute control.
- Pointer-like: Selected based on high activation frequency across prompt families in ICL tasks, followed by ablation to verify causal contribution.

3. Key Results

3.1 Feature Effects Are Rarely Single Directions

Across all SAE variants and tasks, downstream feature effects rarely collapse to a single, context-invariant logit vector. The hypothesis that features act as reusable steering directions is the exception rather than the rule. Most effect clouds do not exhibit high directed-ray concentration.

3.2 Distinction Between Value-Like and Pointer-Like Features

The paper distinguishes two functional roles with distinct geometric properties:

Pointer-Like Features: Associated with prompt-local operations (copying, rule-following).
- Recurrence: A small, compact set of features recurs across different ICL tasks (e.g., LSC, WC, PrOntoQA), particularly in TopK and Matryoshka Batch TopK architectures.
- Geometry: These features predominantly exhibit diffuse effects. Even when they recur across tasks, they do not act as reusable logit directions; their effects scatter across the output space, consistent with the target changing based on the specific value being copied or translated.
- Causality: Joint ablation of these selected sets causes significant accuracy drops, confirming their causal role despite the lack of a stable geometric direction.
Value-Like Features: Associated with static factual attributes (e.g., RAVEL City-Country).
- Geometry: These features exhibit more structured, low-dimensional effects compared to pointer-like features.
- Structure: However, even these structured effects typically span multiple output directions rather than collapsing to a single steering vector.

3.3 Disjoint Feature Sets

The feature sets identified for pointer-like tasks (ICL) and value-like tasks (RAVEL) are disjoint. No features selected for ICL tasks appear in the RAVEL factual-attribute sets, suggesting that the SAE dictionary separates these functional roles, even if their geometric manifestations are complex.

4. Contributions and Significance

Primary Contribution:
The paper introduces FEGA, a framework that shifts the focus of interpretability from activation description to intervention effect geometry. It provides a computable diagnostic for how feature effects are organized, distinguishing between reusable directions, structured low-dimensional spaces, and diffuse clouds.

Significance Claims:

Challenging the Steering Assumption: The results challenge the common assumption that a feature with a clear description and causal effect can serve as a reliable, single-vector control variable. A feature can be interpretable and causally important without providing a stable direction for steering.
Refining Feature Roles: By distinguishing between "value-like" (static information) and "pointer-like" (context-dependent operations) features, the paper explains why steering behavior varies. Pointer-like features, which support operations rather than static concepts, naturally produce diffuse, context-dependent effects.
Methodological Shift: The work argues that feature-level interpretability must distinguish three aspects: what a feature detects, whether it matters causally, and how its effects vary across contexts. FEGA provides the "effect-side audit" necessary to evaluate the third aspect.

Modesty of Claims:
The authors explicitly state that their findings suggest stability in logit-space direction is "often limited." They do not claim that SAE features are useless for steering, but rather that the expectation of a single, reusable steering vector is often incorrect. The paper does not propose new steering algorithms but rather provides a diagnostic tool to understand the limitations of current steering approaches. The geometric labels (e.g., "Diffuse," "Axis") are described as descriptive summaries of the effect cloud rather than mechanistic explanations of the underlying neural computation.