🤖 machine learning

Evidence for feature-specific error correction in LLMs

This paper provides empirical evidence that large language models utilize feature-specific error correction by demonstrating that their residual-stream activations are more robust to perturbations along mixed directions than along privileged "pure" feature directions, a finding consistent with a super- $L^p$ -norm ( $p>2$ ) error correction mechanism that holds across multiple model architectures.

Original authors: Francisco Ferreira da Silva, Stefan Heimersheim

Published 2026-06-25

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Francisco Ferreira da Silva, Stefan Heimersheim

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a Large Language Model (LLM) as a massive, high-tech library where every book represents a piece of knowledge. However, this library has a strange problem: it has far more books (concepts like "wealth," "gender," or "programming") than it has shelves (dimensions) to put them on.

To solve this, the library uses a trick called superposition. It stacks multiple books on top of each other on the same shelf, hoping they don't get too messy. The big question researchers have asked is: How does the library keep these stacked books from getting mixed up and corrupted?

This paper argues that the library uses a specific kind of noise-canceling system (called "feature-specific error correction") that treats the "real books" differently than random noise. Here is how they proved it, using simple analogies.

The Experiment: Pushing the Shelves

The researchers decided to test the library's stability by gently pushing the books on the shelves.

The "Plateau" Effect: They found that if you push a shelf just a tiny bit, nothing happens. The books don't fall off, and the story the model tells doesn't change. This is like a "flat plateau" on a hill; you can walk a little way without going up or down.
The Direction Matters: They discovered that this "pushing" works differently depending on which way you push.
- If you push in a random direction (like shoving the shelf sideways), the shelf is very sturdy. You have to push really hard to make it move.
- If you push in a specific direction (like pushing directly toward the "Wealth" book or the "Gender" book), the shelf is surprisingly sensitive. It moves much easier.

The "Superellipse" Test

To measure exactly how this sensitivity works, they created a mathematical shape called a superellipse (think of it as a shape between a perfect circle and a square).

The Circle (p = 2): If the library treated all directions equally, the shape of the "pushing limit" would be a perfect circle. It wouldn't matter if you pushed straight at a book or at a 45-degree angle between two books; the effort required would be the same.
The Square (p > 2): If the library only cares about the specific books and ignores the space between them, the shape becomes more like a square. You can push hard in the "between" spaces without moving anything, but the moment you aim directly at a book, it moves.

The Finding:
When the researchers tested directions they knew were important (like "Wealth," "Gender," or "Programming Languages"), the shape looked like a square (specifically, a value of roughly 2.3).
When they tested random directions or directions found by chance, the shape looked like a circle (a value of 2.0).

What This Means

This result suggests that the model has a built-in "error correction" system. It is designed to be hyper-sensitive to specific concepts (the "pure" directions) while ignoring the noise that exists in the mixtures of those concepts.

Think of it like a security system in a house:

Random Noise: If someone bumps the wall or the floor (random direction), the alarm doesn't go off. The house is robust.
Specific Triggers: If someone touches the specific "Wealth" button or the "Gender" button, the alarm goes off immediately.
The Mix: If someone tries to press both buttons at the exact same time with half-strength, the alarm is less likely to go off than if they just pressed one button hard.

The paper shows that the model is built to protect its specific "features" from being drowned out by the noise of superposition.

The "Toy Model" Proof

To be absolutely sure their math wasn't just a fluke, the researchers built a tiny, simplified computer model (a "toy model") where they knew exactly how the "books" were stacked.

When they tested this toy model with the real books, it showed the "square" shape (p > 2).
When they rotated their test to aim at the wrong spots, the shape turned back into a circle (p = 2).

This confirmed that their method works: if a system is doing this kind of error correction, it must show this specific mathematical signature.

Summary

The paper provides the first real-world evidence that Large Language Models aren't just randomly jumbling concepts together. They have a sophisticated mechanism that allows them to hold many ideas in the same space while keeping them distinct. They do this by being extra sensitive to specific concepts and extra resistant to random noise, effectively "correcting errors" before they can mess up the model's thinking.

They tested this on six different major AI models (including Gemma, Llama, and Mistral) and found the same pattern in all of them.

Technical Summary: Evidence for Feature-Specific Error Correction in LLMs

Problem Statement

The internal representations of Large Language Models (LLMs) remain poorly understood. A prevailing hypothesis is that LLMs utilize superposition to represent more features than they have dimensions, and potentially perform computation in superposition (CiS). Theoretical work (Hänni et al., 2024) predicts that for CiS to function, the network must implement error correction that privileges specific "feature directions" over generic ones to suppress interference noise. However, this prediction lacks empirical validation. Without ground-truth feature directions, it is difficult to test whether LLMs treat certain directions as privileged. This paper proposes an empirical test to determine if LLMs exhibit Feature-Specific Error Correction (FSEC)—a property where the model is less sensitive to perturbations along non-feature directions than along feature directions.

Methodology

The authors propose a perturbation-based empirical test to measure the geometry of activation plateaus (regions where small perturbations yield negligible downstream response).

Perturbation Setup:
- Residual-stream activations at an early layer ( $\ell=2$ ) are perturbed by rotating the activation vector toward a target direction $d$ while maintaining the vector's norm (norm-matched perturbation).
- The downstream response is measured at the second-to-last layer as the $L_2$ distance between perturbed and unperturbed residual streams.
- The plateau-breaking angle ( $\alpha$ ) is defined as the smallest perturbation angle required to exceed a specific response threshold.
Direction Types:
- Candidate Feature Directions: Constructed via contrastive prompts (e.g., gender, sentiment, languages), MELBO (unsupervised optimization for downstream response), and Sparse Autoencoder (SAE) decoder columns.
- Baseline Directions: Random isotropic directions, PCA directions, and random-difference directions (differences of unpaired activations).
The Superellipse Test:
- The authors perturb activations along mixtures of two directions ( $d_1, d_2$ ) by varying a mixing angle $\phi$ .
- They model the plateau-breaking angle $\alpha(\phi)$ as a function of the projections onto the two directions.
- The response is modeled as an $L_p$ -norm of the decomposition: $R(v) = F(\sum w_i |\hat{f}_i^\top v|^p)$ .
- Theoretical Prediction:
  - If the model responds only to the magnitude of the perturbation (isotropic), the boundary is an ellipse ( $p=2$ ).
  - If the model privileges individual feature directions (FSEC), the boundary bulges outward, forming a superellipse with $p > 2$ . This indicates that a mixture of features requires a larger perturbation to break the plateau than a pure feature direction, implying the network suppresses interference from mixed directions.

Key Contributions

Proposal of FSEC as a Test: The authors formalize FSEC as a test for computation in superposition, modeling the LLM's response to perturbations via the exponent $p$ of the $L_p$ -norm decomposition.
Empirical Evidence: They provide the first empirical evidence that candidate feature directions are privileged over their mixtures.
Cross-Model Replication: The findings are replicated across six diverse LLMs (Gemma-2-9B, Qwen3-1.7B, Llama-3.1-8B, Mistral-7B-v0.3, Aya-Expanse-8B, and Yi-1.5-9B).
Toy Model Validation: The methodology is validated on a toy denoising network with known ground-truth features, confirming that $p > 2$ is recovered for true features and degrades to $p \approx 2$ as directions are rotated away from them.

Results

Contrastive Directions: Across 318 pairs of contrastive directions (covering concepts like wealth, gender, and languages), the fitted superellipse exponent is consistently $p > 2$ (mean $p \approx 2.42$ , median $2.30$).
MELBO and SAE Directions: These candidate feature directions also yield $p > 2$ (MELBO mean $2.21$; SAE mean $2.28$), though slightly lower than contrastive directions.
Baselines: PCA, random, and random-difference directions all cluster around $p \approx 2$ (means ranging from $2.03$ to $2.05$), indicating they are not privileged.
Rotation Analysis: When contrastive directions are rotated toward random orthogonal directions, the exponent $p$ decreases monotonically from $\approx 2.4$ to $\approx 2.0$ , demonstrating that $p$ tracks alignment with feature directions.
Robustness: The results hold across various ablations, including different perturbation layers, measurement layers, response metrics (cosine distance, KL-divergence), and token positions.

Significance and Claims

The paper claims to provide the first empirical evidence supporting the hypothesis that LLMs implement feature-specific error correction, a necessary condition for computation in superposition.

Interpretation of $p > 2$ : The authors argue that $p=2$ (a quadratic form) cannot privilege a large number of feature directions over others due to the limited degrees of freedom in the model's dimension. In contrast, $p > 2$ allows the network to be highly sensitive to specific feature directions while suppressing interference from their mixtures.
Lower Bound: The authors modestly note that their measured $p \approx 2.3$ for LLMs is likely a lower bound. Since their candidate directions (contrastive, MELBO, SAE) are imperfect approximations of the true ground-truth features, the misalignment dilutes the directional sensitivity, biasing $p$ toward 2. Their toy model experiments support this, showing that even moderate misalignment reduces $p$ from the ground-truth range ( $3\text{--}3.4$ ) down to the observed LLM range.
Limitations: The paper explicitly states that while $p > 2$ is a necessary condition for FSEC, it is not sufficient to prove a direction is a feature (preliminary experiments suggest maximizing $p$ alone does not recover features). Furthermore, the existence of activation plateaus does not definitively prove error correction; they could stem from other robustness mechanisms. However, the differential treatment of directions ( $p > 2$ for features vs. $p \approx 2$ for baselines) remains a robust observation of preferential treatment.

In conclusion, the paper establishes a quantitative metric ( $p$ ) that distinguishes candidate feature directions from non-features, offering strong empirical support for the theoretical framework of superposition and error correction in LLMs.