🤖 machine learning

The Geometric Wall: Manifold Structure Predicts Layerwise Sparse Autoencoder Scaling Laws

This paper demonstrates that the scaling laws of sparse autoencoders across neural network layers are not universal but are instead determined by the underlying geometric structure of the activation manifold, specifically its curvature and intrinsic dimension, which predictably dictate reconstruction error floors and width exponents in a transferable manner across different models.

Original authors: Eslam Zaher, Maciej Trzaskowski, Quan Nguyen, Fred Roosta

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Eslam Zaher, Maciej Trzaskowski, Quan Nguyen, Fred Roosta

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: Why AI "Translation" Gets Harder in Some Layers

Imagine you are trying to describe a complex, 3D sculpture (the AI's internal thoughts) using only a set of flat, 2D stickers (the AI's dictionary of features). This is what Sparse Autoencoders (SAEs) do. They try to break down the AI's messy internal data into simple, understandable "stickers" (features) that can be reassembled to recreate the original picture.

For a long time, researchers thought this was like a simple math problem: "If I give you more stickers (wider dictionary), you can describe the sculpture better, and the error drops at a predictable rate." This is the Scaling Law.

The Discovery:
The authors of this paper found that this rule doesn't work the same way for every part of the AI. In some layers of the AI, adding more stickers helps a lot. In others, adding more stickers barely helps at all, no matter how many you add.

Why? Because the "sculpture" the AI is trying to describe changes shape as it moves through the AI's layers.

The Core Metaphor: The Curved Wall vs. The Flat Tile

The paper argues that the AI's internal data isn't a flat sheet of paper; it's a curved, bumpy, twisting manifold (like a crumpled piece of paper or a twisted ribbon).

The Flat Tile (The SAE): The SAE tries to cover this curved surface with flat, straight tiles (linear combinations).
The Mismatch: If the surface is perfectly flat, the tiles fit perfectly. But if the surface is curved, the tiles can never fit perfectly. There will always be tiny gaps or overlaps.
The "Geometric Wall": The paper calls this a "Geometric Wall." It's not a limit on how much computing power you have; it's a limit on how well a flat tool can describe a curved object. No matter how many flat tiles you add, you can't perfectly cover a curved surface.

The Two Main Findings

The researchers tested this on two versions of the Gemma AI model (a small 2B version and a larger 9B version) by looking at all 68 layers of their "residual stream" (the main highway where information flows).

1. The Shape of the Data Predicts the Error

They measured four "geometric summaries" of the data in each layer:

Intrinsic Dimension: How many directions the data actually moves in (is it a line, a sheet, or a complex cloud?).
Curvature: How much the data bends or twists.
Tangent Variation: How quickly the direction of the data changes from point to point.
Heterogeneity: How uneven the "roughness" of the data is.

The Result: They found that layers with high curvature and high intrinsic dimension (very bumpy, complex shapes) had a much harder time being described by the SAE.

Analogy: Imagine trying to wrap a gift. A flat box (low curvature) is easy to wrap with paper. A bumpy, twisted rock (high curvature) is impossible to wrap perfectly, no matter how much paper you use. The "error" (the gaps in the wrapping) is higher for the rock.
The Prediction: The authors built a mathematical model showing that if you know the "shape" (geometry) of a layer, you can predict exactly how well the SAE will perform there.

2. The "Floor" is Set by Geometry

In previous studies, researchers noticed that as you add more dictionary features, the error eventually stops dropping and hits a "floor" (a minimum error you can't get below).

The Paper's Claim: This floor isn't random. It is directly tied to the geometry.
The Finding: In layers where the data is very curved or complex, the "floor" is high (lots of error remains). In layers where the data is flatter, the "floor" is low (very little error remains).
The "Geometric Wall": This floor is the "Geometric Wall." It is the irreducible error left behind because a flat dictionary cannot perfectly match a curved reality.

The "Transferable Law"

One of the coolest parts of the paper is that they found a universal rule.

They learned the relationship between "shape" and "error" on the small 2B model.
They then used that exact same rule to predict the error on the larger 9B model.
The Result: It worked perfectly. This suggests that the way AI data curves and twists is a fundamental property of these models, not just a quirk of one specific size. The "geometry of thought" follows the same rules in small and large brains.

Summary in Plain English

Old View: "If we make the AI's dictionary bigger, it will always get better at explaining itself, following one simple rule."
New View: "The AI's internal data is shaped like a complex, twisting mountain range in some places and a flat plain in others. You can't flatten a mountain with a ruler. The 'bumpier' the data is in a specific layer, the harder it is to explain, and the more error will always remain, no matter how big the dictionary gets."

The paper concludes that we have hit a Geometric Wall. We aren't limited by how much money we spend on computing; we are limited by the fact that we are trying to describe a curved world with flat tools. To do better, we might need tools that can bend and curve along with the data, rather than just adding more flat tiles.

Technical Summary: The Geometric Wall: Manifold Structure Predicts Layerwise Sparse Autoencoder Scaling Laws

Problem Statement
Sparse Autoencoders (SAEs) operationalize the Linear Representation Hypothesis (LRH) by reconstructing model activations as sparse linear combinations of dictionary atoms. This approach implicitly assumes that the activation space is well-approximated by a globally linear structure. However, existing scaling laws for SAE reconstruction loss—specifically the joint power law relating loss to dictionary width ( $n$ ) and sparsity ( $k$ )—have been fitted only at single layers (e.g., GPT-4 layer 5/6) and do not account for the sharp variation in reconstruction error observed across different layers of a network.

The paper argues that this variation is not merely a resource limitation but an empirical trace of a geometric mismatch. While SAEs assume a flat, globally linear basis, the activation manifolds of large language models are curved, and their intrinsic dimension varies systematically with depth. Consequently, no single sparse linear dictionary can uniformly match these manifolds, causing the SAE's width-sparsity scaling to become a layer-dependent function of the manifold's geometry rather than a universal law.

Methodology
The authors conduct the first cross-layer scaling study using 844 residual-stream SAE checkpoints from the Gemma Scope family, covering all 26 layers of Gemma 2 2B and all 42 layers of Gemma 2 9B. The analysis proceeds in two stages:

Stage 1: Per-Layer Scaling Surface Fitting
- For each layer, the authors fit a "no-floor" log-linear scaling surface to the reconstruction loss ( $L$ ) as a function of dictionary width ( $n$ ) and sparsity ( $k$ ).
- Due to limited width coverage at most layers (only two backbone widths available), they fit a 4-parameter surface: $\log L = a_0 + \beta_n \log n + \beta_k \log k + \gamma \log n \log k$ .
- From this, they derive the per-layer width-scaling exponent $\alpha_\ell(k) = -(\beta_{n,\ell} + \gamma_\ell \log k)$ .
- At six "showcase" layers with richer width grids ( $\ge3$ widths), they perform a joint non-linear fit of a 6-parameter "with-floor" surface ( $L = A(k)n^{-\alpha(k)} + B(k)$ ) to identify the asymptotic reconstruction floor $B(k)$ .
Stage 2: Cross-Layer Geometric Regression
- The authors compute four layerwise geometric summaries of the activation manifold using extrinsic Euclidean estimators (TWO-NN, multi-scale PCA, etc.):
  - Intrinsic dimension ( $d_{int}$ ): Local degrees of freedom.
  - Multi-scale curvature ( $\kappa_{ms}$ ): Average departure from the tangent plane across varying radii.
  - Tangent variation ( $\kappa_{tv}$ ): Rate of tangent space rotation.
  - Heterogeneity ( $\nu$ ): Local variation in intrinsic dimension.
- They regress the Stage 1 outputs (specifically the width exponent $\alpha_\ell(k)$ and coefficients $\beta_n, \gamma$ ) against these geometric summaries using Ordinary Least Squares (OLS).
- Hypothesis testing compares a geometry-invariant null ( $H_0$ ) against models where the target depends on single features, low-correlation feature pairs, or all four features.
- Cross-Model Transfer: Coefficients learned on one model (e.g., 2B) are used to predict the per-layer exponents of the other model (9B) to test for a transferable geometric law.

Key Contributions

Geometric Framework for SAE Residuals: The paper provides a pullback information-geometric account explaining why activation spaces may not admit globally efficient sparse linear codes. It posits that the "natural" metric on activation space is the pullback Fisher-Rao metric, while SAEs optimize for ambient Euclidean distance, creating a basis-manifold mismatch.
Geometry-Conditioned Scaling Law: The authors extend the standard width-sparsity scaling law to a geometry-conditioned account. They demonstrate that the scaling law's parameters (specifically the width exponent) are layerwise functions of the activation manifold's geometry.
Cross-Model Geometric Law and Floor Coupling:
- They show that regression coefficients learned on one model successfully predict the per-layer width-scaling exponents of another model, indicating a transferable geometric law.
- At showcase layers, the asymptotic reconstruction floor ( $B$ ) tracks the geometric ordering: layers with higher curvature and intrinsic dimension exhibit higher irreducible floors.

Results

Prediction of Scaling Exponents: Manifold geometry strongly predicts the per-layer width-scaling exponent $\alpha_\ell(k=50)$ . Multi-scale curvature ( $\kappa_{ms}$ ) is the dominant single feature, explaining nearly all cross-layer variation in the 2B model and the majority in the 9B model.
Transferability: The geometric law is transferable. Regression coefficients fitted on Gemma 2 2B predict the 9B exponents with high fidelity ( $R^2 \approx 0.92-0.93$ ), and vice versa, suggesting the relationship is not an artifact of a specific model size.
The Geometric Wall: At layers where the asymptotic floor is identifiable, the floor magnitude correlates with geometric complexity. Higher curvature and intrinsic dimension correspond to higher floors. This is interpreted as an irreducible second-order residual: a flat sparse linear dictionary cannot perfectly approximate a curved manifold, leaving a residual error that does not vanish even as dictionary width approaches infinity.
Mechanistic Interpretation: The "geometric wall" is not a finite-resource ceiling but a geometry-dependent limit. Higher curvature increases the second-order departure from the tangent plane, while higher intrinsic dimension reduces the share of local tangent space coverable by a finite atom budget.

Significance and Claims
The paper claims to recast the SAE limit as a geometry-dependent wall rather than a resource ceiling. It argues that the variation in SAE faithfulness across layers is conditioned by the geometric structure of the activations, which current single-layer scaling practices leave unmodelled.

The authors do not claim SAEs are unsuitable for interpretability. Instead, they assert that the layerwise scaling behavior is fundamentally constrained by the manifold structure the SAE attempts to reconstruct. The findings suggest that the "linear representation hypothesis" is a metric-dependent approximation; the "meaningful linear directions" are not metric-free, and the alignment between the ambient Euclidean metric (used by SAEs) and the predictive pullback metric varies systematically with depth. The work provides a diagnostic framework for understanding where and why SAE reconstruction fails, linking it directly to the intrinsic geometry of the neural network's hidden states.