Technical Summary: Structuring Sparsity: Block-Sparse Featurizers

1. Problem Statement

Current interpretability methods, particularly Sparse Autoencoders (SAEs), decompose neural network activations into sparse combinations of isolated directions (atoms). This approach assumes that concepts are represented by single vectors in activation space. However, recent work suggests that neural concepts often possess internal geometry, manifesting as low-dimensional manifolds, convex regions, or dense features rather than isolated directions.

This creates a mismatch between the phenomenology of neural representations (structured, multi-dimensional) and the apparatus used to analyze them (isolated directions). The paper posits that treating concepts as single directions forces a "shattered" view where a single concept is fragmented across many atoms, failing to capture the intrinsic geometry and continuous variation within a concept.

2. Methodology

2.1 Theoretical Foundation: Structured Sparsity

The authors propose Block-Sparse Featurizers (BSFs) to bridge this gap. Drawing from the literature on structured sparsity (specifically group sparsity), they hypothesize that neural activations are generated as a sparse sum of low-dimensional manifolds.

• Data-Generating Process (DGP): An activation $x$ is modeled as $x = \sum_{i \in S} m_i + \epsilon$, where $m_i$ lies on a low-dimensional manifold $M_i$ and $S$ is a small set of active factors.
• Bayesian Derivation: By placing a spike-and-slab prior on blocks of coordinates (where a "slab" is a uniform distribution over a ball in $\mathbb{R}^b$), the Maximum A Posteriori (MAP) estimate naturally leads to a block-sparse penalty ($\ell_{2,0}$ norm). This penalty enforces sparsity across groups of directions while allowing dense variation within an active group.
• Geometric Interpretation: Unlike SAEs which select single directions, BSFs select subspaces. The "atom" is the subspace itself, and the code within the block represents the position on the manifold.

2.2 Three Block-Sparse Featurizer Variants

The paper implements three distinct architectures to instantiate this principle:

1. Vanilla BSF: Uses a linear encoder and decoder. Sparsity is enforced by a hard projection operator ($\Pi_k$) that retains only the $k$ blocks with the largest $\ell_2$ norms.
2. Grassmannian BSF: Enforces that directions within a block are orthonormal (the decoder blocks lie on the Stiefel manifold). The encoder is tied to the decoder transpose ($z = \gamma x D^\top$), and sparsity is enforced via $\Pi_k$. This explicitly models the block as a subspace.
3. Group Lasso BSF: Uses a linear encoder with a soft-thresholding activation function ($sh_\theta$) that shrinks block norms toward zero. Sparsity is induced via an $\ell_{2,1}$ penalty in the loss function, relaxing the hard constraint for a smooth objective.

2.3 Evaluation Framework

• Synthetic Validation: A controlled toy model where ground-truth manifolds (circles, spheres, tori) are embedded and summed. Recovery is measured by how well a single block reconstructs the specific manifold contribution ($R^2$).
• Real-World Analysis: Applied to DINOv3 (vision foundation model), InceptionV1, and SDXL (diffusion model).
• Metrics:
  • Minimum Description Length (MDL): Used to compare BSFs against SAEs. The cost includes bits for the support (which blocks fire), the code (coordinates within blocks), the residual, and the dictionary.
  • Stable Rank: Used to quantify the intrinsic dimensionality of recovered features.
  • Downstream Tasks: Linear probing on classification, segmentation, and depth estimation.
  • Manifold Steering: Intervening in activation space to control image generation.

3. Key Contributions

1. Theoretical Alignment: The paper formally connects the hypothesis that concepts are manifolds to the structured sparsity literature, deriving block sparsity as the matched prior for an additive mixture of manifolds.
2. Algorithmic Implementation: Introduction of three BSF variants (Vanilla, Grassmannian, Group Lasso) that move beyond the "one-direction-per-concept" assumption.
3. Quantitative Evidence:
   • MDL Analysis: BSFs describe activations more compactly than SAEs. The optimal block dimension is found to be moderate (2–4 dimensions), suggesting concepts are inherently multi-dimensional.
   • Synthetic Recovery: BSFs recover additive manifold superpositions with high fidelity ($R^2 \approx 0.93-0.97$), whereas standard SAEs shatter these structures ($R^2 \approx 0.53$).
4. Discovery of Concept Manifolds:
   • InceptionV1: Demonstrated that "curve detector" neurons and SAE features are actually fragments of a single continuous curve manifold. The BSF recovers this as one connected region and reveals higher-order Fourier modes (e.g., 180° invariance) previously obscured.
   • DINOv3: Discovered manifolds corresponding to physical scene attributes like lighting and shadows, which track illumination consistently across different object identities.
   • SDXL: Identified manifolds for specific objects (e.g., pretzels) where steering within the block's subspace produces semantically coherent variations, unlike steering along isolated directions.

4. Results

• Reconstruction Efficiency: At a fixed distortion floor (determined by task performance tolerance), BSFs achieve lower description lengths than SAEs. The most efficient description occurs when block dimensions are between 2 and 4, even when larger capacities are available.
• Feature Dimensionality: Stable rank analysis reveals that recovered concepts in DINOv3 are typically 2–4 dimensional, confirming that blocks are not merely over-parameterized directions but utilize their internal geometry.
• Spatial Coherence: BSF concept maps are significantly smoother (lower Total Variation and Dirichlet Energy) than SAE maps, indicating they track contiguous scene attributes better.
• Manifold Steering: In SDXL, steering along the learned manifold geometry (using Kohonen maps to traverse the subspace) generates images that vary smoothly along the concept's internal dimensions (e.g., changing the shape of a pretzel) without degrading image quality, a feat difficult with isolated SAE directions.

5. Significance and Claims

The paper claims that block sparsity is the correct inductive bias for decomposing neural representations when those representations are structured as low-dimensional manifolds.

• Neuroscience Connection: The work aligns with visual neuroscience models (e.g., the energy model, complex cells) where features are carried by coordinated groups of neurons rather than single tuned units. BSFs recover the "complex cell" behavior by retaining the internal coordinate (phase/geometry) rather than pooling it away for invariance.
• Interpretability Paradigm: The authors argue that a featurizer is a hypothesis about the data-generating process. By matching the featurizer's structure (blocks) to the representation's geometry (manifolds), one can recover richer, more coherent concepts.
• Limitations and Scope: The paper modestly notes that block sparsity is not a universal atom for all modalities (e.g., language or video may require different structures like temporal or hierarchical blocks). The contribution is specific to the case where representations are additive mixtures of low-dimensional manifolds, a property observed in the visual models tested.

In summary, the paper provides a theoretical and empirical case for moving from "direction-based" to "region-based" (block-based) featurizers, demonstrating that this shift recovers the internal geometry of concepts, improves description efficiency, and enables more precise control over generative models.