Technical Summary: A Geometric Unification of Concept Learning with Concept Cones

Problem Statement

The paper addresses the challenge of interpreting deep neural networks by bridging two dominant but distinct paradigms in concept learning: Concept Bottleneck Models (CBMs) and Sparse Autoencoders (SAEs).

• CBMs are supervised methods that enforce interpretability by aligning model activations with human-labeled concepts. While they provide explicit semantic structure, they rely on predefined taxonomies that may be incomplete or misaligned with the model's internal representations.
• SAEs are unsupervised methods that discover emergent concepts by learning sparse linear representations over an overcomplete dictionary. While scalable, they suffer from non-identifiability: many different dictionaries can reconstruct activations with similar error, making it difficult to determine if learned concepts are meaningful. Furthermore, SAEs are unstable, where minor changes in hyperparameters (sparsity, dictionary size) yield vastly different decompositions.

Currently, there is no principled way to compare these supervised and unsupervised approaches or to evaluate whether an unsupervised SAE has discovered "plausible" human-relevant concepts without a full ground truth.

Methodology: The Concept Cone Framework

The authors propose a unified geometric framework based on dictionary learning and concept cones to analyze both paradigms.

1. Geometric Unification

The paper posits that both CBMs and SAEs solve the same inverse problem: decomposing activation vectors into nonnegative linear combinations of concept directions.

• Dictionary Learning View: Both methods learn a dictionary (rows are concept directions) and codes (coefficients).
• Non-negativity: Both enforce nonnegative codes ($c \geq 0$). In CBMs, this is enforced by the activation function (e.g., sigmoid/ReLU); in SAEs, it is an explicit constraint.
• Concept Cone: The set of all possible activations representable by a dictionary $D$ forms a concept cone, defined as the conic hull of the dictionary atoms:
  $$C_D = \{ v \in \mathbb{R}^d : v = \alpha^\top D, \alpha \in \mathbb{R}^c_+ \}$$
  This geometry unifies the two approaches: they differ not in the structure they learn, but in how they select the specific cone.

2. Partial Anchoring via CBMs

Since SAEs are non-identifiable, the authors use CBMs as partial geometric anchors. The core hypothesis is that a "good" SAE decomposition should contain the concept directions learned by a CBM within its own concept cone. Specifically, a subset of SAE atoms should be able to reconstruct CBM concepts via sparse, nonnegative combinations. This does not require perfect one-to-one alignment but rather that the human-plausible structure exists within the SAE's representational space.

3. Operationalization: Metrics

To quantify the alignment between the SAE cone ($C_{SAE}$) and the CBM cone ($C_{CBM}$), the authors introduce a suite of complementary metrics:

• Coverage: Measures the extent to which CBM concept directions lie within the SAE cone. It is calculated by solving a sparse nonnegative reconstruction problem for each CBM concept against the SAE dictionary. High coverage implies the SAE can represent CBM concepts.
• Geometric Alignment ($\rho_{geom}$): Measures the directional similarity between individual SAE atoms and CBM concepts (using correlation). A top-$k$ version allows for one-to-many or many-to-one mappings.
• Activation Alignment ($\rho_{act}$): Measures whether SAE and CBM concepts activate on similar data samples, accounting for differences in encoders and thresholds.
• Regression Predictability ($R^2$): Fits a linear map from SAE activations to CBM activations to measure global information preservation.
• Sample-level Agreement: Computes precision, recall, and F1 scores based on the top-$k$ active concepts for each sample, mapping SAE indices to CBM concepts.

The authors emphasize that no single metric is sufficient; for instance, high coverage can occur with random dictionaries (artificial inflation), while high geometric alignment does not guarantee meaningful activation patterns.

Key Contributions

1. Unified Framework: Demonstrates that CBMs and SAEs are instances of the same geometric structure (learning a cone in activation space via nonnegative dictionary learning).
2. Partial Alignment Hypothesis: Formalizes the idea that an SAE is plausible if a sparse subset of its atoms can recover independently validated CBM concepts, serving as a sanity check against non-identifiability.
3. Diagnostic Metrics: Introduces a set of metrics (Coverage, Geometric/Activation Alignment, $R^2$, Sample-level Agreement) and provides theoretical clarifications on what each measures and their limitations.
4. Empirical Guidelines: Through extensive benchmarks, identifies the "sweet spots" in SAE design (sparsity and expansion factors) that best recover human-relevant structure.

Experimental Results

The authors evaluated their framework on vision backbones (ResNet-50, ViT, DINOv2) trained on ImageNet and CUB (Caltech-UCSD Birds). They compared various SAE variants (Vanilla, TopK, BatchTopK, JumpReLU, Matching Pursuit, Archetypal).

• Sanity Check: Trained SAEs showed significantly better alignment with CBMs than randomly initialized SAEs. Notably, random SAEs exhibited artificially high Coverage but low alignment and $R^2$, confirming that coverage alone is insufficient and must be paired with other metrics.
• SAE Variants: JumpReLU and Vanilla SAEs generally achieved superior coverage and competitive geometric alignment. Archetypal analysis showed better F1 scores.
• Sparsity Trade-off: There is a tension between sparsity and completeness. Aggressive sparsity (very few active units) yields high geometric alignment (localized prototypes) but lower coverage. Relaxed sparsity improves coverage and reconstruction but disperses structure. An intermediate sparsity regime (approx. 0.01–0.05% active units) provided the best balance.
• Expansion Factor: Increasing the dictionary size (expansion factor) improved coverage and $R^2$ up to a point. A 2x expansion was identified as a sweet spot; beyond this, directional alignment began to diffuse.
• Layer-wise Analysis: In ResNet-50, deeper layers (specifically Layer 4, after the final pooling) showed the strongest semantic alignment with CBM concepts, capturing high-level features. Lower layers captured low-level attributes (edges, colors) that aligned geometrically but lacked semantic correspondence to the CBM taxonomy.

Significance and Claims

The paper claims to provide a principled, geometric foundation for evaluating unsupervised concept discovery. By treating CBMs not as a ground truth but as a "partial semantic anchor," the work offers a structured alternative to existing practices for assessing SAEs.

The authors argue that interpretability does not require choosing between supervision (CBMs) and discovery (SAEs). Instead, the path forward lies in harmonizing the two: using supervised concepts to validate and guide unsupervised discovery within a shared geometric framework. This approach allows researchers to identify inductive biases (such as specific sparsity mechanisms or expansion ratios) that systematically yield more plausible and human-aligned concept dictionaries.

The work concludes that while CBMs are not perfect ground truths (due to human bias and annotation limits), they provide a necessary reference point to distinguish meaningful decompositions from the infinite possibilities allowed by non-identifiability. The proposed metrics serve as diagnostic tools to navigate the trade-offs between expressivity and interpretability in modern neural networks.