Beyond Semantics: Disentangling Information Scope in Sparse Autoencoders for CLIP

1. Problem Statement

While Sparse Autoencoders (SAEs) have become a standard tool for interpreting internal representations in Vision Transformers (ViTs) like CLIP, existing research focuses almost exclusively on the semantic meaning of individual features (e.g., "this feature detects a dog").

The authors identify a critical limitation in this concept-centric approach: knowing what a feature represents does not explain how it aggregates visual evidence. A feature labeled "dog" might rely on localized texture cues (fur) or integrate global scene context. As SAE dictionaries grow and features become finer-grained, semantic labels alone are insufficient to characterize functional roles.

The paper addresses the lack of a metric to quantify the Information Scope of SAE features—specifically, the spatial extent over which a feature aggregates evidence (ranging from localized patch-specific cues to global image-level signals).

2. Methodology

The proposed framework introduces a new interpretability dimension based on Contextual Dependency, quantified via a novel metric called the Contextual Dependency Score (CDS).

A. Theoretical Foundation: Outlier Tokens and Spatial Instability

The authors build on the observation of "outlier tokens" in ViTs—patch tokens with exceptionally high norms that often aggregate global information. They hypothesize that these tokens are highly sensitive to context.

• Shifted Context Cropping (SCC): To test this, they generate two overlapping crops ($I_1$ and $I_2$) from a single image, shifted by a single patch stride ($s=1$).
• Observation: While the pixel content of the overlapping region is identical, the absolute positional embeddings and the surrounding non-overlapping context change.
• Result: Outlier tokens exhibit spatial instability; their locations shift unpredictably between $I_1$ and $I_2$, whereas non-outlier tokens remain spatially stable. This suggests global signals are context-dependent, while local signals are content-anchored.

B. The Contextual Dependency Score (CDS)

To move beyond binary outlier/non-outlier classification, the authors propose CDS to measure the stability of every SAE feature across the continuous spectrum.

1. Selection: Identify the top $k_{CDS}$ images where a specific feature $f_j$ is most active.
2. SCC Application: Apply the SCC method to these images to generate pairs ($I_1, I_2$).
3. Activation Extraction: Pass both crops through the ViT and the trained SAE encoder to extract activation maps for feature $f_j$ on the overlapping grid.
4. Metric Calculation: Compute the Earth Mover's Distance (EMD) between the normalized activation maps of $I_1$ and $I_2$.
   • Low CDS: The feature's activation pattern remains stable despite the shift (indicates Local Scope).
   • High CDS: The feature's activation pattern shifts significantly (indicates Global Scope).
5. Normalization: The EMD is normalized by the grid diagonal distance to ensure the metric is architecture-agnostic.

C. Feature Partitioning

Using the CDS distribution, the authors partition SAE features into two groups:

• Low-CDS Group: Features with stable spatial activation (Local Scope).
• High-CDS Group: Features with unstable spatial activation (Global Scope).

3. Key Contributions

1. New Interpretability Axis: Introduces Information Scope as a critical dimension for understanding SAE features, complementing semantic identity.
2. Contextual Dependency Score (CDS): Proposes a quantitative metric to distinguish between features that rely on local content versus those dependent on global context.
3. Functional Disentanglement: Demonstrates that CDS-based partitioning reveals distinct functional roles for features that semantic analysis alone cannot distinguish.
4. Generalizability: Validates that this phenomenon (outlier instability and CDS separation) holds across multiple ViT architectures (CLIP, DINOv2, SigLIP2, DeiT3) and scales.

4. Experimental Results

The authors validated their findings using linear probing on three downstream tasks: ImageNet classification, ADE20K semantic segmentation, and NYUd monocular depth estimation.

A. Correlation with Outlier Tokens

• High-CDS features are strongly activated by outlier tokens (global aggregators).
• Low-CDS features are strongly activated by non-outlier tokens (local patches).
• This confirms that outlier tokens are structurally composed of features with high contextual dependency.

B. Feature-Group Removal Analysis

By removing specific feature groups from CLIP embeddings and re-training linear probes:

• Image Classification:
  • Removing High-CDS features consistently improved accuracy. This suggests that for linear classifiers, global context features can introduce noise or redundancy that hinders performance when mixed with local features.
  • Removing Low-CDS features generally degraded performance (especially in smaller models), indicating these features carry essential discriminative signals.
• Dense Prediction (Segmentation & Depth):
  • Removing Low-CDS features caused severe performance degradation. This proves that local, patch-level information is critical for pixel-level tasks.
  • Removing High-CDS features had minimal impact or slight improvements, suggesting global context is complementary but not strictly necessary for dense prediction when local cues are preserved.

C. Qualitative Analysis

• Low-CDS features exhibit sharp, spatially grounded activation maps (e.g., specific object parts).
• High-CDS features show diffuse, ambiguous activation patterns that do not localize to specific regions, reflecting broader scene context.

5. Significance

This work fundamentally shifts the paradigm of SAE interpretability in vision models:

• Beyond "What": It moves the question from "What concept does this feature represent?" to "How does this feature integrate information?"
• Diagnostic Tool: CDS provides a robust diagnostic for understanding model behavior, explaining why certain features (outliers) are unstable and how different feature groups contribute to different tasks.
• Model Design Implications: The findings suggest that for tasks requiring fine-grained spatial reasoning (like segmentation), local features are paramount, while global features may need to be handled differently (e.g., via CLS tokens) to avoid interfering with linear probes.
• Generalizability: The discovery that information scope is a universal property across diverse ViT architectures (supervised, self-supervised, and vision-language) highlights a fundamental structural characteristic of transformer-based vision models.

In summary, the paper establishes that Information Scope is a critical, previously overlooked axis for interpreting CLIP and ViT representations, offering a more nuanced and functionally grounded understanding of how these models process visual data.