Measuring Semantic Abstractness of SAE Features via Nonlocality

Technical Summary: Measuring Semantic Abstractness of SAE Features via Nonlocality

Problem Statement
Sparse Autoencoders (SAEs) have become a primary tool for uncovering mechanistic explanations of Large Language Model (LLM) behaviors, such as reasoning and jailbreaking, by isolating causally effective features. However, a critical gap remains in distinguishing between genuinely high-level semantic features and surface-level, token-driven features (e.g., specific lexical cues or formatting patterns). Current feature selection methods—relying on keyword filtering, contrastive datasets, or LLM-generated auto-interpretations—often fail to resolve the abstraction level of a feature. Consequently, successful behavioral steering via intervention does not necessarily imply a mechanistic understanding of the underlying computation; a feature might simply boost the probability of emitting a specific token (e.g., "Wait") rather than representing a complex concept like "uncertainty." There is a need for an LLM-independent, label-free metric to quantify the "mechanistic complexity" or semantic abstractness of SAE features.

Methodology: Feature Nonlocality (FNL)
The authors introduce Feature Nonlocality (FNL) as a quantitative witness of a feature's contextual reach. Inspired by holographic duality, FNL treats the transformer's token sequence as a "time evolution" and measures the "location" of a feature's activation in this temporal space.

The metric is defined as follows:

1. Per-Position Influence: For a specific SAE feature $a$ activated at the final token position $T$ of a prompt $P$, the authors compute the squared gradient norm of the feature's activation with respect to the input embedding at every preceding position $t$ ($1 \le t \le T$). This measures how much the feature's activation depends on specific input tokens.
   $$J_a(t, T, P) := \left\| \frac{\partial z_a(T, P)}{\partial x_t} \right\|_2^2$$
2. Probability Distribution: These influence scores are normalized to form a probability distribution $p_a^{(P)}(t)$ over the prefix positions.
3. Entropy Calculation: The nonlocality $H(a, P)$ is defined as the Shannon entropy of this distribution.
   $$H(a, P) := -\sum_{t=1}^T p_a^{(P)}(t) \log_2 p_a^{(P)}(t)$$
4. Aggregation: A dataset-level FNL score $H(a, D)$ is obtained by averaging the entropy over a set of firing events.

Intuitively, low FNL indicates that a feature's activation is driven by local, specific tokens (low entropy, concentrated influence), while high FNL indicates that the feature integrates information broadly across the context window (high entropy, diffuse influence).

Key Contributions

1. Definition of FNL: The paper proposes FNL as a label-free, gradient-based metric to characterize the typical contextual reach of an SAE feature.
2. Validation of Stability and Geometry: The authors demonstrate that FNL rankings are stable across different datasets (WikiText, GSM8K, Code-Python) and model depths. They also show that FNL correlates with feature geometry, where features with similar decoder vectors (similar write-side functions) share similar nonlocality ranges.
3. Correlation with Semantic Abstractness: FNL is validated against two independent proxy measures of abstraction:
   • Token Injection Susceptibility: High-FNL features show lower susceptibility to being triggered by isolated token injections (false positives), distinguishing them from token-driven features.
   • Paraphrase Invariance: High-FNL features exhibit greater robustness to meaning-preserving paraphrases (lower false negatives) compared to low-FNL features.
4. Downstream Applications:
   • Jailbreak Mitigation Audit: The authors re-analyzed features used to mitigate wrapper-style jailbreaks. They found that the most effective features had near-zero FNL, activating almost exclusively at the beginning of the sequence (positional indicators) rather than recognizing harmful intent. This suggests current mitigation mechanisms rely on surface-level artifacts rather than semantic understanding.
   • Reasoning Steering: In a proof-of-concept experiment on DeepSeek-R1-Distill-Llama-8B, steering high-FNL features improved MATH-500 accuracy by 4.6 points over the unsteered baseline, outperforming steering low-FNL features and random baselines.

Results

• Correlation Metrics: FNL shows a significant negative correlation with token-injection recovery ($\rho \approx -0.46$) and a positive correlation with paraphrase robustness ($\rho \approx 0.27$).
• Discrimination Power: FNL successfully distinguishes context-dependent features from token-driven ones in 73–84% of randomly drawn pairs (AUC 0.73–0.84).
• Jailbreak Findings: In the audit of 25 jailbreak-mitigation features, 21 were identified as positional (0 FNL), and steering these positional features was responsible for the majority of the defense improvement, whereas steering the 4 content-aware features yielded negligible gains.
• Steering Performance: High-FNL steering on DeepSeek-R1-Distill-Llama-8B achieved a MATH-500 accuracy of 0.911 (avg@4), compared to 0.865 for the unsteered baseline. However, the authors note this gain is model-specific; on other models (Gemma-2-9B, Qwen3-8B), high-FNL steering did not consistently outperform baselines.

Significance and Claims
The paper concludes that FNL provides an LLM-independent, label-free, correlational witness of the abstraction level of an SAE feature. It does not claim to be a perfect predictor of steering utility in all contexts but argues that it offers a necessary diagnostic tool to distinguish between superficial lexical triggers and genuine high-level computations.

The authors emphasize that while steering utility validates control, FNL helps validate understanding by revealing whether a feature represents a complex semantic concept or a simple positional cue. This distinction is crucial for developing robust mechanistic explanations of LLM behaviors and for designing more effective safety interventions that target genuine intent recognition rather than surface artifacts. The work is presented as a step toward fine-grained mechanistic interpretability, with the caveat that the steering results are a proof of concept and not a general guarantee that high nonlocality predicts utility.