Technical Summary: Feature Rivalry in Sparse Autoencoder Representations

Problem Statement

While Sparse Autoencoders (SAEs) have become a standard tool for decomposing Large Language Model (LLM) representations into interpretable, monosemantic features, the mechanisms governing how these features interact under conditions of uncertainty remain poorly understood. Specifically, it is unclear whether the competition between multiple simultaneously activating features serves as a mechanistic signature of model uncertainty. This paper investigates Feature Rivalry—defined as the presence of strongly negatively correlated SAE feature pairs at a given layer—as a potential indicator of ambiguity in model processing.

Methodology

The study utilizes Gemma-2-2B and Gemma Scope SAEs (width 16k) across 13 layers (0–24, step 2). The experimental design relies on the PopQA dataset, employing a within-domain entropy split to isolate model uncertainty from dataset-level confounds.

1. Data Preparation: The authors sampled 20 completions per question at temperature 1.0 to compute normalized Shannon entropy ($H(q)$). Questions were split into two conditions:

   • Ambiguous (High Entropy): $H(q) > 0.7$ (200 questions).
   • Unambiguous (Low Entropy): $H(q) < 0.5$ (200 questions).
   • This design ensures domain, format, and answer types remain constant, attributing observed differences solely to model uncertainty.

2. Experiment 1: Rivalry Localization:

   • Metric: The "Rivalry Score" is defined as the 5th percentile of all pairwise Pearson correlations of feature activations across prompts at a specific layer. More negative values indicate stronger rivalry.
   • Analysis: Mann-Whitney U tests compared rivalry distributions between ambiguous and unambiguous conditions across layers.

3. Experiment 2: Causal Intervention (Activation Steering):

   • Setup: The top 20 rival feature pairs were identified. A "rivalry axis" vector ($v_{rivalry}$) was constructed by subtracting the decoder vectors of the two rival features ($W_{dec}[A] - W_{dec}[B]$).
   • Intervention: During generation at layer 10, this vector was added to the last token's hidden state.
   • Comparison: The "flip rate" (fraction of outputs changing) for rivalry-axis steering was compared against a random direction baseline at steering multipliers of 5, 10, and 20.

4. Experiment 3: Predictive Utility:

   • Metric: A per-prompt rivalry score was calculated at layer 10 using the 5th percentile of cosine similarities between the decoder vectors of the top-50 active features.
   • Evaluation: This score was tested as a predictor of answer correctness (binary: ground truth present in output) and compared against softmax confidence (probability of the top token) using AUROC.

Key Results

1. Localization of Uncertainty

Feature rivalry is significantly stronger for ambiguous questions compared to unambiguous ones, but this effect is not uniform across the network.

• Layer 0: Highly significant difference ($p = 5.3 \times 10^{-26}$), suggesting uncertainty manifests as feature competition immediately upon initial encoding.
• Layer 12: A second significant peak ($p = 5.8 \times 10^{-5}$), corresponding to mid-level semantic integration.
• Other layers showed no significant differences after Bonferroni correction.

2. Causal Influence

Activation steering along the rivalry axis produced a measurable causal effect on model outputs.

• At a low steering multiplier (5), rivalry-axis steering resulted in higher output flip rates than random steering in 15 of 20 rival pairs (75%).
• The mean flip rate gap was +0.06 (Rivalry: 0.20 vs. Random: 0.14).
• The effect diminished at higher multipliers (10 and 20), where perturbations became too large and indiscriminate.

3. Predictive Performance

The per-prompt rivalry score serves as a valid, though imperfect, uncertainty signal.

• AUROC for Correctness Prediction: The rivalry score achieved 0.689, significantly outperforming random chance (0.500) but falling short of softmax confidence (0.808).
• Calibration: Both signals showed monotonic trends where higher scores correlated with higher accuracy.
• Distinct Value: Unlike softmax confidence, the rivalry score is derived purely from internal representations without requiring output sampling or ground truth access.

Significance and Claims

The paper posits that Feature Rivalry is a mechanistically grounded signal of uncertainty that offers three primary contributions:

1. Mechanistic Localization: It identifies specific stages in the residual stream (early encoding and mid-level semantic integration) where uncertainty manifests as competition between concept representations, rather than being a monolithic property of the model.
2. Causal Validation: It provides causal evidence that the competition between rival features is upstream of model outputs; steering along the rivalry direction directly influences generation, confirming that these features actively compete during inference.
3. Interpretable Uncertainty Signal: It demonstrates that internal feature competition can predict answer correctness without relying on output distributions. While less accurate than softmax confidence, it offers interpretability (identifying which concepts are competing) and computability in single-pass scenarios.

The authors emphasize that their within-domain entropy split is a methodological improvement over cross-dataset comparisons, ensuring that rivalry differences are attributable to model uncertainty rather than dataset artifacts. They conclude that while rivalry does not surpass softmax confidence in raw predictive power for short-form QA, it provides a unique, interpretable window into the internal dynamics of uncertainty.