Polysemanticity or Polysemy? Lexical Identity Confounds Superposition Metrics

1. Problem Statement

The paper addresses a critical flaw in the interpretation of polysemanticity in Large Language Models (LLMs).

• The Context: Polysemanticity occurs when a single neuron activates for multiple, seemingly unrelated concepts. The prevailing hypothesis (the Superposition Hypothesis) attributes this to compression: models pack more concepts into fewer dimensions than available neurons, forcing them to share neurons as nearly-orthogonal directions.
• The Confound: Standard metrics often label neurons activating for both "financial bank" and "river bank" as polysemantic evidence of superposition. The authors argue this is a lexical confound. The overlap may not be due to compressing two unrelated concepts, but rather the model processing the shared word form ("bank") before disambiguating its meaning.
• The Gap: Current interpretability tools (like Sparse Autoencoders or SAEs) and superposition metrics cannot distinguish between a neuron encoding a compressed concept versus a neuron encoding a word form (lexical identity). This risks overestimating the extent of superposition and misinterpreting model mechanics.

2. Methodology

The authors propose a rigorous framework to disentangle lexical identity from semantic superposition using a 2×2 Factorial Decomposition.

A. 2×2 Factorial Design

They construct sentence pairs across two binary factors: Word Form (Same vs. Different) and Meaning/Sense (Same vs. Different). This yields four conditions:

1. Same-Lemma (SL): Same word, same sense (e.g., bank financial $\leftrightarrow$ bank financial). Overlap = Lexical + Semantic.
2. Polysemantic (PS): Same word, different sense (e.g., bank financial $\leftrightarrow$ bank river). Overlap = Lexical only.
3. Synonym (SYN): Different word, same sense (e.g., bank financial $\leftrightarrow$ institution financial). Overlap = Semantic only.
4. Cross-Lemma (CL): Different word, different sense (e.g., bank $\leftrightarrow$ plant). Baseline overlap.

Key Metric: They define the Lexical Contribution Ratio ($R_{lex}$):
$$R_{lex} = \frac{\bar{M}_{PS} - \bar{M}_{SYN}}{\bar{M}_{SL} - \bar{M}_{CL}}$$

• If $R_{lex} > 0$, word form drives more overlap than meaning.
• If $R_{lex} \approx 1$, word form accounts for almost all overlap.

B. Neuron Classification

To validate this at the individual neuron level, they introduce the Sense Selectivity Index (SSI):

• Sense-Selective Neurons: High Cohen's $d$ between activations for different senses of the same word ($SSI > 2$).
• Sense-Blind Neurons: Low Cohen's $d$, firing reliably for the word regardless of sense ($SSI < 0.5$).
• Form Detectors: Identified via consistency and specificity across all sentences containing a specific word, independent of sense labels.

C. Causal Interventions

The authors use mean-ablation (replacing neuron activations with the dataset mean) to test causality:

• Word Sense Disambiguation (WSD): Measuring accuracy drop when ablating selective vs. blind neurons.
• Model Editing (ROME): Testing if editing one sense (e.g., "river bank") inadvertently damages the other (e.g., "financial bank") when using blind vs. selective neurons.

3. Experimental Setup

• Models: Nine transformers across four architecture families (GPT-2, BERT, ELECTRA, Pythia, LLaMA-2) ranging from 110M to 70B parameters.
• Dataset: SemCor (37k sentences with human-annotated WordNet senses) and CoarseWSD-20 (Wikipedia text) for validation.
• Scope: Analysis of MLP intermediate activations (the standard level for polysemanticity analysis) and Sparse Autoencoder (SAE) features.

4. Key Results

A. The Lexical Confound Dominates

• Consistent Ordering: Across all models and layers, the overlap follows the order: SL > PS > SYN > CL.
• Lexical > Semantic: The PS condition (same word, different sense) consistently produces significantly more neuron overlap than the SYN condition (different word, same sense).
• Magnitude: $R_{lex}$ is positive across all layers and models, often starting near 0.7–0.9 in early layers and decreasing but remaining significant in deeper layers. This proves that sharing a word form drives more activation overlap than sharing a meaning.

B. Impact on Sparse Autoencoders (SAEs)

• Inherited Confound: SAEs trained on these models inherit the bias. 18–36% of learned features for polysemous words are "sense-blind" (they fire for the word form regardless of meaning).
• Mislabeling: Standard SAE interpretation tools often mislabel these features. For example, a feature firing for "foot," "church," "kick," and "dance" (all polysemous words) was labeled "intensifying adjectives," when it was actually detecting the lexical form of the input words.

C. Mechanistic Structure

• Compact Subspace: The lexical confound is not diffuse; it resides in a Lexical Identity Subspace (LIS).
• Removability: Removing just 20 dimensions (≤1% of the activation space) via PCA reduces the PS–SYN gap by 37–51%. At 50 dimensions, the gap reverses, effectively eliminating the confound.

D. Causal Consequences

• Sense Selectivity: Only <1% of neurons are truly sense-selective. Sense-blind neurons encode word identity (85% accuracy) but fail at sense discrimination (~42%, near chance).
• Editing Precision:
  • Standard ROME: Modifies all activated neurons, causing collateral damage (changing the meaning of the untargeted sense).
  • Sense-Selective ROME: By shielding sense-blind neurons, edits become 6.6× more selective (p = 0.002), preserving the untargeted sense while modifying the target.
• Downstream Tasks: Filtering out sense-blind neurons improves Word Sense Disambiguation accuracy by 5–6 percentage points.

5. Significance and Contributions

1. Reframing Polysemanticity: The paper demonstrates that a substantial portion of what is currently measured as "superposition" is actually lexical identity. True superposition (compressing unrelated concepts) is likely less prevalent than previously thought once word forms are controlled for.
2. New Metrics & Tools:
   • $R_{lex}$: A quantitative metric to measure the degree of lexical confounding in any model.
   • Sense-Selective/Blind Classification: A method to filter neurons based on their causal role in sense processing.
   • Lexical Identity Subspace (LIS): A geometric structure that can be identified and removed to "clean" representations.
3. Practical Implications for Interpretability:
   • SAE Evaluation: Researchers must check if SAE features are sense-blind before claiming they represent monosemantic concepts.
   • Model Editing: To perform precise knowledge edits without side effects, practitioners must target sense-selective neurons and avoid sense-blind (lexical) ones.
4. Correctability: The confound is not a fundamental limitation of the model architecture but a correctable artifact. By projecting out the LIS or filtering neurons, models can be made more interpretable and controllable.

Conclusion

The authors conclude that lexical identity is a primary driver of neuron overlap, often masquerading as superposition. By isolating this signal, the field can move toward more accurate measurements of compression and develop safer, more precise methods for steering and editing language models. The paper provides the necessary tools to distinguish between a neuron encoding a "concept" and a neuron encoding a "word."