Sparse Autoencoders Map Brain-LLM Alignment onto Cortical Semantic Topography

Technical Summary: Sparse Autoencoders Map Brain–LLM Alignment onto Cortical Semantic Topography

Problem Statement
A robust finding in computational neurolinguistics is that intermediate layers of Large Language Models (LLMs) best predict human brain responses to language, forming an "inverted-U" relationship across model depth. However, the mechanistic reason for this alignment remains unexplained. Existing theories propose three non-mutually-exclusive accounts: (1) Predictive processing, where intermediate layers generate brain-like prediction errors; (2) Feature discovery, where intermediate layers contain the richest semantic representations before late-layer specialization; and (3) Geometric artifacts, where intermediate layers possess higher-dimensional, linearly decodable representations. Prior work has established that semantic content dominates brain alignment at an aggregate level, but it lacks the granularity to determine which specific semantic features drive alignment and where in the cortex they map.

Methodology
The authors bridge mechanistic interpretability and neural encoding by decomposing LLM representations into interpretable features using Sparse Autoencoders (SAEs).

1. Model Decomposition: SAEs were trained on every fourth layer of GPT-2 XL (1.5B parameters) and Llama-3.1-8B (8B parameters), extracting 16,000–32,000 features per layer. The models were trained on 500M tokens with reconstruction $R^2 \geq 0.95$.
2. Feature Categorization: A two-pass protocol categorized features into five types: Semantic, Syntactic, Lexical, Prediction, and Other.
   • Pass 1: Automated classification via GPT-4.
   • Pass 2: Human validation by two graduate students (Cohen's $\kappa = 0.81$).
3. Neural Data & Encoding: The study utilized the UTS fMRI dataset (8 native English speakers, 27 naturalistic stories). Voxelwise ridge regression models were fitted to predict fMRI responses from feature subsets. Variance partitioning was used to isolate unique contributions against count-matched and variance-matched random baselines.
4. A Priori Topography Prediction: To avoid post-hoc pattern matching, the authors derived five semantic subcategories (Concreteness/Animacy, Event Structure, Affect/Emotion, Social/Mental, Spatial/Locational) and their predicted cortical regions a priori by intersecting findings from three independent neuroscience programs: Binder et al. (2009), Huth et al. (2016), and Deniz et al. (2019).
5. Validation:
   • Convergence Test: Spearman rank correlation and hypergeometric overlap tests compared the observed SAE feature-to-region mapping against the a priori predictions.
   • Causal/Behavioral: Activation patching (ablation) and Generalized Linear Mixed Models (GLMMs) on reading time datasets (Natural Stories, Provo Corpus) were used to validate the causal role and behavioral relevance of semantic features.

Key Contributions

1. Feature-Level Decomposition of the Intermediate-Layer Advantage: The study demonstrates that semantic features alone recover 94% of the peak encoding performance ($r=0.285$) achieved by full SAE-reconstructed activations, substantially outperforming variance-matched random baselines ($p<0.001, d=1.31$). This confirms that semantic content, rather than syntactic structure or geometric dimensionality, is the primary driver of brain–LLM alignment.
2. Formal Validation of Cortical Semantic Topography: The paper provides the first evidence that SAE-discovered features recapitulate known cortical semantic organization at a granularity inaccessible to prior methods. A formal convergence test confirmed that the observed mapping of five semantic subcategories to five brain regions significantly matches a priori predictions derived from independent neuroscience literature (Spearman $\rho=0.72, p<0.001$; Hypergeometric $p=0.007$).
3. Behavioral and Causal Evidence:
   • Reading Times: SAE semantic features predict human reading times beyond lexical controls and standard word-level semantic norms ($\Delta \log Lik = 38.4, p<0.001$).
   • Activation Patching: Ablating semantic features at the peak layer (L24 for GPT-2 XL) caused a significant drop in encoding performance ($\Delta r^2 = -0.14$), providing evidence consistent with a causal role for these features.
4. Generalization: The findings generalize across languages (English, Chinese, French) and models (GPT-2 XL, Llama-3.1-8B).

Results

• Intermediate Layers: Both models exhibited an inverted-U profile, peaking at L20–24 for GPT-2 XL and L12–16 for Llama-3.1-8B.
• Semantic Dominance: Semantic features explained 88% of the model's captured neural variance. Lexical features contributed negligible unique variance ($\Delta R^2 \approx 0.000$), despite being a significant portion of early-layer features.
• Topographic Alignment: Specific subcategories mapped to specific regions as predicted:
  • Concreteness/Animacy $\rightarrow$ Posterior temporal cortex and Angular gyrus.
  • Affect/Emotion $\rightarrow$ Anterior temporal cortex and dmPFC.
  • Social/Mental $\rightarrow$ Inferior frontal cortex and Anterior temporal cortex.
  • Spatial $\rightarrow$ Angular gyrus.
• Prediction Error: An exploratory analysis suggested the brain may also encode unexpected semantic content (prediction errors), as combining raw semantics with prediction errors improved encoding performance ($\Delta r = +0.018, p<0.01$).
• Comparison to Norms: SAE features outperformed simple word-level semantic norms (e.g., concreteness, valence) in both encoding performance ($r=0.285$ vs. $0.194$) and topographic convergence ($\rho=0.72$ vs. $0.31$), highlighting the importance of contextual, fine-grained representations.

Significance and Claims
The paper claims to transform the question of brain–LLM alignment from "whether" semantics drives the relationship to "which specific semantic features drive alignment where in the brain."

• Mechanistic Insight: The authors argue that the intermediate-layer advantage reflects a peak in semantic richness before the model specializes for the training objective (next-token prediction) in later layers.
• Neuroscientific Convergence: The results support the hub-and-spoke architecture of semantic memory, showing that LLMs spontaneously discover a cortical topography that converges with human neuroanatomy.
• Methodological Advance: By applying SAEs to neural data, the study moves beyond aggregate measures to provide a feature-level account that links computational representations to specific neurobiological mechanisms. The authors position this as a single feature-level account spanning two previously disconnected empirical regularities: the intermediate-layer advantage and cortical semantic organization.

Limitations Acknowledged by the Authors
The paper notes several limitations, including:

• The use of GPT-4 for feature labeling may introduce shared biases (mitigated by human validation and cross-LLM checks).
• Mean-ablation is a coarse intervention for causal claims.
• fMRI resolution ($\sim2$s) conflates word-level and compositional semantics.
• The study is a generalization check across three languages (all head-initial) rather than a comprehensive cross-linguistic proof.
• The prediction-error analysis is a simplified proxy for full hierarchical predictive coding.