Technical Summary: Interpreting Brain Responses to Language with Sparse Features from Language Models

Problem Statement

A central challenge in cognitive neuroscience is characterizing the specific features represented by the human language cortex. While Artificial Language Models (LMs) have emerged as powerful tools for probing these representations, existing studies often face criticism for relating one "black box" (the LM) to another (the brain) without yielding clear claims about what specific neural populations represent or compute. Furthermore, standard LM encoding models typically rely on dense residual-stream hidden states, which are difficult to interpret. The authors aim to address these limitations by developing a framework that not only predicts neural responses but also provides interpretable, mechanistic insights into the features driving those responses, specifically distinguishing between primary (general) and idiosyncratic (fine-grained) LM representations.

Methodology

The authors introduce Augmented Sparse Encoding Models, a framework that modifies standard LM encoding approaches in two key ways:

1. Sparse Autoencoder (SAE) Decomposition: Instead of using dense residual-stream features directly, the authors project LM hidden states into a sparse, hierarchically organized feature basis using pre-trained Sparse Autoencoders (SAEs). They utilize two architectures:
   • JumpReLU SAE: A standard high-performing SAE.
   • Matryoshka SAE: A hierarchical SAE trained to produce nested feature sets, where early features capture general, widely applicable information, and later features capture progressively finer-grained, idiosyncratic details.
2. Explicit Surprisal Augmentation: The feature set is augmented with an explicit scalar predictor for surprisal (negative log probability), a widely accepted measure of linguistic processing difficulty.

Data and Analysis Pipeline:

• Dataset: The primary analysis uses a high-field 7T fMRI dataset from eight participants listening to 200 linguistically diverse sentences. Voxels were selected based on reliability (Noise Ceiling Signal-to-Noise Ratio > 0.4) and restricted to the left hemisphere.
• Voxel Subtypes: Voxels were categorized based on their correlation with two principal components (PCs) identified in prior work: PC1 (processing difficulty: easy-to-hard) and PC2 (content: concrete-to-abstract). A "Ghost" population (weak loading on both PCs) was also analyzed.
• Encoding Model: A two-stage pipeline was employed:
  1. Feature Selection: LASSO regression to identify a sparse set of features predicting voxel responses.
  2. Refitting: Ridge regression (always including the surprisal feature) to estimate predictivity.
• Generalization Analysis: To identify coherent subpopulations, the authors tested whether signed sparse features predicting a "source" voxel could generalize to predict "target" voxels within the same population.

Key Contributions and Results

1. Validation of Sparse Features

The study demonstrates that SAE features predict voxel responses to language as accurately as dense residual-stream features. Specifically, the Matryoshka SAE basis achieved performance comparable to dense residual features across all voxel subtypes, while providing a more interpretable, sparse feature set (averaging ~18 features selected per voxel).

2. Dissociation of Processing Difficulty and Content

The framework successfully recovers a dissociation between processing difficulty and content-driven voxel populations at the single-voxel level:

• Hard-to-Process Voxels: These are captured almost entirely by surprisal alone. Adding SAE or residual-stream features provides no significant predictive benefit.
• Abstract and Concrete Voxels: These populations show significant predictive gains when SAE or residual-stream features are included, indicating they are tuned to representational content beyond mere processing difficulty.

3. Interpretation of Content-Driven Voxels

By examining the specific SAE features selected for Abstract and Concrete voxels, the authors refined existing interpretations:

• Abstract Voxels: Driven by features related to emotions, questions, consequences, and failure/error statements. They are suppressed by features related to scenery.
• Concrete Voxels: Driven by features related to people, actions, and pronouns.
• Asymmetry: The analysis revealed asymmetries not visible in PC-based analyses; for instance, "people-related" features suppress Abstract voxels but do not reliably predict Concrete voxels.

4. Discovery of "Ghost" Voxel Subpopulations

The authors identified and interpreted a previously uncharacterized "Ghost" voxel population (stable responses not captured by the main PCs).

• Finding: A coherent subpopulation of 32 voxels was found to be tuned specifically to people-centered content (relationships, pronouns, actions involving people).
• Anatomy: These voxels were located largely outside the core fronto-temporal language network, in the inferior angular gyrus (near the temporoparietal junction) and medial prefrontal cortex, regions associated with social cognition and theory-of-mind.
• Individuality: Unlike the core language network, this subpopulation was driven by features shared across only a subset of individuals (often just one or two participants).

5. Characterization of the Fronto-Temporal Language Network

Analyzing five canonical language regions (IFG, IFGorb, MFG, AntTemp, PostTemp):

• Regional Differences: Frontal regions (particularly MFG) are well-explained by surprisal alone, whereas temporal regions rely significantly more on LM-derived content features.
• Shared Feature Basis: Despite regional differences, the language network relies on a common set of features across regions and individuals. Voxel-to-voxel generalization was high across the network, unlike the "Ghost" population.
• General vs. Idiosyncratic Features: The most prevalent features shared across participants and regions were general, widely-applicable features (e.g., token count, common semantic themes like people/emotions).
• Hierarchy Alignment: Crucially, brain responses are best predicted by the most general features (the first 128 features in the Matryoshka hierarchy), which constitute only 0.4% of the total feature space. Finer-grained, idiosyncratic features (the majority of the SAE space) contributed less to predicting brain responses, suggesting a nontrivial correspondence between brain and LM representations based on general information rather than specific, fine-grained details.

Significance and Claims

The paper claims that Augmented Sparse Encoding Models move beyond treating models and brains as black boxes by providing a framework that:

1. Separates Signal: Distinguishes processing difficulty (surprisal) from representational content (SAE features).
2. Reveals Structure: Identifies that the human language network is organized around a shared, interpretable feature basis dominated by general, primary features rather than idiosyncratic ones.
3. Extends Interpretation: Uncovers novel, coherent voxel subpopulations (e.g., people-tuned voxels) that are spatially distributed and not aligned with dominant low-dimensional response structures.
4. Deepens Correspondence: Suggests that the alignment between artificial and biological language representations is driven by the features that capture the most general information encoded in LMs, implying a deeper structural correspondence than previously implied by encoding accuracy alone.

The authors maintain a modest stance, noting that interpretations are limited to the specific SAE feature basis used and that voxel-level data represents a coarse-grained measure of neural activity. They do not claim to have fully solved the problem of neural representation but provide a validated step toward characterizing how linguistic meaning is organized in the brain.