Do Language Models Align with Brains? Prediction Scores Are Not Enough

Technical Summary: "Do Language Models Align with Brains? Prediction Scores Are Not Enough"

Problem Statement
The paper addresses a critical ambiguity in NeuroAI: the conflation of predictive utility with structural or mechanistic alignment. While large language models (LLMs) are routinely used to predict neural responses to naturalistic language stimuli, a high prediction score does not necessarily imply that the model's internal representations capture brain-relevant computations. A model feature might predict neural data simply because it shares low-level statistical properties with the stimulus (e.g., temporal autocorrelation, lexical frequency, or stimulus timing) rather than reflecting the brain's organizational structure. The authors argue that current evaluation practices often overinterpret positive encoding scores as evidence of biological alignment without sufficient controls to rule out these nonspecific explanations.

Methodology: The L-PACT Framework
To resolve this ambiguity, the authors introduce L-PACT (Language Predictive, Alignment-pattern, Causal, and Turing-bounded Test). L-PACT is not a single benchmark score but a four-level, gate-structured hierarchy designed to separate weak predictive evidence from strong claims of alignment. The framework operates on a locked set of derived neural and model artifacts from primary datasets (Brain Treebank, Podcast ECoG, MEG-MASC) and applies a conjunctive decision rule: a model row must pass all four levels simultaneously to be considered an alignment candidate.

The four evidence levels are:

1. Predictive Adequacy (Level 1): Determines if real model features improve held-out neural prediction relative to both nuisance baselines and severe controls. Severe controls include random matched-dimensionality features, autocorrelation-matched random features, circular shifts, sentence-reset features, reversed-context features, layer-label permutations, and token-order shuffles.
2. Relational Adequacy (Level 2): Tests whether model-to-brain alignment profiles reproduce brain-to-brain alignment patterns. Instead of merely matching a single target, the model must replicate the relational organization among neural units (e.g., how Unit A relates to Unit B) better than severe control profiles do.
3. Counterfactual Mechanism-Stripping Adequacy (Level 3): Evaluates whether removing a candidate mechanism (e.g., surprisal, semantic transition, syntactic boundary) from the model representation produces a selective deficit for matching neural targets. This tests mechanism-specific necessity within the predictor, distinguishing specific degradation from generic feature quality loss.
4. Reliability-Bounded Adequacy (Level 4): Normalizes model evidence against brain-brain ceilings (e.g., split-half, run-to-run, or subject-to-subject reliability). Model scores must approach the variability observed among brain measurements themselves to be considered reliable.

Key Contributions

• L-PACT Framework: A rigorous, source-audited evaluation architecture that separates predictive scores from relational, mechanism-specific, and reliability-bounded claims. It explicitly defines "alignment" as a gate-qualified claim requiring control-robustness, relational organization, and mechanism specificity, distinct from mere "correspondence."
• Source-Audited Dataset Eligibility: The study implements a strict audit of the artifact chain, classifying datasets as primary, secondary, diagnostic, or validation-only based on the availability of brain time series, word events, model features, severe controls, and reliability estimates. Notably, the popular NaturalStories dataset is excluded from primary inference due to incomplete artifact chains.
• Assay-Sensitivity Controls: To ensure that negative results are not due to assay failure, the authors implemented positive controls that do not alter real model decisions. These include:
  • Brain-brain reliability controls: Verifying stable structure in split-half or run-to-run brain data.
  • Brain-as-model controls: Testing if a brain profile from one run/subject can predict another, passing the relational gate.
  • Low-level neural and acoustic-envelope gates: Verifying the detection of word-onset, word-rate, and acoustic envelope signals.
  • Deterministic implanted-signal simulation: A synthetic control where known signal is injected, demonstrating that the L-PACT gates can pass when structure is present.

Results
The analysis covered a locked set of 146 integrated decision rows derived from 414 predictive-control, 2,304 relational, 4,320 mechanism-stripping, and 420 brain-brain ceiling rows.

• No Integrated Passes: Zero real model rows passed the integrated L-PACT decision rule. All 146 integrated rows were classified as control-explained.
• Downgrading of Apparent Positives: Under less stringent, single-criterion rules, many apparent positives emerged (e.g., 35 rows with positive raw Pearson scores, 1,224 rows with positive raw profile similarity). However, L-PACT downgraded these because they failed to survive severe controls, reproduce relational structure, show mechanism-specific necessity, or approach brain-brain reliability ceilings.
• Assay Sensitivity Confirmed: The positive controls succeeded. Brain-brain reliability controls showed stable structure; brain-as-model controls passed the relational gate for Brain Treebank (744/864 rows); low-level neural and acoustic gates detected expected signals; and the synthetic implanted-signal simulation passed all four gates. This confirms that the assay is capable of detecting structure when present, validating the negative result for real language models.
• Specific Failures:
  • Predictive: Real model scores were frequently matched or exceeded by severe controls (e.g., circular shifts or autocorrelation-matched random features).
  • Relational: Model-to-brain profiles did not reproduce brain-to-brain alignment patterns better than shuffled or control profiles.
  • Mechanism Stripping: Stripping mechanisms did not yield selective deficits; observed drops were nonspecific or unsupported by upstream gates.
  • Reliability: Model evidence did not approach the brain-brain reliability ceiling.

Significance and Claims
The paper's central claim is that prediction scores alone are insufficient evidence for structural or mechanism-specific alignment between language models and brains. The authors emphasize that this is a constraint on evidential strength, not a universal verdict that models can never align with brains.

• Distinction of Claims: The study distinguishes between correspondence (reproducible statistical association useful for prediction) and alignment (a stronger claim requiring control-robustness, relational organization, and mechanism specificity).
• Actionable Negative Results: By converting apparent positives into an "auditable control-explained taxonomy," L-PACT provides a clear standard for future work. A future positive claim must be localized to the specific gate it newly satisfies rather than asserted from an aggregate score.
• Conservative Stance: The authors explicitly state they do not claim that all possible model-brain comparisons are negative, nor do they claim that current models will never align. The conclusion is bounded to the analyzed derived artifact set, the specific models tested (including DistilGPT-2, GPT-2, Pythia, and Qwen variants), and the configured controls. The framework is designed to adjudicate strong claims, meaning it trades sensitivity for interpretability to prevent overinterpretation of predictive utility.

In summary, the paper argues that without severe controls, relational profile testing, mechanism stripping, and reliability bounding, positive neural prediction scores risk making computational models appear more biologically informative than the evidence supports. L-PACT provides the methodological infrastructure to separate these claims.