🧬 biology

Training-Driven Representational Geometry Modularization Predicts Brain Alignment in Language Models

This study demonstrates that during the training of large language models, the self-organization of representational geometry into low-complexity modules characterized by reduced entropy and curvature robustly predicts alignment with human brain activity, revealing distinct spatial-temporal trajectories across temporal and frontal regions.

Original authors: Yixuan Liu, Zhiyuan Ma, Likai Tang, Runmin Gan, Xinche Zhang, Jinhao Li, Chao Xie, Sen Song

Published 2026-07-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yixuan Liu, Zhiyuan Ma, Likai Tang, Runmin Gan, Xinche Zhang, Jinhao Li, Chao Xie, Sen Song

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine you are trying to teach a robot to understand human language. You might think the robot just needs to memorize more words or learn more grammar rules. But scientists are starting to wonder: does the robot's "brain" actually start to look and work like a human brain as it learns? This question sits at the intersection of computer science and neuroscience, two fields that usually speak different languages. To understand this, we need to look at how information is stored. Think of a computer's memory not as a list of facts, but as a vast, multi-dimensional landscape. In this landscape, "flat" areas might represent simple, clear ideas, while "bumpy" or "twisted" areas might represent complex, messy, or confusing ones. Scientists call these shapes "representational geometry." The big mystery is: as a computer model trains on millions of sentences, does its internal landscape smooth out and become more like the human brain's landscape? If it does, it suggests that the way humans process language might be a fundamental rule of intelligence, not just a biological accident.

This paper takes a deep dive into that question by watching a family of AI models called Pythia grow up. The researchers didn't just look at the finished, fully-trained models; instead, they acted like time-traveling biologists, observing the models at 19 different stages of their training, from their very first steps to their final form. They tracked two specific "geometric" features of the models' internal layers: entropy (how spread out or chaotic the information is) and curvature (how bumpy or sharp the path of information is). They then compared these changing shapes to real brain scans (fMRI) of humans reading the same sentences.

The study found that as the models trained, their layers didn't just get better at guessing the next word; they spontaneously organized themselves into two distinct teams, or "modules." One team, the low-complexity module, became very smooth and organized, with low entropy and low curvature. The other team, the high-complexity module, stayed bumpy and chaotic. Here is the exciting part: the smooth, low-complexity team was the one that matched the human brain's activity the best. It was like finding that the part of the robot that learned to "calm down" and organize its thoughts was the part that actually thought like a human.

However, this alignment didn't happen everywhere at once. The researchers discovered a fascinating split between different parts of the brain. In the temporal regions (areas near the ears, involved in hearing and basic word meaning), the model's smooth, low-complexity layers started matching the human brain very early in training and stayed that way. It was a quick, stable lock-in. But in the frontal regions (the front of the brain, involved in complex planning and sentence structure), the match was delayed and wobbly. These areas seemed to wait for the "smooth" foundation to be built before they could settle down, sometimes even showing a brief period where the "bumpy" layers matched better before finally switching to the smooth ones.

The paper suggests that curvature (how bumpy the information path is) is a key predictor of this alignment. As the models trained, their internal paths became smoother (lower curvature), and this smoothing strongly predicted better matches with human brain activity. This effect was so strong that even when the researchers accounted for the fact that the model was just getting better at training in general, the "smoothness" of the geometry still mattered. Interestingly, this relationship became much clearer and more reliable in the larger models (up to 1 billion parameters). In the smaller models, the signal was a bit noisy, but as the models grew bigger, the connection between "smooth geometry" and "human-like brain activity" became a robust, detectable fact.

In short, the paper argues that the secret to a model thinking like a human might not just be about having more data, but about how that data gets organized into smooth, low-complexity shapes. It suggests that the human brain prefers these "smooth" representations, and as AI models learn, they naturally evolve toward this state, especially in the parts of the brain responsible for understanding the core meaning of words. While the study doesn't prove that this is the only reason models align with brains, it offers a new, geometric lens to understand why and how these digital minds are starting to mirror our own.

Technical Summary: Training-Driven Representational Geometry Modularization Predicts Brain Alignment in Language Models

Problem Statement

The central challenge addressed is understanding the mechanisms by which Large Language Models (LLMs) align with the neural representation and computation of human language. While previous research has established that LLM activations can predict human fMRI responses to linguistic stimuli, the underlying reasons for this alignment remain debated. Existing explanations often rely on external linguistic features (syntax, semantics) or shared sensitivity to low-level statistical properties, which may not reflect deep functional homology. Furthermore, most studies utilize static snapshots of fully trained models, overlooking the developmental trajectory of alignment and the fine-grained, layer-wise modularity that may emerge during training. This study posits that the intrinsic representational geometry (specifically entropy and curvature) of model layers serves as a shared latent variable, and that its evolution during training drives the alignment with human brain activity.

Methodology

Data and Models

Brain Data: The study utilized the TUCKUTE2024 fMRI dataset, comprising neural responses from five participants during the passive reading of 1,000 sentences. Five Regions of Interest (ROIs) in the left-hemisphere language network were analyzed: IFGorb, IFG, MFG, AntTemp, and PostTemp.
Language Models: The authors employed the Pythia model suite (Transformer-based) across four scales: 70M, 160M, 410M, and 1B parameters.
Training Dynamics: Analysis covered 19 log-spaced checkpoints ranging from step 1 to step 143,000, capturing the co-evolution of model representations and training progress.

Geometric Metrics

For each sentence and layer, two geometric properties were computed:

Entropy ( $E$ ): Calculated as the von Neumann entropy of the normalized Gram matrix of token hidden states. Low entropy indicates compressed, low-rank representations; high entropy reflects distributed, high-dimensional states.
Curvature ( $C$ ): Defined as the mean turning angle of token trajectories in the representation space. High curvature implies sharp, local variations; low curvature indicates smoother, globally consistent trajectories.

Analytical Framework

Modularization: Layer-wise geometric trajectories (vectors of entropy and curvature across checkpoints) were clustered using K-means ( $K=2$ ) to identify low-complexity and high-complexity modules. Robustness was verified via Leave-One-Checkpoint-Out (LOCO) and bootstrap resampling.
Brain Alignment: Linear encoding models (ridge regression with 5-fold cross-validation) mapped layer activations to subject-averaged fMRI responses. Alignment was quantified by Pearson correlation ( $r$ ).
Coupling Analysis:
- Trajectory Coupling: Correlation between geometric metrics and fMRI scores across training checkpoints.
- Conditional Coupling: Fixed-effects regressions ( $fMRI \sim Geometry + \log(t) + \alpha_{layer}$ ) to determine if geometric metrics predict fMRI scores after controlling for training progress and layer identity.
Scale Analysis: The relationship between geometry and alignment was compared across model sizes (70M to 1B) to assess the stabilizing effect of model capacity.

Key Results

1. Emergence of Geometric Modularization

Training induces a stable self-organization of layers into two distinct geometric modules:

Low-Complexity Module: Comprising middle-to-upper layers (e.g., layers 4–15 in Pythia-1B), characterized by consistently lower entropy and curvature.
High-Complexity Module: Comprising shallow and remaining layers, maintaining higher entropy and curvature.
This modularization emerges in mid-to-late training (steps >4k) and remains stable, with the inter-cluster gap widening as training progresses.

2. Differential Brain Alignment Across Modules and ROIs

Global Advantage: The low-complexity module consistently outperforms the high-complexity module in predicting fMRI responses across all language ROIs.
Spatiotemporal Heterogeneity:
- Temporal Regions (AntTemp, PostTemp): Show an early-onset (steps $\le$ 64) and stable advantage for the low-complexity module, with no sign reversal.
- Frontal Regions (IFG, IFGorb): Exhibit a delayed and dynamic differentiation. Notably, IFGorb shows a transient early tendency toward the high-complexity module before switching to a sustained low-complexity advantage.
- MFG: Shows the weakest and least consistent module specificity.

3. Curvature as a Robust Predictor

Co-evolution: Curvature exhibits a strong negative correlation with fMRI encoding scores across the training trajectory (as curvature decreases, alignment increases).
Conditional Predictivity: Even after controlling for training progress and layer fixed effects, reduced curvature remains a robust negative predictor of fMRI scores in most ROIs (AntTemp, IFGorb, PostTemp). In contrast, entropy shows heterogeneous conditional effects (positive in some ROIs, non-significant in others), suggesting a more complex relationship.

4. Scale Effects

Stabilization: The relationship between geometry and alignment strengthens with model scale.
Detectability: In smaller models (70M, 160M), the conditional effect of curvature is weak or inconsistent. In larger models (410M, 1B), the negative predictive power of curvature becomes statistically significant and consistent across all ROIs. Larger models act as stabilizers, making the subtle geometric effects detectable against training noise.

Significance and Claims

The paper claims to establish representational geometry as a critical intermediate phenotype linking training dynamics to model–brain alignment. Key contributions include:

Mechanistic Insight: The study demonstrates that the spontaneous differentiation of layers into low- and high-complexity clusters is a functional reorganization driven by learning, not an architectural artifact.
Biological Plausibility: The consistent advantage of the low-complexity module suggests the brain favors representations compressed into lower-dimensional manifolds, which facilitate linear readout.
Distinct Computational Roles: The dissociation between temporal (early, stable) and frontal (delayed, dynamic) alignment trajectories reflects distinct computational demands: temporal regions require stable, low-ambiguity codes for retrieval, while frontal regions require flexible interaction patterns for unification that stabilize later.
Curvature vs. Entropy: The authors propose that curvature (manifold smoothness) is a universal prerequisite for linear mapping to neural responses ("flatter is better"), whereas entropy acts as a "double-edged sword" balancing compression and capacity, leading to non-monotonic relationships with alignment.
Role of Scale: Model scale is identified not just as a performance booster but as a stabilizer that reveals the underlying mechanistic link between manifold structure and neural encoding, making the geometry–alignment coupling detectable only in sufficiently large models.

The study concludes that training-driven geometric modularization offers a complementary lens to traditional linguistic perspectives for deciphering the mechanisms of model–brain alignment, suggesting that representational smoothing facilitates neural-like linguistic processing.