🤖 machine learning

A Geometric Perspective on Next-Token Prediction in Large Language Models: Three Emerging Phases

By repurposing representation lenses as geometric diagnostic tools to track the evolution of predictive readout subspaces across layers, this study reveals that large language models process next-token prediction through three distinct geometric phases—Seeding Multiplexing, Hoisting Overriding, and Focal Convergence—where increasing model depth primarily enhances candidate disambiguation rather than expanding representational capacity.

Original authors: Gianfranco Lombardo, Giuseppe Trimigno, Stefano Cagnoni

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Gianfranco Lombardo, Giuseppe Trimigno, Stefano Cagnoni

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a Large Language Model (LLM) as a massive, multi-story factory where a piece of information (a word) travels from the ground floor to the roof. At every floor, a team of workers adds a little bit of new information to the package before passing it up. The big mystery this paper solves is: How does the factory decide which word to output next, and where does that decision actually happen?

The authors didn't just ask what the model predicts at each floor; they asked where the prediction lives inside the factory's machinery and how it moves as it goes up.

Here is the story of their discovery, broken down into simple concepts and analogies.

The Tool: The "Representation Lens"

Usually, if you peek inside the factory at the middle floors, the workers seem confused. They are holding a jumbled mix of possibilities, and if you try to guess the final word based on what they are holding, you get it wrong. This is because the workers are holding the information in a "superposition"—like a magician holding a deck of cards face down where all the cards are mixed together.

The researchers used a special tool called a Representation Lens. Think of this as a pair of magical glasses that can rotate and focus the workers' view. Instead of just looking at the messy pile of cards, the lens finds the specific angle where the "winning card" (the correct next word) is actually visible, even if it's buried under other cards.

The Discovery: A Three-Act Play

By tracking how the "winning card" moves through the factory using these glasses, the authors found that the process isn't random. It follows a strict, three-stage geometric dance that happens in almost every model they tested (from small 1-billion parameter models to huge 32-billion ones).

Phase 1: The "Seeding Multiplexing" (The Crowd Gathering)

What happens: As the information enters the factory, the workers start throwing out many possible next words at once. They don't pick a winner yet.
The Analogy: Imagine a crowded room where everyone is shouting out different ideas. The room is full of noise, but the "correct" idea is already there, just mixed in with hundreds of others.
The Geometry: The factory is expanding its "rank" (its capacity to hold many different ideas). The correct word is present, but it's just one voice in a choir. It's visible to the magical lens, but it's not the loudest voice yet.

Phase 2: The "Hoisting Overriding" (The Silent Filter)

What happens: This is the longest part of the journey (about 60% of the factory's height). The workers stop adding new types of ideas. Instead, they start quietly turning down the volume on the wrong ideas and turning up the volume on the right one.
The Analogy: Imagine a sound engineer in a control room. They aren't bringing in new singers; they are just slowly fading out the background noise and boosting the lead singer. The lead singer is still surrounded by the crowd, but they are becoming the clear focus.
The Geometry: The "rank" (the number of active ideas) stays stable. The workers are very precise here, making tiny, almost invisible adjustments that don't change the shape of the room, but they do change who is heard. This is where the model does its heavy lifting: disambiguation (figuring out which of the many candidates is actually correct).

Phase 3: The "Focal Convergence" (The Spotlight)

What happens: In the final stretch, the factory makes a massive, decisive move. It dumps all its energy into a single direction.
The Analogy: The sound engineer suddenly cuts the power to everyone else in the room. The spotlight snaps onto the lead singer. The crowd goes silent, and the singer is now the only thing that matters.
The Geometry: The workers stop being gentle. They make huge, powerful updates that align perfectly with the final output direction. The "rank" shrinks because they are focusing all their energy on just one winner. The magical lens can now see the answer clearly without any help.

The Big Takeaway: Bigger Models = Better Filters, Not Better Starters

The most surprising finding is about how model size affects this process.

The authors found that as you make the factory bigger (adding more floors/layers):

Phase 1 (The Crowd) stays roughly the same size.
Phase 3 (The Spotlight) stays roughly the same size.
Phase 2 (The Silent Filter) gets much longer.

The Metaphor: If you have a small factory, the "filtering" room is small. If you have a giant factory, the "filtering" room is huge.

This means that when we build bigger, smarter AI models, we aren't necessarily making them better at starting with the right ideas or finishing the job. We are giving them a much bigger, more detailed room to sort through the confusion and figure out which of the many possibilities is the right one. The extra power of large models is mostly used for candidate disambiguation—taking a messy pile of "maybe this, maybe that" and turning it into a single, confident "yes."

Summary

The paper reveals that next-token prediction isn't a sudden flash of insight at the end. It's a geometric journey:

Seed: Throw everything into the mix.
Hoist: Quietly filter out the noise (this is where the model gets bigger).
Converge: Lock onto the winner with high energy.

The "magic" of large language models lies in that middle phase, where they have the space and time to carefully separate the signal from the noise.

Technical Summary: A Geometric Perspective on Next-Token Prediction in Large Language Models

Problem Statement

Understanding how Large Language Models (LLMs) process and refine predictive information across their depth remains a central challenge in mechanistic interpretability. While the transformer architecture applies additive updates to a shared residual stream, the geometric evolution of the information required for next-token prediction is unclear. Standard probing methods, such as the LogitLens, suffer from "representational drift," where intermediate states are misaligned with the final unembedding basis, rendering them "blind" to predictive signals present in early layers. Furthermore, the principle of superposition suggests that predictive features may exist in overlapping directions, hidden within the high-dimensional residual stream. This paper addresses the question: Where does predictive information reside in the residual stream, and how does its geometric structure evolve as it propagates through the network depth?

Methodology

The authors propose a framework that repurposes representation lenses (learned affine maps trained to predict the next token from intermediate residual streams) as geometric diagnostic tools rather than mere decoding mechanisms.

Representation Lenses: Following the TunedLens approach, the authors train per-layer affine corrections $(A_\ell, b_\ell)$ to align intermediate representations with the final output distribution via KL distillation.
Predictive Readout Subspace: Instead of analyzing the full correction matrix, the authors perform Singular Value Decomposition (SVD) on the correction matrix $A_\ell$ . They define the predictive readout subspace $S^{(k)}_\ell$ as the $k$ -dimensional subspace spanned by the top- $k$ right singular vectors. This subspace represents the dominant directions in the residual stream where predictive information is concentrated.
Grassmann Manifold Trajectory: The sequence of subspaces $\{S^{(k)}_\ell\}_{\ell=1}^L$ is treated as a trajectory on the Grassmann manifold $Gr(k, d)$. The evolution of this trajectory is quantified using Readout Subspace Similarity (RSS), defined as the mean cosine of principal angles between consecutive subspaces.
Resolution Analysis: The authors vary the resolution parameter $k$ (from 1% to 50% of the hidden dimension $d$ ) to trace a Pareto frontier between Visibility (geometric discriminability between layers) and Energy Retention (spectral fidelity). They identify an intermediate regime ( $k \in \{5\%, 10\%, 15\%\}$ ) that balances these metrics.
Phase Detection: The RSS profile exhibits a unimodal shape (rise, plateau, descent). Using a two-stage Kneedle criterion on the smoothed profile, the authors detect two breakpoints ( $\hat{b}_1, \hat{b}_2$ ) that partition the network depth into three distinct geometric phases.
Experimental Setup: The framework is applied to eight models from two families (QWEN2.5 and OLMO2) ranging from 1B to 32B parameters, trained on the SlimPajama dataset. Metrics include effective rank of hidden states, update orthogonality ( $|\cos(\Delta_\ell, h_{\ell-1})|$ ), and the ratio of FFN to MHA update norms.

Key Contributions

The paper makes two primary contributions:

A Geometric Framework for Depth-Wise Analysis: A method to track the trajectory of predictive information on the Grassmann manifold, revealing that predictive information is present from the earliest layers but hidden in superposition, requiring low-dimensional projection for decoding.
Identification of Three Emerging Geometric Phases: The analysis reveals a consistent tripartite structure in the forward pass of autoregressive models, characterized by distinct geometric behaviors regarding rank, update orthogonality, and subspace stability.

Results and Characterization of Phases

Across all models and scales, the similarity profile follows a unimodal distribution, defining three phases:

Phase 1: Seeding Multiplexing

Geometric Behavior: The predictive subspace reorients and stabilizes (rising RSS). The effective rank of hidden states increases, peaking near the end of this phase.
Updates: Updates are approximately orthogonal to the residual stream. In QWEN2.5, FFN contributions dominate; in OLMO2, contributions are more balanced.
Predictive State: The final token is present among top candidates (Hit@1 $\approx$ 20-33%) but not dominant. The LogitLens remains ineffective, indicating the signal is superposed.
Scaling: The duration of this phase scales weakly with model depth (slope $\le$ 0.26), suggesting a fixed cost for seeding candidates.

Phase 2: Hoisting Overriding

Geometric Behavior: The predictive subspace remains highly stable (near-plateau RSS). The effective rank of hidden states stabilizes.
Updates: Updates are maximally orthogonal to the residual stream (lowest $|\cos(\Delta_\ell, h_{\ell-1})|$ ). The rank of updates peaks in this phase.
Predictive State: The candidate distribution progressively concentrates (Hit@1 rises to $\approx$ 56%, Hit@5/Hit@1 ratio decreases). The LogitLens remains low, implying disambiguation occurs within the subspace without explicit alignment to the final basis.
Scaling: This phase expands linearly with model depth (slope $\approx$ 0.6–1.0), indicating that additional model capacity is primarily absorbed by candidate disambiguation.

Phase 3: Focal Convergence

Geometric Behavior: The predictive subspace reorients sharply toward the unembedding direction (descending RSS). The effective rank of updates decreases, concentrating on fewer directions.
Updates: Updates become highly aligned with the residual stream (highest $|\cos(\Delta_\ell, h_{\ell-1})|$ ). Update norms grow sharply (up to 10 $\times$ ), and FFN contributions dominate across all families.
Predictive State: The candidate distribution collapses to a dominant prediction. The LogitLens performance improves significantly (Hit@1 $\approx$ 30-50%), indicating the final token is now directly readable.
Scaling: The width of this phase remains roughly constant or scales very weakly with depth.

Significance and Claims

The paper claims that the additional capacity of deeper LLMs is largely utilized for candidate disambiguation (Phase 2) rather than for the initial seeding of candidates (Phase 1) or the final convergence (Phase 3). This suggests that the core computational burden of next-token prediction in deep transformers lies in resolving superposed representations within a stable geometric subspace.

The findings align with, yet distinctively extend, mean-field theories of transformer dynamics (e.g., Bruno et al.), mapping the "alignment," "heat," and "pairing" phases to the observed geometric progression. The work establishes that predictive information is not generated late in the network but is present early, hidden in superposition, and geometrically refined through a process of orthogonal updates and subspace concentration.

Limitations: The authors note the analysis is restricted to dense, decoder-only architectures with standard residual connections. The interpretations are based on correlational evidence, and causal interventions (e.g., activation patching) are suggested as future work to provide stronger mechanistic grounding.