🤖 machine learning

Trajectory Geometry of Transformer Representations Across Layers

This paper introduces a probe-free, geometric framework for mechanistic interpretability that characterizes transformer representations as discrete trajectories across layers, revealing universal three-phase dynamics and demonstrating how curvature, convergence, and bifurcation encode computational complexity, semantic similarity, and ambiguity across diverse model families.

Original authors: Vishal Pandey, Gopal Singh

Published 2026-06-09

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Vishal Pandey, Gopal Singh

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a Transformer AI (like the ones that power chatbots) not as a black box that magically turns text into answers, but as a hiker walking through a vast, invisible landscape.

This paper proposes a new way to watch that hiker. Instead of stopping at every step to ask, "What specific word are you thinking of right now?" (which is how most researchers usually work), the authors decided to map the entire path the hiker takes from the start of the trail to the finish. They call this path a "trajectory."

Here is what they discovered about the shape of that path, using simple analogies:

1. The "Magnet" Effect (Semantic Convergence)

The Analogy: Imagine you drop three different colored balls (representing three different sentences about "dogs") into a funnel. At the top, they are far apart. As they fall, they get pulled toward the center of the funnel until they all clump together in a tight pile at the bottom.
The Finding: The researchers found that when the AI processes sentences with similar meanings, their internal "paths" start far apart but gradually curve toward the same spot in the middle and late stages of the network. They call these spots "attractors." It's as if the AI has a built-in magnet that pulls similar ideas together as it thinks deeper.

2. The "Winding Road" vs. The "Straight Line" (Curvature)

The Analogy: Think of two drivers.

Driver A is just changing the color of their car (e.g., "The cat" vs. "The dog"). They drive in a perfectly straight, boring line.
Driver B is solving a complex puzzle or doing a math problem. They have to swerve, turn sharply, and navigate a winding, twisty road to get to the answer.
The Finding: The paper found that "winding roads" (high curvature) happen when the AI is doing hard reasoning or analogies. "Straight lines" (low curvature) happen when the AI is just making small surface-level changes to words. The "twistiness" of the path is a direct measure of how hard the AI is thinking.

3. The "Fork in the Road" (Disambiguation)

The Analogy: Imagine a hiker standing at a fork in the road. At first, they are just standing at the base of the fork, unsure which way to go. As they take a few steps, they slowly commit to the left path, and the distance between the "left path" and the "right path" grows wider and wider until they are miles apart.
The Finding: When the AI encounters a word with two meanings (like "bank" of a river vs. a "bank" for money), the two possible meanings start at the exact same spot. But as the AI processes the sentence, the path splits. About 20–25% of the way through the network, the paths begin to separate clearly, and by the end, they are completely distinct. The AI doesn't decide instantly; it slowly commits to one meaning as it travels down the path.

4. The Three Stages of the Journey (Three-Phase Structure)

The Analogy: The journey isn't random; it always happens in three distinct chapters, no matter which AI model is used:

Chapter 1: The Map (Encoding): The hiker looks at the terrain and figures out where they are. The path changes direction quickly.
Chapter 2: The Trek (Elaboration): The hiker walks steadily through the middle of the forest. This is where the "magnet" effect and the "winding roads" for hard problems happen. The path is stable but doing the heavy lifting.
Chapter 3: The Destination (Output Prep): The hiker sees the finish line and adjusts their stride to get ready to stop. The path changes direction one last time to prepare the final answer.

How They Know This is Real

The authors didn't just guess. They ran "control experiments" to make sure they weren't seeing things that didn't exist:

If they scrambled the order of the layers (like shuffling the chapters of a book), the patterns disappeared.
If they used a model with random, untrained weights (like a hiker with no map), the patterns vanished.
If they randomly assigned meanings to words, the "magnet" effect went away.

The Bottom Line

This paper argues that we can understand how AI thinks by looking at the geometry of its journey. We don't need to stop and ask the AI what it's thinking at every step. Instead, we can just watch the shape of its path:

Straight lines = Easy tasks.
Twisty roads = Hard thinking.
Clumping paths = Similar ideas coming together.
Splitting paths = Choosing between meanings.

It's a new lens that lets us see the "flow" of intelligence without needing to install any extra tools or ask the AI to explain itself.

Technical Summary: Trajectory Geometry of Transformer Representations Across Layers

Problem Statement

Current mechanistic interpretability of transformer models largely operates in two regimes: tracing specific computational circuits (e.g., induction heads, MLP layers) or probing layer-wise embeddings for pre-specified linguistic features. Both paradigms treat network layers as independent snapshots, neglecting the continuous geometric structure of how representations evolve across the network depth. This paper addresses the open problem of understanding how transformer representations evolve across layers, rather than merely what they encode at a static point.

Methodology

The authors reframe the transformer forward pass as a discrete population trajectory through a high-dimensional representation manifold ( $\mathbb{R}^d$ ). Inspired by population trajectory analyses in computational neuroscience, the study avoids dimensionality reduction for metric computation, operating entirely in the full ambient space to avoid projection artifacts.

Core Framework

Trajectory Extraction: For an input sequence, hidden states are extracted at every layer $l$ (from input embeddings $h^{(0)}$ to final outputs $h^{(L)}$ ). Sequence-level representations are obtained via mean pooling over non-padding tokens.
Geometric Metrics: Five metrics are defined directly in $\mathbb{R}^d$ $R^{d}$ :
- Trajectory Length ( $L$ ): Total Euclidean displacement across layers.
- Curvature ( $\kappa$ ): The turning angle between consecutive displacement vectors; mean curvature ( $\bar{\kappa}$ ) measures non-linearity.
- Semantic Convergence Index (CI): Measures the compression of semantically related prompts relative to unrelated ones ( $D_{between} - D_{within}$ ).
- Layerwise Cosine Similarity (SIM): Angular similarity between adjacent layers to detect phase transitions.
- Representational Stability: Similarity between a prompt and its lexical perturbation to measure abstraction of surface form.
Experimental Design:
- Models: Three decoder-only families (GPT-2 Small, TinyLlama-1.1B, Qwen2.5-1.5B).
- Prompts: 150 semantically controlled prompts across five families: Semantic Categories, Lexical Variations, Analogical Reasoning, Multi-step Reasoning, and Ambiguous Concepts.
- Controls: Four rigorous controls (Random Labels, Random Embeddings, Shuffled Layer Ordering, Multiple Projection Methods) to distinguish learned computation from geometric artifacts.

Key Contributions

The paper introduces a trajectory-geometric framework for transformer interpretability that is probe-free and model-agnostic. It defines five high-dimensional metrics to characterize representation dynamics without requiring classifiers or fine-tuning. The authors release an open-source pipeline enabling this analysis for any causal language model.

Key Results

Across all three architectures and validated against controls, the study reports four principal findings:

Semantic Convergence into Attractor Basins: Semantically related prompts undergo statistically significant convergence in middle-to-late layers. The Convergence Index rises sharply from near-zero in early layers to peak values of 0.41–0.58 in later layers ( $p < 0.001$ ). This effect vanishes under random-label controls, suggesting attractor-like dynamics driven by learned semantics.
Curvature as a Complexity Readout: Reasoning and analogy tasks produce trajectories with significantly higher mean curvature than lexical variations.
- Reasoning (Multi-step/Analogy): 0.71–0.83 rad.
- Lexical Variations: 0.27–0.31 rad.
- This difference is consistent across models ( $p < 0.001$ ), suggesting mean curvature encodes computational complexity. Curvature peaks occur in a "computational inflection zone" (approx. 20–45% of network depth), aligning with known induction head and MLP retrieval layers.
Disambiguation as Trajectory Bifurcation: Ambiguous tokens (e.g., "bank" in different contexts) exhibit a measurable trajectory bifurcation. Representations for different contexts start near identical at layer 0 but progressively separate, reaching a 5.6× separation ratio by the final layer. This bifurcation onset is consistent at ~20–25% of network depth across architectures, indicating a universal schedule for progressive disambiguation rather than instantaneous resolution.
Universal Three-Phase Computational Structure: Layerwise cosine similarity reveals a consistent three-phase structure across all models:
- Phase I (Encoding): Rapid representational change (low similarity).
- Phase II (Elaboration): Stabilized similarity, coinciding with semantic convergence and high curvature.
- Phase III (Output Preparation): A secondary drop in similarity as representations recalibrate toward the output vocabulary.
  This structure collapses under random-embedding controls, confirming it is a product of trained computation.

Significance and Claims

The paper argues that the transformer forward pass is not an arbitrary sequence of states but a geometrically structured flow. The significance of this work lies in:

Probe-Free Interpretability: Providing a lens to analyze model behavior without pre-specifying features or training classifiers.
Dynamical Systems Bridge: Connecting computational neuroscience concepts (manifolds, attractors, curvature) to deep learning, offering a unified view of how representations evolve.
Actionable Insights: The findings suggest that interventions for disambiguation or steering should target specific trajectory phases (e.g., before the bifurcation onset) rather than arbitrary layers.
Universality: The consistency of these geometric patterns across models with different scales, architectures, and training corpora suggests that the geometry of the trajectory may be driven by the structure of language itself.

The authors remain modest regarding causality, noting that while geometric structure correlates with semantic and computational properties, the paper establishes correlation, not causation. They identify causal verification via activation patching as the necessary next step.