🤖 machine learning

Transformer Geometry Observatory TGO-II: Representational Similarity Observatory

This paper introduces the Transformer Geometry Observatory-II (TGO-II) framework to reveal that Vision Transformers develop representational complexity and layer specialization through progressive manifold expansion and richer transformations while maintaining strong token interaction structures throughout training.

Original authors: Kaustubh Kapil, Kishor P. Upla

Published 2026-07-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Kaustubh Kapil, Kishor P. Upla

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a Vision Transformer (a type of AI that "sees" images) not as a black box, but as a massive, multi-story factory. Each floor of this factory represents a "layer" where the AI processes the image.

For a long time, scientists knew this factory got better at its job as it trained, but they didn't really understand how the workers on each floor changed their thinking over time. They mostly looked at the final product (did it recognize the cat correctly?) or the tools the workers used (the attention mechanisms).

This paper, TGO-II, introduces a new "observatory" (a high-tech viewing deck) to watch exactly how the workers' internal maps of the world evolve from the first day of training to the last.

Here is what they found, explained through simple analogies:

1. The Workers Stop Copying Each Other (Specialization)

The Analogy: Imagine a group of interns on Day 1. They all look at the same photo and draw almost identical sketches. They are all thinking the same way.
The Finding: As training progresses, the workers on different floors stop doing the same thing. The paper measured how similar the "sketches" (representations) were between floors. They found that the similarity dropped over time.
What it means: The factory floors are becoming specialists. The early floors might focus on edges and shapes, while the later floors focus on complex concepts like "fur" or "wheels." They are no longer just copying each other; they are carving out their own unique roles.

2. The Workspace Gets Bigger and More Complex (Manifold Expansion)

The Analogy: Imagine the interns are drawing on a small, flat piece of paper. At first, they can only draw simple lines. But as they learn, they realize they need more space. They start using a 3D model, then a holographic projection. The "room" they are working in gets bigger and more complex.
The Finding: The paper measured the "Intrinsic Dimensionality," which is basically a count of how many different directions or "degrees of freedom" the AI is using to describe an image. They found this number increased rapidly during training before settling down.
What it means: The AI isn't just getting better at the same old tricks; it is literally expanding its mental workspace. It is learning to describe images using a much richer, more complex set of dimensions than it started with.

3. The Workers Stay Connected (Token Coupling)

The Analogy: A common guess was that as the AI gets smarter, the different parts of the image (like the "cat's ear" and the "cat's tail") would stop talking to each other and become independent, like strangers in a crowd.
The Finding: The paper looked at how much the different parts of the image influenced each other (token covariance). They found that strong connections remained throughout the entire training process. The "ear" and the "tail" never stopped interacting; in fact, their interaction patterns became even more organized and structured.
What it means: The AI doesn't get smart by isolating parts of the image. It gets smart by learning to coordinate the parts of the image together in increasingly sophisticated ways. The "team" stays tightly knit, even as they become experts.

The Big Surprise: The "Transition Zone"

The researchers noticed something weird happening around the 4th and 5th floors of the factory.

Floors 1–4: The workers were changing rapidly, and the "room" was expanding.
Floors 5–12: The workers settled into a new rhythm. The similarity between floors dropped, but the complexity kept growing.
The Hypothesis: The paper suggests this is a "Transition Zone." The first few floors handle the raw, basic input (like the patch of pixels), and after floor 4 or 5, the AI switches to handling abstract, high-level concepts. It's like the factory shifting from "assembly line" work to "creative design" work.

Summary of the New Theory

Before this paper, people might have thought: "The AI gets smart by breaking the image into independent pieces and analyzing them separately."

TGO-II suggests the opposite: The AI gets smart by keeping the pieces tightly connected but making the way they talk to each other much more complex and specialized. It's like a jazz band: the musicians don't stop playing together (they don't decouple); instead, they learn more complex harmonies and improvisations (manifold expansion) while each musician finds their own unique solo style (specialization).

What's Next?

The authors admit they don't know exactly what these complex patterns mean in terms of "meaning" (semantics) yet. They plan to build "TGO-III" to check if this complex geometry actually helps the AI understand what a "cat" or a "car" is, and "TGO-IV" to watch how the individual parts of the image move and change over time.

In short: The AI learns by expanding its mental map, specializing its layers, and keeping its internal parts tightly connected, rather than by breaking things apart.

Technical Summary: TGO-II: Representational Similarity Observatory

Problem Statement
While Vision Transformers (ViTs) have achieved state-of-the-art performance, the geometric evolution of their internal representations during supervised training remains insufficiently understood. Previous research, including the authors' prior work (TGO-I), has focused heavily on attention mechanisms, feed-forward networks, and spectral properties (eigenvalues, effective rank) of feature covariance. However, spectral similarity alone does not confirm whether layers perform similar computations, as layers can share spectral characteristics while occupying distinct representational subspaces. Furthermore, existing analyses often lack a systematic investigation into how representation similarity, manifold complexity, and token interaction structures co-evolve throughout the training process.

Methodology
The authors introduce TGO-II (Transformer Geometry Observatory-II), a framework designed to analyze the representational geometry of a ViT-Small/16 model trained on ImageNet-100 for 100 epochs. The study utilizes a fixed subset of 1,000 validation images to ensure consistent observability. The methodology employs four primary analytical tools:

Representation Similarity: Measured using Centered Kernel Alignment (CKA) and Singular Vector Canonical Correlation Analysis (SVCCA) to quantify the geometric alignment and similarity between layer representations across training.
Intrinsic Dimensionality: Estimated using the Two-Nearest Neighbor Intrinsic Dimension (TwoNN-ID) metric to assess the complexity and degrees of freedom within the learned representation manifold.
Token Interaction Structure: Analyzed via token covariance matrices and a token coupling ratio. This approach examines pairwise relationships between tokens to determine if they become statistically independent (decoupled) or remain coupled as training progresses.
Complementary Views: The framework treats feature-space covariance ( $C_{feat} \propto A^T A$ ) and token-space covariance ( $C_{token} \propto AA^T$ ) as complementary views of the same underlying representation geometry, sharing identical non-zero singular values.

Key Results
The experiments yielded three primary observations regarding the geometric evolution of ViT representations:

Progressive Specialization: Both CKA and SVCCA scores exhibit a progressive decline throughout training. This indicates that as training proceeds, transformer layers become increasingly specialized, developing distinct representational geometries rather than maintaining redundancy. A notable transition occurs between Layers 4 and 5, where similarity drops before stabilizing in deeper layers.
Manifold Expansion: Intrinsic dimensionality (TwoNN-ID) consistently increases during the initial stages of training before stabilizing. This trend suggests a progressive expansion of the representation manifold into a larger set of locally accessible degrees of freedom. The dimensionality peaks at intermediate layers (specifically Layer 8) and decreases in the final blocks, indicating a depth-dependent complexity profile.
Persistent Token Coupling: Contrary to the hypothesis that increasing complexity arises from tokens becoming independent, token covariance analysis reveals that strong interaction structures persist throughout training. Instead of converging to a diagonal matrix, token covariance evolves into organized, stripe-like structures, particularly in deeper layers. This demonstrates that representational complexity grows while tokens remain strongly coupled.

Key Contributions

Framework Introduction: The proposal of TGO-II as a systematic observatory for studying transformer learning dynamics through representation geometry, complementing the spectral focus of TGO-I.
Empirical Evidence of Co-evolution: Providing evidence that increasing representational complexity (manifold expansion) coexists with decreasing inter-layer similarity (specialization) and persistent token coupling.
Refutation of Token Decoupling: Challenging the "Token Diversification" hypothesis by demonstrating that tokens do not attain complete statistical independence; rather, complexity emerges through sophisticated transformations of coupled token systems.
Identification of Transition Zones: Highlighting a distinct behavioral shift around Layers 4–5, suggesting a transition from low-level token interaction processing to abstract, context-dependent representation.

Significance and Claims
The paper claims that these findings motivate a new working hypothesis: Vision Transformers increase representational complexity through progressively richer transformations of highly coupled token systems, rather than through progressive token independence.

The authors position TGO-II as a necessary step beyond spectral analysis, arguing that understanding the geometry of representations themselves is crucial for a complete picture of transformer learning. The work suggests that the observed spectral expansion (noted in TGO-I) is likely driven by the Manifold Expansion Hypothesis, where the underlying representation manifold expands into higher intrinsic dimensions, enabling richer feature relationships without requiring token decoupling.

The paper remains modest regarding semantic claims, noting that while representational specialization is observed, TGO-II does not directly measure semantic content or class separation. Consequently, the "Semantic Expansion" hypothesis remains inconclusive pending future work. The authors frame these findings as a foundation for future observatories (TGO-III through TGO-VI) aimed at linking geometric evolution to semantic structure, token dynamics, attention mechanisms, and optimization landscapes.