🤖 machine learning

Polymorphism Is Rotation: Operational Mechanistic Interpretability from a Two-Layer Transformer to Pythia-70m

This paper demonstrates that independently trained transformers compute identical functions using mutually unintelligible internal coordinates related by a uniform random rotation, a phenomenon termed "polymorphism" that can be resolved via a single orthogonal Procrustes fit to enable the direct transfer of feature dictionaries and steering vectors without retraining.

Original authors: Jordan F. McCann

Published 2026-05-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jordan F. McCann

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you and a friend are both building identical, complex Lego castles from the same instruction manual. You both end up with castles that look exactly the same from the outside and perform the exact same functions (they have the same doors, windows, and towers).

However, inside your castles, the "blueprints" for how the bricks are arranged are completely different. In fact, they are so different that if you tried to use your friend's internal blueprint to understand your own castle, you would get confused. It's not that your castles are different; it's that you are speaking two different "internal languages" to describe the same thing.

This paper calls this phenomenon "Polymorphism." It means "same function, different internal coordinates."

Here is the breakdown of what the researchers found, using simple analogies:

1. The "Decoder" vs. The "Encoder" (The Dictionary Problem)

In AI research, scientists often try to understand what a model is thinking by looking at its "features" (like a dictionary of concepts). They found that if they looked at the end of the process (the "decoder"), the dictionaries looked almost identical between two different models. It was like two people using the exact same dictionary to write a story.

The Twist: The researchers realized that while the dictionary (the decoder) was the same, the way they read the dictionary (the encoder) was completely scrambled.

The Analogy: Imagine two people reading a book. Person A reads it normally. Person B has the book rotated 90 degrees and is reading it upside down. If you just look at the words they point to (the decoder), they match perfectly. But if you try to read Person B's notes using Person A's perspective, the notes make no sense. The "notes" (activations) are in a rotated frame of reference.

2. The Catastrophic Failure

When researchers tried to take the "feature dictionary" from Model A and apply it directly to Model B, it failed completely. The reconstruction of the model's internal thoughts was worse than just guessing the average answer.

The Analogy: It's like trying to use a map of New York City to navigate London. The landmarks (decoder columns) might look similar in a list, but the streets (internal coordinates) are oriented differently. If you follow the New York map in London, you will get lost immediately.

3. The Magic Fix: One Matrix Multiplication

The paper's biggest discovery is how to fix this. You don't need to retrain the models or change their brains. You just need to apply a single mathematical "rotation" (a specific type of math operation called a Procrustes fit) to align the two models.

The Analogy: Imagine you and your friend are both looking at a sculpture, but you are standing on opposite sides of a rotating platform. You see the same object, but from different angles. If you just rotate your platform to match your friend's, suddenly your views align perfectly.
The Result: Once they applied this single "rotation" to Model B's internal data, the dictionary from Model A worked perfectly on Model B. The "internal language" was translated instantly.

4. Is it a Specific Rotation or Random?

The researchers wanted to know: Is this rotation a specific, meaningful shift, or is it just random chaos?

The Finding: It is essentially random. The rotation between two independently trained models is like a random spin on a wheel. It's not a "special" angle; it's just a uniform random rotation.
The Analogy: If you spin a globe randomly and stop it, the orientation is random. The paper shows that every time two AI models are trained from scratch, they end up with a "globe" that is stopped at a random angle relative to each other.

5. What This Means for "Steering" (Changing Behavior)

The paper also tested "steering vectors"—tools used to nudge a model to behave differently (e.g., making it more polite or more creative).

The Finding: If you try to use a "steering nudge" from Model A on Model B without fixing the rotation first, it often fails or even does the opposite of what you want.
The Analogy: Imagine you have a remote control for a toy car (Model A). If you try to use that remote on a different toy car (Model B) that is facing a different direction, pressing "Forward" might make the second car go "Left" or "Backward." You have to first figure out how the second car is facing (the rotation) and adjust your remote's signals accordingly.

Summary of the "Rules"

The paper establishes three main rules for how these models relate:

Same Function, Different Coordinates: Two models trained separately do the same job but use different internal "maps."
The Decoder Lies: Just because the "end of the line" features look similar doesn't mean the models understand things the same way.
The Fix is Cheap: You can align two totally different models with a single, simple math calculation (one matrix multiplication) without needing to retrain them.

The Scale of the Discovery

The researchers proved this on two levels:

The "Toy" Model: A tiny, simple model (104,000 parameters) where they could check every single brick to ensure the theory was true.
The "Real" Model: A much larger, real-world language model (Pythia-70m) with 70 million parameters. The same "random rotation" rule held true here too.

The Bottom Line: AI models are like twins who speak the same language but with different accents and orientations. They do the same things, but to understand one another, you just need to rotate your perspective. Once you do that, their internal secrets become readable.

Technical Summary: Polymorphism Is Rotation

Problem Statement

The field of mechanistic interpretability, particularly regarding Sparse Autoencoders (SAEs), has largely relied on decoder-column cosine similarity as the primary metric for "universality"—the claim that independently trained transformers learn the same internal features. Recent literature reports high cosine similarities (often >0.9) between decoder columns of models trained with different random seeds, interpreting this as evidence that feature dictionaries are shared.

However, this paper identifies a critical failure mode: while decoder columns may align, SAE encoders fail catastrophically when applied to activations from a different seed. Applying an SAE trained on Seed A to the residual stream of Seed B results in negative explained variance (EV), meaning the reconstruction is worse than predicting the constant mean. This suggests that the "universality" metric is measuring only the geometry of the decoder while ignoring a fundamental misalignment in the encoder's reading frame. The core problem is that independently trained models compute the same function but operate in mutually unintelligible interior coordinate systems.

Methodology

The authors employ a rigorous, pre-registered four-bar operational framework to test mechanistic claims, ensuring falsifiability:

Bar B (Behavioural): Mean KL divergence between the model and a constructive specification.
Bar P (Parametric): Maximum per-entry weight Mean Squared Error (MSE) after symmetry alignment.
Bar C (Causal): Correlation between predicted and measured component patch effects.
Bar Pr (Predictive): Correlation between predicted and measured component ablation losses.

Key Methodological Corrections:

Integrated Gradients (IG) vs. Attribution Patching (AP): The authors demonstrate that at convergence, AP anti-correlates with measured patch effects due to noise dominance in gradients near minima. They replace AP with Integrated Gradients, which, by the completeness axiom, provides a mathematically exact predictor of mean-ablation effects (achieving $r > 0.999$ ).
Constructive Specification: Instead of compiling a theoretical spec (e.g., via Tracr), the authors derive a "constructive spec" directly from the trained primary seed by folding RMSNorm and annotating roles, ensuring the spec reflects the actual model implementation.

Experimental Setup:

Toy Scale: A 104k-parameter two-layer transformer trained on bounded-depth Dyck-3 (a bracket-matching task). This allows for full mechanistic interpretation of every weight.
Scale-Up: Nine independently trained Pythia-70m seeds (71M parameters, $d_{model}=512$ ) trained on The Pile.
Cohorts: Experiments compare seeds with shared frozen I/O weights (to isolate interior rotation) and seeds with independent initialization.

Key Contributions & Results

1. The Rotation Hypothesis

The paper proposes that independently trained transformers of the same architecture compute the same function up to an orthogonal rotation of the residual-stream basis.

Polymorphism: The phenomenon where models share a function but possess "mutually unintelligible interior coordinates."
The Fix: A single orthogonal matrix multiplication (Procrustes fit) on a single batch of activations aligns the bases of two seeds.
Recovery: After applying this rotation, cross-seed SAE reconstruction EV is restored to within 0.025 EV of the within-seed baseline on the toy model and 0.85–0.99 EV on Pythia-70m.

2. Statistical Nature of the Rotation

The rotation is not a structured, learnable alignment but a uniform random draw from the orthogonal group $SO(d_{model})$ .

Haar Distribution: The eigenvalue spectrum of the rotation matrix $R$ is statistically indistinguishable from the Haar measure on $SO(d)$ (Kolmogorov–Smirnov test $p \approx 1.000$ ).
Frobenius Distance: The distance $\|R - I\|_F$ matches the theoretical prediction for a random orthogonal matrix ( $\sqrt{2d_{model}}$ ) to within 0.1% for Pythia-70m.
Decoder vs. Encoder: The high decoder cosine similarity (98%) is a static geometric coincidence. The encoder reads from a rotated frame, causing the functional failure.

3. Steering Vector Transfer Regimes

The paper characterizes how steering vectors (diff-of-means interventions) transfer across seeds based on their alignment with the rotation's invariant subspace:

Clean Transfer: Vectors pinned by shared input/output weights (e.g., depth counter in the toy model) transfer without correction.
Partial Transfer: Vectors overlapping the rotated subspace require a dose offset.
Inverted Transfer: Vectors lying primarily in the rotated subspace transfer with inverted signs.
Pythia Scale: In models with no shared I/O (Pythia-70m), all directions are rotated, causing universal inverted transfer. Without rotation correction, steering vectors applied cross-seed degrade performance.

4. Limitations of Parameter Alignment

The authors demonstrate that parameter-level symmetry search (aligning weights via permutations and rotations) cannot fully resolve the misalignment.

Even after optimizing for weight MSE, the activation-level MSE remains high.
Joint optimization of weight and activation loss fails to clear the Bar P threshold; the weight-optimal and activation-optimal rotations sit in disconnected basins of the loss landscape. This confirms that the rotation is an activation-level phenomenon not fully captured by parameter symmetries.

Significance and Claims

The paper claims to resolve the "decoder-cosine puzzle" by showing that universality is rotation-equivalence, not literal coordinate alignment.

Refinement of SAE Universality: Decoder cosine similarity is necessary but not sufficient for feature universality. A complete metric must include cross-seed reconstruction EV and the ability to apply a rotation correction.
Practical Utility: The rotation correction is computationally cheap (one matrix multiplication per model pair) and requires no retraining. It allows for the transfer of SAE feature dictionaries and steering vectors between independently trained models with high fidelity.
Methodological Rigor: By using a pre-registered framework with falsifiable thresholds, the paper distinguishes between "story-fitting" and verified mechanistic claims. The failure of cross-seed Bars P, C, and Pr (without rotation) serves as the empirical motivation for the hypothesis.
Scope: The findings hold across a 104k-parameter toy model and a 71M-parameter family-scale model. The authors explicitly state that frontier-scale (10B+) replication remains an open empirical test, though the experimental design for such a test is straightforward.

The paper concludes that the residual-stream bases of independently trained transformers are, to a first approximation, uniform random samples from the orthogonal group, and that "polymorphism" is the operational reality of this geometric drift.