🤖 AI

Can Cross-Layer Transcoders Replace Vision Transformer Activations? An Interpretable Perspective on Vision

This paper introduces Cross-Layer Transcoders (CLTs) as a sparse, depth-aware proxy model that reconstructs Vision Transformer activations via an encoder-decoder scheme, enabling faithful, layer-resolved interpretability and demonstrating that final representations are concentrated in a small set of dominant layer-wise terms while preserving model performance.

Original authors: Gerasimos Chatzoudis, Konstantinos D. Polyzos, Zhuowei Li, Difei Gu, Gemma E. Moran, Hao Wang, Dimitris N. Metaxas

Published 2026-04-16

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Gerasimos Chatzoudis, Konstantinos D. Polyzos, Zhuowei Li, Difei Gu, Gemma E. Moran, Hao Wang, Dimitris N. Metaxas

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart, but mysterious, robot artist named ViT (Vision Transformer). This robot can look at a picture of a dog and correctly say, "That's a dog!" But if you ask it, "How did you know that?" it just shrugs. It keeps its thinking process inside a black box, layer after layer of complex math that no human can easily read.

For a long time, scientists tried to peek inside this black box using tools called Sparse Autoencoders (SAEs). Think of these like trying to understand a movie by watching just one single frame at a time. You might see a car or a tree in that one frame, but you miss the whole story of how the car drove, the tree swayed, and the plot unfolded. These tools were too focused on individual layers and missed the big picture of how information flows from the beginning to the end of the robot's brain.

The New Idea: The "Cross-Layer Transcoder" (CLT)

This paper introduces a new, smarter tool called a Cross-Layer Transcoder (CLT).

Imagine the robot's brain is a relay race with 12 runners (layers).

Old Way: You only watched Runner #5 and tried to guess what they were thinking. You had no idea what Runners #1 through #4 did to help them.
New Way (CLT): The CLT is like a super-coach who watches the entire race. It understands that Runner #10 (the final layer) didn't just get their energy from Runner #9; they got a huge boost from Runners #1, #3, and #7 too.

The CLT does two amazing things:

1. It's a "Ghost Runner" (Replacement)

The researchers trained the CLT to act as a stand-in for the robot's actual brain cells (specifically the MLP blocks).

The Analogy: Imagine you have a complex, expensive machine. You build a cheaper, simpler, but highly accurate "ghost machine" that mimics exactly how the real one works.
The Result: They replaced the robot's real brain parts with these "ghosts." Surprisingly, the robot didn't get dumber! In fact, for the most important part of the brain (the [CLS] token, which is like the robot's "final conclusion" or "summary"), the ghost machine worked just as well, and sometimes even better. This proves the CLT is a faithful copy of the robot's thinking.

2. It's a "Credit Scorecard" (Interpretability)

This is the coolest part. Because the CLT is built to reconstruct the robot's thoughts by adding up contributions from previous layers, it creates a scorecard.

The Analogy: Imagine the robot's final answer ("It's a dog!") is a team project. The CLT tells you exactly who did what.
- For the "Patch Tokens" (the tiny pieces of the image): The scorecard shows that each layer mostly just helped itself. It's like a group of people where everyone mostly just fixed their own part of the puzzle.
- For the "[CLS] Token" (the final summary): The scorecard shows something wild! The final answer is a team effort from almost every layer. The early layers (which saw just edges and colors) and the middle layers (which saw shapes) all contributed to the final "It's a dog!" decision.

What Did They Discover?

The "Summary" is a Group Effort: The robot's final decision isn't just made in the last second. It's a deep conversation where information from the very first layer travels all the way to the end. If you cut off the top contributors, the robot forgets what it's looking at.
The "Patches" are Local: The tiny pieces of the image mostly just talk to their immediate neighbors. They don't need the whole team to figure out if a specific pixel is part of a fur or a leaf.
We Can Trust the Explanation: Because the CLT can replace the real brain without losing performance, the "scorecard" it generates is trustworthy. It's not just a guess; it's a mathematically proven map of how the robot thinks.

Why Does This Matter?

Think of it like finally getting the blueprint for a black box.

Before, we knew the robot worked, but we didn't know why.
Now, with CLTs, we can say: "Ah, the robot decided it's a dog because Layer 1 saw fur, Layer 5 saw a snout, and Layer 10 combined them all."

This makes AI trustworthy. If we can see exactly how a model makes a decision, we can fix it if it's wrong, control it if it's dangerous, and understand it if it's confusing. The paper shows that we can finally stop guessing and start seeing inside the vision transformer's mind.

1. Problem Statement

Vision Transformers (ViTs) are powerful foundation models, but their internal representations remain largely opaque, hindering interpretability, controllability, and reliability. Existing interpretability methods, such as Sparse Autoencoders (SAEs), typically operate locally within a single layer. They fail to capture:

Cross-layer computational structures: How information flows and transforms across the depth of the network.
Relative layer significance: The specific contribution of earlier layers to the final representation.
Token-specific dynamics: The difference in how global tokens (e.g., [CLS]) versus local patch tokens aggregate information.

The authors propose investigating Cross-Layer Transcoders (CLTs)—previously used in Large Language Models (LLMs)—as a solution for vision models. The core challenge is determining if CLTs can functionally replace ViT MLP blocks while preserving performance and providing a faithful, depth-aware decomposition of the model's internal logic.

2. Methodology: Cross-Layer Transcoders (CLTs)

The authors introduce a framework where CLTs act as sparse, depth-aware proxy models for the Multi-Layer Perceptron (MLP) blocks in ViTs.

A. Architecture and Mechanism

Encoder-Decoder Scheme: For each layer $i$ , a learned linear encoder maps pre-MLP activations ( $x_i$ ) into a sparse feature vector ( $z_i$ ).
Triangular Reconstruction: Unlike standard per-layer transcoders, CLTs reconstruct the post-MLP activation ( $\hat{y}_j$ ) of a target layer $j$ using a linear sum of sparse codes from all preceding layers ( $i \leq j$ ):
$\hat{y}_j = \sum_{i=1}^{j} z_i W_{dec}^{i \to j}$
This enforces a strictly causal structure, preventing information leakage from future layers.
Sparsity Variants: The paper evaluates three non-linear functions ( $\phi$ $ϕ$ ) to enforce sparsity in the latent codes $z$ $z$ :
1. JumpReLU: Uses learned thresholds.
2. ReLU-Top-k: Keeps the top- $k$ positive values.
3. Abs-Top-k: Keeps the top- $k$ values by absolute magnitude (preserving sign).

B. Training and Replacement Strategy

Training: CLTs are trained on frozen CLIP ViT backbones (ViT-B/32 and ViT-B/16) using datasets like CIFAR-100, COCO, and ImageNet-100. The objective minimizes the reconstruction error between the CLT output and the teacher ViT's actual MLP output, plus a sparsity penalty.
Cascaded Replacement: To test functional fidelity, the authors replace the original MLP blocks with the trained CLTs in a "cascaded" manner. The output of a replaced layer becomes the input for the next, allowing the accumulation of reconstruction errors to be measured.

C. Interpretability via Attribution

The linear decomposition allows for Cross-Layer Attribution. The contribution of source layer $i$ to target layer $j$ is quantified by projecting the decoded contribution onto the total reconstruction:
$C_{i \to j}^{proj} = \frac{\langle c_{i \to j}, \hat{y}_j \rangle}{\|\hat{y}_j\|^2}$
This yields a score indicating how much each preceding layer "explains" the current layer's state.

3. Key Contributions

Functional Replacement of MLPs: The paper demonstrates that CLTs can faithfully replace MLP blocks in ViTs, particularly for the [CLS] token and in later layers, often preserving or even slightly improving zero-shot classification accuracy.
Depth-Aware Attribution: Unlike SAEs, CLTs provide a structured, signed decomposition of how information is aggregated across the network depth.
Asymmetry in Token Processing: The study reveals a fundamental architectural difference:
- Patch Tokens: Exhibit diagonal-dominant attribution (each layer primarily explains its own output).
- [CLS] Tokens: Exhibit broad, distributed attribution (the final representation aggregates signals from many preceding layers).
Faithful Credit Assignment: Ablation studies show that the final representation is concentrated in a small subset of dominant layers. Removing the top-scoring layer degrades performance significantly, while retaining only the top-4 layers preserves accuracy.

4. Key Results

A. Reconstruction Fidelity

CLTs achieve high reconstruction quality with layer-averaged Cosine Similarity between 0.92–0.97 and $R^2$ values between 0.89–0.95.
ViT-B/16 (smaller patches) is easier to approximate than ViT-B/32 (larger patches), likely due to simpler per-token activations in the former.
ReLU-Top-k generally performs best, especially in mid-to-late layers.

B. Zero-Shot Classification Performance

[CLS] Token Replacement: Replacing all MLP layers with CLTs for the [CLS] token results in accuracy nearly identical to the baseline (e.g., CIFAR-100: 61.65% baseline vs. ~61.74% with CLTs).
Patch Token Replacement: Replacing early layers for patch tokens causes significant accuracy drops (e.g., dropping from 61.65% to ~51% on CIFAR-100 when replacing all layers). However, replacing only the last few layers (e.g., 10→11) maintains high accuracy.
Conclusion: CLTs are robust proxies for global classification tokens and late-layer processing but struggle with early-layer spatial feature extraction.

C. Interpretability Findings

Concentration of Credit: Ablation experiments confirm that the final representation relies heavily on a small subset of layers. Removing the single highest-contributing layer causes a substantial drop in accuracy (e.g., -5.6% on ImageNet-100), while keeping only the top-4 layers recovers near-baseline performance.
Semantic Evolution: Visual retrieval using CLT sparse codes shows that [CLS] representations evolve from low-level features (color/layout) in early layers to high-level semantic concepts (e.g., "surfing") in deeper layers.

5. Significance and Future Work

This work establishes Cross-Layer Transcoders as a principled framework for structured, cross-layer interpretability in vision models without compromising performance.

Trustworthy AI: It provides a method to audit ViTs by decomposing their decision-making process into additive, layer-resolved contributions.
Beyond SAEs: It moves beyond local feature extraction to model the causal flow of information across the entire network depth.
Future Directions: The authors suggest using CLTs for hierarchical concept discovery, extending the decomposition to include attention heads (for a full circuit analysis), and applying the framework to broader multimodal tasks.

In summary, the paper proves that ViTs can be effectively "re-wired" with sparse, interpretable proxies that not only mimic the original model's behavior but also reveal the specific, concentrated roles different layers play in forming the final visual representation.