💻 computer science

Grad-CAM for Vision Transformers: A Systematic Taxonomy and Audit of Methodological Ambiguity in Explainable AI

This paper presents a systematic taxonomy and audit of 175 studies to expose the widespread methodological ambiguity and lack of rigorous implementation details in adapting Grad-CAM for Vision Transformers, arguing that treating it as a trivial extension of its CNN counterpart obscures critical choices affecting reproducibility and interpretation.

Original authors: Casey Wall, Longwei Wang, Rodrigue Rizk, KC Santosh

Published 2026-08-07

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Casey Wall, Longwei Wang, Rodrigue Rizk, KC Santosh

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to understand how a brilliant, super-fast detective solves a mystery. In the old days, this detective used a specific set of tools: a magnifying glass that looked at tiny, grid-like patches of a crime scene, one by one. This tool was called Grad-CAM. It was famous for lighting up the exact spots on a photo that helped the detective decide, "Aha! This is a cat!" or "This is a car!" Because the detective looked at the photo in a neat grid, the tool could easily point to specific squares and say, "We saw the cat's ear in this square."

But recently, the detective got a massive upgrade. Instead of looking at a grid, the new detective (called a Vision Transformer or ViT) looks at the whole picture at once, breaking it into a long list of little notes called "tokens." It's like reading a story where every word is a clue, rather than looking at a map. The problem is, the old magnifying glass (Grad-CAM) was built for grids, not lists. It doesn't know how to point at a "word" in a list and say, "This word is important!"

So, scientists started trying to force the old magnifying glass onto the new detective's list of notes. They wanted to see why the new detective made its choices. But here's the catch: nobody agreed on how to do it. Some people pointed at the first note, some at the middle, some at the whole list, and some just guessed. This paper is like a detective inspector coming in to audit all the reports. They want to know: "Are we actually using the tool correctly, or are we just pretending to?"

The Audit: A Systematic Mess

The authors of this paper, Casey Wall and their team, decided to play detective themselves. They went on a massive hunt through the scientific literature, looking for every paper that tried to use the old "Grad-CAM" tool on the new "Vision Transformer" detectives. They started with a huge pile of over 550 papers. After a very careful screening process, they narrowed it down to 175 papers that actually tried to apply this method.

What they found was a bit of a mess. They discovered that most of these papers were treating the new detective's list of notes as if it were still the old grid. They were using the tool without really explaining how they were making it fit.

The paper introduces a "taxonomy," which is just a fancy word for a detailed checklist or a menu of choices. The authors realized that when you try to use Grad-CAM on a Vision Transformer, you have to make several big decisions, and almost no one was writing them down:

Where to look: Do you look at the very first note the detective wrote? The middle one? The final summary?
What to measure: Are you measuring how much a specific note matters, or how much the detective's "main thought" (the [CLS] token) cares about the other notes?
How to mix it: If the detective has many different "heads" (like many different perspectives), do you average them all together, or look at them one by one?

The authors ran some tests to show just how much these choices matter. They took a standard model and ran the same picture through it using different settings from their checklist. The result? The "heatmaps" (the glowing pictures showing what the model saw) looked completely different depending on which choices they made. One setting might highlight the dog in the photo, while another might highlight the grass, and a third might show almost nothing at all.

The Big Problem: "Plug-and-Play" Without Instructions

The study found that out of the 175 papers they looked at, a whopping 101 papers (about 58%) just cited the original paper for the old grid-based detective and didn't explain how they adapted it for the new list-based detective. Another 17 papers just pointed to a code library (a GitHub repository) without explaining the math. Only 26 papers (about 15%) actually took the time to explain their specific choices or cite a paper that did.

The authors argue that this is a serious problem. When a scientist says, "We used Grad-CAM to show our model is fair," but they haven't told you which version of Grad-CAM they used, you can't trust the result. It's like if a chef said, "I baked a cake," but didn't tell you if they used flour or sand, or if they baked it for 10 minutes or 10 hours. The result might look like a cake, but you can't be sure it's the same cake everyone else is talking about.

What the Paper Actually Says (and Doesn't Say)

The paper is very careful not to say, "Here is the one correct way to do this." Instead, they suggest that there is no single "correct" answer yet. They found that the field hasn't settled on a standard way to do this.

They explicitly rule out the idea that you can just take the old formula and slap it onto the new model without thinking. They show that doing so creates "methodological ambiguity," which is a fancy way of saying "confusion about how the tool works."

The authors suggest that for the field to move forward, researchers need to stop treating Grad-CAM on these new models as a simple, automatic step. They need to write down exactly which "feature" they picked, which "gradient" they measured, and how they "reshaped" the data back into a picture.

Why This Matters

This isn't just about academic nitpicking. These heatmaps are used to prove that AI models are safe, fair, and trustworthy, especially in important jobs like medical diagnosis or self-driving cars. If the explanation is vague, we can't be sure the AI is actually looking at the right thing. The paper concludes that until scientists start being super specific about their choices, the glowing pictures we see in research papers might be more about looking cool than about telling the truth.

In short, the paper suggests that we need to stop guessing and start writing down our recipes. Until then, the "Grad-CAM" we see on these new AI models is a bit of a mystery, and we can't be sure what it's actually telling us.

Technical Summary: Grad-CAM for Vision Transformers

1. Problem Statement

Gradient-weighted Class Activation Mapping (Grad-CAM) is a standard technique for visualizing decision-making in Convolutional Neural Networks (CNNs), relying on spatial feature maps with clear channel and spatial dimensions. However, Vision Transformers (ViTs) and other token-based architectures do not produce convolutional feature maps; instead, they utilize token embeddings, self-attention mechanisms, and residual streams.

Despite this fundamental architectural mismatch, Grad-CAM is routinely applied to ViTs in the literature without justification, adaptation, or acknowledgment of the structural differences. The paper identifies a critical gap: the term "Grad-CAM on a ViT" does not refer to a unique, well-defined procedure. Unlike CNNs, where the feature map is implicit, ViTs offer multiple potential sources for attribution (e.g., token embeddings, attention matrices, MLP outputs). Current literature often treats the adaptation as a trivial extension of the CNN formulation, leading to:

Methodological Ambiguity: Authors frequently omit details regarding feature extraction locations, gradient targets, token handling (e.g., [CLS] token removal), and aggregation strategies.
Reproducibility Crisis: Without explicit specification, a reported heatmap is one of many possible visualizations selected from a large space of alternatives, making it impossible to reproduce or scientifically evaluate.
Misinterpretation: Visualizations are often presented as evidence of model behavior or visual grounding without clarifying whether they represent token activations, attention weights, or cross-modal interactions.

2. Methodology

The authors employ a two-pronged approach: a systematic literature audit and the development of a descriptive taxonomy.

A. Systematic Literature Audit

Corpus Construction: The authors conducted a search on April 17, 2026, using Google Scholar with queries for "Grad-CAM" AND "Vision Transformer" (or "ViT") restricted to papers from 2021 onward. They filtered for papers available via CVF, NeurIPS, or IEEE Access, resulting in an initial pool of ~550 papers.
Screening: Manual screening identified 175 papers that applied Grad-CAM or Grad-CAM-adjacent methods (e.g., Grad-CAM++, LayerCAM, ScoreCAM) to ViTs or ViT-based architectures (including Swin Transformers, BEiT, CLIP, and BLIP).
Analysis: The papers were coded for publication venue, year, task setting, architecture type, and reporting detail. The analysis focused on how authors justified the adaptation, what feature representations were used, and the level of methodological detail provided.

B. Descriptive Taxonomy of Adaptations

To characterize the methodological gap, the authors propose a taxonomy that explicitly categorizes the choices required to adapt Grad-CAM to ViTs. This taxonomy is not prescriptive (i.e., it does not define a single "correct" method) but descriptive, making implicit choices explicit. Key dimensions include:

Feature Extraction Locations: Identifying where in the transformer block the "feature map" is extracted. The paper defines six primary candidates:
- $F_1$ : LayerNorm output prior to multi-head attention (normalized token embeddings).
- $F_2$ : Softmax-normalized attention matrices (token-to-token interaction strengths).
- $F_3$ : Attention outputs (value-projected token representations).
- $F_4$ : LayerNorm output prior to the Feed-Forward Network (FFN).
- $F_5$ : FFN output.
- $F_6$ : Post-FFN residual output.
- Cross-Attention: In Vision-Language Models (VLMs), the taxonomy distinguishes between context-side representations and cross-attention maps connecting text queries to image keys.
Gradient Targets: Defining the scalar objective ( $y_c$ ) with respect to which gradients are computed. This varies by task (e.g., class logit for classification, ITM score for VLMs, segmentation loss).
Token Handling and Spatial Reconstruction:
- Token Selection: Deciding whether to use gradients from the [CLS] token, average gradients over patch tokens, or exclude non-spatial tokens entirely.
- Reshaping: Mapping the 1D token sequence back to a 2D spatial grid ( $H/P \times W/P$ ). This assumes a raster-scan ordering of patches, which may not hold for all representations (especially those heavily mixed by self-attention).
Aggregation Strategies:
- Linear Combination: How to combine feature dimensions (e.g., averaging gradients over spatial tokens vs. using [CLS] gradients).
- Head Aggregation: For multi-head attention, determining whether to average heads before or after ReLU, or to treat heads as separate channels.

3. Key Results

The audit of 175 papers yielded several critical findings:

Lack of Justification: Approximately 58% (101 papers) referenced only the original CNN-based Grad-CAM paper or non-ViT variants, providing no theoretical justification for applying the method to transformers.
Sparse Methodological Detail: Only 15% (26 papers) explicitly justified their adaptation or cited prior work that did so. Among these, only 13 distinct peer-reviewed adaptations were identified, indicating the field has not converged on a standard convention.
Reliance on Code Repositories: About 10% (17 papers) relied solely on the pytorch-grad-cam library, which implements ViT Grad-CAM without providing a theoretical basis for the adaptation.
Underspecification: Even among papers that claimed to use specific adaptations, critical details were often missing, such as:
- The exact transformer layer used.
- Whether the [CLS] token was included or excluded.
- How attention heads were aggregated.
- The specific gradient target (e.g., raw logit vs. softmax probability).
Visual Variability: Qualitative experiments on a standard ViT-B/16 model demonstrated that different choices (feature location, gradient target, weighting strategy) produce qualitatively different heatmaps for the same input and class. Some maps localized the object, while others highlighted context or appeared inverted.

4. Key Contributions

Systematic Audit: The first large-scale literature review quantifying the prevalence and reporting quality of Grad-CAM applications in ViTs, revealing a widespread lack of methodological rigor.
Descriptive Taxonomy: A framework that categorizes the specific methodological choices (feature location, gradient target, token handling, aggregation) required to adapt Grad-CAM to token-based architectures. This taxonomy serves as a vocabulary to describe existing variations rather than prescribing a single solution.
Identification of Ambiguity: The paper highlights that "Grad-CAM on a ViT" is an underspecified term. It demonstrates that without explicit reporting of the adaptation choices, visualizations cannot be treated as reproducible scientific evidence.
Guidance for Reporting: The authors provide a checklist for authors and reviewers to ensure that ViT Grad-CAM visualizations include necessary details (feature representation, gradient target, token handling, aggregation, and spatial reconstruction) to support claims about model behavior.

5. Significance and Claims

The paper argues that the current practice of applying Grad-CAM to ViTs is often a "plug-and-play" approach that ignores the theoretical mismatch between the method's assumptions (spatial feature maps) and the model's architecture (token sequences).

Scientific Rigor: The authors claim that the current lack of specification undermines the scientific validity of explanations in high-stakes domains (e.g., medical imaging, safety-critical systems). An underspecified heatmap may be visually persuasive but scientifically uninterpretable.
Reproducibility: The study asserts that reproducibility is impossible without explicit reporting of the adaptation choices. The taxonomy allows researchers to distinguish between different attribution procedures that currently share the same label.
Modest Scope: The paper explicitly states it does not propose a single "correct" Grad-CAM adaptation for ViTs. Instead, it aims to clarify the range of methodological choices being made and to improve the transparency of reporting. The goal is to help the community distinguish between reproducible attribution procedures and visually persuasive but underspecified explanations.

In conclusion, the paper calls for a shift in the research community: moving away from treating Grad-CAM on ViTs as a trivial extension of CNN methods toward a more rigorous, architecture-aware approach where every visualization is accompanied by a precise definition of its underlying mathematical and implementation choices.