🤖 machine learning

Gradient-Skipping Relevance Propagation for Efficient Explainability of Vision Transformers

This paper introduces GradSkip, a novel relevance propagation method for Vision Transformers that improves faithfulness and localization by adaptively weighting attention heads and dynamically distributing relevance through residual connections, achieving state-of-the-art performance with over 14 times fewer GFLOPs than existing approaches.

Original authors: Christopher Buratti, Michele Marchetti, Federica Parlapiano, Davide Traini, Domenico Ursino, Luca Virgili

Published 2026-07-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Christopher Buratti, Michele Marchetti, Federica Parlapiano, Davide Traini, Domenico Ursino, Luca Virgili

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a super-smart robot brain called a Vision Transformer (ViT). It's amazing at looking at pictures and telling you what's in them—like spotting a ladybug or a truck. But here's the problem: the robot is a bit of a "black box." You can ask it, "Why did you think that was a truck?" and it just says, "Because I said so." It doesn't show you which parts of the picture made it decide.

Scientists have tried to peek inside the robot's brain to see how it thinks, but their old tools were like using a sledgehammer to crack a nut. They treated every part of the robot's brain as if it were equally important, and they ignored the "shortcut" wires that let information zip right through without changing. This made the explanations messy and often wrong.

Enter GradSkip, a new, snazzy tool invented by Christopher Buratti and his team to fix this mess. Think of GradSkip as a smart detective that doesn't just look at the whole brain at once, but knows exactly which neurons are doing the heavy lifting and which ones are just taking a nap.

The Two Big Mistakes Old Tools Made

The paper points out two big errors in how previous detectives worked:

The "Everyone is Equal" Mistake: Imagine a choir where some singers are soloists hitting perfect high notes, while others are just humming in the background. Old tools treated every singer as if they were singing the same volume. GradSkip realizes that some "attention heads" (the choir sections) are super important for the decision, while others are just noise. It learns to listen to the soloists and ignore the humming.
The "Shortcut" Mistake: Transformers have these cool "skip connections"—like a secret tunnel that lets a message skip a whole room and go straight to the next one. Old tools treated these tunnels as if they were just empty hallways that didn't change anything. But sometimes, the message does change in the main room, and sometimes it doesn't. GradSkip is the first to check: "Did this message take the tunnel, or did it get transformed in the main room?" It figures out exactly how much credit to give each path.

How GradSkip Solves the Mystery

GradSkip uses a clever two-step strategy to figure out what the robot is thinking:

Step 1: The "Who's Who" Filter: It looks at the robot's brain and asks, "Which attention heads are actually doing the work?" It uses a special math trick (combining gradient flow and a "sparsity" measure, kind of like checking how focused a spotlight is) to pick the most important heads. It's like a bouncer at a club who only lets the cool, relevant neurons in and kicks the noisy ones out.
Step 2: The "Path Splitter": When the detective traces the thought process backward, it splits the "relevance" (the credit for the decision) between the main path and the skip path. If the robot mostly just passed the information through without changing it, GradSkip gives most of the credit to the skip path. If the robot did some heavy thinking, it gives credit to the main path. It's a dynamic, fair way to share the credit.

The Proof: It Works, and It's Fast!

The team tested GradSkip on a bunch of pictures, from a standard set of 1,000 images (ImageNet1K) to medical blood cell photos (BloodMNIST). They also tried it on a task where the robot has to color-code every part of an image (segmentation).

Here is what they found:

It's the most honest explanation: When they tested how well the explanations matched the robot's actual decisions, GradSkip was the best. On the ImageNet1K test, it beat the second-best method by 12.50% in one key metric. On the medical blood cell test, it was just as good.
It's incredibly precise: When they asked the robot to point to the object, GradSkip was the most accurate.
It's a speed demon: This is the coolest part. The old, accurate methods were like driving a tank—they worked great but used a ton of fuel. GradSkip is like a sleek sports car. It uses 14.45 times fewer GFLOPs (a measure of computer work) than the second-best accurate method. For classification, it only needed 69.39 GFLOPs, and for segmentation, just 28.52 GFLOPs.

What GradSkip Is NOT

The paper is very clear about what GradSkip doesn't do:

It doesn't work if you can't see inside the robot's brain. You need to be able to run a "backward pass" (checking the gradients), so it can't be used on models where you can't see the internal gears.
It doesn't magically fix everything. In some tricky cases, like when there are multiple objects of the same kind in a picture, it's still really good, but the paper admits it's harder to get perfect scores.
It doesn't look at the "CLS token" (a special token some robots use to summarize the whole image). The authors note that for standard robots that use this token, ignoring it might miss some clues, but for other types (like SegFormer), it works perfectly.

The Bottom Line

GradSkip isn't just a small tweak; it's a smarter way to listen to the robot. By realizing that not all brain parts are equal and by respecting the "shortcut" wires, it gives us a clear, honest, and super-fast look at how Vision Transformers make decisions. The authors suggest that this could be a game-changer for fields like medical imaging, where understanding why a robot made a diagnosis is just as important as the diagnosis itself. And the best part? It does all this without needing a supercomputer to run it.

Technical Summary: Gradient-Skipping Relevance Propagation for Efficient Explainability of Vision Transformers

1. Problem Statement

Vision Transformers (ViTs) have achieved state-of-the-art performance in computer vision but remain difficult to interpret. Existing explainability methods—categorized as gradient-based, attention-based, and perturbation-based—suffer from significant limitations when applied to transformer architectures.

The core issues identified are:

Uniform Head Treatment: Most attention-based approaches assume all attention heads possess uniform importance. However, literature indicates that heads play heterogeneous roles; some encode semantically focused patterns while others attend diffusely. Treating them uniformly dilutes the explanation by mixing informative and uninformative signals.
Neglect of Residual Connections: Existing methods often model skip connections as identity paths or use fixed heuristics. This fails to distinguish between relevance attributed to learned transformations and relevance simply preserved along the identity route, leading to inaccurate relevance attribution.
Efficiency vs. Faithfulness Trade-off: Gradient-based methods are computationally cheap but suffer from unstable attributions in deep transformers. Perturbation-based methods offer high faithfulness but require massive computational costs (thousands of forward passes). Attention-based methods are efficient but often lack causal reliability.

2. Methodology: GradSkip

To address these limitations, the authors propose GradSkip, a relevance propagation method that introduces two conceptual innovations: adaptive head weighting and skip-connection-aware propagation.

2.1 Adaptive Attention Head Weighting

Instead of averaging all heads, GradSkip computes dynamic, head-specific importance weights ( $\omega^l_q$ ) by combining two complementary metrics:

Gradient-Based Flow Metric ( $\tilde{g}^l_q$ ): This quantifies the causal influence of a head on the model's decision by measuring the gradient magnitude weighted by attention scores. A thresholding mechanism filters out heads with negligible gradient flow to reduce noise.
Gini-Based Sparsity Metric ( $\tilde{s}^l_q$ ): This captures the spatial-semantic focus of a head. The Gini coefficient is calculated on the sorted attention values; high Gini coefficients indicate "spiked" attention patterns (focused on specific regions), distinguishing them from diffuse distributions.

The final weight is a convex combination: $\omega^l_q = \alpha \cdot \tilde{g}^l_q + (1-\alpha) \cdot \tilde{s}^l_q$ , where $\alpha$ balances causal relevance and spatial coherence.

2.2 Skip-Connection-Aware Relevance Propagation

GradSkip explicitly models the residual structure of transformer blocks during backward propagation. Rather than treating the skip path as a fixed identity, it dynamically partitions relevance between the attention transformation path and the skip connection path based on their relative causal contributions.

The contribution of each path is quantified using gradient-weighted activation magnitudes:

$\beta^l_{main}$ : The normalized relevance gate for the main attention path.
$\beta^l_{skip}$ : The complementary gate for the skip connection ( $1 - \beta^l_{main}$ ).

The relevance at layer $l-1$ is computed as:
$R^{l-1} = \beta^l_{main} \cdot (\tilde{W}^l)^T R^l + \beta^l_{skip} \cdot R^l$
where $\tilde{W}^l$ is a weighted attention matrix incorporating head importance, gradient magnitudes, and embedding magnitudes. This ensures that blocks acting primarily as identity mappings contribute minimally to the explanation, preserving relevance through the skip connection.

2.3 Implementation Details

The process involves a single forward pass to capture intermediate representations, gradients, and attention maps, followed by a single backward pass to propagate relevance. The method includes gradient rescaling to address flow imbalances and ensure FullGrad-completeness.

3. Key Contributions

The paper outlines three primary contributions:

Adaptive Head Weighting: A mechanism combining gradient-based flow and Gini-based sparsity to capture the causal influence and selectivity of individual attention heads, moving beyond uniform treatment.
Skip-Connection-Aware Propagation: A dynamic rule that partitions relevance between attention and residual paths based on their relative causal contribution, respecting the fundamental residual structure of transformers.
Comprehensive Evaluation: Extensive experiments on classification (ImageNet1K, BloodMNIST) and segmentation (ADE20K-150) benchmarks demonstrating state-of-the-art faithfulness and localization with significantly reduced computational costs.

4. Experimental Results

4.1 Classification Performance

Evaluated on ImageNet1K and BloodMNIST using ViT-Base and DeiT-Base models:

Faithfulness: GradSkip achieved state-of-the-art performance on the Insertion-Deletion metric and the Faithfulness Violation Test. For example, on ImageNet1K with ViT-Base, it outperformed the second-best approach (MUTEX) by up to 12.50% in Insertion-Deletion scores.
Localization: On the Pointing Game metric, GradSkip achieved the highest scores on ViT-Base (0.868) and competitive results on DeiT-Base (0.831).
Efficiency: GradSkip required 69.39 GFLOPs to generate an attribution map. This is 14.45 times fewer operations than MUTEX and 256.66 times fewer than TIS, while maintaining superior faithfulness compared to other efficient methods like LRP variants.

4.2 Segmentation Performance

Evaluated on ADE20K-150 using the SegFormer model:

GradSkip achieved the highest Pixel Accuracy (PA) (0.815) and mean Intersection over Union (mIoU) (0.371).
It outperformed the second-best approach (MUTEX) by 53.94% in mIoU.
Unlike methods like Finer-CAM, which achieved high PA but low mIoU due to over-extended saliency maps, GradSkip demonstrated superior ability to delineate class-specific regions precisely.
Efficiency: It required only 28.52 GFLOPs, comparable to the most efficient LRP-based methods but with significantly higher segmentation accuracy.

4.3 Ablation and Analysis

Head Sparsity: Statistical analysis confirmed a significant correlation between attention sparsity (measured by Gini coefficient) and faithfulness scores. Selecting only the top 25% of heads based on sparsity improved performance, while using the bottom 25% degraded it.
Hyperparameters: The optimal balance between gradient flow and sparsity was found at $\alpha = 0.50$ . Relying solely on one metric resulted in performance drops of up to 26.67%.
Qualitative Analysis: Visualizations showed GradSkip producing focused, spatially precise heatmaps that correctly isolated discriminative regions (e.g., small objects like ladybugs or thin structures like flagpoles) while suppressing background noise, outperforming competitors that exhibited diffuse or scattered activations.

5. Significance and Claims

The paper claims that GradSkip successfully bridges the gap between computational efficiency and explanation faithfulness in Vision Transformers. Its significance lies in:

Architectural Awareness: By explicitly modeling the heterogeneous roles of attention heads and the dynamic nature of residual connections, GradSkip provides explanations that better reflect how the model processes input.
Practical Deployment: It delivers the high faithfulness typically associated with costly perturbation-based approaches but at a fraction of the computational budget (comparable to LRP), making it suitable for practical applications where resources are limited.
Robustness: The method demonstrates consistent performance across different datasets (general and medical), model architectures (ViT, DeiT, SegFormer), and task types (classification and segmentation).

The authors note limitations, specifically that GradSkip requires access to model internals (gradients and activations) and cannot be applied in purely forward, gradient-free settings. Additionally, the current implementation discards the CLS token, which may limit applicability to standard ViTs where the CLS token aggregates global context, though it is directly applicable to variants like SegFormer that do not rely on it.