🤖 AI

Spectral Integrated Gradients for Coarse-to-Fine Feature Attribution

This paper proposes Spectral Integrated Gradients (SIG), a novel feature attribution method that improves upon standard Integrated Gradients by constructing integration paths via singular value decomposition to progressively activate features from coarse global structures to fine details, thereby producing cleaner attribution maps with reduced noise.

Original authors: Soyeon Kim, Seongwoo Lim, Kyowoon Lee, Jaesik Choi

Published 2026-05-20

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Soyeon Kim, Seongwoo Lim, Kyowoon Lee, Jaesik Choi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to explain to a friend why a computer program correctly identified a photo of a dog. You want to point out exactly which parts of the picture (the ears, the nose, the fur) made the computer say, "Yes, that's a dog!"

This is what Feature Attribution does. It tries to highlight the important pixels in an image. One popular method for doing this is called Integrated Gradients (IG).

The Problem: The "All-at-Once" Mess

Think of the standard Integrated Gradients method like trying to build a house by throwing every single brick, window, and piece of furniture onto the foundation at the exact same time.

In the computer world, this method takes a blank black screen and slowly fades the real photo in. But it fades everything in at the same speed. It introduces the big, obvious shapes (the outline of the dog) and the tiny, messy details (the noise in the fur, the background static) all together.

Because it mixes the big shapes with the tiny noise immediately, the computer gets confused. It starts reacting to the "static" and "noise" just as much as the actual dog. The result is a map that looks like static TV snow—lots of scattered, confusing highlights that don't really tell you what the computer is looking at.

The Solution: Spectral Integrated Gradients (SIG)

The authors of this paper, Soyeon Kim and her team, propose a smarter way to build that house. They call their method Spectral Integrated Gradients (SIG).

Instead of throwing everything in at once, SIG uses a mathematical tool called Singular Value Decomposition (SVD). You can think of SVD as a special sorting machine that separates the photo into layers of importance:

The "Big Picture" Layer: This contains the major shapes, the outline of the dog, and the main contrast.
The "Fine Detail" Layer: This contains the texture of the fur, the specific patterns, and the background noise.

How SIG works (The Coarse-to-Fine Approach):
SIG builds the image in a specific order, like a painter starting with a rough sketch and then adding details:

Step 1: It starts by slowly revealing only the "Big Picture" layer. The computer sees the general shape of the dog. It says, "Okay, I see a dog shape."
Step 2: Once the big shape is clear, it starts adding the "Fine Detail" layer. It adds the fur texture and the specific features.

Because the computer understands the "big shape" first, it doesn't get confused by the noise when the details appear later. It knows where to look.

The Analogy: Listening to a Symphony

Imagine trying to understand a complex song.

The Old Way (IG): You turn up the volume on the whole orchestra at once. You hear the violins, the drums, the brass, and the static hiss of the recording all together. It's loud and messy, and hard to figure out which instrument is playing the melody.
The New Way (SIG): You start by listening only to the cellos (the deep, structural notes). Once you understand the rhythm and the main theme, you slowly bring in the violins, then the flutes, and finally the tiny percussion. By the time you hear the high-pitched, noisy sounds, you already know the song's structure, so the noise doesn't distract you.

What Did They Find?

The researchers tested this method on thousands of images (like dogs, flowers, and cats) using different types of computer vision models.

Cleaner Maps: The "heat maps" produced by SIG were much cleaner. They highlighted the actual dog or flower without the scattered "static" noise found in the old method.
Better Accuracy: When they tested how well the method actually identified the important parts, SIG performed better than many other advanced methods.
Speed: It wasn't much slower than the old method. It's like getting a much clearer picture without waiting significantly longer for the result.

In Summary

The paper introduces a new way to explain AI decisions. Instead of flooding the AI with all the information at once (which causes confusion and noise), Spectral Integrated Gradients feeds the AI information in a logical order: Big shapes first, tiny details second. This simple change results in much clearer, more trustworthy explanations of what the AI is actually seeing.

Technical Summary: Spectral Integrated Gradients for Coarse-to-Fine Feature Attribution

Problem Statement

Integrated Gradients (IG) is a widely adopted feature attribution method grounded in axiomatic properties such as sensitivity, implementation invariance, and completeness. However, the quality of IG attributions is heavily dependent on the choice of the integration path. The standard IG approach utilizes a straight-line path that interpolates uniformly between a baseline (typically a black image) and the target input. This uniform interpolation treats all pixel differences equally, simultaneously introducing both dominant structural features and high-frequency noise (fine textures and artifacts) throughout the integration trajectory.

Consequently, the accumulation of gradients along this path often includes erratic contributions from high-frequency noise early in the process. This leads to attribution maps that exhibit scattered importance scores and "speckled" noise patterns, obscuring the true semantic features driving the model's prediction. While alternative path-based methods exist (e.g., Guided IG, Manifold-based IG), they often require iterative per-step gradient computations, reliance on separately trained generative models, or stochastic search, introducing significant computational complexity.

Methodology: Spectral Integrated Gradients (SIG)

The authors propose Spectral Integrated Gradients (SIG), a path-based attribution method that constructs integration paths based on the Singular Value Decomposition (SVD) of the difference between the input and the baseline. The core insight is that SVD provides a natural, data-adaptive hierarchy of visual information:

Large singular values correspond to dominant structural patterns (global shapes, object boundaries, low-frequency energy).
Small singular values correspond to fine-grained details, high-frequency textures, and noise.

SIG leverages this hierarchy to enforce a coarse-to-fine progression. Instead of activating all features simultaneously, SIG progressively activates singular components from largest to smallest. This ensures that the integration path prioritizes robust semantic structures before introducing fine details, thereby suppressing the accumulation of noisy gradients during the early stages of integration.

Technical Formulation

Spectral Decomposition: Let $\Delta = x - x'$ be the difference tensor between the input $x$ and baseline $x'$ . The SVD decomposes this difference into rank-one components: $\Delta = \sum_{i=1}^R \sigma_i u_i v_i^\top$ , where $\sigma_1 \ge \sigma_2 \ge \dots \ge 0$ .
Path Construction via Optimization: The authors frame path construction as an optimization problem where the intermediate difference $\Delta(\alpha)$ at step $\alpha$ must preserve the maximum information from the target difference under a rank constraint. By the Eckart-Young-Mirsky theorem, the optimal solution is the truncated SVD, activating components sequentially.
Continuous Relaxation: To avoid discontinuities inherent in hard rank truncation, SIG introduces a smooth gating function $\phi_i(\alpha; \omega)$ $ϕ_{i} (α; ω)$ . This function controls the activation of the $i$ $i$ -th singular component based on the path progress $\alpha$ $α$ and an overlap parameter $\omega$ $ω$ .
- Components with larger $\sigma_i$ activate earlier (smaller start threshold $s_i$ ).
- The parameter $\omega$ controls the overlap between activation windows. As $\omega \to 1$ , SIG recovers the standard linear IG path; as $\omega \to 0$ , it approaches a strict sequential spectral order. The authors empirically set $\omega = 0.4$ to balance continuity and spectral ordering.
Axiomatic Properties: As a path-based method satisfying the boundary conditions $\gamma(0)=x'$ and $\gamma(1)=x$ , SIG inherits the axiomatic guarantees of IG (Completeness, Sensitivity, Implementation Invariance). Additionally, the authors prove that SIG preserves Geometric Symmetry, ensuring that if the model is invariant to spatial transformations (e.g., rotation), the attribution map transforms equivariantly.

Key Contributions

Spectral Path Construction: The introduction of SIG, which utilizes SVD to construct integration paths that respect the spectral hierarchy of visual information, prioritizing coarse structures over fine details.
Theoretical Justification: Providing a theoretical basis for the path construction via rank-constrained optimization (Eckart-Young-Mirsky theorem), offering an optimality guarantee for the path construction criterion that other methods (e.g., GIG, BIG) lack.
Efficiency: SIG requires only a single SVD computation per image and standard gradient evaluations, adding negligible computational overhead compared to standard IG, unlike methods requiring iterative optimization or generative models.
Comprehensive Evaluation: Extensive empirical validation across diverse datasets (ImageNet, Oxford-IIIT Pet, Oxford 102 Flower) and architectures (ResNet, VGG, Inception) using multiple evaluation metrics.

Experimental Results

The paper evaluates SIG against a comprehensive set of baselines, including Gradient $\times$ Input, standard IG, Guided IG (GIG), Blur IG (BIG), Manifold IG (MIG), and others.

Qualitative Analysis: SIG produces attribution maps that are visibly cleaner, less noisy, and more focused on semantically relevant regions compared to IG and other baselines. It effectively suppresses spurious gradients associated with high-frequency noise and boundary artifacts.
Quantitative Performance (Faithfulness):
- DiffID & Insertion/Deletion: SIG consistently matches or outperforms existing methods across three datasets and three architectures. It achieves the best results in nearly all settings on ImageNet and competitive top-tier results on fine-grained datasets.
- ROAR (RemOve And Retrain): On CIFAR-10, SIG induces the sharpest degradation in model accuracy when top-attributed pixels are removed and the model is retrained, indicating it identifies the most discriminative features.
- Leakage-Aware (ROAD): SIG achieves the best GAP scores on ImageNet, demonstrating robustness against information leakage through perturbation masks.
- Non-Perturbation Localization: SIG achieves the best mean Pointing Game accuracy and SAM 2 IoU scores, confirming superior spatial localization of predicted objects.
Efficiency: SIG maintains a Pareto-optimal position between explanation quality and runtime. It introduces only a marginal computational overhead (~16% increase) over standard IG, significantly outperforming computationally expensive methods like SAMP.

Significance and Claims

The paper claims that SIG offers a simple yet effective solution to the noise accumulation problem in Integrated Gradients. By adhering to a spectral hierarchy, SIG naturally follows a coarse-to-fine progression that aligns with the structural semantics of natural images.

The authors emphasize that while the rank-constrained optimization justifies the path construction, the faithfulness of the resulting attributions is an empirical finding supported by triangulated evaluations (perturbation-based, leakage-aware, and localization metrics), rather than a formal guarantee derived solely from the path construction.

The method is positioned as a practical improvement for image classification tasks where SVD's variance ordering aligns with spatial structure. The authors note modestly that the coarse-to-fine inductive bias is most beneficial for images; a pilot study on text classification suggests the method approaches standard IG performance only when the overlap parameter is large, indicating the spectral ordering may not universally correspond to semantic importance in non-image domains.

The work concludes that SIG improves attribution quality without prohibitive computational costs, making it a viable standard for feature attribution in high-stakes applications requiring reliable model diagnostics.