💻 computer science

FAME: Feature Activation Map Explanation on Image Classification and Face Recognition

This paper introduces FAME, a novel explainable AI method that combines gradient-driven input manipulation with feature activation analysis to generate competitive attribution maps for image classification and face recognition, while demonstrating that traditional Class Activation Mapping assumptions fail in deeper networks.

Original authors: Xinyi Zhang, Manuel Günther

Published 2026-05-13

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Xinyi Zhang, Manuel Günther

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a super-smart AI robot that looks at photos and tells you what it sees, like identifying a bear in a forest or confirming that two photos show the same person. The problem is, this robot is a "black box." It gives you the answer, but it won't tell you why it thinks that. Did it see the bear's nose? The fur? Or maybe it just got confused by the green grass in the background?

This paper introduces a new tool called FAME (Feature Activation Map Explanation) to help us peek inside the robot's brain and see exactly which parts of the photo matter most.

Here is how the paper explains it, using simple analogies:

The Old Way: The "Guessing Game"

Before FAME, there were two main ways to try to understand these robots:

The "Zoomed-Out Map" (CAM/Grad-CAM): Imagine the robot breaks the photo into a tiny grid of tiles. It looks at each tile and says, "This tile is important!" Then, it stretches that tiny grid back out to the size of the original photo to show you where the important spots are.
- The Flaw: The authors discovered that for deep, complex robots, this "grid" doesn't actually correspond to just one small spot in the photo. A single tile in the robot's brain might be looking at the bear's ear and the tree behind it at the same time. Stretching the grid out creates a blurry, misleading map. It's like trying to guess the ingredients of a soup by only looking at the spoonful you took from the middle; you might miss the carrots at the bottom or the herbs on top.
The "Scratch-and-Check" (Perturbation Methods): This method tries to understand the robot by randomly covering parts of the photo with black squares or noise and seeing if the robot gets confused.
- The Flaw: This is a bit clumsy. If you cover a part of the face with a black square, you create a sharp, unnatural edge that might confuse the robot for the wrong reasons. It's like trying to figure out how a car engine works by randomly throwing rocks at it and seeing what breaks. It's messy and relies on luck.

The New Way: FAME (The "Guided Sculptor")

The authors created FAME to combine the best of both worlds. Instead of guessing or randomly scratching the image, FAME acts like a guided sculptor.

Here is how it works:

The Goal: FAME asks the robot, "What is the smallest change I can make to this photo to make you change your mind?"
The Process: It uses the robot's own internal math (gradients) to gently nudge the pixels of the image. It doesn't just throw random noise; it pushes the pixels in the exact direction that would confuse the robot.
The Result: The areas where the robot needed the most "nudging" to change its mind are the most important parts. If the robot was sure it saw a bear, but a tiny change to the bear's nose made it say "I don't know," then the nose was the key.

FAME takes these "nudges" (which look like a rough, noisy map) and smooths them out with a gentle blur, creating a clean, easy-to-read heat map that shows exactly where the robot is looking.

What Did They Find?

The team tested FAME on two big tasks: Image Classification (naming objects) and Face Recognition (matching faces).

Proving the Old Maps Wrong: They showed that for deep, modern robots, the old "Zoomed-Out Map" method is unreliable. The robot's brain connects distant parts of the image together, so assuming a small grid tile equals a small photo spot is false.
Better at Face Recognition: When trying to match two faces, FAME was much better than the competition.
- If a person was wearing sunglasses, FAME correctly ignored the sunglasses and focused on the eyes and forehead.
- If the photo was taken from a weird angle, FAME still found the matching features (like the bridge of the nose).
- Other methods often got confused by the background or the sunglasses, but FAME stayed focused on the face.

The Bottom Line

FAME is a new, smarter way to explain AI. Instead of guessing or randomly breaking the image, it gently "prods" the AI to reveal what it is actually looking at. The authors found that this method produces clearer, more accurate maps of importance than previous tools, especially for complex, deep neural networks.

Note: The paper mentions that because FAME has to do this "prodding" step-by-step, it is currently slower than the other methods, but the results are more accurate.

Technical Summary: FAME – Feature Activation Map Explanation

Problem Statement

Deep learning models, particularly in image classification (IC) and face recognition (FR), achieve high accuracy but operate as "black boxes," obscuring the reasoning behind their predictions. Existing Explainable AI (XAI) methods fall into two primary categories, each with significant limitations:

Gradient-based Class Activation Mapping (CAM): Methods like Grad-CAM compute attributions based on feature map activations and upsample them to image resolution. This approach relies on the assumption that feature map elements are influenced only by the underlying pixel regions (local receptive fields). The authors argue this assumption fails in deeper networks where receptive fields are large, overlapping, and non-local, leading to misleading attribution maps.
Perturbation-based Methods: Techniques like CorrRISE perturb input images with fixed patches (e.g., black squares or noise) to observe output changes. These methods often introduce artificial edges or fail to remove information effectively (as modern networks handle blurred inputs well). Furthermore, they rely on stochastic random masking and manually defined parameters, lacking a principled approach to perturbation.

Methodology: Feature Activation Map Explanation (FAME)

The authors propose FAME, a unified framework that bridges gradient-based and perturbation-based paradigms. FAME is built upon Layerwise Origin Target Synthesis (LOTS), an adversarial attack technique originally designed to modify images to force a network toward a specific target output at a certain layer.

Core Mechanism

Instead of using fixed patches or simple gradient upsampling, FAME treats the generation of an attribution map as an optimization problem:

Iterative Perturbation: Starting with an input image $x$ , FAME iteratively updates the image to $\bar{x}$ using normalized gradients to minimize a specific loss function $\mathcal{L}$ relative to a target $t$ at a chosen network layer.
$\bar{x} \leftarrow \bar{x} - \eta \frac{\nabla_{\bar{x}}}{\max |\nabla_{\bar{x}|}}$
where $\nabla_{\bar{x}} = \frac{\partial \mathcal{L}(f^l(\bar{x}) \mid t^l)}{\partial \bar{x}}$ .
Attribution Derivation: The attribution map is derived from the absolute difference between the original image and the perturbed image: $\Delta x = |\bar{x} - x|$ .
Smoothing: The resulting pixel-level perturbation map is smoothed using a Gaussian kernel and max-normalized to produce the final attribution map $e \in [0, 1]^{H \times W}$ .

Task-Specific Adaptations

FAME adapts the loss function and target based on the task:

Feature Map Analysis: To test the validity of CAM assumptions, the target is set to zero for specific feature map elements $a[k]$ . The loss $\mathcal{L}_a = \|a[k] - 0\|_1$ identifies which input pixels must be removed to nullify a specific feature activation.
Image Classification: The loss is defined directly on the logit of a target class ( $L_{cls} = z_o$ ), backpropagating to the input to highlight pixels critical for that class prediction.
Face Recognition: The method minimizes or maximizes the similarity score $s$ $s$ between a gallery image ( $x_g$ $x_{g}$ ) and a probe image ( $x_p$ $x_{p}$ ).
- Similar ( $e^+$ ): Minimize similarity ( $L_+ = s$ ) to find regions supporting the match.
- Dissimilar ( $e^-$ ): Maximize similarity ( $L_- = 1-s$ ) to find regions causing the mismatch.
- Unlike related methods (e.g., FGGB), FAME does not require arbitrary thresholding to separate similar and dissimilar regions.

Key Contributions

Refutation of CAM Assumptions: The paper provides empirical evidence that the assumption of local receptive fields in CAM-based methods does not hold for deeper networks. Using FAME, the authors demonstrate that feature map elements in deep networks (e.g., IResNet101) aggregate information from large, shifted, and overlapping regions of the input, rendering simple upsampling unreliable.
Unified Framework: FAME offers a single, optimization-based approach applicable to both IC and FR tasks without requiring task-specific architectural changes or manual threshold selection.
Principled Perturbation: By using gradient-driven perturbations rather than fixed patches, FAME avoids introducing artificial artifacts (like sharp edges from black squares) and provides a more natural interpretation of how input changes affect network outputs.

Experimental Results

The authors evaluated FAME on ImageNet (IC) and three Face Recognition datasets (AR Face, SCface, CFP) using various architectures (ResNet, VGG, ConvNeXt, IResNet).

Image Classification

Qualitative: FAME produces more compact and semantically relevant attribution maps compared to Grad-CAM and Grad-CAM-EW, which often highlight diffuse background regions or focus only on specific parts (e.g., just the head of a bird).
Quantitative: On ImageNet, FAME achieves competitive Intersection over Union (IoU) scores, slightly trailing FullGradCAM but outperforming standard Grad-CAM variants. Crucially, in ROAD-Delete evaluations (measuring performance degradation as salient pixels are removed), FAME achieves the highest scores at low removal ratios (10% and 30%), indicating it identifies the most critical pixels for the model's decision more effectively than competitors.

Face Recognition

Qualitative: FAME generates spatially coherent maps that align with human intuition. For occluded faces (e.g., scarves, sunglasses), FAME correctly focuses on visible identity cues (eyes, forehead) while ignoring occluded areas. In contrast, methods like CorrRISE and FGGB often produce noisy maps or highlight irrelevant background regions.
Quantitative: Across multiple FR protocols and network depths, FAME consistently outperforms Grad-CAM, CorrRISE, and FGGB in both Delete (accuracy drop) and Insert (accuracy recovery) metrics. Notably, FAME achieves the highest Insert scores (up to 96.7% on CFP), demonstrating its ability to identify regions that, when restored, rapidly recover verification accuracy.

Significance and Claims

The paper claims that FAME provides a task-adaptive, optimization-based framework that yields stable and model-relevant explanations. Its primary significance lies in:

Correcting Misconceptions: It challenges the reliance on naive spatial upsampling in deep networks, showing that deeper architectures encode globally integrated patterns rather than localized features.
Robustness: FAME demonstrates superior robustness to variations in pose, occlusion, and resolution compared to state-of-the-art XAI methods.
Generality: It eliminates the need for arbitrary thresholds (a limitation of FGGB) and fixed perturbation parameters (a limitation of CorrRISE), offering a more principled approach to attribution.

The authors acknowledge that FAME is computationally slower than gradient-based methods due to its iterative nature but suggest that speed can be improved via early stopping or larger step sizes with minimal impact on quality. Future work is proposed to extend FAME to transformer architectures and object detection tasks.