💻 computer science

TTE-CAM: Built-in Class Activation Maps for Test-Time Explainability in Pretrained Black-Box CNNs

The paper introduces TTE-CAM, a test-time framework that converts pretrained black-box CNNs into self-explainable models by replacing their classification heads with a convolution-based layer, thereby achieving state-of-the-art predictive performance while providing faithful, built-in explanations without the trade-offs of existing post-hoc or inherently interpretable methods.

Original authors: Kerol Djoumessi, Philipp Berens

Published 2026-03-31

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Kerol Djoumessi, Philipp Berens

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a brilliant but mysterious doctor, let's call him Dr. Black-Box. Dr. Black-Box is amazing at diagnosing diseases from X-rays and eye scans. He gets the diagnosis right almost every time. But there's a catch: when you ask him, "Why did you think this patient has pneumonia?" he just shrugs. He can't explain his reasoning because his brain is a tangled web of millions of connections that even he doesn't fully understand.

In the real world, doctors can't just trust a "black box." They need to know why a diagnosis was made before they can trust it with a patient's life.

The Old Ways: The "After-the-Fact" Detective vs. The "Slow Learner"

Before this paper, researchers had two main ways to try to understand Dr. Black-Box, and both had big problems:

The "After-the-Fact" Detective (Post-hoc Methods):
Imagine Dr. Black-Box makes a diagnosis, and then you hire a detective to look at the X-ray and guess what the doctor was looking at. The detective draws a glowing circle on the image saying, "I think the doctor was looking here!"
- The Problem: The detective is just guessing. Sometimes the circle is right, but often it's wrong or misleading. It's an approximation, not the real reason the doctor made the decision.
The "Slow Learner" (Inherently Interpretable Models):
This is like training a new doctor from scratch who is forced to explain every step of their thinking as they go.
- The Problem: This new doctor is very honest and clear, but they aren't as smart as Dr. Black-Box. They make more mistakes because being forced to explain things slows them down and limits their ability to learn complex patterns.

The New Solution: TTE-CAM (The "Built-in Flashlight")

The authors of this paper, Kerol Djoumessi and Philipp Berens, came up with a clever trick called TTE-CAM.

Think of Dr. Black-Box's brain as a factory assembly line. At the very end of the line, there is a "Final Decision Maker" (the classification head) who looks at the processed parts and shouts, "Pneumonia!" or "No Pneumonia!"

The problem is that this Final Decision Maker is a "Fully Connected Layer"—a messy, opaque switchboard that doesn't show its work.

TTE-CAM's Magic Trick:
Instead of firing the Final Decision Maker and hiring a new, slower one, the authors simply swapped the switchboard for a transparent, built-in flashlight.

Here is how they did it:

The Swap: They took the messy, opaque switchboard at the end of the factory and replaced it with a simple, transparent tool (a 1x1 convolution layer).
The Secret Sauce: They didn't throw away the old knowledge. They took the exact weights (the "muscle memory") from the old switchboard and transferred them directly into the new transparent tool.
The Result: Because the new tool is built the same way the old one was, Dr. Black-Box doesn't need to retrain. He keeps his super-high accuracy. But now, instead of just shouting a diagnosis, the tool automatically projects a "heat map" onto the X-ray showing exactly which parts of the image triggered the decision.

Why is this a Big Deal?

No Retraining Needed: You don't have to teach the AI anything new. It's like taking a Ferrari and swapping the dashboard for one that has a clear map on it, without changing the engine. The car still goes just as fast.
It's Honest (Faithful): Unlike the "After-the-Fact" detective who guesses, this flashlight is part of the machine's actual decision-making process. If the light shines on a spot, it's because the machine actually used that spot to make the decision.
It's Fast: The machine gives you the answer and the explanation in one single step. Other methods often have to run the machine multiple times just to figure out the explanation.

The Test Drive

The authors tested this on two real medical tasks:

Diabetic Retinopathy: Checking eye scans for damage.
Pneumonia: Checking chest X-rays for lung infections.

The Results:

Accuracy: The model kept its perfect scores. It didn't get any dumber.
Explanations: The "heat maps" it produced were just as good (and sometimes better) than the guesses made by the "After-the-Fact" detectives. They correctly highlighted the diseased areas in the lungs and eyes.

The Bottom Line

TTE-CAM is like giving a super-smart but silent AI a voice. It allows us to use the most powerful, accurate AI models we have today, but forces them to show their work while they are working, without making them slower or less accurate. It bridges the gap between "being smart" and "being understandable," which is a huge step toward getting AI into real hospitals.

1. Problem Statement

Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance in medical image analysis but remain "black boxes," hindering their adoption in high-stakes clinical settings due to a lack of interpretability. The field currently faces a fundamental trade-off between two existing approaches:

Post-hoc Methods: Techniques like GradCAM or Integrated Gradients generate saliency maps after prediction. While they do not require retraining, they are inherently unfaithful because the explanation is derived from the model rather than driving the output.
Inherently Interpretable Architectures: Models designed to be self-explainable are faithful (predictions are computed directly from the explanation) but often require complex training procedures or sacrifice predictive performance compared to standard black-box CNNs.

The core challenge addressed by this paper is how to convert high-performing, pretrained black-box CNNs into self-explainable models without retraining and without a drop in predictive accuracy.

2. Methodology: TTE-CAM

The authors propose TTE-CAM (Test-Time Explainability via Class Activation Maps), an architectural reformulation that transforms a standard pretrained CNN into a self-explainable model at inference time.

Architectural Reformulation:
- The method targets the classification head of a pretrained CNN.
- It removes the Global Average Pooling (GAP) layer and the Fully Connected Layer (FCL).
- It replaces the FCL with a $1 \times 1$ convolutional layer having $C$ filters (where $C$ is the number of classes).
Weight Initialization:
- Since an FCL is mathematically equivalent to a $1 \times 1$ convolution, the original pretrained weights are directly transferred to the new convolutional layer.
- Crucially, no retraining is required.
Mechanism:
- The reformulated layer produces built-in Class Activation Maps (CAMs) as its output.
- These maps are spatially averaged to compute class scores, which are then passed through a softmax to generate the final prediction.
- Because the prediction is computed from the activation maps, the explanation is faithful by construction.
Comparison Baselines:
- The method was compared against five post-hoc baselines across three families:
  - Gradient-free: CAM, ScoreCAM.
  - Gradient-based: GradCAM, LayerCAM.
  - Backpropagation-based: Integrated Gradients (IG).

3. Key Contributions

Bridging the Faithfulness-Performance Gap: TTE-CAM demonstrates that high-performing black-box CNNs can be converted into self-explainable models without retraining, preserving the original predictive accuracy while providing faithful explanations.
Test-Time Efficiency: Unlike post-hoc methods that may require multiple forward passes (e.g., ScoreCAM) or one pass per class (e.g., GradCAM), TTE-CAM generates faithful explanations in a single forward pass.
No Retraining Overhead: By leveraging the mathematical equivalence between FCLs and $1 \times 1$ convolutions, the method utilizes existing pretrained weights immediately, avoiding the computational cost of training inherently interpretable models from scratch.

4. Experimental Results

The method was evaluated on two public medical imaging datasets:

Kaggle Fundus (Diabetic Retinopathy): Binary classification (No DR vs. DR).
RSNA Chest X-Ray (Pneumonia): Pneumonia detection.

Predictive Performance:
Replacing the FCL with the $1 \times 1$ convolution preserved the predictive performance exactly:

DR Dataset: Accuracy = 0.899, AUC = 0.923.
Pneumonia Dataset: Accuracy = 0.953, AUC = 0.988.

Explanation Quality:
TTE-CAM was evaluated using three metrics: Top-k Sensitivity, Top-k Localization, and Activation Precision.

Qualitative: Visual comparisons (Figure 1) showed TTE-CAM produced heatmaps similar to standard CAM and competitive with other post-hoc methods.
Quantitative: TTE-CAM achieved performance comparable to or competitive with the five post-hoc baselines.
- On the Fundus dataset, TTE-CAM achieved a Top-k Precision of $0.33 \pm 0.28$ and Top-k Sensitivity of $0.629$.
- On the CXR dataset, it achieved an Activation Precision of $0.12 \pm 0.09$ and Top-k Sensitivity of $0.953$.
- Notably, TTE-CAM matched the performance of the original CAM (which it is based on) while offering a unified inference pipeline.

5. Significance and Limitations

Significance:

TTE-CAM offers a practical solution for clinical deployment where both high accuracy and model transparency are required.
It eliminates the "faithfulness vs. performance" trade-off by making the explanation the mechanism of prediction itself, without sacrificing the benefits of transfer learning.
It reduces computational overhead during inference compared to many gradient-based post-hoc methods.

Limitations:

Resolution: Like all CAM-based methods, reliance on low-resolution feature maps can result in coarse explanations, limiting fine-grained localization (observed in the DR dataset).
Architectural Constraints: The method requires the final feature map channels to match the classifier input size. This restricts applicability to architectures like ResNet and DenseNet, excluding models like VGG where the final layer structure differs significantly.
Future Work: The authors suggest extending this mechanism to Vision Transformers (ViTs) for built-in attention map explanations and addressing the resolution constraints.