Explaining Digital Pathology Models via Clustering Activations

Technical Summary: Explaining Digital Pathology Models via Clustering Activations

Problem Statement
The adoption of deep learning models in clinical digital pathology is hindered by a "trust gap." While Convolutional Neural Networks (CNNs) can accurately classify tissue slides (e.g., detecting prostate cancer), they function as opaque "black boxes." Pathologists are reluctant to trust these models because standard explainability techniques, such as occlusion, GradCAM, or relevance propagation, only provide one-dimensional, local saliency maps. These maps highlight regions contributing to a single prediction but fail to explain the reasoning behind the decision or the global behavior of the model. Unlike human pathologists who synthesize various morphological features and tissue types, current methods do not reveal the internal semantic representations the model uses to distinguish between healthy and cancerous tissue.

Methodology
The authors propose a clustering-based explainability technique that visualizes the global behavior of CNN models by grouping similar feature activations. The method operates as follows:

1. Feature Extraction: For a given CNN model, feature vectors are extracted from a specific convolutional layer $\ell$ for every spatial location $(i, j)$ of input image tiles. These vectors represent the activation values across all channels $C$.
2. Non-Negative Matrix Factorization (NMF): To ensure semantic interpretability, the authors employ NMF on the feature vectors. Since NMF requires non-negative inputs, they utilize the outputs of the layer after the ReLU activation function.
3. Clustering Process:
   • Training: A training matrix $V$ of feature vectors is decomposed into non-negative matrices $W$ and $H$ by minimizing the Frobenius norm $\|V - WH\|_F^2$. The matrix $H$ represents $K$ class vectors (cluster centroids), while $W$ represents the intensity (weight) of each class for every feature vector.
   • Inference: For new data, $H$ remains fixed, and $W$ is optimized. Each spatial location is assigned to the cluster with the highest intensity weight.
4. Whole Slide Image (WSI) Adaptation: To handle the massive scale of WSIs (hundreds of thousands of pixels) and the issue of tile overlap:
   • The 2-pixel border of feature vectors in each tile is discarded to mitigate border artifacts.
   • Feature vectors from overlapping tiles at the same spatial location are averaged to create a single, consistent representation per pixel.
   • Tiles with no tissue overlap (background) are discarded to focus computation on relevant areas.

Key Contributions

• Global vs. Local Explanation: Unlike saliency maps that highlight specific regions for a single decision, this method reveals the model's global behavior by clustering internal representations into distinct semantic categories.
• Semantic Segmentation without Supervision: The technique generates a semantic segmentation of the input image based on learned clusters, despite the underlying model being trained solely for binary classification (cancer vs. non-cancer) and never for segmentation.
• Handling WSI Challenges: The approach is specifically designed to manage the high dimensionality and sparse tissue distribution characteristic of digital pathology slides.
• Validation against Expert Knowledge: The method was evaluated on a prostate cancer detection model, demonstrating that the resulting clusters correspond to observable morphological features recognized by pathologists.

Results
The technique was evaluated using a prostate cancer detection model (VGG-16 backbone) trained on 87 WSIs from the Masaryk Memorial Cancer Institute. The authors trained clustering models with $K=4, 6,$ and $8$ classes.

• Pathologist Evaluation: A resident pathologist analyzed the $K=6$ clustering output and identified that the clusters corresponded to specific morphological structures:
  • Classes 2, 4, and 5 were identified as corresponding to carcinomatous areas (e.g., chains of densely packed nuclei, small lumina).
  • Classes 1, 3, and 6 corresponded to sparse tissue structures, edges, or normal epithelium.
• Quantitative Analysis:
  • A logistic regression model trained solely on the cluster weights achieved high performance in predicting cancer positivity (Accuracy: 0.984, AUC: 0.998).
  • The regression coefficients confirmed that classes 2, 4, and 5 strongly correlate with positive cancer labels, while classes 1, 3, and 6 correlate with negative labels, aligning with the pathologist's qualitative assessment.
• Comparison with GradCAM:
  • While classes 2, 4, and 5 showed high correlation with GradCAM heatmaps, the Intersection over Union (IoU) between the clusters and GradCAM masks was low.
  • This indicates that while the clusters overlap with the regions GradCAM deems important, the clustering method provides more nuanced, fine-grained information about what specific features within those regions the model is utilizing.

Significance and Claims
The paper claims that this clustering-based approach offers a robust tool for understanding the inner workings of CNNs in digital pathology. By moving beyond simple saliency maps to reveal the global semantic behavior of the model, the method helps demystify the "black box" nature of AI. The authors argue that providing interpretable insights into the specific morphological features a model prioritizes can significantly increase pathologist confidence, thereby facilitating the faster and safer adoption of AI models in clinical practice. The method is presented as applicable to any model architecture utilizing spatial feature vectors, with future work suggested for inspecting foundational transformer models.