🤖 AI

Efficient KernelSHAP Explanations for Patch-based 3D Medical Image Segmentation

This paper introduces an efficient KernelSHAP framework for patch-based 3D medical image segmentation that accelerates inference through logit caching and restricts computation to regions of interest, demonstrating that while regular supervoxels maximize perturbation metrics, organ-aware supervoxels provide superior clinical interpretability and effectiveness in identifying false-positive drivers.

Original authors: Ricardo Coimbra Brioso, Giulio Sichili, Damiano Dei, Nicola Lambri, Pietro Mancosu, Marta Scorsetti, Daniele Loiacono

Published 2026-04-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ricardo Coimbra Brioso, Giulio Sichili, Damiano Dei, Nicola Lambri, Pietro Mancosu, Marta Scorsetti, Daniele Loiacono

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

🏥 The Big Problem: The "Black Box" Doctor

Imagine a super-smart AI doctor that looks at 3D CT scans of your body to find tumors or map out organs for radiation therapy. It's incredibly accurate, but it's a "Black Box." It gives you an answer ("Here is the tumor"), but it won't tell you why it made that decision.

In medicine, knowing why is just as important as the answer. If the AI is wrong, we need to know if it was looking at the right spot or if it got confused by a weird shadow.

🧩 The Old Way: Too Slow to Ask "Why"

To figure out what the AI is thinking, researchers use a method called KernelSHAP. Think of this like a detective trying to solve a mystery by testing every possible clue.

The Detective's Job: The detective asks, "If I hide this part of the image, does the AI still see the tumor?" Then, "What if I hide that part?"
The Problem: A 3D medical scan is huge (millions of tiny pixels called voxels). If you try to test every single pixel or even small groups of pixels, the detective has to run the AI model thousands of times.
The Result: It's like trying to taste every single grain of sand on a beach to find a specific shell. It takes forever and is too expensive to do in real life.

⚡ The New Solution: The "Smart Detective"

The authors of this paper built a super-efficient version of this detective specifically for 3D medical scans. They made two major upgrades:

1. Zooming In (The "Spotlight" Strategy)

Instead of checking the entire body, the system asks the doctor to point a flashlight (a Region of Interest) at the specific area they are worried about.

The Analogy: Imagine you are looking for a lost ring in a messy room. Instead of checking the whole house, you only check the rug where you think it fell. The AI only does the heavy math on that specific rug and the immediate area around it, ignoring the rest of the house.

2. The "Cheat Sheet" (Patch Caching)

This is the real magic trick. The AI usually looks at the image in small chunks (patches), like a jigsaw puzzle.

The Old Way: Every time the detective hides a piece of the puzzle to test it, the AI has to re-solve the entire puzzle from scratch.
The New Way: The system keeps a Cheat Sheet (a cache) of the answers for the puzzle pieces that didn't change.
- If you hide a piece on the left side of the image, the AI looks at its Cheat Sheet and says, "Hey, the right side didn't change! I already know the answer for that." It reuses the old answer instead of doing the work again.
The Result: This saves a massive amount of time (15% to 30% faster), making it possible to run these tests in minutes instead of hours.

🧱 How They Group the Pixels (The "Lego" Analogy)

To make the explanation easy for a human to understand, the AI doesn't look at individual pixels. It groups them into "blocks." The paper tested three ways to build these blocks:

Full Organs (The "Big Blocks"):
- Analogy: Grouping pixels by whole organs (e.g., "The whole Liver" is one block).
- Pros: Very easy to understand. "The Liver is important."
- Cons: Too broad. It can't tell you which part of the liver caused a mistake.
Regular Grid (The "Uniform Bricks"):
- Analogy: Cutting the image into perfect, identical cubes, like a giant Lego structure, ignoring where the organs actually are.
- Pros: Mathematically very precise at finding errors.
- Cons: It cuts right through organs. A single block might be half-heart and half-lung. This is confusing for a doctor.
Hybrid (The "Smart Bricks"):
- Analogy: This is the winner. It starts with the uniform Lego bricks but then snaps them apart if they cross an organ boundary. So, you get a "Heart-Brick" and a "Lung-Brick" that fit perfectly together.
- Result: It gives you the mathematical precision of the grid but the anatomical sense of the organs. It's the best balance for doctors.

🎯 What They Found

Speed: The "Cheat Sheet" (caching) worked great. It made the process fast enough to be practical.
Accuracy vs. Clarity: The "Regular Grid" was great at mathematically proving what the AI saw, but the explanations looked like noise to a human. The "Hybrid" blocks gave explanations that made sense to doctors (e.g., "The AI got confused because of a shadow inside the liver, not the whole liver").
Finding Mistakes: The system was particularly good at spotting False Positives (when the AI thinks a healthy spot is a tumor). The Hybrid blocks helped pinpoint exactly where the AI was hallucinating.

🏁 The Bottom Line

This paper gives us a tool to open the "Black Box" of 3D medical AI. By being smart about where to look and by reusing old calculations, they made it possible to ask the AI, "Why did you do that?" in a way that is fast enough for real hospitals and clear enough for human doctors to trust.

In short: They built a faster, smarter detective that uses a cheat sheet to explain medical AI decisions without wasting time or confusing the doctors.

1. Problem Statement

Deep learning models, particularly nnU-Net, are the standard for 3D medical image segmentation (e.g., delineating organs-at-risk in radiotherapy). However, these models are "black boxes," making it difficult to audit errors or understand why a specific segmentation mask was generated.

The Challenge: Existing Explainable AI (XAI) methods face significant hurdles in 3D medical imaging:
- Gradient-based methods (e.g., Grad-CAM) are computationally efficient but often layer-dependent and struggle to provide model-agnostic, clinically meaningful insights in multi-structure settings.
- Perturbation-based methods (e.g., KernelSHAP) offer model-agnostic, faithful explanations by measuring the impact of feature removal. However, they are prohibitively expensive for 3D patch-based inference. A naive implementation requires thousands of forward passes over highly overlapping sliding-window patches to evaluate feature coalitions, leading to massive computational redundancy.
- Interpretability Gap: There is no consensus on how to define "interpretable units" in volumetric data (e.g., whole organs vs. geometric supervoxels) or how to aggregate voxel-level evidence into robust scores.

2. Methodology

The authors propose an efficient KernelSHAP framework tailored for patch-based 3D segmentation (specifically nnU-Net on CT data). The approach is built on three core pillars:

A. Localized Computation (ROI & Receptive Field)

Instead of explaining the entire 3D volume, the method focuses on a User-Defined Region of Interest (ROI) (e.g., a specific organ or a connected component of a prediction).

The system computes a minimal bounding box around the ROI.
It expands this box to include the Receptive Field (RF) support, ensuring that all input voxels capable of influencing the prediction within the ROI are included.
Perturbations are restricted to this $B_{RF}$ volume, while voxels outside remain fixed, drastically reducing the search space.

B. Efficient Inference via Patch Logit Caching

To overcome the computational bottleneck of KernelSHAP (which requires evaluating many coalitions):

Baseline Caching: The model first runs a single sliding-window inference on the unperturbed input within the $B_{RF}$ , storing the resulting patch logits in a CPU memory dictionary.
Conditional Recomputation: During coalition evaluation, if a perturbation (masking) does not affect a specific sliding-window patch, the system retrieves the cached baseline logits (Cache Hit).
Selective Forward Pass: Logits are only recomputed for patches that contain masked voxels (Cache Miss).
Fusion: The cached and recomputed logits are fused using the standard nnU-Net Gaussian-weighted accumulation scheme, ensuring the output remains identical to a full re-run.

C. Interpretable Units and Aggregation Strategies

The framework tests three types of feature partitions (units) and four scoring functions:

Interpretable Units:
1. Full Organs: Anatomical structures segmented by TotalSegmentator (high semantic meaning, low feature count).
2. Regular FCC: Face-Centred Cubic lattice tessellation (geometrically regular, organ-agnostic).
3. Hybrid: FCC cells subdivided by organ boundaries (combines geometric regularity with anatomical constraints).
Aggregation/Value Functions:
1. True Positive (TP): Rewards logit support for correctly predicted voxels.
2. False Positive (FP): Penalizes logit support for incorrectly predicted voxels.
3. Dice Score: Measures agreement between perturbed and baseline masks.
4. Soft Dice: A logit-based surrogate that rewards consistent voxels and penalizes new false positives.

3. Key Contributions

Efficient KernelSHAP for 3D: A novel framework that reduces computational cost by restricting analysis to the ROI/RF and utilizing patch logit caching, making Shapley value estimation feasible for 3D patch-based pipelines.
Systematic Evaluation of Units: A comparative study demonstrating that the choice of interpretable unit (Organ vs. Geometric vs. Hybrid) significantly impacts both the faithfulness (perturbation metrics) and clinical interpretability of the explanations.
Trade-off Analysis: Identification of a clear trade-off where geometric units (Regular FCC) maximize perturbation-based metrics (faithfulness) due to target overlap, while anatomical units (Hybrid/Organ) provide more clinically actionable insights, particularly for debugging false positives.

4. Experimental Results

The framework was evaluated on 40 whole-body CT scans (8 used for testing) for lymph node and spleen segmentation in radiotherapy planning.

Computational Efficiency:
- Patch caching yielded significant savings. Full Organs achieved a 32.4% cache hit rate, while Hybrid achieved 30.2%.
- Total runtime was reduced substantially compared to a naive implementation (e.g., Hybrid reduced runtime from a theoretical hours-long process to ~2 hours for 2000 coalitions).
Quantitative Metrics (MoRF/LeRF Curves):
- Regular FCC Supervoxels achieved the highest AOPC (Area Over Perturbation Curve) and ABPC (Area Between Perturbation Curves). However, the authors note this is partly because these units often overlap the target directly, making the model sensitive to their removal regardless of clinical relevance.
- Hybrid Supervoxels performed best under False Positive aggregation with normalized metrics, effectively localizing the specific sub-regions within an organ causing spurious activations.
Qualitative Findings:
- Full Organs provided stable, high-level explanations but lacked granularity.
- Hybrid units revealed heterogeneous contributions within single organs (e.g., one part of a spleen driving a false positive while another supports the prediction), offering superior debugging capabilities.
- FP-based scores were crucial for identifying "destabilizing" evidence, whereas TP/Dice scores highlighted "stabilizing" evidence.

5. Significance and Limitations

Significance:

This work bridges the gap between rigorous, model-agnostic explainability (KernelSHAP) and the practical constraints of 3D medical imaging.
It provides a clinically actionable workflow: By using Hybrid units and FP-aggregation, clinicians can pinpoint exactly where and why a model is making false positive errors, which is critical for safety in radiotherapy planning.
The caching mechanism makes Shapley-based explanations viable for real-world 3D datasets.

Limitations:

Validation Scale: Due to high computational costs, validation was limited to 8 test volumes.
Perturbation Artifacts: The use of hard masking (setting voxels to -1024 HU/air) may introduce out-of-distribution artifacts that do not perfectly mimic natural image variations.
Lack of Clinical Ground Truth: The study relies on quantitative faithfulness metrics (AOPC/ABPC) rather than direct clinician evaluation of the explanations' utility.
Fixed Grids: Current supervoxel strategies (FCC) may not perfectly align with complex anatomical boundaries, potentially biasing attributions.

Future Work:
The authors plan to explore data-driven, boundary-adherent 3D supervoxels (e.g., 3D SLIC) and more realistic in-distribution perturbations (e.g., inpainting) to further improve anatomical alignment and clinical relevance.