Bridging the Gap: Explainability Metrics for AI Image-Based Clinical Diagnostics

Technical Summary: Bridging the Gap: Explainability Metrics for AI Image-Based Clinical Diagnostics

Problem Statement
While Artificial Intelligence (AI), particularly Convolutional Neural Networks (CNNs), demonstrates potential in enhancing diagnostic accuracy for medical imaging, its integration into clinical workflows is hindered by the "black-box" nature of these models. Clinicians often lack trust in systems that cannot provide interpretable reasoning aligned with clinical judgment. Current evaluation frameworks rely heavily on statistical metrics (e.g., accuracy, sensitivity, specificity), which quantify predictive correctness but fail to assess whether the model's internal reasoning aligns with human expert logic. This creates a critical gap: a model may achieve high accuracy while relying on spurious correlations or irrelevant image features, rendering it clinically untrustworthy despite its statistical performance.

Methodology
The study proposes a staged evaluation framework to assess AI performance in diagnosing infections in chronic wounds, integrating explainability as a core metric.

1. Dataset and Model Training: The authors utilized a public dataset of chronic wound images (545 infected, 697 non-infected) acquired under real-world clinical conditions. Three CNN architectures—ResNet-50, EfficientNet V2B0, and DenseNet121—were trained using transfer learning. To address variability in lighting and acquisition devices, a GAN-based illumination normalization network (Pix2Pix-based) was applied as a preprocessing step. The models were fine-tuned using standard techniques (dropout, early stopping, learning rate reduction) and evaluated on a held-out test set of 200 images.
2. Explainability Techniques: Two post-hoc explainability methods were applied to a subset of 10 correctly classified infected wound images:
   • Saliency Maps: A gradient-based method computing the partial derivative of the output with respect to input pixels to identify regions of high influence.
   • LIME (Local Interpretable Model-agnostic Explanations): A perturbation-based method that segments images into superpixels (using SLIC clustering), masks them, and trains a local linear surrogate model to approximate the CNN's behavior.
3. Clinical Ground Truth: Two chronic wound specialists independently reviewed the same 10 images and reached a consensus on the specific pixel regions indicative of infection. These annotations served as the clinical reference standard.
4. Proposed Metric (Explainability Agreement Index - EAI): To quantify the alignment between AI explanations and clinical reasoning, the authors introduced the Explainability Agreement Index (EAI). The EAI is calculated for true positive cases where both the AI and clinicians agree on the diagnosis. It is defined as the ratio of pixels identified as influential by the explainability method (LIME or Saliency) that overlap with the clinician-annotated regions, divided by the total number of clinician-annotated pixels.
   • Formula Concept: $EAI = \frac{|Pixels_{AI} \cap Pixels_{Clinician}|}{|Pixels_{Clinician}|}$

Key Results

• Diagnostic Performance: The CNN models achieved moderate classification accuracy on the test set, with EfficientNet V2B0 performing best at 72% accuracy, 64% sensitivity, and 80% specificity. ResNet-50 achieved 71% accuracy, and DenseNet121 achieved 66%.
• Explainability Performance: Despite reasonable classification accuracy, the alignment between AI-generated explanations and clinical reasoning was low.
  • LIME: Mean EAI values ranged from 0.2119 (ResNet-50) to 0.2999 (EfficientNet V2B0).
  • Saliency Maps: Mean EAI values ranged from 0.1946 (DenseNet121) to 0.2811 (ResNet-50).
• Discrepancy: The low EAI scores (ranging approximately between 0.19 and 0.30) indicate that while the models correctly identified infected wounds, the visual regions they highlighted as "important" for the decision rarely matched the regions clinicians identified as diagnostically relevant.

Key Contributions

1. Introduction of the Explainability Agreement Index (EAI): The paper proposes a novel, quantitative metric designed to measure the concordance between AI-generated visual explanations and clinician annotations. Unlike traditional metrics that only assess prediction correctness, EAI evaluates the reasoning behind the prediction.
2. Empirical Validation of the "Accuracy-Explainability" Gap: The study provides concrete evidence that high classification accuracy does not guarantee clinical interpretability. The significant disparity between accuracy (up to 72%) and EAI (approx. 0.20–0.30) highlights that current models may be "right for the wrong reasons."
3. Evaluation of Standard xAI Methods in Clinical Context: The study demonstrates that widely used explainability techniques (LIME and Saliency maps) struggle to consistently capture clinically relevant features in complex medical imaging tasks like wound assessment, suggesting a need for domain-specific refinement.

Significance and Claims
The authors assert that the proposed EAI framework offers a practical pathway to bridge the gap between algorithmic performance and clinical trustworthiness. The study claims that for AI to be safely integrated into healthcare, validation must extend beyond statistical metrics to include the alignment of model reasoning with clinical expertise.

The paper modestly concludes that while the EAI is a step forward, current explainability methods are insufficient for clinical deployment without further refinement. It emphasizes that explainability is a necessary, though not sufficient, dimension of trustworthy AI, noting that factors like model calibration and uncertainty estimation also play critical roles. The authors position the EAI not as a final solution, but as a tool to quantitatively assess and drive improvements in the transparency of medical AI systems.