Technical Summary: Beyond Accuracy in Malaria Diagnosis

Problem Statement

Malaria remains a leading cause of mortality in sub-Saharan Africa, where diagnostic infrastructure is often scarce. While deep learning (DL) offers a path toward automated screening, clinical adoption is hindered by two primary factors: the high computational cost of complex models and the "opacity" of their decision-making processes. In high-stakes medical settings, clinicians require not only accurate predictions but also mechanistic justifications for those predictions. Furthermore, existing evaluations often focus solely on predictive accuracy, neglecting critical dimensions such as model efficiency, robustness to image corruption (common in low-resource microscopy), and the reliability of post-hoc explanations.

Methodology

This study benchmarks four distinct deep learning architectures on the NLM-Malaria dataset, evaluating them across three dimensions: predictive performance, computational efficiency, and explainability.

1. Model Architectures:
The study compares models with varying design philosophies and capacities:

• ResNet-18: A standard CNN utilizing skip connections.
• EfficientNet-B0: A CNN emphasizing efficiency via compound scaling of depth, width, and resolution.
• MobileNet-v3-Large: A lightweight CNN using depthwise separable convolutions.
• Vision Transformer (ViT-B/16): A transformer-based architecture replacing convolutions with patch-based self-attention.
  All models were pre-trained on ImageNet and fine-tuned for binary classification (healthy vs. infected) using 5-fold stratified cross-validation.

2. Explainable AI (XAI) Framework:
Four post-hoc attribution methods were employed to generate saliency maps:

• Gradient-based: Grad-CAM and Integrated Gradients (Itgd-Grad).
• Gradient-free/Score-based: Score-CAM.
• Game-theoretic: SHAP (Gradient-based variant).
  These methods were used to visualize which image regions influenced the model's decision.

3. Evaluation Metrics:

• Predictive Performance: Accuracy, F1-score, AUC, Sensitivity, Specificity, PPV, and NPV.
• Efficiency: Parameter count, FLOPs, inference latency, and peak memory usage (measured on CPU).
• Robustness: Performance under three types of image corruption (Gaussian noise, Salt-and-Pepper noise, and Gaussian blur) at varying severity levels. Metrics included mean accuracy and mean predicted confidence.
• Explanation Quality:
  • Qualitative: Visual inspection of saliency maps and consistency analysis (agreement/disagreement maps) between different XAI methods.
  • Quantitative: Insertion and Deletion scores, measuring how quickly model confidence changes as important (or unimportant) image patches are revealed or occluded.
• Statistical Analysis: The Friedman test followed by Wilcoxon signed-rank tests with Bonferroni correction were used to determine statistical significance across models.

Key Results

1. Predictive Performance and Efficiency:

• All four architectures achieved comparable diagnostic performance. The Friedman test yielded a p-value of 0.4717, indicating no statistically significant differences in accuracy, F1-score, or AUC among the models.
• Lightweight models (MobileNet-v3 and EfficientNet-B0) matched the performance of heavier counterparts (ResNet-18 and ViT) while offering substantial efficiency gains. MobileNet-v3 demonstrated the lowest latency (14.37 ms) and FLOPs (0.22 G), whereas ViT-B/16 was the most computationally expensive (293.8 ms latency, 16.8 G FLOPs).
• An ensemble of the four models achieved the highest accuracy (0.972), but the marginal gain over individual lightweight models was small.

2. Robustness to Corruption:

• Models demonstrated strong robustness to Salt-and-Pepper noise but were more sensitive to Gaussian noise and blur.
• A critical finding was that model confidence degraded faster than accuracy under increasing noise levels. This suggests that confidence scores can serve as a practical signal for triggering human review when image quality is poor, even if the prediction remains technically correct.

3. Explainability and XAI Robustness:

• Localization: CAM-based methods (Grad-CAM, Score-CAM) consistently localized diagnostically relevant regions (the parasite). In contrast, fine-grained methods (Integrated Gradients, SHAP) produced less targeted explanations, often distributing high-attribution pixels broadly across the image, particularly with heavier backbones like ResNet-18.
• Consistency: There was structured disagreement between Grad-CAM and Score-CAM, concentrated in anatomically meaningful areas. Conversely, disagreement between Integrated Gradients and SHAP appeared as unstructured, diffuse noise.
• XAI Fragility: Crucially, no XAI method was robust to image corruption. Explanation reliability degraded significantly at noise levels plausible in clinical practice, even when the underlying predictions remained accurate. This indicates that explanations cannot be trusted as stable signals in noisy environments.
• ViT Behavior: The Vision Transformer focused on highly localized regions (specifically the chromatin dot), contrasting with the more diffuse activation patterns of CNNs.

Significance and Claims

The paper argues that for resource-constrained clinical settings, lightweight, efficient-by-design models (MobileNet-v3, EfficientNet-B0) are sufficient for malaria diagnosis, as they match the predictive performance of larger models without the computational overhead.

However, the study highlights a significant limitation in the current state of responsible AI deployment: the vulnerability of post-hoc explanations. The authors claim that while models may remain accurate under noise, their explanations become unreliable. This decoupling of prediction accuracy and explanation stability suggests that relying on XAI for clinical validation in noisy, real-world environments requires extreme caution. The work supports the deployment of efficient architectures but cautions that the interpretability of these systems is not guaranteed under the imperfect conditions typical of malaria-endemic regions.