Weakly Supervised Pneumonia Localization from Chest X-Rays Using Deep Neural Network and Grad-CAM Explanations

Technical Summary: Weakly Supervised Pneumonia Localization from Chest X-Rays

Problem Statement
Pneumonia remains a leading cause of global morbidity and mortality, with chest X-rays serving as the primary diagnostic tool. However, accurate diagnosis is often hindered by subtle lesions, radiologist variability, and the "black-box" nature of deep learning models. While Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) have shown state-of-the-art classification capabilities, they typically lack interpretability. Furthermore, fully supervised localization methods require costly, pixel-level annotations (e.g., segmentation masks or bounding boxes), which are difficult to acquire at scale. This study addresses the dual challenge of achieving high diagnostic accuracy without pixel-level labels while providing clinically meaningful, transparent explanations for model decisions.

Methodology
The authors propose a unified weakly supervised learning (WSL) framework that utilizes only image-level labels (normal vs. pneumonia) to perform both classification and spatial localization. The core methodology involves:

• Dataset Curation: The study utilizes the publicly available Chest X-ray dataset [3]. To prevent data leakage, the authors re-partitioned the data at the patient level (70% training, 15% validation, 15% testing), ensuring no patient IDs appeared in multiple splits. Images were resized to 224×224, converted to pseudo-RGB, and augmented with rotations, flips, brightness adjustments, and noise.
• Model Benchmarking: Seven pre-trained architectures were evaluated under identical training conditions:
  • CNNs: ResNet-18, ResNet-50, DenseNet-121, EfficientNet-B0, MobileNet-V2, and MobileNet-V3.
  • Transformer: Vision Transformer (ViT-B/16).
• Training Strategy: All models were initialized with ImageNet weights. To address the severe class imbalance (404 normal vs. 3,692 pneumonia images), the authors employed Focal Loss (down-weighting easy samples) and random over-sampling of the minority class. Training utilized the Adam optimizer with early stopping based on validation ROC-AUC.
• Localization via Grad-CAM: To generate heatmaps without pixel-level ground truth, the study applied Gradient-weighted Class Activation Mapping (Grad-CAM).
  • For CNNs, gradients were computed from the last convolutional layer.
  • For ViT, the formulation was adapted to capture activations and gradients from the final MLP block of the last transformer encoder, treating patch embeddings as spatial units.
• Quantitative Evaluation: Since pixel-level masks were unavailable, a Lung Attention Ratio (LAR) was introduced. This metric calculates the proportion of Grad-CAM activation energy falling within a coarse, fixed lung region of interest (ROI), serving as a proxy for anatomical consistency.

Key Results

• Classification Performance: All seven models achieved high discriminative performance, with test accuracies ranging from 96% to 98%.
  • ResNet-18 and EfficientNet-B0 achieved the highest test accuracy (98%) and F1-scores (0.987), with ROC-AUC and PR-AUC values exceeding 0.997.
  • MobileNet-V3 delivered comparable accuracy (97%) with a significantly lower parameter count (4.9M), demonstrating its suitability for resource-constrained environments.
  • Deeper models like ResNet-50 and DenseNet-121 showed slightly lower generalization, suggesting mild overfitting.
• Localization and Interpretability:
  • Grad-CAM visualizations confirmed that models focused on radiologically relevant opacities in the middle and lower lung zones for pneumonia cases.
  • MobileNet-V3 exhibited the most compact and noise-free localization, with a high Lung Attention Ratio (LAR) of 0.693 for bacterial pneumonia and low variance.
  • ViT-B/16 showed strong discrimination between normal and bacterial pneumonia (LAR 0.172 vs. 0.605) but displayed broader, more diffuse attention patterns for viral pneumonia, consistent with global self-attention mechanisms.
  • DenseNet-121 occasionally showed spurious activations in normal cases, while EfficientNet-B0 and ResNet-50 displayed some diffuse or off-target hotspots.

Key Contributions

1. Rigorous Benchmarking: The study provides a fair comparison of six CNN architectures and one Vision Transformer under identical training protocols, including patient-wise data splitting to eliminate leakage.
2. Weakly Supervised Localization: It demonstrates that clinically meaningful heatmaps can be generated using only image-level labels via Grad-CAM, removing the dependency on expensive pixel-level annotations.
3. Quantitative Interpretability: The introduction of the Lung Attention Ratio (LAR) offers a lightweight, quantitative method to evaluate the anatomical consistency of explainable AI outputs in the absence of ground-truth segmentation.
4. Efficiency Analysis: The work identifies MobileNet-V3 as an optimal trade-off between diagnostic accuracy and computational cost, validating its potential for real-time, edge, and mobile health applications.

Significance and Claims
The paper claims that explainable, weakly supervised deep learning methods can effectively bridge the gap between "black-box" image classification and clinically interpretable decision support. By validating that models focus on relevant lung regions without manual segmentation training, the study supports the use of such frameworks to enhance transparency and clinical trust in AI-assisted pneumonia screening. The authors conclude that compact architectures, when combined with focal loss and patient-wise splitting, can achieve near state-of-the-art accuracy while remaining computationally efficient, facilitating deployment in diverse clinical settings.

Future Directions
The authors modestly note that the current research is limited to a single dataset. They suggest future work should involve evaluation on larger, more diverse datasets (e.g., RSNA Pneumonia, NIH ChestX-ray14) to enhance robustness. Additionally, they propose exploring multi-label thoracic disease localization, radiologist reader studies, and further mobile deployment optimizations to strengthen the framework's translational impact.