💻 computer science

Swin Transformer Based Multi-Label Chest Disease Classification

This paper presents a comprehensive benchmarking study on the NIH ChestX-Ray14 dataset demonstrating that a Swin Transformer V2-B framework, enhanced by specific strategies for class imbalance and threshold optimization, outperforms existing state-of-the-art deep learning models in multi-label chest disease classification.

Original authors: Abdullah Bakr, Ashraf Ullah, Abdul Jabbar

Published 2026-09-08

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Abdullah Bakr, Ashraf Ullah, Abdul Jabbar

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Every day, millions of people walk into clinics and hospitals for a simple chest X-ray, a procedure that has become the most common way doctors look inside the human body. These images are vital for spotting conditions like pneumonia, fluid around the lungs, or an enlarged heart, yet reading them is a demanding task. A single image can hide multiple diseases at once, and even experienced specialists can miss subtle signs or struggle to distinguish between conditions that look nearly identical. While artificial intelligence has shown great promise in helping doctors interpret these scans, a major hurdle remains: the data used to teach these computers is often unbalanced. Some diseases appear frequently in medical records, while others are rare, causing standard computer programs to become experts at spotting the common issues but failing when faced with the uncommon ones.

A team of researchers set out to solve this problem by testing a new generation of artificial intelligence against older, established methods. They focused on a massive collection of over 112,000 chest X-rays from more than 30,000 patients, a dataset that includes images labeled with up to fourteen different diseases. The goal was not just to build a single model, but to run a fair, head-to-head comparison of six different types of deep learning systems. These systems ranged from traditional designs, which process images by scanning them with small filters like a human eye moving across a page, to newer architectures that use a mechanism called "attention" to look at the entire image at once and decide which parts matter most. The researchers also introduced a specific training technique designed to force the computer to pay extra attention to the rare diseases, ensuring it did not simply ignore them in favor of the common ones.

The study revealed that a newer type of architecture, known as the Swin Transformer, significantly outperformed all other methods tested. Unlike older models that might miss the forest for the trees, or newer models that look at the whole picture but struggle with fine details, this winning system combines the best of both worlds. It breaks the image into small windows and analyzes them locally, then shifts its focus to connect those windows together, allowing it to see both the tiny details of a small lung nodule and the broad shape of an enlarged heart simultaneously. When tested on the same set of images, this model achieved a higher accuracy score than any of the five other systems, including a famous benchmark model that has been the standard for years. It successfully identified the presence of diseases across the board, proving that this specific way of processing visual information is better suited for the complex, multi-layered nature of chest X-rays.

To ensure the results were trustworthy, the researchers took great care to prevent the computer from memorizing the answers. They split the data so that images from the same patient never appeared in both the training group and the testing group, guaranteeing that the system was truly learning to recognize disease patterns rather than just recognizing specific people. They also adjusted the system's decision-making process for each disease individually, fine-tuning the sensitivity so it would not miss a rare condition just because it was uncommon. The results showed that while the new model took longer to train than some of the simpler systems, the extra time yielded a more reliable diagnosis. The researchers also used a visualization tool to look inside the model's "mind," confirming that when it flagged a disease, it was actually focusing on the relevant parts of the lung or heart, rather than random background noise.

The findings suggest that the future of automated medical diagnosis may lie in these more sophisticated, attention-based systems. While the new model did not solve every challenge—some difficult-to-spot diseases still proved tricky, and the overall accuracy scores, while the best in the study, left room for improvement—it demonstrated a clear advantage over the traditional methods that have dominated the field for years. By showing that a system capable of understanding both local details and global context can better handle the messy reality of multiple co-occurring diseases, this work provides a strong foundation for building tools that can one day assist doctors in making faster, more accurate decisions for patients around the world.

Technical Summary: Swin Transformer Based Multi-Label Chest Disease Classification

Problem Statement
The paper addresses the challenges inherent in the automated multi-label classification of chest X-rays (CXR), specifically focusing on two critical issues: class imbalance and the complexity of multi-label prediction. In clinical practice, a single chest radiograph often presents multiple co-occurring pathologies (e.g., cardiomegaly, pleural effusion, and pneumonia simultaneously), requiring models to predict multiple independent labels rather than a single class. Furthermore, the distribution of diseases in medical datasets is highly skewed, with "No Finding" cases dominating and rare diseases representing a small fraction of the data. Existing studies often compare models under inconsistent training conditions or focus on single architectures, making fair benchmarking difficult. The goal is to develop a robust framework that accurately classifies 14 distinct thoracic diseases while mitigating the effects of data imbalance and preventing data leakage.

Methodology
The study presents a comprehensive benchmarking framework evaluated on the NIH ChestX-Ray14 dataset, comprising 112,120 frontal-view radiographs from 30,805 patients. The methodology is structured around five core modules:

Data Handling and Pre-processing:
- Patient-Level Splitting: To prevent data leakage, the dataset is split at the patient level (Training: 69,219; Validation: 17,305; Test: 25,596), ensuring images from the same patient do not appear in both training and testing sets.
- Pre-processing: Images are resized to architecture-specific dimensions (e.g., 224×224 for ResNet-50, 256×256 for Swin Transformer V2-B) and normalized using ImageNet statistics.
- Augmentation: Training data undergoes random horizontal flipping, rotation (±10°), and color jittering to mitigate overfitting.
Model Architecture and Training:
- Comparative Study: Six deep learning architectures are evaluated under identical conditions: four CNNs (ResNet-50, VGG-19, DenseNet-121, Inception V3) and two Transformer-based models (ViT-B/16, Swin Transformer V2-B).
- Output Layer: All models are adapted for multi-label classification by replacing the final layer with a 14-output sigmoid head.
- Loss Function: To address class imbalance, Focal Loss (with $\alpha=1, \gamma=2$ ) is employed instead of standard Binary Cross-Entropy. This down-weights easy examples and forces the model to focus on hard, misclassified cases.
- Optimization: Models are trained using the AdamW optimizer, OneCycleLR learning rate scheduling, and Mixed Precision (AMP).
Post-Processing and Evaluation:
- Threshold Optimization: Per-class thresholds are optimized on the validation set to maximize the F1-score for each disease independently.
- Metrics: Performance is assessed using AUC-ROC, F1-score, Precision, Recall, and Confusion Matrices.
- Explainability: Grad-CAM++ is applied to the proposed Swin Transformer model to generate heatmaps, verifying that the model attends to clinically relevant anatomical regions.

Key Contributions
The paper claims the following specific contributions:

Rigorous Benchmarking: It validates six state-of-the-art deep learning models under strictly identical training conditions on the NIH ChestX-Ray14 dataset, providing a fair comparison between CNNs and Transformers.
Proposed Framework Superiority: The study demonstrates that the Swin Transformer V2-B outperforms the other state-of-the-art benchmark DL techniques and the CheXNet baseline when evaluated under identical single-model conditions.
Data Integrity: The implementation of patient-level data splitting ensures the elimination of data leakage, a common pitfall in medical imaging studies.
Optimization Strategies: The application of Focal Loss for class imbalance and per-class threshold optimization for all six models enhances classification performance.
Explainability: The integration of Grad-CAM++ with a custom multi-label target provides visual evidence of the model's decision-making process.

Results
Extensive experiments yield the following findings:

Performance: The Swin Transformer V2-B achieves the highest mean AUC of 0.8495, outperforming all other benchmarked models. It surpasses the second-best model (VGG-19, AUC 0.8341) and the Transformer baseline ViT-B/16 (AUC 0.8312).
Architecture Comparison: Transformer-based architectures (average AUC 0.8409) generally outperform CNN-based architectures (average AUC 0.8214) across all metrics, suggesting that attention mechanisms better capture the long-range spatial dependencies required for thoracic disease analysis.
Per-Class Performance: Swin V2-B shows strong performance on visually distinct and well-represented diseases (e.g., Hernia AUC 0.935, Cardiomegaly AUC 0.915) but faces challenges with visually subtle pathologies like Infiltration (AUC 0.729) and Nodule (AUC 0.792).
Efficiency Trade-off: While Swin V2-B offers the highest accuracy, it requires the longest training time (5.83 hours) compared to ViT-B/16 (2.57 hours) and other CNNs, highlighting a trade-off between computational cost and performance gains.
Comparison with CheXNet: The study's DenseNet-121 implementation achieved an AUC of 0.8163, lower than the original CheXNet's 0.8410. The authors attribute this difference to CheXNet's use of model ensembling and additional training resources, noting that their Swin V2-B results were achieved in a single-model, single-run setting.

Significance and Claims
The paper concludes that Swin Transformer V2-B represents a significant advancement for multi-label chest disease classification. The authors argue that the hierarchical shifted-window attention mechanism of the Swin Transformer is particularly effective for medical imaging, as it captures both fine-grained local details (e.g., nodules) and broader structural context (e.g., cardiomegaly) more effectively than global attention (ViT) or purely convolutional filters (CNNs).

The study emphasizes that while Focal Loss and threshold optimization mitigate class imbalance, the difficulty of distinguishing visually ambiguous pathologies remains a challenge, as evidenced by the relatively low F1-scores (0.23–0.31) across all models. The authors modestly claim that their work provides a robust baseline for future research, demonstrating the superiority of Transformer architectures in this domain without relying on ensembling techniques. They acknowledge limitations, including the lack of external validation on other datasets and the absence of quantitative explainability evaluation against radiologist annotations.

Technical Summary: Swin Transformer Based Multi-Label Chest Disease Classification

More like this