💻 computer science

Squeeze-and-Excitation Channel Attention in a Compact CNN for 24-Class Clinical Skin Disease Classification

This paper presents a compact, 5.01M-parameter CNN enhanced with Squeeze-and-Excitation channel attention that achieves 88.55% accuracy on a balanced 24-class clinical skin disease dataset, demonstrating significant performance gains over baseline models while acknowledging specific evaluation limitations prior to clinical deployment.

Original author: Md. Ariful Islam

Published 2026-11-10

📖 4 min read☕ Coffee break read

CC BY 4.0

Original author: Md. Ariful Islam

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the author. For technical accuracy, refer to the original paper. Read full disclaimer

In many parts of the world, particularly in rural areas with few medical specialists, a person with a skin condition often waits weeks to see a doctor or tries to treat the problem themselves with over-the-counter creams. This gap in care has spurred researchers to build computer programs that can look at a photograph of a skin lesion and suggest a diagnosis. These programs rely on a type of artificial intelligence called a convolutional neural network, which is essentially a digital system designed to recognize patterns in images much like the human eye and brain do. To make these systems smarter without making them too heavy to run on the modest computers found in small clinics, scientists have begun adding "attention" mechanisms. Think of this as a way for the computer to learn which parts of an image matter most, allowing it to focus on the specific texture or color of a rash while ignoring the surrounding skin or background clutter. The question driving recent work is whether these attention tools can significantly improve a small, efficient computer program trained from the ground up to recognize a wide variety of skin diseases.

A researcher at East Delta University in Bangladesh set out to answer this question by building a compact computer model specifically for classifying clinical photographs of skin. The goal was to create a system that is accurate enough to be useful but light enough to run on hardware that small clinics actually possess, rather than relying on massive, pre-trained systems that require powerful servers. To train this model, the researcher gathered a public collection of skin disease images and organized them into twenty-four distinct categories, ranging from common inflammatory conditions like eczema to infections like chickenpox, as well as a category for healthy skin. To ensure the computer learned fairly, the researcher carefully balanced the dataset by selecting exactly the same number of images for every single category, resulting in a total of 68,568 photographs. The computer was then taught to recognize these twenty-four classes from scratch, without any prior knowledge from other tasks, using a specific architectural design that included the attention mechanism in every stage of its learning process.

The results showed that this approach worked remarkably well. When tested on a separate set of images it had never seen before, the model correctly identified the skin condition in nearly 89 percent of cases. This performance was significantly better than what would be expected by random guessing, and it outperformed several other standard, well-known computer architectures that were tested under the same conditions. The study included a controlled experiment to prove that the improvement came specifically from the attention mechanism. When the researcher removed the attention blocks from the model and trained the rest of the system exactly the same way, the accuracy dropped noticeably. Adding the attention blocks back in boosted the performance by nearly nine percentage points, all while increasing the total size of the computer program by less than one percent. This suggests that the ability to focus on the most important features of an image is a highly efficient way to improve accuracy without demanding more computing power.

Despite these strong numbers, the researcher is careful to note that the study has specific limits before the system can be used in a real hospital. The images used for testing were split randomly, meaning that photos of the same patient or the same lesion could accidentally end up in both the training and testing groups, which might make the results look better than they truly are. Furthermore, the dataset did not include information about the skin tone of the patients, so it is unknown how well the system would work on darker skin types, which are common in South Asia. The model also did not address skin cancers, as those specific categories were excluded from the training data to keep the focus on other common conditions. The researcher concludes that while this compact, attention-focused model is a promising starting point for helping doctors in low-resource settings, it requires further testing on separate groups of patients and diverse skin types before it can be trusted for clinical use. The work demonstrates that with the right design, small, efficient computer programs can learn to see skin diseases with a high degree of accuracy, offering a potential tool to bridge the gap where dermatologists are scarce.

Technical Summary: Squeeze-and-Excitation Channel Attention in a Compact CNN for 24-Class Clinical Skin Disease Classification

Problem Statement
Access to dermatologists is severely limited in low-resource regions, such as rural Bangladesh, creating a critical need for automated triage tools that are both accurate and computationally lightweight enough to run on modest hardware. While large-scale Convolutional Neural Networks (CNNs) have demonstrated expert-level performance in dermatology, they often rely on heavy pre-trained backbones and transfer learning. This study addresses the specific question of how much Squeeze-and-Excitation (SE) channel attention contributes to a compact CNN trained from scratch on a broad, multi-class set of clinical skin photographs, aiming to balance performance with model size.

Methodology
The research proposes a compact SE-CNN architecture and a rigorous experimental protocol:

Dataset Curation: A class-balanced dataset of 68,568 clinical skin-disease images was constructed from a public Kaggle collection. The author excluded generic or overlapping tumor folders (e.g., melanoma, basal cell carcinoma) to focus on 24 specific classes: 23 disease categories (including inflammatory conditions, infections, and vascular lesions) and one healthy skin class. Each category was randomly undersampled to exactly 2,857 images to ensure perfect balance without synthetic duplication.
Network Architecture: The proposed SE-CNN is trained from scratch without transfer learning. It consists of four convolutional stages with filter counts of 64, 128, 256, and 512. Each stage includes two 3×3 convolutions with ReLU and Batch Normalization, followed by a Squeeze-and-Excitation (SE) block and 2×2 max pooling. The SE block learns channel weights via global average pooling and a two-layer bottleneck (reduction ratio $r=16$ ), rescaling feature maps to emphasize informative channels. The network concludes with global average pooling, a fully connected layer, dropout, and a 24-way softmax. The total parameter count is 5.01 million.
Training Protocol: Models were trained using the Adam optimizer, categorical cross-entropy loss, and a batch size of 64. Data augmentation (rotation, zoom, shifts, shear, flipping) was applied only to the training set. The dataset was split 80/20 into training and held-out sets. Early stopping was monitored on the held-out split.
Comparative Analysis: The study employed a controlled ablation study on a reduced subset of the data to isolate the impact of SE blocks. A "Baseline CNN" identical to the SE-CNN but without SE blocks was trained under identical conditions. Additionally, standard architectures (MobileNetV2, DenseNet121, ResNet50, EfficientNetB0) were trained using the same pipeline for comparison.

Key Contributions

Compact SE-CNN: Development of a 5.01 M parameter network for 24-class clinical skin disease classification trained entirely from scratch.
Balanced Benchmark: Creation of a 68,568-image, perfectly class-balanced dataset with no duplicated images across partitions.
Controlled Ablation: An experimental isolation of the SE block's contribution, demonstrating performance gains with negligible parameter overhead.
Explicit Limitations: A transparent accounting of evaluation constraints, including image-level splitting and the reuse of the held-out set for early stopping.

Results

Full Dataset Performance: On the held-out split (13,728 images), the SE-CNN achieved 88.55% top-1 accuracy and a macro F1 score of 0.888. The micro-averaged ROC AUC was 1.00. Performance varied by class; healthy skin and specific infections (e.g., chickenpox, shingles) achieved F1 scores ≥0.99, while eczema and psoriasis/lichen planus showed lower performance (F1 ~0.61–0.66) due to visual overlap.
Ablation Study: On a reduced subset, adding SE blocks to the baseline CNN increased validation accuracy from 64.2% to 73.1% and test macro F1 from 0.49 to 0.59. This improvement was achieved with less than a 1% increase in parameter count.
Baseline Comparison: On the same subset, the SE-CNN (72.9% validation accuracy) significantly outperformed MobileNetV2 (50.8%) and DenseNet121 (46.5%). ResNet50 and EfficientNetB0 failed to converge under the shared training pipeline, suggesting incompatibility with the specific data scaling and augmentation rather than inherent architectural weakness.

Significance and Claims
The paper claims that the ablation study provides direct evidence that SE channel attention adds approximately nine percentage points to validation accuracy and ten points to test F1 for less than 1% additional parameters. This supports the utility of lightweight attention modules for dermatology triage in low-resource settings where model size is a constraint.

The author explicitly states that the results are a "starting point" for clinical triage but are not yet ready for clinical deployment. They highlight several limitations:

Data Leakage Risk: The use of image-level splitting (rather than patient-level) may inflate performance metrics if multiple images of the same patient appear in both training and test sets.
Validation Bias: The held-out split was reused for early stopping, meaning it is not a fully independent test set.
Generalizability: The lack of skin-tone metadata prevents assessment of performance on darker skin types, which are prevalent in the target demographic.
Source Artifacts: The near-perfect classification of certain classes may be partly due to the model recognizing source-specific image prefixes rather than disease features.

Future work is proposed to address these issues through patient-level separation, external validation on datasets like Fitzpatrick 17k, and the inclusion of saliency maps (e.g., Grad-CAM) to verify that the model attends to lesions rather than background artifacts.

Technical Summary: Squeeze-and-Excitation Channel Attention in a Compact CNN for 24-Class Clinical Skin Disease Classification

More like this