Melanoma Classification Through Deep Ensemble Learning and Explainable AI

This paper proposes a reliable melanoma classification system that combines an ensemble of three state-of-the-art deep transfer learning networks with Explainable AI (XAI) techniques to overcome the "black box" limitations of traditional deep learning models and enhance diagnostic trust.

The Gist

Imagine your skin is like a vast, complex landscape. Sometimes, small "storms" (skin lesions) appear on this landscape. Most of these storms are harmless clouds (benign), but a few are dangerous hurricanes (melanoma). Catching that hurricane early is the difference between a safe day and a life-threatening crisis.

Currently, doctors act like expert storm chasers. They look at pictures of these skin spots through a special magnifying glass (dermoscopy) to tell the difference. But even the best chasers can get tired, distracted, or see things differently. Sometimes, they need to cut a piece of the skin out (a biopsy) to be sure, which is painful and risky.

This paper introduces a new kind of Digital Storm Chaser built by computers. Here is how it works, broken down into simple parts:

1. The Problem: The "Black Box" Mystery

Artificial Intelligence (AI) has gotten really good at spotting these storms. It can look at a picture and say, "That's a hurricane!" with high accuracy. But there's a catch: AI is a "Black Box."

Imagine a wizard who predicts the weather perfectly but refuses to tell you why. You might trust them once, but if they make a mistake, you have no idea what they were looking at. In medicine, doctors can't just trust a magic box; they need to know why the AI thinks a spot is dangerous. If the AI is looking at a stray hair or a weird shadow instead of the actual tumor, that's a disaster.

2. The Solution: The "Council of Experts" (Ensemble Learning)

Instead of relying on one super-smart AI, the authors built a Council of Experts. They took three different, highly trained AI models (named ResNet-101, DenseNet-121, and Inception v3) and put them in a room together.

• The Analogy: Think of it like a jury. If one juror makes a mistake, the others might catch it.
• How they vote: They tried different ways to let the jury decide:
  • Majority Vote: "Who has the most votes?"
  • Average Opinion: "Let's take the average confidence of all three."
  • Weighted Opinion: "Let's listen more to the expert who has been right the most often."

The team found that giving a weighted vote (listening more to the most accurate experts) worked the best. This "Council" was more accurate than any single expert working alone.

3. The Training: Cleaning the Messy Data

The AI needed to learn, but the data it was given was messy.

• The Imbalance: Imagine a classroom with 100 students, but 98 are wearing blue shirts (benign spots) and only 2 are wearing red shirts (cancerous spots). The AI would just guess "Blue" every time and be "right" 98% of the time, but it would miss every single cancer case. The researchers had to balance the class so the AI learned to spot the red shirts, too.
• The Noise: The photos were often blurry, dark, or had hair covering the spot. The researchers used digital tools to brighten the images, sharpen the edges, and crop out the distractions, giving the AI a clearer view.

4. The Magic Trick: Explainable AI (XAI)

This is the most important part. The authors didn't just want the AI to give an answer; they wanted it to show its work.

They used a tool called SHAP (which sounds like a friendly name but is actually a complex math method).

• The Analogy: Imagine the AI is looking at a photo of a skin spot. The SHAP tool acts like a highlighter pen.
  • It paints the parts of the image the AI thinks are dangerous in Red.
  • It paints the parts it thinks are safe in Blue.

What they discovered:

• Good News: The AI correctly highlighted the actual irregular edges of the skin lesion (the real storm). This builds trust.
• Bad News: The AI sometimes got distracted!
  • In some photos, it highlighted hair crossing the spot, thinking the hair made it dangerous.
  • In others, it got confused by circular shadows (like a camera lens effect) and thought those were part of the tumor.
  • It even looked at the healthy skin around the spot, which shouldn't matter as much.

5. The Result

By combining the "Council of Experts" with the "Highlighter Pen" (SHAP), the system achieved:

• High Accuracy: It correctly identified cancerous spots about 86% of the time.
• Trust: Doctors can now see where the AI is looking. If the AI highlights the hair instead of the tumor, the doctor knows to ignore the AI's suggestion for that specific case.

The Bottom Line

This paper is about building a smarter, more honest medical assistant. It doesn't just say "Yes" or "No"; it points to the evidence. While it's not perfect yet (it still gets confused by hair and shadows), it's a huge step toward a future where AI helps doctors catch deadly skin cancer earlier, with less pain for the patient and more trust in the diagnosis.

Future Goal: The researchers plan to teach the AI to ignore the hair and shadows even better, so it focuses only on the real danger.

Technical Summary

1. Problem Statement

Melanoma is the most aggressive form of skin cancer, with early detection being critical for a >99% 5-year survival rate. However, current diagnostic methods face significant challenges:

• Clinical Limitations: Visual inspection via dermoscopy has an accuracy of only ~80% and varies between specialists. Biopsies are invasive, costly, and carry infection risks.
• Data Challenges: Automated Deep Learning (DL) approaches struggle with class imbalance (malignant cases are rare compared to benign ones), data scarcity (lack of annotated data), and image quality issues (low contrast, noise, irregular borders).
• The "Black Box" Issue: While DL models achieve high accuracy, their lack of transparency (explainability) hinders trust and adoption in clinical settings. Medical professionals need to understand why a model makes a specific prediction to validate its reliability.

2. Methodology

The authors propose a machine learning framework that combines Deep Ensemble Learning with Explainable AI (XAI). The methodology consists of five phases:

A. Data Preparation

• Datasets: Utilized the ISIC Challenge 2019 and ISIC Challenge 2020 datasets, containing thousands of dermoscopy images labeled as malignant (melanoma) or benign.
• Imbalance Handling: The datasets exhibited extreme class imbalance (e.g., 1.8% malignant in ISIC 2020). The authors addressed this by down-sampling the majority class (benign) to create a balanced dataset of 10,212 images (5,106 benign, 5,106 malignant).
• Pre-processing: Images underwent enhancement to improve contrast and sharpness using PIL and OpenCV (color enhancement, sharpness, brightness, contrast). Images were center-cropped and normalized (divided by 255) to reduce position variance and training time.

B. Base Model Training

Five state-of-the-art pre-trained Transfer Learning models were fine-tuned on the balanced dataset:

1. VGG-19
2. ResNet-50
3. ResNet-101
4. DenseNet-121
5. Inception v3

• Training Strategy: Models were trained using the Adam optimizer, Categorical Cross-Entropy loss, and a learning rate of 0.0001.
• Regularization: L1 and L2 regularization were applied to prevent overfitting.
• Data Augmentation: Online augmentation (horizontal/vertical flips, rotation, zoom, shear, shifts) was used to increase data diversity.
• Selection: Based on performance metrics (Accuracy and ROC-AUC), ResNet-101, DenseNet-121, and Inception v3 were selected as the base learners for the ensemble.

C. Ensemble Learning Framework

The authors experimented with four fusion mechanisms to combine the predictions of the three base models:

1. Hard Majority Voting: Selects the class predicted by the majority.
2. Soft Majority Voting (Probability Averaging): Averages the probability scores.
3. Max Rule: Selects the prediction with the highest confidence score.
4. Weighted Probability Averaging: Assigns weights to each model based on their performance.

Innovation in Weighting: Instead of assigning weights based solely on accuracy, the authors proposed a novel weighting scheme using the Hyperbolic Tangent (tanh) function. This function calculates weights ($w_i$) based on a combination of multiple metrics (Precision, Recall, F1-score, and ROC-AUC) for each base learner:
$$w_i = \sum_{m \in M_i} \tanh(m)$$
This approach rewards models with high performance across all metrics, not just accuracy.

D. Explainable AI (XAI)

To address the "black box" problem, the authors integrated SHAP (SHapley Additive exPlanations) analysis.

• Technique: Used a gradient explainer to compute SHAP values for the ensemble's output feature maps.
• Visualization: Generated heatmaps where red indicates features positively contributing to the prediction (high impact) and blue indicates negative contributions. This allows clinicians to see exactly which pixels (e.g., lesion edges, hair, artifacts) influenced the decision.

3. Key Results

Performance Metrics

The proposed ensemble framework outperformed individual base models and previous state-of-the-art works:

• Best Individual Model: DenseNet-121 achieved 83.90% Accuracy and 0.91 ROC-AUC.
• Proposed Ensemble (Weighted Average with Tanh):
  • Accuracy: 85.80% (an improvement of ~1.9% over the best single model).
  • Precision: 86.58%
  • Recall (Sensitivity): 84.36%
  • F1-Score: 85.46%
  • ROC-AUC: 0.93 (indicating superior class separability).

Comparison with State-of-the-Art

The proposed method achieved higher accuracy and F1-scores compared to recent works by Gessert et al. (2020), Setiawan (2020), and others on similar ISIC datasets.

SHAP Analysis Insights

• Trustworthiness: SHAP visualizations confirmed that models correctly focused on the lesion boundaries and relevant features (highlighted in red).
• Diversity: Different models focused on different parts of the lesion (e.g., ResNet-101 focused on bottom edges, DenseNet-121 on top-right), validating the benefit of ensembling.
• Artifact Detection: The analysis revealed that models sometimes misinterpreted artifacts (e.g., hair, circular microscopic effects, vignetting) as diagnostic features. For instance, hair overlapping the lesion increased the probability of a specific class, while hair outside the lesion did not. This highlights the need for better pre-processing (occlusion removal).

4. Key Contributions

1. Novel Ensemble Strategy: Introduction of a weighted averaging ensemble where weights are dynamically calculated using a Hyperbolic Tangent function applied to a multi-metric performance profile (Precision, Recall, F1, AUC), rather than just accuracy.
2. Explainability Integration: Successfully integrated SHAP analysis into a deep ensemble framework for melanoma, providing visual evidence of why a prediction was made, thereby increasing clinical trust.
3. Robustness on Imbalanced Data: Demonstrated that effective class balancing and transfer learning can yield competitive results even with small, imbalanced medical datasets.
4. Diagnostic Insight: Identified specific failure modes where models are influenced by non-diagnostic artifacts (hair, lighting effects), offering a clear direction for future improvements in image pre-processing.

5. Significance and Future Work

This paper addresses a critical gap in medical AI: the trade-off between high accuracy and interpretability. By proving that an ensemble of transfer learning models combined with XAI can achieve high diagnostic accuracy while providing transparent reasoning, the study moves the field closer to clinical deployment.

Future Directions:

• Sensitivity Enhancement: Further improving Recall (Sensitivity) to minimize False Negatives (Type II errors), which is crucial in medical diagnosis.
• Advanced Pre-processing: Implementing automated lesion segmentation and occlusion removal (e.g., hair removal algorithms) to prevent models from learning from irrelevant artifacts.
• Clinical Validation: Validating the SHAP explanations against the actual diagnostic criteria used by dermatologists to ensure the AI's "reasoning" aligns with medical expertise.