🤖 machine learning

Synthetic Melanoma Image Generation and Evaluation Using Generative Adversarial Networks

This paper presents a systematic benchmarking study demonstrating that StyleGAN2-generated synthetic melanoma images, which preserve diagnostically relevant features and are difficult for dermatologists to distinguish from real ones, effectively mitigate class imbalance and improve melanoma detection performance when used for data augmentation.

Original authors: Pei-Yu Lin, Yidan Shen, Neville Mathew, Renjie Hu, Siyu Huang, Courtney M. Queen, Cameron E. West, Ana Ciurea, George Zouridakis

Published 2026-03-17

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Pei-Yu Lin, Yidan Shen, Neville Mathew, Renjie Hu, Siyu Huang, Courtney M. Queen, Cameron E. West, Ana Ciurea, George Zouridakis

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to spot a dangerous intruder (melanoma) in a crowded room of harmless visitors (benign moles). The problem? The intruders are incredibly rare. For every 100 people in the room, maybe only 2 are intruders. If you only show the robot the real photos you have, it gets confused. It starts thinking, "Oh, everyone looks like a harmless visitor," because it hasn't seen enough intruders to learn what they really look like.

This is the big problem doctors and AI researchers face with skin cancer detection: there just aren't enough pictures of melanoma to train smart computers.

This paper is like a cooking competition where four different chefs (AI models) try to bake fake but perfect-looking cakes (synthetic melanoma images) to fill up the robot's training diet. The goal isn't to trick people into eating fake cake; it's to give the robot enough "practice" so it becomes an expert at spotting the real thing.

Here is the breakdown of what they did, using some everyday analogies:

1. The Contestants (The AI Chefs)

The researchers tested four different "recipes" (AI architectures) to see which one could bake the most realistic fake melanoma images at high resolution (512x512 pixels, which is like a high-definition photo):

DCGAN: The old-school chef. It tried, but the cakes came out a bit blurry and lacked the fine details (like the specific texture of a mole).
StyleGAN3 (Two versions): The fancy, modern chefs. They were very good at capturing the variety of cakes, but one version kept adding weird, invisible grid lines to the frosting (artifacts), and the other made cakes that looked great individually but all looked exactly the same (lacking variety).
StyleGAN2: The Goldilocks chef. It wasn't the flashiest, but it found the perfect balance. It baked cakes that looked incredibly real, had all the right textures, and didn't have any weird glitches.

2. The Taste Test (How they judged the cakes)

How do you know a fake cake is good? You don't just look at it; you test it in three ways:

The Math Test (FID Score): They used a computer program to measure how close the "flavor profile" of the fake cakes was to the real ones. Lower scores meant the fake cakes were more similar to the real ones. StyleGAN2 scored the best.
The Robot Test (The Frozen Classifier): They took a super-smart robot that had already learned to spot real melanoma from real photos. They showed it the fake cakes. If the robot said, "That looks like a melanoma!" it meant the fake cake was convincing enough to fool a machine expert. 83% of the StyleGAN2 cakes fooled the robot.
The Human Test (The Dermatologists): This was the ultimate test. They showed 200 photos (100 real, 100 fake) to two expert skin doctors. The doctors had to guess which were real and which were fake.
- The Result: The doctors only got it right 66.5% of the time. Since they were guessing randomly, they should have only been right 50% of the time. This means the fake images were so good that even experts struggled to tell them apart!

3. The Payoff (Why this matters)

The most exciting part of the paper is what happened when they used these fake cakes to train a new robot.

Before: The robot was trained only on real photos. Because there were so few melanoma photos, the robot was bad at spotting them (it missed many).
After: They added thousands of the StyleGAN2 fake melanoma images to the training mix. Suddenly, the robot had a much better "diet."
The Result: The robot's ability to spot real melanoma improved significantly. It became much better at finding the dangerous intruders in the crowd.

The Big Takeaway

Think of this like a flight simulator. You can't train a pilot to handle a storm by only flying in perfect weather. You need a simulator that creates a realistic storm.

This paper proves that StyleGAN2 is the best simulator for melanoma. It can create "fake storms" (synthetic melanoma images) that are so realistic, they don't fool the eye, but they do teach the computer how to survive the real thing.

In short: By using AI to generate fake but perfect pictures of skin cancer, the researchers solved the problem of "not enough data," making future AI doctors much better at saving lives.

1. Problem Statement

Melanoma is the most lethal form of skin cancer, where early detection is critical for patient survival. While deep learning has advanced automated skin lesion analysis, progress is hindered by two primary bottlenecks:

Data Scarcity: A lack of large, high-quality, expert-annotated datasets.
Severe Class Imbalance: In standard datasets (e.g., ISIC), melanoma images are vastly underrepresented compared to benign lesions. This imbalance biases decision boundaries in machine learning models, reducing their generalization capabilities.

Existing generative approaches often rely on early GAN architectures that produce low-resolution images lacking fine-grained dermoscopic details, or they rely solely on feature-space metrics that do not guarantee clinical utility or downstream recognizability.

2. Methodology

The study presents a systematic benchmarking framework to evaluate four Generative Adversarial Network (GAN) architectures for high-resolution (512 × 512) melanoma synthesis.

Architectures Evaluated

The authors compared:

DCGAN: A baseline deep convolutional GAN.
StyleGAN2: A modular architecture separating content and style.
StyleGAN3-T: Focuses on translation equivariance and realistic textures.
StyleGAN3-R: Focuses on rotation and translation equivariance to minimize positional bias.

Datasets and Preprocessing

Training Data: Two expert-annotated benchmarks: ISIC 2018 (1,061 melanoma images after filtering) and ISIC 2020 (7,227 melanoma images).
Preprocessing: Images were resized to 512 × 512.
- DCGAN: Used bilateral filtering, normalization, and extensive augmentation (rotations/flips).
- StyleGANs: Used only horizontal flipping, relying on internal augmentation mechanisms.
Hardware: Training conducted on a DGX-2 system with 16 NVIDIA Tesla V100 GPUs.

Hyperparameter Optimization

A key focus was optimizing the R1 regularization strength ( $\gamma$ ), which stabilizes training. The authors explored $\gamma$ values of 0.8, 1.6, 8.0, and 10.0.

Multi-Faceted Evaluation Protocol

To move beyond simple visual inspection, the study employed a rigorous, five-part evaluation:

Distribution-Level Metrics: Fréchet Inception Distance (FID) to measure global distribution similarity.
Sample-Level Metrics: Fréchet Medoid Distance (FMD) to measure representativeness of individual samples and detect mode collapse.
Qualitative Inspection: Dermoscopic analysis based on the 7-point checklist (e.g., pigment network, border irregularity).
Downstream Classification: A frozen EfficientNet-B6 classifier (trained on real data) tested on synthetic images to verify if they retain discriminative disease features.
Expert Human Evaluation: Two board-certified dermatologists independently classified real vs. synthetic images to assess perceptual realism.

3. Key Contributions

Systematic Cross-Architecture Benchmark: The first comprehensive comparison of DCGAN, StyleGAN2, and StyleGAN3 variants specifically for melanoma synthesis under unified experimental conditions.
Domain-Specific Hyperparameter Guidance: Identification that lighter R1 regularization ( $\gamma = 0.8$ ) yields superior FID scores for melanoma synthesis, likely due to the homogeneity of dermoscopic features requiring less aggressive gradient smoothing.
Clinically Grounded Validation: A novel protocol combining computational metrics with downstream classifier performance and independent expert dermatologist assessment, proving that synthetic data can be diagnostically relevant.

4. Key Results

Quantitative Performance (FID & FMD)

StyleGAN2 achieved the best balance of metrics. At $\gamma=0.8$ , it attained FID scores of 24.8 (ISIC 2018) and 7.96 (ISIC 2020).
StyleGAN3-R achieved the lowest FID (26.47) at $\gamma=8$ but suffered from severe "mesh-like" artifacts in ~60% of images, rendering them clinically unsuitable.
StyleGAN3-T showed poor FID (246.42) but low FMD, indicating it generated realistic individual samples but failed to capture the full morphological diversity (partial mode collapse).
DCGAN performed poorly on both metrics (FID 66.49), failing to capture high-resolution details.

Downstream Utility

Recognizability: A frozen EfficientNet classifier correctly identified 83% of StyleGAN2-generated images as melanoma, confirming the preservation of disease-specific features.
Augmentation Impact: When synthetic images were added to a class-imbalanced training set (shifting the benign:melanoma ratio from 98:2 to 65:35), the melanoma detection AUC improved from 0.925 to 0.945, and the F1 score increased from 0.168 to 0.259.

Human Evaluation

Two board-certified dermatologists achieved an average accuracy of 66.5% in distinguishing synthetic from real images (significantly above the 50% chance level, but low for experts).
Inter-rater agreement was low ( $\kappa = 0.17$ ), suggesting no consistent visual "tells" or artifacts across the synthetic samples.
The StyleGAN2 discriminator itself only achieved 59.5% accuracy, further indicating the high fidelity of the generated images.

5. Significance and Conclusion

This study demonstrates that StyleGAN2, when properly tuned, can generate high-resolution melanoma images that are indistinguishable from real clinical data by both machines and human experts.

Clinical Impact: The generated images are not merely visually similar; they preserve the specific dermoscopic features required for diagnosis.
Practical Application: Synthetic data effectively mitigates class imbalance, leading to measurable improvements in the performance of melanoma detection models (higher AUC and F1 scores).
Future Outlook: The framework provides a validated pathway for integrating synthetic data into dermatological AI pipelines to improve robustness and generalization, potentially extending to other data-scarce skin conditions.

The authors conclude that while diffusion models are emerging, GANs (specifically StyleGAN2) currently offer the most thoroughly characterized and effective solution for medical image synthesis in this domain.