💻 computer science

Synthetic Data Generation for Long-Tail Medical Image Classification: A Case Study in Skin Lesions

This paper proposes a diffusion-model-driven synthetic data augmentation pipeline featuring a novel inpainting model and an Out-of-Distribution post-selection mechanism to effectively address long-tail class imbalance in medical image classification, achieving significant performance gains—particularly on rare classes—on the ISIC2019 skin lesion dataset.

Original authors: Jiaxiang Jiang, Mahesh Subedar, Omesh Tickoo

Published 2026-05-06

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jiaxiang Jiang, Mahesh Subedar, Omesh Tickoo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are training a student to recognize different types of skin spots. In the real world, some spots are very common (like a "Head" class), while others are extremely rare but dangerous (like a "Tail" class).

The problem is that your student only gets to study 10,000 pictures of the common spots but only 50 pictures of the rare, dangerous ones. Naturally, the student becomes an expert at spotting the common ones but fails miserably when they see a rare one. This is the "Long-Tail" problem in medical AI.

This paper proposes a clever solution: Don't just study harder; create more study material.

Here is how they did it, explained with simple analogies:

1. The Problem: The Unbalanced Library

Think of the medical dataset as a library. The "Head" classes have entire shelves full of books (images). The "Tail" classes have only a single, dusty book on the bottom shelf. When the AI tries to learn, it ignores the bottom shelf because there's nothing there to read. But in medicine, that single book might describe a deadly disease, so ignoring it is dangerous.

2. The Solution: The "Magic Photocopier" (Diffusion Model)

Instead of waiting for more real patients to show up, the authors built a "Magic Photocopier" (a Diffusion Model) that can generate new, realistic pictures of those rare skin spots.

The Inpainting Trick: Imagine you have a photo of a skin spot, but you erase the spot itself, leaving just the surrounding skin. The AI's job is to "paint" a new spot back into that empty space.
The Twist: They taught this AI to look at the surrounding skin and the specific type of disease they want to create, then "draw" a brand new, realistic version of that rare disease into the blank space.

3. The Safety Check: The "Quality Control Inspector" (OOD Filtering)

Here is the catch: When you ask an AI to imagine something it has never seen much of, it sometimes gets confused and draws nonsense (like a blue skin spot or a spot that looks like a cartoon). These are called Out-of-Distribution (OOD) samples—fake data that doesn't belong in the real world.

If you feed these fake, weird pictures to your student, they might get confused and learn the wrong things.

So, the authors added a Quality Control Inspector. Before the new pictures are added to the student's study pile, this inspector checks them. If a picture looks too weird or doesn't fit the "real world" rules of that disease, the inspector throws it away. Only the clean, realistic, "in-distribution" pictures get to stay.

4. The Result: A Better Student

The authors tested this on a famous dataset of skin lesions (ISIC2019).

The Setup: They took the rare diseases (the ones with very few samples), generated hundreds of new, high-quality synthetic images using their "Magic Photocopier," filtered out the bad ones, and mixed them with the original photos.
The Outcome: They trained a standard AI classifier on this new, balanced pile of data.
- Overall Score: The AI got much better at recognizing all types of spots.
- The Big Win: The rare diseases saw a massive improvement. For the rarest class (Dermatofibroma), the accuracy jumped by over 28%.
- Comparison: Their method beat other top-tier methods that tried to fix the problem by changing the math (loss functions) or building complex team-based AI models. They did it simply by giving the AI better, more balanced data to learn from.

5. The "Sweet Spot" Finding

They also discovered a "Goldilocks" zone for their Quality Control.

If they kept too many generated images (including the weird ones), the AI got confused.
If they kept too few, they didn't get enough help for the rare diseases.
They found that keeping about 20% to 60% of the generated images (after filtering) was the perfect balance to boost performance without adding noise.

Summary

In short, the paper says: "Don't just try to teach a student with a tiny, unbalanced textbook. Use a smart, safety-checked photocopier to print more pages for the rare chapters, and your student will learn much better."

This approach didn't require building a super-complex new brain for the AI; it just gave the existing brain a much fairer set of study materials.

Technical Summary: Synthetic Data Generation for Long-Tail Medical Image Classification

Problem Statement
Long-tailed class distributions are pervasive in multi-class medical datasets, posing significant challenges for deep learning models. In these scenarios, a few dominant "head" classes contain the majority of samples, while numerous clinically critical "tail" classes (often corresponding to rare, high-risk diseases) are severely under-represented. Models trained on such skewed data typically achieve competitive overall accuracy but fail to generalize to tail classes. Existing solutions—categorized into loss function design, model architecture improvements, and class re-balancing—offer only marginal improvements. These methods often rely on meticulous hyperparameter tuning, exhibit limited generalization across different models, and struggle to scale due to the deterministic nature of handcrafted data augmentation. Furthermore, while diffusion models have shown success in generating natural images, their application to domain-specific, long-tailed medical imaging remains under-explored, particularly regarding the generation of diverse yet clinically meaningful samples.

Methodology
The authors propose a diffusion-model-driven synthetic data augmentation pipeline tailored for Long-Tail Medical Image Classification (LTMIC), specifically demonstrated on skin lesion classification. The approach consists of three primary stages:

Inpainting Diffusion Model Architecture:
The core of the method is a novel inpainting diffusion model. Unlike standard text-embedding models which are unsuitable for highly specialized medical descriptions, the authors train a simple fully connected class embedding network. The model takes four inputs: the original dermoscopy image ( $x$ ), a binary area-of-interest mask ( $b$ ), the background of the dermoscopy image ( $d$ ), and the class label ( $y$ ). The image and background are encoded into latent space via a pre-trained Variational Auto Encoder (VAE). The binary mask is rescaled to match the latent dimensions. These components are concatenated and fed into a UNet parameterized by $\theta_d$ to estimate the noise added during the forward diffusion process. A class embedding network parameterized by $\theta_c$ provides the label conditioning.
Fine-Tuning with Sample Selection:
To ensure the diffusion model learns the distribution of each class equally well, an oversampling strategy is employed during fine-tuning. The authors leverage findings from prior work suggesting that "approximately-in-distribution" (AID) samples (those slightly far from the distribution center) are most important for training generative models. Consequently, a self-supervised medical anomaly detection method is used to compute anomaly scores for each sample. Weights are assigned based on the distance of a sample from the median anomaly score of its class; samples closer to the AID region are assigned higher weights and repeated to balance the fine-tuning dataset across all classes.
Synthetic Generation and OOD Filtering:
The fine-tuned model generates synthetic images by taking the background images and class labels of the original training set as input. Due to the stochastic nature of diffusion models, the initial synthetic dataset ( $D_{syn}$ ) contains "noisy" samples, including Out-of-Distribution (OOD) images that may not be clinically valid. To address this, a medical OOD filtering mechanism is applied to $D_{syn}$ to produce a clean synthetic dataset ( $D_{clean}^{syn}$ ). The final training dataset is formed by combining the original long-tail dataset ( $D_{LT}$ ) with $D_{clean}^{syn}$ . A hyperparameter $\gamma$ controls the percentage of clean samples retained from the synthetic set for each class.

Key Contributions
The paper outlines three main contributions:

Novel Pipeline: A synthetic data generation pipeline integrating an inpainting diffusion model with OOD filtering specifically designed to address the LTMIC problem.
Specialized Architecture: A new inpainting diffusion model architecture tailored for medical image generation, utilizing a dedicated class embedding network and background conditioning rather than generic text embeddings.
Sampling Strategy: A novel OOD-based sampling strategy for fine-tuning diffusion models on medical long-tail datasets, which prioritizes AID samples to improve the quality of generated data for tail classes.

Experimental Results
The method was evaluated on the ISIC2019 skin lesion classification dataset, a benchmark containing 25,331 images across 8 classes with severe imbalance (ranging from 12,875 samples for Nevus to only 62 for Vascular Lesion). The experiments utilized 5-fold cross-validation and a simple CNN-based classifier (without ensembling or external data) to isolate the impact of the data generation method.

Overall Performance: The proposed approach achieved a Balanced Multiclass Accuracy (BMA) of 0.802, significantly outperforming current State-of-the-Art (SOTA) methods, including multi-model ensembles (MRE) and other diffusion-based approaches (e.g., CBDM, Dreambooth). Notably, it surpassed the MRE method by approximately 4% in BMA without requiring external data.
Tail Class Improvement: The most significant gains were observed in tail classes. For the class with the fewest samples (Dermatofibroma, DF), the method yielded an accuracy improvement of over 28% compared to the baseline without synthetic data.
Impact of OOD Filtering: The study found that OOD filtering is crucial for balancing performance. While synthetic data without filtering improved tail classes significantly, it introduced noise that slightly degraded head class performance. The filtering process restored head class performance while maintaining the gains in tail classes.
Hyperparameter Sensitivity: An ablation study on $\gamma$ (the retention rate of clean synthetic samples) revealed that performance improves sharply with even a small amount of synthetic data but plateaus or decreases as $\gamma$ approaches 1 (no filtering). The optimal range for $\gamma$ was identified between 0.2 and 0.6.

Significance and Claims
The authors claim that their results demonstrate the effectiveness of diffusion-based augmentation in mitigating long-tail imbalance and enhancing the robustness of medical classification. By generating diverse, realistic, and clinically meaningful synthetic samples, the method addresses the data scarcity of rare diseases without relying on complex model ensembles or specialized loss functions that are sensitive to hyperparameters. The paper concludes that while the current approach treats data generation and classification as independent processes, future work will focus on the joint optimization of these components to further enhance LTMIC performance. The method is presented as model-agnostic, capable of integration with any classification architecture.