💻 computer science

DermaFlux: Synthetic Skin Lesion Generation with Rectified Flows for Enhanced Image Classification

DermaFlux is a rectified flow-based generative framework that synthesizes clinically grounded skin lesion images from natural language descriptions to address data scarcity and class imbalance, significantly enhancing binary classification performance compared to existing diffusion-based methods.

Original authors: Stathis Galanakis, Alexandros Koliousis, Stefanos Zafeiriou

Published 2026-03-18

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Stathis Galanakis, Alexandros Koliousis, Stefanos Zafeiriou

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to spot skin cancer. You want the robot to be a super-expert doctor who can look at a mole and say, "This is dangerous," or "This is harmless."

The problem? The robot is starving for data.

Real medical photos of skin lesions are hard to get. They are private, protected by strict privacy laws, and often, there just aren't enough of them—especially for rare types of cancer. It's like trying to teach someone to recognize a specific type of rare bird by showing them only three blurry photos. The robot gets confused, misses the dangerous ones, or panics over harmless freckles.

Enter DermaFlux. Think of DermaFlux not as a photographer, but as a master art forger who is so good at copying reality that the copies are indistinguishable from the real thing. But instead of copying paintings, it creates fake skin lesions to train the robot.

Here is how it works, broken down into simple concepts:

1. The "Recipe" vs. The "Random Guess"

Most AI image generators (like the ones that make funny pictures of cats) work a bit like a sculptor starting with a block of marble and chipping away random pieces until a shape appears. This is called "diffusion." It's creative, but sometimes the result is a bit messy or doesn't follow the doctor's instructions perfectly.

DermaFlux is different. It uses something called "Rectified Flows."

The Analogy: Imagine you are trying to get from your house to a friend's house.
- Old Way (Diffusion): You take a random walk, bumping into trees, going in circles, and hoping you eventually find the right street.
- DermaFlux Way (Rectified Flow): You are given a straight, high-speed train track that goes directly from your house to your friend's. No detours, no confusion. It's a direct, efficient path from "nothing" (noise) to "perfect image."

2. The "Medical Translator"

To make these fake images useful, they need to look like real medical cases. You can't just say, "Make a mole." You need to say, "Make a mole that is uneven, has jagged edges, and has three different shades of brown."

The researchers used a smart AI (Llama 3.2) to act as a Medical Translator.

They took thousands of real photos of skin lesions.
The AI looked at them and wrote a detailed "recipe" for each one, describing exactly how asymmetrical it was, how jagged the border was, and what the colors looked like.
This turned a simple picture into a structured instruction manual.

3. The "Specialized Apprentice"

The team took a massive, pre-trained AI model (Flux.1) that was already good at making images. Instead of retraining the whole giant brain (which would take forever and cost a fortune), they used a technique called LoRA (Low-Rank Adaptation).

The Analogy: Imagine a world-famous chef who knows how to cook everything. You don't need to teach them how to hold a knife or chop onions. You just give them a specialized recipe card for "Skin Cancer Cuisine." They keep all their existing skills but learn to focus specifically on making perfect moles. This is fast, cheap, and very effective.

4. The "Training Gym"

Now, the team has a gym full of fake but perfect skin lesions. They use these to train their "Doctor Robot" (a classifier).

The Result: When they trained the robot using these DermaFlux images, the robot got significantly smarter.
If they gave the robot a tiny amount of real photos (2,500) and mixed in the fake ones, the robot's accuracy jumped by 8% compared to other top models.
It was even 9% better than robots trained on fake images made by the older, "random walk" style generators.

Why Does This Matter?

Think of it like a flight simulator.

Before: Pilots (doctors/AI) could only train on a few real flights. If they encountered a weird storm (a rare cancer), they might crash.
Now: DermaFlux generates millions of "perfect storms" and "weird weather patterns" that are medically accurate. The pilot can practice thousands of times in the simulator. When they finally get on a real plane, they are ready for anything.

In short: DermaFlux is a smart, efficient machine that writes detailed medical descriptions and turns them into realistic fake skin lesions. It fills the gaps in our data, allowing AI doctors to learn faster, make fewer mistakes, and save more lives, all without needing to invade anyone's privacy to get more real photos.

1. Problem Statement

Skin lesion classification systems face significant hurdles due to the scarcity of large, diverse, and well-annotated clinical datasets. Key challenges include:

Data Scarcity & Privacy: Strict ethical and legal regulations limit the availability of real-world dermatological images.
Class Imbalance: Public datasets often suffer from an imbalance between benign and malignant lesions, leading to poor generalization in deep learning models.
Limitations of Current Generative Models: Existing solutions based on diffusion models (e.g., Stable Diffusion) struggle with:
- Stochasticity: Reliance on stochastic denoising reduces sampling efficiency and determinism.
- Control: Difficulty in precisely controlling clinically meaningful attributes (e.g., asymmetry, border irregularity, color variation) during synthesis.
- Alignment: Weak semantic alignment between textual descriptions and synthesized visual features due to limited context capacity in standard CLIP-based encoders.

2. Methodology

The authors propose DermaFlux, a text-to-image generative framework based on Rectified Flows rather than diffusion processes.

A. Dataset Curation (Image-Text Pairs)

To train the model, the authors constructed a large-scale, open, and curated dataset of approximately 508,344 image-text pairs derived from public datasets (ISIC 2019–2024, Derm12345, PAD20, etc.).

Caption Generation: Since original metadata lacked detailed morphological descriptors, the authors used Llama 3.2 (a Vision-Language Model) to generate structured captions.
Clinical Structure: Prompts were designed to follow the ABCDE dermatological criteria, specifically focusing on Asymmetry, Border irregularity, and Color variation.
Validation: Clinical experts verified the medical plausibility of the generated captions.

B. Model Architecture & Adaptation

Backbone: DermaFlux is built upon Flux.1, a rectified flow-based model. Unlike diffusion models that learn a stochastic denoising path, rectified flows learn a deterministic transport mapping between Gaussian noise ( $z_0$ ) and the data distribution ( $z_1$ ) via a linear interpolation path.
Encoders: Flux.1 utilizes dual text encoders (CLIP and T5-XXL), providing richer contextual conditioning and better text-image alignment compared to standard diffusion models.
Fine-Tuning Strategy: The authors employed Low-Rank Adaptation (LoRA) to fine-tune the 12B parameter Flux.1 model.
- Only the attention layers of the transformer were updated (approx. 612M trainable parameters).
- The backbone weights were frozen to preserve general generative capacity while adapting to the dermatological domain.
Training: The model was trained for five epochs across three resolutions (128×128, 256×256, 512×512) using an ODE sampler with 20 interpolating steps for inference.

3. Key Contributions

Structured Dataset: Creation of a 500k+ image-text dataset with clinically grounded, attribute-level captions explicitly encoding asymmetry, border irregularity, and color variation.
DermaFlux Framework: Development of a rectified flow-based generator that offers a deterministic alternative to diffusion-based synthesis, improving sampling efficiency and attribute control.
Performance Benchmarking: Demonstration that synthetic data generated by DermaFlux significantly outperforms diffusion-based synthetic data (e.g., Derm-T2IM) in downstream classification tasks.

4. Experimental Results

The authors evaluated the framework using ResNeXt-50 and Vision Transformer (ViT) classifiers on a held-out test set of ~9,100 lesions.

Comparison with Diffusion Models:
- When classifiers were trained exclusively on synthetic data, DermaFlux improved ViT accuracy by 9.03% over diffusion-based baselines.
- In mixed real/synthetic training scenarios, DermaFlux improved ViT accuracy by 3.47%.
Data Augmentation Impact:
- Adding DermaFlux synthetic data to limited real datasets (e.g., 2,500 real images) improved binary classification accuracy by up to 6%.
- The model showed the most significant gains in low-data regimes (2,500–5,000 real samples), enhancing class separability (ROC-AUC).
State-of-the-Art (SOTA) Comparison:
- A ViT model trained on 2,500 real images + 4,375 DermaFlux synthetic samples achieved 78.04% accuracy and an AUC of 0.859.
- This result surpasses the next best dermatology model (MAKE/DermLIP) by 8 percentage points.
- The model achieved near-perfect sensitivity (TPR $\approx$ 1.0) at a false positive rate of 0.75, outperforming existing models in identifying malignant lesions while maintaining specificity.

5. Significance

Clinical Utility: DermaFlux addresses the critical bottleneck of data scarcity in medical AI. By generating clinically coherent images, it allows for the training of robust classifiers without relying on massive, privacy-sensitive real-world datasets.
Technical Advancement: The paper validates Rectified Flows as a superior architecture for medical image synthesis compared to traditional diffusion models, offering better control over specific clinical attributes and higher sampling efficiency.
Cost-Effectiveness: The approach demonstrates that high-performance medical AI can be achieved with significantly fewer real-world annotations (2,500 images) when augmented with high-quality, structured synthetic data, reducing the cost and time of dataset curation.

Availability: Code, data, and pretrained models are available at dermaflux.github.io.