⚡ electrical engineering

MLFFM-SegDiff: A Multi-Level Feature Fusion Diffusion Model for Skin Lesion Segmentation

The paper proposes MLFFM-SegDiff, a multi-level feature fusion diffusion model that integrates a dual-path U-Net encoder, an attention-based fusion module, and a boundary-sensitive loss to overcome challenges like blurred boundaries and artifacts in dermoscopic images, achieving state-of-the-art performance on skin lesion segmentation benchmarks.

Original authors: Jingjun Gu, Chaojie Shen, Yifeng Cao, Wei Zhang, Yiliu Li, Aobo Fan

Published 2026-06-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jingjun Gu, Chaojie Shen, Yifeng Cao, Wei Zhang, Yiliu Li, Aobo Fan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to trace the outline of a faint, blurry stain on a piece of fabric. Sometimes the stain looks like a perfect circle; other times, it's a weird shape with fuzzy edges that blend right into the fabric. Now, imagine there are also hairs, shadows, and wrinkles on the fabric that make it even harder to tell where the stain ends and the fabric begins.

This is exactly the challenge doctors face when looking at skin lesion images (pictures of moles or skin spots) to see if they might be dangerous. The goal is to draw a perfect line around the spot so computers can measure it and check for disease.

The paper you provided introduces a new tool called MLFFM-SegDiff to solve this problem. Here is how it works, explained simply:

The Problem: The "Blurry Stain"

Existing computer programs often struggle because:

The edges are fuzzy: The spot doesn't always have a sharp line.
The size varies: Spots can be tiny or huge.
The background is noisy: Hair and shadows look like part of the spot.

The Solution: A "Smart Painter" (Diffusion Model)

The authors use a type of AI called a Diffusion Model. Think of this model as a painter who starts with a canvas covered in static (like TV snow) and slowly cleans it up, step-by-step, until the image of the skin spot appears.

Usually, this painter just looks at the static and tries to guess what's underneath. But in this new method, the painter gets a helper and a special toolkit.

The Three Secret Weapons

1. The Dual-Path Team (The "Two-Eyed" Approach)

Old way: The painter looked at the static noise and the original photo separately, then tried to mash them together.
New way (Dual-Path Encoder): The model has two "eyes" working together. One eye looks at the noisy static, and the other looks at the clear photo of the skin. They constantly talk to each other. If one eye sees a hair that looks like a spot, the other eye says, "No, that's just a hair." This teamwork helps them understand the spot better than looking at just one thing.

2. The Multi-Level Fusion Module (The "Layer Cake" Strategy)

Old way: When the painter moved from the "rough sketch" phase to the "fine detail" phase, they only passed along the most recent sketch. They forgot the tiny details from the beginning.
New way (MLFFM): Imagine a construction site. You have the blueprints (deep, big-picture ideas) and the raw materials (shallow, tiny details like texture). The new Multi-Level Feature Fusion Module is like a super-efficient foreman. It takes the big blueprints and the tiny raw materials, mixes them together perfectly, and hands them to the painter. This ensures the final drawing has both the correct shape and the sharp, fuzzy edges.

3. The Boundary-Sensitive Loss (The "Red Pen" Focus)

Old way: The teacher grading the painter's work treated the middle of the spot and the edge of the spot the same.
New way: The model is given a special rule: "If you mess up the middle, it's okay. But if you mess up the edge, you get a bigger penalty." This forces the AI to pay extra attention to the blurry, hard-to-see borders, making the outline much more accurate.

The Results: Did it Work?

The authors tested this new "painter" on three different sets of skin images (like three different art classes). They compared it against other famous AI models.

The Score: The new model won in almost every category. It was better at finding the whole spot (Recall) and matching the shape perfectly (Dice and Jaccard scores).
The Analogy: If other models were like students who drew a circle around a messy stain, MLFFM-SegDiff was the student who carefully traced every wiggly, fuzzy edge of the stain, even when it was hard to see.

The Catch (Limitations)

The paper admits two things:

Speed: Because this model has to "clean up" the image step-by-step (like erasing and redrawing many times), it takes a bit longer to finish the picture than simpler models.
Over-enthusiasm: Sometimes, to make sure it doesn't miss any part of the spot, it might accidentally include a tiny bit of the background. It's better at finding the spot than it is at being perfectly strict about what isn't the spot.

Summary

In short, this paper presents a new AI system that acts like a highly collaborative artist. By using two "eyes" to look at the problem, mixing "big picture" and "tiny detail" information, and focusing extra attention on the fuzzy edges, it draws a much more accurate outline of skin spots than previous methods. This helps doctors get a clearer view of what they are treating.

Technical Summary: MLFFM-SegDiff

Problem Statement
Skin lesion segmentation is a critical prerequisite for computer-aided dermato-logical diagnosis, directly influencing subsequent feature extraction and disease classification. However, achieving precise segmentation in dermoscopic images remains challenging due to several factors: blurred or ambiguous lesion boundaries, low contrast between lesions and surrounding skin, significant variations in lesion shape and size, and interference from artifacts such as hair, shadows, and reflections. While recent diffusion models have shown promise in medical image segmentation through their progressive denoising capabilities, existing diffusion-based methods for skin lesions often suffer from insufficient cross-level feature interaction and inadequate recovery of boundary details.

Methodology
The paper proposes MLFFM-SegDiff, a multi-level feature fusion diffusion model designed to address the limitations of current segmentation frameworks. Built upon a conditional diffusion framework where the dermoscopic image serves as guidance and the lesion mask is recovered via iterative reverse denoising, the architecture introduces three primary innovations:

Dual-Path U-Net Encoder: Unlike standard fusion strategies, the model employs a dual-path encoder to jointly model the noisy mask branch and the image guidance branch. This structure enhances the interaction between original image features and noisy mask features during the extraction process. The encoder processes features through residual blocks and linear attention, generating two types of skip features: direct skip features ( $h_i$ ) for local information preservation and guidance-enhanced skip features ( $b_i$ ) for further fusion.
Multi-Level Feature Fusion Module (MLFFM): To overcome the limitations of conventional same-level skip connections, the MLFFM is inserted between the encoder and decoder. This module processes the enhanced skip features ( $b_i$ $b_{i}$ ) through:
- Attention Enhancement: Applying spatial and channel attention to emphasize informative regions and channels.
- Scale Alignment: Aligning features from different encoder levels to a common spatial resolution using adaptive pooling or bilinear interpolation.
- Cross-Level Fusion: Fusing the aligned features via element-wise multiplication and convolutional transformations.
  This allows the decoder to jointly exploit shallow boundary details and deep semantic representations from multiple levels simultaneously.
Configurable Boundary-Sensitive Loss: The training objective incorporates a boundary-sensitive weighting strategy. A boundary attention map assigns higher weights to pixels closer to the lesion contour. This term is configurable; when enabled, it encourages the model to prioritize ambiguous lesion boundaries during the denoising process without compromising overall stability.

Key Contributions
The paper outlines three main contributions:

The proposal of MLFFM-SegDiff, a novel diffusion model that integrates dual-path feature extraction, multi-level skip feature fusion, and boundary-aware optimization specifically for skin lesion segmentation.
The design of specific architectural components: a dual-path U-Net encoder to strengthen feature interaction, the MLFFM to enhance cross-level semantic and boundary representation, and a configurable boundary-sensitive loss term.
Extensive experimental validation demonstrating that the proposed method outperforms representative baselines (DermoSegDiff, Generic U-Net, SwinUNETR, and U-Net) across multiple evaluation metrics on three public datasets.

Experimental Results
The method was evaluated on the ISIC2018, PH2, and HAM10000 datasets using five-fold cross-validation. Performance was measured using Accuracy, F1-score, Jaccard index, Precision, Recall, Specificity, and Dice coefficient.

Overall Performance: MLFFM-SegDiff achieved the best average performance across all three datasets. The average Jaccard index reached 0.8546, and the average Dice coefficient reached 0.9207.
Dataset Specifics:
- On ISIC2018, the model achieved the highest Accuracy (0.9504), F1-score (0.8877), Jaccard (0.7985), Recall (0.9199), and Dice (0.8877).
- On PH2 and HAM10000, the proposed method achieved the best performance across all seven evaluation metrics.
Ablation Studies: Experiments confirmed that the combination of the Dual-Path U-Net (D-U-Net), MLFFM, and Edge Loss yields the best Dice and sensitivity scores, particularly on PH2 and HAM10000, indicating the complementary nature of these modules.
Visual Analysis: Qualitative results show that MLFFM-SegDiff produces masks that adhere more closely to target regions, particularly in cases with irregular boundaries or ambiguous appearances, compared to baseline methods.

Significance and Claims
The paper claims that MLFFM-SegDiff provides a feasible and effective approach for automatic skin lesion segmentation by effectively improving lesion region overlap and recall performance. The authors assert that the proposed multi-level feature fusion mechanism successfully addresses the challenge of integrating shallow boundary details with deep semantic structures, which is often insufficient in existing diffusion-based methods.

The authors maintain a modest stance regarding the limitations of their work. They acknowledge that while the method improves overlap and recall, Precision and Specificity on the ISIC2018 dataset were not the highest among compared methods, suggesting room for improvement in background suppression and fine-grained boundary control. Furthermore, they note that the iterative denoising process inherent to diffusion models may result in higher computational costs compared to feed-forward networks. Future work is proposed to focus on efficient sampling strategies and broader validation on larger, multi-center clinical datasets.