🤖 machine learning

DualMiT-Net: Local-Global Transformer-Convolutional Fusion for Breast Mass Segmentation in Mammographic Regions of Interest

The paper introduces DualMiT-Net, a dual-branch network that fuses local mass details from a Mix Transformer encoder with global breast context from an EfficientNet encoder via a spatially gated decoder to achieve state-of-the-art breast mass segmentation on the CBIS-DDSM dataset.

Original authors: Alibek Kamiluly, Milana Muratova, Yash Patel, Fan Li

Published 2026-08-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Alibek Kamiluly, Milana Muratova, Yash Patel, Fan Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Breast cancer remains one of the most significant health challenges facing women today, yet early detection offers a powerful defense. When the disease is found while still localized, the five-year survival rate exceeds ninety-nine percent. To find these early signs, doctors rely on mammograms, which are X-ray images of the breast. However, reading these images is a demanding task. The tumors, or masses, often blend into the surrounding tissue, lacking sharp edges or clear contrast, and their shapes can be irregular and difficult to trace. Even for experienced radiologists, drawing the exact outline of a mass can take time and vary from one reader to another. This is where computer science steps in, aiming to teach machines to see these hidden boundaries with the same precision a human eye strives for, but with the consistency and speed that only software can provide.

Researchers have long tried to help computers understand medical images using deep learning, a type of artificial intelligence that learns by analyzing thousands of examples. Early attempts used networks that were very good at spotting small, local details but sometimes struggled to understand the bigger picture of the surrounding tissue. More recent models borrowed ideas from how humans process vision, using systems that can look at distant parts of an image and connect them. Yet, a specific challenge remained for breast masses: a computer needs to see the fine, jagged edge of the tumor to define it, but it also needs to see the wider area of the breast to understand what the tumor is sitting next to. A tight, close-up view shows the edge, while a wider view shows the context, but most computer models have historically tried to do both with a single, fixed view.

To solve this, a team of researchers at Lawrence Technological University developed a new system called DualMiT-Net. Instead of forcing the computer to choose between a close-up and a wide shot, this system uses two separate "eyes" to look at the same mass simultaneously. One eye focuses tightly on the lesion, zooming in to capture the texture and the precise boundary of the mass. The other eye steps back to take in a broader view of the surrounding breast tissue, providing the context needed to understand where the mass ends and healthy tissue begins. These two perspectives are processed by different parts of the computer brain, designed to handle fine details and broad patterns respectively.

The magic of the system lies in how these two views talk to each other. As the computer analyzes the image, it shares information between the close-up view and the wide view at the deepest levels of its processing. Then, as it builds the final picture, it uses a special gate to decide how much of the wide, contextual information should be added to the detailed close-up at every single point. This allows the system to keep the sharp edges of the tumor while using the surrounding tissue to clarify confusing areas. The researchers also found that feeding the computer a specific type of image preparation helped the most. They took the standard mammogram, adjusted its brightness to highlight the most important details, and added a layer of texture information that responds to the specific patterns found in breast tissue, such as the spiky edges often seen in tumors.

When tested on a large, public collection of mammograms containing hundreds of cases, this dual-view system performed with remarkable accuracy. The researchers measured how well the computer's outline matched the expert-drawn outlines provided by human doctors. The system achieved a score of 0.9375 on a scale where a perfect match is 1.0, a result that was consistent across multiple training runs. This score was significantly higher than six other standard computer models that were tested under the exact same conditions. The new system was better at recovering the full area of the tumor and keeping the boundary lines close to the expert marks, even in difficult cases where the mass had low contrast or fuzzy edges.

The study also carefully checked what happened when parts of the system were removed to see which parts were truly essential. They found that while the specific tools used to refine the image helped, the core strength of the system came from the simple idea of using two different views at once. The system did not rely on a single trick but on the combination of local detail and global context working together. While the current model requires the computer to already know roughly where the mass is located, it represents a significant step forward in automating the difficult task of outlining these tumors. By mimicking the way a radiologist might shift their focus between a detail and the whole picture, this new approach offers a more reliable way to define the boundaries of breast masses, potentially helping doctors make faster and more consistent diagnoses.

Technical Summary of DualMiT-Net

Problem Statement
Breast mass segmentation in mammography remains a challenging task due to the inherent difficulties in medical imaging: masses often exhibit low contrast against surrounding tissue, possess irregular shapes, and have boundaries that blend seamlessly with normal breast structures. While deep learning has advanced medical image segmentation, standard models often struggle to simultaneously capture fine-grained boundary details and the broader contextual information necessary to distinguish lesions from surrounding tissue. Furthermore, many existing approaches rely on a single image view, forcing the model to balance the need for tight boundary precision with the need for surrounding tissue context within a single input.

Methodology
The authors propose DualMiT-Net, a dual-branch encoder-decoder architecture designed specifically for lesion-centered Regions of Interest (ROIs) in mammograms. The framework operates on two aligned inputs derived from the same lesion:

Local View: A tight crop centered on the lesion, resized to $256 \times 256$ pixels, intended to preserve fine shape, texture, and boundary details.
Global View: A wider crop centered on the same lesion, resized to $512 \times 512$ pixels, intended to capture the surrounding breast tissue context.

Network Architecture:

Dual Encoders: The local branch utilizes a Mix Transformer (MiT-B5) encoder to model relationships across the lesion ROI and generate multi-scale features. The global branch employs an EfficientNet-B5 encoder to efficiently capture the wider image context.
Deep Feature Sharing: Information is exchanged between the two branches at the two deepest encoder levels. This includes a Global-to-Local (G2L) sharing mechanism where global features are projected and added to the local representation before decoding.
Context-Gated Fusion Decoder: A single decoder progressively fuses features from coarse to fine resolution. At each decoding stage, global features are integrated into the local feature stream via a spatial gate. This gate, computed using a query-key-value mechanism, determines how much global context is added at each spatial location, ensuring that wider context is utilized only where beneficial without disrupting local boundary precision.
Feature Recalibration: The decoder incorporates Spatial and Channel Squeeze-and-Excitation (scSE) modules to recalibrate features, emphasizing useful channels and spatial locations.

Input Representation and Preprocessing:
The authors evaluated four input representations and selected a percentile-windowed mammogram combined with a Gabor texture response. The input tensor is constructed as $X = [I_w, I_w, I_g]$ , where $I_w$ is the intensity-windowed image and $I_g$ is the Gabor filter response. This approach leverages the Gabor filter's sensitivity to oriented texture patterns (such as spiculation and margin texture) while maintaining compatibility with ImageNet-pretrained weights. Preprocessing includes lesion-centered cropping, canonical patch construction using Principal Component Analysis (PCA) for orientation and scale normalization, and nearest-neighbor interpolation for mask resizing.

Experimental Protocol:
The model was trained and evaluated on the mass subset of the Curated Breast Imaging Subset of the Digital Database for Screening Mammography (CBIS-DDSM). A strict patient-level split was used to prevent data leakage, resulting in 753 training pairs, 91 validation pairs, and 92 test pairs. The network was optimized using a weighted sum of Binary Cross-Entropy and soft Dice loss. Experiments were conducted with three random seeds, and results were reported using both base weights and Exponential Moving Average (EMA) weights.

Key Contributions:

Dual-View Architecture: The introduction of a dual-branch system that explicitly separates local lesion detail from global tissue context, processing them through specialized encoders (MiT-B5 and EfficientNet-B5) before fusing them.
Deep Cross-Branch and Spatially Gated Fusion: A mechanism for exchanging features at deep levels and selectively incorporating global context during decoding via a spatial gate, allowing the model to adaptively use surrounding tissue information.
Texture-Aware Input: The demonstration that a Gabor-enhanced, percentile-windowed input representation yields superior performance compared to standard windowing or histogram equalization approaches.

Results:
Across three training runs, the full DualMiT-Net with EMA weights achieved a mean Dice coefficient of 0.9375 ± 0.0011 and a mean Intersection over Union (IoU) of 0.8834 ± 0.0020.

Comparison with Baselines: DualMiT-Net outperformed six standard encoder-decoder baselines (including U-Net, U-Net++, FPN, and PSPNet variants) trained under identical settings. The strongest baseline (U-Net with ResNet-50) achieved a Dice of 0.9025, representing an absolute improvement of 0.0350 in Dice and 0.0594 in IoU for DualMiT-Net.
Comparison with Literature: The model achieved higher Dice and IoU scores than previously published methods on the CBIS-DDSM mass subset, including Cross-view VAE (Dice 0.9246) and Connected-ResUNets (Dice 0.8952).
Ablation Studies: Removing the scSE module or the G2L sharing mechanism resulted in only marginal performance drops (Dice decreasing by ~0.0003–0.0004), indicating that the core dual-view architecture is the primary driver of performance, while the refinement modules provide incremental gains.
Boundary Accuracy: The model achieved a 95th-percentile Hausdorff Distance (HD95) of approximately 6.88 pixels, with qualitative analysis showing that errors were primarily localized to lesion margins rather than gross segmentation failures.

Significance and Claims
The paper claims that the combination of local mass information with wider breast context provides a robust solution for accurate and consistent breast mass segmentation. The authors assert that their dual-view approach effectively addresses the trade-off between boundary precision and contextual understanding, which is critical for lesions with low contrast or irregular shapes.

The study is modest regarding its scope, noting that the current evaluation is lesion-centered, relying on expert annotations to define the ROI and normalize orientation. Consequently, the results reflect segmentation performance on known lesion regions rather than end-to-end detection and segmentation on full mammograms. The authors acknowledge that the model was trained on digitized screen-film mammograms (CBIS-DDSM) without random data augmentation, and future work is required to validate generalization across different imaging systems (e.g., full-field digital mammography) and to develop automatic lesion localization capabilities. Nevertheless, the results support the hypothesis that complementary local and global views significantly enhance segmentation quality in mammographic ROIs.

Technical Summary of DualMiT-Net

More like this