MMIF-MESFusion：Coupling Modality-Specific Enhancement and Statistics- Guided Adaptive Fusion for Medical Image Fusion

Technical Summary: MMIF-MESFusion

Problem Statement
Multimodal medical image fusion aims to synthesize anatomical details (from CT/MRI) and functional metabolic information (from PET/SPECT) into a single image to eliminate "information blind spots" in clinical diagnostics. However, existing deep learning approaches face three primary challenges:

1. Heterogeneous Feature Discrepancies: Different modalities possess distinct physical imaging mechanisms (e.g., high-frequency anatomical textures in MRI vs. low-frequency metabolic intensity in PET), which standard unified encoders fail to address effectively.
2. Insufficient Long-Range Modeling: While Transformers offer global modeling, their quadratic complexity is prohibitive for high-resolution medical images. Conversely, standard CNNs are limited by local receptive fields, and direct applications of Mamba (State Space Models) often struggle with the non-causal spatial structure of 2D/3D images, leading to impaired spatial continuity.
3. Inefficient Fusion Strategies: Current fusion methods often rely on computationally intensive attention mechanisms or simple concatenation, failing to dynamically balance structural and functional information based on the holistic statistical properties of feature maps.

Methodology: The MESFusion Framework
The authors propose MESFusion, a lightweight, end-to-end framework designed to couple modality-specific enhancement with statistics-guided adaptive fusion. The architecture follows an encoder-fusion-decoder paradigm comprising four key stages:

1. Modality-Aware Enhancement (MAE):

   • Instead of a shared preprocessing module, MAE employs differentiated strategies for anatomical (MRI) and functional (PET/SPECT/CT) images.
   • For MRI, it utilizes multi-scale gradient extraction (3×3 and 5×5 convolutions) and structure-aware attention to enhance high-frequency anatomical textures.
   • For Functional images, it performs channel-based statistical modeling (computing global mean and standard deviation) to create energy-aware attention weights, preserving intensity distributions and highlighting metabolic regions.

2. Spatial Gated Mamba (SGMam) Backbone:

   • To address the limitations of standard Mamba in 2D medical imaging, the authors introduce SGMam, which integrates linear global modeling with local feature extraction.
   • It features a Spatial Gating (SPG) branch using a 7×7 depth-wise convolution to generate a spatial weight map, suppressing background redundancy and enhancing salient structural details.
   • It employs a 2D Selective Scan (SS2D) mechanism with a multi-path scanning strategy to capture long-range dependencies and global context with linear computational complexity, preserving fine-grained anatomical continuity.

3. Statistical-Guided Adaptive Fusion (SDAF):

   • This module replaces heavy attention mechanisms with a lightweight, distribution-aware fusion strategy.
   • It extracts first-order (mean) and second-order (standard deviation) statistics from the feature channels of both modalities.
   • These statistics are concatenated and processed through a bottleneck network to generate dynamic channel-wise weights, allowing the model to adaptively balance structural and functional information based on the specific content of the input images.

4. Decoder and Loss Function:

   • A hierarchical decoder reconstructs the fused image using cascaded convolutions with SiLU and Tanh activations to ensure radiometric fidelity.
   • The training utilizes a composite loss function combining Intensity Loss (for global balance and target enhancement), Gradient Loss (for edge preservation), and Structural Similarity Loss (MS-SSIM) (for perceptual consistency).

Key Contributions

1. Problem-Driven Architecture: The paper introduces a framework specifically designed for heterogeneous medical modalities, moving away from generic stacking of modules to a tailored encoder-fusion-decoder approach.
2. Modality-Specific Enhancement: The MAE module bridges the representation gap between anatomical and functional images by applying distinct enhancement strategies before deep interaction.
3. Lightweight Spatial-Gated Mamba: The SGMam module successfully adapts the Mamba architecture for 2D medical images, combining local inductive biases with global context modeling while maintaining linear complexity.
4. Statistical-Guided Fusion: The SDAF layer offers an efficient alternative to attention-based fusion, utilizing channel-wise statistical priors to dynamically calibrate weights without significant computational overhead.

Experimental Results
The authors evaluated MESFusion on three public datasets (CT-MRI, PET-MRI, SPECT-MRI) and an external cross-dataset test (NAF-PROSTATE PET/CT).

• Quantitative Performance: MESFusion achieved competitive or superior results across multiple metrics, including Visual Information Fidelity (VIF), Correlation Coefficient (CC), Edge-based Fusion Quality (QAB/F), and Multi-Scale Structural Similarity (MS-SSIM). Notably, it demonstrated strong performance in PSNR and MS-SSIM, indicating superior structural preservation.
• Ablation Studies: Removing any core component (MAE, SGMam, or SDAF) resulted in measurable degradation in visual quality and metric scores, confirming the synergistic effect of the proposed modules. The inclusion of both mean and standard deviation in SDAF was shown to be superior to using mean alone.
• Generalization: On the unseen NAF-PROSTATE dataset, the method maintained robust fusion quality, outperforming other methods in preserving lesion contrast and anatomical boundaries despite distribution shifts.
• Efficiency: The proposed method is highly efficient, with significantly lower FLOPs (3.511G) and parameter counts (31.6K) compared to state-of-the-art methods like DDBFusion and DM-FNet, while achieving faster inference times (0.056s).

Significance and Claims
The paper claims that MESFusion provides a robust and efficient solution for multimodal medical image fusion by effectively addressing the heterogeneity of feature distributions and the computational costs of global modeling. The authors emphasize that their approach achieves a favorable trade-off between structural preservation, functional information retention, and computational efficiency. The framework is presented as a viable option for resource-constrained clinical environments, offering visually coherent results with clear anatomical contours and precise functional representation. The authors acknowledge that the current loss function weighting relies on manual tuning and identify adaptive loss-weight optimization as a direction for future work.