D3Seg: Dependency-Aware Diffusion for Brain Tumor Segmentation with Missing Modalities

Technical Summary: D3Seg – Dependency-Aware Diffusion for Brain Tumor Segmentation with Missing Modalities

Problem Statement
Accurate brain tumor segmentation using multi-parametric MRI is essential for treatment planning. Standard clinical protocols typically acquire four sequences: T1, contrast-enhanced T1 (T1ce), T2, and FLAIR. However, in practice, acquiring all modalities is not always feasible due to varying institutional protocols or patient contraindications to contrast agents. Existing segmentation methods often suffer significant performance degradation when modalities are missing. Current approaches generally fall into two categories:

1. Fusion-based methods: These rely on naive feature concatenation or statistical aggregation of available modalities but lack mechanisms to explicitly compensate for missing inputs.
2. Reconstruction-based methods: These attempt to synthesize all missing modalities or learn shared latent representations. However, reconstructing all missing features indiscriminately can introduce unnecessary computational overhead.
3. Knowledge Distillation: These rely on teacher-student networks but incur additional training overhead and depend heavily on teacher network reliability.

Furthermore, existing models often struggle with the delineation of the Enhancing Tumor (ET), a clinically critical sub-region, particularly when T1ce data is absent, often leading to under-segmentation of minority classes and overconfidence in dominant classes like edema.

Methodology
The authors propose D3Seg, a dependency-aware diffusion-imputed segmentation network designed to maintain stable performance under missing-modality settings. The architecture processes each available MRI modality through independent 3D convolutional encoders. The core components are:

1. Multi-hop Modality Graph Fusion (MMGF):
   Instead of simple average fusion, D3Seg models higher-order inter-modality dependencies. It constructs a modality adjacency graph where nodes represent modality embeddings (derived via global average pooling). An adjacency matrix is computed based on the cosine similarity between embeddings. When a modality is missing, the corresponding rows/columns are masked, except for T1ce features at the bottleneck, which utilize either available data or diffusion-imputed representations. The model propagates information across multiple graph hops (up to 3) using learnable hop weights, allowing information to flow through intermediate modalities to enrich the representation of available ones. This is applied at the network bottleneck and skip connections.

2. Diffusion-Based Latent T1ce Imputation:
   Recognizing the critical role of T1ce for delineating the tumor core, the authors introduce a lightweight, unconditional diffusion mechanism. This operates directly in the latent feature space of the T1ce encoder (at the bottleneck). A lightweight transformer-based denoiser (3 attention layers) learns to remove injected Gaussian noise to synthesize T1ce feature representations when the modality is missing. This approach avoids full reconstruction of all modalities, focusing only on the clinically critical T1ce features, thereby maintaining parameter efficiency.

3. Error-Guided Decision Refinement (EGDR):
   To address the tendency of models to be overconfident in dominant classes (e.g., edema) and under-segment minority classes (e.g., enhancing tumor), a decision refinement module operates in the probability space. A lightweight error predictor estimates a voxel-wise error likelihood map. This map guides an adaptive redistribution of probability mass, transferring confidence from edema (ED) to the enhancing tumor (ET) in regions prone to under-segmentation. This ensures a more balanced prediction of tumor sub-regions.

Key Contributions
The paper outlines three primary contributions:

1. Dependency-Aware Fusion: A novel MMGF approach that constructs a modality adjacency graph to model direct and indirect inter-modality relationships via multi-hop propagation, enabling more expressive aggregation than simple fusion.
2. Targeted Latent Imputation: A diffusion-based mechanism that synthesizes clinically critical T1ce feature representations at the network bottleneck. This provides targeted compensation for missing T1ce information without the computational cost of reconstructing all modality features.
3. Decision Refinement: A unique module that explicitly mitigates dominant-class overconfidence by adaptively redistributing probability mass toward minority tumor subregions (specifically ET) under missing-modality ambiguity.

Experimental Results
The model was evaluated on the BraTS 2023 dataset (1,251 scans), split into 70% training, 10% validation, and 20% testing. Performance was measured using the Dice Similarity Coefficient (DSC) for Whole Tumor (WT), Tumor Core (TC), and Enhancing Tumor (ET) across various missing-modality configurations.

• Performance Gains: D3Seg consistently outperformed state-of-the-art methods (including M3AE, M3FeCon, and IM-Fuse) across multiple missing-modality scenarios.
  • It achieved approximately 1.5–2.0% Dice improvement on the Enhancing Tumor (ET) and around 1.0% on Tumor Core (TC) compared to the current state-of-the-art (IM-Fuse).
  • Notably, the model maintained competitive or superior performance even with fewer parameters (38M vs. 47M for IM-Fuse) and lower computational cost (236 vs. 249 GFLOPs).
• Ablation Studies: Experiments confirmed that each component contributes to performance. Specifically, MMGF improved ET Dice by 1.3% over a Mamba-fusion-only baseline, while diffusion-based imputation added a further 1.0% gain to TC Dice. The full model achieved the highest scores (92.5% WT, 80.8% TC, 64.9% ET in the missing-T1ce setting).
• Qualitative Analysis: Visual results demonstrated that MMGF yields more localized and contrast-enhanced responses in the ET area compared to standard fusion, leading to better alignment with ground truth.

Significance and Claims
The authors claim that D3Seg offers a robust solution for clinical scenarios where complete MRI acquisition is impossible. By combining graph-based dependency modeling with targeted diffusion imputation and probability-space refinement, the model addresses the specific weaknesses of existing methods: the inability to compensate for missing inputs and the bias toward dominant classes. The paper emphasizes that these improvements are achieved while maintaining computational efficiency, making the approach practical for real-world deployment. The results suggest that explicitly modeling inter-modality dependencies and selectively imputing critical features is more effective than indiscriminate reconstruction or simple fusion.