A Unified Benchmark of Deep Learning Models for Multi-task 3D Brain Tumor Segmentation from Magnetic Resonance Imaging

Technical Summary: A Unified Benchmark of Deep Learning Models for Multi-task 3D Brain Tumor Segmentation from MRI

Problem Statement
Automatic brain tumor segmentation from Magnetic Resonance Imaging (MRI) is critical for diagnosis, treatment planning, and monitoring. While deep learning architectures have advanced significantly, objective comparison of these methods remains challenging. Published studies often utilize heterogeneous experimental conditions, including different datasets, preprocessing pipelines, training protocols, and evaluation metrics. Furthermore, many comparisons focus solely on segmentation accuracy while neglecting computational efficiency, a vital factor for real-world clinical deployment. This lack of standardization makes it difficult to determine whether performance differences stem from intrinsic architectural capabilities or experimental variations.

Methodology
To address these limitations, the authors developed a unified, reproducible experimental benchmark evaluating five representative 3D segmentation architectures under identical conditions. The study compares three architectural families:

1. Convolutional Neural Networks (CNNs): 3D U-Net and SegResNet.
2. Transformer-based Models: Swin UNETR.
3. State Space Models (SSMs): SegMamba and SegMambaV2 (derived from the Mamba framework).

Datasets and Experimental Protocol
The evaluation was conducted on two distinct clinical scenarios from the Brain Tumor Segmentation (BraTS) challenges:

• BraTS 2023: Intracranial meningioma segmentation, characterized by well-defined boundaries but variability in size and location.
• BraTS 2024: Post-treatment glioma segmentation, a more complex scenario involving resection cavities, edema, and necrotic tissue.

All models were trained using a standardized pipeline implemented via the MONAI library. Key methodological controls included:

• Data Handling: Identical preprocessing (rigid co-registration, resampling to 1mm³ isotropic resolution, skull-stripping), data augmentation (random crops, flips, intensity scaling), and 5-fold cross-validation splits derived from annotated training data.
• Training Configuration: All models were trained from scratch using the AdamW optimizer and a combined Dice Cross-Entropy loss function to address class imbalance. Hyperparameters (learning rates, batch sizes) were standardized, with minor adjustments for SSM stability (e.g., gradient clipping).
• Evaluation Metrics: Performance was assessed using volumetric overlap (Dice Similarity Coefficient, Intersection over Union), boundary accuracy (95th-percentile Hausdorff Distance), and computational efficiency (inference time and parameter count).

Key Results
The experiments yielded the following findings across both datasets:

• Overall Performance: State Space Models (SSMs), specifically SegMamba and SegMambaV2, consistently achieved the highest global segmentation accuracy and the most precise boundary delineation (lowest HD95) across both meningioma and post-treatment glioma tasks.
• Sub-region Analysis: SSMs demonstrated superior robustness in segmenting rare and small sub-regions (e.g., Non-Enhancing Tumor Core). However, Transformer-based Swin UNETR showed competitive performance, particularly in delineating specific anatomical structures like the Enhancing Tumor in post-treatment scenarios.
• Efficiency Trade-offs:
  • 3D U-Net offered the fastest inference (0.2s) and lowest parameter count (2M) but yielded the lowest accuracy.
  • SegResNet provided a strong balance between accuracy and efficiency, suitable for hardware-constrained environments.
  • Swin UNETR required significantly higher computational resources (62M parameters) and inference time (1.4s), with variable accuracy depending on the dataset.
  • SegMambaV2 achieved the best accuracy-efficiency trade-off for high-performance needs, outperforming the baseline SegMamba in speed (approximately 1.55–1.70s vs. 1.85–1.95s) despite having a larger parameter count (~138M vs. ~67M).
• Comparison with State-of-the-Art: When compared to top-performing challenge entries (which utilized ensembling and extensive training), the single-model architectures in this benchmark achieved highly competitive Dice scores, particularly the SSMs. However, the authors note that direct comparison of spatial error metrics (HD95) is complicated by differences in evaluation methodologies (lesion-wise vs. voxel-wise).

Significance and Contributions
The paper claims four primary contributions:

1. Unified Framework: It proposes a fully reproducible experimental framework that evaluates modern 3D brain tumor segmentation architectures under identical training and evaluation conditions, eliminating confounding experimental variables.
2. Comprehensive Comparison: It provides a systematic comparison of CNN, Transformer, and SSM architectures across two clinically distinct and challenging BraTS datasets.
3. Accuracy-Efficiency Analysis: It offers practical insights into the trade-offs between predictive performance and computational cost, highlighting the suitability of different paradigms for specific clinical deployment scenarios.
4. Architectural Behavior: It analyzes how different model families behave across varying tumor morphologies and clinical contexts, identifying the strengths of SSMs in global context modeling and boundary precision.

The authors conclude that while SSMs currently offer the most robust performance for complex 3D segmentation tasks, CNNs remain viable for efficiency-critical applications, and Transformers retain value for specific anatomical delineations. The study underscores that architectural choices must be weighed against both clinical requirements and computational constraints.