💻 computer science

MNet++: Extended 2D/3D Networks for Anisotropic Medical Image Segmentation

This paper validates the reproducibility of the MNet architecture for anisotropic medical image segmentation within the nnU-Net framework and introduces two lightweight extensions—a learned Fusion Gating mechanism and a VMamba state-space module—that further enhance segmentation accuracy and consistency across variable voxel spacings.

Original authors: Kirsten Odendaal, Rade Bajic

Published 2026-06-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Kirsten Odendaal, Rade Bajic

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Fixing "Chunky" Medical Scans

Imagine you are trying to build a 3D model of a house using a stack of 2D blueprints.

The Problem: In the real world, medical scans (like MRIs and CTs) are often "chunky." The pictures inside the slice (left-to-right) are very sharp and detailed, but the distance between the slices (top-to-bottom) is wide and blurry. This is called anisotropy.
The Old Way:
- If you use a 2D-only brain (a computer looking at one slice at a time), it sees the sharp details but misses how the house connects vertically. It's like trying to guess the shape of a building by looking at one floor plan and ignoring the stairs.
- If you use a 3D-only brain (looking at the whole stack at once), it gets confused by the "chunky" gaps between slices. It tries to guess the connection, but often overfits or gets the shape wrong because the data is sparse.

The Original Solution: MNet (The Hybrid Architect)

The original paper introduced MNet, a clever "hybrid" architect. Instead of choosing between 2D or 3D, MNet has two teams working simultaneously:

Team 2D: Focuses on the sharp details within each slice.
Team 3D: Focuses on how the slices connect vertically.

These teams constantly talk to each other and combine their findings. The original MNet was great, but it had a rigid rulebook. It decided how to combine the teams' work using fixed math (like "always add them together" or "always subtract"). It couldn't adapt if one team was having a bad day or if a specific part of the image was tricky.

The New Study: MNet++ (The Upgraded Architect)

The authors of this paper did two main things:

They rebuilt MNet from scratch to prove it actually works as advertised.
They gave MNet two new "superpowers" to make it smarter and more efficient.

Superpower 1: The "Smart Switch" (Fusion Gating)

In the original MNet, the two teams (2D and 3D) were forced to mix their notes using a fixed recipe.

The Upgrade: The new Fusion Gating is like a smart traffic controller. Instead of a fixed rule, it looks at every single pixel and asks: "Right now, is the 2D team more reliable, or is the 3D team?"
How it works: It learns to blend the two teams' information dynamically. If the vertical connection is blurry, it trusts the 2D team more. If the slice is noisy, it trusts the 3D team.
The Result: This made the model slightly better at drawing the edges of organs (like the prostate) without slowing it down much.

Superpower 2: The "Long-Distance Look" (VMamba)

Medical scans often have thick slices, making it hard for the computer to "see" the whole picture from top to bottom. Standard 3D computers have a short memory span.

The Upgrade: The authors added a module called VMamba. Think of this as a telescope that looks down the "depth" of the scan (the z-axis).
How it works: Instead of trying to process the whole 3D block at once (which is heavy and slow), VMamba scans the slices one by one in a very efficient sequence. It remembers what it saw at the top of the stack while it's looking at the bottom.
The Result: This helped the model understand the "big picture" of the organ better, especially for the liver, leading to the most accurate results in the study.

Did It Work? (The Results)

The team tested these upgrades on two types of medical data:

Prostate MRI (PROMISE): These scans are very "chunky" (large gaps between slices).
Liver CT (LiTS): These scans are sharper but still have gaps.

The Findings:

Reproduction: They successfully rebuilt the original MNet and got almost the exact same scores as the original authors. This proves the original idea was solid.
The Upgrades:
- The Smart Switch (Fusion Gating) improved the accuracy slightly and made the model more consistent.
- The Long-Distance Look (VMamba) was the star for the liver scans, achieving the highest accuracy scores (95.8%) and making the model more stable.
Robustness: Even when they artificially made the scans "chunkier" (wider gaps between slices), the new models didn't fall apart as much as the old ones. They stayed reliable.

The Catch (Limitations)

The authors were honest about their constraints:

Computer Power: They didn't have the massive supercomputers the original authors might have used. They had to train the models for fewer "epochs" (learning cycles).
Data Size: For the liver dataset, they used a smaller sample size (50 cases instead of 131). This made it harder to train the model to recognize tricky, small tumors perfectly. The liver segmentation was great, but the tumor segmentation was still a bit hit-or-miss, likely because the tumors are rare and irregular, not because the model is broken.

Summary

Think of MNet++ as taking a good, reliable car (the original MNet) and adding adaptive cruise control (Fusion Gating) and a better navigation system (VMamba). It doesn't change the engine, but it makes the car drive smoother, handle bumps (anisotropy) better, and get to the destination (accurate segmentation) more reliably, especially when the road conditions (image quality) aren't perfect.

Technical Summary: MNET++: EXTENDED 2D/3D NETWORKS FOR ANISOTROPIC MEDICAL IMAGE SEGMENTATION

Problem Statement

Three-dimensional medical image segmentation faces a persistent challenge regarding anisotropic voxel spacing, where the spacing between slices (z-axis) is significantly larger than the in-plane resolution (x and y axes). This is common in clinical MRI and CT acquisitions. Standard 3D Convolutional Neural Networks (CNNs) often overfit to sparsely sampled through-plane regions, while 2D CNNs fail to capture volumetric context. While hybrid approaches like 2.5D U-Nets or the original MNet attempt to reconcile these regimes, they often rely on hand-crafted fusion rules or lack modern mechanisms for efficient long-range contextual reasoning along the depth axis.

Methodology

The study focuses on two primary objectives: the faithful reproduction of the original MNet architecture and the introduction of lightweight extensions to enhance its adaptability and efficiency.

1. Reproduction Framework

The authors re-implemented MNet within the nnU-Net framework to ensure consistency with standard preprocessing pipelines and to verify reported performance under realistic compute constraints.

Datasets: Experiments were conducted on PROMISE12 (prostate MRI) and a controlled subset of LiTS (liver CT).
Configuration: The reproduction utilized the original MNet architecture (5 encoder/decoder stages with parallel 2D/3D streams) trained for 150 epochs using Dice loss and deep supervision.
Anisotropy Testing: Models were evaluated across varying z-spacings (1 mm, 2.2/2.0 mm, and 4 mm) to assess robustness.

2. Proposed Extensions

Two specific architectural modifications were introduced to the re-implemented MNet:

Fusion Gating Mechanism:
- Problem: The original MNet uses fixed, hard-coded operations (addition, subtraction, or concatenation) to merge 2D and 3D feature maps.
- Solution: A learned Fusion Gating module replaces these static rules. It predicts a soft gate $g \in [0, 1]$ to interpolate between 2D and 3D features: $y = g \odot x_{2D} + (1-g) \odot x_{3D}$ .
- Variants: The study explored Channel Gating (global confidence per channel) and Spatial Gating (location-specific trust maps shared across channels). This allows the model to adaptively weight feature streams based on local image quality or anisotropy severity.
VMamba Integration:
- Problem: Standard 3D convolutions struggle to model long-range dependencies along the z-axis in thick-slice scans without excessive computational cost.
- Solution: The authors integrated VMamba (a state-space recurrent module) into the bottleneck stages (3, 4, and 5) of the encoder.
- Mechanism: The input volume is reshaped to treat each spatial position $(H, W)$ as an independent sequence along the depth $D$ . A z-axis selective scan processes these sequences with $O(D)$ complexity, capturing inter-slice dependencies efficiently. This reduces the parameter count by replacing heavy 3D convolutions in the bottleneck.

Key Results

Reproduction Validation

The re-implemented MNet successfully matched the performance trends of the original paper within the accepted reproducibility margin ( $\pm 2\text{--}3\%$ ):

PROMISE: Achieved 89.0 ± 0.9% Dice Similarity Coefficient (DSC), within 0.8% of the original 89.8%.
LiTS: Achieved 94.3 ± 1.9% for liver and 54.6 ± 3.1% for tumor segmentation. The lower tumor score is attributed to the smaller training subset (50 cases vs. 131) and reduced training epochs (150 vs. 500).

Performance of Extensions

Spatial Gating: Improved PROMISE performance by +0.8% (reaching 90.0% DSC) with negligible parameter overhead (<3% inference cost). It demonstrated superior stability under varying anisotropy, with a smaller DSC drop (1.5 points) between 1mm and 4mm spacing compared to the baseline.
VMamba: Reduced model parameters from 8.77M to 7.42M. It achieved the strongest liver segmentation performance on LiTS (95.8 ± 0.7% DSC) and improved performance consistency, reducing PROMISE Dice variation to ±0.7%.
Combined Model: The combination of Spatial Gating and VMamba achieved 95.9 ± 0.3% DSC on LiTS liver, confirming the complementary nature of adaptive fusion and depth-aware modeling.

Anisotropy Robustness

Both extensions preserved MNet's robustness to anisotropy. Across 1–4 mm voxel spacing, the Dice variation remained approximately 1.5%, demonstrating that learned fusion and state-space modeling do not compromise the network's ability to handle variable slice thickness.

Significance and Claims

The paper claims three primary contributions:

Reproducibility: It validates that the hybrid 2D/3D fusion strategy of MNet is feasible and robust under realistic compute budgets, confirming the original paper's core premise that adaptive dimensional representation is key for anisotropic segmentation.
Adaptive Fusion: It demonstrates that replacing hard-coded fusion rules with learned gating mechanisms allows the network to dynamically blend 2D and 3D features, improving boundary precision and stability without significant computational overhead.
Efficient Long-Range Modeling: It shows that integrating VMamba blocks can efficiently model long-range depth dependencies in anisotropic volumes, enhancing feature coherence and reducing parameter counts.

Limitations and Modest Conclusions

The authors maintain a modest stance regarding their findings:

Dataset Constraints: The LiTS results, particularly for tumor segmentation, are limited by the use of a 50-case subset and shorter training schedules. The authors note that tumor performance is heavily dependent on lesion morphology and data volume, and full-dataset training with extended schedules may be required to unlock further gains.
Convergence: While 150 epochs were sufficient for the PROMISE dataset (where performance plateaued by epoch 100–120), the LiTS loss curves showed erratic behavior, suggesting that longer training or more data might be necessary for complex anatomical structures.
Future Work: The study acknowledges that further testing is required to provide definitive conclusions, particularly regarding the generalization of these extensions to other multi-class tasks and the potential of advanced VMamba variants (e.g., bidirectional scanning) on larger computational resources.

In summary, the study confirms the reproducibility of MNet and provides evidence that adaptive fusion and state-space attention can further strengthen segmentation reliability under anisotropic conditions, though definitive conclusions on tumor segmentation require more extensive validation.