Submanifold Sparse Convolutional Networks for Automated 3D Segmentation of Kidneys and Kidney Tumours in Computed Tomography

Technical Summary: Submanifold Sparse Convolutional Networks for Automated 3D Segmentation of Kidneys and Kidney Tumours

Problem Statement
Accurate delineation of kidney tumours in Computed Tomography (CT) is critical for quantitative analysis and precision oncology but remains a bottleneck due to the time-consuming nature of manual segmentation. While deep learning, particularly U-Net architectures, has shown promise, standard dense convolutional neural networks (CNNs) face significant computational hurdles when processing large, high-resolution 3D medical volumes. A single CT scan can contain tens of millions of voxels, often exceeding the memory capacity of typical GPUs, necessitating downsampling or patch-based inference which can compromise resolution and context. Furthermore, dense representations are inefficient as they process vast amounts of background information (e.g., air, bones) that is irrelevant to the segmentation task.

Methodology
The authors propose a two-stage 3D segmentation framework leveraging Submanifold Sparse Convolutional Networks (SSCNs) and voxel sparsification to address these computational constraints while maintaining high resolution.

1. Voxel Sparsification: The process begins by filtering input images to retain only voxels within specific Hounsfield Unit (HU) ranges relevant to renal tissue. In the first stage, a threshold range of (-53.4, 283.2) HU is applied, preserving 99% of segmentation voxels while removing ~77% of background voxels.
2. Stage 1: ROI Finder (Detection): A low-resolution sparse U-Net processes the sparsified volume to identify a Region of Interest (ROI) containing kidneys and masses. The output is a dilated bounding box to ensure all relevant structures are captured.
3. Stage 2: Full Segmentation: The identified ROIs are used to crop the original high-resolution scans. A second, high-resolution sparse U-Net performs refined segmentation on these cropped volumes.
4. Network Architecture: The model utilizes a custom sparse U-Net based on ConvNeXtV2 blocks, implemented via a modified MinkowskiEngine that supports depth-wise convolutions. The architecture employs an encoder-decoder structure with skip connections via element-wise summation (to reduce memory footprint) and deep supervision in the final three decoder stages. The loss function is a weighted Dice loss to address class imbalance.
5. Training: The method was evaluated on the KiTS23 dataset (489 training cases) using 5-fold cross-validation. Data augmentation included affine transformations, intensity scaling, and noise injection.

Key Contributions

• First Application of SSCNs in CT Tumour Segmentation: This work represents the first application of Submanifold Sparse Convolutional Networks to the semantic segmentation of tumours in CT images, extending their use beyond high-energy physics and limited skull reconstruction tasks.
• Two-Stage Sparse Pipeline: The authors demonstrate a novel two-stage approach that combines low-resolution detection with high-resolution sparse refinement, enabling native 3D processing at full resolution without downsampling the entire volume.
• Resource Efficiency: The study provides a direct comparison between the proposed sparse architecture and equivalent dense implementations, as well as established baselines (nnU-Net and the foundation model SegVol), highlighting significant reductions in computational cost.

Results
Evaluated on the KiTS23 dataset using 5-fold cross-validation, the proposed method achieved the following Dice Similarity Coefficients (DSC):

• Kidneys + Masses: 95.8%
• Tumours + Cysts: 85.7%
• Tumours Alone: 80.3%

Comparative Performance:

• vs. nnU-Net: The sparse method achieved comparable overall performance (macro-average 87.3% vs. 86.7% for nnU-Net). While slightly lower for "Kidneys + Masses," it performed numerically better for lesion-focused outputs ("Tumour + Cyst" and "Tumour alone").
• vs. SegVol: The foundation model (zero-shot) localized kidneys well (91.4% DSC) but failed on small, heterogeneous lesions (19.5% DSC for tumours alone), underscoring the need for task-specific training.
• vs. Dense Implementation: Compared to an equivalent dense U-Net architecture, the sparse approach reduced inference time by up to 60% and VRAM usage by up to 75%.
• Hardware Feasibility: The dense baseline frequently encountered Out-of-Memory (OOM) errors on 16GB GPUs, whereas the sparse model successfully processed full-resolution volumes on the same hardware.

Significance and Claims
The paper claims that SSCNs offer a feasible and resource-efficient alternative for 3D medical image segmentation. The authors assert that their method achieves accuracy competitive with state-of-the-art dense frameworks (including top-performing KiTS23 teams) while substantially reducing the computational resources required for training and inference.

Key implications highlighted by the authors include:

• Clinical Deployment: The reduced memory footprint allows for full-resolution 3D processing on hardware with limited VRAM (e.g., 16GB GPUs), facilitating deployment in clinical settings with constrained computational resources.
• Sustainability: By lowering inference time and memory usage, the approach contributes to reduced energy consumption and a smaller carbon footprint for large-scale medical imaging workflows.
• Generalizability: While the study focuses on kidneys, the authors suggest the sparsification and SSCN methodology could be extended to other organs and tumour types, as well as other modalities like MRI or PET where voxel intensities can be thresholded to isolate relevant structures.

The authors remain modest regarding their claims, noting limitations such as the dataset size (489 cases), the lack of access to the KiTS23 test set labels for direct leaderboard comparison, and the absence of post-processing steps which were used by top-ranking teams in the challenge. They identify future work in exploring larger multi-center datasets and alternative single-stage sparse designs.