Robust Renal Mass Segmentation on CT: A Validation Study of an AI-Based Framework

Technical Summary: Robust Renal Mass Segmentation on CT

Problem Statement
Renal mass segmentation is critical for clinical workflows involving quantitative assessments, such as monitoring kidney volume changes in diseases like autosomal dominant polycystic kidney disease (ADPKD) or staging renal tumors based on diameter and volume. Current clinical practice relies heavily on subjective visual assessment, which suffers from inter-rater variability and is time-consuming. While general medical image segmentation models (e.g., TotalSegmentator) have advanced, they often underperform on specialized tasks like renal mass segmentation, particularly when tumors significantly alter organ shape and internal structure. Furthermore, existing models trained on specific datasets (e.g., KiTS) often struggle to generalize to external, heterogeneous clinical cohorts, highlighting a gap between algorithmic development and robust clinical adoption.

Methodology
The authors developed Renal-Net, a robust segmentation algorithm based on the state-of-the-art nnU-Netv2 framework (specifically the Residual Encoder Large configuration).

• Training Data: The model was trained exclusively on two publicly available datasets to ensure reproducibility and generalizability:
  1. KiTS23: 489 CT scans from US patients with suspected renal cancer, annotated with kidney, tumor, and cyst labels.
  2. Radboudumc Kidney Abnormality Dataset: 215 contrast-enhanced CT scans from Dutch oncology patients, containing healthy kidneys and various lesions (cysts, masses, metastases).
  • Data Harmonization: To merge these datasets, the authors unified the annotation protocols. Cysts and tumors from KiTS23 were merged into a single "renal mass" category to align with the Radboudumc dataset. The kidney region definitions (inclusion/exclusion of the hilum) were left as originally annotated to expose the model to annotation variability, hypothesizing this improves robustness.
• Architecture & Post-processing: The model utilizes a full-resolution nnU-Net. A post-processing pipeline was applied to remove predicted renal mass regions not attached to the kidney (unless the volume exceeded 100 cm³, to account for large tumors suppressing kidney visibility) and to filter out small axial diameters (<3 mm) to reduce false positives.
• Validation Strategy: The study employed a large-scale validation approach across three independent test datasets:
  1. Radboudumc Private Test Set: 50 CT scans (20 healthy, 30 with lesions).
  2. TCGA-KIRC: 28 scans of clear cell renal cell carcinoma patients.
  3. Charité Universitätsmedizin Berlin: 1,510 preoperative CT scans from 917 patients with diverse tumor subtypes and contrast phases.
• Evaluation Metrics: Performance was quantified using the Dice coefficient (overlap) and the 95th percentile Hausdorff distance (boundary accuracy). Detection performance was assessed via precision, recall, and F1-scores with an Intersection over Union (IoU) threshold of 0.5. Subgroup analyses were conducted for patient sex, age, CT contrast phase, and tumor histologic subtype.

Key Contributions

1. Development of Renal-Net: A specialized, robust segmentation model for kidneys and renal masses trained solely on public data.
2. Large-Scale External Validation: The study provides the largest external validation in this domain to date, testing on over 1,500 CT scans across three distinct medical centers with diverse demographics and imaging protocols.
3. Comprehensive Subgroup Analysis: Detailed analysis demonstrating the model's robustness across patient sex, age, contrast phases, and tumor histologic subtypes.
4. Open Science: The model weights, code, and a ready-to-use algorithm are publicly available via Zenodo, GitHub, and grand-challenge.org.

Results

• Segmentation Performance: Renal-Net significantly outperformed reference models (TotalSegmentator and the BAMF model) across all datasets for the "Kidney," "Kidney+Mass," and "Renal Mass" regions (corrected p-values < 0.001 for Dice scores).
  • On the TCGA-KIRC dataset, Renal-Net achieved a mean Dice of 0.95 for the kidney region and 0.86 for renal masses, compared to 0.80 and 0.05 respectively for TotalSegmentator.
  • On the Charité dataset, Renal-Net and the BAMF model showed comparable performance, though Renal-Net demonstrated superior performance on difficult cases (5th percentile Dice of 0.53 vs. 0.39 for BAMF on renal masses).
• Healthy Kidneys: On healthy kidney subsets, Renal-Net performed comparably to TotalSegmentator (Dice ~0.93-0.95), though TotalSegmentator showed slightly lower HD95 scores.
• Human-Level Performance: On the Radboudumc private test set, Renal-Net's performance (Dice 0.95 for kidney+mass) fell within the range of human inter-observer variability (Dice 0.941).
• Detection: Renal-Net demonstrated superior detection performance (Precision, Recall, F1-score) compared to baselines across all test sets. For instance, on the Charité dataset, Renal-Net achieved an F1-score of 0.78 compared to 0.70 for the BAMF model.
• Robustness: Subgroup analyses revealed consistent high performance across different patient sexes, age groups, CT contrast phases (early venous, delayed venous, arterial), and tumor subtypes (ccRCC, pRCC, chrRCC, oncocytoma).

Significance and Claims
The paper claims that combining publicly available datasets with the nnU-Net framework is sufficient to develop a segmentation model that generalizes effectively to external, diverse clinical settings without the need for proprietary training data. The authors emphasize that their approach prioritizes robustness over marginal gains in accuracy on specific datasets, achieved by training on heterogeneous data with varying annotation protocols and imaging characteristics.

The study positions Renal-Net as a tool to support objective and reproducible clinical workflows, such as tracking kidney volume changes for active surveillance or disease progression monitoring. The authors acknowledge limitations, including the model's inability to process non-contrast CT scans and the lack of pediatric or non-Western demographic data in the validation sets. They conclude that while the model is ready for clinical testing and further research, future work should address these gaps and explore the potential of separating cystic and solid lesion segmentation.