Med-PU: Point Cloud Upsampling for High-Fidelity 3D Medical Shape Reconstruction

Technical Summary: Med-PU for High-Fidelity 3D Medical Shape Reconstruction

Problem Statement
High-fidelity 3D anatomical reconstruction is essential for downstream clinical tasks, including preoperative planning, radiotherapy target delineation, and orthopedic implant design. However, a significant gap exists between standard voxel-wise segmentation outputs (derived from medical imaging) and clinically usable 3D models. Directly converting discrete voxel masks into smooth, watertight, and anatomically faithful meshes is challenging due to partial observations, image noise, and inter-subject variability. Traditional Statistical Shape Models (SSMs) attempt to address variability but rely on manual landmark annotation, explicit correspondence, and Principal Component Analysis (PCA), which limits their scalability and automation on large medical datasets. Conversely, modern deep learning segmentation networks often produce meshes that are jagged, topologically inconsistent, or over-smoothed, particularly when input resolution is limited.

Methodology
The authors propose Med-PU, a knowledge-driven framework that bridges the gap between segmentation outputs and high-quality 3D meshes by integrating volumetric medical image segmentation with deep point cloud upsampling. The pipeline operates in three sequential stages:

1. Volumetric Segmentation: The framework utilizes SAM-Med3D (a 3D extension of the Segment Anything Model) to perform voxel-wise segmentation on CT data, generating coarse anatomical masks.
2. Point Extraction and Upsampling: These masks are converted into sparse point clouds. A deep upsampling network, trained on large-scale 3D medical shape data (specifically the MedShapePelvic subset of MedShapeNet), processes these sparse points. Unlike classical SSMs, this network learns an implicit anatomical prior directly from the data distribution rather than relying on explicit parametric modeling. The network maps the sparse, segmentation-derived distribution toward the ground-truth anatomical manifold, effectively densifying and regularizing the point set.
3. Surface Reconstruction: The resulting dense point cloud undergoes surface reconstruction using the Marching Cubes algorithm (with a voxel grid size of 1.5) to produce smooth, topologically consistent triangular meshes.

The learning objective minimizes a symmetric Chamfer discrepancy between the predicted upsampled points and the target dense points, embedding shape regularity and population variability into the network parameters without requiring landmarks or PCA.

Key Contributions

• Implicit Prior Integration: The paper introduces a pipeline that couples SAM-Med3D-based segmentation with deep point cloud upsampling to learn implicit anatomical priors. This approach densifies and regularizes sparse segmentation-derived point sets into smooth, watertight meshes without manual landmark annotation or PCA.
• Comprehensive Evaluation: The method is trained on the MedShapePelvic dataset and validated on the Pelvic1k dataset. It is benchmarked against state-of-the-art upsampling methods (including PU-GCN, Grad-PU, and PUCRN) using a suite of geometry and surface metrics (Chamfer/Hausdorff distances, Point-to-Surface error, F-score, Normal Consistency, Edge-aware measures, Area-Length Ratio, Manifoldness Rate, and Connected Component Discrepancy).
• Anatomy-Agnostic Potential: While validated specifically on pelvic bones, the authors posit that the framework is anatomy-agnostic and applicable to other skeletal regions and organs, offering a generalizable bridge from voxel segmentation to 3D models.

Experimental Results
Experiments conducted on pelvic CT datasets demonstrate that Med-PU consistently improves surface quality and anatomical fidelity compared to baseline methods:

• Upsampling Performance: In 4× upsampling tasks, the PUCRN-based implementation of Med-PU outperformed other methods (MPU, PU-GCN, Grad-PU) across various input scales (512 to 4,096 points) in terms of Chamfer Distance (CD), Hausdorff Distance (HD), and Point-to-Surface (P2F) error. Notably, training on the domain-specific MedShapePelvic dataset yielded significantly better results than training on the synthetic PU1k dataset, highlighting the importance of domain-aligned supervision.
• Surface Quality: Quantitative analysis of the reconstructed meshes showed that Med-PU (specifically with PUCRN) achieved superior mesh quality metrics, including higher Area-Length Ratios (indicating better triangle shape) and reduced Connected Component Discrepancies (indicating fewer topological artifacts) compared to raw segmentation results.
• Visual Fidelity: Visualizations confirmed that the proposed framework generates smoother surfaces with more detailed structures and fewer layer artifacts compared to direct Marching Cubes reconstruction from voxel masks.

Significance and Claims
The paper claims that Med-PU serves as a practical, generalizable tool to bridge the gap between segmentation outputs and clinically usable 3D models. By shifting from explicit parametric modeling to learned implicit priors, the framework enables flexible, automated reconstruction under varying input densities. The authors emphasize that their approach avoids the scalability and automation constraints of classical SSMs while addressing the jaggedness and topological inconsistencies of direct mesh reconstruction. Although currently validated on the pelvis, the study suggests the approach is robust across input densities and applicable to broader biomedical contexts, provided that domain-specific training data is available to capture the specific anatomical variations of the target region.