Depth to Anatomy: Organ Localization from Depth Images for Automated Patient Table Positioning in Radiology Workflow

Technical Summary: Depth to Anatomy

Problem Statement
In clinical radiology, the scan preparation and planning phase is a significant bottleneck, consuming an average of 5.7 minutes (11% of total examination time). This process relies heavily on manual patient positioning by radiologic technologists, who use external landmarks and laser guides to approximate the region of interest, followed by low-resolution scout imaging to verify alignment. This manual workflow is operator-dependent, time-consuming, and prone to variability, which can lead to inconsistent positioning, repeat scans, and reduced scanner throughput. While automated patient positioning systems exist, they often rely on external landmark detection or do not explicitly infer the 3D locations and shapes of internal anatomy. There is a need for a system that can accurately localize internal organs directly from external body surface data to automate table positioning without relying on iterative scout scans.

Methodology
The authors propose a learning-based framework, Pix2Vox, that predicts 3D volumetric organ locations and shapes directly from a single 2D anterior depth image of the body surface.

• Data Generation: The model is trained on a dataset of 10,020 whole-body MRI scans from the German National Cohort (NAKO). Since paired depth images and organ segmentations are not natively available, the authors developed a deterministic pipeline to synthesize training data:
  • Depth Synthesis: MRI volumes are processed to generate orthographic top-down depth projections simulating a sensor mounted above the scanner table.
  • Ground Truth Extraction: The TotalSegmentatorMRI tool is used to generate 3D masks for 56 anatomical structures. These are refined via morphological operations and connected component analysis, resulting in a final set of 41 target structures (including bones and soft tissues).
  • Domain Gap Bridging: To simulate real-world conditions (e.g., hospital gowns, blankets), a stochastic augmentation pipeline adds simulated creases and bulges to the synthetic depth maps.
• Network Architecture: The core of the method is a hybrid 2D-to-3D convolutional neural network.
  • Encoder: A standard 2D U-Net encoder processes the input depth image to extract spatial features.
  • 2D-to-3D Conversion: Custom layers bridge the dimensional gap. At the bottleneck and skip connections, 2D feature maps are unsqueezed along a new feature axis and processed by 3D convolutions to create volumetric representations.
  • Decoder: A 3D U-Net decoder reconstructs the full volumetric segmentation masks for the 41 target organs.
• Training: The model is trained using a combined Dice and binary cross-entropy loss with data augmentations (shifts, scaling, rotations). Post-processing via connected component analysis removes spurious predictions.

Key Contributions

1. Hybrid Architecture: Introduction of a Pix2Vox architecture that performs direct end-to-end inference from a single 2D depth image to 3D volumetric organ segmentations, bypassing intermediate body model fitting or explicit geometric reconstruction.
2. Large-Scale Dataset: Creation and utilization of a dataset comprising 10,020 synthetic depth-anatomy pairs derived from diverse MRI scans, enabling robust modeling of anatomical variability.
3. Generalization: Demonstration that a model trained purely on synthetic data generalizes to real-world depth images acquired from volunteers in an MRI scanner, including scenarios with clothing.

Results
The model was evaluated on a held-out test set of 998 samples and compared against a "Mean Model" (spatial prior) and a "2.5D CNN" (multi-planar projection baseline).

• Localization Accuracy: The Pix2Vox model achieved a mean absolute detection offset (DOE) of 10.99 ± 5.54 mm for organ bounding boxes. It significantly outperformed the mean model in coronal localization (e.g., reducing right boundary error from ~19.86 mm to 7.87 mm). While the 2.5D CNN showed superior lateral precision, Pix2Vox provided more balanced 3D localization, particularly in the anterior-posterior axis where the 2.5D method struggled.
• Segmentation Fidelity: The model achieved a mean Dice Similarity Coefficient (DSC) of 0.44 ± 0.2 (0.48 ± 0.20 in the results section) across all structures. Performance varied by organ size and surface correlation:
  • High performance: Lungs (0.78), Liver (0.71), Femur (0.62).
  • Lower performance: Thyroid gland (0.24), Trachea (0.32).
• Surface Distance: The mean Average Symmetric Surface Distance (ASSD) was 7.69 ± 5.68 mm (8.34 ± 8.11 mm in the results section).
• Real-World Evaluation: Qualitative tests on real Kinect v2 depth images showed the model could reconstruct spatially consistent internal structures. However, the authors noted specific artifacts, such as narrowing in femoral regions and peak-like artifacts in lung segmentations, attributed to surface curvature sensitivity and sensor noise.

Significance and Claims
The paper claims that depth-only organ localization is technically feasible for supporting automated patient positioning in radiology workflows. By achieving a mean absolute detection offset within the clinically relevant threshold of 10–20 mm, the proposed method can potentially:

• Reduce setup time by minimizing manual table adjustments.
• Decrease reliance on scout imaging for geometric planning.
• Minimize inter-operator variability.
• Improve patient comfort and scanner throughput.

The authors emphasize that while the method does not yet resolve fine-grained boundaries of non-surface-correlated tissues perfectly, it provides sufficient spatial information to center target anatomy within the scanner's imaging isocenter. The code and pretrained models are made publicly available to support reproducibility.