CAOA -- Completion-Assisted Object-CAD Alignment

Technical Summary: Completion-Assisted Object-CAD Alignment (CAOA)

Problem Statement
Accurately aligning CAD models to their corresponding objects within indoor RGB-D scans is a fundamental challenge in 3D semantic reconstruction. This task requires estimating a 9-Degree-of-Freedom (DoF) pose (position, rotation, and scale). However, performance is frequently hindered by the inherent limitations of indoor scans, including noise, incompleteness, and segmentation errors that cause geometric distortions. Existing methods often struggle because point cloud completion algorithms are typically trained on synthetic datasets that fail to generalize to real-world scans due to the synthetic-to-real domain gap. Furthermore, standard alignment approaches often lack mechanisms to handle object symmetry, leading to ambiguities in pose estimation.

Methodology: CAOA
The authors propose Completion-Assisted Object-CAD Alignment (CAOA), a modular framework designed to enhance alignment accuracy by integrating point cloud completion with symmetry-aware pose estimation. The pipeline consists of three primary modules:

1. Context-Aware Point Cloud Completion Module (CAPCM):
   To address noise and missing geometry, CAOA employs a completion module that processes incomplete object point clouds. Unlike standard completion methods that rely solely on the object itself, CAPCM is "context-aware." It incorporates point clouds from the surrounding environment (within a defined context radius) alongside the target object. This additional contextual information helps the algorithm generate completions that are geometrically consistent with the scene, preventing over- or under-generation of points. The module is trained using a novel real-world dataset (S2C-Completion) and a specialized synthetic dataset (SN-Indoor) designed to mimic indoor occlusion and sensor noise.

2. Symmetry Encoder Module (SEM):
   To mitigate ambiguities arising from symmetric objects, CAOA introduces a Symmetry Encoder. This module utilizes a 3D Transformer-based feature encoder to analyze the CAD model (in its canonical pose) and generate a symmetry vector. This vector indicates whether an object possesses symmetry and the nature of that symmetry. By explicitly encoding this information, the system improves robustness during the alignment phase, particularly for rotation and scale estimation.

3. Object-CAD Alignment Module (OCAM):
   The final module estimates the 9-DoF pose. It takes the completed object point cloud from CAPCM, the symmetry vector from SEM, and the corresponding CAD model as inputs. Using a Siamese-style network with a 3D CNN backbone, OCAM extracts features and regresses translation, rotation, and scale parameters. The training process utilizes a composite loss function combining Chamfer Loss (modified to be symmetry-aware) and a weighted L1 loss for pose parameters.

Key Contributions
The paper introduces several specific contributions to bridge the gap between synthetic training and real-world application:

• S2C-Completion Dataset: An expert-annotated dataset containing over 8,500 object-CAD pairs derived from ScanNet and Scan2CAD. This dataset provides precise ground truth alignments for real-world indoor single-object completion, serving as a new benchmark for the task.
• SN-Indoor Dataset: A synthetic dataset generated from ShapeNet using techniques tailored to indoor environments, including multi-view ray-casting, non-linear occlusion planes, and Gaussian noise injection. This approach aims to reduce the domain gap between synthetic training data and real-world scans.
• Context-Aware Completion Strategy: A method for point cloud completion that integrates surrounding scene context to improve the geometric consistency of the generated point clouds.
• Symmetry-Aware Alignment: A training methodology that leverages a symmetry encoder and a symmetry-aware loss formulation to learn robust pose features, addressing the specific challenges of symmetric objects.

Experimental Results
The authors evaluated CAOA on the Scan2CAD benchmark using the ScanNet validation set.

• Alignment Accuracy: CAOA achieved a 17% improvement in class-average accuracy and a 16% improvement in overall accuracy compared to state-of-the-art methods (including SceneCAD, End-to-End, and CIS2VR).
• Ablation Studies:
  • Completion: The inclusion of the CAPCM module resulted in significant performance gains, improving overall accuracy by approximately 15% and weighted average accuracy by 19%.
  • Dataset Generalization: Training completion models on the proposed SN-Indoor dataset significantly reduced the Chamfer Distance error on real-world data compared to training on standard synthetic datasets (PCN, ShapeNet-55/34).
  • Symmetry: Incorporating the Symmetry Encoder (SEM) provided an approximate 2% improvement in alignment accuracy.
• Runtime: The method demonstrates high efficiency, averaging approximately 0.58 seconds per scene on the ScanNet validation set, making it suitable for dynamic applications.

Significance and Claims
The paper claims that CAOA significantly enhances the robustness and accuracy of object-CAD alignment by directly addressing the limitations of noisy and incomplete point clouds through context-aware completion and symmetry encoding. The authors emphasize that the introduction of S2C-Completion and SN-Indoor is critical for bridging the synthetic-to-real domain gap, enabling completion algorithms to generalize effectively to real-world indoor scenes. By releasing the code, datasets, and annotation tools, the work aims to establish a new standard for benchmarking and advancing 3D semantic reconstruction tasks. The authors note that while the current approach separates completion and pose estimation training, future work could explore unified strategies to further guide completion objectives with pose estimation goals.