Shape Representation using Gaussian Process mixture models

1. Problem Statement

The paper addresses the challenge of representing 3D shapes efficiently and accurately, particularly in scenarios involving sparse data.

• Limitations of Explicit Representations: Traditional methods like point clouds and meshes are storage-intensive, lack continuity, and require complex indexing for surface lookups. They struggle with noise and varying point densities common in photogrammetry.
• Limitations of Neural Implicit Representations: While functional models (e.g., DeepSDF, NeRF) are storage-efficient and continuous, they typically rely on heavy neural architectures. These models are often optimized for generalization across object classes rather than capturing the high-fidelity, idiosyncratic details of a single specific object. They also require massive training datasets.
• The Gap: There is a need for a lightweight, object-specific functional representation that can reconstruct complex geometries from sparsely sampled point clouds without the computational burden of deep learning.

2. Methodology

The authors propose a novel Gaussian Process (GP) Mixture Model that represents surface geometry as a collection of continuous directional distance fields (DDFs).

Core Concept

Instead of a single global model, the object's surface is modeled as a mixture of local GP priors anchored at strategic reference points (typically located inside the object).

Key Steps:

1. Reference Point Extraction:

   • The sparse input point cloud is partitioned into regions.
   • Reference points are computed using structural decomposition methods like skeletonization or distance-based clustering (e.g., k-means).
   • These points serve as the centers for local coordinate systems.

2. Parametrization of Directional Distance Fields (DDFs):

   • For each reference point $C$, the surface is parameterized in spherical coordinates $(\phi, \theta)$.
   • The distance $d$ from $C$ to the surface along a specific direction is modeled as a function $d = f(\phi, \theta)$.
   • This transforms the 3D reconstruction problem into a regression problem on a 2D spherical domain.

3. Gaussian Process Modeling:

   • A local GP is trained for each reference point to predict the distance $d$ given a direction $(\phi, \theta)$.
   • Kernel Selection: The authors utilize the Rational Quadratic (RQ) kernel, which is shown to be superior to Radial Basis Function (RBF) or Matérn kernels for handling multi-scale behaviors in complex geometries.
   • The GP provides a probabilistic distribution over the distance, offering both a prediction and uncertainty estimation.

4. Mixture Model & Surface Coverage:

   • To handle complex topologies (e.g., concavities, holes) where a single reference point cannot see the entire surface, multiple GPs are combined.
   • The likelihood of a point $P$ belonging to the surface is modeled as a weighted sum of the local GP predictions:
     $$p(P|o) = \sum_{k=1}^{K} p(P|C_k; o) \pi(C_k; o)$$
   • Overlap Strategy: To avoid "borderline errors" where points are ambiguously assigned to reference points, the authors introduce inter-cluster overlap during training. This ensures smooth transitions between local patches.

3. Key Contributions

1. Lightweight Functional Representation: The method completely abstains from deep neural networks, relying solely on GPs. This makes it computationally efficient and interpretable.
2. Object-Specific High-Fidelity: Unlike generalizable neural models, this approach is tailored to reconstruct the specific details of a single object from sparse data.
3. Probabilistic Directional Distance Fields: It leverages the ability of GPs to model continuous functions from irregularly sparse data, providing a robust framework for handling occlusions and complex topologies via a mixture of local models.
4. Comprehensive Evaluation: The method is rigorously tested on both synthetic (ShapeNetCore) and real-world industrial (IndustryShapes) datasets.

4. Experimental Results

The authors evaluated their method against state-of-the-art competitors: DeepSDF (Park et al., 2019a) and NKSR (Huang et., 2023).

• Datasets:
  • ShapeNetCore: 120 models each for Planes, Chairs, and Sofas.
  • IndustryShapes: Screwdriver components (industrial assembly context).
• Metrics: Chamfer Distance (dC), Precision, Recall, and F-score.
• Performance:
  • Accuracy: The GP mixture model achieved the lowest Chamfer distance and the highest F-scores across all categories.
    • Example (Planes): F-score of 90.3% (vs. 87.6% for DeepSDF and 79.5% for NKSR).
    • Example (Screwdrivers): F-score of 95.8% (vs. 32.7% for Polynomial and 87.2% for RBF baseline).
  • Detail Preservation: Qualitative results show the method captures fine details (e.g., thin protrusions on planes, holes in screwdrivers) that neural methods tend to over-smooth or miss.
  • Efficiency: Training takes approximately 30 seconds per object on a single RTX 4090 GPU for 10,000 input points.
• Ablation Studies:
  • Kernel Choice: The Rational Quadratic (RQ) kernel consistently outperformed RBF, Matérn, and Linear kernels, demonstrating better adaptability to varying shape complexities.
  • Reference Points: While increasing the number of reference points improves accuracy, the placement is critical. For complex shapes (like chairs), automatic clustering (k-means) was inferior to manual or skeleton-based placement, highlighting the need for intelligent reference point selection.

5. Significance and Conclusion

This work demonstrates that Gaussian Processes are a powerful, underutilized tool for 3D shape representation.

• Advantages: It offers a "sweet spot" between the storage efficiency of implicit models and the detail fidelity of explicit models, without the training overhead of deep learning.
• Applications: The method is particularly suitable for digital twins, heritage documentation, and industrial inspection where high-fidelity reconstruction of specific objects from sparse sensor data is required.
• Future Work: The authors aim to automate the selection of reference points using learned mixture models and provide theoretical guarantees for full surface coverage regardless of shape complexity.

In summary, the paper proposes a GP-mixture-based functional representation that effectively bridges the gap between discrete sparse data and continuous geometry, outperforming current neural implicit methods in accuracy and efficiency for object-specific reconstruction.