Technical Summary: Artic-O: End-to-End Articulated Object Reconstruction via Latent Geometry Learning

1. Problem Statement

Reconstructing articulated objects (e.g., cabinets, drawers) from sparse multi-state images is a challenging task that requires recovering three distinct but interdependent components: complete 3D geometry, the segmentation of movable (active) parts, and the parameters defining the motion mechanism (joints).

Existing approaches face significant limitations:

• Geometry-based methods require clean, complete 3D inputs (point clouds or meshes), which are often unavailable in real-world scenarios.
• Image-based methods often rely on per-instance optimization (slow, time-consuming) or retrieval from geometry libraries (limited fidelity).
• State-of-the-art (SoTA) limitations: Recent methods like LARM [Yuan et al. 2025] improve efficiency by using feed-forward novel-view synthesis but separate geometry reconstruction from articulation estimation. This decoupling can propagate reconstruction errors into motion estimation and fails to enforce consistency between shape, active parts, and motion parameters. Furthermore, LARM requires a separate post-processing stage to estimate articulation, lacking an end-to-end pipeline.

2. Methodology: Artic-O

Artic-O is an end-to-end, feed-forward framework that simultaneously predicts complete object geometry, active-part segmentation, and articulation parameters from sparse multi-state images (typically two endpoint states with multiple views). The core innovation is extending latent geometry learning from static object reconstruction to articulated object reconstruction.

2.1 Architecture Overview

The framework consists of three primary components:

1. State-Aware Image Encoder:

   • Based on the NOVA3R/VGGT architecture, this encoder takes sparse images from two articulation states ($I_0, I_1$) as input.
   • It employs a state-aware token embedding mechanism, adding learnable embeddings to image patch tokens to distinguish between the two observed states.
   • It outputs geometry latents ($z_{geo}$) and image tokens ($f_{img}$). The geometry latents are conditioned on the image evidence but reside in a pretrained latent space.

2. Frozen Flow-Matching Shape Decoder:

   • Artic-O utilizes a pretrained, frozen flow-matching (FM) decoder trained on complete object point clouds.
   • Instead of predicting geometry directly in image space, the model maps sparse observations into the latent space of this decoder.
   • The decoder reconstructs the complete point cloud ($\hat{P}$) by integrating a predicted velocity field from sampled noise. Crucially, this decoder is kept frozen during training, serving as a strong prior for recovering occluded and interior geometry.

3. Image-Grounded Part Reasoning & Articulation Heads:

   • Part Memory: To efficiently connect visual evidence with geometry, the model distills image tokens and geometry latents into a compact set of "part memory" tokens via cross-attention.
   • Segmentation Head: Using the part memory, point-wise geometry features ($f_{pcd}$), and image tokens, the model predicts a binary mask ($\hat{S}$) distinguishing the active moving part from the static context.
   • Articulation Head: A slot-driven mechanism uses the representation of the active slot and point-wise features to predict joint parameters. It supports both prismatic (linear translation) and revolute (rotation) motions, predicting axes, origins, and motion ranges. For revolute joints, it employs a dense voting mechanism to locate the axis.

2.2 Training Strategy

To balance the conflicting objectives of generative geometry reconstruction and discriminative part-level reasoning, Artic-O employs two specific training strategies:

• Geometry-to-Articulation Curriculum:

  • Stage 1: The model is trained solely on the flow-matching loss ($L_{fm}$) to adapt the image encoder to the two-state articulated setting, ensuring the latent space captures valid geometry before introducing part-level supervision.
  • Stage 2: Segmentation and articulation heads are introduced. The image encoder is updated with a lower learning rate, while the heads are trained with a higher rate to learn quickly without destabilizing the geometry prior.

• Decoupled Two-Pass Training:

  • Within a single training step, the encoder runs once, but the frozen decoder runs twice with different noise schedules.
  • Pass 1 (FM Pass): Uses standard noisy trajectory samples to optimize geometry reconstruction ($L_{fm}$).
  • Pass 2 (Seg/Art Pass): Uses a mixture of noisy samples and "clean" samples (where $t=1$, i.e., ground truth geometry) to train segmentation and articulation. This ensures the heads learn from unambiguous part identities (clean) while remaining robust to imperfect reconstructions (noisy) encountered at inference.

3. Key Contributions

• End-to-End Feed-Forward Pipeline: Artic-O is the first method to unify complete geometry reconstruction, active-part segmentation, and articulation parameter prediction in a single feed-forward pass, eliminating the need for per-instance optimization or separate post-processing stages.
• Latent Geometry Prior for Articulation: By leveraging a frozen flow-matching decoder, the method recovers complete interior geometry and occluded structures that view-conditioned methods often miss, providing a stable structural context for articulation reasoning.
• Image-Grounded Part Memory: A novel module that distills multi-view, multi-state visual evidence into a compact representation, enabling effective separation of active parts from static contexts.
• Curriculum and Two-Pass Training: A tailored training regimen that balances the generative nature of geometry reconstruction with the discriminative requirements of part segmentation and motion estimation.

4. Experimental Results

Experiments were conducted on the PartNet-Mobility dataset, following the evaluation protocol of LARM.

• Geometry Reconstruction: Artic-O outperforms LARM, reducing Chamfer Distance (CD) from 0.019 to 0.017 and improving the F-score from 93.8% to 95.4%. Qualitative results show more complete interior volumes and better handling of occluded regions.
• Articulation Accuracy: The method achieves comparable or superior accuracy across joint metrics. Notably, motion-range accuracy ($M_r$) improved from 87.0% to 93.5%, and axis-angle success improved from 96.7% to 97.1%.
• Efficiency: The most significant improvement is in inference speed. Artic-O processes an object in 0.32 seconds, compared to approximately 9 minutes for LARM (a ~1680x speedup), as it avoids iterative optimization and novel-view synthesis loops.
• Ablation Studies:
  • Removing the frozen geometry prior degrades both geometry and articulation.
  • Removing the image-grounded part memory significantly reduces axis-angle success (by ~19.8 points).
  • Removing the curriculum or two-pass training leads to substantial drops in both geometry quality and articulation accuracy.
• Real-World & Multi-Part: The model generalizes to real-world iPhone captures without test-time optimization and can be extended to multi-part objects via independent part-wise inference, outperforming LARM in these scenarios as well.

5. Significance and Claims

The paper claims that Artic-O represents a significant step toward scalable and efficient reconstruction of interactive 3D assets. By moving away from decomposed pipelines and optimization-based approaches, Artic-O demonstrates that:

1. Consistency: Jointly learning geometry and articulation in an end-to-end manner yields better consistency between shape and motion than separated stages.
2. Feasibility: High-quality articulated reconstruction is feasible in real-time (sub-second) without requiring clean 3D inputs or expensive per-instance tuning.
3. Generalization: The latent geometry approach allows the model to infer plausible complete shapes and motion mechanisms from sparse, incomplete visual evidence, making it suitable for downstream applications like digital twins, AR/VR, and embodied intelligence where rapid asset generation is critical.

The authors maintain a modest stance, noting that while the method handles independently moving parts effectively, it does not currently address complex kinematic chains, though it covers many everyday articulated objects.