PartNeXt: A Next-Generation Dataset for Fine-Grained and Hierarchical 3D Part Understanding

1. Problem Statement

While 3D part understanding is critical for computer vision, graphics, and robotics, existing datasets suffer from significant limitations that hinder scalability and model performance:

• Lack of Texture and Geometry Fidelity: The dominant dataset, PartNet, relies on untextured geometries. Its annotation process often requires remeshing, which deforms the original geometry and strips away texture information (color, material). This limits the ability of models to learn from visual cues essential for human-like perception.
• Annotation Scalability and Usability: PartNet's annotation interface requires expert-level 3D modeling skills (e.g., manually drawing cutting curves on meshes). This creates a high barrier to entry for crowdsourcing, limiting dataset expansion and increasing annotation costs.
• Limited Granularity and Hierarchy: Existing datasets often lack fine-grained, hierarchical labels or are restricted to a small number of object categories, failing to support open-vocabulary or complex structural reasoning.

2. Methodology

The authors introduce PartNeXt, a dataset designed to overcome these hurdles through a novel data pipeline and annotation system.

A. Data Collection and Preprocessing

• Sources: Models are curated from Objaverse, ABO (Amazon-Berkeley Objects), and 3D-FUTURE, ensuring a wide diversity of appearances and geometries.
• Filtering: High-quality textured meshes are selected. Models with animations, excessive face counts (>130k), or architectural/scanned nature are removed.
• Category Classification: Since Objaverse lacks explicit categories, the authors use CLIP text encoders to match object captions (from Cap3D) against ~100 predefined categories, filtering for high similarity scores (>0.75) and selecting the top 50 categories.

B. Hierarchy Definition

• AI-Assisted Design: To define consistent, functionality-aware part hierarchies across 50 categories, the authors leverage GPT-4o. The AI is guided by principles of functionality, hierarchy, exhaustiveness, atomicity, and consistency.
• Human-in-the-Loop: AI-generated hierarchies are reviewed and refined by human experts. Visual reference images for each part node are also generated by GPT-4o to reduce ambiguity for annotators.

C. Scalable Annotation System

The core innovation is a fully web-based, dual-panel annotation interface designed for non-expert crowdsourcing:

• Dual-Panel Layout:
  • Left Panel: Displays the original, unannotated textured mesh.
  • Right Panel: Shows the currently segmented result.
  • Workflow: Annotators select faces from the left panel and move them to the right, effectively visualizing interior parts that might be occluded in a single view.
• Selection Tools:
  • Connected Component Selection: Click-to-select entire connected mesh regions (handling vertex duplication).
  • Bounding Box Selection: Selects faces within a 2D projected box for quick region selection.
  • Per-Face Selection: Manual face-by-face control for precision.
• Key Advantage: Annotations are performed directly on the original textured mesh, eliminating the need for remeshing and preserving native texture and geometry.

3. Key Contributions

1. PartNeXt Dataset: A large-scale dataset containing 23,519 high-quality, textured 3D models across 50 object categories, with a total of 350,187 annotated part instances.
2. Hierarchical & Fine-Grained Labels: Unlike previous datasets, PartNeXt provides deep hierarchical structures (depth 4–10) covering both external and internal components, with leaf nodes representing atomic parts.
3. Texture-Aware Annotations: The dataset preserves original textures and geometry, enabling research into texture-based part segmentation and affordance analysis.
4. Novel Benchmarks:
   • Class-Agnostic Part Segmentation: Evaluates models on identifying parts without category priors.
   • 3D Part-Centric Question Answering (QA): A new benchmark for 3D Large Language Models (3D-LLMs) testing part counting, classification, and grounding (bounding box prediction).
5. Annotation Interface: A scalable, web-based tool that democratizes 3D part annotation, significantly reducing the expertise barrier.

4. Results and Experiments

The authors benchmarked state-of-the-art (SOTA) methods on PartNeXt to demonstrate the dataset's difficulty and utility.

• Part Segmentation:
  • SOTA methods (PartField, SAMPart3D, SAMesh) showed significant performance drops compared to their performance on PartNet.
  • SAMesh tended to over-segment; PartField struggled with connected regions; SAMPart3D failed in weak-texture areas.
  • Conclusion: Current methods struggle with fine-grained, leaf-level parts and maintaining semantic consistency.
• 3D Part-Centric QA:
  • Leading 3D-LLMs (PointLLM, ShapeLLM, 3D-LLM) performed poorly on part counting and grounding tasks, especially in category-agnostic settings.
  • This highlights a major gap in current models' ability to perform open-vocabulary part reasoning.
• Interactive Segmentation (Point-SAM):
  • Training Point-SAM on PartNeXt yielded substantial gains over training on PartNet (e.g., IoU@10 improved from ~60.3% to 65.9%).
  • This confirms that the higher quality, diversity, and texture fidelity of PartNeXt directly translate to better model generalization.

5. Significance

• Foundation for Next-Gen Models: PartNeXt addresses the critical lack of high-fidelity, textured, and hierarchically structured data, serving as a foundational resource for training the next generation of 3D foundation models.
• Bridging the Gap: By preserving textures and original geometry, it enables research into how visual cues (color, material) influence part understanding, a dimension largely ignored by previous datasets.
• Scalability: The proposed annotation workflow proves that high-quality 3D part annotation can be scaled via crowdsourcing without requiring expert 3D modeling skills, setting a new standard for future dataset creation.
• Research Direction: The benchmarks reveal that current 3D-LLMs and segmentation models are far from human-level understanding of object structure, providing a clear roadmap for future research in 3D reasoning and grounding.