💻 computer science

Artiverse: A Diverse and Physically Grounded Dataset for Articulated Objects

The paper introduces Artiverse, a diverse and physically grounded dataset comprising 5,400 high-quality articulated 3D objects with detailed functional and physical annotations, developed via an efficient semi-automated pipeline to advance research in part mobility analysis, object generation, and physics-based interaction.

Original authors: Denys Iliash, Jiayi Liu, Egor Fokin, Qirui Wu, Ali Mahdavi-Amiri, Manolis Savva, Angel X. Chang

Published 2026-05-26

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Denys Iliash, Jiayi Liu, Egor Fokin, Qirui Wu, Ali Mahdavi-Amiri, Manolis Savva, Angel X. Chang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a giant digital toy box. For years, the toys inside were mostly static statues—beautiful to look at, but you couldn't actually play with them. If you tried to pull a drawer on a digital dresser, it would just clip right through the wood like a ghost. If you tried to spin a chair, it would stay frozen in place.

The paper introduces Artiverse, a massive new collection of 3D objects that are not just statues, but fully functional, "living" digital toys. Think of it as upgrading your toy box from a collection of plastic models to a set of high-tech, physics-ready action figures that actually work the way real objects do.

Here is the breakdown of what they built and how they did it, using simple analogies:

1. The "Living" Toy Box (The Dataset)

Artiverse contains over 5,400 human-made objects (like chairs, toasters, and filing cabinets) across 88 different categories.

The Problem with Old Datasets: Previous collections were like "paper dolls." They had the right shape, but if you tried to move a part, it didn't know how to connect. They also lacked "inside" details (like the empty space inside a microwave) and didn't know how heavy things were.
The Artiverse Solution: Every object in this new box is physically grounded.
- It has weight: The computer knows a plastic handle weighs less than a metal one.
- It has size: A chair is the size of a real chair, not a giant or a miniature.
- It has "bones" and "joints": Just like a human has knees and elbows, these objects have hinges, sliders, and swivels. The computer knows exactly how a door swings or how a drawer slides.
- It has an "inside": If you open a microwave, you can see the turntable inside. If you open a fridge, you see the shelves.

2. The "Robot Chef" Assembly Line (The Annotation Process)

Building 5,400 complex, moving toys by hand would take a human team years. To speed this up, the researchers built a semi-automated assembly line.

Think of it like a robot chef trying to build a sandwich:

The Robot Guesses: The AI looks at a 3D model and guesses, "This part is the handle, and this part is the door. I think the door swings on a hinge."
The Human Taste-Tester: A human expert looks at the robot's guess. "Good job on the handle, but the door is attached to the wrong side. Fix that."
The Result: This teamwork allowed them to cut the manual work time by about 30%. The robot does the heavy lifting, and the humans just fix the mistakes, ensuring everything is perfect.

3. Why This Matters (The Experiments)

The paper tests if this new "toy box" is actually useful for training computer programs (robots and AI).

The "How Does It Move?" Test: They taught a computer to look at a static picture of a chair and guess how the parts move. When they trained the computer using Artiverse, it got much better at guessing than when they trained it on the old, "ghost-like" datasets. It learned that chairs have swivel bases and that drawers slide out.
The "Build It From Scratch" Test: They showed a computer a picture of a toaster and asked it to build a 3D version that actually works. The computer trained on Artiverse built a toaster with a working lever and realistic internal parts, whereas computers trained on older data struggled to make things that actually moved correctly.
The "Robot Simulator" Test: They put these objects into a physics simulator (a video game engine for robots). They successfully programmed a virtual robot arm to open a cabinet drawer. Because the drawer had real weight, real joints, and real friction, the robot didn't just "glitch" through it; it actually pulled it open.

Summary

Artiverse is a massive library of 3D objects that are ready for the real world. It's not just a picture of a chair; it's a digital chair you can sit on, spin, and lift, with all the correct physics, materials, and moving parts. By combining AI guesses with human experts, the researchers created a dataset that helps teach robots and AI how to understand, interact with, and manipulate the physical world around them.

Technical Summary: Artiverse

Problem Statement

Existing 3D datasets for articulated objects often lack the geometric fidelity, functional realism, and physical grounding required for realistic simulation and embodied AI. While large-scale repositories (e.g., ShapeNet, Objaverse) offer diverse geometry, they are predominantly static. Specialized articulated datasets (e.g., PartNet-Mobility, AKB-48) provide motion annotations but are limited in joint complexity (mostly single-DoF), visual realism (simplified textures, missing interiors), and physical attributes (mass, material, metric scale). Recent efforts like ArtVIP and PhysX-3D have begun addressing physical properties but remain limited in scale and category diversity. Consequently, there is a lack of high-quality, physically grounded assets that capture the full spectrum of functional interactivity—structural decomposition, kinematic dependencies, and physical plausibility—necessary for advancing functional modeling and simulation.

Methodology

The authors introduce Artiverse, a dataset of 5,402 human-authored articulated objects across 88 categories, aggregated from 10 diverse 3D repositories (including ShapeNet, ABO, 3D-Future, HSSD, Objaverse, and others). The construction of Artiverse relies on a semi-automated annotation pipeline designed to balance scalability with high-quality human verification.

1. Data Selection and Preprocessing

Models are filtered from multiple repositories using a hybrid retrieval workflow combining automated feature analysis (DuoDuoCLIP embeddings) and human oversight. Objects are standardized to a consistent $y$ -up coordinate frame, zero-centered, and scaled to metric units. Metric scales are inferred using LLMs (GPT-4o) to estimate physical dimensions for sub-categories where original data is missing.

2. Semi-Automated Annotation Pipeline

The pipeline consists of four sequential stages, with human verification integrated at critical points:

Functional Part Segmentation:
- Template-Driven: Category-specific templates define functional part labels (e.g., handles, lids, shelves) and motion types.
- Few-Shot Segmentation: The ASIA model (Adaptive 3D Segmentation using few image annotations) is trained on a small set of manually annotated exemplars per category to generate initial 2D part masks on multi-view renderings.
- 3D Aggregation: 2D masks are projected onto 3D mesh over-segmentations. Unlabeled internal surfaces are assigned labels via distance-based propagation. A union-find algorithm clusters connected segments into disjoint part instances.
- Verification: Human annotators correct segmentation errors via a 3D UI, followed by expert verification.
Interior Completion:
- To ensure physical realism, the pipeline addresses missing internal geometries in three scenarios:
  1. Partial Articulated Parts: Extends geometry for parts like drawers where the interior box is missing, ensuring plausibility across the full motion range.
  2. Missing Articulated Parts: Procedurally inserts missing internal components (e.g., dishwasher baskets, microwave turntables) by referencing templates from similar objects.
  3. Missing Interaction Affordance: Adds structural elements like shelves and racks to hollow furniture to support realistic interaction.
Motion Annotation:
- Automated Proposal: Geometric heuristics infer joint parameters (type, axis, origin, limits) using PCA-aligned bounding boxes and contact region analysis between kinematically dependent parts. Kinematic graphs are constructed based on spatial connectivity and template constraints.
- Verification: Human experts validate motion plausibility and adjust joint parameters using an interactive web interface that supports motion copying and instant previews.
Physical Properties:
- Mass and Material: Per-part mass is estimated by computing volume (via tetrahedralization for solid parts or shell offsetting for hollow parts) and multiplying by material density. Material labels and density ranges are assigned based on semantic labels and LLM-inferred ranges.

3. Efficiency

The semi-automated approach reduces manual annotation time by 32.0% for segmentation and 33.5% for motion annotation compared to fully manual processes. Approximately 50.12% of automatic proposals require no human correction.

Key Contributions

Artiverse Dataset: A diverse collection of 5,402 articulated objects with rich annotations including functional parts, kinematic dependencies, multi-DoF joints (e.g., universal, cylindrical), and physical attributes (metric scale, material, mass). It covers 88 sub-categories, significantly expanding the diversity and complexity of existing datasets.
Semi-Automated Pipeline: A scalable workflow integrating few-shot segmentation, geometric reasoning, and multi-stage human verification to produce high-quality annotations efficiently.
Benchmarking and Validation: The paper demonstrates the dataset's utility through:
- Part Mobility Analysis: Training the FPNGroupMot model on Artiverse yields superior generalization compared to training on PartNet-Mobility, though the dataset's complexity presents new challenges for existing models.
- Articulated Object Generation: Evaluating image-conditioned generation (SINGAPO, Articulate-Anything) shows that models trained on Artiverse produce more structurally and geometrically consistent results than those relying solely on VLM priors.
- Simulation Readiness: Assets are released in GLB, USD, and URDF formats, enabling seamless integration into physics engines (e.g., Genesis) for embodied AI tasks.

Results

Statistics: Artiverse contains 24,607 parts and 23,640 joints, with a significant portion being complex multi-DoF joints (480 2-DoF joints), compared to the single-DoF focus of PartNet-Mobility.
Cross-Dataset Evaluation: Models trained on Artiverse outperform those trained on PartNet-Mobility in part segmentation and motion prediction metrics (e.g., F1 scores), indicating that the dataset's diversity and quality enhance model generalization.
Generation Performance: In image-to-articulated-object generation, SINGAPO trained on Artiverse achieves higher gIoU and lower collision rates (AOR) than models relying on VLM priors, highlighting the importance of structured data priors for detailed articulation synthesis.

Significance

The paper claims that Artiverse serves as a critical resource for advancing functional understanding of articulated objects. By providing physically grounded, diverse, and kinematically complex assets, it enables:

The development of more robust learning-based methods for part mobility analysis and articulated object generation.
The creation of realistic interactive scenes and immersive virtual environments.
The training and evaluation of embodied agents in physics-based simulators for interactive tasks like object manipulation.

The authors position Artiverse not just as a data collection, but as a benchmark that exposes the limitations of current methods in reasoning over complex, multi-DoF articulations, thereby motivating future research in unified reasoning for articulated behaviors. The dataset is released with standardized formats to facilitate immediate adoption in simulation and generative modeling tasks.