PARSE: Part-Aware Relational Spatial Modeling

The paper introduces PARSE, a framework utilizing part-level geometric relations encoded in Part-centric Assembly Graphs to resolve spatial ambiguities, which is validated through the creation of the PARSE-10K dataset and demonstrated to significantly enhance both object layout reasoning in vision-language models and the physical realism of generated 3D scenes.

The Gist

Imagine you are trying to build a complex Lego castle, but instead of just snapping blocks together, you have to figure out exactly which tiny bump on one block fits into which specific hole on another. If you get it wrong, the tower wobbles and falls.

Most current AI models trying to understand 3D spaces are like children who only know the names of the big Lego blocks ("Table," "Chair," "Book"). They know a book goes "on" a table, but they don't know which part of the book (the cover? the spine?) touches which part of the table. This leads to AI hallucinations where books float in mid-air or chairs phase through the floor.

PARSE is a new framework that teaches AI to stop looking at objects as whole blobs and start seeing them as collections of specific parts with specific jobs.

Here is a breakdown of the paper using simple analogies:

1. The Problem: The "Vague Architect"

Imagine an architect who gives you instructions like, "Put the lamp near the sofa."

• The Old Way: The AI hears this and might put the lamp inside the sofa, or floating three feet above it, or leaning against the wall on the other side of the room. It's technically "near," but physically impossible or weird.
• The Issue: Words like "on," "against," or "near" are too fuzzy for a computer to build a stable 3D world.

2. The Solution: The "Part-Centric Assembly Graph" (PAG)

The authors created a new blueprint called a PAG. Think of this not as a drawing of a room, but as a recipe for assembly.

Instead of saying "Lamp on Table," the PAG says:

"The flat bottom surface of the Lamp's base must touch the top surface of the Table's center."

• The Analogy: It's like a master carpenter's instruction manual. It doesn't just say "attach the leg to the table"; it specifies, "align the dovetail joint of the leg with the mortise of the table."
• The Result: By breaking things down into parts (legs, cushions, shelves, covers), the AI can calculate exactly how to place them so they don't crash into each other and actually stay put.

3. The Engine: The "Spatial Configuration Solver"

Once the AI has this detailed recipe (the PAG), it needs a tool to build the scene. This is the Solver.

• How it works: Imagine a puzzle solver that works from the bottom up.
  1. It starts with the floor.
  2. It places the table legs, locking them to the floor.
  3. It places the table top, locking it to the legs.
  4. It places the book, locking its bottom cover to the table top.
• The Magic: At every step, the solver checks the physics. If a chair is too heavy for a wobbly table, the solver knows to move it. It creates a "collision-free" zone, ensuring every object has a solid place to stand before adding the next one.

4. The Dataset: "PARSE-10K"

To teach an AI this skill, you need millions of examples. The authors built PARSE-10K, a massive library of 10,000 3D indoor scenes.

• The Difference: Previous datasets were like photos of messy rooms where the computer couldn't tell where one object ended and another began. PARSE-10K is like a perfectly organized museum where every single object is tagged with its parts.
• Why it matters: It's the difference between showing a student a blurry photo of a car and showing them a 3D model where you can pop off the tires, the hood, and the engine to see how they connect.

5. The Results: Smarter AI

The team tested this by teaching a smart AI (Qwen3-VL) using PARSE-10K.

• Before Training: The AI was good at guessing, but often wrong about how things touched. It might say a cup is "on" a table when it's actually "under" it.
• After Training: The AI became a spatial genius.
  • Visual Reasoning: It could look at a picture and correctly identify that a "pen is leaning against the side of the cup," not just "near" it.
  • Scene Generation: When asked to generate a 3D room, the AI didn't just scatter objects randomly. It built rooms where chairs had four legs touching the floor, books were stacked neatly, and lamps were plugged in (metaphorically). The scenes looked physically real and stable.

Summary

Think of PARSE as teaching an AI the difference between "The book is on the table" (a vague idea) and "The book's bottom cover is resting on the table's top surface" (a physical reality).

By forcing the AI to think in terms of parts and contacts, they created a system that can understand, reason about, and generate 3D worlds that actually make sense to our human eyes and physics. It's the difference between a child stacking blocks randomly and an engineer building a skyscraper.

Technical Summary

Here is a detailed technical summary of the paper "PARSE: Part-Aware Relational Spatial Modeling".

1. Problem Statement

Current spatial intelligence systems struggle with inter-object relations because existing representations are too coarse.

• Limitations of Current Methods:
  • Linguistic Prepositions: Terms like "on," "in," or "against" are ambiguous. For example, "a book on a table" does not specify if the spine or cover is contacting the surface.
  • Object-Level Scene Graphs: Traditional scene graphs treat objects as indivisible units. They lack the granularity to specify which regions of objects support, contain, or contact one another.
• Consequences: This ambiguity leads to physically inconsistent layouts, unstable 3D scene generations, and poor performance in tasks requiring fine-grained spatial reasoning (e.g., packing, stacking, embodied manipulation).
• Goal: To develop a representation that bridges high-level language descriptions and low-level spatial configurations by modeling interactions at the part level.

2. Methodology

The authors propose PARSE (PArt-aware Relational Spatial modEling), a framework centered on two core components:

A. Part-centric Assembly Graph (PAG)

A hierarchical, directed acyclic graph (DAG) that explicitly models geometric relations between specific object parts.

• Nodes:
  • Object Nodes ($V_O$): Represent semantic queries (e.g., "chair") rather than specific 3D models, allowing for compositional diversity.
  • Part Nodes ($V_P$): Represent geometric components (e.g., "legs," "seat") with labeled surfaces (e.g., "top," "bottom," "front"). These serve as the interfaces for contact.
• Edges:
  • Object-Level Edges ($E_{obj}$): Encode coarse relations (e.g., "left of," "near") to guide macroscopic layout.
  • Part-Level Geometric Edges ($E_{part}$): Encode fine-grained constraints (e.g., "book cover bottom" on "table surface top"). These define precise physical interactions.
• Structure: The graph is a DAG, ensuring a valid sequential assembly order (no circular dependencies), which makes the constraint satisfaction problem computationally tractable.

B. Part-Aware Spatial Configuration Solver

A procedural engine that instantiates PAGs into physically valid 3D scenes through a coarse-to-fine process:

1. Coarse Localization: Uses object-level edges to prune the 2D support surface, excluding occupied regions.
2. Part-Level Alignment: Instantiates specific 3D assets and enforces geometric constraints (e.g., coplanarity, contact) between specific part surfaces. If parts are not explicitly named, the solver infers them (e.g., finding the lowest part of an object for an "on" relation).
3. Pose Sampling & Validation: Samples a final pose from the constrained subspace and validates it for collisions.
4. Physics Refinement: Runs a brief dynamic simulation (using Sapien) to ensure stability and realism.

3. Key Contributions

1. The PARSE Framework

A novel representation and solver that moves beyond object-level abstraction to part-level geometric constraints, enabling the generation of collision-free, physically plausible scenes with complex contact structures.

2. PARSE-10K Dataset

A large-scale dataset of 10,000 high-quality 3D indoor scenes constructed using the PARSE framework.

• Scale & Diversity: Contains ~17,372 part-segmented assets across 132 categories, averaging 49.9 objects per scene.
• Annotations: Features dense part-level contact graphs and fully part-segmented object instances, a feature missing in existing datasets (e.g., 3D-FRONT, HSSD-200).
• Generation: Scenes are derived from real-image layout priors, ensuring semantic plausibility.

3. Benchmarking and Applications

The paper demonstrates the utility of PARSE-10K in two distinct domains:

• Spatial Reasoning (VLMs): Fine-tuning Vision-Language Models (VLMs) on PARSE-10K significantly improves their ability to understand part-level contacts and generate accurate scene graphs.
• 3D Scene Generation: Using PAGs as structural priors in diffusion models leads to scenes with higher object counts, richer contacts, and greater physical realism compared to baselines trained on standard datasets.

4. Results

A. Spatial Reasoning (VLM Evaluation)

The authors fine-tuned Qwen3-VL on PARSE-10K and evaluated it against state-of-the-art models (GPT-5, Gemini-2.5-Pro, Claude-Opus-4) on three tasks:

• Visual Relation MCQ: The fine-tuned model achieved 97.4% accuracy (vs. 85.0% for Gemini-2.5-Pro).
• Part-level Contact MCQ: Achieved 86.2% accuracy (vs. 75.6% for Gemini-2.5-Pro).
• Scene Graph Generation (SGG): The model showed substantial gains in Recall, Precision, and F1 Score, particularly in grounding relations to specific object parts.
• Key Insight: Generalist VLMs struggle with visual grounding and part-level specificity; PARSE-10K supervision directly addresses these gaps.

B. 3D Scene Generation

• Qualitative: Scenes generated using PARSE-10K and PAG control exhibited more complex arrangements (stacking, leaning) and better physical stability than those generated by InstructScene trained on 3D-FRONT.
• Quantitative (User Study): In a study with 20 participants:
  • Complexity: 47.5% preference for PAG-conditioned PARSE-10K scenes vs. 7.5% for the baseline.
  • Realism: 38.8% preference vs. 33.8% for the baseline.
  • Contact Fidelity: 45.0% preference vs. 28.8% for the baseline.
• Finding: Conditioning on PAGs is crucial; without it, models trained on PARSE-10K still struggled with physical plausibility due to the dataset's high complexity.

5. Significance

• Paradigm Shift: Moves the field from object-centric to part-centric spatial modeling, resolving the ambiguity inherent in natural language spatial descriptions.
• Data Gap Filling: Provides the first large-scale dataset with explicit, dense part-level contact annotations, which is essential for training models in robotics, embodied AI, and physical simulation.
• Physical Consistency: Demonstrates that explicit geometric constraints (PAGs) are superior to purely learned priors for generating physically stable and structurally complex 3D environments.
• Future Impact: Sets a foundation for more advanced embodied agents that can reason about how objects interact (e.g., "the cup rests on the coaster, not the table") rather than just where they are.

Limitations & Future Work

• Manual Construction: PAG construction currently requires manual effort and sensitivity to canonical poses.
• Geometric Complexity: The current directional-face model handles parallel surfaces well but may struggle with highly oblique contacts.
• Future Directions: Learning part-part relations directly from geometry, developing flexible contact representations, and integrating PARSE into embodied manipulation tasks.