Technical Summary: PolyLayout

Problem Statement

Generating physically plausible 3D room layouts is critical for home furnishing retail, enabling customers to visualize products in their own spaces. However, a significant gap exists between academic research and real-world application. Existing solutions primarily focus on algorithmic strategies for furniture placement within idealized, rectangular rooms. They largely neglect the complex, non-rectangular geometries (e.g., L-shaped, T-shaped, or cut walls) and strict architectural constraints (doors, windows, alcoves) prevalent in real homes. Furthermore, purely data-driven approaches often treat layout generation as a black-box mapping, failing to enforce hard spatial constraints. This leads to layouts where objects collide, violate ergonomic relationships, or protrude beyond boundaries. Conversely, semantic models like Large Language Models (LLMs) and Vision-Language Models (VLMs) excel at functional reasoning but lack the geometric precision required for exact coordinate placement and cannot natively handle irregular polygonal boundaries as hard constraints.

Methodology

PolyLayout introduces a hybrid, hierarchical framework designed to bridge the gap between semantic reasoning and geometric feasibility. The system decouples layout generation into three distinct stages, processing a room topology (in glTF format) and a predefined furniture inventory.

1. Input Processing and Functional Clustering (Stage 1)

The system first reconstructs the true floor polygon by traversing wall endpoints, preserving non-rectangular plans exactly. It extracts structural openings (doors/windows) and assigns inward-facing normals to walls.

• Clustering: The furniture inventory is partitioned into "functional bundles" (coherent groups like a bed with nightstands) and standalone items using a rule-based taxonomy.
• Internal Arrangement: Within each bundle, items are arranged relative to a mandatory anchor (e.g., a desk for a work area) using deterministic, rule-based templates. This resolves fine-grained geometry (clearances, surface placement) before global placement occurs.

2. VLM-Guided Macro-Routing (Stage 2)

A Vision-Language Model (VLM) handles the high-level semantic placement of bundles and standalone items.

• Context: The VLM receives a multi-modal input: a top-down rendering of the room with color-coded walls and openings, structured text descriptions of dimensions and openings, and item dimensions.
• Retrieval-Augmented Generation (RAG): A spatial plan is synthesized by retrieving curated interior design guidelines (e.g., walkway clearances) to inform the VLM.
• Semantic Suggestion: The VLM outputs relative placement suggestions (e.g., "center of wall," "next to door," "in front of sofa") rather than raw coordinates. These suggestions are restricted to a reference-dependent set to ensure feasibility.

3. Geometric Realization via Dynamic Placement (Stage 3)

A deterministic solver translates semantic suggestions into precise, collision-free 3D poses.

• Dependency Ordering: Objects are placed in a specific order (e.g., large anchors first, wall-mounted items last) based on a breadth-first traversal of reference dependencies.
• Feasibility Testing: A candidate pose is accepted only if the object lies fully within the floor polygon, clears all openings, and overlaps no previously placed objects.
• Local Search: If the primary target pose is infeasible, the solver performs a discrete search around the semantic anchor (e.g., sliding along the wall) to find a valid position. If no zero-overlap pose exists, it either assigns the globally minimum-overlap pose or reports the item as unplaced, avoiding forced collisions.

Key Contributions

1. Hybrid Hierarchical Pipeline: An end-to-end architecture that separates layout generation into clustering, macro-routing, and micro-refinement, effectively decoupling semantic intent from geometric constraints.
2. Constraint-Aware Macro-Routing: A VLM-guided strategy that treats irregular polygonal boundaries and fixed architectural openings as hard constraints, translating them into actionable spatial anchors for real-world floor plans.
3. Industry-Ready Micro-Arrangement: Deterministic, rule-based geometric algorithms that resolve final constraints with collision-aware guarantees, ensuring the output is suitable for retail applications where physical feasibility is non-negotiable.

Experimental Results

The system was evaluated on a production-scale catalog of IKEA products across various room topologies, including rectangular rooms without openings, rectangular rooms with doors/windows, and five types of non-rectangular plans (L, T, U, beveled, cut).

• Performance on Rectangular Rooms: Compared to baselines Holodeck and LayoutVLM, PolyLayout achieved the highest perceptual plausibility score (2.74/4.0) while maintaining perfect in-bounds placement (1.000) and a high placement ratio (0.973). It resolved the trade-off between completeness and validity that plagued the baselines (LayoutVLM had high completeness but low plausibility; Holodeck had high plausibility but low completeness).
• Latency: PolyLayout operates at CPU-level latency (70 seconds), significantly faster than LayoutVLM (907 seconds) and comparable to Holodeck (~62 seconds).
• Generalization to Irregular Geometries: PolyLayout was the only method capable of handling non-rectangular boundaries and fixed openings natively. As geometry complexity increased, the system degraded gracefully, maintaining 100% in-bounds placement and high completeness (>95%), with only a slight dip in perceptual plausibility.
• Human Evaluation: In a blind study, human raters preferred PolyLayout layouts in 61% of cases over the baselines. The VLM judge's scores correlated well with human preferences (72% agreement on the preferred layout).

Significance and Claims

The paper claims that PolyLayout successfully bridges the gap between academic layout generation and real-world retail requirements. By combining the semantic reasoning capabilities of VLMs with deterministic geometric solvers, the system produces layouts that are not only functionally coherent but also physically valid within complex, irregular home environments.

The authors position the system as a tool to lower the barrier for customers entering the "blank-room" problem, providing a concrete, plausible starting point for refinement. They acknowledge limitations, noting that the system is currently limited to single rooms from a specific catalog and that the fine-placement templates are hand-authored, representing a scalability cost. However, the framework demonstrates that hybrid approaches can achieve high perceptual quality and geometric compliance at low latency, extending capabilities to irregular boundaries that existing methods do not natively support.