Technical Summary: Function2Scene

Problem Statement

Current text-driven 3D indoor scene synthesis methods predominantly rely on object-centric prompts (e.g., "a bedroom with a queen bed and two nightstands"). While these methods can generate visually plausible scenes, they often fail to address the core of interior design: functionality. Real-world design briefs are functional specifications describing who occupies a space, what they do there, and the specific constraints their needs impose (e.g., "a bedroom for a couple where one partner reads late while the other sleeps early").

Existing Large Language Model (LLM)-based approaches struggle with these high-level functional specifications. When tasked with generating scenes directly from such prompts, they frequently produce layouts that are visually coherent but functionally invalid, failing to support specific activities, ergonomic needs, or environmental constraints. Furthermore, the design knowledge in deep learning-based methods is often implicit and uninterpretable, making it difficult to ensure a layout supports human use.

Methodology: Function2Scene

The authors propose Function2Scene, a framework that shifts the paradigm from object-centric generation to functionality-first layout design. The system operates through a pipeline that parses natural-language functional specifications into a structured set of design constraints and iteratively refines a layout to satisfy them.

1. Constraint Taxonomy

The system grounds its design logic in a taxonomy of 17 criteria derived from interior design literature, organized into four categories:

• Spatial (S1–S5): Governs geometric validity, boundary attachment, spatial relationships (grouping), scale/proportion, and visual composition.
• Ergonomic (E1–E4): Ensures safe and comfortable navigation and use, covering circulation paths, interaction clearances (e.g., door swings, chair pull-outs), reachability, and body fit/posture.
• Activity (A1–A4): Validates that the layout supports specific tasks, including zone allocation, multi-activity compatibility, sightlines/privacy, and workflow sequencing.
• Environmental (N1–N4): Addresses light access, glare prevention, acoustic separation, and ventilation/thermal comfort.

These constraints are assigned to six priority tiers (T1–T6), where lower-tier constraints (e.g., geometric validity) must be satisfied before higher-tier ones are considered.

2. Pipeline Architecture

The system functions in two main stages:

A. Initialization

• Parsing: An LLM parses the raw functional prompt to extract occupant personas, activities, and a structured list of constraints based on the taxonomy. It also generates a reformulated, LLM-friendly scene description.
• Room Structure Generation: The system generates the room's architectural shell (walls, floors, doors, windows) encoded in a custom JSON-based Domain-Specific Language (DSL). This step includes a human verification checkpoint to ensure the structural geometry is correct before furnishing.
• Furniture Initialization: An initial furniture layout is generated. The authors note that LLMs at this stage often produce spatially invalid results (overlaps, blocked paths), necessitating the refinement stage.

B. Iterative Evaluation and Refinement (Check-and-Repair Loop)
Rather than relying on a single generation pass, Function2Scene employs an iterative loop guided by a tool-augmented agent:

1. Constraint Evaluation: The agent evaluates constraints sequentially by priority. It invokes specialized tools to gather data:
   • Geometric Tools: Compute quantitative measures (e.g., overlap ratios, pathfinding, clearance widths).
   • LLM Tools: Perform contextual reasoning on structured scene data (e.g., reachability, workflow logic).
   • VLM Tools: Assess holistic visual quality and balance from rendered images.
2. Layout Refinement: If a constraint is violated, the LLM generates a targeted refinement action (e.g., repositioning, resizing, or reorienting furniture) based on design principles. Adjustments are applied locally to avoid disrupting previously satisfied constraints.
3. Termination: The process continues until all constraints are assessed. A final re-evaluation of Tier 1 constraints ensures foundational layout quality has not been compromised by later adjustments.

Key Contributions

1. Functionality-First Framing: The work reframes indoor scene synthesis from placing plausible objects to designing spaces that support human use, shifting input from object lists to functional specifications.
2. Customizable Constraint Taxonomy: It introduces a design constraint taxonomy rooted in interior design literature, coupled with an LLM-driven method to automatically customize these constraints to specific personas and activities.
3. Iterative Tool-Augmented Framework: It proposes a check-and-repair loop that combines geometric measurements, LLM reasoning, and VLM visual assessment to iteratively refine layouts, addressing the limitations of direct LLM generation.

Results and Evaluation

The authors evaluated Function2Scene using 30 professionally written interior design cases from Architectural Digest, covering diverse room types and personas (e.g., retired couples, drag queens, remote workers).

• Perceptual Study: A two-alternative forced-choice (2AFC) study with 32 valid participants compared Function2Scene against three state-of-the-art baselines: Holodeck, I-Design, and LayoutVLM.
• Performance: Function2Scene was preferred in 94.3% of pairwise comparisons overall. It outperformed baselines across both original functional prompts and parsed prompt conditions.
• Ablation Study: The study confirmed that the evaluation tools are critical. Removing the tools while retaining iterative updates resulted in worse performance than removing both, indicating that iterative refinement without grounded spatial feedback is counterproductive.

Significance and Claims

The paper claims that Function2Scene successfully addresses the gap between current generative AI capabilities and real-world interior design workflows. By moving away from implicit statistical priors toward explicit, functional design constraints, the system produces layouts that are not only visually coherent but also functionally valid for specific human needs.

The authors position this work as a step toward LLM-driven pipelines that better suit professional design workflows. They acknowledge limitations, noting that current inputs require detailed, professionally written specifications, whereas real users often start with vague demands. Future work is suggested to include conversational interfaces to help users refine their needs and more advanced simulation tools (e.g., embodied simulation, physical lighting) to enhance constraint verification.