Technical Summary: Code-as-Room

Problem Definition

Designing realistic and functional 3D indoor rooms is critical for applications in interior design, virtual reality, gaming, and embodied AI. However, manual construction is labor-intensive, and existing automated methods face significant limitations. Traditional procedural methods are constrained by hand-crafted rules and predefined categories, failing to capture complex real-world spatial relationships. Recent Multimodal Large Language Model (MLLM) approaches have shown promise but struggle with precise spatial information when relying solely on text descriptions. Furthermore, existing image-conditioned agents tasked with holistic room generation from top-down views often suffer from instability, infinite looping, and an inability to recover fine-grained spatial details.

The core challenge addressed by this work is generating a complete, editable, and executable 3D indoor scene from a single top-down view image. The goal is to produce Blender code that accurately represents room structure, object placement, geometry, materials, and lighting, aligning strictly with the visual priors of the input image.

Methodology: Code-as-Room (CaR)

The authors propose Code-as-Room, an MLLM-based agentic framework equipped with a structured execution harness. Unlike direct generation methods, CaR decomposes the complex task into a principled, multi-stage pipeline that transforms a top-down image into executable Blender code.

1. Structured Execution Pipeline

The framework operates through five distinct phases, organized into ten specialized stages:

• Image-based Scene Structuring:

  • Stage 1 (Spatial Semantic Analysis): The VLM parses the top-down image to extract functional zones, object hierarchies, and architectural elements (walls, doors, windows), assigning identifiers and placement types.
  • Stage 2 (Scene Graph Construction): A deterministic skeleton is derived, constructing an object-centric scene graph that defines relationships between architectural features, major furniture, and minor objects (stored in a "sidecar" for later use).

• Layout Code Generation:

  • Stage 3 (Major Layout with Visual Feedback): The system generates initial layout code for major furniture. It employs a render–critique–revise loop where the current code is rendered into a top-down image, evaluated by a VLM-based critic for overlap, boundary consistency, and spatial relations, and then revised. This loop runs for a fixed number of iterations (e.g., 5) to ensure layout stability.
  • Stage 4 (Auxiliary Layout): Wall-mounted objects and visually salient minor objects (e.g., rugs, plants) are added to the layout, while tiny surface-bound items are deferred.

• Layout-Grounded Object Profiling:

  • Stage 5: The system parses the fixed layout to ground the VLM, generating fine-grained object descriptions (color, material, style) and a global room-style description, ensuring visual details align with the established spatial structure.

• Object-Level Code Generation:

  • Stage 6 (Geometry Replacement): The coarse layout proxies are replaced with detailed 3D geometry primitives based on the inferred object profiles. A hybrid strategy is used for tiny objects: simple geometric placeholders are created first, followed by asset retrieval from a library to replace them with matching 3D models.

• Interior Decoration Code Generation:

  • Stage 8 (Material Assignment): Part-level PBR materials are assigned based on the object dictionary and global style.
  • Stage 9 (Texture Generation): High-capacity image generation models synthesize texture maps for floors, walls, and decorative elements, which are injected into the scene.
  • Stage 10 (Lighting & Post-Hoc Optimization): The system infers lighting styles (natural and artificial) and sets up rendering parameters. A deterministic correction pass fixes common implementation issues (e.g., missing materials, invalid paths) and resolves boundary or overlap violations via local search.

2. Cross-Stage Memory System

A critical component of the framework is the Cross-Stage Memory Module. This shared memory maintains the persistent state of the pipeline, storing typed artifacts from each stage. Downstream stages read only predefined views of this memory. This design mitigates the "context forgetting" problem inherent in long agentic workflows, ensuring that spatial information and object details derived in early stages are preserved and accurately utilized in later stages.

Key Contributions

1. Top-Down Image-Guided Paradigm: The paper introduces a novel paradigm where a single top-down image serves as a global spatial prior to guide the complete generation of an indoor room, moving beyond text-only or perspective-image inputs.
2. Structured Execution Harness: The authors propose a structured agentic workflow that orchestrates MLLMs through a coarse-to-fine pipeline. This harness ensures stable and coherent 3D room synthesis by decomposing the task into manageable stages with visual feedback loops.
3. Dedicated Benchmark: A new benchmark for code-based 3D room synthesis is introduced, encompassing diverse evaluation protocols for visual content understanding, spatial reasoning, vision-to-code generation, and holistic scene quality.

Experimental Results

The authors evaluated Code-as-Room using three leading VLM backbones: Gemini-3 Flash, Gemini-3.1 Pro, and GPT-5.5.

• Benchmark Performance: Compared to direct single-pass generation and the existing VIGA framework, Code-as-Room demonstrated superior performance across all metrics.
  • Spatial Reasoning: CaR significantly improved Layout IoU (up to 73.2% vs. 16.8% for direct Gemini), Rotation Accuracy, and Support Accuracy.
  • Code Generation: The framework achieved high execution rates (up to 100% for Gemini variants), whereas direct generation often failed to produce runnable code.
  • Visual Quality: Human evaluations showed that CaR-generated scenes had higher similarity to input images, better usability, and higher acceptability (requiring fewer manual corrections) compared to baselines.
• Ablation Studies:
  • Memory: Removing the cross-stage memory resulted in a significant drop in Layout IoU (from 73.2% to 58.0%) and Object Recall, confirming its necessity for maintaining spatial consistency.
  • Visual Feedback: Performance improved with up to 5 feedback iterations. However, increasing iterations to 10 led to a decline in metrics, suggesting that excessive revision can cause layout drift.

Significance and Claims

The paper claims that Code-as-Room addresses the instability and lack of precision in current image-to-3D room generation. By representing 3D rooms as executable Blender code, the framework enables scenes that are not only visually aligned with input images but also editable, renderable, and structured.

The authors emphasize that the success of the system relies not just on the base VLM capabilities but on the structured agentic workflow that organizes visual parsing, spatial reasoning, and code synthesis. The introduction of the cross-stage memory module is highlighted as a key innovation for mitigating context forgetting in complex, multi-stage generation tasks.

The work positions itself as a step toward more reliable and practical tools for automated 3D content creation, particularly for scenarios where precise spatial alignment with floor plans or top-down sketches is required. The authors note limitations, such as the current focus on top-down views and the reliance on asset retrieval for highly complex geometric details, suggesting future work in video generation for neural rendering and handling arbitrary-view inputs.