ShellMaker: Language-Guided Exterior Completion under Structural Constraints

Technical Summary: ShellMaker: Language-Guided Exterior Completion under Structural Constraints

Problem Statement
While recent advancements in indoor scene generation have enabled the automated synthesis of plausible interiors with structured floor plans and wall layouts, these systems typically terminate at the building shell. They fail to generate coherent, stylistically consistent exteriors (facades and roofs) that align with the generated interiors. Existing 3D generative models, including text-to-3D and image-to-3D approaches, are generally unconstrained and struggle to adhere to the hard structural constraints required for exterior completion. Specifically, generated exteriors must strictly preserve the fixed building footprint, wall geometry, and the precise spatial semantics of door and window openings to remain compatible with the interior layout. Current methods often violate these constraints, producing assets that are geometrically misaligned or structurally incoherent with the underlying scaffold. Furthermore, large-scale paired datasets linking structured building layouts to fully textured exterior meshes are scarce, limiting the training of controllable generative models for this specific task.

Methodology
ShellMaker is a language-guided framework designed to complete building exteriors from structured scaffolds while strictly enforcing structural constraints. The system operates on a format-agnostic scaffold representation (derived from indoor scene generators, CityGML, or CAD models) and a text-based style prompt. The synthesis process is decomposed into three coordinated stages:

1. Structure Generation Stage:

   • Scaffold Parsing: Heterogeneous inputs (indoor scene graphs, CityGML, BIM/CAD) are converted into a unified schema containing wall segments, floor polygons, and opening metadata.
   • Footprint Decomposition: Complex footprints are partitioned into simpler subregions using either straight-skeleton decomposition (medial axis) or greedy rectangle decomposition to facilitate roof generation.
   • Parametric Roof Generation: The system generates roof geometry from five canonical types (flat, gable, pyramid, hip, half-hip) controlled by parameters such as pitch angle and overhang. Roofs can be generated as a single mass or as subdivided compositions over complex footprints.

2. Stylization Stage:

   • Part-Aware Prompt Refinement: An instruction-tuned Large Language Model (LLM) refines coarse user prompts (e.g., "Victorian bakery") into structured, part-specific prompts for individual architectural elements (walls, roofs, windows, doors). This ensures the LLM generates detailed geometric and decorative cues without including environmental context that would complicate compositing.
   • Two-Stage Text-to-3D Synthesis: Refined prompts are processed by Nano Banana to generate reference images, which are then reconstructed into textured meshes with PBR materials by Trellis-2.
   • Joint Wall-Roof Texture Retrieval: Instead of generating textures from scratch, ShellMaker employs a compatibility-aware retrieval mechanism. It precomputes a compatibility matrix based on material co-occurrence, color harmony (Lab*), frequency alignment (2D FFT), and semantic similarity (CLIP). At runtime, it selects wall and roof texture pairs that maximize both prompt alignment and cross-surface compatibility.

3. Assembly Stage:

   • Geometry-Aware Assembly: The system integrates the scaffold, generated roof, and assets into a single mesh.
   • Shape-Conforming Cutting: A 2D boundary polygon is extracted from generated assets to drive boolean subtraction for wall openings, ensuring precise alignment with predefined door and window locations.
   • UV Parameterization: World-space planar UVs are assigned with automatic rescaling based on the texture's dominant spatial frequency to maintain consistent perceptual texture density across different materials.
   • Detailing: Optional procedural generation adds roof utilities (gutters, downspouts) and ornaments (chimneys, dormers), clipped to conform to the roof geometry.

Key Contributions

1. Task Formulation: The paper formulates the problem of language-guided exterior completion under hard scaffold constraints, transforming partial building scaffolds into complete, style-consistent assets.
2. Modular Pipeline: A novel pipeline integrates procedural structure generation, LLM-driven part-level stylization, and compatibility-aware material retrieval. This modular approach limits cross-stage error propagation and facilitates asset reuse.
3. Geometry-Aware Assembly: The introduction of an assembly process that enforces strict preservation of footprint, wall boundaries, and opening placements, resulting in editable, part-structured exterior meshes.

Results
The authors evaluated ShellMaker on 200 diverse scaffolds (100 from indoor generators, 50 CityGML, 50 CAD) using 60 style prompts and a library of 500+ textures.

• Quantitative Metrics: ShellMaker achieved a Footprint IoU of 0.992 and an Opening Center L2 error of 0.091, significantly outperforming baselines (Trellis-2, SpaceControl, SAT-Skylines) which struggled to maintain structural integrity (e.g., Trellis-2 Footprint IoU: 0.492). ShellMaker also matched or exceeded baselines in style alignment metrics (CLIP-Sim and UNI3D).
• Perceptual Evaluation: In a 2AFC study using GPT-4.1 as a judge, ShellMaker achieved win rates above 70% against all baselines, with particularly high scores in structural plausibility (84.1% vs. Trellis-2).
• Generalization: The system demonstrated consistent performance across different input formats, though CAD inputs with noisy geometry showed slightly lower structural metrics.

Significance and Claims
The paper claims that ShellMaker bridges the gap between perceptually realistic generation and layout-faithful exterior synthesis. By operating on format-agnostic scaffolds, it enables the automatic completion of exteriors for buildings generated by indoor scene pipelines, CityGML data, or CAD models without modifying the underlying pipeline. The authors assert that their method improves architectural coherence, material harmony, and constraint satisfaction compared to independent retrieval pipelines and unconstrained generative baselines. The work represents a practical step toward the automated synthesis of complete building assets that possess both coherent interiors and structurally valid, stylistically consistent exteriors. The authors acknowledge limitations in handling degenerate part generation and the current lack of per-floor facade articulation, noting that future work could explore joint layout generation and generative texture synthesis.