Yo'City: Personalized and Boundless 3D Realistic City Scene Generation via Self-Critic Expansion

This paper introduces Yo'City, an agentic framework that leverages large models for hierarchical planning and a self-critic expansion loop to generate personalized, boundless, and spatially coherent 3D realistic city scenes, outperforming existing state-of-the-art methods across multiple evaluation metrics.

The Gist

Imagine you want to build a massive, realistic 3D city for a video game or a virtual reality tour. In the past, this was like trying to build a Lego city the size of Manhattan by hand: it took forever, required a team of architects, and if you wanted to add a new neighborhood later, you'd have to tear down half the city to make it fit.

Existing AI tools tried to help, but they were like blind painters. They could paint a few buildings, but they often got lost, made the buildings float in the air, or created a city where the "downtown" looked nothing like the "residential area." They lacked a master plan.

Enter Yo'City. Think of Yo'City not as a single painter, but as a super-organized construction crew led by a brilliant project manager. Here is how it works, broken down into simple steps:

1. The Master Planner (The "City-Block-District" Strategy)

Instead of just saying "Make a city," Yo'City breaks the job down into three layers, just like real urban planning:

• The City Level: A high-level AI (the Global Planner) looks at your request (e.g., "A magical Harry Potter theme park") and draws a rough map. It decides, "Okay, we need a big 'Wizarding District' here, a 'Muggle Shopping Mall' there, and a 'Quiet Residential Zone' over there."
• The District Level: A second AI (the Local Designer) zooms in. It takes the "Wizarding District" and says, "This specific block needs a tall clock tower, cobblestone streets, and shops that look like they're from the 1800s."
• The Grid Level: It breaks the whole city into tiny squares (like a chessboard). Every single square gets a specific, detailed instruction on what to build.

The Analogy: Imagine building a giant puzzle. Old methods tried to glue the pieces together one by one, often realizing the picture didn't match. Yo'City first prints the full picture on the back of the box (the plan), then builds every piece of the puzzle simultaneously so they all fit perfectly.

2. The "Self-Critic" Builder (The Produce-Refine-Evaluate Loop)

Once the plan is ready, Yo'City starts building the 3D models. But it doesn't just build and hope for the best. It uses a Self-Critic system:

1. Produce: It generates a 2D "top-down" drawing of a building block.
2. Refine: It looks at the drawing and says, "Wait, that roof looks weird," or "There are too many trees." It fixes the image.
3. Evaluate: A third AI acts as a strict inspector. It checks: "Does this look like the description? Is the texture clear? Are the buildings floating?" If the score is too low, it sends the image back to be fixed.

The Analogy: It's like a chef making a cake. They bake it (Produce), taste it and fix the frosting (Refine), and then a food critic tastes it to see if it's good enough to serve (Evaluate). If the critic says "too salty," the chef starts over on that specific cake until it's perfect.

3. The "Boundless" Expansion (Growing the City)

The coolest part of Yo'City is that the city never has to stop growing. If you say, "Hey, I want to add a new hospital," Yo'City doesn't just slap it on the edge.

• It looks at the existing city map.
• It asks: "Where does a hospital belong? It should be near the residential area for easy access, but far away from the noisy factory."
• It uses a Scene Graph (a digital map of relationships) to find the perfect spot that makes logical sense, then builds it seamlessly.

The Analogy: Imagine a living organism. If you add a new organ to a body, it has to connect to the right blood vessels and nerves. Yo'City is like a smart doctor that knows exactly where to place a new "organ" (a new city block) so the whole "body" (the city) stays healthy and connected.

Why is this a big deal?

• No More "Cartoon" Cities: Old AI cities often looked blurry or like a video game from the 90s. Yo'City creates crisp, realistic textures and buildings that look like real photos.
• Infinite Growth: You can keep adding neighborhoods, parks, and skyscrapers forever, and the city will always feel like one cohesive place, not a patchwork of mismatched pieces.
• Your Imagination is the Limit: Whether you want a "Cyberpunk Tokyo," a "Medieval European Village," or a "Harry Potter Theme Park," Yo'City understands the specific rules of that world and builds it accordingly.

In short, Yo'City is the difference between a child randomly stacking blocks and a master architect using a blueprint, a quality control team, and a smart growth plan to build a city that feels alive, real, and endless.

Technical Summary

Here is a detailed technical summary of the paper "Yo'City: Personalized and Boundless 3D Realistic City Scene Generation via Self-Critic Expansion."

1. Problem Statement

Generating realistic, personalized, and infinitely expandable 3D city scenes is a fundamental challenge for applications like virtual reality, digital twins, and urban planning. Existing methods face several critical limitations:

• Lack of Personalization: Most approaches rely on training single diffusion models on specific datasets (e.g., satellite maps), making it difficult to generate scenes based on flexible, user-defined text prompts.
• Scalability and Consistency: Current generative methods often struggle to maintain global structural consistency when scaling to city-sized environments. They frequently produce "flat" generations that lack the intrinsic hierarchical organization of real cities (e.g., districts, blocks).
• Expansion Limitations: Existing frameworks (like SynCity) often use autoregressive, tile-by-tile generation without explicit planning, leading to error accumulation, spatial incoherence, and simplified geometry (cartoonish or blurry textures) in large-scale scenes.
• Static Nature: Most methods generate a static scene and lack mechanisms for continuous, user-interactive evolution or expansion while maintaining spatial and semantic coherence.

2. Methodology: The Yo'City Framework

Yo'City is a multi-agent framework that leverages the reasoning and compositional capabilities of off-the-shelf Large Language Models (LLMs) and Vision-Language Models (VLMs). It operates through a "Planning-Generation-Expansion" pipeline without requiring training on city-specific datasets.

A. Hierarchical "City–District–Grid" Planning

Inspired by real-world urban structures, Yo'City employs a coarse-to-fine thinking strategy:

1. Global Planner:
   • Takes a high-level user prompt (e.g., "A modern business city with a Harry Potter theme park").
   • Estimates city size ($H \times W$ grid) and identifies functional districts (e.g., CBD, Residential, Recreation).
   • Allocates these districts onto the grid map, defining their sizes, locations, and adjacency.
   • RAG Integration: Uses Retrieval-Augmented Generation (RAG) to fetch real-world urban data (e.g., from Wikipedia) for reference cities (e.g., "New York-like") to ensure factual grounding and realistic spatial logic.
2. Local Designer:
   • Refines the coarse district blueprints into detailed, grid-level textual descriptions.
   • Ensures spatial and stylistic coherence across grids within the same district, defining architectural styles, building densities, landmarks, and context.

B. 3D Generation via "Produce–Refine–Evaluate" Loop

Instead of direct text-to-3D, Yo'City uses a two-stage process for each grid:

1. Isometric Image Synthesis:
   • Produce: Generates an initial isometric image on a predefined ground platform to ensure scale consistency.
   • Refine: An image editing model removes the platform and refines surfaces to correct artifacts and enhance diversity.
   • Evaluate: A specialized VLM evaluator assesses the image for text alignment, realism, and layout rationality. If the score is low, the prompt is rewritten, and the loop repeats.
2. Image-to-3D Conversion: High-quality isometric images are converted into 3D assets using a pre-trained image-to-3D model (Hunyuan3D).
3. Assembly: All 3D grid assets are arranged according to the global layout and connected with roads and ground surfaces.

C. Relationship-Guided Self-Critic Expansion

To enable boundless city growth, Yo'City introduces a mechanism for iterative expansion:

• User Input: The user requests a new grid (e.g., "Add a museum").
• Scene Graph Construction: A VLM analyzes the existing city and the new request to generate a description for the new grid and constructs a scene graph. This graph encodes qualitative spatial relationships (e.g., "near," "far") between the new grid and existing districts.
• Optimization: A distance- and semantics-aware optimization function determines the optimal placement ($x^*$) for the new grid:
  • Spatial Objective ($L_{dist}$): Minimizes distance to "near" zones and maximizes distance from "far" zones (e.g., placing schools near residential areas, but far from industrial zones).
  • Semantic Regularization ($L_{sem}$): Ensures the new grid's description is semantically compatible with its neighbors using Sentence-BERT embeddings.
• Result: The new grid is synthesized and seamlessly integrated, allowing the city to evolve indefinitely while maintaining logical coherence.

3. Key Contributions

1. Yo'City Framework: A novel multi-agent system for personalized, boundless 3D city generation driven by text, eliminating the need for training on massive city datasets.
2. Hierarchical Planning Strategy: A "City–District–Grid" approach that combines global structural planning with fine-grained local design, addressing the lack of spatial hierarchy in previous methods.
3. Self-Critic Expansion Mechanism: A scene-graph-based expansion module that optimizes new grid placement based on both spatial distance constraints and semantic coherence, enabling truly open-world city evolution.
4. Comprehensive Benchmark: The authors constructed a diverse dataset of 100 city descriptions and designed a multi-dimensional evaluation metric covering semantics, geometry, texture, layout, and realism.

4. Experimental Results

Yo'City was evaluated against state-of-the-art methods (Trellis, Hunyuan3D, CityCraft, SynCity) using both automated metrics and human/VLM judges.

• Quantitative Performance:
  • Semantic Consistency: Achieved the highest VQAScore (0.7151), significantly outperforming baselines (e.g., SynCity: 0.6975, CityCraft: 0.5639).
  • Visual Quality: Won pairwise comparisons in Geometric Fidelity (93.5%), Texture Clarity (94.0%), Layout Coherence (96.5%), and Overall Realism (95.0%) against GPT-5 and human judges.
• Qualitative Analysis:
  • Yo'City produces scenes with clear textures, high-fidelity details (e.g., windows), and consistent scales.
  • Baselines like SynCity often suffer from spatial inconsistency (dense clusters in some tiles, sparse in others) and blurry textures.
  • Yo'City successfully handles diverse styles (e.g., "Silk Road," "Disneyland," "Victorian") and complex constraints.
• Efficiency:
  • Due to its parallel generation strategy (generating all grids simultaneously rather than autoregressively), Yo'City is more efficient than SynCity, especially for larger city sizes.
• Ablation Studies:
  • Removing the coarse-to-fine planning strategy significantly degraded layout coherence and realism, proving the necessity of the hierarchical agent structure.
  • The expansion mechanism showed high stability (low Coefficient of Variation in VQAScore) across multiple expansion steps.

5. Significance and Impact

• Paradigm Shift: Moves 3D city generation from data-driven, static reconstruction to agentic, reasoning-driven, and interactive creation.
• Scalability: Solves the "boundless" problem by enabling continuous, logical city expansion without error accumulation, a major hurdle for autoregressive methods.
• Applications: Directly applicable to Virtual Reality (VR), Digital Twins for urban planning, Gaming (procedural world generation), and Robotics (simulation environments).
• Accessibility: By relying on off-the-shelf LLMs and VLMs, it lowers the barrier to entry for generating complex 3D environments without requiring specialized training data or massive computational resources for model training.

In summary, Yo'City represents a significant advancement in generative AI for 3D environments, successfully bridging the gap between high-level user intent and low-level geometric realization through a structured, self-criticizing multi-agent framework.