🤖 AI

MAGIC: Transition-Aware Generation of Navigable Multi-Scene Game Worlds with Large Language Models

The paper presents MAGIC, a four-stage prompt-to-project system that leverages large language models to automatically generate navigable, consistent multi-scene game worlds with functional transitions, addressing key challenges in cross-scene consistency, in-scene navigability, and transition validation through a novel pipeline and evaluation agent.

Original authors: Tsz Hei Fan, Choi Wing Fung, Yuxuan Wan, Shuqing Li, Michael R. Lyu

Published 2026-07-14

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Tsz Hei Fan, Choi Wing Fung, Yuxuan Wan, Shuqing Li, Michael R. Lyu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you're trying to build a massive, interconnected video game world where you can walk from a cozy bedroom, through a door, and instantly find yourself in a spooky dungeon. For human game designers, this is a nightmare of paperwork. They have to manually draw maps, ensure the door on the left side of the bedroom matches the door on the right side of the dungeon, and double-check that no giant bookshelf has been placed in front of the door, blocking the player's path. It's like trying to build a house of cards where every card has to be perfectly aligned with the one above and below it, or the whole thing collapses.

Enter MAGIC, a new system that acts like a super-organized, hyper-logical architect. Instead of just building one room at a time and hoping they fit together later, MAGIC plans the entire journey before it even lays a single brick.

The Problem with "One Room at a Time"

Previous AI tools were great at designing a single, beautiful room. But if you asked them to build a whole world by just repeating the process, it would fail miserably. Imagine asking a painter to paint a hallway by painting one wall, then walking to the next wall and painting it without ever looking at the first one. The doors wouldn't line up, the floors would be different heights, and the result would be a confusing mess.

The paper identifies three specific reasons why this "one room at a time" approach fails:

The "Lost Connection" Problem: The AI forgets that a door in Room A needs to lead to Room B. It might invent a door that leads nowhere or a door that doesn't match the one on the other side.
The "Furniture Blockage" Problem: The AI might put a giant sofa right in front of the door. The door exists, but you can't walk through it.
The "Did It Work?" Problem: No one checks if the door actually opens. Previous tools just looked at how pretty the room was, never testing if you could actually walk through the game.

How MAGIC Fixes It: The Four-Stage Pipeline

MAGIC solves this by treating the game world like a train system with a master schedule, rather than a collection of isolated stops.

Stage 1: The Master Planner
First, MAGIC takes your simple text idea (like "a house with a secret basement") and acts as a strict project manager. It doesn't just write a description; it draws a transition graph. Think of this as a subway map. It decides exactly which rooms exist, where the doors (portals) are, and what kind of "magic effect" happens when you walk through them (like a fade-out or a wipe). It creates a shared blueprint that every future step must follow, ensuring no room gets lost.

Stage 2: The Blueprint Checker
Next, it designs the furniture for each room. But here's the trick: before it finalizes the design, it runs a flood-fill test. Imagine pouring virtual water into the room. If the water can reach every corner and every door, the room is "navigable." If the water gets stuck behind a virtual sofa, MAGIC knows the design is broken. It then rearranges the furniture until the "water" can flow freely to every exit. This ensures you won't get stuck in a room with a blocked door.

Stage 3: The Builder
Once the blueprints are perfect and the doors are reachable, MAGIC builds the actual 3D models and writes the computer code (scripts) that tells the game engine, "When the player touches this door, load the next scene." It's like assembling the LEGO set according to the instructions, making sure the pieces snap together correctly.

Stage 4: The Stitcher
Finally, it takes all the separate room files and stitches them into one single, playable game project.

The "Play-Tester" Robot

The most exciting part is how MAGIC checks its own work. The authors built a special evaluation agent—basically a robot player that runs inside the finished game. This robot doesn't just look at pictures; it actually plays the game. It spawns in the starting room, walks to every door, tries to open it, and sees if it successfully teleports to the next room. It even takes photos of the doors to make sure they look like what was requested.

What the Numbers Say

The authors tested this system on 100 different multi-scene game cases, ranging from simple loops to complex branching paths. The results were impressive:

Success Rate: MAGIC produced a working, playable game project for every single one of the 100 cases.
Accuracy: When checking if the transitions worked correctly, MAGIC achieved a 0.99 precision (meaning almost every transition it made was correct), 0.95 recall (meaning it found almost all the required transitions), and a 0.96 F1 score (a balanced measure of both).
Comparison: Compared to other methods (like a standard AI baseline or a tool called Holodeck), MAGIC was far better at keeping the rooms connected and preventing blocked paths. For instance, while other methods often missed doors or created blocked paths, MAGIC's "flood-fill" check ensured 0.9952 connectivity, meaning the robot could reach almost every walkable spot in the game.

What MAGIC Can't Do Yet

It's important to know where MAGIC stops. The paper explicitly states that this system currently only works for indoor scenes (like houses and offices) and only within the Unity game engine. It also only understands English text prompts and currently supports just two types of transition effects: "FadeInOut" and "IrisWipe."

If you ask it to build a giant outdoor forest or a spaceship, it can't do it yet. Also, if the system runs out of tries to fix a blocked door, it will give you the "best effort" layout, which might still have a tiny blockage, though this happened very rarely in their tests.

The Bottom Line

MAGIC isn't just a tool that draws pretty pictures; it's a system that understands the logic of moving between spaces. By planning the connections first, checking if you can actually walk through the doors, and then building the game, it turns a single sentence of text into a fully navigable, multi-room adventure. While it's not a magic wand that solves every game design problem (yet), it proves that we can finally automate the tricky, laborious job of connecting game worlds without breaking them.

Technical Summary: MAGIC – Transition-Aware Generation of Navigable Multi-Scene Game Worlds

Problem Statement
Contemporary 3D games rely on multi-scene navigation, where players clear objectives in bounded spaces and cross portals (doors, stairwells) to transition to new environments. While recent Large Language Models (LLMs) and Multimodal LLMs (MLLMs) have significantly reduced the cost of synthesizing single furnished interiors, they fail to generate connected multi-scene worlds. Naive repetition of single-scene generation leads to three critical failures:

Cross-Scene Consistency: Stateless prompting causes models to hallucinate or drop cross-scene references, resulting in portals that lack consistent endpoints or effects on both sides.
In-Scene Navigability: Even if portals are named correctly, the placement of furniture may physically block access to them, rendering the transition impossible. Single-scene metrics do not detect this.
Evaluation Gap: Existing text-to-3D metrics measure visual fidelity or prompt alignment but never execute a transition. Consequently, mislinked portals or broken scripts remain invisible to standard evaluation.

Methodology: The MAGIC Pipeline
MAGIC (Multi-scene Automated Game worlds generator with Intelligent Connectivity) is a four-stage, prompt-to-project system designed to generate runnable multi-scene Unity projects from a single natural-language prompt.

Planning Stage:
- An LLM processes the user's multi-scene prompt to generate individual scene descriptions and a transition-aware intermediate representation (IR).
- The IR is formulated as a graph $A = (S, P, \delta, E)$ , where $S$ are scene descriptions, $P$ are portals, $E$ are transition effects, and $\delta$ defines the symmetric transition relation between scenes.
- A verification loop ensures every scene description explicitly mentions the required number and type of portals to maintain graph connectivity.
Scene Specification Stage:
- This stage expands scene descriptions into structured specifications for Unity, including object lists and placement constraints in a Domain-Specific Language (DSL).
- Portal Handling: Objects and connections (doors/windows) are tagged with an isPortal attribute. A verification step ensures all expected portals are generated.
- Navigability Validation: To prevent blockage, the system employs a flood-fill algorithm on a 2D occupancy grid ( $G \in \{0, 1\}^{r \times c}$ ). It simulates walkable space and verifies that every portal is reachable from all walkable cells. If blockage is detected, the object placement is regenerated iteratively until connectivity is maximized.
Scene Generation Stage:
- Assets (meshes for walls, floors, objects) are generated using the Scenethesis composition algorithm.
- Script Generation: Transition scripts (LevelLoader) are automatically generated and bound to the correct portal colliders. Unlike previous approaches, these scripts are attached to portal objects rather than the camera to ensure reliable triggering.
- The output is a single-scene executable Unity project.
Combination Stage:
- Individual scene projects are merged into a single multi-scene Unity project. The combination algorithm stitches the scenes together using the transition scripts generated in Stage 3, ensuring scenes reference each other correctly.

Key Contributions

End-to-End Framework: MAGIC is the first system to generate interconnected, navigable multi-scene game worlds from a single prompt, utilizing a shared intermediate representation and a portal-reachability validator to enforce consistency and navigability.
Multi-Scene Transition Benchmark: The authors introduce a new benchmark of 100 instances derived from MIT Indoor 67 and MMIS datasets. It covers scene graphs with looping and branching patterns and provides ground-truth cross-scene transitions and portal reachability data.
Transition-Focused Evaluation Agent: A novel agent that operates directly on the packaged Unity project. It executes transitions in-play, navigates from spawn points to portals, and uses an MLLM judge to assess portal appearance. It reports Precision, Recall, F1, Approach Rate, and Portal Match Rate.
Comprehensive Evaluation: The paper provides both stage-by-stage and end-to-end evaluations, demonstrating the system's ability to recover ground-truth portals and produce navigable layouts superior to LLM baselines and Holodeck.

Results
Evaluated on 100 multi-scene cases, MAGIC achieved the following:

End-to-End Transition Identification: 0.99 Precision, 0.95 Recall, and 0.96 F1 score.
Navigability: MAGIC achieved a connectivity score of 0.9952 (near-perfect reachability), significantly outperforming the LLM baseline (0.8718) and Holodeck (0.8545).
Executability: MAGIC produced an executable project for 100% of test cases. In contrast, the LLM baseline failed to produce any executable projects due to missing essential components or file naming errors.
Portal Generation: MAGIC demonstrated high recall (0.9487), identifying nearly all ground-truth portals, while maintaining high precision (0.8333).

Significance and Claims
The paper claims that MAGIC addresses the specific obstacles left unsolved by single-scene generators: consistency, navigability, and the lack of execution-based evaluation. By introducing a transition-aware intermediate representation and a flood-fill validator, the system ensures that generated worlds are not just visually coherent but functionally navigable. The authors position MAGIC as a significant step toward automating the labor-intensive process of authoring connected game levels, noting that while current limitations exist (indoor scenes only, specific transition effects, English prompts), the pipeline successfully bridges the gap between text-to-3D synthesis and playable, multi-scene game development. The work establishes that with proper structural constraints and execution-based validation, LLMs can be guided to generate complex, interconnected 3D environments.