🤖 AI

InfiniteDiffusion: Bridging Learned Fidelity and Procedural Utility for Open-World Terrain Generation

The paper introduces InfiniteDiffusion, a training-free algorithm that enables real-time, infinite, and seed-consistent terrain generation by reformulating diffusion sampling to combine the high fidelity of learned models with the unbounded utility of traditional procedural noise.

Original authors: Alexander Goslin

Published 2026-05-06

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Alexander Goslin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build a video game world that is truly infinite—bigger than the entire Earth, with no edges. For decades, developers have used a "mathematical recipe" called Procedural Noise (like Perlin noise) to do this. It's like having a magic ink that can spill out forever, creating hills and valleys instantly. It's fast, it never runs out of memory, and if you type in a specific code (a "seed"), you get the exact same mountain every time.

The Problem: While this magic ink is great for speed, the landscapes it creates look a bit fake. They are too smooth and repetitive. They lack the chaotic, realistic details of real mountains, river basins, and eroded canyons.

The Other Problem: Recently, AI image generators (called Diffusion Models) have become amazing at creating hyper-realistic pictures. They can make a mountain look exactly like a photograph. But these AI models are like painters who only have a small canvas. They can't paint an infinite world; they get confused if you ask them to keep going forever, and they usually need to save the whole picture in memory before showing it to you.

The Solution: InfiniteDiffusion
The authors of this paper, led by Alexander Goslin, created a new method called InfiniteDiffusion. Think of it as a translator that teaches the hyper-realistic AI painter how to use the infinite magic ink.

Here is how it works, using simple analogies:

1. The "Lazy Painter" (Lazy Evaluation)

Usually, to generate a huge image, you have to paint the whole thing at once. But InfiniteDiffusion is a "lazy" painter. It only paints the specific square of the world you are looking at right now.

The Analogy: Imagine you are walking through a forest. You don't need to see the whole forest at once. You only need to see the trees immediately around you. If you turn left, the painter instantly paints the trees on the left. If you turn right, it paints those. It never paints the trees you aren't looking at, so it never runs out of memory.

2. The "Overlapping Stencils" (Sliding Windows)

To make sure the world looks seamless (so you don't see a hard line where one painted square ends and the next begins), the AI uses a technique called "sliding windows."

The Analogy: Imagine trying to paint a mural on a wall using a small stencil. You paint one section, then slide the stencil over slightly so it overlaps the previous section. You blend the paint where they overlap so there is no seam. InfiniteDiffusion does this mathematically across an infinite grid, blending millions of tiny overlapping predictions into one giant, seamless world.

3. The "Team of Artists" (Hierarchical Modeling)

Creating a whole planet is too hard for one AI model. So, the authors built a team of three artists working together:

The Architect (Coarse Model): This artist draws the big picture first. They decide where the continents and oceans go, using simple sketches or user drawings.
The Sculptor (Core Model): This artist takes the big sketch and adds the major features: huge mountain ranges, deep valleys, and river systems.
The Detailer (Decoder): This artist zooms in and adds the tiny, realistic details: the texture of the rocks, the specific shape of a cliff, and the snow on the peak.
The Result: This team approach ensures the world looks realistic up close but still makes sense from space.

4. The "Magic Seed" (Seed Consistency)

One of the most important features is that the world is deterministic.

The Analogy: If you give the AI the same "seed" (a starting number), it will generate the exact same mountain at the exact same location, every single time, no matter when you ask for it. You could teleport to a mountain in the middle of the ocean, and the AI would instantly generate it, and it would look exactly the same as it did the last time you visited. This is crucial for video games, so players don't see the world change when they walk away and come back.

5. The "Speed Demon" (Performance)

The paper claims this system is incredibly fast.

The Claim: On a standard consumer graphics card (like one you might buy for a gaming PC), the system can generate terrain 9 times faster than a fighter jet flies.
What this means: If you are flying a plane over this world, the computer can generate the ground in front of you faster than you can fly over it. This allows for real-time exploration of a planet-sized world without any lag.

Summary

The paper introduces InfiniteDiffusion, a way to combine the realism of modern AI with the infinite, fast, and consistent nature of old-school game math. They tested this by building Terrain Diffusion, a system that can generate a realistic, infinite planet in real-time. They even plugged it into the game Minecraft, replacing the game's native world generator with their AI, allowing players to explore a procedurally generated world that looks like a photograph but behaves like a video game.

The key takeaway is that they solved the "infinite canvas" problem for AI, allowing us to have worlds that are both limitless and photorealistic.

Technical Summary: InfiniteDiffusion

Problem Statement

Procedural terrain generation has long relied on noise functions (e.g., Perlin, Simplex) due to their ability to provide seamless infinite extent, seed-consistency, and constant-time random access. However, these methods are fundamentally limited in realism, lacking the hierarchical, multi-scale organization of natural geography (continents, river basins, erosion). Conversely, modern diffusion models offer unprecedented fidelity and control but are typically confined to bounded canvases. Existing attempts to extend diffusion to infinite domains often sacrifice core procedural properties (such as seed-consistency or random access) or fail to maintain global coherence across large scales. The paper addresses the challenge of bridging the high fidelity of learned generative models with the functional utility of procedural noise for open-world generation.

Methodology

1. InfiniteDiffusion Algorithm

The core contribution is InfiniteDiffusion, a training-free generalization of MultiDiffusion designed for lazy, unbounded generation.

Reformulation: Unlike standard MultiDiffusion, which assumes a finite set of windows over a fixed canvas, InfiniteDiffusion reformulates the sampling process over an infinite domain ( $\mathbb{Z}^2$ ). It defines a lazy evaluation strategy where only the windows intersecting a queried region $R$ are computed.
Lazy Evaluation & Caching: The algorithm maintains two sparse infinite tensors, $A_t$ (accumulated numerator) and $B_t$ (accumulated weights), to store window contributions. When a region is queried, the system recursively evaluates only the necessary windows, caching results to avoid recomputation.
Truncated $T$ : To make queries tractable, the authors redefine the diffusion model $\Phi$ as an arbitrary denoising function (e.g., a few-step consistency model) rather than a single atomic step. This allows the truncation of the diffusion schedule ( $T$ ) to 1 or 2 steps without significant loss of coherence, decoupling the number of inference steps from the internal diffusion schedule.
Properties: The algorithm guarantees:
- Seed Consistency: The entire infinite output is deterministically determined by a single seed, independent of query order.
- Constant-Time Random Access: Querying any fixed-size region is $O(1)$ , independent of location or previous queries.
- Parallelization: Window updates at a fixed timestep are independent and can be executed in parallel.

2. Infinite Tensor Framework

To support the storage and manipulation of infinite tensors without exceeding memory limits, the authors introduce an open-source Infinite Tensor framework. This Python library abstracts sliding window computation, allowing models to operate on implicitly infinite images using bounded transient caches and tiled disk storage.

3. Terrain Diffusion Framework

The authors apply InfiniteDiffusion to create Terrain Diffusion, a learned procedural terrain generator.

Data: A global dataset merging MERIT DEM (land) and ETOPO1 (ocean) with WorldClim climate data, processed into equal-area 2048×2048 tiles to minimize polar distortion.
Preprocessing:
- Signed Square-Root Transform: Applied to elevation data to compress high-relief values and distribute variance more uniformly, improving training stability.
- Laplacian Encoding: To handle the large dynamic range of Earth's elevations, the model predicts a low-frequency component (blurred/downsampled) and a high-frequency residual. A denoising step re-extracts the low-frequency channel from the noisy output to eliminate low-frequency errors while preserving high-frequency detail.
Hierarchical Architecture:
- Coarse Model: Generates basic continental structure (23km/pixel) from user sketches or procedural noise.
- Core Latent Model: A latent diffusion model (EDM2 + sCM backbone) that refines the coarse structure into 46km tiles in latent space, predicting low-frequency elevation channels and residual latents.
- Consistency Decoder: A diffusion decoder that expands the latents into high-fidelity elevation maps.
- Conditioning: The core model is conditioned on coarse elevation patches and climatic variables (temperature, precipitation) to ensure global coherence.

Key Results

Visual Fidelity: On a consumer GPU (RTX 3090 Ti), the system generates realistic terrain with sharp ridges, coherent river basins, and smooth transitions.
- FID Scores: InfiniteDiffusion with $T=2$ achieves an FID of 14.78 for tiled generation, nearly matching the non-tiled base consistency model (12.72) and significantly outperforming Perlin blending (186.70) and naive tiling (74.44).
- Artifact Reduction: Increasing $T$ from 1 to 2 effectively eliminates most artifacts introduced by the infinite tiling process.
Performance & Latency:
- Speed: The system generates terrain at interactive rates. On a 512×512 tile (90m resolution), the Time to First Tile (TTFT) is ~1.72s, and the Time to Second Tile (TTST) is ~0.66s.
- Traversal Speed: The generation speed exceeds orbital velocity (approx. 7,700 m/s) by a factor of 9. A vehicle traveling at highway speeds encounters a 170x performance buffer.
- Scalability: The hierarchical approach allows for zooming into infinite worlds with amortized costs that remain $O(S^2)$ regardless of cascade depth.
Integration: The framework was successfully integrated into the Minecraft engine, replacing the native world generator to stream real-time, biome-consistent terrain.

Significance and Claims

The paper claims that InfiniteDiffusion successfully bridges the gap between the high fidelity of diffusion models and the functional utility of procedural noise. By retaining seed-consistency and constant-time random access, it enables the creation of "practical infinite virtual worlds" that are both realistic and interactive.

The authors position Terrain Diffusion as the first practical learned procedural terrain generator capable of streaming an entire planet in real time on consumer hardware. They argue that this approach moves beyond the limitations of noise-based methods (which lack structure) and bounded diffusion methods (which lack scale), offering a foundation for the next generation of open-world generation. The paper modestly notes limitations, such as the inefficiency of scattered point queries (due to window-based generation) and the challenge of acquiring training data for the coarsest layers of the hierarchy, suggesting synthetic data as a potential solution.