💻 computer science

TerraDiT- $\Omega$ : Unified Spatial Control for Satellite Image Synthesis with Any Geospatial Primitive

The paper introduces TerraDiT- $\Omega$ , a unified generative framework that synthesizes satellite imagery directly from diverse native geospatial primitives (such as polygons, polylines, bounding boxes, and points) via a novel Geometry-Aware Local Attention mechanism, thereby enabling flexible spatial control for urban planning and enhancing downstream GeoAI tasks through controllable synthetic data augmentation.

Original authors: Brian Wei, Srikumar Sastry, Daniel Cher, Eric Xing, Nathan Jacobs

Published 2026-07-01

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Brian Wei, Srikumar Sastry, Daniel Cher, Eric Xing, Nathan Jacobs

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you want to build a realistic model city out of clay, but instead of sculpting it by hand, you have a magical robot that can instantly create the city for you. The problem is, the robot is used to making pictures of forests and beaches (natural images), where things are soft and blend together. But satellite images of cities are different: they are made of sharp, distinct shapes like roads, buildings, and parks, all packed tightly together.

If you try to tell the robot, "Put a building here," using a simple dot or a fuzzy outline, it gets confused. It might put the building in the wrong spot, or make the road look like a river. Previous attempts to fix this involved turning the precise city plans into blurry pixel maps (like a low-resolution photo) or just giving the robot a few random dots. Both methods were clunky: one lost the fine details, and the other was too vague.

Enter TerraDiT-Ω: The "Universal City Planner"

This paper introduces a new AI system called TerraDiT-Ω. Think of it as a super-smart architect that can take instructions in any format you have, whether it's a detailed blueprint, a rough sketch, or just a few sticky notes.

Here is how it works, broken down into simple concepts:

1. Speaking the Language of Maps

Most AI models need instructions converted into a specific "pixel" format (like a digital coloring book where you color every square). TerraDiT-Ω is different. It speaks the native language of maps: Geospatial Primitives.

Polygons: Like drawing the exact shape of a park with a pen.
Polylines: Like drawing a winding road.
Boxes: Like putting a square around a building.
Points: Like dropping a pin on a tree.

The system doesn't force you to convert these shapes into blurry pixels. It understands them directly, just like a human city planner understands a blueprint.

2. The "Geometry-Aware" Magic

The secret sauce of this system is a new tool they call Geometry-Aware Local Attention (GALA).

Imagine you are trying to paint a picture of a highway.

Old AI: Might see a "road" instruction and paint a straight line, but if the road curves, it gets lost. It treats the road like a generic blob.
TerraDiT-Ω: Uses GALA to "feel" the shape. If you give it a curved polyline, GALA tells the AI, "Hey, this road bends! Make sure the pixels follow that exact curve." If you give it a box, it knows to fill that specific rectangle.

It's like giving the artist a magnet that pulls the paint exactly where the shape dictates, ensuring that a highway stays a highway and a building stays a building, even when they are crowded together.

3. Flexible Budgets (The "Budget" Analogy)

One of the biggest problems in making these maps is that drawing every single building perfectly (high annotation budget) takes forever and costs a lot of money. But just dropping a few pins (low annotation budget) isn't enough to get a good result.

TerraDiT-Ω is like a flexible contractor:

Low Budget: You give it a few points (pins). It does its best to guess the layout.
Medium Budget: You give it boxes. It gets more accurate.
High Budget: You give it precise polygons and lines. It creates a perfect, high-fidelity city.

The beauty is that it uses the same brain for all three scenarios. You don't need different robots for different budgets; one robot adapts to whatever information you give it.

4. Why This Matters (The Results)

The authors tested this system and found two main things:

Better Pictures: When they asked the AI to generate satellite images based on these shapes, the results were much more realistic and structurally correct than previous methods. The roads actually connected, and buildings didn't float in the sky.
Better Training Data: They used the AI to create fake satellite images to help train other AI systems (like those that detect cars or map land use). Because TerraDiT-Ω's fake images were so accurate, the other AI systems learned faster and performed better on real-world tasks.

Summary

In short, TerraDiT-Ω is a new way to generate satellite images that respects the precise geometry of the real world. Instead of forcing map data into a blurry, pixelated mold, it takes the raw shapes (lines, boxes, dots) and uses a special "geometry-sensing" mechanism to build a realistic image that matches the blueprint perfectly, no matter how much or how little detail you provide.

Technical Summary: TerraDiT-Ω

Problem Statement

While generative models have advanced significantly in natural image synthesis, applying them to satellite imagery remains challenging due to the unique structural complexity of remote sensing (RS) scenes. Unlike natural images, satellite scenes are defined by dense, interconnected arrangements of small, semantically distinct geometries. Existing approaches attempt to bridge this gap by adapting natural image frameworks using either dense rasters (e.g., segmentation masks) or sparse prompts (e.g., point annotations). However, these methods suffer from two fundamental limitations:

Geometric Degradation: Converting native geospatial primitives (vectors) into pixel-based rasters flattens complex spatial relationships, such as overlapping overpasses or intersecting land-use zones, leading to a loss of structural fidelity.
Workflow Misalignment: Dense semantic maps require prohibitive annotation efforts (averaging 2.5 hours per tile), while sparse signals like point prompts underspecify crucial geometric layouts. This forces practitioners to deploy separate architectures for different annotation budgets, hindering scalable, end-to-end GeoAI workflows.

Current models also struggle to handle the varying degrees of structural detail across different input formats, often relying on generic learnable tokens that fail to capture the specific geometric priors of rotated or elongated objects common in RS.

Methodology

The authors propose TerraDiT-Ω, a unified spatial control framework that generates satellite imagery directly from native geospatial primitives (polygons, polylines, bounding boxes, and points) without requiring format conversion. The system is built upon a Flow-based Latent Diffusion Transformer (DiT) backbone.

Core Components

Unified Primitive Encoder:
- This module maps diverse geospatial formats into a shared representation.
- It processes polygons, polylines, bounding boxes, and points using Fourier feature mappings, concatenated with instance captions encoded by a frozen LongCLIP model.
- Missing primitives are represented by learned null embeddings, allowing the model to handle mixed or partial annotation budgets.
Geometry-Aware Local Attention (GALA):
- To address the limitations of implicit spatial learning, the authors introduce GALA, a conditioning mechanism that injects explicit geometric cues into the attention space.
- MetaRBF+: For each instance token, this module predicts the parameters of a rotated anisotropic Gaussian prior $(\sigma_x, \sigma_y, \theta)$ . Unlike previous axis-aligned priors (e.g., in TerraDiT-Σ), this allows the attention mechanism to adapt to oriented and elongated geometries.
- Spatial Geometry Field (SGF) Modulation: For richer geometric conditions (polygons, polylines, boxes), the model computes a dense spatial field.
  - For polygons and boxes, a Signed Distance Field (SDF) with sigmoid falloff is used to create sharp boundaries.
  - For polylines, a distance-based contrast function expands the line support to capture visual thickness (e.g., for rivers).
- The final attention prior is a normalized product of the Gaussian kernel and the spatial geometry field, multiplicatively modulating the attention logits between image and primitive tokens. This ensures that points induce soft local focus, while polygons and polylines enforce strict geometric adherence.
Training and Data:
- The model is trained on the Git-10M dataset, utilizing OpenStreetMap (OSM) vector data for primitives and global text/geolocation for context.
- Training involves a continuous-time diffusion framework with velocity prediction, representation alignment (REPA) using a satellite-specific DINOv3 encoder, and adaptive dropout strategies to ensure robustness across varying annotation densities.

Key Contributions

Unified Framework: TerraDiT-Ω is the first generative framework to directly operate on any native geospatial primitive, enabling state-of-the-art controllable synthesis across a spectrum of annotation budgets from sparse points to precise polygons.
Geometry-Aware Local Attention (GALA): The proposed conditioning mechanism explicitly preserves geometric cues in vector primitives, significantly improving fidelity in densely structured scenes by modeling rotation and spatial constraints.
Versatile Synthetic Data Augmentation: The single unified architecture supports synthetic data generation for multiple downstream tasks (segmentation, detection, road extraction, classification) without requiring task-specific pipelines or rasterization.

Experimental Results

The authors evaluated TerraDiT-Ω on three held-out datasets: Git-Rand-15k (random global sampling), Git-Spatial-15k (spatial holdout), and Git-Dense-3.5k (high-density scenes).

Synthesis Quality: TerraDiT-Ω consistently outperforms general image models (e.g., SDXL, PixArt) and prior geospatial baselines (e.g., GeoSynth-OSM, VectorSynth, TerraDiT-Σ) across FID, sFID, and LPIPS metrics. Notably, performance scales with annotation complexity; the model achieves the best results when conditioned on the full set of primitives ( $\Omega$ ).
Spatial Fidelity: Quantitative metrics (SSIM) and qualitative visualizations demonstrate that TerraDiT-Ω maintains correct relational connectivity (e.g., overpasses, road networks) where raster-based baselines fail. The model effectively reduces layout hallucinations by providing exact geometric details.
Downstream Utility: Synthetic data generated by TerraDiT-Ω improves downstream performance on four key RS tasks:
- Object Detection (DIOR): Improved mAP@50-95 and mAP@50.
- Land-Cover Segmentation (OpenEarthMap): Increased mIoU and F1 scores.
- Road Graph Extraction (City-Scale): Enhanced topological connectivity (TOPO) and local geometry (APLS).
- Scene Classification (AID): Significant accuracy gains, particularly in low-data regimes.
Ablation Studies: Removing rotation ( $\theta$ ) or SGF modulation degrades performance, confirming the necessity of explicit geometric modeling. GALA outperforms other attention mechanisms (GSA, IMA, ALA) in adhering to complex geometries.

Significance and Claims

The paper claims that TerraDiT-Ω represents a significant step toward robust, end-to-end GeoAI workflows. By eliminating the need for format conversion between vector primitives and raster masks, the framework bridges the gap between expensive dense annotations and imprecise sparse prompts.

The authors emphasize that their approach is naturally compatible with existing geospatial data sources (like OSM) and supports controllable layouts across varying annotation budgets. This flexibility allows for the generation of high-fidelity synthetic data that directly enhances downstream model performance without the computational overhead of precomputing dense masks or the geometric loss of rasterization. The work positions itself as a versatile, unified solution for satellite image synthesis that aligns with the structural realities of remote sensing.