💻 computer science

D2-CDIG: Controlled Diffusion Remote Sensing Image Generation with Dual Priors of DEM and Cloud-Fog

The paper proposes D2-CDIG, a novel remote sensing image generation framework that leverages a diffusion model with dual priors from Digital Elevation Models (DEM) and cloud-fog information to achieve precise, high-quality, and realistic control over both terrain and atmospheric phenomena.

Original authors: Zuopeng Zhao, Ying Liu, Kanyaphakphachsorn Pharksuwan, Su Luo, Xiaoyu Li, Maocai Ning

Published 2026-05-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zuopeng Zhao, Ying Liu, Kanyaphakphachsorn Pharksuwan, Su Luo, Xiaoyu Li, Maocai Ning

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are an artist trying to paint a realistic satellite photo of the Earth. You want to show a specific mountain range, but you also want to control exactly how much fog or cloud covers it.

The Problem:
Existing "AI artists" (previous computer programs) are good at painting, but they struggle with two things:

The Ground: They often get the shape of the mountains and valleys wrong because they don't look at a 3D map of the terrain.
The Sky: They treat clouds and fog like random decorations. They can't easily say, "Make the clouds thick here and thin there," or "Add a little bit of fog to the valley."

Because of this, the pictures they make often look fake, especially when the weather is complex or the terrain is rugged.

The Solution: D2-CDIG
The authors of this paper created a new tool called D2-CDIG. Think of it as a "Double-Controller" system for an AI painter. Instead of giving the AI one set of instructions, they give it two separate, specialized guides working together:

The "Ground Guide" (DEM): This guide holds a Digital Elevation Model (DEM). Imagine this as a 3D wireframe map of the mountains and valleys. The AI uses this to ensure the ground looks exactly right—no flat mountains or upside-down hills.
The "Sky Guide" (Cloud-Fog): This guide holds information about clouds and fog. It acts like a dimmer switch or a slider for the weather. You can slide it to say, "I want 10% cloud cover," or "I want heavy fog in the valleys."

How It Works (The Creative Analogy):
Imagine building a house.

Old Methods: You try to build the walls and paint the sky at the same time with one brush. If you get the sky wrong, you might accidentally mess up the roof.
D2-CDIG: It uses two separate construction crews.
- Crew A (Ground): They build the foundation and walls based strictly on the blueprints (the DEM). They don't worry about the weather.
- Crew B (Sky): They come in later and paint the sky and add clouds. They look at the finished house and decide exactly where the clouds should sit to look realistic.
- The Magic: These two crews talk to each other. The sky crew knows not to put a cloud floating in the middle of a mountain peak because the ground crew told them, "That's a cliff." This ensures the clouds look like they are actually interacting with the ground (like fog settling in a valley).

Key Features:

The Cloud Slider: The user gets a simple control knob. If you turn it to "50%," the AI generates an image with exactly 50% cloud coverage. It's not a guess; it's a precise calculation.
Layered Learning: The AI learns the ground details in the early stages of "thinking" and learns the cloud details in the later stages. This keeps the two things from getting confused.

What They Found:
The paper tested this new tool against other AI methods.

Better Pictures: The images looked more real, with sharper mountains and more natural-looking clouds.
More Control: Unlike other tools that just guessed the weather, D2-CDIG let the user decide exactly how cloudy the scene should be.
Useful Data: The images were so good that if you used them to train another AI (like one that identifies crops or buildings), that second AI performed almost as well as if it had been trained on real photos.

In Short:
D2-CDIG is a new way to generate satellite images that separates the job of drawing the land from the job of drawing the weather. By using a 3D map for the ground and a "cloud slider" for the sky, it creates highly realistic, controllable images that are much better than what was possible before.

Technical Summary: D2-CDIG

Problem Statement
Existing controllable remote sensing image generation methods primarily rely on traditional techniques such as land object segmentation and edge detection. While these approaches provide geographic data, they fail to fully leverage complex terrain structures or atmospheric conditions. Consequently, generated images often lack accuracy and naturalness when simulating complex terrains and atmospheric phenomena like clouds, fog, and climate changes. Current models struggle to capture the intricate interactions between surface features and atmospheric effects, leading to unrealistic outputs that fail to support high-precision downstream tasks or large model training.

Methodology
The paper proposes D2-CDIG (Controlled Diffusion Remote Sensing Image Generation with Dual Priors of DEM and Cloud-Fog), a framework that integrates diffusion models with a dual-prior control mechanism. Built upon the Stable Diffusion v1.5 architecture, the method introduces a dual-branch ControlNet design to decouple terrain and atmospheric generation:

Dual-Prior Knowledge: The system incorporates two distinct control signals:
- Digital Elevation Model (DEM): Serves as the ground prior to control terrain structure.
- Cloud-Fog Information: Serves as the atmospheric prior to control weather conditions.
Dual-Branch Architecture:
- Ground Branch: Utilizes a CNN-based encoder (ResNet-18) to process DEM data. It injects control signals into the lower and mid-level U-Net blocks (blocks 2, 4, 6, 8) to guide local terrain geometry, edges, and textures.
- Atmospheric Branch: Utilizes a Transformer-based encoder (ViT-Small) to process cloud-fog masks. It injects signals into the higher-level U-Net blocks (blocks 10, 12, 14, 16) to govern global composition, semantic structure, and cloud morphology.
Layered Injection Strategy: Control signals are injected at specific feature levels to ensure physical consistency. Ground features anchor the generation early in the process, while atmospheric features modulate the output later, allowing the model to consider global illumination and topography when rendering atmospheric effects.
Refined Cloud Control: A "cloud-density slider" ( $\delta \in [0, 1]$ ) is introduced, calibrated via a power-law function ( $C_{cov} = 0.95 \cdot \delta^{0.85} \times 100\%$ ) to map user input to precise physical cloud coverage percentages.
Training Objective: The model is trained using a joint loss function combining standard diffusion noise prediction loss with perceptual losses for ground consistency ( $L_{ground}$ ) and atmospheric similarity ( $L_{atmosphere}$ ), weighted by hyperparameters $\alpha=0.6$ and $\beta=0.4$ .

Key Contributions

Dual-Prior Controlled Generation: The introduction of a method that simultaneously considers surface features (via DEM) and atmospheric changes (via cloud-fog) using a dual-branch control network, filling the research gap in combining terrain and atmospheric control.
Decoupled Generation Process: The design of parallel Ground and Atmospheric branches that allow for independent yet coordinated control, ensuring natural transitions between terrain and weather conditions.
Precise Cloud Control Mechanism: The development of a cloud-density slider and meteorological parameter system that allows users to flexibly and precisely adjust cloud thickness, shape, and distribution during inference.
Layered Injection: A training strategy where ground and atmospheric signals are injected at different feature levels (low-level for terrain, high-level for atmosphere) to ensure seamless integration.

Experimental Results
The authors evaluated D2-CDIG against state-of-the-art baselines (SD v1.5, DiffusionSat, ControlNet, CRS-Diff, T2I-Adapter) across three tasks: Text-to-Image, DEM-to-Image, and Multi-Conditional Generation.

Quantitative Performance: D2-CDIG achieved the best results across all metrics. In DEM-guided generation, it attained an SSIM of 0.303, PSNR of 16.930 dB, and FID of 59.974, outperforming the second-best method (CRS-Diff) significantly.
Robustness: The model demonstrated superior stability across varying cloud coverage levels (5% to 70%). At 50% cloud coverage, D2-CDIG maintained an SSIM of 0.321 and FID of 63.175, whereas baseline methods showed significant degradation.
Downstream Utility: In land-cover segmentation tasks using a DeepLabV3+ model trained on augmented data, D2-CDIG-generated images yielded an mIoU of 0.678, nearly matching the performance of models trained on real data (0.683) and significantly outperforming other augmentation strategies.
Transferability: The framework demonstrated zero-shot transferability to Sentinel-2 (10m resolution) data, with performance improving substantially after fine-tuning.

Significance and Claims
The paper claims that D2-CDIG offers a flexible and precise solution for remote sensing image generation, addressing the limitations of traditional methods that overlook atmospheric phenomena. By decoupling terrain and atmospheric generation, the method produces images with higher detail richness, realism, and physical consistency. The authors state that D2-CDIG provides a high-quality data foundation for training large remote sensing models and downstream tasks. Furthermore, the method is designed to be sensor-agnostic, relying on universally available DEM and cloud-fog inputs rather than sensor-specific bands, making it applicable across diverse remote sensing platforms. The work aims to bridge the gap between data demand and supply, aiding scientists and policymakers in making informed decisions regarding the planet's changing surface and atmosphere.

Technical Summary: D2-CDIG

More like this