💻 computer science

GeoDiffusion:Enhancing Self-Supervised Monocular Depth Estimation in Dynamic Scenes

The GeoDiffusion framework enhances self-supervised monocular depth estimation in dynamic scenes by leveraging a diffusion model trained on refined pseudo-ground-truth labels to overcome photometric inconsistencies caused by motion and occlusion, achieving high accuracy on benchmarks like Cityscapes and KITTI without requiring ground-truth depth during inference.

Original authors: Fang Dai, Chengzhi Lyu, Hong He, Pan Wang, Lei Zhang

Published 2026-10-01

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Fang Dai, Chengzhi Lyu, Hong He, Pan Wang, Lei Zhang

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

To navigate the world, a vehicle must understand not just what is in front of it, but how far away those things are. For decades, engineers have relied on expensive laser scanners or pairs of cameras to build this three-dimensional map, much like human eyes work together to judge distance. However, these tools are heavy, costly, and difficult to calibrate. A more flexible alternative exists: using a single standard camera, the kind found on most smartphones, to guess the depth of a scene. This is known as monocular depth estimation. While computers are quite good at this when looking at a still, quiet room, they struggle immensely when the world is moving. In a busy street, cars drive by, pedestrians cross the road, and the camera itself is in motion. These changes break the mathematical rules that computers usually use to figure out distance, causing the system to see ghosts or miss obstacles entirely.

A team of researchers from the Hubei University of Automotive Technology has developed a new approach to solve this specific problem of motion. They created a system called GeoDiffusion, which teaches a computer to estimate depth in dynamic, moving scenes without needing a human to provide the correct answers. Instead of relying on pre-labeled data, which is expensive and hard to gather, the system learns by watching video and correcting its own mistakes. The core idea involves a two-step process where a "teacher" model first makes a rough guess at the depth of a scene. This guess is then cleaned up by a "student" model that uses a technique called diffusion. In simple terms, diffusion is a method where the computer starts with a noisy, blurry version of the depth map and gradually removes the noise, step by step, to reveal a clear and accurate picture. This process is guided by the teacher's initial guess and the visual details of the scene itself.

The researchers tested their method on two major datasets used by the autonomous driving community: Cityscapes, which features complex urban traffic, and KITTI, a collection of driving footage from Germany. In these tests, the GeoDiffusion system proved to be more accurate than previous methods that did not use this diffusion technique. On the Cityscapes dataset, it reduced the error in depth estimation to a level of 0.105, beating the next best method which sat at 0.115. On the KITTI dataset, the improvement was even more pronounced, with the new system achieving an error rate of 0.090 compared to 0.096 for the leading alternative. These numbers represent the average difference between the computer's guess and the actual distance, meaning a lower number indicates a sharper, more reliable vision. The system also managed to keep the model small, containing about 7.6 million parameters, which is fewer than many competing models, allowing it to run quickly on standard hardware.

What makes this work particularly significant is how it handles the chaos of the real world. Traditional methods often assume the world is static, or they require the computer to identify moving objects separately using complex semantic labels. GeoDiffusion bypasses this need. It uses a consistency check to identify which parts of the image are moving or unreliable, such as a car speeding past or a reflection on a wet road, and ignores those areas when learning. It then focuses on the stable parts of the scene to refine its understanding of depth. By combining this selective focus with the step-by-step refinement of the diffusion process, the system learns to see through the confusion of motion blur and changing light. The result is a depth map that remains stable even when the camera is shaking or objects are rushing past.

The study confirms that this approach works well in practice, but the authors note that it is not without limitations. The system relies on the initial "teacher" model to generate its starting guesses, which means if the teacher is biased or makes a mistake in very difficult weather conditions, the student might learn from that error. The researchers suggest that future work will focus on making the system even more robust against extreme weather and reducing its dependence on these initial guesses. For now, however, the findings demonstrate a clear path forward. By teaching computers to clean up their own noisy observations, rather than just memorizing correct answers, we can build vehicles that see the road more clearly, even when the world around them is in constant motion.

Technical Summary: GeoDiffusion

Problem Statement

Monocular depth estimation is critical for autonomous driving tasks such as path planning and obstacle detection. While traditional methods like LiDAR and stereo cameras are accurate, they are often costly or require complex calibration. Self-supervised monocular depth estimation offers a cost-effective alternative by leveraging photometric consistency without ground-truth depth labels. However, existing self-supervised approaches predominantly assume static environments. In real-world dynamic scenes—characterized by moving objects, motion blur, occlusions, and rapid lighting changes—this assumption fails. These factors disrupt photometric consistency, leading to inaccurate pose estimation and degraded depth predictions. Previous attempts to address dynamics often rely on auxiliary tasks like semantic segmentation or motion segmentation, which either require costly annotations or precise trajectory modeling, limiting their practicality.

Methodology

The authors propose GeoDiffusion, a self-supervised framework that integrates diffusion models to enhance depth estimation in dynamic driving scenes. The architecture consists of a teacher-student paradigm and a conditional diffusion process:

Robust Pseudo-Depth Construction:
- A pre-trained Lite-Mono-8M model acts as a teacher to generate multi-scale pseudo-depth maps.
- A multiview consistency module filters these maps by reprojecting depth across frames. Pixels with high variance (indicating motion, occlusion, or specular reflections) are marked as inconsistent and excluded. This creates reliable pseudo-ground-truth labels for training without manual annotation.
Conditional Depth Diffusion (Student Network):
- The student network performs depth estimation as an iterative denoising task.
- Forward Diffusion: Noise is progressively added to the depth map.
- Reverse Diffusion: The student network predicts the depth map of a target frame by denoising in discrete stages ( $T$ $T$ ). This process is guided by:
  - Multiscale Image Features: Extracted by a Lite-Mono DepthNet encoder.
  - Discriminative Priors: The teacher's depth predictions ( $D_{teacher}$ ) serve as conditional inputs to guide the denoising trajectory, ensuring geometric consistency and structural fidelity.
- The student uses a compact noise predictor that balances multi-scale features via learnable attention weights.
Pose Estimation:
- A separate module, utilizing a pre-trained ResNet-18 encoder and a four-layer convolutional decoder, predicts the relative 6-DOF pose between consecutive frames.
Training Objective:
The model is trained using a weighted sum of three loss functions:
- Photometric Consistency Loss ( $L_{ph}$ ): Measures the difference between the synthesized target frame and the actual target frame using SSIM and $L_1$ loss.
- Knowledge Distillation Loss ( $L_{KD}$ ): Guides the student to align with the reliable pseudo-depth from the teacher, filtered by the consistency mask.
- DDIM Loss ( $L_{DDIM}$ ): Enforces coherence between the student's predictions and the intended diffusion trajectory to ensure stability.

During inference, the teacher model and auxiliary networks are discarded; only the lightweight student model is used.

Key Contributions

GeoDiffusion Framework: A novel self-supervised monocular depth framework that utilizes diffusion models to reduce errors specifically in dynamic scenes.
Robust Pseudo-Label Generation: A method for generating label-free supervision by combining a pre-trained teacher model with multiview consistency verification to filter out unreliable regions caused by motion and occlusion.
Discriminative Prior Injection: The introduction of a module that injects teacher-derived depth as a discriminative prior during the diffusion process, improving geometric and structural fidelity.
State-of-the-Art Performance: The method achieves superior accuracy and efficiency on standard benchmarks without relying on semantic or motion segmentation labels.

Experimental Results

The authors evaluated GeoDiffusion on the Cityscapes and KITTI datasets, comparing it against state-of-the-art methods including Monodepth2, RA-Depth, and DNA-Depth.

Cityscapes: GeoDiffusion achieved an AbsRel of 0.105, SqRel of 0.905, and RMSE of 5.750, outperforming previous self-supervised models (e.g., RA-Depth had an AbsRel of 0.111 and RMSE of 5.791). Qualitative results showed improved accuracy for dynamic objects.
KITTI: The method achieved an AbsRel of 0.090, SqRel of 0.609, and RMSE of 4.180, surpassing leading methods like RA-Depth (AbsRel 0.096, RMSE 4.216).
Efficiency: The model contains approximately 7.6 million parameters, which is more compact than RA-Depth (9.98M) and Monodepth2 (14.3M). Inference speed on an RTX 3090 is approximately 9.2 ms per frame, indicating suitability for real-time applications.
Ablation Studies: Experiments confirmed that the integration of Robust Pseudo-depth Construction (RPC), Discriminative Prior Enhancement (DPE), Knowledge Distillation ( $L_{KD}$ ), and Photometric Loss ( $L_{PH}$ ) systematically improves performance, with the full model yielding the best metrics.

Significance and Claims

The paper claims that GeoDiffusion successfully addresses the limitations of self-supervised depth estimation in dynamic environments by leveraging the generative and denoising capacities of diffusion models. By utilizing a teacher-student distillation approach with multiview consistency filtering, the framework achieves high accuracy without the need for expensive ground-truth depth labels or auxiliary semantic/motion annotations.

The authors position GeoDiffusion as a significant step forward for 3D perception in autonomous driving, offering a balance of high accuracy, structural robustness, and computational efficiency. However, the paper modestly acknowledges limitations: the reliance on pseudo-ground truths from a pre-trained teacher may introduce biases under challenging lighting or weather conditions, potentially affecting generalization. Future work is proposed to focus on reducing this reliance and improving robustness against dynamic objects.