⚡ electrical engineering

Monocular Depth Estimation From the Perspective of Feature Restoration: A Diffusion Enhanced Depth Restoration Approach

This paper proposes a novel Monocular Depth Estimation approach that reframes the task as feature restoration using an Invertible Transform-enhanced Indirect Diffusion module to recover degraded encoder features and an Auxiliary Viewpoint-based Low-level Feature Enhancement module to refine local details, achieving state-of-the-art performance on multiple benchmarks.

Original authors: Huibin Bai, Shuai Li, Hanxiao Zhai, Yanbo Gao, Chong Lv, Yibo Wang, Haipeng Ping, Wei Hua, Xingyu Gao

Published 2026-04-10

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Huibin Bai, Shuai Li, Hanxiao Zhai, Yanbo Gao, Chong Lv, Yibo Wang, Haipeng Ping, Wei Hua, Xingyu Gao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Guessing Distance from a Single Photo

Imagine you are looking at a photograph of a street. It's a flat, 2D image. But your brain instantly knows that the car in the foreground is close, the trees in the middle are further away, and the mountains in the back are very far. This is Monocular Depth Estimation (MDE): teaching a computer to guess how far away things are just by looking at a single picture.

The problem is, this is a "trick question" for computers. A single photo doesn't actually contain distance information; it's an illusion. The computer has to guess, and often, it gets it wrong, especially for faraway objects or fine details.

The Problem: The "Blurry Blueprint"

Current computer vision systems work like a factory assembly line. They have an Encoder (the part that looks at the photo and breaks it down into pieces) and a Decoder (the part that puts the pieces back together to guess the distance).

The researchers found a flaw in this process:

The Encoder gets "tired": As the computer processes the image through many layers, the important "high-level" information (like "that's a car" or "that's a road") gets a bit degraded or "noisy," like a photocopy of a photocopy.
The Decoder gets confused: When the computer tries to guess the distance using these slightly blurry blueprints, the result isn't perfect.
The "Diffusion" Trap: Some new methods use a technique called Diffusion (think of it like a sculptor starting with a block of stone and chipping away noise to reveal a statue). However, previous attempts had a problem: they were trying to sculpt the statue based only on the final shape (the depth map) without checking if the stone itself (the internal features) was being shaped correctly. This caused the sculpture to wobble or drift off-course during the process.

The Solution: Restoring the Blueprint

The authors propose a new way of thinking: Don't just guess the distance; fix the blueprint first.

They treat the problem as Feature Restoration. Instead of asking, "What is the distance?", they ask, "How do we restore the original, high-quality 'mental image' of the scene so the distance guess becomes easy?"

Here is how their three main tools work:

1. The "Magic Mirror" (InvT-IndDiffusion)

This is the core invention. Imagine you are trying to clean a dirty window to see the view outside.

The Old Way: You wipe the window, look at the view, and if the view looks a bit blurry, you wipe again. But sometimes, you wipe in a way that makes the view look clear from the outside but smears the glass on the inside.
The New Way (InvT-IndDiffusion): The researchers built a "Magic Mirror" (an Invertible Decoder). This mirror has a special rule: if the view outside gets clearer, the glass inside must also get cleaner. It forces the computer to ensure that every step of the "cleaning" process (the diffusion steps) keeps the internal features aligned with the final goal. It prevents the computer from taking a "shortcut" that looks good temporarily but ruins the final result.

2. The "Sidekick Camera" (AV-LFE)

Sometimes, you have more than one photo of the same scene (like a car with a front camera and a side camera).

The Analogy: Imagine you are trying to judge how far a tree is. If you only have one photo, it's hard. But if you have a second photo from a slightly different angle, you can triangulate the distance much better.
The Tool: They created a module called AV-LFE (Auxiliary Viewpoint Low-level Feature Enhancement). It acts like a helpful sidekick. If a second camera angle is available, this module grabs the fine details (like the texture of the bark or the edge of a sign) from the side view and pastes them into the main view to sharpen the details. If there is no second camera, the module simply sits quietly and doesn't interfere.

The Results: Sharper, Deeper, Better

When they tested this new system (called IID-RDepth) on real-world driving datasets (like the KITTI dataset):

It sees further: It got much better at guessing the distance of faraway objects (like distant road signs or mountains), which is usually the hardest part.
It sees finer details: The edges of objects were sharper.
It's a huge improvement: Compared to the previous best methods, their system reduced errors by nearly 38% when using the "sidekick camera" feature. Even without it, it was still significantly better than the competition.

Summary

Think of this paper as a team of mechanics who realized that the car (the computer vision model) wasn't running well because the engine (the internal features) was getting dirty. Instead of just polishing the hood (the final output), they invented a new cleaning fluid (the InvT-IndDiffusion) that cleans the engine while ensuring the car drives straight, and they added a turbocharger (the AV-LFE) that uses extra air (side views) to make the engine run even smoother.

The result? A computer that can look at a flat photo and understand the 3D world with human-like precision.

1. Problem Statement

Monocular Depth Estimation (MDE) is an ill-posed problem where a single 2D image must be used to infer 3D depth. While current mainstream methods use encoder-decoder architectures (e.g., U-Net, Transformers) with multi-scale feature processing, they face several limitations:

Feature Degradation: The quality of encoder features significantly impacts prediction accuracy, but current architectures do not fully optimize these features.
Latent Feature Deviation in Diffusion: Recent attempts to use Diffusion Models for MDE often rely on indirect supervision (using the final sparse depth map as the loss signal) because ground truth depth is often sparse (from LiDAR) and not directly mappable to latent features. This causes a misalignment in the optimization trajectory: during iterative inference, the latent features may deviate from the "assumed ground truth feature" required to generate the correct depth, leading to suboptimal results.
Lack of Low-Level Detail: Standard diffusion processes often struggle to preserve fine-grained local details, especially when auxiliary information (like multi-view data) is available but underutilized.

2. Methodology

The authors propose IID-RDepth (Invertible transform enhanced Indirect Diffusion for Restored Depth Estimation), a framework that treats MDE as a feature restoration problem. The core idea is to restore degraded high-level encoder features into an "assumed ground truth feature" that yields the optimal depth map.

A. Core Framework: IID-RDepth

The architecture follows a U-Net structure with a frozen pre-trained encoder (Swin-Transformer) and a novel decoder based on diffusion.

Feature Restoration Perspective: Instead of generating depth directly from noise, the model treats the encoder's output as a degraded version of an ideal feature. The goal is to denoise and restore this feature.
InvT-IndDiffusion (Invertible Transform-enhanced Indirect Diffusion):
- Problem: In indirect diffusion, minimizing the depth loss at each step does not guarantee the latent feature moves closer to the true feature, causing deviation over iterations.
- Solution: The authors introduce a decoder function ( $f_m$ ) that satisfies the bi-Lipschitz condition.
- Mechanism: By constructing an invertible decoder (using Affine Coupling Layers), the distance between the predicted depth and ground truth depth is positively correlated with the distance between the latent feature and the assumed ground truth feature.
- Mathematical Guarantee: If the decoder is invertible and bi-Lipschitz continuous, minimizing the depth loss ( $||D_{pre} - D_{gt}||$ ) mathematically forces the latent feature optimization ( $||F_n - F_{gt}||$ ) to converge toward the correct direction, preventing feature deviation during iterative inference.
Diffusion Process:
- Forward: Noise is added to the high-level encoder features (treated as degradation).
- Reverse: A denoising network predicts the noise and degradation, iteratively restoring the features. Multi-scale features from the encoder act as mutual conditional maps to guide the restoration.

B. Auxiliary Viewpoint-based Low-level Feature Enhancement (AV-LFE)

To address the lack of local detail and utilize multi-view data (common in autonomous driving):

Module: A plug-and-play module that fuses features from a main viewpoint and an auxiliary viewpoint.
Operation: It uses deformable convolutions to align auxiliary features with the main view, then fuses them via convolution layers to enhance the "shortcut connections" in the U-Net decoder.
Flexibility: It operates in two modes:
- Compatible Mode: Only the AV-LFE is trained; the rest of the network is frozen. It works if auxiliary views exist and is neutral if they don't.
- Fully Trainable Mode: The entire network is trained for optimal performance.

3. Key Contributions

Feature Restoration Formulation: The paper reframes MDE as a feature restoration task, demonstrating through ablation that optimizing high-level features yields significantly greater performance gains than optimizing low-level features.
InvT-IndDiffusion: A novel diffusion module that solves the "feature deviation" problem in indirect supervision. By enforcing a bi-Lipschitz condition via an invertible decoder, it ensures that iterative optimization steps remain consistent with the target depth, even without direct feature supervision.
AV-LFE Module: A flexible, plug-and-play module that leverages auxiliary viewpoints to enhance low-level spatial details without compromising the base model's performance when such views are unavailable.
State-of-the-Art Performance: The method achieves superior results on standard benchmarks, particularly in long-range depth estimation and fine-grained detail preservation.

4. Experimental Results

The method was evaluated on the KITTI and DDAD datasets.

KITTI Benchmark:
- Compared to the baseline PixelFormer, IID-RDepth improved RMSE by 4.09% (standard setting).
- With the AV-LFE module in Fully Trainable Mode, RMSE improved by 37.77%.
- Statistical analysis (paired t-test) confirmed that the method outperforms baselines in 97.2% of images with $p < 0.05$ .
- Long-Range Performance: The method showed a 9.02% improvement in RMSE for distances between 50m and 80m, attributed to the high-level feature restoration.
DDAD Benchmark:
- The method was tested as a plug-and-play module on different backbones (NewCRFs and PixelFormer), showing consistent improvements (up to 16.03% RMSE reduction), proving its robustness across architectures.
Ablation Studies:
- Diffusion Steps: Using 6 inference steps yielded better results than 1 step, validating the iterative refinement capability.
- Decoder Type: The proposed Invertible Decoder outperformed both Transformer-based and Convolution-based decoders, validating the necessity of the bi-Lipschitz constraint for indirect diffusion.

5. Significance

This paper addresses a critical theoretical gap in applying diffusion models to dense prediction tasks with sparse supervision. By proving that invertible transformations can align the optimization trajectories of latent features and final task outputs, the authors provide a robust solution to the feature deviation problem. This approach not only sets a new state-of-the-art for Monocular Depth Estimation but also offers a generalizable framework for other inverse problems where direct supervision of intermediate latent variables is impossible. The ability to seamlessly integrate auxiliary multi-view data further enhances its practical utility in autonomous driving and 3D reconstruction scenarios.