Last-Layer-Centric Feature Recombination: Unleashing 3D Geometric Knowledge in DINOv3 for Monocular Depth Estimation

1. Problem Statement

Monocular Depth Estimation (MDE) is the task of predicting pixel-wise scene depth from a single RGB image. It is an inherently ill-posed problem because a single 2D image can correspond to infinite 3D scenes.

• Current Limitations: While Vision Foundation Models (VFMs), particularly DINO-based Transformers, have significantly improved MDE accuracy, existing approaches follow a unified paradigm: they uniformly sample a fixed set of intermediate transformer layers to build multi-scale features.
• The Gap: This uniform sampling implicitly assumes that geometric information is distributed evenly across all layers. The authors argue this underutilizes the specific structural 3D cues encoded in VFMs, particularly the rich geometric knowledge found in deeper layers.

2. Methodology: Last-Layer-Centric Feature Recombination (LFR)

The proposed solution is a plug-and-play module inserted between the ViT backbone and the prediction head (DPT decoder). It consists of three main stages:

A. Systematic Layer-wise Analysis

Before designing the method, the authors conducted a statistical analysis of DINOv3-L features:

• Findings: Deeper layers exhibit stronger depth predictability and better capture inter-sample geometric variations. The final layer acts as a "geometric anchor," while intermediate layers contain complementary information.
• Conclusion: Deeper features should play a central role, and intermediate features should be selected based on their complementarity to the last layer, not just their depth index.

B. Minimal-Similarity-Based Feature Selection

Instead of fixed uniform sampling, LFR adaptively selects intermediate layers:

1. Anchor: The features of the last layer ($F_L$) serve as the dominant geometric representation.
2. Selection Criterion: The algorithm computes the average cosine similarity between the last layer's class token and the image tokens of all preceding layers.
3. Strategy: It selects the $K$ layers with the lowest similarity scores (Minimal-Similarity).
   • Rationale: Layers with low similarity to the last layer are semantically most complementary, maximizing information gain and minimizing redundancy.

C. Feature Recombination via Lightweight Adapters

Selected auxiliary features are fused with the last-layer features:

• Concatenation: Auxiliary features ($F_{ak}$) and dominant features ($F_L$) are concatenated along the channel dimension.
• Adapter: A compact linear adapter (Down-projection $\to$ Non-linear Activation $\to$ Up-projection) processes the concatenated features.
• Gating: A learnable scalar gate ($g^k$), modulated by a tanh function, controls the contribution of the fused features.
• Residual Connection: The final recomposed feature is calculated as:
  $$F_{rk} = \tanh(g^k) \cdot A^k([F_{ak}; F_L]) + F_L$$

D. Prediction Head & Loss

• Multi-level Aggregation: Unlike standard DPT which uses only the final layer for prediction, LFR performs depth regression independently for each recomposed feature level. The final depth map is a weighted sum of these predictions, where weights are derived from the recomposed class tokens.
• Loss Functions: The model is trained using a combination of Scale-Invariant Log Loss and a Hierarchical Normalization (HN) Loss to balance global coherence and local detail.

3. Key Contributions

1. Systematic Analysis: The first layer-wise analysis of DINOv3 for 3D geometry, demonstrating that deeper layers encode denser, more explicit geometric information and that intermediate layers offer complementary cues.
2. LFR Module: A simple, effective, and plug-and-play recombination module that adaptively selects complementary intermediate layers based on a minimal-similarity criterion and fuses them with the last-layer anchor.
3. State-of-the-Art Performance: The method achieves new SOTA results on standard benchmarks with minimal computational overhead and no modification to the backbone architecture.

4. Experimental Results

The method was evaluated on NYU Depth v2 (indoor) and KITTI (outdoor), with zero-shot generalization tested on SUN RGB-D and Argoverse.

• NYU Depth v2:
  • DINOv3-L + LFR achieved an AbsRel of 0.057 and RMSE of 0.206, outperforming the baseline DINOv3-L (0.060 AbsRel) and prior SOTA methods like Depth Anything v2 (0.056 AbsRel) and generative models like DAR-L (0.056 AbsRel).
  • It achieved superior performance with significantly fewer parameters and no iterative inference (unlike diffusion-based models).
• KITTI:
  • DINOv3-L + LFR achieved an AbsRel of 0.043 and RMSE of 1.711, surpassing all previous methods, including those requiring massive pre-training or generative frameworks.
• Zero-Shot Generalization:
  • Models trained on NYU transferred to SUN RGB-D with an AbsRel of 0.099 (vs. 0.101 for baseline).
  • Models trained on KITTI transferred to Argoverse with an AbsRel of 0.204 (vs. 0.219 for baseline), demonstrating robust adaptation to unseen environments.
• Ablation Studies: Confirmed that the Minimal-Similarity selection strategy outperforms maximal similarity or random selection. The recombination module and HN loss provided consistent incremental gains.

5. Significance

• Unlocking VFM Potential: The paper provides a principled strategy to unlock the latent 3D geometric knowledge in foundation models without retraining the backbone.
• Efficiency: It offers a highly efficient alternative to computationally expensive generative models (diffusion/autoregressive) for dense 3D tasks, achieving comparable or better accuracy with a single forward pass.
• Theoretical Insight: It challenges the standard "uniform sampling" paradigm in transfer learning, suggesting that layer-wise heterogeneity in VFMs should be exploited through adaptive, complementarity-based feature fusion.
• Generalizability: The "Last-Layer-Centric" approach is architecture-agnostic and can likely be extended to other dense prediction tasks beyond depth estimation.