Technical Summary: DepthPrior

Problem Statement

Object detectors struggle to reliably detect small and distant objects due to scale variation, low resolution, and background clutter. While depth information holds the potential to improve detection, existing approaches typically require complex, model-specific architectural modifications (e.g., multi-branch networks, specialized convolutions, or feature fusion) that hinder practical deployment. Furthermore, standard supervised training treats objects uniformly regardless of distance, ignoring the systematic relationship between object distance and detection difficulty. This leads to a training bias where models prioritize nearby objects (which have low loss variance) and systematically underfit distant objects (which exhibit high loss variance). Additionally, standard inference post-processing applies uniform confidence thresholds across all depths, discarding valid detections of distant objects that receive systematically lower confidence scores.

Methodology

The paper introduces DepthPrior, a framework that utilizes depth as prior knowledge to inform training and inference without modifying detector architectures. The framework consists of three components:

1. Theoretical Foundation

The authors formalize the depth-detection relationship, proving that detection loss variance scales with depth (heteroscedasticity). Under the inverse-square law of light intensity, signal quality degrades with distance, increasing aleatoric uncertainty. Standard Stochastic Gradient Descent (SGD) with uniform weighting naturally fits low-variance (nearby) samples first, leading to the systematic underfitting of distant objects.

2. Training Components

To counteract this bias, DepthPrior employs two loss-level interventions:

• Depth-Based Loss Weighting (DLW): Implements variance-compensating weights based on the theoretical derivation. It applies an exponential weighting function to each object's loss based on its normalized depth. This scales contributions across the batch (inter-image relationship), prioritizing distant objects without requiring architectural changes.
• Depth-Based Loss Stratification (DLS): Decomposes the detection loss per image into "close" and "distant" intervals using a binary depth mask. These intervals are assigned separate weights (e.g., higher weight for distant objects), allowing explicit control over the learning dynamics for specific depth ranges (intra-image relationship).

3. Inference Component

• Depth-Aware Confidence Thresholding (DCT): Addresses the systematic confidence degradation of distant objects during inference. Instead of a uniform threshold, DCT learns depth-dependent confidence thresholds using cubic B-splines. These thresholds are optimized on a validation set to maximize true positives while constraining false positives, effectively recovering valid detections that would otherwise be discarded by standard post-processing.

Key Contributions

1. Theoretical Formalization: The paper provides a theoretical proof that depth induces heteroscedasticity in detection loss, creating a systematic training bias toward nearby objects that standard uniform supervision fails to address.
2. Modular Framework: DepthPrior offers a plug-and-play solution that exploits depth priors without architectural modifications, maintaining independence from the specific detector choice (anchor-based or anchor-free) and deployment domain.
3. Empirical Validation: The framework is validated across four diverse benchmarks (KITTI, MS COCO, VisDrone, SUN RGB-D) and two detector families (EfficientDet and YOLOv11), demonstrating effectiveness in both training and inference stages.

Experimental Results

Experiments demonstrate that DepthPrior significantly improves the detection of small and distant objects:

• Performance Gains: On the KITTI dataset, DepthPrior achieved up to +9% mAPS (mean Average Precision for small objects) and +7% mARS (mean Average Recall for small objects) compared to baselines.
• Inference Recovery: The Depth-Aware Confidence Thresholding (DCT) component achieved high recovery rates, with true detections recovered at a ratio of 95:1 against false detections in specific configurations. Across all datasets and architectures, DCT recovered 6,041 additional true detections with only 184 additional false detections on validation data.
• Generalization: The learned depth-threshold relationships transferred effectively to held-out inference data (90% split), recovering an additional 22,042 true detections with a favorable 9:1 ratio of true to false recoveries.
• Comparison to Baselines: Architectural integration methods (e.g., feature fusion, multi-task learning) performed poorly or failed to converge, whereas DepthPrior's loss-level and post-processing interventions consistently outperformed the baseline and simple depth-proportional weighting strategies.

Significance

The paper claims that DepthPrior offers a significant advancement by treating depth as a guiding prior rather than a fused feature. This approach avoids the modality imbalance and computational overhead associated with multi-modal architectures. By addressing the root cause of performance degradation—depth-induced heteroscedasticity in training and confidence miscalibration in inference—the framework improves safety-critical detection of distant objects without requiring additional sensors, architectural changes, or performance costs. The work establishes that geometric priors can be effectively leveraged through supervision-level adjustments and adaptive post-processing to enhance the robustness of standard object detectors.