One Scene, Two Depths: Probing Geometric Ambiguity in Monocular Foundation Models

Problem Statement

Monocular depth estimation (MDE) foundation models typically operate under a "single-depth" paradigm, mapping a single RGB image to one scalar depth value per pixel. This assumption holds for opaque scenes but fails in layered 3D scenes (e.g., transparent glass), where a single camera ray intersects multiple geometrically valid surfaces (a foreground glass pane and a background object).

The paper argues that current supervised training collapses this multi-layer geometry into a single "dataset-shaped scalar target." The resulting depth label is not an intrinsic truth of the scene but a convention determined by annotation choices, sensor limitations (e.g., LiDAR vs. ultrasound), and training data biases. Consequently, frozen foundation models do not learn "layer-free" geometry; instead, they learn a depth-layer preference—a bias toward reporting either the proximal (foreground) or distal (background) layer when ambiguity exists. Standard evaluation metrics, which reward agreement with a single ground truth, penalize valid alternative interpretations, obscuring the model's true geometric reasoning capabilities.

Methodology

1. MultiDepth-3k (MD-3k) Benchmark
To measure this preference without requiring dense, sensor-neutral metric ground truth (which is difficult to obtain for transparent layers), the authors introduce MD-3k, a sparse ordinal benchmark.

• Data: 3,161 RGB images sourced from the GDD dataset, containing transparent objects.
• Annotation: Instead of dense depth maps, each image is annotated with one sparse pair of points in an ambiguous region. Crucially, the annotation provides two ordinal labels: the relative depth order for the transparent foreground layer and the visible background layer.
• Subsets: The dataset is split into a "Same" subset (where foreground and background depth orders agree) and a "Reverse" subset (where they conflict). The "Reverse" subset is the critical test case where a single-output model cannot satisfy both layers simultaneously.

2. Laplacian Visual Prompting (LVP)
The authors propose a training-free method to probe whether a frozen model's depth-layer preference can be modulated by changing the input representation.

• Mechanism: LVP applies a discrete Laplacian kernel (a high-frequency spectral transform) to the input RGB image. The resulting signed residuals are normalized and mapped back to the model's input range (e.g., 0–255).
• Hypothesis: By emphasizing high-frequency edges, LVP acts as a spectral probe that may steer the frozen model toward a different valid depth hypothesis (e.g., switching from foreground to background preference) without retraining or modifying weights.

3. Evaluation Metrics

• Spatial Relationship Accuracy (SRA): Measures the fraction of point pairs where the predicted depth ordering matches the ground truth for a specific layer (foreground or background).
• Depth-Layer Preference ($\alpha$): Defined as $SRA(2) - SRA(1)$. A positive $\alpha$ indicates a background preference; negative indicates foreground.
• Multi-Layer Spatial Relationship Accuracy (ML-SRA): Evaluates a pair of hypotheses (e.g., RGB output and LVP output). It checks if the pair, when optimally assigned to the two layers, satisfies the ordinal constraints for both layers simultaneously.

Key Contributions

1. Characterization of Depth-Layer Preference: The paper provides the first systematic characterization of how diverse monocular depth foundation models (including Depth Anything v1/v2, DPT, Marigold, etc.) exhibit distinct, model-specific biases toward foreground or background layers under standard RGB input.
2. MD-3k Benchmark: Introduction of a real-world benchmark that makes depth-layer preference measurable via sparse ordinal relations, bypassing the need for impossible dense multi-layer metric ground truth.
3. Input-Dependent Modulation: Demonstration that Laplacian Visual Prompting (LVP) can significantly alter the expressed depth-layer preference of frozen models. For certain models, LVP acts as a "switch" that reveals a complementary geometric hypothesis, allowing a single-output model to effectively cover multiple valid interpretations when queried with different inputs.

Results

• Diverse Biases: Under standard RGB input, models show strong but inconsistent biases. For instance, indoor-tuned Depth Anything v2 variants tend to prefer the foreground (glass), while outdoor-tuned variants and generative models (like Marigold) often prefer the background.
• LVP Effectiveness: LVP induces substantial shifts in preference for specific models (e.g., DPT, Depth Pro, and DAv2-L) but has little effect on others (e.g., DAv1, some generative models).
  • DAv2-L Case: The strongest example is DAv2-L. Under RGB, it is foreground-biased ($\alpha \approx -30.6$). Under LVP, it shifts to background-biased ($\alpha \approx +68.2$).
• ML-SRA Performance:
  • A "collapsed" baseline (duplicating a single depth map) achieves a strict ceiling of 56.4% on the Reverse subset of MD-3k because it cannot satisfy conflicting ordinal constraints.
  • The RGB/LVP pair using DAv2-L achieves 75.5% overall ML-SRA and 52.2% on the Reverse subset. This represents a significant gain over the single-hypothesis ceiling, proving that the frozen model can express complementary ordinal relations when the input representation changes.
• Ablation: The effect is specific to high-frequency prompting; low-pass Gaussian prompts fail to modulate the layer preference. The effect is not unique to the Laplacian operator but generalizes to other high-frequency transforms (Sobel, Fourier), though LVP remains a robust default.

Significance and Claims

The paper claims that standard RGB inference captures only a single slice of a richer geometric posterior. By treating ambiguity not as noise but as a measurable geometric structure, the authors demonstrate that:

1. Ambiguity is Intrinsic: Depth foundation models encode multiple valid geometric hypotheses, but standard training and evaluation force them to collapse these into a single, biased convention.
2. Modulation is Possible: Simple, training-free spectral input transformations (LVP) can unlock these latent, complementary hypotheses in frozen models.
3. Rethinking Evaluation: Future depth systems should be evaluated on their ability to represent and preserve multiple valid 3D interpretations rather than forcing a single scalar output.

Limitations Acknowledged:

• LVP is model-dependent and not a universal layer extractor; it fails on semi-transparent surfaces where frequency cues are entangled.
• The benchmark is sparse and focused on transparency; it does not provide dense metric validation.
• LVP often degrades standard single-depth accuracy on non-ambiguous scenes, indicating it is a diagnostic/probing tool rather than a replacement for standard inference in clear scenes.

The authors invite the community to adopt an "ambiguity-aware" lens for depth supervision and evaluation, where multiple valid 3D interpretations are treated as geometric structure to be measured and expressed.