Diagnosing and Repairing Shape-Prior Shortcuts in Long-Range Single-Shot Fringe Projection Profilometry

Technical Summary: Diagnosing and Repairing Shape-Prior Shortcuts in Long-Range Single-Shot Fringe Projection Profilometry

Problem Statement
Learning-based single-shot fringe projection profilometry (FPP) has achieved success in close-range regimes (e.g., endoscopic, tabletop) where fringe contrast and triangulation geometry are favorable. However, the long-range regime (standoff > 1 m) remains largely unaddressed due to three coupled challenges: (1) the inverse-square falloff of light intensity degrades signal-to-noise ratio and physical ground truth reliability; (2) the single-shot formulation is intrinsically ill-posed because fringe-order information is unavailable from a single image, leading to depth errors that scale quadratically with distance; and (3) existing architectures lack mechanistic interpretability, leaving error diagnosis without principled guidance. Furthermore, the field lacks a shared benchmark, with inconsistent normalization, loss functions, and evaluation protocols across studies.

Methodology
The authors present a unified "diagnose-repair-verify" study utilizing FPP-ML-Bench, the first open-source photorealistic synthetic benchmark for single-shot FPP. Built on the VIRTUS-FPP framework within NVIDIA Isaac Sim, the dataset comprises 15,600 fringe images of 50 diverse objects at 1.5–2.1 m standoff.

The methodology proceeds in four stages:

1. Systematic Ablation: The authors isolate design choices (normalization, background processing, multi-frame composition, loss functions, and architecture) to establish a best-performing baseline.
2. Mechanistic Interpretability (MI) Diagnosis: Using linear probing, Grad-CAM, and out-of-distribution (OOD) flat-plane testing, the authors investigate whether the baseline network learns physics-based fringe decoding or geometric shortcuts.
3. Architectural Repair: Based on the diagnosis, the authors introduce PhiCalNet, a phase-intermediate architecture. Unlike standard depth-regressing networks, PhiCalNet outputs a wrapped-phase representation $(\sin \phi, \cos \phi)$. A fixed, differentiable calibration layer then deterministically maps this phase to depth using FPP triangulation physics. This design removes the shape-prior solution from the hypothesis space by architectural constraint rather than loss penalty.
4. Verification: The repair is verified via:
   • MI: Re-applying the interpretability battery to PhiCalNet to confirm it learns phase rather than shape.
   • Uncertainty Quantification (UQ): Applying pixel-wise conformal prediction on a snapshot ensemble to determine if the model's self-reported uncertainty aligns with its failure modes.
   • Counterfactuals: Comparing PhiCalNet against a Physics-Informed Neural Network (PINN) baseline where physics is enforced via a soft loss penalty rather than architecture.

Key Findings and Results

• Baseline Optimization: Systematic ablations revealed that individual depth normalization (decoupling scale from shape) improved object Mean Absolute Error (MAE) by 9.1× over raw depth. Contrary to intuition, background fringes are essential signal, not noise; removing them degraded performance by 2.8–7.3×. Hybrid L1 loss ($\alpha=0.7$) and UNet architecture outperformed TransUNet, ResUNet, and Pix2Pix. The best baseline (UNet) achieved 14.54 mm object MAE, but residual analysis suggested a representational limit rather than a capacity issue.
• The Shortcut Diagnosis: Mechanistic analysis of the UNet baseline revealed it solves the task via object-boundary shape priors rather than fringe-phase decoding.
  • Linear Probing: Geometric features (edges) were 2.82× easier to decode from internal activations than depth values.
  • Grad-CAM: Attention correlated more strongly with object edges (1.28×) than fringe patterns.
  • OOD Test: On a featureless flat plane, the network collapsed to near-zero depth predictions, failing to utilize the valid fringe patterns present, confirming reliance on shape templates.
• PhiCalNet Performance: By forcing the network to output wrapped phase, PhiCalNet reduced object MAE by 3.3× (from 14.54 mm to 4.46 mm) compared to the UNet baseline.
  • A PINN counterfactual (enforcing physics via soft loss on a depth-regressing network) yielded no measurable gain, isolating the architectural framing as the operative factor.
  • The residual error of PhiCalNet was localized to 0.103% of pixels at the $\pm\pi$ wrap discontinuity, a structural limit of continuous regression of a discontinuous target.
  • A 3-frame extension of PhiCalNet further reduced MAE to 1.16 mm, validating the information-theoretic deficit of single-shot acquisition.
• Uncertainty Quantification: Pixel-wise conformal prediction demonstrated that PhiCalNet's errors are structurally legible. Rejecting the top 5% of pixels by snapshot disagreement reduced PhiCalNet's RMSE by 64% (20.6 mm $\to$ 7.4 mm), whereas the same procedure reduced the UNet baseline's RMSE by only 3.5%. This confirms that PhiCalNet's failures are concentrated in specific geometric loci where the model correctly identifies high uncertainty, unlike the diffuse errors of the baseline.

Significance and Claims
The paper claims three primary contributions:

1. Benchmarking: The introduction of FPP-ML-Bench provides a standardized, reproducible evaluation surface for long-range single-shot FPP, addressing the lack of shared infrastructure in the field.
2. Mechanistic Insight: This is the first application of mechanistic interpretability to FPP, revealing that standard deep learning approaches in this domain often rely on geometric shortcuts (shape priors) rather than physical phase decoding, a failure mode exacerbated by the ill-posed nature of single-shot long-range reconstruction.
3. Architectural Solution: The authors demonstrate that architectural constraints (outputting phase rather than depth) are more effective than loss penalties (PINNs) for enforcing physical consistency. PhiCalNet successfully removes the shape-prior shortcut from the hypothesis space.

The authors frame the convergence of Mechanistic Interpretability, Residual Error Analysis, and Uncertainty Quantification at a single physical locus (the $\pm\pi$ wrap discontinuity) as a methodological template for physics-informed learning. They argue that when a model's failure mode has a specific geometric or physical locus, these three diagnostic tools can converge to provide a coherent explanation of model behavior, a condition often absent in diffuse failure modes found in other domains like language or medical imaging.

The paper concludes that the residual error in single-shot FPP is fundamentally information-theoretic (due to the lack of fringe-order information) rather than a modeling limitation, and that while PhiCalNet significantly improves accuracy, the remaining error is structurally bounded by the physics of the problem.