Repairing Shape-Prior Shortcuts in Long-Range Single-Shot Fringe Projection Profilometry

Technical Summary: Repairing Shape-Prior Shortcuts in Long-Range Single-Shot FPP

Problem Statement
Single-shot fringe projection profilometry (FPP) aims to achieve dense, non-contact 3D reconstruction from a single fringe image, enabling operation under non-stationary conditions (moving objects, variable lighting) where conventional multi-step phase-shifting FPP fails. However, in the long-range regime (standoff > 1 m), single-shot reconstruction is inherently ill-posed. The mapping from a single fringe intensity image to depth is non-injective without fringe-order information because the intensity depends on a $2\pi$-periodic cosine function. Consequently, an incorrect fringe order induces depth errors that grow quadratically with distance ($Z^2$).

Previous work (Haroon et al., 2026b) established that standard deep learning baselines (specifically depth-regressing UNets) fail to solve this problem on long-range benchmarks. Instead of decoding the fringe phase to recover depth, these networks exploit a "shape-prior shortcut." They detect object boundaries and fill in learned shape templates, bypassing the fringe physics entirely. This failure is a property of the hypothesis space: because the mapping is non-injective, optimizers trained with standard $L_1/L_2$ regression collapse onto the shape-prior surrogate. Adding more data or model capacity does not resolve this, as the shortcut remains available within the search space.

Methodology: PhiCalNet
To repair this failure, the authors introduce PhiCalNet, a phase-intermediate architecture designed to remove the shape-prior solution from the hypothesis space by construction, rather than merely discouraging it via loss penalties.

1. Architectural Framing: Unlike standard networks that regress depth directly ($I \to Z$), PhiCalNet outputs a wrapped-phase representation $(\sin \phi, \cos \phi)$. The network never produces depth as a direct learned quantity.
2. Fixed Differentiable Calibration: The output phase is mapped to depth through a fixed, non-learnable, differentiable calibration layer ($\mathcal{C}$) that applies the deterministic FPP triangulation physics. This ensures that the only path to accurate depth is for the network to accurately predict the wrapped phase.
3. Auxiliary Fringe Order: To resolve the non-injectivity of single-shot reconstruction, PhiCalNet takes the fringe order ($k$) as declared auxiliary side information (e.g., from gray-code or multi-frequency decoding). The network learns the mapping $I \to (\sin \phi, \cos \phi)$, while the unwrapping ($\Phi = \phi + 2\pi k$) and triangulation are handled by the fixed pipeline.
4. Loss Function: The training employs a composite loss:
   • $\mathcal{L}_{circ}$: A geodesic phase term on the unit circle, weighted to up-weight errors near the $\pm \pi$ wrap boundary.
   • $\mathcal{L}_{\nabla}$: A smoothness regularizer on the phase gradients.
   • $\mathcal{L}_{d}$: A Hybrid $L_1$ loss on the final depth (computed via the calibration layer), which activates a gradient path back through the fixed calibration to the backbone.

Key Contributions

1. Architectural Repair over Loss Penalties: The paper demonstrates that enforcing physics via a soft loss penalty (a Physics-Informed Neural Network or PINN baseline) yields no improvement over the standard UNet. The improvement comes strictly from the architectural choice of routing reconstruction through a fixed calibration layer, which forces the network to represent phase.
2. Mechanistic Interpretability: The authors apply a battery of interpretability probes (linear probing, dual-mode Grad-CAM, and flat-plane out-of-distribution testing) to verify that PhiCalNet learns the intended physics.
   • Linear Probing: Phase is the most decodable internal feature, whereas depth is the least decodable (unlike the UNet baseline where depth-as-shape is consolidated in the decoder).
   • Grad-CAM: In "phase mode," the network attends to fringe patterns; in "depth mode," attention shifts to object boundaries only after gradients flow through the calibration layer.
   • Flat-Plane Test: PhiCalNet correctly recovers a uniform depth on a featureless plane (where shape priors fail), whereas the UNet baseline collapses to near-zero depth.
3. Uncertainty Quantification (UQ): The paper applies pixel-wise conformal prediction to FPP for the first time. It shows that the model's uncertainty (snapshot ensemble disagreement) is concentrated precisely at the $\pm \pi$ wrap discontinuities, the same locus identified by the residual error analysis.
4. Methodological Template: The convergence of mechanistic interpretability (MI), residual error analysis, and uncertainty quantification (UQ) at a single geometric locus is proposed as a "diagnose-repair-verify" template for physics-informed learning.

Results
Evaluated on the FPP-ML-Bench (15,600 fringe images, 50 objects at 1.5–2.1 m standoff):

• Accuracy: PhiCalNet reduces the object Mean Absolute Error (MAE) by 3.3$\times$ compared to the best UNet baseline (4.46 mm vs. 14.54 mm).
• Residual Analysis: The remaining error is highly localized. The entire Root Mean Square Error (RMSE) tail is carried by 0.103% of object pixels located at the $\pm \pi$ wrap discontinuity. This is identified as a structural limit of continuous regression on a discontinuous target, not a fitting failure.
• Multi-Frame Extension: Extending the input to three phase-shifted frames reduces the object MAE to 1.16 mm, confirming that the single-shot residual is information-theoretic.
• Fringe-Order Sensitivity: The system remains within 5% of its oracle accuracy even with up to 5% pixel-wise fringe-order corruption, indicating the oracle assumption is a formalism rather than a strict deployment requirement.
• UQ Efficacy: Rejecting the top 5% of pixels with the highest snapshot disagreement reduces PhiCalNet's RMSE by 64% (20.6 mm $\to$ 7.4 mm), whereas the same procedure on the UNet baseline yields only a 3.5% reduction.

Significance and Claims
The paper claims that the failure of single-shot FPP networks is not a capacity issue but a representational one, where optimizers exploit shape priors to bypass ill-posed physics. The significance of PhiCalNet lies in its architectural framing, which removes the shape-prior shortcut by making phase the primary output and depth a deterministic consequence of fixed physics.

The authors modestly frame their contributions as:

1. A specific architectural solution (PhiCalNet) that outperforms loss-level physics enforcement.
2. An empirical and analytical characterization of the residual error as a wrap-boundary artifact.
3. A verification workflow where MI and UQ converge on the same failure locus, offering a template for diagnosing and repairing physics-informed learning systems where failure modes have specific geometric loci.

The paper does not claim to solve the sim-to-real transfer problem or to eliminate the need for fringe-order information in single-shot setups; rather, it isolates the architectural cause of the shortcut learning and demonstrates that fixing the architecture yields a structurally legible failure mode that can be effectively managed via uncertainty quantification.