Technical Summary: P2GS – Physical Prior-guided Gaussian Splatting for Photometrically Consistent Urban Reconstruction

1. Problem Statement

While 3D Gaussian Splatting (3DGS) has established itself as a powerful tool for real-time, high-fidelity rendering in autonomous driving simulation, it suffers from a fundamental limitation: photometric inconsistency. Conventional 3DGS implicitly assumes consistent exposure and tone mapping across all input views. However, real-world driving data violates this assumption due to:

• Heterogeneous camera pipelines and Image Signal Processors (ISPs).
• Per-camera dynamic exposure control.
• Varying outdoor illumination conditions.

When standard 3DGS is trained on such data, it "bakes" exposure discrepancies and sensor noise directly into the learned radiance field. This results in severe artifacts, including illumination mismatches, color shifts, and visible seams between views, particularly in static background regions critical for simulation. Existing extensions (e.g., StreetGS, DrivingGaussian) focus on dynamic object separation but fail to address cross-view photometric consistency. Furthermore, prior attempts to model illumination (e.g., GaussHDR, Luminance-GS) either require multi-exposure supervision, rely on unstable pseudo-HDR synthesis, or apply 2D corrections that lack physical grounding.

2. Methodology: P2GS

The authors propose P2GS, a physically grounded framework that reconstructs an exposure-invariant High Dynamic Range (HDR) radiance field from Low Dynamic Range (LDR) images without requiring HDR supervision. The method operates by shifting the optimization from the LDR space to the linear HDR domain, disentangling intrinsic scene radiance from camera-specific photometric responses.

Core Components

1. Linear HDR-Domain Radiance Representation:
   Instead of fitting spherical harmonic (SH) coefficients directly to observed LDR pixel values, P2GS optimizes them as linear HDR radiance. This creates a physically meaningful field ($\hat{I}_{linear}$) where scene radiance is decoupled from camera gain and tone mapping.

2. Per-View Exposure and Tone Modeling:
   The framework introduces learnable parameters for each view $i$:

   • Exposure Scale ($e_i$): A positive scalar representing the camera gain.
   • Tone-Mapping Function ($T_i$): Approximated via a learnable gamma correction ($\gamma_i$) to model the nonlinear camera response.
     The image formation model is defined as: $I_i = T_i(e_i \cdot \hat{I}_{linear})$.

3. Principle of Invariant Radiance (PIR):
   This is the theoretical cornerstone of P2GS. It posits that in the linear HDR domain, the ratio of observed radiances across different camera views depends solely on the relative exposure scales:
   $$\frac{I^{(i)}_{linear}}{I^{(j)}_{linear}} = \frac{e_i}{e_j}$$
   This principle enforces that the underlying scene radiance remains invariant regardless of the camera's exposure settings.

Unified Optimization Strategy

P2GS employs a unified optimization framework minimizing a total loss function ($\mathcal{L}_{total}$) composed of three terms:

1. Photometric Reconstruction ($\mathcal{L}_{photo}$): Minimizes the error between observed LDR images and synthesized images (rendered HDR $\to$ exposure $\to$ tone mapping).
2. Relative Exposure Consistency ($\mathcal{L}_{exp}$): Enforces the PIR constraint by ensuring that the ratio of rendered HDR intensities between view pairs matches the ratio of their estimated exposure scales. This prevents the radiance field from absorbing exposure variations.
3. Regularization ($\mathcal{L}_{reg}$):
   • Anchors the absolute scale of exposure parameters to prevent global scale ambiguity.
   • Penalizes excessive variance in exposure estimates across views to ensure stability.
   • Applies a soft prior to tone parameters ($\gamma_i$) around the sRGB standard ($\approx 2.2$) to prevent non-physical solutions.

At inference, the method averages the learned exposure and tone parameters across all training views to produce a stable, flicker-free rendering.

3. Key Contributions

The paper outlines three primary contributions:

• P2GS Framework: An explicit HDR-domain extension of 3DGS that embeds the Principle of Invariant Radiance to separate intrinsic radiance from exposure and tone variations, operating without HDR supervision.
• Unified Optimization: A fully differentiable strategy that jointly optimizes HDR radiance, per-view exposure, and tone mapping using a photometric loss, PIR-based constraints, and physical regularization.
• State-of-the-Art Performance: Demonstration of superior photometric consistency and HDR fidelity on multi-camera driving datasets with heterogeneous ISPs, outperforming existing baselines in both reconstruction and novel view synthesis.

4. Experimental Results

The authors evaluated P2GS on the Waymo Open Dataset (real-world) and CARLA (synthetic, controlled illumination).

• Quantitative Performance (Waymo):

  • P2GS achieved the highest SSIM (0.939) and lowest LPIPS (0.209) among all compared methods, including vanilla 3DGS, PVG, and Luminance-GS.
  • It significantly reduced the HDR Inconsistency Score (HIS) and Std-Luminance, indicating superior temporal and inter-view brightness stability. Notably, P2GS achieved lower inconsistency scores than the raw ground truth, suggesting it effectively removes dataset-induced illumination noise that standard methods retain.
  • In Novel View Synthesis (NVS), P2GS maintained consistent superiority, demonstrating robustness to varying lighting conditions.

• Qualitative Results:

  • Baseline methods (3DGS, PVG, Luminance-GS) exhibited visible seams, color shifts, and uneven shading where views with different exposures were stitched.
  • P2GS produced photometrically uniform reconstructions with smoothly matched brightness and color across all views, effectively suppressing seam artifacts on static structures like roads and walls.

• Robustness (CARLA):

  • Under synthetic datasets with controlled, varying ISO sensitivities (simulating extreme exposure noise), P2GS showed marginal performance drops compared to significant degradation in baselines (e.g., Luminance-GS and affine-transformed 3DGS).
  • This confirms P2GS's ability to disentangle exposure variations and preserve a view-invariant radiance field even under severe corruption.

• Ablation Studies:
  Removing either the exposure consistency loss ($\mathcal{L}_{exp}$) or the regularization terms ($\mathcal{L}_{reg}$) resulted in performance degradation across all metrics, validating the necessity of the unified physical formulation.

5. Significance and Claims

The paper claims that P2GS addresses a critical gap in autonomous driving simulation: the lack of photometrically stable 3D representations. By decoupling camera-dependent distortions from true scene radiance, P2GS enables:

• Reliable Closed-Loop Simulation: Providing a physically coherent background essential for training perception models and simulators where illumination consistency is paramount.
• Exposure-Invariant Controllability: Allowing for continuous, artifact-free re-rendering under arbitrary exposure scales, a capability impossible for standard 3DGS.
• Scalability: Offering a solution that works on sparse, large-scale driving data without requiring controlled environments or multi-exposure inputs.

The authors conclude that while P2GS focuses on static backgrounds and does not yet explicitly separate material properties from external lighting (a future direction), it establishes a strong, physically consistent foundation for large-scale urban reconstruction in autonomous driving.