Differentiable Ray Tracing with Gaussians for Unified Radio Propagation Simulation and View Synthesis

Technical Summary: Differentiable Ray Tracing with Gaussians for Unified Radio Propagation Simulation and View Synthesis

Problem Statement
Current wireless digital twins face a critical bottleneck: the disconnect between high-fidelity visual scene reconstruction and the geometric requirements for radio-frequency (RF) propagation simulation. While explicit neural representations like 3D Gaussian Splatting (3DGS) enable real-time, photorealistic novel view synthesis, they optimize for alpha-composited optical appearance rather than ray-intersectable geometry. Consequently, standard 3DGS pipelines cannot directly compute the deterministic multi-bounce paths required for physical RF modeling. Conversely, traditional RF simulation relies on manually constructed meshes (CAD or point-cloud-derived), which are expensive to build, difficult to update, and often lack the fine geometric details present in real environments. Furthermore, existing differentiable RF methods often rely on synthetic meshes or implicit fields that struggle to recover explicit, geometry-dependent propagation paths or generalize to sparse measurements.

Methodology
The authors propose a unified framework that embeds Gaussian primitives into a hardware-accelerated ray tracing structure, enabling differentiable RF propagation simulation directly within visually reconstructed neural scenes. The method operates through three primary stages:

1. Unified Gaussian Representation:
   The system utilizes 2D Gaussian Splatting (2DGS) to model the scene. Unlike standard 3DGS, 2DGS represents splats as planar Gaussians (elliptical disks) with zero extent along the z-axis, making them well-suited for precise ray-plane intersections. Each Gaussian is defined by mean position, scale, rotation, spherical harmonics, opacity, and an edge probability. To support RF simulation, the method assigns semantic material labels to Gaussians via a post-processing step using a Dinov3 head trained on ADE20K.

2. Differentiable Ray Tracing and Path Computation:
   The framework approximates each Gaussian with a lightweight polygonal mesh to enable hardware-accelerated ray intersection testing. It employs a visibility matrix between receivers and voxelized Gaussian mean points to efficiently identify coarse propagation paths. These paths are then refined to satisfy Fermat's principle of least time by optimizing interaction points within their local tangent planes. A novel path pruning strategy eliminates duplicate paths by checking interaction point proximity and normal coherence, while also leveraging per-Gaussian edge probabilities to discard non-physical reflections near geometric edges.

3. Training and Optimization:
   The model is trained using RGB, depth, and normal images alongside a sparse point cloud. The loss function combines photometric reconstruction (L1 + D-SSIM) with geometric regularizations:

   • Normal Regularization: Ensures Gaussians align with surface normals derived from depth gradients and monocular normal estimates (MoGe-2).
   • Depth Regularization: Uses logarithmic loss on ray-traced depth maps to smooth reconstructions.
   • Edge Regularization: Aligns predicted edge probabilities with binary masks derived from monocular normal gradients.

   For RF simulation, the framework introduces a Structural Path Alignment (SPA) loss. This loss matches simulated and measured Channel Impulse Responses (CIRs) by identifying corresponding multipath components (MPCs) based on time delay and minimizing the power difference, rather than simply matching aggregate metrics like RMS delay spread. This allows for the differentiable optimization of material parameters (relative permittivity and conductivity) and channel gain.

Key Contributions

1. Unified Differentiable Representation: A single Gaussian-based scene representation that simultaneously supports photorealistic rendering, multi-bounce geometric path tracing, and differentiable electromagnetic modeling.
2. Real-World Multimodal Dataset: A dataset comprising RGB-D images, calibrated camera poses, and channel measurements (including D-band and 234 GHz scenarios) to validate the approach.
3. Differentiable Alignment Objective: A novel loss function that optimizes channel parameters by matching simulated and measured CIRs, enabling the recovery of physically meaningful CIRs from visual-only reconstructions without manual geometric modeling.

Experimental Results
The method was evaluated on two real-world scenes: a corridor (D-band, 110–170 GHz) and an auditorium (234 GHz).

• Propagation Accuracy: In the corridor scene, the method achieved an RMS delay spread ($\tau_{rms}$) absolute error of 0.11 ns, outperforming point-cloud-based baselines (NimbusRT v1) and matching the accuracy of mesh-based simulators (Sionna) despite using automatically reconstructed geometry.
• Differentiable Optimization: In the auditorium scene, the SPA loss successfully optimized material parameters to reproduce measured power delay profiles. While the baseline loss (matching $\tau_{rms}$ directly) showed lower error in some splits, it failed to capture the correct physical path structure. The proposed SPA loss provided more faithful supervision, correctly identifying dominant reflection paths, though it occasionally missed late-arriving paths in complex scenarios.
• Visual Quality: The method maintained novel view synthesis quality comparable to state-of-the-art 2DGS and 3DGS baselines (PSNR ~24–30 dB) while using significantly fewer Gaussians (under 1 million per scene) and offering faster training times than hardware-accelerated RT baselines like 3DGRT.

Significance
The paper claims that this work bridges the gap between computer vision and wireless channel modeling by establishing a single spatial representation that generalizes across optical and RF domains. By enabling the computation of physically meaningful multi-bounce paths directly from visually reconstructed scenes, the approach removes the dependency on manual CAD geometry and supports scalable, measurement-driven digital twins. The authors posit that this integration allows for the learning of electromagnetic parameters from RF measurements within a differentiable pipeline, facilitating tighter integration between vision-based scene understanding and RF simulation for applications in wireless communications, robotics, and sensing. The work acknowledges limitations, such as the exclusion of diffraction and scattering, and notes that experimental validation is currently limited to two scenes due to the scarcity of paired RGB-D and channel measurement datasets.