Complex-Valued 2D Gaussian Representation for Computer-Generated Holography

Technical Summary: Complex-Valued 2D Gaussian Representation for Computer-Generated Holography

Problem Definition
Computer-Generated Holography (CGH) aims to synthesize holograms that reconstruct 3D scenes by capturing light intensity, interference, and diffraction. Unlike natural images, holograms exhibit dense, high-frequency, and random spatial variations. Existing representation methods struggle with this domain:

• Implicit Neural Representations (INRs) and Autoencoders tend to favor smooth, low-frequency data, failing to capture the high-frequency details inherent in holograms.
• Per-pixel optimization creates a massive parameter search space, hindering scalability and efficiency.
• Existing Gaussian-based methods (e.g., 3D Gaussian Splatting variants) are often designed for natural images or 3D novel view synthesis (NVS) and do not explicitly model wave optics (interference and diffraction) or the complex-valued nature of holographic fields.

Methodology
The authors propose a structured representation using Complex-Valued 2D Gaussian Primitives specifically optimized for single-view hologram estimation.

1. Primitive Definition:
   The method defines a primitive $G_n$ parameterized by pre-activation position ($\tilde{\mathbf{x}}_n$), scales ($\tilde{\mathbf{s}}_n$), rotation ($\theta_n$), color amplitudes ($\mathbf{c}_n$), opacity ($\tilde{\alpha}_n$), and per-channel phase ($\boldsymbol{\varphi}_n$).

   • Unlike naive approaches that pair two real-valued Gaussians for real and imaginary parts (requiring 18 parameters and two render passes), this unified formulation uses 12 parameters and a single rendering pass.
   • The contribution of a primitive at pixel $p$ is $H_n(p) = \alpha_n \mathbf{c}_n g_n(p) \exp(j \boldsymbol{\varphi}_n)$, where $g_n(p)$ is the 2D Gaussian spatial distribution.

2. Theoretical Foundation (Gabor's Theory):
   Inspired by Gabor's uncertainty principle, the authors argue that Gaussian functions achieve the minimum space–frequency uncertainty ($\Delta x \Delta f_x = 1/2$). This makes them the optimal compact primitive for representing holograms, which require simultaneous encoding of spatial and frequency information. The method demonstrates that while Gaussians do not surpass per-pixel quality at full parameter budgets, they retain reconstruction fidelity significantly better as the parameter budget shrinks.

3. Differentiable Pipeline:

   • Rasterization: A custom differentiable rasterizer for complex-valued 2D Gaussians is developed, handling complex arithmetic (trigonometric evaluation for real/imaginary decomposition) and gradient computation for amplitude and phase.
   • Light Propagation: The pipeline integrates a GPU-optimized Band-limited Angular Spectrum Method (BLASM) kernel. This simulates free-space light propagation, allowing the system to optimize the hologram field $H$ such that its propagated intensity $|U(d)|^2$ matches target images at multiple depth planes.
   • Loss Function: Optimization minimizes a reconstruction loss combining MSE, structural similarity (SSIM), and specific terms for defocus regions, supervised against multi-plane focal stacks.

4. POH Conversion:
   Since commercial displays are typically Phase-Only Holograms (POH), the authors introduce a conversion procedure:

   • Smooth POH: Uses Double Phase-Amplitude Coding (DPAC) to spatially multiplex amplitude and phase.
   • Random POH: Uses the complex Gaussian representation as structural guidance to optimize a random phase-only hologram, jointly minimizing intensity and complex field differences.

Key Contributions

• Novel Representation: Migration of complex-valued 3D Gaussians to complex-valued 2D Gaussian primitives for direct hologram optimization, reducing the parameter search space by a 5:1 ratio compared to dense per-pixel parameterization.
• System Implementation: Development of a differentiable rasterizer and a GPU-optimized BLASM kernel for end-to-end training, achieving significant speedups and memory efficiency over standard PyTorch implementations.
• Conversion Framework: A procedure to adapt the complex representation to practical Smooth and Random POH formats, decoupling the representation from hardware-specific encoding.
• Theoretical Justification: Demonstration that the proposed primitive attains minimum space–frequency uncertainty, providing a theoretical basis for its efficiency in hologram representation.

Results
Extensive experiments on the DIV2K dataset and hardware validation (using a LASOS MCS4 laser and Jasper JD7714 SLM) show:

• Quality: The method achieves 30.7 dB PSNR for complex holograms, outperforming prior Gaussian-based methods (e.g., Image-GS at 17.2 dB, GI at 22.6 dB) and learned representations (MLP, SIREN). For Random POH, it reaches 29.4 dB PSNR, a 4.1 dB gain over Wirtinger Holography.
• Efficiency:
  • VRAM Usage: Reduced by ~30% compared to standard autodiff-based implementations.
  • Optimization Speed: Accelerated by 50% (e.g., 1.4 minutes vs. 5.1 minutes for GWS).
  • Rendering Speed: Achieves up to 3200× faster rendering compared to GWS-based approaches.
• Scalability: The method scales to resolutions of 3200×1800 without Out-of-Memory (OOM) errors.
• Ablation: The unified complex formulation yields +6.3 dB higher PSNR and 33% fewer parameters compared to naive paired real-valued Gaussians.

Significance
The paper claims that by reducing the hologram parameter search space through a theoretically grounded, compact representation, it enables more scalable hologram estimation for next-generation CGH systems. The work bridges the gap between efficient Gaussian-based rendering and the physical constraints of wave optics, offering a general intermediate representation that can be converted to various practical hologram formats while maintaining high reconstruction fidelity. The authors note that while the method focuses on center-eyebox reconstruction, it provides a foundation for future work in real-time video and camera-in-the-loop techniques.