AEGIR: Modeling Area Emitters for Indoor Inverse Rendering using Gaussian Splatting

Technical Summary: AEGIR – Modeling Area Emitters for Indoor Inverse Rendering Using Gaussian Splatting

Problem Statement
Inverse rendering aims to decompose observed images into geometry, materials, and illumination. However, this problem is inherently ill-posed due to the tight coupling between surface appearance and lighting. While recent advances in Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have improved novel view synthesis, extending these to inverse rendering remains challenging. Existing methods typically approximate scene lighting using global environment maps, discrete point lights, or implicit neural representations. These approaches fail in bounded indoor environments because they ignore the physical spatial extent of real-world emitters (e.g., lamps, ceiling panels). Consequently, they produce incorrect light attenuation, unrealistic shadow boundaries, and often absorb residual lighting effects directly into the surface albedo, leading to poor disentanglement of materials and illumination.

Methodology
The authors propose AEGIR (Area Emitters for Gaussian Inverse Rendering), a framework that explicitly models local area emitters within a relightable Gaussian Splatting representation. The methodology consists of three core components:

1. Explicit Local Area Emitters:
   Instead of point lights or global maps, AEGIR parameterizes lighting as localized 3D primitives. Each emitter is defined by:

   • Spatial properties: A center position, local orthonormal axes, and physical scales (anisotropic ellipsoids).
   • Emission profile: An RGB emission color and a Super-Gaussian angular falloff distribution.
   • Flexibility: By combining anisotropic scaling with a learnable Super-Gaussian power parameter ($\rho$), a single primitive can represent diverse sources, ranging from isotropic point lights to directional, elongated fixtures (e.g., fluorescent tubes) with sharp or soft falloffs.
   • Initialization: Emitters are initialized by unprojecting high-intensity pixels from multi-view images, clustering them via DBSCAN, and estimating initial parameters using Principal Component Analysis (PCA). An adaptive controller dynamically splits or prunes emitters during optimization based on energy and scale thresholds.

2. Deferred Rendering and Light Transport:
   AEGIR utilizes a deferred rendering pipeline coupled with 2D Gaussian Splatting (2DGS) to ensure reliable surface normals and geometry.

   • G-Buffer: 2D Gaussians are rasterized to generate a buffer containing albedo, roughness, metallic, and normals.
   • Sampling: The rendering equation is evaluated using a Monte Carlo estimator with Multiple Importance Sampling (MIS). This combines cosine-weighted hemisphere sampling (for diffuse transport) with direct light sampling from the explicit emitter surfaces.
   • Visibility: Soft shadows and occlusions are handled via differentiable ray tracing through the 2D Gaussian scene, computing a visibility term that accounts for accumulated opacity and occlusion distance.
   • Indirect Light: A single-bounce approximation is used during training (using secondary Gaussian hits) and a more rigorous single-bounce trace during inference.

3. Joint Optimization and Training Curriculum:
   To resolve the ambiguity between materials and lighting, the framework employs a progressive optimization strategy with a combined loss function ($L_{total} = L_{photo} + L_{prior} + L_{TV} + L_{light}$):

   • Contrast-Aware Photometric Loss: Upweights high-contrast pixels to prioritize high-frequency lighting cues.
   • Diffusion Priors: Uses a pretrained DiffusionRenderer to provide pseudo-ground truth for materials, preventing the model from baking lighting into albedo.
   • Edge-Aware Total Variation: Encourages piecewise constant materials on flat surfaces while preserving boundaries.
   • Emitter Regularization: Enforces physical plausibility by penalizing excessive scaling and enforcing color neutrality.
   • Curriculum: The process begins with a geometric baseline (DN-Splatter), followed by joint optimization of geometry, materials, and emitters, and concludes with a final stage where only emitters are optimized to refine lighting without altering materials.

Key Contributions

1. Novel Emitter Formulation: An explicit area emitter representation using anisotropic 3D ellipsoids with Super-Gaussian angular falloff, enabling the modeling of diverse real-world light shapes and emission profiles.
2. Joint Optimization Framework: A system that simultaneously recovers explicit light sources, scene geometry, and Physically Based Rendering (PBR) materials (albedo, roughness, metallic) from multi-view images, achieving better disentanglement through radiometric and geometric constraints.
3. Physically Grounded Rendering Model: A differentiable pipeline that evaluates illumination directly from area emitters, supporting physically consistent downstream tasks.

Results
The method was evaluated on synthetic datasets (Hypersim, FIPT-Synthetic) and real-world indoor scans (Replica, ScanNet++), comparing against state-of-the-art methods like GS-ID, IRGS, IRIS, and NeILF++.

• Lighting Estimation: In controlled re-rendering experiments using ground-truth geometry and materials, AEGIR achieved the highest PSNR (31.42 dB for indoor lighting) and lowest LPIPS, significantly outperforming methods relying on environment maps or point lights. It successfully recovered smooth indoor shading and accurate soft shadows.
• Novel View Synthesis (NVS) & Material Recovery: AEGIR achieved superior or competitive NVS performance (e.g., 21.50 PSNR on Hypersim) while maintaining higher albedo accuracy. Unlike baselines that often trade material accuracy for rendering quality by baking lighting into albedo, AEGIR maintained a better balance.
• Efficiency: The optimization process takes approximately 55 minutes on a single NVIDIA RTX 4090, making it roughly three times faster than NeILF++ and comparable to other efficient baselines.
• Ablation Studies: Removing components such as diffusion priors or focus maps resulted in significant drops in albedo accuracy and lighting estimation, confirming the necessity of the full pipeline for stable disentanglement.

Significance and Limitations
The paper claims that AEGIR bridges the gap between neural scene modeling and physically based rendering by explicitly modeling the spatial extent of light sources. This leads to more consistent decomposition of geometry, materials, and lighting, enabling realistic applications such as controlled relighting and virtual object insertion.

The authors modestly acknowledge current limitations:

• Shading artifacts may arise from noisy normals.
• The computational cost of shadow tracing increases with the number of emitters.
• The current material model does not handle mirror-like or transparent materials.
• There are broader ethical considerations regarding the potential misuse of such technology for generating deceptive media.

Despite these limitations, AEGIR is presented as a reliable approach for recovering illumination in complex indoor environments, offering a flexible foundation for realistic scene editing.