💻 computer science

DelowlightSplat: Feed-Forward Gaussian Splatting for Lowlight 3D Scene Reconstruction

The paper proposes DelowlightSplat, a feed-forward Gaussian splatting framework that integrates a lightweight lowlight adapter and cost-volume-based multi-view inference to enable robust 3D scene reconstruction and clean novel-view synthesis from sparse, low-light images.

Original authors: Fuzhen Jiang, Zengtian Xie, Zhuoran Li

Published 2026-05-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Fuzhen Jiang, Zengtian Xie, Zhuoran Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build a perfect 3D model of a room using only a few photos. Usually, if you take these photos in a bright, sunny room, your computer can easily figure out where the walls, furniture, and objects are. It's like putting together a puzzle where the pieces have clear, sharp edges.

But what happens if you try to do this in a pitch-black room with just a flashlight? The photos are grainy, the colors look weird (maybe everything looks blue or green), and the edges are blurry. If you try to build the 3D model directly from these bad photos, the computer gets confused. It might think a shadow is a hole in the wall, or it might make the furniture look like a melted blob.

This is the problem the paper DelowlightSplat is solving.

The Problem: "The Foggy Puzzle"

The authors explain that current AI tools for 3D reconstruction are great at handling bright photos but fail miserably in low light.

The Noise: In the dark, cameras get "noisy" (like static on an old TV).
The Color Shift: The colors get distorted (everything looks too orange or too blue).
The Confusion: Because the photos look so different from each other, the computer can't match the same object in Photo A with the same object in Photo B. Without matching these "dots," it can't build the 3D shape.

The Old Way: "Fix the Photo, Then Build"

Before this new method, people tried a two-step process:

Step 1: Take the dark, blurry photo and run it through a separate program to make it bright and clear (like using a photo editor to "enhance" a picture).
Step 2: Feed that "fixed" photo into the 3D builder.

Why this failed: The photo editor fixed the picture individually. It might make Photo A look warm and yellow, but Photo B look cool and blue. When the 3D builder tried to combine them, they didn't match up anymore. It's like trying to build a puzzle where someone painted the pieces different colors just to make them look nice individually, but now they don't fit together.

The New Solution: "DelowlightSplat"

The authors created a new system called DelowlightSplat. Think of it as a smart construction crew that doesn't just "fix" the photos, but learns how to "translate" them so they can work together.

Here is how it works, using a simple analogy:

1. The "Translator" (The Lowlight Adapter)

Instead of just brightening the image, the system uses a small, smart helper called a Lowlight Adapter.

Analogy: Imagine you are trying to talk to a friend who is speaking a dialect you don't understand. A translator doesn't just shout louder (brighten the image); they translate the meaning so you can understand each other.
What it does: This adapter looks at the dark, noisy photo and adds just the right amount of "correction" to make the features (like edges and textures) easier for the computer to recognize and match with other photos. It keeps the photos consistent with each other, rather than changing them randomly.

2. The "3D Builder" (Feed-Forward Gaussian Splatting)

Once the photos are "translated" and ready, the main system builds the 3D scene.

Analogy: Imagine a cloud of millions of tiny, glowing balloons (these are the "Gaussians"). Each balloon has a position, a size, a color, and a transparency.
What it does: The system looks at the "translated" photos and instantly figures out where to place these millions of balloons to recreate the room. It predicts the shape and color of the room directly, skipping the slow, step-by-step optimization that older methods used.

3. The Result: A "Clean" Room

The magic trick is that even though the input photos were dark and messy, the system builds a clean, bright, and sharp 3D model.

Analogy: It's like looking at a dark, foggy window and having a machine that instantly draws a crystal-clear picture of the garden outside, even though the glass is dirty. The output isn't a "fixed" photo; it's a brand new, perfect 3D view of the scene.

The Proof

The authors tested this by taking a large dataset of normal photos and artificially making the "input" photos dark and blurry (like turning down the lights and adding fog), while keeping the "target" photos clean to check the results.

Direct Method: When they fed the dark photos straight into the 3D builder, the result was a blurry, floating mess.
Old Two-Step Method: When they fixed the photos first and then built, the result was better but still had weird color shifts and blurry edges.
DelowlightSplat: Their new method produced a sharp, accurate 3D model that looked almost as good as if the photos had been taken in bright daylight.

In Summary

DelowlightSplat is a new tool that lets computers build perfect 3D models from bad, dark photos. It does this by using a smart "translator" to fix the photos just enough so they can talk to each other, and then instantly building a clean, bright 3D world from that conversation, skipping the messy steps of the past.

Technical Summary: DelowlightSplat

Problem Statement

Novel-view synthesis and 3D reconstruction from sparse posed images are critical for robotics and AR/VR. While recent feed-forward 3D Gaussian Splatting (3DGS) methods have successfully amortized per-scene optimization by directly predicting 3D Gaussians from sparse views, they fail significantly under low-light conditions. In low-light regimes, reduced contrast, sensor noise, and color bias degrade feature matchability. This breakdown in correspondence disrupts cost-volume aggregation, leading to incorrect geometry and over-smoothed appearances.

Existing workarounds, such as a "restore-then-reconstruct" pipeline (enhancing 2D images before reconstruction), are insufficient. Per-image enhancement often introduces view-dependent color shifts and illumination inconsistencies, failing to explicitly optimize for multi-view consistency, which results in unstable downstream reconstruction. Furthermore, there is a scarcity of realistic multi-view supervision data where target views are clean while context views are degraded, making it difficult to train models specifically for this task.

Methodology: DelowlightSplat

The authors propose DelowlightSplat, a lowlight-aware, two-view feed-forward 3DGS framework designed to reconstruct clean, normally illuminated 3D scenes directly from low-light inputs. The pipeline consists of three main stages:

1. Controllable Multi-view Lowlight Benchmark

To address the lack of training data, the authors constructed a benchmark based on a large-scale posed-view dataset. Instead of degrading entire scenes, they degrade only the context views (input images) while keeping the target views (ground truth) clean. The degradation operator $\mathcal{L}$ applies four probabilistic transformations sequentially to context images:

Gamma Darkening: $I \leftarrow I^\gamma$ with $\gamma > 1$ .
Exposure Scaling: $I \leftarrow s \cdot I$ with $0 < s < 1$ .
RGB Channel Shift: Independent scaling of R, G, and B channels.
Blur: Convolution with a Gaussian kernel.

This protocol isolates the impact of low-light inputs on reconstruction while preventing the model from simply learning to imitate low-light appearances.

2. Lowlight Adapter

A lightweight front-end module is introduced to map low-light inputs into a representation domain suitable for multi-view reasoning. Unlike standalone 2D enhancement models, this adapter is optimized to improve matchability for the subsequent pipeline.

Mechanism: It predicts a residual correction $\Delta(I_i)$ using a small stack of convolutional residual blocks.
Output: The adapted image is computed as $\tilde{I}_i = \text{clip}(I_i + \lambda \cdot \tanh(\Delta(I_i)))$ .
Benefit: This residual formulation allows the model to stay close to identity for mild degradations (avoiding over-enhancement) and is trained end-to-end with the reconstruction backbone.

3. Feed-forward Gaussian Splatting Network

The adapted context views $\{\tilde{I}_i\}$ are processed by a cost-volume-based multi-view encoder:

Feature Encoding: Extracts 2D features and performs cross-view interaction to aggregate geometrically consistent evidence.
Geometry Construction: Predicts depth and density-related quantities at each pixel. These are back-projected into 3D to form Gaussian centers ( $\mu$ ). Covariances ( $\Sigma$ ) and opacities ( $\alpha$ ) are parameterized to ensure positive definiteness and valid ranges.
Appearance: Each Gaussian carries Spherical Harmonics (SH) coefficients ( $h$ ) to represent view-dependent color.
Training Objective: The network is supervised using clean target images ( $I^\star_t$ ) via a combination of pixel-wise loss (e.g., $\ell_2$ or Charbonnier) and perceptual loss.

Key Contributions

A Controllable Multi-view Lowlight Benchmark: A training and evaluation framework that degrades only context views with probabilistic transformations while maintaining clean target views, enabling systematic ablation of low-light effects.
DelowlightSplat Pipeline: A novel two-view feed-forward 3DGS pipeline that integrates a lightweight Lowlight Adapter for residual enhancement. This approach improves cross-view matchability and directly predicts clean 3D Gaussians for novel-view rendering, avoiding the inconsistencies of two-stage pipelines.

Experimental Results

The method was evaluated against two baselines:

Lowlight MVSplat: Directly feeding low-light images into a standard feed-forward reconstructor.
DarkIR $\to$ MVSplat: A two-stage pipeline using a strong 2D restoration model (DarkIR) followed by reconstruction.

Quantitative Performance (Low-light Novel-View Synthesis):
DelowlightSplat significantly outperformed both baselines:

PSNR: Improved from 8.49 (Lowlight MVSplat) and 17.11 (DarkIR $\to$ MVSplat) to 23.08.
SSIM: Improved from 0.372 and 0.723 to 0.833.
LPIPS: Reduced from 0.355 and 0.225 to 0.162 (lower is better).

Qualitative Observations:

Direct reconstruction resulted in over-smoothed textures and geometric artifacts (e.g., floating structures).
The two-stage baseline improved brightness but introduced view-dependent color shifts and residual blur due to inconsistent per-image restoration.
DelowlightSplat produced sharper edges, stable geometry, and consistent color/shading across viewpoints.

Significance and Claims

The paper claims that DelowlightSplat effectively bridges the gap between standard feed-forward 3D reconstruction and low-light conditions. By integrating lowlight adaptation directly into the reconstruction pipeline rather than treating it as a separate pre-processing step, the method explicitly optimizes for multi-view consistency. The results demonstrate that this approach yields higher reconstruction fidelity and perceptual quality under diverse low-light degradations (gamma, exposure, color shift, and blur) compared to existing direct or two-stage methods.

The authors note that future work could extend the model to handle more views, longer temporal contexts, and real-world sensor artifacts such as severe motion blur and spatially varying noise.