💻 computer science

NTIRE 2026 3D Restoration and Reconstruction in Real-world Adverse Conditions: RealX3D Challenge Results

This paper presents the results of the NTIRE 2026 RealX3D Challenge, which evaluated 33 teams' methods for robust 3D restoration and reconstruction in extreme low-light and smoke-degraded environments, highlighting key design principles and significant progress in handling real-world scene degradation.

Original authors: Shuhong Liu, Chenyu Bao, Ziteng Cui, Xuangeng Chu, Bin Ren, Lin Gu, Xiang Chen, Mingrui Li, Long Ma, Marcos V. Conde, Radu Timofte, Yun Liu, Ryo Umagami, Tomohiro Hashimoto, Zijian Hu, Yuan Gan, Tianh

Published 2026-04-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Shuhong Liu, Chenyu Bao, Ziteng Cui, Xuangeng Chu, Bin Ren, Lin Gu, Xiang Chen, Mingrui Li, Long Ma, Marcos V. Conde, Radu Timofte, Yun Liu, Ryo Umagami, Tomohiro Hashimoto, Zijian Hu, Yuan Gan, Tianhan Xu, Yusuke Kurose, Tatsuya Harada, Junwei Yuan, Gengjia Chang, Xining Ge, Mache You, Qida Cao, Zeliang Li, Xinyuan Hu, Hongde Gu, Changyue Shi, Jiajun Ding, Zhou Yu, Jun Yu, Seungsang Oh, Fei Wang, Donggun Kim, Zhiliang Wu, Seho Ahn, Xinye Zheng, Kun Li, Yanyan Wei, Weisi Lin, Dizhe Zhang, Yuchao Chen, Meixi Song, Hanqing Wang, Haoran Feng, Lu Qi, Jiaao Shan, Yang Gu, Jiacheng Liu, Shiyu Liu, Kui Jiang, Junjun Jiang, Runyu Zhu, Sixun Dong, Qingxia Ye, Zhiqiang Zhang, Zhihua Xu, Zhiwei Wang, Phan The Son, Zhimiao Shi, Zixuan Guo, Xueming Fu, Lixia Han, Changhe Liu, Zhenyu Zhao, Manabu Tsukada, Zheng Zhang, Zihan Zhai, Tingting Li, Ziyang Zheng, Yuhao Liu, Dingju Wang, Jeongbin You, Younghyuk Kim, Il-Youp Kwak, Mingzhe Lyu, Junbo Yang, Wenhan Yang, Hongsen Zhang, Jinqiang Cui, Hong Zhang, Haojie Guo, Hantang Li, Qiang Zhu, Bowen He, Xiandong Meng, Debin Zhao, Xiaopeng Fan, Wei Zhou, Linzhe Jiang, Linfeng Li, Louzhe Xu, Qi Xu, Hang Song, Chenkun Guo, Weizhi Nie, Yufei Li, Xingan Zhan, Zhanqi Shi, Dufeng Zhang, Boyuan Tian, Jingshuo Zeng, Gang He, Yubao Fu, Weijie Wang, Cunchuan Huang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build a perfect 3D model of a room using a bunch of photos. Usually, you'd take these photos in bright, clear weather. But what if you tried to do it in a pitch-black cave or while standing inside a thick cloud of smoke? The photos would be dark, blurry, or full of "fog," making it nearly impossible to tell where the walls are or what the furniture looks like.

This paper is the report card for a global contest called NTIRE 2026, where 33 teams of scientists and engineers tried to solve this exact problem. Their goal was to figure out how to take these "ruined" photos (either too dark or smoky) and turn them into a clear, accurate 3D model.

Here is a simple breakdown of what happened, how they did it, and who won.

The Challenge: Two Different Disasters

The contest had two tracks, like two different levels in a video game:

Track 1 (The Dark Room): Teams had to fix photos taken in extreme low light.
Track 2 (The Smoke Machine): Teams had to fix photos taken through thick smoke.

They used a special dataset called RealX3D, which contains real-world scenes captured in these terrible conditions. The teams had to feed these bad photos into their computers and get a clean 3D picture out the other side.

The Secret Weapon: "3DGS"

Almost everyone used a technology called 3D Gaussian Splatting (3DGS).

The Analogy: Imagine trying to paint a 3D sculpture. Instead of using a solid block of clay, you use millions of tiny, fluffy balls of paint (Gaussians). You throw these balls into the air, and they stick together to form the shape of the object.
The Problem: When the photos are dark or smoky, the computer gets confused about where to put the balls. It might think a shadow is a hole, or that smoke is a wall.
The Solution: The teams had to invent clever ways to "clean up" the photos before or while throwing the paint balls, so the computer knows exactly what the real object looks like.

How the Winners Did It (The Strategies)

The teams used some very creative tricks to win. Here are a few examples of their approaches:

The "Teamwork" Approach (FuME-GS): Instead of relying on one method to fix the dark photos, this team used four different "fixers" at the same time. It's like asking four different art restorers to fix a painting, then combining their best parts to make one perfect image.
The "Three-Headed Monster" (CISP-GS): This team built a system with three different branches. One branch focused on the brightness, another on the camera settings, and a third on the fine details. They let all three work together and averaged their results to get a super-stable image.
The "Physics Detective" (Smoke-GS & GenSmoke-GS): For the smoke challenge, some teams didn't just try to "erase" the smoke. They built a mathematical model of how smoke behaves (like how light scatters). They taught the computer the rules of smoke physics so it could mathematically subtract the smoke from the image, rather than just guessing.
The "Generative Artist" (GenSmoke-GS): One team used a powerful AI (a Multi-Modal Large Language Model) to act like an artist. They told the AI, "Here is a smoky photo; please redraw the details you can't see, but don't change the shape of the building." The AI filled in the missing details based on what it knows about the world.

The Results

Out of 279 people who signed up, only 33 teams managed to finish the challenge with a working solution.

The Winners:
- For the Dark Room: The team DimV won with their method called FuME-GS. They got the clearest picture by fusing multiple enhancement techniques.
- For the Smoke: The team PLBBL won with GenSmoke-GS. They used a mix of physics and AI generation to clear the air.

Why This Matters

The paper concludes that while we have gotten very good at making 3D models in perfect studios, the real world is messy. These winners showed us that by combining image cleaning (fixing the photo) with 3D reconstruction (building the model), we can finally see through the dark and the smoke.

The paper doesn't promise that your phone will do this tomorrow, but it proves that the technology is moving in the right direction. The organizers hope this helps future robots and self-driving cars navigate safely even when it's pitch black or foggy outside.

In short: It was a race to see who could best clean up a messy photo and turn it into a clear 3D world, and the winners proved that with the right mix of math, physics, and AI, even the darkest and smokiest scenes can be reconstructed.

1. Problem Statement

The paper addresses the critical challenge of 3D scene reconstruction (specifically using Neural Radiance Fields and 3D Gaussian Splatting) under real-world adverse conditions. While current state-of-the-art methods (e.g., NeRF, 3DGS) perform well in controlled, clean environments, they fail significantly when input images are degraded by:

Extreme Low-Light: Leading to noise, loss of texture, and poor feature extraction.
Smoke/Haze: Causing scattering, reduced contrast, color shifts, and occlusion of geometric details.

The scarcity of real-world multi-view datasets with paired degraded/clean ground truth further hinders progress. The paper introduces the RealX3D Challenge to benchmark robust pipelines for these specific degradations.

2. Methodology: The RealX3D Challenge

The challenge was organized into two distinct tracks, both utilizing a subset of the RealX3D benchmark containing 7 distinct scenes per track.

Dataset: Real-world multi-view image pairs (degraded vs. clean) with ground-truth camera poses.
- Training: 1 validation scene (paired) + 4 development scenes (degraded only).
- Testing: 3 scenes (degraded only).
Tracks:
1. Track 1: 3D Low-light Enhancement: Focuses on recovering illumination, reducing noise, and preserving structure in dark environments.
2. Track 2: 3D Smoke Restoration: Focuses on removing scattering effects, restoring contrast, and correcting color shifts caused by smoke.
Evaluation:
- Metric: Novel View Synthesis (NVS) quality measured by PSNR (primary) and SSIM (tiebreaker) against ground-truth clean images.
- Protocol: 33 teams submitted valid results out of 279 registered participants.

3. Key Contributions & Methodological Trends

The paper reviews 27 submitted methods, revealing several dominant technical strategies used by top-performing teams:

A. Hybrid "Restoration-then-Reconstruction" Pipelines

Most top teams (e.g., FuME-GS, GenSmoke-GS, 3DLLR) decouple the problem into two stages:

2D Pre-processing: Using advanced 2D restoration networks (e.g., Retinexformer, Zero-DCE, DarkIR, Diffusion models) to generate pseudo-clean images or enhanced candidates.
3D Reconstruction: Feeding these enhanced images into 3D Gaussian Splatting (3DGS) or NeRF pipelines.
- Innovation: Many teams use ensemble strategies (e.g., GenSmoke-GS, IC-GS) where multiple restoration models or 3D branches are trained and averaged to reduce variance and hallucinations.

B. Physics-Informed and Pseudo-GT Generation

To avoid overfitting to noisy 2D priors, several methods incorporate physical models:

Atmospheric Scattering Models: Used in Smoke-GS and SDG-GS to explicitly model smoke transmission ( $T$ ) and airlight ( $A$ ), allowing the 3DGS to learn the "clean" scene radiance ( $J$ ) by inverting the degradation equation.
Pseudo-GT Generation: Methods like IDEAL and AdaTone-GS generate pseudo-ground truth using depth estimators (Depth-Anything) or analytical tone curves, then use Appearance Decoders (MLPs) to absorb view-dependent artifacts, preventing 2D noise from baking into the 3D geometry.

C. Multi-Branch and Frequency-Split Architectures

CISP-GS and NAKA-GS utilize multi-branch frameworks where different branches handle specific aspects (e.g., analytical enhancement vs. deep learning enhancement vs. frequency separation).
Frequency Splitting: Separating low-frequency (structure/color) and high-frequency (texture) components allows models to preserve geometric consistency while injecting sharp details from deep enhancers.

D. Joint Optimization (End-to-End)

Some approaches, such as DarkIR-GS and LLE-GS, attempt end-to-end training where the 3DGS renderer back-propagates gradients to the 2D restoration module, ensuring the restoration is optimized specifically for the 3D reconstruction task rather than generic 2D metrics.

4. Results

The quantitative results (Table 1) highlight significant progress over baselines:

Track 1 (Low-light):
- Winner: FuME-GS (DimV) achieved the highest PSNR (23.38) and SSIM (0.80).
- Key Insight: The winning method used a multi-stage fusion of multiple enhancement models (Retinexformer, Zero-DCE, etc.) combined with a fusion-guided 3DGS initialization.
- Runner-up: CISP-GS (22.78 PSNR) utilized a multi-branch framework with ISP supervision.
Track 2 (Smoke):
- Winner: GenSmoke-GS (PLBBL) achieved the highest PSNR (20.21) and SSIM (0.73).
- Key Insight: This method combined ConvIR-UDPNet for initial restoration, DCP for dehazing, and a Multi-Modal Large Language Model (MLLM) for structural refinement to prevent geometry drift, followed by 3DGS-MCMC ensembling.
- Runner-up: Smoke-GS (18.67 PSNR) introduced a specific "Smoke Medium Module" to model view-dependent scattering.

General Observation: Top-performing methods consistently outperformed simple baselines by integrating multi-model ensembles, physics-based constraints, and robust initialization strategies (e.g., using VGGT or COLMAP with specific pruning).

5. Significance and Impact

Benchmarking Real-World Robustness: The paper establishes RealX3D as a critical benchmark for evaluating 3D reconstruction in non-ideal conditions, moving the field beyond synthetic or clean-capture datasets.
Design Principles: It identifies that the most effective strategies for adverse conditions are not single-model solutions but hybrid pipelines that combine:
1. Robust 2D restoration priors.
2. Physics-based degradation modeling.
3. Ensemble methods to mitigate hallucinations.
4. Explicit separation of geometry (structure) and appearance (illumination/color).
Practical Applications: The advancements directly benefit autonomous driving (night/smoke scenarios), robotic navigation in hazardous environments, and augmented reality in uncontrolled lighting.
Community Progress: With 33 valid teams submitting novel architectures, the challenge demonstrates a rapid maturation of techniques for handling 3D scene degradation, setting a new standard for future research in robust 3D vision.