💻 computer science

NTIRE 2026 The 3rd Restore Any Image Model (RAIM) Challenge: Multi-Exposure Image Fusion in Dynamic Scenes (Track 2)

This paper introduces the NTIRE 2026 3rd Restore Any Image Model (RAIM) Challenge, which established a benchmark for multi-exposure image fusion in dynamic scenes to address artifacts like ghosting and misalignment, attracting 114 teams and 987 submissions to advance HDR imaging capabilities.

Original authors: Lishen Qu, Yao Liu, Jie Liang, Hui Zeng, Wen Dai, Guanyi Qin, Ya-nan Guan, Shihao Zhou, Jufeng Yang, Lei Zhang, Radu Timofte, Xiyuan Yuan, Wanjie Sun, Shihang Li, Bo Zhang, Bin Chen, Jiannan Lin, Yuxu

Published 2026-04-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Lishen Qu, Yao Liu, Jie Liang, Hui Zeng, Wen Dai, Guanyi Qin, Ya-nan Guan, Shihao Zhou, Jufeng Yang, Lei Zhang, Radu Timofte, Xiyuan Yuan, Wanjie Sun, Shihang Li, Bo Zhang, Bin Chen, Jiannan Lin, Yuxu Chen, Qinquan Gao, Tong Tong, Song Gao, Jiacong Tang, Tao Hu, Xiaowen Ma, Qingsen Yan, Sunhan Xu, Juan Wang, Xinyu Sun, Lei Qi, He Xu, Jiachen Tu, Guoyi Xu, Yaoxin Jiang, Jiajia Liu, Yaokun Shi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to take a perfect photo of a busy street scene at sunset. You have a camera that can't capture everything in one go: the bright sky is too white (blown out), and the dark alleyways are too black (crushed).

To solve this, you take three or five photos in rapid succession:

One photo is super bright (to see the shadows).
One is super dark (to see the sky).
One is in the middle.

The Problem: The world isn't a painting; it's alive. While you were snapping these photos, a car drove by, a person walked past, or your hand shook slightly. If you just stack these photos on top of each other, you get a messy "ghost" image where the person looks like they have three arms, or the car is smeared across the street.

The Challenge (NTIRE 2026 RAIM Track 2):
This paper describes a high-stakes competition where 114 teams of AI researchers tried to build the ultimate "digital photo editor." Their goal? To take those messy, misaligned, multi-exposure photos and fuse them into one perfect, crystal-clear image that looks like it was taken with a super-camera, even though the scene was moving and shaking.

Here is how the competition worked, explained through simple analogies:

1. The Training Ground (The Dataset)

The organizers didn't just give the teams a few easy pictures. They created a gym for AI.

The Workout: They provided 100 "training sequences" (like practice drills) where the AI had to learn to handle moving cars, flickering lights, and shaky hands.
The Exam: Then, they gave them 100 "test sequences" with fewer photos to work with (5 instead of 7), making the job harder. It's like asking a chef to make a gourmet meal with fewer ingredients than they practiced with.

2. The Rules of the Game (Evaluation)

How do you judge if the AI did a good job?

The Scorecard: They used a mix of math and human feeling.
- PSNR/SSIM: These are like a "pixel-perfect" ruler. They check if the AI's image matches the "perfect" answer exactly.
- LPIPS: This is the "human eye" test. It checks if the image looks natural or if it looks weird and plastic.
The Final Verdict: The winner wasn't just the one with the highest math score; they also had to prove their code actually worked and wasn't cheating.

3. The Winning Strategies (The Teams)

Think of the teams as different types of chefs trying to solve the same recipe problem. Here are a few of the top contenders and their "secret sauces":

WHU-VIP (The Smart Editor):
- The Analogy: Imagine a photo editor who decides, "I don't need all 7 photos; the middle one plus the two brightest/darkest ones are enough."
- The Trick: They built a system that ignores the "noisy" parts of the photos. If a car moved and created a ghost, their AI has a "reliability gate" that says, "Ignore that pixel, it's a ghost," and fills it in with the correct detail from another photo. They also used a "curriculum learning" strategy, teaching the AI easy scenes first, then harder ones, just like a teacher.
SHL (The Big Picture Guide):
- The Analogy: Imagine trying to paint a tiny detail on a map while blindfolded. SHL gave the AI a "bird's-eye view" of the whole scene (a downsampled version) to help it understand the context.
- The Trick: They added a "Neighboring Pixel Relationship" loss. Think of this as a strict rule: "If the edge of a building is straight in the input, it must stay straight in the output." This prevented the AI from blurring edges.
nunucccb (The Transformer Detective):
- The Analogy: This team used a "Vision Transformer," which is like a detective that looks at the whole picture at once to understand relationships, rather than just looking at one small square at a time.
- The Trick: They used a "Spatial Attention" mechanism. It's like having a spotlight that shines only on the parts of the image that are moving, telling the AI, "Pay extra attention here to fix the misalignment."
I2 Group & Transsion (The Wavelet Weaver):
- The Analogy: Imagine taking a photo and separating it into "structure" (the shape of things) and "texture" (the fine details like hair or brick).
- The Trick: They used a mathematical tool called a "Wavelet Transform" to split the photos. They fixed the "structure" in one branch and the "texture" in another, then stitched them back together. This prevented the "ghosting" that usually happens when you try to fix both at once.

4. The Result

The winner, WHU-VIP, didn't just get the highest math score; they found the perfect balance. They removed the "ghosts" (the blurry double-images of moving people) better than anyone else while keeping the fine details sharp.

Why does this matter?
This isn't just about winning a contest. This technology is what makes your smartphone camera work. When you take a photo in low light or of a moving pet, your phone is secretly doing this exact "multi-exposure fusion" magic to give you a clear, bright, and sharp picture. This challenge helps engineers build better cameras for the future.

In short: 114 teams taught AI how to be a master photo editor, learning to fix shaky, moving, and poorly lit photos into one perfect shot. The winner proved that the best AI doesn't just crunch numbers; it understands the "story" of the image.

1. Problem Definition

The paper addresses the challenge of Multi-Exposure Image Fusion (MEF) in dynamic scenes. Unlike static benchmarks, real-world photography involves:

Scene Motion: Moving objects between exposure captures.
Camera Jitter: Handheld instability causing global misalignment.
Illumination Variation: Changes in lighting conditions between frames.
Exposure Bracketing: Combining frames with different exposure levels (EV) to recover High Dynamic Range (HDR) details.

The core difficulty lies in fusing these frames to produce a single, high-quality image without ghosting artifacts, misalignment, or color distortion, while maintaining computational efficiency for mobile deployment.

2. Dataset and Benchmark Design

The organizers introduced a new benchmark specifically designed to reflect realistic mobile photography constraints:

Training Set: 100 sequences, each containing 7 exposure levels. Paired with reference fusion images for supervised learning.
Test Set: 100 sequences, each containing 5 exposure levels. The reduced number of frames increases the difficulty, requiring methods to remain stable with less information.
Key Characteristics: The dataset includes object motion, local saturation/underexposure, and global misalignment, making it suitable for evaluating robustness against dynamic artifacts.

3. Evaluation Protocol

The challenge utilized a multi-dimensional evaluation strategy:

Quantitative Metrics: A composite leaderboard score derived from PSNR, SSIM, and LPIPS (Learned Perceptual Image Patch Similarity).
- Formula: $Score = 30 \cdot \frac{PSNR}{50} + 22.5 \cdot \frac{SSIM - 0.5}{0.5} + 30 \cdot (1 - \frac{LPIPS}{0.4})$ .
Secondary Metrics: DISTS, NIQE, and subjective quality assessments.
Efficiency Constraints: Runtime, parameter count, and FLOPs were recorded to ensure practical applicability.
Reproducibility: Final ranking required the submission of runnable code and pre-trained weights to verify that leaderboard scores could be reproduced locally.

4. Methodology: Top Performing Approaches

The challenge attracted 114 teams with 987 submissions. The top-performing methods employed diverse architectural strategies:

WHU-VIP (Winner):
- Architecture: Based on AFUNet (Swin Transformer backbone).
- Key Innovations:
  - Input Reduction: Uses only 3 frames (0 EV, ±2 EV) instead of all available frames to reduce motion artifacts and improve efficiency.
  - Spatial Feature Transform (SFT): Modulates features spatially based on local illumination.
  - DCM-RG (Data Consistency Module with Reliability Gating): Predicts pixel-wise gates to suppress unreliable regions (misaligned or over/under-exposed) during fusion.
  - Curriculum Learning: A 4-stage training strategy progressing from basic alignment to perceptual enhancement and quantization-aware training.
SHL (Runner-up):
- Strategy: Lightweight global guidance.
- Mechanism: Concatenates a downsampled full-resolution image as a global context prior to the model input.
- Loss Function: Introduces a Neighboring Pixel Relationships (NPR) Loss to explicitly constrain local structural consistency and preserve high-frequency details, preventing blurring.
- Ensemble: Combines predictions from three different backbones (Restormer, Uformer, MST++) via weighted averaging.
nunucccb (3rd Place):
- Architecture: HDR-Transformer based on Context-Aware Vision Transformer (CA-ViT).
- Mechanism: Uses spatial attention to align non-reference features against a reference feature ( $f_3$ ) to suppress foreground motion artifacts. Followed by a reconstruction network with dilated convolutions to expand the receptive field.
untrafusion:
- Strategy: End-to-end framework modeling physical exposure priors.
- Mechanism: Uses a luminance-prior guided exposure alignment module to handle large pixel-value differences. Employs a scale-evolving training strategy (progressive tiling) to handle noise and structural artifacts in the non-linear JPG domain.
I2 Group & Transsion:
- Strategy: Wavelet-based decomposition.
- Mechanism: Separates inputs into Low-Frequency (LF) and High-Frequency (HF) components using Discrete Wavelet Transform (DWT).
  - LF Branch: Uses DehazeFormer-s for intensity and long-range dependencies.
  - HF Branch: Uses a lightweight residual fusion module to refine details without introducing ghosting.
NTR:
- Strategy: Metric-aligned diffusion.
- Mechanism: Fine-tunes a masked diffusion autoencoder (TimeDiffiT) with a time-conditioned U-Net. The loss function is explicitly optimized to match the official challenge score (combining L1, SSIM, and LPIPS).

5. Key Results

Winner: WHU-VIP achieved the highest final score (58.889), demonstrating the best balance between PSNR, SSIM, and perceptual quality (LPIPS).
Performance Trends:
- SHL achieved the highest raw PSNR and SSIM but suffered slightly in perceptual quality (LPIPS), resulting in a lower overall rank.
- Robustness: The top 3 teams maintained stable performance across both Test Stage 1 and Stage 2, whereas lower-ranked teams showed significant performance drops, highlighting the difficulty of generalizing to unseen motion and exposure variations.
Visual Quality: Winning methods successfully suppressed ghosting artifacts and preserved fine textures in challenging dynamic regions compared to baseline methods.

6. Significance and Contributions

Bridging the Gap: The challenge successfully narrows the gap between academic research and practical mobile HDR deployment by enforcing efficiency constraints and using a dynamic, real-world dataset.
New Benchmark: Established a reproducible benchmark for dynamic-scene MEF that emphasizes motion robustness and artifact suppression, moving beyond static scene assumptions.
Methodological Insights: Demonstrated that curriculum learning, global guidance priors, wavelet decomposition, and metric-aligned training are effective strategies for handling dynamic fusion.
Resource Availability: The organizers released the dataset, reference images, and code from all participating teams, fostering further research in computational photography.

This work underscores that while high-fidelity reconstruction is achievable, the primary challenge in dynamic MEF remains the consistent handling of misalignment and ghosting without sacrificing perceptual naturalness or computational efficiency.