💻 computer science

CLIP-Guided Data Augmentation for Night-Time Image Dehazing

This paper presents a unified framework for the NTIRE 2026 Night Time Image Dehazing Challenge that leverages CLIP-guided data augmentation to construct domain-aligned training sets, employs a two-stage NAFNet training strategy, and combines inference-time enhancements to effectively address the complex degradation patterns of nighttime haze under limited supervision.

Original authors: Xining Ge, Weijun Yuan, Gengjia Chang, Xuyang Li, Shuhong Liu

Published 2026-04-08

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Xining Ge, Weijun Yuan, Gengjia Chang, Xuyang Li, Shuhong Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to take a beautiful photo of a city street at night, but a thick, glowing fog has rolled in. The streetlights are blinding, the shadows are pitch black, and the whole scene looks muddy and blurry. This is the problem of Nighttime Image Dehazing.

The paper you shared is a "recipe" for a computer program that cleans up these messy photos. The authors realized that teaching a computer to fix these specific night-time photos is incredibly hard because there aren't many "before and after" examples to learn from.

Here is how they solved it, explained with simple analogies:

1. The Problem: The "Foreign Student" Dilemma

Usually, to teach a computer how to clean a foggy night photo, you show it thousands of examples. But for night-time fog, those examples are rare.

The Trap: The authors considered using photos of daytime fog to help teach the computer. But that's like trying to teach a student how to swim in the ocean by only showing them videos of people swimming in a pool. The water is different, the currents are different, and the student gets confused. If you mix too many "daytime" examples with the few "nighttime" ones, the computer gets lost and stops learning the specific rules of the night.

2. The Solution: The "Smart Librarian" (CLIP)

Instead of throwing every available foggy photo into the training mix, the team built a Smart Librarian using a tool called CLIP.

How it works: Imagine you have a huge library of foggy photos from all over the world. You tell the librarian, "I only want books that feel like a nighttime foggy street."
The Filter: The librarian (CLIP) looks at every single photo and asks, "Does this look like the target night scene?" If a photo is a bright, sunny day with fog, the librarian says, "No, that's too different," and puts it back on the shelf. If a photo has the right mix of darkness and artificial light, the librarian says, "Yes, this fits!"
The Result: They ended up with a small, carefully curated "training set" of 59 photos that are all very similar to the problem they are trying to solve. This prevents the computer from getting confused by "foreign" data.

3. The Training: The "Two-Step Dance"

Once they had the right photos, they didn't just dump them all into the computer at once. They used a Two-Stage Training method.

Step 1: Learning the Basics. First, they taught the computer only on the real night-time photos. Think of this as teaching a musician to play a specific song perfectly before they try to improvise. The computer learns the specific "vibe" of night fog.
Step 2: Expanding the Horizons. Once the computer mastered the night photos, they introduced the filtered "foreign" photos (the ones the librarian approved). Now, the computer can say, "Okay, I know how night fog works, and now I see that this daytime fog is actually quite similar in some ways." This makes the computer smarter and more robust without forgetting the basics.

4. The Final Touch: The "Panel of Judges" (Inference)

When the computer is finally ready to clean a new photo, it doesn't just make one guess. It uses a Panel of Judges strategy.

The Mirror Trick (Self-Ensemble): Imagine looking at a painting in a mirror, then turning it upside down, then sideways. The computer looks at the photo in 8 different ways (flipped, rotated, etc.) and makes a guess for each. Then, it averages all 8 guesses. This cancels out any weird mistakes the computer might make in one specific angle.
The Time-Traveler (Snapshot Fusion): The computer was saved at three different times during its training (like taking a photo of a student at the 1st month, 3rd month, and graduation). The final result is a weighted mix of these three versions. The "graduation" version gets the most weight because it's the best, but the earlier versions help smooth out any rough edges.

The Bottom Line

The authors didn't invent a brand-new, super-complex brain for the computer. Instead, they built a smart pipeline:

Filter the data so the computer only learns from relevant examples.
Train in steps so it learns the basics before getting advanced.
Combine multiple guesses at the end to ensure the result is stable and clear.

It's a reminder that sometimes, the best way to solve a hard problem isn't to build a bigger, more complicated machine, but to be smarter about what you feed the machine and how you guide it.

1. Problem Statement

Nighttime image dehazing presents a significantly more complex challenge than daytime dehazing due to the coupling of multiple degradation factors:

Complex Degradation: Unlike daytime haze (primarily scattering), nighttime haze involves low illumination, non-uniform lighting, strong artificial light sources, and halo diffusion.
Data Scarcity: High-quality paired target-domain data (nighttime hazy/clear pairs) is extremely limited, leading to domain drift and training instability.
Domain Mismatch: Naively introducing external data (e.g., daytime haze datasets) without control causes distribution mismatch, which can weaken model adaptation rather than improve it.
Optimization Challenges: Simply increasing network capacity often fails under limited supervision, leading to unstable optimization and poor generalization.

2. Methodology

The authors propose a unified framework that avoids complex network redesign. Instead, it focuses on the co-design of data screening, stage-wise training, and inference-time enhancement. The pipeline consists of three main components:

A. CLIP-Guided Data Curation (Data Level)

To address the scarcity of nighttime data, the authors augment the training set using external datasets (I-HAZE, Dense-Haze, HAZE1K) but strictly filter them to ensure domain consistency.

Mechanism: A pre-trained CLIP visual encoder (ViT-B/32) is used to extract feature representations from both the target nighttime images (NTHazy) and candidate external samples.
Selection Criteria: External samples are retained only if they exhibit high semantic similarity to the target domain in terms of visual content, illumination, and degradation morphology.
Result: This selective strategy reduced the external pool to a curated set of 59 total pairs (25 original NTHazy + 34 filtered external pairs), ensuring the training distribution remains aligned with the nighttime target domain.

B. Stage-Wise Restoration Training (Optimization Level)

The authors employ a two-stage training strategy using NAFNet as the backbone to prevent the model from being biased by the larger but imperfectly aligned external data.

Stage 1 (Target Adaptation): The model is trained exclusively on the 25 original NTHazy pairs. This allows the network to internalize the specific characteristics of nighttime haze, low-light imaging, and target-domain texture statistics.
Stage 2 (Domain Expansion): The CLIP-filtered auxiliary data is introduced, and the Stage 1 model is fine-tuned on the combined dataset. Since the model already possesses a "nighttime prior," it can absorb the broader degradation patterns from the external data without overwriting the target-domain behavior.

C. Inference-Time Enhancement (Inference Level)

To improve output stability and reconstruction quality without modifying learned parameters, three strategies are combined:

TLC (Local Enhancement): Utilizing NAFNetLocal with TLC-style local enhancement to reconcile local statistics with global context, stabilizing restoration under complex illumination.
×8 Self-Ensemble: The input is transformed via 8 combinations of vertical flip, horizontal flip, and transpose. Predictions are mapped back and averaged to improve stability.
Weighted Snapshot Fusion: Instead of relying on a single checkpoint, the model fuses outputs from three checkpoints (80K, 100K, and 200K iterations).
- Weights: 0.04 (80K), 0.01 (100K), and 0.95 (200K).
- Rationale: The final checkpoint (200K) receives the dominant weight as it best fits the target domain, while earlier snapshots act as mild regularizers against specific artifacts.

3. Key Contributions

Domain-Consistent Data Screening: Introduced a CLIP-based filtering mechanism to construct an extended training set that mitigates target-domain data scarcity while minimizing domain shift from external sources.
Stage-Wise Training Framework: Demonstrated that a two-stage optimization process (Target Adaptation $\rightarrow$ Domain Expansion) is more effective than joint training or simple data mixing for small-sample scenarios.
Practical Pipeline: Showed that a coordinated design of data organization and inference strategies can achieve robust results without the need for heavy backbone redesign or complex architectural changes.

4. Experimental Results

The method was evaluated on the NHM-20 public benchmark (20 aligned image pairs) and the official NTIRE 2026 hidden test set.

Quantitative Performance (NHM-20):
- Luminance Recovery: The method achieved significant improvements in Y-channel PSNR (from 23.47 baseline to 24.89 with Weighted Ensemble) and SSIM (from 0.887 to 0.905).
- Limitations: RGB-channel metrics and LPIPS (perceptual quality) did not surpass the hazy input baseline. This indicates the pipeline is highly effective at recovering brightness structure but struggles with color fidelity and perceptual naturalness.
Qualitative Analysis: Visual comparisons show stable brightness structure recovery and effective removal of haze, though some color artifacts remain.
Ensemble Effect: The weighted snapshot fusion stabilized performance across diverse scenes, though the 100K checkpoint yielded the best PSNR and the 80K the best SSIM individually, highlighting the trade-off managed by the fusion weights.

5. Significance

This paper provides a practical and effective solution for low-data regimes in image restoration. It highlights that in scenarios like nighttime dehazing, where paired data is scarce and degradation is complex, data curation and training strategy are often more critical than increasing model complexity. The proposed framework offers a reproducible, robust pipeline that balances structural recovery with domain adaptation, serving as a strong baseline for future research in adverse weather and low-light image restoration.