🤖 AI

Zero-TIG: Temporal Consistency-Aware Zero-Shot Illumination-Guided Low-light Video Enhancement

This paper presents Zero-TIG, a novel zero-shot learning framework that leverages Retinex theory and optical flow-based temporal feedback to enhance low-light and underwater videos with improved visual quality, temporal consistency, and color fidelity without requiring paired ground truth data.

Original authors: Yini Li, Nantheera Anantrasirichai

Published 2026-04-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yini Li, Nantheera Anantrasirichai

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to watch a movie, but the projector is broken. The screen is pitch black, the image is grainy with static, and the colors look weird—like everything is underwater or covered in a gray fog. Now, imagine trying to watch a series of these movies where the flickering is so bad it hurts your eyes.

This is the problem Zero-TIG solves. It's a new computer program designed to take dark, noisy, shaky videos and turn them into clear, bright, smooth masterpieces—without ever having seen a "perfect" version of the video before.

Here is how it works, broken down into simple stories and analogies:

1. The Problem: The "Blind Painter" Dilemma

Usually, to teach a computer how to fix a dark photo, you show it thousands of pairs: a "bad" photo and the "perfect" version next to it. The computer learns by copying the perfect one.

But in the real world (especially underwater or in deep night), we rarely have those "perfect" reference photos. It's like trying to teach an artist to paint a sunset by only showing them a black-and-white sketch, without ever showing them the real sunset. Most existing methods fail here because they are "blind" without that reference data.

Zero-TIG is different. It's a Zero-Shot learner. It doesn't need a reference. It figures out how to fix the video just by looking at the bad video itself, using the rules of physics and light.

2. The Core Idea: The "Magic Flashlight" (Retinex Theory)

The paper uses a concept called Retinex Theory. Imagine you are in a dark room with a flashlight.

The Object: The thing you are looking at (a red apple). This is the Reflectance. It stays the same color no matter how you shine the light.
The Light: The beam of the flashlight. This is the Illumination. It changes depending on how you move the light.

Zero-TIG acts like a smart detective. It looks at the dark, noisy video and says, "Okay, I need to separate the object from the bad lighting." It strips away the darkness (the bad light) and cleans up the object, then shines a new, perfect light on it.

3. The Two-Part Engine

The system has two main parts working together:

Part A: The "Cleanup Crew" (Enhancement Module)

Think of this as a team of three specialists cleaning a dirty window:

The Dust Remover (LD-Net): First, they wipe away the heavy dust and static noise so the image is clearer.
The Light Analyst (IE-Net): Next, they figure out exactly how the light is hitting the scene. They separate the "shadow" from the "object."
The Polisher (RD-Net): Finally, they polish the object itself, removing any remaining smudges and making the colors pop.

Part B: The "Steady Hand" (Temporal Feedback Module)

This is the paper's biggest innovation.
If you just fix every frame of a video one by one, the result looks like a strobe light. One frame is bright, the next is dim, the next is a different color. It's flickering.

Zero-TIG solves this with a Time-Traveling Feedback Loop:

Imagine you are painting a mural. You finish one section (Frame 1).
Before you start the next section (Frame 2), you look at your finished work from Frame 1.
You use Optical Flow (a way to track how pixels move, like tracking a ball flying through the air) to see exactly how the scene moved from Frame 1 to Frame 2.
You "warp" (stretch and align) your finished Frame 1 to match the position of Frame 2.
You use that aligned, perfect Frame 1 as a guide to help you paint Frame 2.

The Analogy: It's like a dance instructor. Instead of teaching each dancer a move in isolation, they watch the previous dancer's smooth motion and use that rhythm to guide the next dancer. This ensures the video flows smoothly without jerky jumps or flickering lights.

4. The Underwater Challenge: The "Blue Filter"

When filming underwater, light gets absorbed. Red light disappears first, leaving everything looking blue or green. It's like looking through a thick blue swimming pool filter.

Zero-TIG has a special trick for this. It doesn't just guess the colors; it calculates the "white balance" for each color channel (Red, Green, Blue) separately. It's like having a smart filter that knows, "Oh, the Red channel is missing, so I'll boost it specifically to make the coral look pink again, not just make the whole image brighter."

5. The Result: A Smooth, Clear Movie

The paper tested this on dark videos and underwater footage.

Without training data: It didn't need a "teacher" with perfect videos.
No flickering: The video plays smoothly, frame-to-frame.
Better than the pros: In many cases, it looked better than expensive, supervised methods that do have perfect training data.

Summary

Zero-TIG is like a smart, self-teaching video editor that:

Separates the object from the bad lighting.
Cleans up the noise.
Remembers what the previous frame looked like to ensure the next frame matches perfectly (no flickering).
Fixes underwater colors automatically.

It turns a shaky, dark, grainy mess into a cinematic masterpiece, all by teaching itself how to see the light.

1. Problem Statement

Low-light and underwater videos suffer from significant degradation, including poor visibility, low contrast, high noise, motion blur, and color distortion (particularly in underwater environments due to wavelength-dependent light attenuation).

Limitations of Existing Methods: Most state-of-the-art approaches rely on supervised learning requiring paired ground-truth datasets (low-light vs. normal-light), which are scarce, especially for underwater scenes.
Temporal Inconsistency: Applying image-based enhancement methods frame-by-frame to videos often results in temporal flickering and artifacts.
Zero-Shot Gap: While zero-shot methods exist for images, they often fail to maintain temporal continuity or handle the specific color distortions of underwater data effectively without paired training data.

2. Methodology: Zero-TIG

The proposed Zero-TIG framework is a self-supervised, zero-shot learning approach that requires only a single input video for training. It integrates Retinex theory with a novel Temporal Feedback Module. The architecture consists of two primary components:

A. Enhancement Module

Based on the Zero-IG framework, this module processes the input low-light image ( $I_t$ ) through three subnetworks:

Low-Light Denoising Network (LD-Net): Performs initial denoising using two downsamplers ( $G_1, G_2$ ) to generate a preliminary denoised image ( $I_t^{LP}$ ). This mitigates noise before illumination estimation.
Illumination Estimation Network (IE-Net): Decomposes the denoised image into Reflectance ( $R_t^{IE}$ ) and Illumination ( $S_t^{IE}$ ) components based on Retinex theory ( $I_t^{LP} = R_t^{IE} \cdot S_t^{IE}$ ).
Reflection Denoising Network (RD-Net): Refines the concatenated reflectance and illumination maps to remove residual noise, producing the final enhanced reflectance ( $R_t^{RD}$ ) and illumination ( $S_t^{RD}$ ).

B. Temporal Feedback Module

To ensure temporal consistency and reduce flickering, this module recursively incorporates information from the previous frame ( $t-1$ ) into the current frame ( $t$ ):

Preprocessing: The current frame is pre-denoised ( $I_t^{LD}$ ) and undergoes Histogram Equalization (HE) to align intensity distributions.
Optical Flow (OF) Estimation: A pre-trained RAFT network computes the optical flow displacement map between the previous enhanced output ( $R_{t-1}^{RD}$ $R_{t - 1}^{R D}$ ) and the pre-processed current frame ( $\hat{I}_t^{LD}$ $\hat{I}_{t}^{L D}$ ).
- Optimization: Images are downsampled by a factor of 3 to reduce noise impact on OF estimation and accelerate computation.
Warping & Fusion: The previous enhanced reflectance and illumination maps are warped to the current frame using the computed optical flow. These warped maps are concatenated with the current frame's features and fed back into the IE-Net and RD-Net for continuous refinement.
- Initialization: For the first frame, warped maps are initialized as zero vectors.

C. Loss Functions & Underwater Adaptation

The model is optimized using 11 loss functions, including residual, consistency, smoothness, and variance losses.

Adaptive White Balance for Underwater Data: Standard losses assume uniform RGB channel distributions, which fails underwater due to light absorption. Zero-TIG introduces channel-specific constraints:
- It calculates the mean luminance ( $Y_L$ ) for each RGB channel independently.
- The illumination constraints ( $L_{over}$ and $L_{pix}$ ) are extended to sum over channels ( $c \in \{R, G, B\}$ ), allowing the network to adaptively balance colors and achieve automatic white balance without ground truth.

3. Key Contributions

Zero-Shot Video Enhancement: A novel method that achieves high-quality low-light video enhancement without paired training data, addressing the scarcity of ground-truth datasets.
Temporal Feedback Mechanism: A unique module combining histogram equalization, optical flow (RAFT), and image warping to recursively align previous frames with current frames, significantly reducing flickering and noise.
Underwater Color Correction: An adaptive white balance strategy that computes channel-specific gains to correct wavelength-dependent color distortion in underwater videos.
Comprehensive Optimization: Integration of 11 loss functions to jointly optimize denoising, illumination estimation, and temporal consistency.

4. Experimental Results

The method was evaluated on the BVI-RLV (low-light) and BVI-Coral (underwater) datasets.

Quantitative Performance (Low-Light):
- Compared against state-of-the-art self-supervised methods (Zero-DCE, SGZSL, RUAS, Zero-IG, EnlightenGAN) and supervised upper bounds (BVI-CDM, BVI-Mamba).
- Metrics: Achieved competitive PSNR (19.34 dB) and SSIM (0.790) scores. With histogram matching, it reached 28.052 PSNR and 0.854 SSIM, outperforming most self-supervised baselines and approaching supervised methods.
- LPIPS: Achieved a low LPIPS score (0.360), indicating high perceptual similarity to ground truth.
Temporal Consistency:
- Evaluated using Mean Absolute Brightness Differences (MABD). Zero-TIG demonstrated significantly lower MABD values compared to Zero-IG, confirming superior stability and reduced flickering over time.
Underwater Performance:
- Evaluated using non-reference metrics (UIQM, UCIQE, CLIP-IQA).
- Zero-TIG outperformed Zero-IG in all metrics, producing clearer edges, better contrast, and effective color correction for coral and sand textures.

5. Significance

Practical Applicability: By eliminating the need for paired ground-truth data, Zero-TIG is highly applicable to real-world scenarios where capturing reference images is impossible (e.g., deep-sea exploration, surveillance in darkness).
Temporal Stability: The introduction of the temporal feedback module solves a critical bottleneck in video enhancement—flickering—making the output suitable for continuous monitoring and automated decision-making.
Versatility: The framework successfully handles both general low-light conditions and the specific physical challenges of underwater imaging (color attenuation), offering a unified solution for diverse visual enhancement tasks.

Code Availability: The source code is publicly available at https://github.com/liyinibristol/Zero-TIG.