BiEvLight: Bi-level Learning of Task-Aware Event Refinement for Low-Light Image Enhancement

BiEvLight is a bi-level learning framework that addresses the noise coupling challenge in low-light image enhancement by dynamically optimizing event denoising as a task-aware prior, thereby significantly improving enhancement quality on real-world noisy datasets.

The Gist

Imagine you are trying to take a beautiful photo of a city street at night. You have two tools:

1. A Standard Camera: It takes a picture, but because it's so dark, the photo is grainy, blurry, and full of "snow" (noise).
2. An Event Camera: This is a special, futuristic sensor that doesn't take pictures. Instead, it acts like a super-fast motion detector. It only "speaks" when something moves or changes brightness. In the dark, it's supposed to be amazing because it doesn't get blurry. However, in the dark, this sensor gets confused. It starts "hallucinating" random sparks of light (noise) because it's trying too hard to see in the dark.

The Problem:
Previous methods tried to combine these two tools by simply gluing them together. They took the noisy event camera data and the grainy photo, mixed them, and hoped for the best.

• The Flaw: If you clean the event camera data too much before mixing, you lose the important details (like the shape of a car). If you don't clean it enough, the "hallucinated" sparks ruin the final photo. It's a "Goldilocks" problem that nobody could solve perfectly because the two tools weren't talking to each other.

The Solution: BiEvLight
The authors of this paper created a new system called BiEvLight. Think of it not as two separate tools working side-by-side, but as a Master Chef and a Sous Chef working in a kitchen with a special feedback loop.

Here is how it works, using simple analogies:

1. The "Master Chef" (The Enhancement Task)

This is the main goal: making the final low-light photo look perfect. The Master Chef is looking at the blurry, dark photo and trying to sharpen it.

2. The "Sous Chef" (The Event Denoising Task)

This is the helper. Its job is to clean up the noisy data from the Event Camera so the Master Chef can use it.

3. The Secret Sauce: "Bilevel Learning" (The Feedback Loop)

In old systems, the Sous Chef cleaned the ingredients once and handed them over. If the Sous Chef cleaned too much, the Master Chef had nothing to work with. If they cleaned too little, the dish was ruined.

BiEvLight changes the rules:

• The Conversation: The Master Chef tastes the dish (the enhanced image) and says, "Hey, Sous Chef, you cleaned the onions too much! I lost the texture. Please keep a little bit of the onion skin next time."
• The Adjustment: The Sous Chef listens, adjusts their cleaning technique, and hands over a slightly different batch of ingredients.
• The Loop: This happens constantly. The Master Chef guides the cleaning, and the cleaner provides better ingredients for the cooking. They learn together, in real-time, to find the perfect balance.

4. The "Flashlight" Strategy (Gradient-Guided Denoising)

How does the Sous Chef know what to keep and what to throw away?

• Imagine the blurry photo has faint outlines of trees and cars (gradients).
• The system uses these faint outlines as a Flashlight.
• If the Event Camera sees a "spark" of noise that doesn't match the outline of a tree in the photo, the Flashlight says, "That's fake! Throw it away."
• If the spark does match the outline, the Flashlight says, "Keep that! That's a real edge."
  This ensures that the noise is removed without erasing the important details.

The Result

By letting the "cooking" (enhancement) and the "cleaning" (denoising) talk to each other, BiEvLight produces results that are:

• Sharper: You can see details in the dark that were previously invisible.
• Cleaner: The "snow" and random sparks are gone.
• More Natural: It doesn't look like a computer generated it; it looks like a real, high-quality photo.

In a nutshell: Instead of treating the noise removal and the image enhancement as two separate, static steps, BiEvLight makes them a dynamic team that constantly helps and corrects each other, resulting in the clearest possible night-vision photos.

Technical Summary

1. Problem Statement

The paper addresses the challenge of Low-Light Image Enhancement (LLIE) in dynamic scenes. While traditional frame-based cameras suffer from motion blur and noise in low-light conditions, event cameras offer a promising alternative due to their high dynamic range and microsecond temporal resolution. However, existing event-based LLIE methods face two critical bottlenecks:

1. Dual Degradation & Noise Coupling: In low-light environments, event cameras generate significant Background Activity (BA) noise due to internal circuit fluctuations and dark current. Simultaneously, images suffer from low Signal-to-Noise Ratio (SNR). Existing methods often fail to address this dual degradation, leading to severe noise coupling during the fusion of event and image data.
2. Static Pre-processing Limitation: Current approaches typically treat event denoising as a static pre-processing step before enhancement. This creates a trade-off dilemma:
   • Over-denoising: Removes critical structural details (edges) needed for enhancement.
   • Under-denoising: Allows residual noise to propagate into the fusion stage, degrading the final image quality.
   • Crucially, this static approach lacks task-aware feedback; the denoised events are not optimized specifically for the downstream enhancement objective.

2. Methodology: BiEvLight Framework

The authors propose BiEvLight, a hierarchical, task-aware framework that reformulates event denoising and image enhancement as a bi-level optimization problem. The framework consists of two main components:

A. Spatially-adaptive Gradient-guided Denoising

To address the lack of guidance in single-modality denoising, the authors leverage the strong gradient correlation between images and events.

• Mechanism: Based on the brightness constancy assumption, events are triggered by motion at object edges, which aligns with image gradients.
• Strategy: The method uses the gradient of the reflectance component (derived from the low-light image) to guide event denoising.
• Spatial Adaptivity: A global threshold is insufficient due to spatial heterogeneity. The authors introduce a spatially-adaptive threshold ($q$) calculated within sliding windows. This allows the model to:
  • Preserve sparse events in smooth regions.
  • Aggressively suppress noise in textured regions.
• Outcome: This creates a "gradient-guided prior" that implicitly filters artifacts while preserving true structural signals.

B. Bi-level Learning of Task-Aware Optimization

Instead of a static pipeline, BiEvLight treats denoising as an upper-level optimization problem constrained by the lower-level enhancement task.

• Lower-Level (Enhancement): Optimizes the enhancement network ($\theta$) to reconstruct high-quality images using the denoised events ($\hat{x}_e$) and low-light images ($x_{low}$).
• Upper-Level (Denoising): Optimizes the denoising network ($w$) to produce events that maximize the performance of the lower-level enhancement task.
• Interaction: The enhancement task provides gain feedback to calibrate the denoising process. If the enhancement network struggles, the denoising network adapts to provide better structural cues. Conversely, refined events provide high-SNR cues to the enhancement branch.
• Optimization: The authors employ a one-step truncated Iterative Differentiation (ITD) strategy to approximate the bi-level gradient, avoiding the computational complexity of explicit Hessian inversion.

3. Key Contributions

1. BiEvLight Framework: A novel task-aware bi-level learning framework that redefines event denoising as an optimization problem constrained by the enhancement task, enabling collaborative optimization rather than sequential pre-processing.
2. Gradient-Guided Denoising: A spatially-adaptive strategy that utilizes image gradients to guide event stream denoising, effectively solving the issue of insufficient denoising in heavily noisy regions without losing structural details.
3. State-of-the-Art Performance: Extensive experiments demonstrate significant improvements over existing methods on real-world datasets.

4. Experimental Results

The method was evaluated on two real-world datasets: SDE (Real-world noise Dataset) and SDSD.

• Quantitative Performance:

  • On the SDE-out dataset, BiEvLight achieved a PSNR of 24.36 dB, outperforming the previous SOTA (EvLight) by 1.92 dB.
  • It achieved a PSNR of 26.16 dB* (improving by 1.73 dB over SOTA) and SSIM of 0.7451.
  • On the SDSD dataset, it also secured the best results, with a PSNR improvement of 1.41 dB over EvLight on the indoor (SDSD-in) task.
  • Overall, the method showed average improvements of 1.30 dB in PSNR, 2.03 dB in PSNR*, and 0.047 in SSIM across datasets.

• Qualitative Results:

  • Visual comparisons show BiEvLight reconstructs clearer edges in dark areas compared to methods like ELIE, EvLight, and URWKV.
  • Event Visualization: The denoised events produced by BiEvLight clearly reveal text and structural details that were completely obscured by noise in the raw event streams, validating the effectiveness of the gradient-guided strategy.

• Ablation Studies:

  • Removing the bi-level learning (using joint or alternating training) resulted in lower performance due to gradient conflicts or lack of task interaction.
  • Removing the spatially-adaptive gradient guidance led to over-smoothing or residual noise.

5. Significance

BiEvLight represents a paradigm shift in event-based low-light enhancement. By moving away from the "denoise-then-enhance" static pipeline, it establishes a dynamic, bidirectional feedback loop between denoising and enhancement. This ensures that the event data is refined specifically to serve the needs of the enhancement task, effectively breaking the performance bottleneck caused by noise coupling and structural loss. The work provides a robust solution for autonomous driving and surveillance applications operating in challenging, dynamic low-light environments.