Event-Illumination Collaborative Low-light Image Enhancement with a High-resolution Real-world Dataset

Technical Summary: Event-Illumination Collaborative Low-light Image Enhancement (EIC-LIE)

1. Problem Statement

Existing event-based low-light image enhancement (LIE) methods face three primary limitations:

1. Neglect of Global Illumination: Current approaches heavily rely on direct feature fusion to recover High Dynamic Range (HDR) details from event streams but often overlook the essential global illumination information emphasized in Retinex-based LIE techniques.
2. Event Noise Sensitivity: Event signals are inherently sensitive to noise in low-light conditions due to the scarcity of available photons. Existing strategies, such as fixed Signal-to-Noise Ratio (SNR) maps, lack reliability and often result in noticeable noise in enhanced outputs.
3. Lack of High-Quality Datasets: There is a scarcity of high-resolution, real-world event-image datasets with ground truth (GT). Existing datasets (e.g., SDE) suffer from low resolution, temporal misalignment, color inaccuracies, and restrictions to static or slow-moving scenes.

2. Methodology

The authors propose EIC-LIE, a framework designed to facilitate collaborative interaction between event data and illumination priors while dynamically filtering noise.

A. Event-Illumination Collaborative Interaction (EICI)
This module establishes bidirectional feature transmission between illumination and event representations through two processes:

• Forward Gathering: Utilizes a covariance-based cross-attention mechanism to gather HDR features from event streams and global illumination priors into a shared latent representation. This allows the primary feature to selectively absorb relevant contextual information.
• Backward Injection: Reuses the attention matrix computed during the gathering phase to decompose the fused latent features back into modality-specific components (illumination and event). This ensures that complementary information is shared while preserving modality integrity, enabling continuous mutual refinement.

B. Illumination-Aware Event Filter (IAEF)
To address the noise sensitivity of event signals, the authors introduce a filter that leverages global illumination statistics derived from RGB images:

• Illumination-aware Kernel Extraction: Calculates filter kernels based on illumination features, encoding lighting priors to distinguish between noise and meaningful event responses.
• Event-driven Weight and Offset Extraction: Dynamically predicts weights and spatial offsets for event features. This allows the filter to adaptively reference neighboring events, improving robustness against noise and misaligned event locations.

C. Data Acquisition System
To overcome dataset limitations, the authors constructed a beam-splitter-based hybrid imaging system:

• Hardware: Uses dual beam splitters to coaxially align two high-resolution RGB cameras (Sony IMX273, 1440×1080) and an advanced event camera (Sony&Prophesee IMX636, 1280×720).
• Synchronization: Employs an external controller for precise temporal synchronization and optical sleeves for spatial alignment.
• Output: Captures synchronized low-light images, normal-light images, and event streams across dynamic scenes.

3. Key Contributions

1. EIC-LIE Framework: A novel framework that integrates event and illumination information through collaborative bidirectional interaction (EICI) and illumination-aware noise filtering (IAEF).
2. Algorithmic Innovations:
   • Design of the EICI module to merge HDR features with global illumination priors and complement modal-specific content.
   • Development of the IAEF to dynamically filter event noise using brightness statistics from images.
3. RLE Dataset: Construction of the RLE (Real-world Low-light Event) dataset, the first high-resolution (1024×768), real-world event-based LIE dataset featuring high-quality, time-synchronized event-image pairs in both indoor and outdoor dynamic scenes.

4. Experimental Results

The authors evaluated EIC-LIE on five datasets (RLE, SDE, and SDSD) against state-of-the-art (SOTA) methods, including image-based approaches (e.g., Retinexformer, MambaLLIE) and event-based approaches (e.g., EvLight, ELIE).

• Performance on RLE: EIC-LIE achieved a PSNR of 23.63 dB and SSIM of 0.7670, surpassing the previous best (EvLight) by 0.95 dB in PSNR and 0.0469 in SSIM.
• Performance on SDE: On the SDE-indoor/outdoor datasets, the method improved PSNR by up to 1.01 dB and SSIM by 0.0695 compared to EvLight.
• Performance on SDSD: Achieved improvements of 1.24 dB in PSNR and 0.0312 in SSIM over EvLight.
• Efficiency: The model requires significantly fewer parameters (2.13M) compared to EvLight (22.73M), demonstrating higher efficiency.
• Qualitative Analysis: Visual comparisons show that EIC-LIE produces images with more coherent edges, natural colors, and effective noise suppression compared to methods that exhibit noise interference or smooth, detail-less edges.

5. Significance and Claims

The paper claims that EIC-LIE addresses the critical gap in existing event-based LIE methods by effectively integrating global illumination priors with HDR event data. By introducing the RLE dataset, the authors provide a robust benchmark that overcomes the resolution and alignment limitations of previous datasets. The results demonstrate that collaborative interaction between modalities, guided by illumination statistics, significantly enhances image quality in real-world low-light scenarios, outperforming current SOTA methods in both quantitative metrics and visual fidelity.