Technical Summary: ControlLight

Problem Statement

Existing deep learning-based low-light enhancement (LLE) methods face two primary limitations in real-world applications:

1. Lack of Controllability: Most models are trained on datasets where each low-light image has a single, fixed supervision target (a specific "normal-light" ground truth). This forces the model to learn a fixed enhancement strength, preventing users from flexibly adjusting the brightness or restoration intensity to suit personal preferences or varying scene requirements.
2. Generalization and Consistency Issues: While large-scale generative image editing models (e.g., FLUX.2-klein) demonstrate strong generalization and the ability to recover plausible details, they often suffer from hallucinations, structural distortions, and pixel shifts. Furthermore, standard fine-tuning of these models for continuous control often leads to inconsistent outputs or structural drift when the enhancement strength is varied. Additionally, existing continuous control methods often lack reliable intermediate supervision, leading to unstable trajectories.

Methodology

The authors propose ControlLight, a framework designed to achieve controllable, consistent, and generalizable low-light enhancement. The methodology consists of three core components:

1. Light100K: Continuous Pseudo-Paired Data Construction

To address the scarcity of real-world paired data with continuous supervision, the authors constructed Light100K.

• Data Collection: Approximately 30,000 high-quality real-world low-light images were collected and filtered for semantic relevance and degradation levels.
• Pseudo-Target Generation: A pretrained FLUX.2-klein-9B model was used to generate "normal-light" counterparts ($I_1$) for each low-light input ($I_0$).
• Edge-Consistency Filtering: Pairs with significant structural misalignment were removed, resulting in ~20,000 high-consistency pairs.
• Retinex-Inspired Interpolation: Instead of simple RGB alpha blending (which flattens local contrast and mixes illumination with reflectance), the authors employed a Retinex-based interpolation strategy. They decomposed images into Reflectance ($R$) and Illumination ($L$) components. Continuous enhancement was modeled as a smooth transition in the illumination component (log-domain interpolation) while conservatively interpolating the reflectance. This generates a continuous sequence of pseudo-ground-truths ($I_s$) for enhancement strengths $s \in \{0.2, 0.4, 0.6, 0.8\}$.

2. Misalignment-Aware Weighted Flow Matching Loss ($L_{wFM}$)

The authors observed that even visually consistent pseudo-pairs can contain subtle edge misalignments due to the generative nature of the base model. Standard flow matching would force the model to learn and amplify these misalignments, causing structural artifacts.

• Edge Difference Mapping: A structural edge-difference map is computed between the input and the pseudo-target in the log-luminance domain to isolate high-frequency structural discrepancies.
• Weighted Supervision: A spatial weight map is generated where regions with significant edge misalignment (far from input edges) are down-weighted.
• Loss Function: The standard flow matching loss is modified to apply these weights, encouraging the model to preserve the input image's structure while learning the illumination enhancement, thereby reducing structural drift.

3. ControlLight Model Architecture

• Base Model: The framework fine-tunes FLUX.2-klein-9B using Low-Rank Adaptation (LoRA).
• Continuous Control Mechanism: The enhancement strength $s$ is directly incorporated as the LoRA scaling factor ($W' = W + s \cdot AB$). Unlike methods that assume linearity only at inference, ControlLight enforces this linearity during training by pairing specific $s$ values with their corresponding pseudo-ground-truths ($I_s$) and computing the weighted flow matching loss.
• Inference: Users can input a low-light image and a fixed restoration prompt, adjusting $s$ continuously from 0 to 1 to control the enhancement intensity.

Key Contributions

1. Light100K Dataset: A large-scale, continuous low-light enhancement dataset containing real-world degraded images and structure-consistent pseudo-enhanced targets with fine-grained supervision across multiple illumination strengths.
2. Misalignment-Aware Weighted Flow Matching Loss: A novel loss function that identifies and down-weights unreliable edge regions in diffusion-generated pseudo-targets, ensuring structural consistency and preventing the amplification of generative artifacts during continuous enhancement.
3. ControlLight Framework: A controllable LLE model that achieves state-of-the-art performance in both enhancement quality and linear controllability, outperforming existing continuous and non-continuous methods on real-world benchmarks.

Experimental Results

The authors evaluated ControlLight on five benchmarks: LOL-v1 and LWSR (paired), and DICM, LIME, and RealIR-Bench (non-reference).

• Enhancement Quality: On paired benchmarks, ControlLight achieved state-of-the-art results on most perceptual metrics (CLIPIQA, MANIQA, MUSIQ) and competitive results on NIQE. On real-world non-reference benchmarks, it consistently outperformed all baselines, including traditional methods (Retinexformer, CIDNet) and other diffusion-based approaches (CLE Diffusion).
• Controllability: Evaluated using $\delta_{smooth}$ (trajectory smoothness) and CLIP-Dir (semantic directionality), ControlLight demonstrated superior linear controllability compared to universal continuous editing methods like ConceptSlider, KSlider, and CLE Diffusion. It produced smoother transitions and more semantically consistent enhancement trajectories.
• Ablation Studies:
  • Removing the weighted flow matching loss ($L_{wFM}$) resulted in higher structural inconsistency (higher LI-LPIPS) and lower perceptual quality.
  • Comparing interpolation strategies showed that Retinex-based interpolation provided a more physically reasonable quality gradient and richer degradation cues compared to direct alpha blending.

Significance and Claims

The paper claims that ControlLight establishes a new state-of-the-art for low-light enhancement by successfully bridging the gap between the generative power of large-scale image editing models and the strict structural consistency required for image restoration.

Key claims include:

• Real-World Applicability: By constructing a dataset from real-world images and using a structure-preserving loss, the model generalizes effectively to real-world degradations where traditional methods often fail.
• User-Centric Control: The ability to continuously adjust enhancement strength ($s=0$ to $s=1$) allows users to tailor results to specific preferences without sacrificing visual consistency or realism.
• Structural Integrity: The proposed loss function effectively mitigates the "hallucination" and structural drift common in generative restoration, ensuring that the enhanced image remains faithful to the original scene structure.

The authors conclude that ControlLight offers a robust solution for practical low-light enhancement scenarios, delivering superior consistency, controllability, and generalization compared to existing approaches.