💻 computer science

CANDLE: Illumination-Invariant Semantic Priors for Color Ambient Lighting Normalization

CANDLE is a novel method for color ambient lighting normalization that leverages illumination-robust DINOv3 semantic priors through multi-layer guidance and frequency refinement to effectively recover intrinsic object colors under challenging multi-colored illumination, achieving state-of-the-art performance on the CL3AN dataset and top rankings in the NTIRE 2026 ALN challenge.

Original authors: Rong-Lin Jian, Ting-Yao Chen, Yu-Fan Lin, Chia-Ming Lee, Fu-En Yang, Yu-Chiang Frank Wang, Chih-Chung Hsu

Published 2026-04-07

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Rong-Lin Jian, Ting-Yao Chen, Yu-Fan Lin, Chia-Ming Lee, Fu-En Yang, Yu-Chiang Frank Wang, Chih-Chung Hsu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to take a beautiful photo of a red apple. But instead of normal sunlight, you are standing under a weird, multi-colored disco light. The apple looks purple, the shadows look green, and the shiny spots are blown out white. Your goal is to "fix" the photo so the apple looks red again, the shadows look natural, and the shine is realistic.

This is the problem of Color Ambient Lighting Normalization (ALN). It's like trying to clean a dirty window, but the dirt is made of different colored lights, and the window itself is made of different materials (glass, metal, fabric) that react to light differently.

Here is how the paper CANDLE solves this, explained simply:

1. The Problem: Old Tools Don't Work

Previous methods tried to fix these photos using two main tricks:

The "Geometry" Trick: They looked at the shape of the object (like the curve of the apple) to guess where the light was hitting. But if the light is a crazy neon blue, the shape doesn't tell you what color the apple should be.
The "Frequency" Trick: They tried to separate the image into "blurry" parts (colors) and "sharp" parts (edges). But when the colors are totally wrong, this just makes a mess.

The Analogy: Imagine trying to guess what a person looks like under a red flashlight. If you only look at their shadow (geometry) or the blur of the light (frequency), you can't tell if they are wearing a blue shirt or a green shirt. You need to recognize who they are first.

2. The Big Discovery: "The Brain That Doesn't Blink"

The researchers noticed something amazing about a specific AI model called DINOv3.

Most AI models get confused when the lighting changes color.
DINOv3, however, is trained to recognize objects, not just colors. If you show it a red apple under blue light, it still "knows" it's an apple. It ignores the weird lighting and focuses on the object's true identity.

The Analogy: Think of DINOv3 as a super-smart art critic who has seen every painting in the world. Even if you shine a green laser on a painting of a sunset, the critic still knows, "Ah, that's a sunset, not a green swamp." The researchers realized: If we let this "critic" guide the photo repair, we can fix the colors correctly.

3. The Solution: CANDLE

They built a new system called CANDLE (Color Ambient Normalization with DINO Layer Enhancement). Think of it as a three-step repair shop:

Step A: The "Smart Guide" (D.O.G.)

Instead of guessing the lighting based on shadows, CANDLE asks the "Smart Critic" (DINOv3) for help at every stage of the repair.

How it works: As the computer tries to fix the image, it constantly checks with the critic: "Is this part of the apple? Is this part of the table?"
The Magic: The system uses layers of the critic's brain. Shallow layers help with tiny details (like the texture of the apple skin), and deep layers help with the big picture (making sure the whole apple is red, not purple).
Analogy: It's like having a team of guides. One guide knows the map of the city (the big picture), and another guide knows the specific street signs (the tiny details). They work together to get you home.

Step B: The "Color Filter" (BFACG)

Sometimes, even with a good guide, the computer gets confused in very bright, shiny spots (like a highlight on a car hood). It might accidentally turn the whole car blue.

The Fix: CANDLE splits the work. One part of the system fixes the shape (edges, lines), and a separate part fixes the color. They only mix them together carefully at the end.
Analogy: Imagine painting a room. You use one brush for the walls (color) and a different brush for the trim (edges). If you use the same brush for both, you might get paint on the trim where it doesn't belong. CANDLE keeps the brushes separate until the very last step.

Step C: The "Noise Cleaner" (SFFB)

When computers fix images, they often copy-paste parts of the original bad image to fill in gaps. This can accidentally bring back the bad colored light.

The Fix: CANDLE uses a special filter (like a sieve) to catch the "bad light" before it gets pasted into the final image. It keeps the sharp details but throws away the weird color tints.
Analogy: It's like straining pasta. You want the pasta (the details), but you want to drain out the starchy water (the bad lighting).

4. The Results: A New Champion

The researchers tested CANDLE on a tough challenge called CL3AN (a dataset full of weird, multi-colored lights).

The Score: CANDLE beat the previous best methods by a significant margin. It was like going from a blurry, color-drunk photo to a crisp, natural-looking one.
The Competition: In the NTIRE 2026 Challenge (a big contest for image fixing), CANDLE took 3rd place for the "Color Lighting" track and 2nd place for the "White Lighting" track.
Why it matters: It proved that using "semantic" knowledge (understanding what an object is) is much better than just using "geometric" knowledge (understanding where the light hits) when the lights are crazy.

Summary

CANDLE is a new way to fix photos taken under weird, multi-colored lights. Instead of guessing based on shadows or math, it uses a super-smart AI "critic" (DINOv3) to understand what the objects actually are. It then uses a split-brain approach to fix colors and shapes separately, ensuring the final photo looks natural, sharp, and true to life.

In one sentence: CANDLE teaches a computer to ignore the crazy disco lights and focus on the true identity of the objects, resulting in perfectly restored photos.

1. Problem Statement

Ambient Lighting Normalization (ALN) aims to recover a uniformly illuminated image from observations corrupted by spatially varying, multi-source lighting. While traditional shadow removal focuses on single white-light shadows, Color ALN addresses a significantly more ill-posed problem involving:

Multi-colored illumination: Strong chromatic shifts that alter the perceived color of objects.
Complex interactions: Self-shadows, specular highlights, color spill, and material-dependent reflectance.
Limitations of existing methods: Current approaches rely on geometric priors (e.g., surface normals) or frequency-domain representations. These fail when illumination-induced chromatic bias dominates, as geometric cues cannot determine an object's intrinsic color once the color shift is severe, and frequency methods often conflate illumination shifts with material identity.

2. Key Observation

The authors identify a critical gap in existing priors: Geometric and low-level priors are insufficient under strong chromatic illumination.

Hypothesis: Self-supervised semantic features remain consistent between colored-light inputs and ambient-lit ground truth (GT), whereas geometric or supervised features drift significantly.
Evidence: Experiments comparing DINOv3, CLIP, and ResNet-50 show that DINOv3 features maintain high patch-wise cosine similarity across domains, even in severely corrupted regions. This is attributed to DINOv3's self-supervised training objective, which encourages representations invariant to appearance variations while preserving object- and material-consistent semantic structures.

3. Methodology: CANDLE Framework

The proposed CANDLE (Color Ambient Normalization with DINO Layer Enhancement) framework consists of two core components designed to leverage these illumination-invariant priors:

A. DINO Omni-layer Guidance (D.O.G.)

This module replaces traditional surface-normal injection with adaptive multi-layer semantic guidance.

Feature Extraction: Features are extracted from a frozen DINOv3 ViT-L/16 at layers $L = \{6, 12, 18, 24\}$ .
Prompt Selection Fusion (PSF): Instead of injecting a single fixed layer, PSF adaptively selects and fuses the most relevant DINO layers for each encoder stage. Shallow stages use low-level features for texture boundaries, while deep stages use high-level features for global color consistency.
DINO-Residual Fusion Block (DRFB): The fused semantic features are injected into the backbone via query-based cross-attention. The backbone features act as queries, while the DINO priors act as keys and values. This allows the network to selectively retrieve semantically consistent regions from the prior, conditioned by a learnable scale factor $\gamma$ .

B. Color-Frequency Refinement

Since semantic guidance constrains the encoder but does not fully prevent decoder-side failures (chromatic collapse in highlights and detail contamination via skip connections), CANDLE introduces a refinement module:

Bifurcated Feature-Adaptive Color Guidance (BFACG): Decouples structural and chromatic restoration in the decoder. Intermediate features are routed through parallel branches: one for geometry-consistent detail ( $F_{str}$ ) and one for illumination-invariant color correction ( $F_{chr}$ ). An edge-aware map modulates the fusion, ensuring color correction does not bleed into structural boundaries.
Spectral Feature Fusion Block (SFFB): Addresses illumination bias reintroduced by skip connections. It uses Haar wavelet decomposition to separate skip features into low-frequency (global color bias) and high-frequency (structural detail) subbands. A learned gate selectively suppresses the low-frequency components before reconstructing the feature, preventing illumination leakage.

4. Training Strategy

Loss Function: Combines $L_1$ loss and SSIM loss for structural consistency.
Progressive Pipeline (Color Track):
1. Stage 1: Train full architecture with progressive patch sizes.
2. Stage 2: Freeze backbone, train a lightweight NAFNet refiner for residual correction.
3. Stage 3: Joint fine-tuning of backbone and refiner.
Ensemble Strategy (White Track): Combines CANDLE with other models (e.g., DenseSR) to leverage complementary inductive biases for luminance-dominated scenarios.

5. Experimental Results

Quantitative Performance (CL3AN Benchmark)

PSNR Gain: CANDLE achieves 21.066 dB, surpassing the previous state-of-the-art (RLN2) by +1.22 dB and PromptNorm by +1.85 dB.
Ablation Studies:
- Replacing geometric priors with D.O.G. yielded a massive +1.85 dB gain over geometric guidance alone.
- The full architecture (D.O.G. + BFACG + SFFB) outperformed variants using naive concatenation or single-layer injection.
- D.O.G. alone accounted for a +3.39 dB gain over the unguided baseline.

NTIRE 2026 ALN Challenge Rankings

Color Lighting Track: Ranked 3rd in perceptual ranking (LPIPS) and achieved the 2nd best LPIPS score (0.1964) among top contenders.
White Lighting Track: Ranked 2nd in fidelity (PSNR) and 3rd in perceptual ranking. Notably, it achieved the lowest FID (49.838) among all submissions, indicating superior distribution alignment with real-world data.

6. Significance and Contributions

New Prior Paradigm: The paper establishes that self-supervised semantic features (DINOv3) are superior to geometric or frequency priors for color ALN, providing a robust solution to the "chromatic disentanglement" problem.
Architectural Innovation: The introduction of D.O.G. (adaptive multi-layer semantic injection) and Color-Frequency Refinement (BFACG + SFFB) creates a robust framework that handles both global semantic consistency and local detail/color recovery.
State-of-the-Art Performance: CANDLE sets new benchmarks on the CL3AN dataset and demonstrates strong generalization across both chromatic (colored light) and luminance (white light) dominant conditions, validating the approach's versatility.
Practical Impact: The method successfully addresses real-world challenges like highlight saturation and material-dependent reflectance, which have historically hindered image restoration in complex lighting environments.