💻 computer science

AD-Relight: Training-Free Banner Relighting via Illumination Translation with Diffusion Priors

AD-Relight is a novel, training-free framework that leverages diffusion priors to adaptively relight custom Photoshop-generated ad banners, enabling their seamless and realistic integration into diverse scene lighting conditions without requiring millions of training images.

Original authors: Rameshwar Mishra, A V Subramanyam

Published 2026-04-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Rameshwar Mishra, A V Subramanyam

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a director filming a scene in a cozy, sunlit living room. The light is streaming in from the left, casting long, soft shadows across the wooden floor. Suddenly, you want to paste a digital advertisement for a new sneaker right onto that floor.

If you just "stick" the image on there (like a sticker), it looks fake. The sneaker is flat, bright, and ignores the shadows. It looks like a cartoon cutout floating on a real floor. This is what most current advertising software does.

On the other hand, if you try to use a super-smart AI artist to "paint" the sneaker into the scene, the AI might get confused. Because it was trained mostly on people's faces (portraits), it might try to make the sneaker look like a face, or it might paint the sneaker with the wrong color of light, making it look like it belongs in a dark basement instead of a sunny room.

Enter AD-Relight.

The authors of this paper created a new tool called AD-Relight to solve this specific problem: making digital ads look like they actually belong in the lighting of the real world, without needing to retrain the AI from scratch.

Here is how it works, broken down into three simple steps:

1. The "Texture Match" (Shade Alignment)

First, the tool looks at the floor where the ad is going. It asks, "Is the floor rough? Is it shiny? Is it dark on the left and bright on the right?"

The Analogy: Imagine you are putting a new sticker on a wall. Before you stick it, you rub the sticker with a cloth to make it look as dusty or shiny as the wall behind it.
What AD-Relight does: It takes the flat, Photoshop-made ad and gently "dusts" it with the texture and shadow patterns of the floor it's being placed on. This ensures the ad isn't just a flat image; it has the same "roughness" and shadow direction as the floor.

2. The "Light Detective" (Differential Probing)

This is the cleverest part. The tool uses a pre-existing, super-smart AI (called IC-Light) that is great at lighting up faces but bad at lighting up floor ads. Instead of asking the AI to do the whole job, the tool uses a "detective" trick.

The Analogy: Imagine you want to know how much a specific lamp contributes to a room's brightness. You take a photo of the room with the lamp on, then take another photo with the lamp off. By subtracting the second photo from the first, you isolate exactly what that one lamp added.
What AD-Relight does: It asks the smart AI, "What does this room look like with the floor?" and then "What does it look like without the floor?" The difference between those two answers tells the tool exactly how the light hits that specific spot. It steals this "lighting secret" from the AI without needing to teach the AI anything new.

3. The "Final Polish" (Relighting)

Now, the tool takes the "lighting secret" it just stole and applies it to the ad.

The Analogy: It's like taking a black-and-white photo and using a projector to shine the exact same colored light onto it that was in the original room.
What AD-Relight does: It combines the "lighting secret" with the texture-matched ad. It also adds a soft shadow underneath the ad, so it doesn't look like it's floating. The result is an ad that looks like it was physically there, lit by the same sun or lamp as the rest of the room.

Why is this a big deal?

No New Training: Usually, to teach an AI to light up ads, you'd need to feed it millions of pictures of ads on floors. That takes years and huge computers. AD-Relight does this without any new training. It just uses the existing AI's brain in a smarter way.
Better Results: The paper tested this against other methods.
- Simple "Stickers": Looked fake and flat.
- Other AI methods: Looked weird or changed the ad's design.
- AD-Relight: Looked real.
Human Proof: When real people were asked to look at the results, they consistently picked AD-Relight as the one that looked most realistic, especially in tricky lighting situations (like a shiny wooden floor with strong shadows).

In short: AD-Relight is a clever trick that takes a smart AI designed for faces, tricks it into revealing how light hits a floor, and uses that information to make digital ads look like they truly belong in the scene. It's the difference between a sticker on a wall and a painting that looks like it's part of the wall.

1. Problem Statement

The paper addresses a critical gap in personalized advertising for streaming services: the realistic integration of Photoshop-generated ad banners into video frames.

Current Limitations: Existing ad-placement pipelines typically rely on simple geometric warping, which ignores the scene's lighting conditions, resulting in banners that look "pasted on" with inconsistent shading.
Diffusion Model Failures: While state-of-the-art diffusion-based object insertion and relighting models (e.g., IC-Light, SpotLight) exist, they struggle with ad banners. These models are trained primarily on portrait-style objects or specific lighting distributions. When applied to floor-level banners with complex, non-uniform lighting gradients, they often fail to preserve the banner's identity, reproduce the correct lighting direction, or avoid generating artifacts (e.g., reproducing floor textures instead of scene lighting).
Training Data Scarcity: Training a new diffusion model specifically for ad banners would require millions of labeled images, which is computationally expensive and labor-intensive.

Goal: Develop a training-free framework that can relight custom ad banners to match the specific illumination, shading, and shadows of the target scene without requiring additional model training.

2. Methodology: AD-Relight

The authors propose AD-Relight, a novel, three-stage, training-free framework. It leverages a pre-trained, publicly available diffusion relighting model (IC-Light) as a backbone but adapts its behavior at test time through a differential probing strategy.

Stage 1: Shade Alignment

Before relighting, the method ensures the banner's base shading matches the target region.

Texture Simulation: Since real-world banners are printed on textured materials, a pre-trained diffusion model generates a texture map ( $T$ ) which is alpha-blended with the input logo ( $I_L$ ).
Retinex-based Decomposition: Inspired by Retinex theory, the method decomposes the input region ( $I_O$ $I_{O}$ ) and the banner ( $I_L$ $I_{L}$ ) into Illuminance (lighting) and Reflectance (structure) components.
- The illuminance channel is factorized into a low-frequency shading component ( $S$ ) and a high-frequency structural component ( $S_t$ ) using a Gaussian filter.
Transfer: The shading component ( $S_O$ ) of the original scene is transferred to the structural component of the banner ( $S_{L,t}$ ) to create a new illuminance map ( $I_{L,ill}$ ). This ensures the banner inherits the spatial shading gradients of the target area.

Stage 2: Light Feature Estimation (Differential Probing)

This is the core innovation. Instead of training a new model, the framework "probes" the pre-trained IC-Light model to extract region-specific lighting priors.

Hypothesis: Diffusion models exhibit locally linear behavior regarding illumination changes.
Process:
1. Input A: The full frame ( $B_O$ ) containing the target region.
2. Input B: A masked frame ( $B_M$ ) where the target region is removed.
3. Inference: Both inputs are passed through the IC-Light backbone to generate outputs $Out_1$ and $Out_2$ .
Differential Feature ( $\epsilon$ ): The difference between the two outputs ( $\epsilon = Out_1 - Out_2$ $ϵ = O u t_{1} - O u t_{2}$ ) is calculated.
- Mathematically, this isolates the illumination contribution ( $L_\phi^O$ ) of the specific target region, effectively extracting a spatially aligned lighting residual.
- This $\epsilon$ serves as a reusable, region-specific lighting prior without needing to retrain the model.

Stage 3: Relighting the Custom Logo

The extracted lighting prior is used to relight the banner.

Background Construction: A weighted combination of the differential feature ( $\epsilon$ ) and a low-frequency light gradient ( $G$ ) derived from the original region is used as the background input ( $B_\epsilon$ ) for the relighting model. This balances global trends with specific region details.
Soft Shadow Estimation: To enhance realism, an adaptive shadow map is generated using Otsu's method on the illuminance map. This creates a smooth attenuation factor ($df$) that is applied to the relit logo, simulating soft shadows without hard transitions.
Final Output: The relit logo is blended with the shadow map and the original output to produce the final seamless integration.

3. Key Contributions

Problem Identification: The paper highlights the specific failure of existing diffusion models and warping techniques in handling floor-level ad banners with complex lighting gradients.
AD-Relight Framework: A novel, three-stage, training-free approach that adapts a pre-trained diffusion model at test time. It uniquely combines Retinex-based shading transfer with a differential probing strategy to extract illumination priors.
Theoretical Formulation: Provides a principled mathematical formulation for the "differential illumination feature," demonstrating how to isolate lighting contributions from a black-box diffusion model.
Comprehensive Evaluation: Extensive quantitative and qualitative benchmarks, including a user study, proving superiority over existing methods.

4. Results and Evaluation

Quantitative Results

The method was evaluated on a benchmark of 560 cases (7 logos, 80 frames) against four baselines: Simple Warping, TF-ICON, IC-Light, and SpotLight.

Metrics: SSIM (Structural Similarity), LPIPS (Perceptual Loss), and ILL-SIM (Illumination Cosine Similarity).
Performance: AD-Relight achieved the best scores across all metrics:
- SSIM: 0.95 (vs. 0.92 for SpotLight).
- LPIPS: 0.03 (lower is better; vs. 0.07 for IC-Light).
- ILL-SIM: 0.92 (vs. 0.79 for SpotLight), indicating superior lighting consistency.

LLM and User Studies

GPT-4o Evaluation: An LLM-based assessment ranked AD-Relight as the preferred method in 90–100% of cases across criteria like light gradient, consistency, and scene realism.
Human User Study: 25 participants evaluated 324 responses. Users consistently preferred AD-Relight outputs, particularly in challenging scenarios involving strong light gradients and floor placements.

Ablation Studies

The authors validated the necessity of each stage. Removing the differential feature ( $\epsilon$ ), the shading alignment, or the gradient blending resulted in significant drops in SSIM and ILL-SIM, confirming that all three stages are critical for high-quality results.

5. Significance and Impact

Training-Free Efficiency: By avoiding the need for millions of training images, AD-Relight offers a computationally efficient solution that can be deployed immediately using existing open-source models.
Realism in Advertising: It solves a major bottleneck in programmatic advertising, allowing for the seamless integration of custom banners that respect the physics of the scene (lighting direction, intensity, and shadows).
Generalizability: The differential probing technique could potentially be applied to other tasks requiring the extraction of specific scene properties from pre-trained generative models without fine-tuning.
Limitations & Future Work: The current work focuses on single-frame relighting. The authors note that temporal coherence for video is a future direction, as the current method does not explicitly enforce consistency across frames.