💻 computer science

DANTE-W: Diffuse Albedo Neural Texturing in the Wild

DANTE-W is a neural texturing framework that recovers high-fidelity, relightable diffuse albedo textures from unstructured in-the-wild image collections by fusing view-space generative priors with physically principled neural rendering to overcome the shading and shadow limitations of classical mesh texturing.

Original authors: Guangyu Wang, Tianheng Lu, Ruqi Huang, Lu Fang

Published 2026-07-01

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Guangyu Wang, Tianheng Lu, Ruqi Huang, Lu Fang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a box of hundreds of photos taken of a beautiful, old stone pavilion on a sunny day. You want to turn these photos into a 3D model that you can spin around, zoom in on, and even change the time of day for (like turning noon into sunset).

The problem with current technology is that when you build this 3D model, the "skin" or texture you paint onto it gets the sunlight and shadows baked in. It's like taking a photo of a person wearing a hat, printing that photo onto a 3D mannequin, and then trying to move the mannequin into a dark room. The hat's shadow is still permanently stuck on the mannequin's face, making it look weird and unrealistic.

DANTE-W is a new tool that fixes this. Here is how it works, using some simple analogies:

1. The "Magic Paint" vs. The "Real Stone"

Think of the photos you took as a mix of two things:

The Stone: The actual color and texture of the object (like the roughness of the brick or the pattern of the wood). This is called Albedo.
The Sunlight: The shadows, the bright highlights, and the way the light hits the object. This is called Irradiance.

Old methods just mashed these two together. If you tried to change the lighting later, the "baked-in" shadows would stay, ruining the effect.

2. The "Smart Assistant" (Diffusion Priors)

The researchers used a very smart AI assistant (a "diffusion model") that is great at guessing what an object's true color should be, even if it's in the dark or in shadow.

The Analogy: Imagine you are trying to guess the color of a shirt in a dark room. Your smart assistant looks at the photo and says, "I think that's actually a red shirt, not a dark purple one."
The Problem: If you just listen to the assistant blindly, it might get the details wrong or make the shirt look blurry because it's guessing based on 2D photos, not a real 3D object. It might also "hallucinate" (make up) details that don't exist.

3. The "Frequency Filter" (The Secret Sauce)

This is the clever part of DANTE-W. The researchers realized that shadows and sunlight are usually smooth and blurry (low frequency), while fine details like cracks, scratches, and patterns are sharp and crisp (high frequency).

The Analogy: Imagine you have a smooth, blurry painting of a landscape (the sunlight) and a sharp, high-definition photograph of a leaf (the texture).
The Solution: DANTE-W separates these two. It uses the smart assistant to figure out the smooth lighting and shadows. Then, it looks at the original photos to grab the sharp details. It forces the "sharpness" to belong to the texture and the "smoothness" to belong to the light.

4. The Result: A "Re-lightable" 3D World

By doing this separation, DANTE-W creates a 3D model where the texture is pure "stone color" without any baked-in shadows.

Why it matters: Now, if you want to move your 3D pavilion from noon to midnight, the shadows disappear and new, realistic shadows appear based on the new moonlight. The stone still looks like stone, not like a flat photo.

How Fast and Good Is It?

Speed: It's surprisingly fast. It can process a huge scene with thousands of photos in about 5 to 8 minutes on a standard gaming computer. That's more than 7 times faster than the old standard tools.
Quality: In tests, it produced much clearer images and fewer "ghostly" errors than other AI methods. It even created a new dataset called GigaLit (a collection of tricky, real-world scenes) to prove it works better than anything else currently available.

In short: DANTE-W is a tool that takes messy, sun-drenched photos and separates the "object" from the "light," allowing you to create 3D models that look real and can be lit up from any angle, just like in a movie.

Technical Summary: DANTE-W: Diffuse Albedo Neural Texturing in the Wild

1. Problem Statement

Reconstructing high-quality textured 3D assets from unstructured, in-the-wild image collections remains a significant challenge, particularly for large-scale scenes. Existing approaches suffer from two primary limitations:

Classical Mesh Texturing: Traditional methods (e.g., multi-view stereo followed by texture optimization) directly blend captured pixel colors. This results in textures with "baked-in" shading and shadows, compromising visual fidelity when the scene is relit under novel lighting conditions.
Intrinsic Decomposition & Diffusion Methods: While recent neural inverse rendering and diffusion-based generative models show promise in separating lighting from reflectance, they often fail in complex, large-scale scenarios. They struggle with 3D consistency across viewpoints, lose fine-grained details due to the probabilistic nature of generative modeling and VAE information loss, and are often incompatible with standard graphics pipelines.

The core problem addressed is the need for a framework that can recover high-fidelity, 3D-consistent diffuse albedo textures from unstructured image collections, effectively disentangling surface reflectance from illumination effects to enable realistic relighting.

2. Methodology: DANTE-W

The authors propose DANTE-W, a neural texturing framework that integrates seamlessly with traditional 3D reconstruction pipelines. Given a reconstructed mesh and its surface parameterization, the method operates through the following stages:

2.1 View-Space Albedo Priors

To address the ill-posed nature of intrinsic decomposition, the method leverages state-of-the-art diffusion models (e.g., RGB↔X, DiffusionRenderer) to generate initial albedo estimates from input images. These view-space priors are lifted into a coherent 2D neural texture space defined on the UV coordinates of the mesh. This is achieved by minimizing the difference between the neural texture's output and the screen-space albedo generated by the diffusion models.

2.2 Physically Principled Neural Rendering

Recognizing that naive lifting of diffusion priors leads to over-smoothed results and view inconsistencies, the authors introduce a physically principled neural rendering framework to recover fine-grained details. The core innovation lies in frequency-based disentanglement:

Albedo Representation: The diffuse albedo ( $a_d$ ) is modeled using a high-resolution 2D hash encoding in the UV space. This allows the network to capture high-frequency, intricate textural details.
Irradiance Representation: The diffuse irradiance ( $s_d$ ) is modeled using a low-resolution 3D hash encoding in the 3D scene space, combined with per-image learnable embeddings. This captures the smoother, low-frequency lighting variations and shadows.
Disentanglement Mechanism: By explicitly enforcing a discrepancy in the maximal grid resolutions ( $V^{(a)}_{L_a} \gg V^{(s)}_{L_s}$ ), the framework encourages high-frequency information to flow into the albedo component while the irradiance component handles the lighting. This resolves the ambiguity between surface detail and shading.

2.3 Optimization and Rendering

The system jointly optimizes the learnable hash entries and MLP decoders to minimize a composite loss function:

Albedo Regularization ( $\mathcal{L}_a$ ): Aligns the neural texture with the view-space diffusion priors.
Neural Rendering Loss ( $\mathcal{L}_c$ ): Renders the final RGB color using the Lambertian shading model ( $c = a_d \odot s_d$ ) and compares it against the raw input images.

The resulting neural texture can be baked into a standard UV map, making it compatible with modern graphics engines (e.g., Blender) for real-time rendering and flexible editing.

3. Key Contributions

DANTE-W Framework: A neural texturing approach that recovers high-fidelity diffuse albedo for in-the-wild scenes by lifting inconsistent view-space diffusion priors into a consistent, expressive neural texture representation.
Frequency-Based Disentanglement: A physically principled neural rendering strategy that separates fine-grained albedo details from irradiance by differentiating the frequency bands of their respective neural representations (high-res 2D for albedo, low-res 3D for lighting).
GigaLit Benchmark: The introduction of a new benchmark dataset containing diverse, in-the-wild scenes and synthetic objects with complex textures and unstructured viewpoints, designed to evaluate diffuse albedo recovery and relighting under challenging conditions.

4. Experimental Results

The authors evaluated DANTE-W on the GigaLit benchmark and the public Stanford-ORB dataset, comparing against classical texturing (Metashape Pro), vanilla diffusion outputs, and other intrinsic decomposition methods.

Quantitative Performance: On the GigaLit synthetic subset, DANTE-W outperformed diffusion-based renderers by +2.63 dB to +3.85 dB in PSNR for albedo recovery and achieved ~25% lower LPIPS. For relighting tasks, it showed even greater improvements, with +6.13 dB to +7.15 dB in PSNR and over 40% reduction in LPIPS compared to state-of-the-art diffusion methods.
Qualitative Performance: Visual comparisons demonstrate that DANTE-W effectively removes baked-in shadows and shading while preserving fine details (e.g., text on surfaces, intricate patterns) that are often lost or hallucinated by other methods. It maintains 3D consistency across varying viewpoints.
Efficiency: The optimization process is highly efficient, taking approximately 5–8 minutes per scene on a single RTX 3090 GPU, which is over 7× faster than the multi-view mesh texturing process of Metashape Pro, while maintaining comparable or superior visual quality.

5. Significance and Claims

The paper claims that DANTE-W represents a substantial advancement toward robust and high-fidelity relighting applications for in-the-wild scenes. By bridging the gap between coarse diffusion priors and expressive 3D representations, the method overcomes the inherent lighting-material ambiguity that plagues existing inverse rendering techniques.

The authors emphasize that their approach enables hyper-realistic free-view relighting by producing albedo textures that are free from illumination artifacts. They position the work as a seamless integration with established 3D reconstruction and graphics pipelines, making it practical for domains such as virtual reality, filmmaking, and gaming.

Limitations: The authors modestly note that the current method is designed for Lambertian surfaces and does not account for non-Lambertian effects (e.g., specular highlights). Future work is suggested to extend the framework to advanced scene representations (like Gaussian Splatting) and include specular materials.