💻 computer science

GTAvatar: Bridging Gaussian Splatting and Texture Mapping for Relightable and Editable Gaussian Avatars

GTAvatar proposes a novel method that bridges 2D Gaussian Splatting with UV texture mapping to reconstruct high-fidelity, relightable, and intuitively editable head avatars from a single monocular video by embedding Gaussian primitives into a canonical UV domain.

Original authors: Kelian Baert, Mae Younes, Francois Bourel, Marc Christie, Adnane Boukhayma

Published 2026-03-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Kelian Baert, Mae Younes, Francois Bourel, Marc Christie, Adnane Boukhayma

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a digital actor, a 3D head that looks so real it could fool your grandmother. For a long time, creating these actors was a trade-off: you could have them look perfectly realistic (like a high-definition photograph), or you could have them be easy to edit (like a clay model you can reshape), but rarely both.

GTAvatar is a new invention that finally gives you the best of both worlds. It combines the "photorealism" of a modern camera with the "editability" of a painter's canvas.

Here is how it works, broken down into simple concepts:

1. The Problem: The "Spray Paint" vs. The "Canvas"

Imagine trying to paint a realistic portrait.

Old Method (3D Mesh): This is like sculpting with clay. You have a smooth surface (a canvas) where you can paint a texture (skin, freckles, makeup). It's easy to edit, but if you want to capture tiny, complex details like the way light hits a wet lip or the translucency of an ear, the clay model often looks a bit stiff or blurry.
The New Method (Gaussian Splatting): This is like taking a million tiny, glowing spray-paint dots and spraying them into the air to form a shape. It creates incredibly realistic, photo-like images instantly. However, because it's just a cloud of independent dots, you can't easily "paint" on it. If you want to change the color of the lips, you'd have to find every single dot that makes up the lips and change them one by one. It's a mess.

2. The Solution: The "Sticky Note" Trick

GTAvatar solves this by taking that cloud of spray-paint dots (Gaussians) and sticking them onto a pre-existing map (a UV texture map), like a standard 3D character model used in video games.

Think of it this way:

Imagine you have a 3D mannequin (the FLAME model) that has a flat map of its skin laid out on a table (the UV map).
GTAvatar takes those millions of "spray-paint dots" and says, "You belong to this specific spot on the flat map."
Instead of each dot carrying its own color, the dots now just point to a shared, continuous painting on the table.

3. Why This is a Game-Changer

Because the dots are now connected to a single, continuous map, three magical things happen:

You Can Paint Directly: Since the dots are just reading from a map, you can open that map in Photoshop (or any image editor), add a tattoo, change the eye color, or smooth out a wrinkle. When you save the file, the 3D avatar instantly updates. It's like editing a texture on a video game character, but the character looks photorealistic.
The Lighting Works Automatically: The system uses "Physics-Based Rendering" (PBR). Think of this as giving the avatar a set of realistic materials (skin, plastic, metal). If you change the lighting in the room (from sunny to candlelight), the skin reacts naturally. It knows how to reflect light, how rough the skin is, and where the shine should be, just like real skin.
It's Fast: Even though it's doing complex math, it's efficient enough to run in real-time. You can animate the face, change the expression, and see the result instantly.

4. The "Magic Glue" (UV Regularization)

There was one big risk: If you just stuck the dots to the map randomly, the texture would look like a shattered mirror (discontinuous and glitchy).

The authors invented a special "glue" (mathematically called UV Regularization). This ensures that if two dots are next to each other in 3D space, they must be next to each other on the flat map. This keeps the texture smooth and continuous, so when you edit it, you don't get weird artifacts or blurry patches.

5. What Can You Do With It?

The paper shows off some cool demos:

Relighting: Take a video of a person in a dark room and instantly make it look like they are standing in bright sunlight or under neon club lights.
Editing: Remove a scar, add a mustache, or change the hair color without re-training the whole AI model.
Animation: Make the avatar talk, smile, or look around, and the texture stays perfectly aligned.

The Bottom Line

Before GTAvatar, you had to choose between a realistic photo that you couldn't touch, or a cartoon character you could edit easily. GTAvatar builds a bridge between the two. It takes the "spray paint" technology that makes things look real and sticks it onto a "painting canvas" that makes things easy to edit.

It's like taking a hyper-realistic photograph of a face and turning it into a digital clay model that you can sculpt, paint, and light up however you want, all while keeping the photo-realistic quality.

1. Problem Statement

While recent advancements in 3D Gaussian Splatting (3DGS) and 2D Gaussian Splatting (2DGS) have enabled photorealistic, real-time reconstruction of head avatars from monocular videos, they suffer from a critical limitation: lack of editability.

The Trade-off: Traditional mesh-based methods allow for intuitive editing via UV texture maps (e.g., changing makeup, removing wrinkles), but struggle with high-frequency details and complex geometries (like hair or translucent skin) due to topological rigidity. Conversely, Gaussian Splatting offers superior fidelity and geometry handling but treats each primitive as an isolated entity with its own color, lacking the shared structure (UV space) required for coherent, semantic editing.
The Gap: Existing attempts to embed Gaussians into UV space often result in discontinuous, sparse texture maps that are difficult to edit or require complex, multi-stage neural baking processes (e.g., FATE), which can degrade sharp details. Furthermore, most Gaussian avatars lack robust relighting capabilities combined with editable material properties.

2. Methodology

GTAvatar proposes a novel framework that bridges the gap between the fidelity of 2D Gaussian Splatting and the editability of UV texture mapping. The pipeline reconstructs a relightable, animatable, and editable head avatar from a single monocular RGB video.

Core Architecture

Base Representation: The method uses FLAME (a 3D morphable model) as a canonical template. 2D Gaussian primitives are bound to the triangles of the FLAME mesh.
Deformation: Gaussians deform with the mesh using Linear Blend Skinning (LBS) and barycentric interpolation, allowing for realistic facial expressions and poses.
Material Modeling: Instead of assigning a single color to each Gaussian, the system samples Physically Based Rendering (PBR) material attributes (Albedo, Roughness, Specular Reflectance, and Normal maps) from learnable global texture maps.

Key Technical Innovations

Efficient UV Mapping (The "Bridge"):
- The core contribution is a computationally efficient method to map the local tangent space $(s, t)$ of a 2D Gaussian splat to a continuous patch in the canonical UV domain of the FLAME mesh.
- Mechanism: For a ray-splat intersection, the method approximates the intersection point on the splat's reference triangle plane. It then uses a single matrix multiplication per intersection to transform local splat coordinates into global UV coordinates.
- Formula: The mapping $M(s,t)$ is derived using Jacobians ( $J_v$ for world space, $J_{uv}$ for UV space) and a pseudo-inverse, allowing the system to compute UV coordinates without expensive iterative solvers during rendering.
Deferred Shading & Relighting:
- The system employs a deferred shading framework using the Cook-Torrance BRDF model.
- It utilizes a split-sum approximation to handle environment lighting efficiently. A learnable RGB cubemap represents the environment lighting, enabling the avatar to be relit under arbitrary conditions.
- Normal Mapping: To capture high-frequency geometric details without increasing the primitive count, the method uses a residual normal map sampled from a texture. This normal is transformed from tangent space to world space to drive the shading calculations.
Training & Regularization:
- The model is trained end-to-end using a monocular video.
- UV Regularization ( $L_{uv}$ ): A novel loss term is introduced to ensure that all Gaussians contributing to a single pixel map to a consistent location in the UV texture. This prevents "ghosting" and ensures the texture map remains continuous and editable.
- Physical Priors: The system uses diffusion-based priors for albedo and statistical priors (FLAME PCA) to guide the disentanglement of lighting and material properties, ensuring realistic skin appearance.

3. Key Contributions

Unified Representation: The first Gaussian head avatar framework that simultaneously supports real-time rendering, arbitrary relighting, and intuitive UV-based texture editing without requiring a two-stage neural baking process.
Efficient UV Mapping: A novel, differentiable mapping technique that projects 2D splat intersections to UV space via a single matrix multiplication, maintaining high rendering speeds despite the added complexity of texture sampling.
Semantic Control: By anchoring Gaussians to a standard 3DMM UV map, the method enables semantic editing (e.g., changing eye color, adding tattoos, smoothing skin) directly in the texture domain, which propagates consistently across all poses and expressions.
Compactness: The use of global texture maps for material attributes allows for a significantly smaller model size compared to state-of-the-art relightable avatars while maintaining or exceeding reconstruction quality.

4. Results

The authors evaluated GTAvatar on the INSTA and HDTF datasets, comparing it against state-of-the-art methods like HRAvatar, FLARE, and FATE.

Reconstruction Quality: GTAvatar achieves state-of-the-art performance in PSNR, SSIM, and LPIPS metrics, outperforming HRAvatar and other baselines.
Model Efficiency: Despite using global textures, GTAvatar achieves comparable or better reconstruction quality with fewer Gaussian primitives (e.g., 10k primitives vs. 80k in baselines) due to the high representational power of the texture maps.
Relighting: The method demonstrates high-quality relighting under various environment maps, comparable to or better than HRAvatar and FLARE.
Editing Capabilities: The paper showcases successful edits including:
- Adding decals (stars) and changing hair/eye colors.
- Removing skin imperfections or adding makeup.
- Editing normal maps to alter surface geometry perception.
- These edits remain consistent across different poses and lighting conditions.
Performance: The method renders at >170 FPS for static geometry and ~80 FPS for dynamic geometry on an NVIDIA RTX A5000, proving that the texture sampling overhead is manageable.

5. Significance

GTAvatar represents a significant step forward in digital human reconstruction by solving the "editability vs. fidelity" dilemma.

For Industry: It provides a pipeline suitable for visual effects, gaming, and virtual production where artists need to modify avatars (e.g., aging, stylization) without retraining the entire model.
For Research: It demonstrates that explicit UV parameterization can be successfully integrated into implicit Gaussian representations, opening new avenues for combining the best of mesh-based and neural rendering techniques.
Ethical Consideration: The authors explicitly address the potential for deepfakes, emphasizing that the technology is intended for legitimate applications (VFX, VR) and advocating for ethical guidelines and regulations.

In summary, GTAvatar successfully merges the geometric flexibility of Gaussian Splatting with the artistic control of traditional texture mapping, creating a versatile, high-fidelity, and editable avatar representation.