💻 computer science

Sculpting NeRF Geometry: Human-Preference Fine-Tuning of a 3D-Aware Face GAN

This paper presents a method to fine-tune a pretrained 3D-aware face GAN (EG3D) using reinforcement learning from human feedback directly on neural radiance field density values, enabling the generation of user-preferred 3D face geometries without requiring explicit mesh supervision or shape priors.

Original authors: Archer Moore, Mingming Gong, Liam Hodgkinson

Published 2026-06-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Archer Moore, Mingming Gong, Liam Hodgkinson

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a master sculptor (a computer program called EG3D) who is incredibly talented at painting portraits. If you ask this sculptor to "make a face," they can paint a stunning, photorealistic image of a person. However, if you ask them to build the actual 3D clay model underneath that painting, the result is often a mess. The nose might be too sharp, the cheeks might have weird bumps, or the sides of the face might look like crumpled paper. The painting looks great, but the 3D structure underneath is broken.

This paper introduces a new way to teach this sculptor how to fix the clay model without showing them a perfect clay model to copy. Instead, they learn by listening to a human's opinion on which models look "better."

Here is how they did it, broken down into simple steps:

1. The Problem: The "Uncanny Valley" of 3D

Current AI models are great at making 2D pictures of faces. But when they try to figure out the 3D shape behind the picture, they often get it wrong. It's like a magician who can draw a perfect rabbit on a piece of paper, but if you try to pull a real rabbit out of the hat, it's made of cardboard and falls apart. The paper notes that even the best current models (like EG3D) have these "cardboard" flaws, especially around the nose and sides of the face.

2. The Solution: A "Taste Tester" for Invisible Clay

Usually, to fix 3D models, researchers try to turn the invisible computer data into a visible mesh (a wireframe) and then fix the wireframe. This paper says, "No, let's skip the wireframe."

Instead, they built a "Taste Tester" (a reward model) that looks directly at the invisible "density" of the 3D face.

The Analogy: Imagine the 3D face is made of invisible fog. Some parts are thick fog (the skin), and some parts are thin fog (empty space). The AI sculptor creates this fog. The "Taste Tester" doesn't need to see a solid statue; it just sniffs the fog. It knows that a smooth, continuous fog pattern looks like a real face, while a jagged, broken fog pattern looks like a glitch.

3. The Training: "This One, Not That One"

The researchers didn't need thousands of experts or complex instructions. They just needed one person to play a simple game:

The AI generates a bunch of 3D faces.
The human looks at them and picks the "best" one and the "worst" one.
The AI learns: "Oh, the human likes the smooth nose and dislikes the bumpy side."

They did this about 4,300 times. The AI learned a "score" for how good a 3D shape looks, based entirely on human preference.

4. The Fine-Tuning: Sculpting with a Safety Net

Once the AI had this "Taste Tester," they let it tweak the sculptor (EG3D).

The Goal: Make the 3D clay model smoother and more realistic.
The Safety Net: They were worried that if they changed the clay too much, the 2D painting would stop looking like the same person. So, they added a rule: "You can change the shape, but you must keep the face looking like the original person."

The Result:

The Fix: The AI successfully smoothed out the weird bumps on the nose and sides of the face.
The Trade-off: The 2D pictures got slightly less "perfect" (a tiny drop in image quality), but the 3D shape became much more realistic.
The Verdict: When they showed the "Before" and "After" faces to other people, 74.4% of the time, people preferred the new, fixed 3D shape.

5. Why This is Special

Most other methods try to fix 3D shapes by:

Using text prompts (e.g., "Make a nose that looks like a Greek statue").
Converting the shape into a mesh first (which is slow and loses detail).
Looking at the 2D pictures from different angles.

This paper's method is unique because:

No Text Needed: It works on any face the AI generates, not just ones described by words.
Direct Reading: It reads the "fog" (density) directly, skipping the messy step of turning it into a wireframe.
Simple Data: It learned from just one person's opinions, proving you don't need a massive army of experts to teach an AI what looks good.

Summary

Think of it like teaching a child to draw a 3D cube. Instead of giving them a ruler and a protractor (the complex math), you just show them two drawings and say, "This one looks like a real box, that one looks like a crumpled piece of paper." The child learns the feeling of a good shape. This paper did exactly that for AI, teaching it to sculpt better 3D faces by listening to human preferences, resulting in faces that look real in 3D without losing their identity in 2D.

Technical Summary: Sculpting NeRF Geometry: Human-Preference Fine-Tuning of a 3D-Aware Face GAN

1. Problem Statement

While 3D-aware generative models (such as EG3D) have achieved photorealistic 2D image synthesis from 2D image collections, they frequently fail to produce realistic 3D geometry. Despite high-quality RGB outputs, the underlying 3D shapes often exhibit unrealistic discontinuities, irregularities, and artifacts (e.g., grooves on the nose or sides of the face). Existing approaches to improving 3D quality typically rely on explicit surface representations (meshes), require text-prompt conditioning, or depend on large-scale multi-view rendering and mesh extraction pipelines. These methods often struggle with "sink traps" in latent space where editing prompts no longer alters geometry, or they require complex, multi-dimensional human ratings. The core challenge addressed is how to fine-tune the 3D geometry of an unconditional 3D-aware GAN directly from human preferences without relying on external mesh priors, text conditioning, or explicit surface extraction.

2. Methodology

The proposed approach fine-tunes a pretrained 3D-aware face GAN (EG3D) using Reinforcement Learning from Human Feedback (RLHF), but with a distinct architecture that operates directly on the Neural Radiance Field (NeRF) density field.

A. Reward Model Architecture
Instead of scoring rendered 2D images or explicit mesh tokens, the reward model $r_\theta$ operates directly on the continuous 3D density field ( $\sigma$ ) of the NeRF.

Input: The model extracts a "frontal face slab" from the generator's $\sigma$ volume (a cropped region of the 3D grid containing the face). This slab is normalized to a range of $[0, 100]$ .
Backbone: A 3D U-Net with a Squeeze-and-Excitation ResNet backbone (ResNet-SE-3D-UNet) processes the $\sigma$ slab to extract global features, which are then mapped by an MLP to a scalar quality score.
Training Data: Preferences are collected by ranking batches of 3D geometries (visualized via Marching Cubes) from the generator. The dataset consists of 4,346 pairwise comparisons ( $x_w \succ x_l$ ) derived from a single annotator. To ensure robustness, each batch is augmented with a high-quality "anchor" sample drawn from the center of the latent space.
Loss: The reward model is trained using a pairwise preference loss (modified from InstructGPT) to maximize the score of the preferred sample over the dispreferred one.

B. Fine-Tuning Procedure
The generator is fine-tuned using a modified GAN loop that incorporates the learned reward signal.

Generator Loss ( $L_G$ ): The standard GAN loss is augmented with two terms:
1. Reward Loss ( $L_r$ ): Minimizing the negative of the reward score to encourage higher geometry quality. A clamp function limits the reward gradient to prevent instability.
2. Consistency Loss ( $L_c$ ): An $L_1$ distance constraint between the density field of the fine-tuned generator and the original pretrained generator. This ensures that while the geometry is reshaped, the 2D appearance and identity remain qualitatively similar.
Optimization: The process involves 20,000 images (20kimg) of fine-tuning on the FFHQ dataset, using a modified GAN loop update rather than Proximal Policy Optimization (PPO) or score-distillation sampling.

3. Key Contributions

Direct Density-Based Preference Learning: The paper demonstrates that 3D shape quality can be learned directly from human rankings of a small number of generator outputs by operating on the implicit $\sigma$ density field, eliminating the need for mesh extraction or text prompts.
Unconditional Reward Model: A quality-scoring module is developed that requires no language-prompt conditioning or color information. It operates solely on the 3D density representation, avoiding the "sink trap" issues associated with text-conditioned rewards.
Efficient Data Requirements: The method achieves robust geometric improvements using a modest dataset of 4,346 pairwise comparisons from a single annotator, significantly smaller than the datasets required by contemporary 3D RLHF methods (e.g., 16,000–25,000 pairs).
Selective Geometry Improvement: The approach successfully reshapes 3D geometry (smoothing noses and face sides) while preserving 2D identity and appearance, with only a bounded increase in FID (from 4.09 to 6.66).

4. Results

Reward Model Performance: The $\sigma$ -field representation significantly outperforms depth maps and point clouds in predicting human preferences. On held-out "hard" within-distribution pairs (excluding high-quality anchors), the $\sigma$ -based model achieves 91% accuracy, compared to 74% for the best depth-map model and chance levels for point-cloud models.
User Study: In an external user study involving 40 face pairs (before and after fine-tuning), users preferred the fine-tuned geometries in 74.4% of pairwise comparisons (Cohen's $h = 1.135$ , indicating a large effect size).
Geometric vs. Appearance Trade-off:
- FID: The FID-50k score increased from 4.09 (pretrained) to 6.66 (fine-tuned), indicating a measurable but bounded distributional cost.
- Identity Preservation: Visual inspection and quantitative metrics (LPIPS and face-recognition cosine similarity) confirm that the 2D identity and overall appearance remain qualitatively similar, with only minor differences in lighting and hair strands.
- No Mode Collapse: Analysis against the truncation-mean face shows that the fine-tuned samples remain geometrically closer to their original seeds than to the mean face (93–98% of cases), ruling out a simple collapse to a mean shape.
Attribution Analysis: SHAP and Integrated Gradients analyses reveal that the $\sigma$ -based reward model attends to semantically meaningful regions (nose, mouth, cheeks), whereas depth-map and point-cloud models tend to attend to face edges and boundaries, leading to less effective or distorted geometry corrections.

5. Significance and Claims

The paper claims that human preferences can be effectively utilized to fine-tune implicit 3D representations (NeRF density fields) without the need for explicit mesh priors or text conditioning. By decoupling the reward signal from text prompts and operating directly on the density volume, the method avoids the failure modes of prompt-conditioned systems and the computational overhead of mesh extraction.

The authors position this work as a proof of concept demonstrating that a single annotator's preferences, when distilled into a reward model operating on the generator's internal density field, can selectively improve 3D geometry quality. The method is presented as a data- and compute-efficient alternative to existing 3D RLHF pipelines, achieving significant geometric improvements while maintaining the qualitative fidelity of the 2D appearance. The authors acknowledge limitations, specifically that the reward model is currently bound to the specific generator architecture (EG3D) and its $\sigma$ distribution, and does not transfer cleanly to other 3D generators (e.g., PanoHead, SphereHead) without retraining, due to differences in numerical density regimes.