Condition Matters in Full-head 3D GANs

Technical Summary: Condition Matters in Full-Head 3D GANs (BalanceHead)

Published at: ICLR 2026

1. Problem Statement

The paper addresses a fundamental limitation in existing 3D-aware Generative Adversarial Networks (GANs) designed for full-head (360°) synthesis. While early 3D GANs focused on near-frontal views, recent full-head models (e.g., PanoHead, SphereHead) attempt to cover the entire head. However, these models typically use the view angle (camera yaw/pitch) as the primary conditioning signal.

The authors identify two critical issues with view-conditioning:

• Directional Bias: The model's generative capacity becomes tethered to the conditional view. Consequently, the quality and diversity of the generated head are significantly higher at the conditional view (usually the front) than at non-conditional views (the back), leading to global incoherence and artifacts in the rear regions.
• Mode Collapse & Instability: Removing conditioning entirely leads to rapid mode collapse, but traditional view-conditioning fails to encourage the generator to follow the true semantic distribution, often causing it to stall after learning a few successful modes.

2. Methodology

To decouple the generative capability from the viewing direction, the authors propose BalanceHead, which shifts the conditioning paradigm from view-based to semantic-based.

A. View-Invariant Semantic Conditioning:
Instead of providing the camera angle as input, the model is conditioned on a view-invariant semantic feature. The authors anchor all views of a subject to the CLIP image feature of the frontal view. Since the frontal view contains the most comprehensive information (facial identity, hairstyle, clothing), using it as a shared condition for all views (front, side, and back) ensures semantic alignment and eliminates directional bias.

B. BalanceHead360 Dataset Generation:
Since high-quality 360° multi-view datasets are rare, the authors developed a novel two-stage pipeline using FLUX.1 Kontext (a 2D diffusion model):

1. Front-view Synthesis: Real near-frontal images are enhanced and converted into high-quality, centered frontal views.
2. Multi-view Expansion: These frontal views are expanded into full 360° collections using diverse view prompts.
3. Automated Filtering: A visual-language model (Qwen2.5-VL) acts as an agent to filter out images with artifacts, identity drift, or global incoherence, resulting in a massive 11.2-million-image balanced dataset.

C. ViCiCo Loss (View-image and Condition-image Consistency Loss):
To prevent "multiple-face" artifacts (where the model generates face-like patterns in the back-head region), the authors introduce ViCiCo Loss. This loss enforces consistency between the generated image and the semantic condition by using a discriminator to identify mismatched pairs (e.g., a side-view image paired with a front-view semantic condition).

D. Architecture:
The model utilizes a hy-plane representation (building on HyPlaneHead) and a StyleGAN2 backbone, integrating the semantic condition and the ViCiCo loss into the training pipeline.

3. Key Contributions

• New Conditioning Paradigm: Proposes the first semantic-conditional 3D-aware GAN that uses view-invariant features to solve directional bias.
• Large-Scale Synthetic Dataset: Constructs BalanceHead360, a 11.2M image dataset that provides balanced distribution of quality, quantity, and diversity across all 360° views.
• Robust Training Strategy: Demonstrates that 3D-aware GANs can effectively learn consistent 3D structures even from "imperfect" (slightly 3D-inconsistent) 2D generative data.
• Novel Loss Function: Introduces ViCiCo loss to suppress common 3D head artifacts.

4. Results

• 360° Synthesis: BalanceHead achieves state-of-the-art (SOTA) performance. Quantitative metrics (FID-random) show significantly lower error rates compared to view-conditioned models, indicating much higher quality and diversity in non-frontal views.
• Diversity: The model shows superior identity diversity (measured by MKNND) and can generate a wide range of attributes, including complex hairstyles, hats, and accessories.
• GAN Inversion: In single-view inversion tasks (using PTI), BalanceHead demonstrates higher fidelity and better global coherence, successfully reconstructing the "invisible" back-head regions of a target subject.
• Generalizability: The semantic conditioning approach was shown to be compatible with various other 3D representations (tri-plane, tri-grid, spherical tri-planes), consistently improving their performance.

5. Significance

This work represents a potential paradigm shift in 3D generative modeling. It proves that researchers do not need to rely solely on expensive, perfectly consistent multi-view datasets. Instead, by leveraging the massive, high-quality, but slightly inconsistent output of 2D diffusion models and combining it with view-invariant semantic conditioning, one can build highly stable, diverse, and photorealistic 3D generative models. This has profound implications for digital avatars, virtual reality, and automated 3D content creation.