Technical Summary: Source-Face Authenticity Detection for 3D Gaussian Heads

Problem Definition

Recent advancements in single-image 3D Gaussian head reconstruction have enabled the generation of highly realistic, freely renderable digital heads from a single portrait. While beneficial for digital humans and AR/VR, this technology poses significant risks to identity authentication and privacy. If a 3D Gaussian head is reconstructed from a forged or manipulated 2D portrait, it creates a persistent, renderable identity asset that can facilitate impersonation.

The core challenge addressed in this work is Source-Face Authenticity Detection: determining whether the underlying source face of a reconstructed 3D Gaussian head is real or fake, given only a single rendered view of the 3D head. Existing 2D forgery detectors fail in this domain due to two primary factors:

1. Information Loss: The 3D reconstruction and re-rendering processes compress or alter original 2D forgery traces, leaving only subtle, localized evidence that standard detectors struggle to capture.
2. View Inconsistency: Changes in rendering viewpoint alter the visibility and projection of these cues. Detectors trained on standard 2D supervision often overfit to view-specific rendering shortcuts rather than learning view-robust authenticity features.

Methodology

The authors propose a two-stage detector designed to preserve fine-grained appearance information and enforce feature consistency across rendered views.

Stage I: Self-Supervised Fine-Tuning

This stage adapts a visual backbone to the 3D Gaussian Splatting (3DGS) rendering domain using two self-supervised objectives:

• Masked Autoencoding (MAE): To encourage the retention of fine-grained local appearance information, the model randomly masks a high ratio (75%) of image patches. The backbone must reconstruct the missing patches from visible ones. This process forces the encoder to learn structural and subtle texture details (including Gaussian artifacts) that are critical for distinguishing real from fake, without relying on explicit authenticity labels.
• Multi-View Contrastive Learning: To address view inconsistency, the model treats different rendered views of the same 3D head as positive pairs and views from different identities as negative pairs. This objective pulls representations of the same head together in the latent space regardless of the viewpoint, promoting viewpoint-invariant authenticity features.

Stage II: Multi-Level Feature Fusion and Classification

In the second stage, the visual backbone is frozen. Instead of relying solely on the final CLS token, the method concatenates CLS tokens from low-, middle-, and high-level network depths.

• Rationale: Different network depths exhibit complementary spatial attention patterns. Low-level tokens capture fine-grained texture, while high-level tokens capture semantic consistency.
• Classification: The normalized CLS tokens from these selected layers are concatenated and fed into a Multi-Layer Perceptron (MLP) for the final real/fake classification.

Key Contributions

1. First Large-Scale Benchmark: The authors introduce the first dataset for real/fake 3D Gaussian head detection. It comprises approximately 360,000 multi-view renderings from 16,372 unique identities. The dataset includes:
   • Real Sources: 5 real-face datasets (e.g., CelebV-HQ, FFHQ).
   • Fake Sources: 9 forgery methods covering text-to-image generation, face swapping, and expression-driven synthesis.
   • Reconstruction: 5 single-image 3D Gaussian head reconstruction methods (e.g., CAP4D, FaceLift, Any3DAvatar).
2. Systematic Benchmarking: A unified evaluation protocol was established to test representative 2D forgery detectors (both general-purpose and face-specific) on 3DGS data, revealing their limitations in retaining fine-grained cues and maintaining cross-view consistency.
3. Novel Detector Architecture: A two-stage framework combining masked autoencoding and multi-view contrastive learning, followed by multi-level CLS token fusion, which achieves state-of-the-art performance.

Experimental Results

The proposed method was evaluated against strong baselines (including AIDE, Effort, PGC, and face-specific detectors like IID and ForAda) on an identity-disjoint benchmark.

• Overall Performance: The proposed method achieved the highest accuracy (89.51%) and Macro-F1 (89.51%) on the standard test set, outperforming the strongest baseline (Effort) by a significant margin. It also achieved superior AUC (95.36%) and AP(fake) (96.03%).
• Generalization (OOD): The method demonstrated robust generalization to unseen forgery methods and reconstruction pipelines:
  • Unseen Generators: When tested on held-out forgery methods (e.g., GPT-Image-2, HyperSwapper, PersonaLive), the proposed method consistently ranked first across all metrics. Notably, on the GPT-Image-2 OOD subset, it achieved 96.54% accuracy and Macro-F1.
  • Unseen Reconstruction: When tested on an unseen 3D reconstruction method (Any3DAvatar), it maintained high accuracy (89.24%), significantly outperforming baselines which dropped to ~82% or lower.
• Ablation Studies:
  • Removing the MAE branch or Contrastive Learning branch individually degraded performance, confirming the necessity of both detail preservation and view consistency.
  • Fusing multi-level CLS tokens (low, mid, high) outperformed using single-layer tokens or only the final layer, validating the complementary nature of features across network depths.
  • Using the DINOv3 ViT-H+/16 backbone yielded the best results, though the framework remained superior to baselines even with smaller backbones.

Significance and Claims

The paper claims to be the first work to systematically address the authenticity detection of 3D Gaussian heads reconstructed from single portraits. It highlights that the transition from 2D images to 3D renderable assets fundamentally changes the nature of forgery traces, rendering existing 2D detectors ineffective.

The authors emphasize that their approach does not rely on detecting specific manipulation artifacts but rather on learning robust representations that preserve fine-grained appearance details and maintain consistency across views. By introducing a large-scale benchmark and a dedicated detector, the work aims to provide a foundation for securing identity authentication and protecting face privacy in emerging 3D digital human applications.

Note: The authors modestly acknowledge that expression-driven forgeries remain a challenging subset (achieving 76.24% accuracy in specific splits) and suggest future work to expand detection capabilities in this category.