Symmetry Matters: Auditing and Symmetrizing 3D Generative Models

Technical Summary: Symmetry Matters: Auditing and Symmetrizing 3D Generative Models

Problem Statement
Symmetry is a fundamental geometric prior ubiquitous in natural and artificial objects, serving as a crucial cue for human perception and a structural constraint in computer vision tasks. Despite the prevalence of reflection symmetry in widely used 3D datasets like ShapeNet, standard benchmarks for 3D generative models (evaluating metrics such as Coverage, 1-NNA, CD, and EMD) rarely assess whether this prior is preserved in generated outputs. The authors posit that current models may achieve competitive scores on standard metrics while failing to maintain structural consistency, specifically reflection symmetry. The core research question is whether current 3D generative models preserve the symmetry present in their training data, and if not, whether this failure stems from data imperfections, sampling noise, or the learned generative process itself.

Methodology
The paper employs a multi-stage approach to audit, diagnose, and remedy symmetry failures in unconditional point cloud generation:

1. Symmetry Audit and Measurement:

   • The authors introduce a Normalized Symmetry Score based on Chamfer Distance (NSCD). This metric quantifies reflection symmetry by computing the CD between an object and its reflection across the $x=0$ plane, normalized by the squared diagonal of the object's bounding box to ensure scale invariance.
   • They audit eight representative 3D generative models (including DPM, PVD, LION, DiT-3D, XCube, SLIDE 3D, and SPVD variants) trained on ShapeNet.

2. Controlled Dataset Verification:

   • To determine if the symmetry gap is inherited from the data, the authors construct a mirrored-objects dataset. This involves taking ShapeNet objects, concatenating them with their reflections, and resampling to create an explicitly symmetrized distribution.
   • Models are retrained on this mirrored dataset to observe if a stronger symmetry prior in the data translates to better symmetry preservation in generation.

3. Mechanistic Interpretability Inspection:

   • The authors apply mechanistic interpretability techniques to diagnose where symmetry breaking occurs within the pipeline.
   • Sampling Level: For diffusion-based models, they test if reflecting the initial Gaussian noise results in a reflected final shape. They measure the NSCD between the generation from original noise and the generation from reflected noise.
   • Latent Level: For models with encoders (e.g., LION), they measure the normalized Euclidean distance between the latent codes of an object and its reflection. A large distance indicates the encoder does not treat symmetric inputs as equivalent.

4. Data-Centric Intervention:

   • Instead of modifying architectures or loss functions, the authors propose a half-objects training strategy.
   • Training: Objects are split along the symmetry plane ($x=0$), and only the right half ($x \ge 0$) is used for training.
   • Sampling: Generated half-objects are mirrored across the $x=0$ plane and concatenated to reconstruct full shapes. A dynamic shift mechanism is introduced to prevent geometric collapse at the boundary.

Key Results

• Symmetry Gap: Current state-of-the-art models exhibit a consistent "symmetry gap." Even when standard metrics (1-NNA, COV) indicate competitive performance, the NSCD of generated samples is significantly higher (worse) than the ground-truth ShapeNet distribution.
• Data vs. Process: Retraining models on the explicitly symmetrized mirrored dataset did not eliminate the gap. In many cases, the models trained on mirrored data still produced outputs with higher symmetry errors than the original training data, suggesting the failure is induced by the learned generative process, not just the data.
• Mechanistic Findings:
  • Sampling: Reflecting the input noise does not consistently produce a reflected output shape, indicating the denoising process is not symmetry-aware.
  • Latent Space: Encoders map symmetric objects to distant latent representations, failing to encode the symmetry prior coherently.
• Intervention Efficacy: The half-objects training approach substantially improved symmetry across all tested backbones. The resulting models (denoted with 'S-') achieved significantly lower NSCD values (often better than the original ShapeNet ground truth) while maintaining competitive performance on standard distributional metrics (1-NNA, COV, CD, EMD).
• Post-hoc Mirroring Limitations: An ablation study showed that simply splitting and mirroring generated objects (post-hoc) often duplicates local geometric errors (e.g., missing parts or deformations), whereas training on half-objects learns a robust distribution of partial geometries.

Significance and Claims
The paper claims that reflection symmetry is a strong prior in 3D data that is currently overlooked by standard evaluation protocols. The authors argue that:

1. Evaluation Gap: Standard benchmarks are insufficient for assessing structural priors; a symmetry-aware evaluation protocol is necessary alongside metrics like 1-NNA and COV.
2. Generative Failure: The lack of symmetry in generated shapes is not merely a data artifact but a failure of the learned generative process to encode structural consistency.
3. Simple Remedy: A data-centric intervention (training on half-objects) is a simple, architecture-agnostic method to enforce symmetry, yielding geometrically consistent and visually plausible results without sacrificing distributional diversity.
4. Mechanistic Insight: The work demonstrates the utility of mechanistic interpretability in 3D generation, revealing that symmetry breaking is visible in both sampling dynamics and latent space organization.

The authors conclude that future 3D generative models should explicitly incorporate symmetry as a design principle, either through training strategies or architectural constraints, to ensure the physical and semantic integrity of generated objects.