TopoMesh: High-Fidelity Mesh Autoencoding via Topological Unification

1. Problem Statement

The current paradigm for high-fidelity 3D generation relies on a VAE-Diffusion pipeline, where the Variational Autoencoder (VAE) is responsible for compressing irregular 3D meshes into a latent space and reconstructing them. However, existing VAEs face a fundamental representation mismatch:

• Ground Truth (GT): Real-world meshes have arbitrary, variable topology (irregular connectivity, varying vertex counts).
• Network Prediction: Most VAEs predict fixed-structure implicit fields (e.g., Signed Distance Fields on regular grids) or use decoders like Marching Cubes (MC) that constrain vertices to grid edges.

Consequences:

• Loss of Sharp Features: Fixed-grid approaches inherently smooth out sharp edges and corners.
• Indirect Supervision: Due to the lack of explicit vertex-to-vertex correspondence between the GT and the prediction, prior works rely on indirect signals like SDF losses or rendering losses. These signals suffer from ambiguity, occlusion, and limited resolution, failing to provide clear gradients for fine geometric details.
• Training Instability: Attempts to supervise both topology and geometry simultaneously often lead to a "tug-of-war" where geometry losses destabilize topology learning.

2. Methodology: TopoMesh

TopoMesh addresses these issues through Topological Unification, ensuring both the input (GT) and the output (prediction) share a unified Dual Marching Cubes (DMC) topological framework. This enables explicit, direct supervision at the mesh level.

The framework consists of two core modules:

A. Topo-Remesh (Preprocessing)

To unify the input, arbitrary GT meshes must be converted into DMC-compliant representations without losing sharp features.

• L∞ Distance Metric: Traditional remeshing uses $L_2$ distance, which rounds sharp corners during surface dilation. TopoMesh introduces an $L_\infty$ distance metric that considers the local surface structure (planes of incident triangles) rather than just point-to-point distance. This preserves sharp angles and corners.
• GPU-Accelerated Pipeline: The process involves Voxelization, Flood-fill, $L_\infty$ SDF calculation, Isosurface Extraction (using Manifold Dual Contouring), and Compression. It achieves watertight, sharp-feature-preserving meshes at $1024^3$ resolution in ~15 seconds.

B. Topo-VAE (The Autoencoder)

• Sparse Voxel-Point Encoder:
  • Encodes input mesh vertices (point cloud) into sparse voxel features.
  • Efficiency: Uses a sparse local attention mechanism. Instead of global attention (which is computationally intractable for millions of points), each voxel attends only to points within it. This reduces memory from $O(N \times P)$ to $O(P)$ and allows all voxels to share a single learnable query token.
• Decoupled Topology and Geometry Decoder:
  • Builds upon FlexiCubes but decouples parameters into two sets:
    • Topology: Occupancy ($o$) and triangulation weights ($\gamma$).
    • Geometry: Magnitude ($u$), interpolation weights ($\alpha, \beta$), and deformation vectors ($\delta$).
  • Process: The decoder first determines mesh connectivity (faces) using only topological parameters, then calculates vertex positions using all parameters. This separation prevents the entanglement of supervision signals.

C. Training Strategy

• Explicit Mesh-Level Loss: Because the topology is unified, the model applies direct losses:
  • Topology Loss: Binary Cross-Entropy on grid corner occupancy.
  • Vertex Loss: $L_1$ loss on vertex positions.
  • Normal Loss: $L_1$ loss on face normals (handling the specific triangulation strategy of DMC).
• Teacher Forcing: To prevent training instability caused by the "tug-of-war" between topology and geometry, the decoder is fed ground-truth topology during training. This allows geometry parameters to learn stable gradients under correct topological configurations. At inference, the network predicts both.
• Progressive Resolution & Voxel Pruning: Training starts at coarse resolutions and moves to fine ones. GT-guided voxel pruning ensures computational efficiency without removing critical surface voxels.

3. Key Contributions

1. Topological Unification Paradigm: A novel VAE framework that unifies GT and predictions under a shared DMC structure, enabling direct supervision on topology, vertex positions, and face orientations.
2. Topo-Remesh Algorithm: A robust, fully GPU-accelerated remeshing algorithm using an $L_\infty$ metric to convert arbitrary meshes into DMC-compliant formats while preserving sharp features.
3. Sparse Voxel-Point VAE: An efficient encoder using sparse local attention and a decoupled decoder that separates topology and geometry for stable training.
4. Advanced Training Strategy: Introduction of Teacher Forcing and progressive resolution training to resolve convergence issues in mesh autoencoding.
5. State-of-the-Art Performance: Demonstrated superior reconstruction fidelity, particularly in preserving sharp features and fine details compared to existing methods.

4. Experimental Results

• Remeshing Performance: Topo-Remesh outperforms baselines (Mesh2SDF, ManifoldPlus, Dora, Sparc3D) in preserving sharp edges (highest F1 and ANC scores) while being significantly faster (~18.5s vs. >60s for others) and producing smaller file sizes.
• Autoencoding Quality:
  • Sharp Feature Preservation: Achieved an 8% improvement in F-Score and significantly higher F1-Sharp scores (measuring preservation of edges/corners) compared to SOTA methods like SparseFlex and Trellis.
  • Visuals: Successfully reconstructs complex details (e.g., railings, sharp corners, holes) that other methods smooth out or fail to recover.
• 3D Generation: When used as a backbone for image-to-3D generation, TopoMesh produces results with sharper geometric details and better alignment with input images compared to Trellis and Direct3D-S2.

5. Significance

TopoMesh represents a paradigm shift in 3D generative modeling by solving the representation mismatch bottleneck. By establishing explicit mesh-level correspondences, it eliminates the need for ambiguous indirect supervision (like rendering losses), allowing for lossless reconstruction of fine geometric details. This paves the way for high-fidelity 3D content creation in gaming, VR, and robotics, where preserving sharp features and complex topology is critical. The method also provides a scalable, efficient architecture for processing high-resolution 3D data.