Technical Summary: Intrinsic and Triangulation-Agnostic Attention for Mesh Learning

Problem Statement

While attention mechanisms and transformers have revolutionized learning in domains like text, images, and audio, their application to triangle meshes has not yielded comparable performance gains. Existing approaches face a dichotomy:

1. Point Cloud Transformers: These often discard the mesh structure, treating vertices as unordered point sets. This ignores critical metric and topological information inherent to the mesh, leading to under-performance.
2. Mesh-Based Architectures: Current state-of-the-art mesh learning methods (e.g., PoissonNet, DiffusionNet) utilize geometric approaches based on Partial Differential Equations (PDEs) and Finite Element Methods (FEM). While these are intrinsic (respecting the Riemannian manifold structure) and triangulation-agnostic (invariant to how the surface is discretized), they lack the powerful feature aggregation capabilities of attention mechanisms.

The core problem is that standard attention mechanisms are neither intrinsic nor triangulation-agnostic. They treat vertices as discrete tokens without regard for the continuous surface they approximate, causing performance to degrade when the mesh triangulation changes or when high-frequency geometric details are required.

Methodology

The authors propose a novel attention layer designed specifically for meshes by modifying the attention mechanism from the ground up using principles from geometry processing. The method ensures that all quantities (queries, keys, values) and operations respect the continuous nature of the underlying manifold.

1. Intrinsic and Triangulation-Agnostic Quantities

To achieve triangulation agnosticism, the queries ($q$), keys ($k$), and values ($v$) are not treated as arbitrary vectors but as discretizations of continuous functions defined over the manifold.

• Generation: These quantities are predicted by an intrinsic, triangulation-agnostic backbone network (e.g., PoissonNet or DiffusionNet). This ensures that $q_i \approx q(x_i)$, where $q(x)$ is a continuous function, regardless of the specific mesh triangulation.

2. Triangulation-Agnostic Attention Mechanisms

The paper defines two types of attention operations:

• Cross-Attention (Mesh to Discrete Set): When attending from mesh vertices to a discrete set of external parameters (e.g., pose vectors), the standard discrete attention mechanism is sufficient, provided the queries are intrinsic. Theoretical analysis shows that if the query is a good approximation of a continuous function, the resulting feature aggregation error is linearly bounded by the discretization error.

• Self-Attention (Mesh to Mesh): This is the core contribution. The authors reformulate attention as a continuous operation over the surface:

  • Continuous Formulation: The attention weight between two points $s$ and $t$ on the surface is defined as $\alpha(s, t) = \frac{e^{\langle q(s), k(t) \rangle}}{\int e^{\langle q(s), k(r) \rangle} dr}$. The final feature is a weighted integral of the value function.
  • Discretization: To compute this efficiently on a mesh, the authors employ a standard Finite Element Method (FEM) discretization using lumped-mass quadrature.
  • Mass-Weighted Attention: Instead of a simple softmax over dot products, the attention weights are normalized by the lumped mass ($M_{jj}$) of each vertex. The aggregated feature $f_i$ is computed as:
    $$f_i = \frac{\sum_j M_{jj} w_{ij} v_j}{\sum_k M_{kk} w_{ik}}$$
    where $w_{ij} = e^{\langle q_i, k_j \rangle}$.
  • Theoretical Guarantee: The authors prove that this mass-weighted discretization approximates the continuous attention definition with an error of $O(\epsilon + h^2)$, where $\epsilon$ is the function approximation error and $h$ is the maximal edge length. This ensures the output remains triangulation-agnostic.

3. Architecture Integration

The proposed attention layers are integrated into existing mesh learning backbones. In each block, the backbone predicts the intrinsic queries, keys, and values, which are then processed by the mass-weighted attention mechanism. This allows the network to leverage existing optimized CUDA kernels for attention while maintaining geometric correctness.

Key Contributions

1. Intrinsic Attention Layer: A new attention mechanism that operates directly on triangle meshes by treating vertices as samples of continuous functions and using mass-weighted quadrature for discretization.
2. Triangulation Agnosticism: The method is proven to be invariant to changes in mesh triangulation, a property lacking in standard attention and many existing mesh transformers.
3. Retroactive Performance Boost: The authors demonstrate that this simple modification can be applied to older architectures (e.g., DiffusionNet from 2022) to outperform current state-of-the-art methods (e.g., PoissonNet from 2025) without requiring new backbone designs.
4. Theoretical Foundation: A formal proof showing that the proposed mass-weighted discretization well-approximates the continuous attention definition with linearly bounded error.

Experimental Results

The method was evaluated on four benchmarks, consistently outperforming both point-cloud transformers and state-of-the-art mesh-based architectures:

1. Predicting High-Frequency Intrinsic Signals:

   • The model predicts Fourier-like high-dimensional embeddings of vertices.
   • Result: Achieved a 6dB improvement in PSNR over the state of the art (PoissonNet). Unlike baselines (HodgeFormer, PTV3), the method maintained accuracy on meshes with significantly different triangulations (re-triangulated, coarsened, or subdivided).

2. Shape Deformation:

   • The network predicts vertex displacements conditioned on pose parameters.
   • Result: Significant improvement in granularity, specifically achieving articulated finger control which previous methods failed to capture accurately.
   • Efficiency: A version using a 2022-era backbone (DiffusionNet) with the new attention layers outperformed the 2025 state-of-the-art (PoissonNet without attention).

3. Dense Correspondence:

   • The model computes dense correspondences between full and partial shapes using a naive nearest-neighbor matching on predicted features.
   • Result: Outperformed DiffuMatch (a diffusion-based SOTA) on 4 out of 5 datasets (FAUST, SCAPE, SH19, D-Inter) without specific training for correspondence tasks. The method produced smooth, continuous mappings, whereas competitors exhibited discontinuities on out-of-distribution shapes.

4. Surface Descriptors (Heat Kernel Signatures):

   • Predicting HKS features on general meshes (Thingi10K).
   • Result: Achieved 2x higher accuracy (lower MSE) compared to PoissonNet.

Significance and Claims

The paper claims that the lack of intrinsicality and triangulation-agnosticism in standard attention mechanisms is the primary reason for their failure to match the performance of geometric networks on meshes. By embedding these geometric principles directly into the attention mechanism via mass-weighted FEM discretization, the authors provide a "simple and powerful" approach that:

• Surpasses current SOTA in multiple geometry processing tasks.
• Generalizes robustly to different mesh discretizations and out-of-distribution shapes.
• Can be retroactively applied to existing architectures to yield immediate performance gains.

The authors note limitations, including the requirement for manifold meshes (single connected component) due to the backbone and scalability issues with self-attention on very large meshes. They suggest future work in extending the mechanism to non-manifold cases and exploring conditioning on other modalities (vision, language).