Look Both Ways Before You Cross: Lifting Cross Fields From 2D Visual Priors

Technical Summary: Look Both Ways Before You Cross: Lifting Cross Fields From 2D Visual Priors

Problem Statement
Quadrilateral (quad) meshes are the standard representation for 3D modeling due to their suitability for animation, parameterization, and editing. A critical requirement for high-quality quad meshes is feature alignment: the ability to align quad edges with both local geometric cues (e.g., sharp creases, curvature) and global semantic features (e.g., the axis of a limb, the symmetry of a face). Existing automated methods typically rely on geometric quantities, such as principal curvature directions, to guide alignment. However, these geometric signals are often local and fail to capture global semantic understanding or handle shapes where geometry is ambiguous (e.g., flat surfaces with semantic axes). Furthermore, manual alignment requires significant expertise and is time-intensive. While data-driven methods exist, many are limited by the scarcity of 3D training data or rely on models trained on geometric proxies rather than artist-created semantic layouts.

Methodology: CrossLift
The authors present CrossLift, a technique that generates feature-aligned cross fields on 3D meshes by leveraging 2D visual priors derived from text-to-image generative models. The method operates in three primary stages:

1. 2D Prior Generation and Gradient Extraction:

   • The input triangle mesh is rendered from six canonical views (front, back, left, right, top, bottom) to generate depth maps.
   • These depth maps, combined with a text prompt describing a quad mesh texture, condition a text-to-image model (specifically Flux in the experiments) to synthesize multi-view images depicting a wireframe quad mesh aligned with the object's features.
   • From these generated images, the system extracts per-pixel alignment directions. This involves computing image gradients using a Scharr kernel, filtering for high-magnitude gradients corresponding to grid lines, and applying a coherence filter to remove artifacts at grid intersections.

2. 3D Back-Projection and Two-Stage Interpolation:

   • The extracted 2D gradient vectors are back-projected onto the 3D mesh surface tangent space using the camera Jacobian and view transformations.
   • To consolidate these sparse and potentially conflicting signals into a smooth, unified cross field, the authors employ a two-stage interpolation process:
     • Stage 1 (Per-View Solve): For each view, the back-projected gradients are interpolated into a per-face cross field. This step uses a constrained linear system minimizing a smoothness energy (harmonic weights) and an alignment energy (matching the projected gradients). This ensures that each view contributes a dense, consistent signal regardless of the original gradient density in the image.
     • Stage 2 (Multi-View Solve): The per-view cross fields are consolidated into a single global cross field. This solve introduces custom confidence-based weights to resolve conflicts between views.
       • View Alignment Weight: Crosses on faces aligned with the view direction are weighted higher, as perspective distortion is minimized.
       • Coherence Weight: If multiple views agree on a face's alignment, the weight is high; if views conflict, the weight is lowered, allowing the smoothness term to dominate and resolve ambiguity.
   • The final field is normalized to unit length, resulting in a 4-RoSy (four-way rotationally symmetric) cross field.

3. Quad Mesh Extraction:

   • The resulting cross field is integrated to produce a parameterization (using polyvector curl correction and MiQ).
   • Integer isolines of this parameterization are extracted to form the final quad mesh.

Key Contributions

• Visual-Guided Alignment: The paper introduces a novel approach to quad meshing that utilizes the strong semantic and geometric understanding of pre-trained 2D generative models, bypassing the limitations of purely geometric or sparse 3D data-driven approaches.
• Two-Stage Interpolation: The authors propose a specific interpolation pipeline that back-projects gradients to the surface before solving, followed by a multi-view consolidation. This design prevents projection artifacts (where orthogonal surface vectors appear non-orthogonal in 2D) and ensures equal contribution from all views.
• Modularity: The method is modular, compatible with various 2D image generation models (demonstrated with Flux, Gemini 3, and ChatGPT 5.2) and can accept alternative 2D signals, such as user-drawn coarse lines on 2D renders.
• Robustness: The approach demonstrates robustness to surface noise (e.g., wrinkles on a tablecloth) by relying on global semantic priors rather than local geometric noise, and it handles occluded regions through smooth interpolation.

Results and Evaluation
The authors evaluate CrossLift on a diverse set of organic and mechanical shapes, comparing it against QuadriFlow, QuadWild, and NeurCross.

• Qualitative Results: CrossLift produces quad meshes that align with both local geometric features (e.g., studs on a Lego brick) and global semantic features (e.g., the symmetry of a lion's face or the flow of a cat's fur). In contrast, baseline methods often produce wavy edge flows (QuadriFlow/QuadWild) or overly rigid, axis-aligned grids that ignore semantic features (NeurCross).
• Quantitative Results: On the QuadWild300 dataset, CrossLift achieves a higher mean Scaled Jacobian (SJ) (0.9797 vs. 0.9674 for QuadriFlow), indicating superior quad quality with less stretching and twisting. It also achieves the lowest percentage of Irregular Vertices (IV) (1.4133%), suggesting smoother edge flow and fewer singularities.
• Applications: The paper demonstrates applications in texture-aligned quad meshing (aligning quads to texture patterns rather than geometry) and interactive design using coarse, user-drawn 2D lines as guidance.

Significance and Claims
The paper claims that CrossLift addresses the "non-trivial" challenge of aligning quads to feature directions by shifting the guidance signal from local geometry to global visual priors. The authors argue that while geometric quantities are good proxies, they cannot guarantee semantic alignment. By leveraging text-to-image models trained on billions of images, CrossLift captures nuanced semantic features that existing methods miss. The work highlights that this visual guidance allows for robust meshing on shapes with ambiguous geometry or noise, and provides a pathway for intuitive user control without requiring deep 3D modeling expertise. The authors note limitations, specifically that the method is not guaranteed to be equivariant to rigid transformations of the input mesh, though high-level alignment remains consistent.