Learning Convex Decomposition via Feature Fields

This paper introduces a novel, self-supervised feature field learning approach that enables the first feed-forward model for open-world 3D convex decomposition, producing high-quality, generalizable results across diverse representations like meshes, CAD models, and Gaussian splats to accelerate applications such as collision detection.

The Gist

Imagine you have a giant, intricate, and oddly shaped jigsaw puzzle piece made of clay. Now, imagine you need to put this piece into a box, but the box only accepts simple, blocky shapes like cubes, spheres, or pyramids.

This is the problem of Convex Decomposition. In the world of 3D computer graphics (like video games and robot simulations), computers struggle to calculate how complex, curvy objects bump into each other. It's too slow and messy. To fix this, we need to break those complex shapes down into a pile of simple, "convex" blocks (shapes where if you draw a line between any two points inside, the line never leaves the shape).

For a long time, doing this was like trying to solve a Rubik's cube blindfolded. It was slow, required human artists to do it manually, or used old math tricks that were too slow for the internet age.

This paper introduces a new, super-smart way to do this automatically, using a method the authors call "Learning Feature Fields." Here is how it works, explained with everyday analogies:

1. The Old Way vs. The New Way

• The Old Way (The Sculptor): Imagine a sculptor trying to carve a complex statue out of a block of wood. They have to chisel away piece by piece, checking constantly if the piece is still "convex." It takes forever, and if the statue is weird, the sculptor might get stuck.
• The New Way (The Paint-by-Numbers): Instead of carving, imagine you have a magic paintbrush. You paint the surface of the object with different colors. If two spots are the same color, they belong in the same block. If they are different colors, they belong in different blocks.

The authors' method is like that magic paintbrush. It doesn't try to cut the shape directly. Instead, it paints a "feature map" over the object.

2. The Magic Paintbrush: "Feature Fields"

The core idea is to teach a computer to paint the object with invisible "colors" (mathematical numbers) that tell it which parts should stick together.

• The Rule of the Game: The computer learns a simple rule: "If you can draw a straight line between two points without hitting the outside air, they should have the same color."
• The Training: The computer looks at millions of 3D shapes. It picks two points.
  • If the line between them stays inside the object, it says, "Okay, these two points are friends! Give them similar colors."
  • If the line goes outside the object (like cutting through a hollow leg of a chair), it says, "Nope, these are strangers! Give them very different colors."

By doing this millions of times, the computer learns to "paint" the object so that all the parts that naturally form a convex block have similar colors, and the parts that shouldn't be together have different colors.

3. The Clustering (The Grouping)

Once the computer has painted the whole object with these invisible colors, the final step is easy. It just looks for groups of similar colors and says, "Okay, all these red spots go in Box A, all these blue spots go in Box B."

Then, it wraps a tight bubble (a "convex hull") around each color group. Suddenly, your complex, curvy object is perfectly represented by a pile of simple, bouncy blocks.

Why is this a Big Deal?

The paper highlights three superpowers of this new method:

1. It's Fast (The Express Lane):
   Old methods were like solving a maze every time you wanted to simulate a crash. This new method is like having a GPS that instantly tells you the route. It can process a 3D shape in seconds, making it perfect for real-time video games and robot training.

2. It's "Open-World" Ready (The Chameleon):
   Previous AI models were like students who only studied for one specific test. If you gave them a shape they hadn't seen before (like a weird alien creature or a scanned real-world object), they failed. This new model is like a genius student who understands the concept of shapes. It works on:

   • CAD models (blueprints).
   • 3D Scans (messy, real-world photos of objects).
   • Gaussian Splats (a new, fuzzy way of representing 3D scenes).
     It doesn't care what the input looks like; it just sees the geometry and knows how to break it down.

3. It's Adjustable (The Zoom Lens):
   Sometimes you want a rough approximation (just a few big blocks), and sometimes you want a super-detailed one (hundreds of tiny blocks). Because the computer learned a continuous "painting" of the object, you can just turn a dial to decide how many blocks you want, and it instantly re-groups the colors to match.

Real-World Impact

Why do we care?

• Video Games: When a car crashes in a game, the physics engine needs to know how the metal bends. If the car is one complex mesh, the math takes too long. If it's broken into 50 simple blocks, the crash happens instantly and looks realistic.
• Robotics: Robots need to know how to pick up a weirdly shaped mug without dropping it. They need to quickly calculate if the mug will fit in their gripper or if it will collide with a table. This method gives them that super-fast calculation power.

Summary

Think of this paper as teaching a computer to see the "skeleton" of any 3D object instantly. Instead of struggling to cut a complex shape into pieces, it learns to "feel" the shape and naturally group the parts that belong together, turning a chaotic mess of polygons into a neat, efficient pile of building blocks. It's the difference between manually sorting a pile of mixed Lego bricks and having a machine that instantly snaps them into their correct, pre-built structures.

Technical Summary

Here is a detailed technical summary of the paper "Learning Convex Decomposition via Feature Fields".

1. Problem Statement

Convex decomposition is the process of approximating a complex, non-convex 3D shape with a union of simpler convex bodies. This is a critical preprocessing step for:

• Physical Simulation: Accelerating collision detection (the primary bottleneck in rigid body dynamics).
• Other Applications: Signed-distance computation, motion animation, and skeleton extraction.

Challenges:

• Computational Complexity: Finding the optimal decomposition (minimum number of strictly convex components) is an NP-hard combinatorial problem.
• Limitations of Traditional Methods: Classical algorithms (e.g., V-HACD, CoACD) use branch-and-bound or heuristic search. While robust, they are computationally expensive, often slow, and struggle with complex geometries or open-world data.
• Limitations of Existing Learning Methods: Previous deep learning approaches (e.g., BSP-Net, Cvx-Net) typically rely on reconstruction objectives or fixed primitive sets. They often fail to generalize to "open-world" objects (diverse topologies) and are generally limited to specific datasets like ShapeNet. They also struggle with non-mesh inputs like Gaussian splats.

2. Methodology

The authors propose a novel feature learning framework that reformulates convex decomposition as a continuous embedding problem, enabling a feed-forward, self-supervised model.

A. Core Insight: Convexity as Feature Clustering

Instead of directly optimizing a discrete set of primitives, the method learns a continuous feature field $f: \mathcal{M} \to \mathbb{R}^k$ defined over the surface of the shape.

• Goal: Points that belong to the same convex component should have similar features (small distance in feature space), while points that cannot belong to the same convex component should have dissimilar features.
• Geometric Definition: A shape is convex if the line segment connecting any two points lies entirely within the shape. The method leverages this definition to define "convex pairs" (points connected by a line inside the volume) and "non-convex pairs" (lines exiting the volume).

B. Self-Supervised Contrastive Loss

Since ground-truth convex decompositions are unavailable for open-world data, the authors introduce a purely geometric, self-supervised loss:

1. Triplet Sampling: For a source point $x$, the model samples:
   • Positive Pair ($p$): A point such that the segment $xp$ is inside the shape (convex pair).
   • Negative Pair ($n$): A point such that the segment $xn$ exits the shape (non-convex pair).
   • Hard Negative Sampling: The method preferentially samples negative points spatially close to $x$ to create challenging training examples.
2. Contrastive Objective: The loss function minimizes the distance between features of positive pairs and maximizes the distance for negative pairs (using a log-normalized similarity metric). This forces the network to learn a feature space where convexity is encoded geometrically.

C. Architecture

• Input: A point cloud sampled from the input shape (supports meshes, point clouds, and Gaussian splats).
• Encoder: A PVCNN encoder processes the input point cloud.
• Representation: Features are projected onto three axis-aligned 2D planes (Triplane representation).
• Processing: A 2D CNN downsamples the triplanes, followed by a Transformer module, and finally a transposed 2D CNN upsamples them to reconstruct the final feature triplanes.
• Output: A dense feature field defined over the 3D space, allowing feature retrieval at any query point.

D. Inference: Recursive Decomposition

At inference time, the model predicts the feature field for a new shape.

1. Clustering: Features are sampled on the surface and clustered (using agglomerative clustering for meshes or k-means for point clouds) to partition the shape.
2. Recursive Strategy: Instead of fixing the number of clusters, the algorithm uses a divide-and-conquer approach. It recursively splits clusters until the concavity metric (deviation from the convex hull) of each component falls below a user-specified threshold $\epsilon$. This allows dynamic control over decomposition granularity.

3. Key Contributions

1. New Formulation: The first formulation of convex decomposition as a contrastive feature learning problem, enabling the creation of the first feed-forward open-world model for this task.
2. Self-Supervised Geometric Loss: A novel loss function derived strictly from the geometric definition of convexity, eliminating the need for ground-truth decomposition labels.
3. Scalability and Generalization: The model generalizes across diverse 3D modalities (meshes, CAD models, 3D scans, and Gaussian splats) and open-world datasets (Objaverse), unlike previous methods restricted to specific categories.
4. Performance: Achieves state-of-the-art results in both decomposition quality (lower concavity) and reconstruction accuracy compared to classical and learning-based baselines.

4. Experimental Results

The method was evaluated on V-HACD, PartObjaverse-Tiny, and ShapeNet datasets.

• Quantitative Performance:
  • Outperformed classical baselines (V-HACD, CoACD) and learning-based baselines (BSP-Net, Cvx-Net) in Concavity (geometric fit) and Reconstruction Error (Chamfer distance) across all datasets.
  • Achieved lower concavity scores with fewer components, indicating more efficient decompositions.
• Qualitative Analysis:
  • Successfully captures inclined convex regions that classical methods miss (due to axis-aligned cut assumptions).
  • Preserves large convex structures (e.g., submarine hulls) and separates nearby parts effectively.
  • Generalizes to unseen categories and complex geometries where other learning methods fail.
• Applications:
  • Collision Detection: Demonstrated a 5x speedup in physics simulation steps (8ms vs 40ms) when using the decomposed convex hulls compared to original meshes.
  • Multi-Modal Input: Successfully processed CAD models, 3D scans, and AI-generated Gaussian splats without retraining.

5. Significance and Impact

This work bridges the gap between computational geometry and deep learning for 3D processing.

• Efficiency: It replaces slow, iterative search algorithms with a fast, single-pass feed-forward network.
• Open-World Readiness: By removing the reliance on category-specific ground truth, it enables convex decomposition for the vast, unstructured 3D data generated by modern AI and scanning pipelines.
• Versatility: The ability to handle Gaussian splats and point clouds directly makes it highly relevant for emerging 3D representation formats in robotics and simulation.

Limitations: The model currently struggles with highly incomplete geometry (scenes) and extremely thin structures (e.g., fan frames), suggesting future work should focus on scene-scale training and noise-injected data.