VoMP: Predicting Volumetric Mechanical Property Fields

VoMP is a fast, feed-forward deep learning method that predicts spatially-varying volumetric mechanical properties (Young's modulus, Poisson's ratio, and density) for 3D objects by aggregating multi-view features through a Geometry Transformer and decoding them via a physically plausible material manifold learned from a novel, multi-source annotated dataset.

The Gist

Imagine you have a digital 3D model of a coffee mug, a pile of leaves, or a bowling ball. Right now, most computer programs see these objects as just "shapes." They know the mug is round and the leaves are jagged, but they don't know what the objects are made of. To a computer, a mug made of glass and a mug made of foam look exactly the same until you try to drop them.

If you want to simulate a realistic scene—like a bowling ball crashing through a bed of pillows—you usually have to manually tell the computer: "Okay, the frame is steel, the springs are metal, and the pillow is soft foam." This is slow, boring, and requires an expert to guess the right numbers.

VoMP (Volumetric Mechanical Property Fields) is a new AI tool that does this guessing for you, instantly and accurately.

Here is how it works, explained with some everyday analogies:

1. The "X-Ray Vision" Problem

Most 3D models are like hollow shells. If you look at a 3D model of a tree, you see the leaves and the bark, but the computer doesn't know what's inside the trunk. Is it solid wood? Is it hollow? Is it filled with water?

VoMP is special because it doesn't just look at the surface; it looks inside. It treats the object like a 3D grid of tiny cubes (voxels), similar to how a CT scan breaks a human body into slices. It predicts the material properties for every single cube inside the object, not just the ones you can see from the outside.

2. The "Translator" (The Vision-Language Model)

How does the AI know that a shiny, gray object is "aluminum" and not "steel"?

• The Old Way: You had to manually look up a table of materials and type in the numbers.
• The VoMP Way: The system uses a "Vision-Language Model" (think of it as a super-smart librarian who has read every material science textbook ever written).
  • It looks at the object's texture and color.
  • It asks the librarian: "This looks like a metal chair leg. What are the real-world numbers for stiffness and weight?"
  • The librarian gives it a range of realistic numbers.
  • VoMP then uses these clues to fill in the 3D grid with the correct "recipe" for that material.

3. The "Safety Net" (The Latent Space)

Here is the tricky part: If you ask an AI to guess numbers, it might accidentally say, "This foam has the weight of a lead brick," or "This steel is as stretchy as a rubber band." That would break the physics simulation.

To prevent this, the researchers built a Safety Net (called a "Latent Space").

• Imagine a map of all possible real-world materials. On this map, you have steel, wood, rubber, and water.
• If you try to draw a point in the "impossible zone" (like a material that is both heavier than lead and stretchier than a balloon), the Safety Net pulls it back to the nearest real material.
• This ensures that every prediction VoMP makes is physically possible. You can't accidentally create a "magic material" that breaks the laws of physics.

4. The "Instant Chef" (Feed-Forward)

Many previous AI tools were like chefs who had to taste the soup, adjust the salt, taste it again, and adjust the pepper for every single dish they made. This took forever.

VoMP is like a super-fast automated chef.

• You hand it the 3D model (the ingredients).
• It instantly knows exactly how much "salt" (stiffness), "pepper" (density), and "spice" (elasticity) to add to every part of the dish.
• It does this in 3 seconds.
• It works on any shape: a mesh (like a video game model), a cloud of dots (Gaussian Splats), or even a blurry 3D scan.

Why Does This Matter?

Think about Digital Twins (virtual copies of real factories or cities). If you want to test if a new bridge design will hold up in a storm, or if a robot arm will break when it picks up a heavy box, you need accurate physics.

• Before VoMP: Engineers spent days manually assigning materials to every part of a 3D model.
• With VoMP: You upload the 3D model, and in seconds, the computer knows exactly how heavy, stiff, and bouncy every part is.

The Result: You can drop a virtual bowling ball on a virtual bed, and the springs will compress, the mattress will sag, and the ball will bounce exactly the way it would in real life, all without a human ever touching a spreadsheet. It turns static 3D shapes into living, breathing, physics-ready worlds.

Technical Summary

1. Problem Statement

Accurate physical simulation (e.g., for Digital Twins, robotics, and Real-2-Sim workflows) requires spatially varying mechanical properties—specifically Young's Modulus ($E$), Poisson's Ratio ($\nu$), and Density ($\rho$)—throughout the volume of 3D objects.

• Current Limitations: Existing 3D capture methods and repositories rarely contain these annotations. Artists and engineers must manually guess or copy-paste coarse material presets, a labor-intensive and subjective process.
• Prior Art Issues: Previous learning-based approaches often rely on:
  • Per-object optimization (slow, not feed-forward).
  • Simulator-specific parameters (not transferable across different physics engines).
  • Surface-only predictions (ignoring internal volume, critical for fidelity).
  • Coarse categorical labels rather than precise physical values.
  • Heavy reliance on Vision-Language Models (VLMs) at inference time, leading to brittleness and high latency.

2. Methodology: VoMP

VoMP is a feed-forward deep learning model designed to predict physically valid volumetric mechanical property fields for any 3D representation (Meshes, 3D Gaussian Splats, NeRFs, SDFs) that can be voxelized and rendered.

A. Core Architecture

The pipeline consists of three main stages:

1. Feature Aggregation (§4.1):
   • The input geometry is voxelized.
   • Multi-view images are rendered from the object.
   • DINOv2 features are extracted from these views and projected onto the 3D voxel grid.
   • Crucially, features are aggregated for interior voxels, not just the surface, allowing the model to infer internal material composition.
2. Geometry Transformer (§4.2):
   • A Transformer encoder (based on the TRELLIS architecture) processes the voxel features.
   • It outputs per-voxel material latent codes ($z \in \mathbb{R}^2$).
   • To handle large assets, it employs stochastic sampling of voxels during training.
3. Material Latent Space Decoding (MatVAE) (§3):
   • The latent codes are decoded by a pre-trained Variational Autoencoder (MatVAE).
   • MatVAE maps the 2D latent space back to the physical triplet $(E, \nu, \rho)$.
   • Key Innovation: The latent space is trained exclusively on a database of real-world measured material values. This acts as a "continuous tokenizer," ensuring that any point in the latent space decodes to a physically plausible material, preventing the generation of impossible property combinations.

B. Training Data Generation (§5)

To overcome the lack of volumetric material datasets, the authors propose an automatic annotation pipeline:

• Source Data: High-quality 3D assets with part-segmentation (e.g., NVIDIA SimReady, Residential, Vegetation packs).
• Material Database (MTD): A dataset of 100,000+ real-world material triplets collected from engineering databases (MatWeb, Cambridge, etc.).
• VLM Annotation: A Vision-Language Model (Qwen 2.5 VL-72B) is used to annotate parts.
  • Input: Full object render, part texture sphere, part name, and ranges of the 3 closest real-world materials from MTD.
  • Output: Precise $(E, \nu, \rho)$ triplets for each part.
  • Result: A dataset of 1,664 objects with ~37 million annotated voxels.

3. Key Contributions

1. First Feed-Forward Volumetric Predictor: The first method to estimate simulation-ready mechanical fields ($E, \nu, \rho$) in a single forward pass without per-object optimization.
2. Representation Agnostic: Works across Meshes, 3D Gaussian Splats, NeRFs, and SDFs.
3. Physically Valid Latent Space: Introduces MatVAE, a latent space constrained by real-world physics, guaranteeing that interpolated or predicted materials are physically plausible.
4. New Benchmark & Dataset: Created a new benchmark (GVM) with detailed volumetric annotations and a Material Triplet Dataset (MTD) for training.
5. Speed & Accuracy: Achieves significant speedups (seconds vs. hours) while outperforming prior art in accuracy.

4. Results

• Quantitative Performance:
  • VoMP significantly outperforms baselines (NeRF2Physics, PUGS, Phys4DGen, Pixie) on all metrics (ALDE, ALRE, ADE, ARE).
  • Young's Modulus Error: VoMP achieves an ALRE of 0.0409, compared to 0.1346 (NeRF2Physics) and 0.1688 (PUGS).
  • Density Error: VoMP achieves an ARE of 0.0921, compared to 1.0365 (NeRF2Physics).
• Speed:
  • VoMP processes objects in ~3.6 seconds (including rendering and voxelization).
  • Baselines take 200s to 1450s (e.g., NeRF2Physics: ~1454s).
  • This represents a 50x–100x speedup.
• Simulation Quality:
  • Simulations using VoMP-predicted fields (using FEM and Simplicits solvers) produce realistic deformations (e.g., falling bowling balls, falling ficuses) without manual tuning.
  • The method correctly predicts internal structures (e.g., dirt inside a pot, metal frames inside furniture) which surface-only methods miss.
• Validity:
  • Predicted materials fall within the ranges of real-world measured materials (MTD), whereas baselines often predict "invalid" values that might work in specific simulators but are physically impossible.

5. Significance

VoMP bridges the gap between visual 3D capture and high-fidelity physical simulation. By automating the assignment of accurate, volumetric mechanical properties, it:

• Democratizes Digital Twins: Enables rapid creation of physics-accurate virtual replicas from real-world scans.
• Accelerates Robotics & Simulation: Removes the bottleneck of manual material tuning for Sim-2-Real transfer.
• Enables New Workflows: Allows for realistic interaction with 3D Gaussian Splats and other novel representations in physics engines, which was previously impossible due to the lack of volumetric data.

The paper establishes a new standard for "simulation-ready" 3D assets, moving from subjective, coarse material presets to objective, physics-grounded volumetric fields.