Reduced-order Neural Modeling with Differentiable Simulation for High-Detail Tactile Perception

Technical Summary: Reduced-order Neural Modeling with Differentiable Simulation for High-Detail Tactile Perception

Problem Statement
High-resolution tactile perception is essential for dexterous robotic manipulation, yet simulating the deformation of elastomeric sensors (such as GelSight) at the necessary fidelity remains computationally prohibitive. Traditional approaches face a trade-off:

• Finite Element Methods (FEM): Offer high accuracy but require complex, adaptive remeshing to handle sharp contacts and large deformations, hindering scalability and real-time integration.
• Material Point Methods (MPM): Provide flexibility and robustness to topological changes but suffer from heavy memory costs when high particle densities are required to capture fine surface details like wrinkles and micro-indentations.
• Proxy-based Methods: While efficient, they often lack physical realism by approximating soft contact through geometric penetration, failing to capture accurate stress-strain responses.

Consequently, existing differentiable simulators struggle to balance physical fidelity with the computational efficiency required for robotic learning and optimization.

Methodology
The authors propose a reduced-order neural simulation framework that couples coarse-grained MPM dynamics with an implicit neural decoder. The core insight is that the high-dimensional state space of detailed tactile deformation can be compressed into a low-dimensional latent manifold.

1. Reduced-Order Formulation: Instead of evolving the full particle configuration $q_t$ (which scales linearly with particle count $N$), the method solves for a compact latent coordinate $z_t \in \mathbb{R}^r$ (where $r \ll n$). The full state is reconstructed via a neural decoder $f_\theta(z_t)$.
2. Latent Dynamics: The system evolves the latent state by minimizing an energy functional projected onto the subspace manifold. This involves an inertial penalty term and potential energy, ensuring the latent dynamics faithfully approximate high-dimensional evolution.
3. Neural Architecture: An autoencoder structure is trained on paired high-resolution ($10^6$ particles) and low-resolution ($10^4$ particles) MPM simulations.
   • Encoder: Maps full particle configurations to latent coordinates.
   • Decoder: Reconstructs high-fidelity particle states (position and deformation gradient) from latent codes.
   • Consistency Loss: A specific loss term ($L_{cons}$) penalizes deviations in velocity and deformation gradients between the reconstructed high-resolution state and the coarse MPM prediction, ensuring physical consistency in momentum and strain evolution.
4. Tactile Rendering: The framework decouples global mechanics from detail reconstruction. The coarse MPM solver handles global dynamics efficiently, while the neural field refines the deformation field. Tactile images are generated by tracing view rays through the reconstructed high-resolution surface, accounting for light refraction at the gel-air interface, directly from the reduced-order latent state.

Key Contributions

• Hybrid Framework: A novel integration of differentiable MPM with reduced-order neural modeling, enabling sub-particle tactile detail reconstruction from low-dimensional latent states without the memory overhead of full-resolution simulation.
• Physical Grounding: The approach maintains differentiability and preserves the underlying physics of MPM, ensuring stable and plausible interactions suitable for gradient-based optimization.
• Super-Resolution Rendering: The method generates realistic depth images and surface meshes with fine-scale features (e.g., hair-like textures, wrinkles) directly from the reduced model, bypassing the need for full-resolution MPM computation during inference.

Experimental Results
The framework was validated through simulation and real-world experiments using a UR5e robot equipped with a GelSight Mini sensor.

• Efficiency: Compared to the TacIPC baseline, the proposed method achieves over 65% faster simulation and 40% lower memory usage. In runtime comparisons, it is approximately 3× faster than TacIPC and 4.3× faster than Tacchi, while maintaining comparable geometric accuracy (Chamfer distance).
• Accuracy: In tactile rendering tasks, the method improves accuracy by 25% (measured by SSIM and PSNR) compared to baselines, significantly reducing Mean Absolute Error (MAE).
• Fidelity: Qualitative and quantitative results demonstrate superior preservation of complex surface morphologies, including microscopic ridges and localized indentations, which are often smoothed out or lost in low-resolution simulations or FEM-based approaches.
• 3D Reconstruction: The method produces structurally coherent 3D meshes from rendered tactile images with lower error than Tacchi or TacIPC, avoiding depth-induced artifacts.

Significance and Claims
The paper claims that this reduced-order neural model enables high-detail, physically grounded tactile simulation with substantial efficiency gains. By decoupling global mechanics from local detail reconstruction, the framework makes high-fidelity tactile perception feasible for robotic interaction and optimization tasks that were previously limited by computational cost. The authors emphasize that their method achieves the accuracy of full-resolution MPM or FEM solvers at a fraction of the cost, facilitating real-time robotic simulation and gradient-based learning.

Limitations and Future Work
The authors note that the current formulation targets quasi-static or low-speed indentation regimes; highly dynamic interactions (fast sliding, rolling, impact) involving inertial effects are not explicitly modeled. Additionally, generalization to unseen materials is limited due to training under fixed constitutive settings. The work focuses on geometric fidelity rather than absolute force estimation, and tactile rendering assumes fixed illumination. Future directions include incorporating isosurface-based contact representations and dynamic sliding interactions.