Proximity3D: Shape from Capacitive Proximity on Sensing Manifold

Technical Summary: Proximity3D: Shape from Capacitive Proximity on Sensing Manifold

Problem Definition
In robotic manipulation, the "near-field" region—the narrow gap between a robot and an object prior to contact—is often occluded from line-of-sight sensors like cameras. While non-contact capacitive proximity sensing offers a solution by detecting conductive objects via electric field perturbations, existing approaches typically treat these signals merely as collision heuristics or simple distance estimates. They fail to recover the rich spatial information encoded in the signals.

The core challenge addressed by this paper is the inverse problem of reconstructing 3D object geometry from sparse, indirect capacitive proximity fields captured on a curved, non-planar sensing manifold (e.g., a woven textile). Unlike standard 3D reconstruction which relies on dense visual or depth data, capacitive scans provide only a few hundred channel responses. These responses are indirect (reflecting complex capacitive coupling rather than direct depth) and highly dependent on the local geometry and orientation of the sensor's electrode layout.

Methodology: Proximity3D
The authors propose Proximity3D, a feedforward reconstruction pipeline that maps multi-view capacitive fields directly to 3D object meshes. The system operates in three main stages:

1. Manifold Sensing Attention (MSA):
   To address the non-uniform and curved nature of the sensor, the network processes data directly on the sensing manifold. It employs a novel attention mechanism that aggregates neighboring channel responses based on their local electrode tangent frames.

   • Each channel is initialized with its capacitive response and a geometry descriptor (position and 6D rotation).
   • The MSA operator maps neighboring electrodes into the local tangent frame of a target electrode, encoding local displacement and relative orientation.
   • This allows the network to learn the directional dependencies of the capacitive coupling, effectively interpreting the sparse signals in the context of the sensor's physical layout.

2. Multi-View Fusion via Shape Tokens:
   The per-view features (aggregated via MSA) are fused with their known sensor poses (rotation and translation) using Fourier encoding. These are combined with learnable "shape tokens" and "register tokens" within a transformer architecture.

   • The shape tokens act as a compact, global representation of the object's geometry, selectively aggregating evidence from all views.
   • This stage bridges the gap between sparse, pose-dependent sensor data and a global 3D representation.

3. Shape Decoding with Pretrained Priors:
   Due to the extreme sparsity of the input data, the pipeline leverages a strong pretrained shape prior (specifically the TRELLIS-2 decoder).

   • The shape tokens are cross-attended to a structured 3D latent grid.
   • A learned query predicts occupancy probabilities and latent features for each cell in the grid.
   • The resulting sparse latent representation is fed into the frozen TRELLIS-2 decoder to synthesize the final 3D mesh.

Surrogate Forward Model for Data Generation
Recognizing that physical data collection is time-consuming, the authors introduce a surrogate forward model to generate training data.

• This model learns to map "geometric priors" (derived from raycasting in the sensor's local tangent frame) to realistic capacitive responses.
• It is implemented as a Conditional Diffusion Transformer (DiT) that incorporates the same MSA module used in the reconstruction pipeline.
• The model is pre-trained on Finite Element Method (FEM) simulations and fine-tuned on a small set of physical measurements to align with real-world hardware distributions.

Key Contributions

1. Proximity3D Framework: The first method to effectively reconstruct 3D shapes from multi-view capacitive proximity fields captured on a curved sensing manifold.
2. Manifold Sensing Attention (MSA): A novel attention mechanism that explicitly integrates local electrode geometry and connectivity to interpret sparse, direction-dependent capacitive signals.
3. Surrogate Capacitive Model: A generative approach that learns realistic sensor responses from local geometry, enabling scalable data generation for training without extensive physical collection.

Experimental Results
The system was evaluated using a hemispherical woven sensor with 100 electrode channels and a robotic arm, as well as a palm-mounted sensor on a dexterous hand.

• Reconstruction Quality: On 200 unseen test shapes, the MSA-based approach significantly outperformed baselines using vanilla self-attention and Graph Attention Networks (GAT). Using 64 views, the method achieved a Chamfer Distance (CD) of 0.0078 and an Average Surface Error of 0.0071.
• View Density: Performance improved with view count up to 64 views, after which it plateaued, indicating efficient information fusion.
• Surrogate Validation: The surrogate forward model, when fine-tuned on real data, achieved high correlation (Pearson 0.9990) with physical measurements, validating its utility for data generation.
• Robotic Application: In a grasp planning task using a palm-mounted sensor, reconstructions enabled a 90% success rate for force-closure grasps, significantly outperforming planning based on ground-truth convex hulls (78%), Oriented Bounding Boxes (68%), and Axis-Aligned Bounding Boxes (65%).

Significance and Limitations
The paper claims that Proximity3D represents a significant step toward embodied near-field geometric awareness, allowing robots to recover object geometry before contact. This capability is crucial for applications like grasp planning and safe human-robot interaction.

The authors note specific limitations:

• Material Constraints: The method is currently limited to conductive objects due to the reliance on electric field coupling. However, experiments suggest robustness across different conductive materials (stainless steel, gold, silver, copper, aluminum) and internal structures (solid vs. hollow) without material-specific retraining.
• Weak Evidence: In cases of insufficient capacitive evidence (e.g., deep concavities or high genus shapes), the reconstruction relies more heavily on the pretrained prior, leading to graceful degradation rather than total failure, though accuracy decreases.
• Surrogate Approximation: While the surrogate model is effective, the paper acknowledges that some reconstruction errors may stem from the approximation of physical responses by the generative model.

In conclusion, the work demonstrates that sparse, indirect capacitive signals on curved manifolds can be transformed into high-fidelity 3D meshes, opening a new route for robotic sensing that does not require line-of-sight or physical contact.