ParCo-SDF: Learning Prior-Free Partial-to-Complete Signed Distance Fields of Deformable Objects

Technical Summary: ParCo-SDF

Problem Statement
Deformable object manipulation (DOM) faces significant challenges due to the infinite degrees of freedom and continuous geometric changes inherent in deformable objects (DOs). These properties frequently lead to severe self-occlusions and robot-induced occlusions, obscuring critical state information required for control. While recent approaches utilize Implicit Neural Representations (INRs) to model continuous surfaces, they often rely on object-specific shape priors (e.g., predefined templates or symmetry) to stabilize training and infer complete geometry from partial observations. Such priors limit generalization to unseen deformation scenarios. Furthermore, existing conditioning mechanisms, such as input concatenation or hypernetworks, struggle with scalability, representational capacity, or training stability when handling diverse instances and continuous deformation states. The core problem addressed is the reconstruction of complete, high-fidelity signed distance fields (SDFs) for deformable objects from partial, temporally ordered point-cloud observations without relying on explicit shape priors.

Methodology: ParCo-SDF
The authors propose ParCo-SDF, a two-stage framework designed for prior-free partial-to-complete SDF reconstruction. The architecture conditions a shared implicit network on a latent code derived from a sliding window of partial observations $\{P_{t-T+1}, \dots, P_t\}$.

1. Temporal Geometry Encoding (Stage 1):
   Instead of aggregating raw point sets, which is sensitive to irregular sampling, the encoder transforms observations into a compact latent space.

   • Point-level Feature Lifting: Raw coordinates are mapped to a higher-dimensional space using learnable Fourier-feature mapping to enhance spatial expressiveness.
   • Local Geometric Encoding: A Dynamic Graph Convolutional Neural Network (DGCNN) processes each lifted observation to generate a permutation-invariant, geometry-aware latent code ($\tilde{z}_i$) that captures local partial structures.
   • Temporal Aggregation: A self-attention mechanism aggregates the sequence of observation-level codes to infer the deformation state at time $t$, producing a final latent code $z_t$. This allows the model to capture structural similarities across the sequence without explicit priors.

2. FiLM-Conditioned SDF Prediction (Stage 2):
   The latent code $z_t$ conditions a shared implicit SDF network (based on SIREN architecture) via Feature-wise Linear Modulation (FiLM).

   • Shift-Only Modulation: To improve optimization stability and avoid amplifying sinusoidal pre-activations, the modulator predicts layer-wise shift parameters ($\beta^{(\ell)}$) rather than scaling factors.
   • Parameter Efficiency: Unlike hypernetworks that predict entire weight matrices (leading to quadratic parameter growth), this approach predicts shift parameters that scale linearly with network width. This enables a larger implicit network capacity with stable training.
   • Training Objective: The model is trained using a weighted sum of three losses:
     • Surface Supervision ($L_{surface}$): Enforces zero-level set consistency and correct surface normals using a Focal loss formulation to handle class imbalance in sparse, highly deformed configurations.
     • Eikonal Regularization ($L_{eik}$): Enforces the SDF property $\|\nabla f_\theta(x)\| = 1$ at off-surface points.
     • Latent Space Regularization ($L_z$): Penalizes the Euclidean norm of the latent code to prevent unbounded growth and improve generalization.

Key Contributions

• Prior-Free Reconstruction: The method achieves robust partial-to-complete geometry reconstruction without relying on object-specific shape priors, enabling generalization to unforeseen structures and deformations.
• Temporal Geometry Encoding: By aggregating a sequence of partial observations via self-attention, the framework captures structural information necessary to resolve occlusions that are ambiguous in single frames.
• Stable, Expressive Conditioning: The introduction of a FiLM-based shift-only modulation mechanism allows for a larger shared implicit network with stable training, overcoming the scalability and instability issues of hypernetwork-based conditioning.
• Robustness to Occlusion: The framework is specifically designed to handle severe self-occlusions and robot-induced occlusions common in DO manipulation.

Experimental Results
The method was evaluated on a rubber band manipulation dataset generated in Isaac Sim, involving sequences with elongation, bending, and twisting. The model was trained on four sequences and evaluated on a held-out sequence with unseen deformation trajectories.

• Metrics: Performance was measured using Chamfer Distance (CD) for geometric fidelity and Topology Success Rate (TSR) for topological correctness (genus preservation).
• Comparison: ParCo-SDF outperformed the state-of-the-art baseline, INR-DOM (which uses a hypernetwork and skeleton loss).
  • ParCo-SDF reduced CD by 23.6% and improved TSR by 21.9% compared to INR-DOM.
  • The baseline (INR-DOM) exhibited higher variance in reconstruction errors, suggesting instability under varying occlusion patterns, whereas ParCo-SDF demonstrated significantly lower variance.
• Ablation: Removing the temporal aggregation layers (ParCo-SDF-T) resulted in a performance gap, confirming the critical role of temporal cues in inferring complete topology from partial observations.
• Qualitative: In cases of severe occlusion, ParCo-SDF successfully reconstructed the torus topology of the rubber band, whereas the baseline often produced spurious connections and surface artifacts.

Significance and Claims
The paper claims that ParCo-SDF provides a promising foundation for prior-free deformable geometry reconstruction. By leveraging temporal geometry encoding and FiLM-based conditioning, the method achieves robust, high-fidelity reconstruction under severe occlusions and previously unseen deformation states. The authors note that while current experiments are limited to simulation and a single object category (rubber bands), the framework addresses fundamental limitations in generalization and stability found in existing INR-based approaches. Future work is identified as investigating generalization across diverse deformable objects and real-world robotic manipulation scenarios.