Technical Summary: PhyPush

Problem Statement

Accurate estimation of object physical properties, specifically mass and friction coefficients, is critical for reliable and adaptive robotic manipulation in unstructured environments. While interactive perception offers a mechanism to infer these latent properties, existing approaches face significant scalability and deployment barriers. Most state-of-the-art methods rely on specialized, expensive hardware such as force/torque (F/T) sensors, tactile arrays, or multi-camera motion-capture systems. Furthermore, many techniques require multiple, diverse interactions to converge on an estimate, increasing identification time and reducing operational efficiency. These constraints limit the applicability of such systems to controlled laboratory settings and hinder their deployment on standard robotic arms lacking specialized sensing capabilities.

Methodology

The paper introduces PhyPush, a sensorless framework designed to estimate an object's mass ($m$) and kinetic friction coefficient ($\mu$) using only the kinematic data from a single translational push. The system operates without external sensors, relying solely on the end-effector velocity profile available on standard robotic arms.

Core Architecture

PhyPush utilizes a Transformer-based encoder-only architecture with a dual-stream design:

1. Input Processing: The system records the end-effector velocity sequence ($v_{ee}$) at 100 Hz during a single push. The input window captures both the transient phase (initial impact and acceleration) and the steady-state phase (constant velocity sliding).
2. Transformer Encoder: A standard Transformer encoder processes the velocity sequence to capture long-range temporal dependencies and global contextual understanding.
3. Dual-Stream Estimation: The encoded features are fed into two independent Multi-Layer Perceptron (MLP) streams to separately estimate mass and friction.
4. Physical Property Attention: Instead of simple averaging, the model employs a dual-stream attention mechanism. Trainable global queries selectively attend to the most informative time intervals for each property. This allows the network to focus on the transient phase for mass estimation (dominated by Newtonian inertia) and the steady-state phase for friction estimation (dominated by Coulomb friction).

Physics-Guided Loss Formulation

A central component of the methodology is the physics-guided loss function, which constrains the learning process using fundamental physical laws rather than relying solely on data-driven statistical correlations. The loss is derived from:

• Newton's Second Law ($F_{net} = ma$): Enforced during the transient phase to ensure consistency between net force, estimated mass, and object acceleration.
• Coulomb Friction Model ($F_{fric} = \mu F_{norm}$): Enforced during the steady-state phase to ensure the estimated friction coefficient aligns with the observed friction force.

The authors propose three loss variants to address different supervision levels:

1. Semi-Supervised Force-Based Loss ($L_{force}$): Uses ground-truth force and acceleration data to enforce physical equations directly.
2. Unsupervised Force-Based Loss ($L_{force, un}$): Introduces a constraint coupling the estimated mass and friction ($\hat{m}$ and $\hat{\mu}$) mathematically, allowing training without explicit force labels by enforcing consistency between the two predictions.
3. Unsupervised Force/Acceleration-Based Loss ($L_{force, acc, un}$): A hybrid formulation that integrates both mass and friction into a single acceleration consistency equation, further deepening the coupling between the two estimators.

These unsupervised terms are introduced via a curriculum-style annealing schedule to ensure stable training.

Key Contributions

1. Sensorless Estimation Framework: The introduction of PhyPush, which estimates mass and friction using only kinematic end-effector velocity, eliminating the need for F/T sensors, tactile arrays, or external motion capture.
2. Physics-Guided Transformer Architecture: A novel dual-stream Transformer that embeds constraints from Newton's Second Law and Coulomb friction directly into the training objective. This approach improves physical consistency and generalization compared to traditional data-driven losses.
3. Comprehensive Evaluation: Extensive validation in both simulation and real-world settings, demonstrating that physics-guided learning outperforms data-driven baselines, particularly in out-of-distribution (OOD) scenarios.

Experimental Results

Simulation Performance

Evaluated on a Franka Emika robot in a simulated environment with 31,759 pushing samples, PhyPush was tested against a baseline method (re-implemented from prior work) that relies on privileged F/T sensor data.

• Generalization: While the F/T sensor baseline performed well on in-domain data, PhyPush variants consistently outperformed it on out-of-distribution (OOD) test sets involving unseen masses, friction coefficients, or both.
• Error Reduction: In the most challenging OOD setting ($D_{m,\mu}^{OOD}$), PhyPush achieved lower estimation errors for both mass and friction than the baseline, despite the baseline having access to full force information.
• Loss Comparison: Models trained with physics-guided losses ($L_{force}$, $L_{force, un}$, $L_{force, acc, un}$) significantly outperformed the purely data-driven loss ($L_{data}$). Specifically, the $L_{force, un}$ variant reduced friction estimation error by over 28% compared to the data-only loss in OOD conditions.
• Physical Consistency: Models trained with physics-guided losses demonstrated higher $R^2$ scores when validating predictions against Newton's Second Law and Coulomb's Law, indicating that the model learns underlying physical principles rather than mere statistical correlations.

Real-World Performance

Experiments were conducted on 47 distinct object-surface configurations, including novel objects (e.g., tin cans, plastic containers) not seen during training.

• Robustness: Physics-guided variants consistently tracked ground-truth trends more closely than the data-driven baseline.
• Bias Correction: The data-driven baseline exhibited systematic biases (underestimating mass and overestimating friction), whereas the physics-guided models maintained accuracy and physical consistency.
• Hardware Independence: The system successfully generalized to unseen objects and surfaces without specialized contact sensors, relying only on the robot's internal velocity data.

Significance and Claims

The paper claims that PhyPush demonstrates the viability of low-cost, sensor-efficient physical property estimation. By leveraging the direct coupling between a robot's kinematic response and an object's dynamics, the framework achieves high accuracy without the overhead of specialized hardware.

The authors emphasize that physics-guided learning is the key enabler for robustness. By grounding the learning process in fundamental laws, the model avoids overfitting to specific training distributions and maintains performance in challenging, unseen real-world conditions. The work suggests that a single push, combined with a physics-informed Transformer, is sufficient for reliable estimation, offering a scalable solution for embodied AI systems operating in unstructured environments.

The paper concludes modestly, noting current limitations such as the assumption of predominantly translational pushes and the reliance on velocity measurements. It identifies future directions including the incorporation of rotational dynamics and the development of adaptive pushing policies via reinforcement learning to further expand the range of estimable physical responses.