FAWAM: Force-Aware World Action Models for Closed-Loop Contact-Rich Manipulation

Technical Summary: FAWAM – Force-Aware World Action Models for Closed-Loop Contact-Rich Manipulation

1. Problem Statement

Contact-rich robotic manipulation tasks (e.g., wiping, peeling, pushing) require continuous regulation of interaction with the environment. While visual observations are standard, they are often insufficient for inferring critical interaction states such as weak contact, excessive pressure, or jamming. Existing methods that incorporate force sensing typically treat it as an additional observation modality for policy learning or use it for low-level reactive control. However, these approaches fail to explicitly model the evolution of future contact states (e.g., future wrenches) and lack predictive references to guide execution-time corrections. Consequently, robots struggle to anticipate contact outcomes and adjust proactively when sensed feedback deviates from expectations.

2. Methodology

The authors propose FAWAM (Force-Aware World Action Model), a closed-loop framework that integrates force signals across three levels: perception, prediction, and execution. The system operates on two distinct frequencies: a Force-Envisioned Action Model running at 1 Hz for high-level planning, and a Force-Guided Residual Corrector running at 10 Hz for online adjustment.

A. Force-Envisioned Action Model
This component extends a world action model (based on GE-Act) to incorporate force awareness:

• Force Conditioning: Recent 6-axis force/torque histories ($w_{t-H_f+1:t}$) are encoded into a compact contact feature ($c_t$) via a lightweight MLP. This feature modulates the action generation pathway using AdaLN-Zero-style modulation, where force-conditioned offsets are added to the standard AdaLN-Zero parameters (shift, scale, gate) within the DiT blocks.
• Joint Prediction: Instead of using an isolated force-prediction head, the model replaces the original action projection with a joint linear projection. This outputs concatenated future action trajectories ($A_t$) and future wrench trajectories ($W_t$). This design forces the decoder to learn a shared representation that captures the coupling between robot motions and their physical contact consequences.
• Training: The model employs a two-stage training strategy. First, a video generation model is fine-tuned using latent-space flow matching. Second, the force-aware action decoder is trained with a combined loss function ($L_{stage2} = L_{action} + \lambda_F L_{force}$) to simultaneously predict future motions and contact forces under the same flow-matching framework.

B. Force-Guided Residual Correction
To address unexpected contact variations (e.g., sudden resistance) during the execution of an action chunk, a residual corrector is introduced:

• Inputs: The corrector takes the base action chunk, the predicted wrench reference, real-time wrench feedback, and the current proprioceptive state.
• Error Computation: It explicitly computes a wrench tracking error ($e^w_t$) by comparing measured force feedback ($w^{meas}$) with the predicted wrench trajectory ($w^{pred}$).
• Gating Mechanism: A gating head predicts a scalar gate ($g^{res}_t$) to determine if correction is necessary. A binary mask ($\kappa_t$) is applied based on a threshold to prevent the residual model from perturbing nominal executions.
• Output: The system outputs a residual action ($\Delta a^{res}$) which is added to the base action only when the gate is active, refining the execution online.
• Data Collection: Correction data is collected using a force-feedback leader-follower teleoperation system (FACTR). Human operators intervene when abnormal contact occurs, and the difference between the corrected command and the base policy command serves as the residual target.

3. Key Contributions

1. FAWAM Framework: A force-aware world action model that leverages force signals throughout perception, prediction, and execution levels to enable robust contact-rich manipulation.
2. Force-Guided Residual Correction: A novel mechanism that utilizes the discrepancy between predicted forces and sensed feedback to correct actions during execution, moving beyond simple reactive control to proactive, prediction-guided adjustment.
3. Empirical Validation: Extensive real-world experiments demonstrating significant performance gains over state-of-the-art baselines.

4. Experimental Results

Experiments were conducted on a Franka Research 3 (FR3) robotic arm across four tasks: Erase Whiteboard, Peel Cucumber, Pivot Box, and Wipe Vase. The tasks involved diverse contact conditions induced by varying surface inclinations, table heights, and object positions.

• Performance: FAWAM achieved an average success rate (SR) of 85.00%.
  • This represents a 36.25% improvement over vision-only baselines (e.g., $\pi0.5$ at 47.50%, GE-Act at 48.75%).
  • It shows a 21.25% improvement over existing force-aware baselines (ForceVLA and TA-VLA, both at 63.75%).
• Ablation Analysis:
  • The base policy without residual correction (FAWAM w/o Res) achieved a 73.75% success rate, indicating that the force-envisioned action model alone provides a strong foundation.
  • The addition of the residual corrector provided the final boost to 85.00%, validating the effectiveness of online correction.
  • The ablation study confirmed that all three components (Force-conditioned Observation, Joint Prediction, and Residual Correction) contribute positively to performance.

5. Significance and Claims

The paper claims that contact-rich manipulation is not merely a reactive observation-to-action mapping but a proactive prediction paradigm. By explicitly modeling the evolution of future contact states and using these predictions as online references, FAWAM allows robots to anticipate contact outcomes and correct deviations before failures occur. The authors assert that their systematic integration of force signals—moving beyond simple observation fusion to predictive modeling and residual correction—yields robust behaviors adaptable to dynamic environmental variations and external disturbances. The results suggest that future contact prediction is a critical missing component in current visuomotor policies for tasks requiring sustained physical interaction.