💻 computer science

WorldContact: A Contact-Centric World Model for Scalable Robot Learning

The paper introduces WorldContact, a contact-centric world model that efficiently generates high-quality training data for deformable-object manipulation, achieving a 10x speedup over the source simulator and significantly improving real-world robot policy success rates from 65% to 95%.

Original authors: Caoliwen Wang, Mengdi Wang, Heng Zhang, Shixun Huang, Siyuan Chen, Chao Liu, Anpei Chen, Zhendong Wang, Peter Yichen Chen, Huamin Wang

Published 2026-09-18

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Caoliwen Wang, Mengdi Wang, Heng Zhang, Shixun Huang, Siyuan Chen, Chao Liu, Anpei Chen, Zhendong Wang, Peter Yichen Chen, Huamin Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Robots that can handle the messy, unpredictable nature of the real world face a fundamental hurdle: they need to learn how objects behave when touched, pushed, or lifted. Unlike a chess piece that moves in fixed patterns, a shopping bag, a shirt, or a piece of dough changes shape constantly, and those changes alter how they interact with everything else. To learn these skills safely and quickly, researchers often rely on computer simulations, where robots can practice thousands of times without breaking anything. However, simulating soft, deformable objects is notoriously slow and difficult because the computer must calculate the physics of every tiny fold and stretch in real-time, a process that often requires breaking time down into tiny, painstakingly small fractions of a second to avoid errors.

A team of researchers has developed a new approach called WorldContact, a specialized computer model designed to predict how soft objects move and deform when a robot interacts with them. Instead of calculating every microscopic step of a physical simulation, this model learns from a small set of high-quality examples to predict the outcome of an action over a much longer period. By focusing specifically on where the robot, the object, and the environment touch one another, the system can generate training data for robots at ten times the speed of traditional simulators. When tested on a real robot, this method allowed the machine to learn a complex task—lifting a shopping bag—much more effectively than if it had relied only on the original, slower simulation data.

The core challenge in teaching robots to manipulate soft objects lies in the sheer complexity of the physics involved. When a robot grabs a bag, the fabric folds, slides against itself, and presses against the table, creating a web of interactions that change every millisecond. Traditional simulators handle this by taking tiny steps through time, checking for collisions and calculating forces at each instant to ensure the bag doesn't pass through the table or the robot's hand. While accurate, this method is computationally expensive, making it slow to generate the massive amounts of practice data robots need to learn new skills. The researchers behind WorldContact recognized that while the physics are complex, the most critical factor in predicting the future state of a soft object is understanding the contact points: where the fabric touches the gripper, where it touches the table, and where different parts of the fabric touch each other.

To solve the speed problem, the team built a model that skips the tiny, incremental steps of traditional simulation. Instead, it learns to predict the entire change in the object's shape over a larger time interval, roughly twenty times longer than the steps used in the source simulator. The model is trained on a limited set of high-quality trajectories, which are paths of movement recorded from either a precise physical simulation or real-world interactions. It analyzes these examples to understand how local geometry and motion around specific points on the object influence the whole. By focusing on these contact zones, the system can forecast how the object will deform and move without needing to resolve every single micro-interaction that a standard physics engine would require.

The researchers tested this approach using a dataset of sixteen different shopping-bag manipulation tasks, ranging from simple collisions to complex grasping and lifting scenarios. They used an automated agent to generate the initial training data, ensuring that every movement was physically valid and free of errors like objects passing through one another. Once the model was trained, it was used to generate a vast amount of additional practice data. The results showed that WorldContact could produce state rollouts—predictions of how the object would move over time—ten times faster than the original simulator, excluding the time needed to render images or save files. This speedup allowed the team to create a much larger dataset for training a robot's control policy, which is the set of rules the robot follows to decide its next move.

The true test of this method came when the researchers applied it to a real-world task: teaching a robot to lift a shopping bag. They started with a pre-trained robot policy and fine-tuned it using data from the simulations. When the robot was trained only on the original, limited set of twenty-five simulation trajectories, it succeeded in lifting the bag on its first try in sixty-five percent of attempts. However, when the same policy was fine-tuned using the expanded dataset generated by WorldContact, the success rate jumped to ninety-five percent. This dramatic improvement demonstrated that the additional data, created efficiently by the new model, provided the robot with a much richer understanding of how to handle the bag without dropping it or failing to grasp it.

The system works by breaking the object down into thousands of points, or vertices, and encoding information about how each point relates to its neighbors and the environment. It pays special attention to four types of contact: spatial contact between different parts of the object, topological contact between points that are connected in the object's structure, controllable contact with the robot's arms and grippers, and environmental contact with surfaces like tables. By combining these local details with a global understanding of the scene, the model can predict the future state of the entire object with high fidelity. In visual comparisons, other methods often failed to maintain the connection between the gripper and the bag, causing the bag to drop, or they produced physically impossible deformations. WorldContact, by contrast, maintained stable contact and realistic movement throughout the lifting process.

This work suggests a new path for scalable robot learning, where the bottleneck is no longer the availability of data but the efficiency of generating it. By using a contact-centric model to amplify a small set of high-quality experiences, researchers can rapidly adapt robot policies to new tasks and objects. While the current success was measured in simulation and on a specific set of bag-lifting tasks, the approach points toward a future where robots can learn complex manipulation skills from limited real-world interactions, using intelligent models to fill in the gaps. The researchers note that challenges remain, particularly in ensuring the model works perfectly when the real world differs from the training data, but the results offer a compelling proof that efficient data generation can significantly boost a robot's ability to act in the physical world.

Technical Summary: WorldContact

Problem Statement

Adapting robots to new objects and tasks, particularly in the domain of deformable-object manipulation, requires extensive interaction experience that is often costly to obtain in the real world. While world models can predict the outcomes of robot-object interactions to supply this experience, constructing them is challenging due to the complex, changing physical behaviors of deformable objects (e.g., fabric). These objects exhibit dynamic geometry, material properties, and physical interactions (friction, sliding, grasping) that are difficult to model reliably across varying configurations. Furthermore, existing numerical simulators, while accurate, require small integration time steps to resolve rapid motion and prevent interpenetration, making them computationally expensive for generating the large-scale datasets needed for robot policy learning.

Methodology

The authors propose WorldContact, a contact-centric neural world model designed to predict object dynamics efficiently from a limited set of high-quality interaction trajectories. The system operates as a learned simulator that bridges the gap between high-fidelity physics simulation and scalable data generation.

1. State Representation

The model represents the scene using two components:

Object State ( $S_t$ ): Describes all objects whose dynamics are to be predicted, including vertex positions ( $X_t$ ), velocities ( $V_t$ ), rest topology ( $E$ ), and attributes ( $C$ ) such as object identity and rest-state coordinates.
Prescribed State ( $B_t$ ): Specifies external inputs, including commanded robot joint rotations and gripper openings ( $a_t$ ), fixed kinematic structures ( $G$ ), and sampled environmental geometry ( $X^E_t, V^E_t, C^E_t$ ).

The model learns the transition function $F_\theta$ that predicts state changes ( $\Delta X_t, \Delta V_t$ ) given the current object state and the future prescribed state:
$(\Delta X_t, \Delta V_t) = F_\theta(S_t, B_{t+\Delta t})$

2. Contact-Centric Vertex Tokenizer

A core innovation is the encoding of local geometry and motion based on four distinct contact types for each vertex:

Spatial Contact: Neighbors in Euclidean space across all objects (e.g., bag strap touching bag body).
Topological Contact: Neighbors in the rest-state topology (UV space) of the same object.
Controllable Contact: Neighbors from robot links, grippers, and rigidly held objects (capturing direct and mediated contact).
Environmental Contact: Neighbors from environmental geometry (e.g., table surfaces).

These contacts are aggregated using learned compact-support kernels to form vertex tokens that combine local state with contact information.

3. Hierarchical Architecture

To handle long-range coupling in deformable objects (e.g., stretching a cloth affects distant vertices) without the computational cost of full self-attention over all vertices, WorldContact employs a hierarchical approach:

Hierarchical Token Encoder: Uses a token-merging strategy (based on cosine similarity) to reduce the number of tokens involved in global self-attention. This creates a compact set of global tokens that summarize local geometric and contact information.
Global Token Decoder: Global tokens capture long-range dependencies via self-attention. These are then used in a cross-attention mechanism to refine the original vertex tokens, allowing each vertex to retrieve relevant global context while retaining local details.
Output: A pointwise MLP predicts the state corrections ( $\Delta x, \Delta v$ ).

4. Training and Stability

The model is trained on high-quality trajectories (from simulation or real-world) using an autoregressive rollout objective that minimizes position and velocity errors. To improve long-horizon stability, the authors introduce a differentiable topology projection ( $P$ ) within the prediction loop. This projection smooths the predicted displacement field and limits topology-edge distortion, ensuring that the generated trajectories remain physically plausible during autoregressive generation.

5. Data Generation Pipeline

WorldContact operates with a time step 20 $\times$ larger than the source numerical simulator (0.04s vs. 0.002s). It predicts state changes over these larger intervals directly, bypassing the need for small integration steps. The predicted state trajectories are then rendered into observation modalities (RGB views) and paired with the conditioning robot actions to create a dataset for fine-tuning Vision-Language-Action (VLA) policies.

Key Contributions

WorldContact Model: A contact-centric world model constructed from limited high-quality data that encodes local geometry and multi-type contact (spatial, topological, controllable, environmental) to predict object dynamics under robot actions.
Scalable Data Pipeline: A pipeline that renders predicted trajectories and pairs them with conditioning actions, enabling the scaling of demonstrations for policy fine-tuning.
Empirical Validation: A comprehensive study across 16 shopping-bag manipulation tasks demonstrating a 10 $\times$ speedup in state-rollout generation compared to the source simulator and improved real-robot performance.

Results

The authors evaluated WorldContact on 16 shopping-bag manipulation tasks (including lifting, supporting, and collision scenarios).

Computational Efficiency: On a single NVIDIA H100 GPU, WorldContact generates state rollouts 10 $\times$ faster than the source numerical simulator (excluding rendering and disk I/O).
Dynamics Fidelity: Compared to baselines (RoboCraft, RoboCook, AdaptiGraph), WorldContact achieved significantly lower Mean Squared Error (MSE) for position (0.0002 vs. ~0.03–0.06) and velocity, and a drastically lower penetration rate (0.35% vs. 10–25%).
Real-Robot Performance: In a bag-lifting task, a VLA policy fine-tuned on only 25 source simulation trajectories achieved a 65% success rate (13/20 trials). When the same policy was fine-tuned on an expanded dataset of 150 trajectories generated by WorldContact, the success rate increased to 95% (19/20 trials).

Significance and Claims

The paper claims that WorldContact provides a practical solution for efficient data generation to adapt robot control policies to new manipulation settings. By leveraging a contact-centric formulation and a larger time step, the model generates high-quality training experience from limited source data, effectively scaling the dataset without the prohibitive cost of running full physics simulations.

The authors emphasize that their approach separates dynamics validation from observation generation, maintaining explicit vertex states for physical checks (like penetration) while rendering observations for policy training. They conclude that this method supports the adaptation of robot policies to new objects and tasks, though they acknowledge that prediction errors under distribution shift and differences between rendered and real observations remain challenges for future work.