💻 computer science

Enfold: Folding World-Generator Computation into Predictive Representations for Efficient Embodied Control

Enfold is a novel framework that distills the multi-level computational process of world generative models into a predictive representation inferred solely from the current visual and linguistic context, enabling embodied agents to achieve strong control with significantly reduced latency by internalizing future-generative structure without needing to materialize future trajectories at every step.

Original authors: Weili Zeng, Yitong Xing, Fulong Liu, Chengqun Yang, Antao Xiang, Feng Tian, Jingnan Gao, Jisong Cai, Xin Wang, Xiaomin Wu, Yao Mu, Yichao Yan

Published 2026-07-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Weili Zeng, Yitong Xing, Fulong Liu, Chengqun Yang, Antao Xiang, Feng Tian, Jingnan Gao, Jisong Cai, Xin Wang, Xiaomin Wu, Yao Mu, Yichao Yan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to make a sandwich. The old way of doing this was to give the robot a crystal ball. Before it moved a single hand, the robot would use its crystal ball to generate a full, high-definition video of the future: seeing the bread get sliced, the cheese slide onto the loaf, and the plate being set down. Only after watching this entire movie in its mind would the robot decide what to do next. It's like trying to drive a car by watching a full movie of the road ahead before you even turn the steering wheel. It's powerful, but it's also incredibly slow and computationally expensive, like trying to render a blockbuster movie just to decide whether to turn left or right.

This paper comes from the field of robotics and artificial intelligence, specifically focusing on "world models." A world model is essentially an AI that learns how the physical world works by predicting what happens next. The key idea the authors are playing with is "predictive representations." Think of this as the difference between memorizing a whole movie script versus understanding the rules of the story. You don't need to re-watch the whole movie to know that if you drop a glass, it will shatter; you just need to understand the physics of gravity and fragility. The big question the authors ask is: Can we teach a robot to understand the structure of the future without actually forcing it to generate the full, heavy video every single time it needs to move?

The paper introduces a new method called Enfold. The name is a play on "unfolding" a future (which is what the old, slow methods do) and "enfolding" that future knowledge into the present moment. Instead of making the robot generate a full video of the future before acting, Enfold teaches a "predictive encoder" to absorb the computation of how that future would look.

Here is how it works: The researchers use a powerful "teacher" AI (a video generator) that is really good at imagining the future. As this teacher AI processes a video of what will happen, it goes through many internal steps, organizing the scene, the objects, and the interactions. Enfold watches these internal steps and learns to predict them using only the current picture and the robot's instruction. It's like a student watching a master chef cook a complex dish. Instead of the student trying to cook the whole dish from scratch every time they want to make a sandwich, they learn the chef's internal "muscle memory" and "kitchen logic." They learn to anticipate the next step based on the current state of the pan, without needing to simulate the entire meal in their head first.

The paper finds that this approach is a game-changer for speed and efficiency. In tests, the Enfold robot was able to make decisions 3.7 times faster than a previous state-of-the-art method called Fast-WAM, and an optimized version called Enfold-Flash was 10.1 times faster. Despite being much faster, it didn't lose its smarts; it actually performed slightly better on complex tasks, achieving a 97.8% success rate on one benchmark and 91.77% on another.

Crucially, the paper argues against the idea that a robot must generate a full video to act intelligently. They show that by "enfolding" the future-generative computation into a compact representation, the robot can still adapt if the world changes. For example, if a human moves a plate while the robot is planning to grab it, Enfold doesn't just replay a pre-recorded script; it instantly recalculates the future based on the new reality and adjusts its actions. The authors suggest that this proves the robot has learned a flexible, predictive understanding of the world, rather than just memorizing a fixed path.

In short, Enfold suggests that we don't need to render the entire future to control a robot. We just need to teach the robot to carry the logic of the future in its pocket, allowing it to act instantly and adaptively, turning a slow, video-rendering process into a lightning-fast, intuitive decision.

Technical Summary: Enfold – Folding World-Generator Computation into Predictive Representations for Efficient Embodied Control

1. Problem Statement

World generative models (WGMs) have emerged as powerful substrates for embodied control, capable of synthesizing future observations to anticipate scene evolution, object interactions, and long-range motion. However, current deployment strategies for these models face a fundamental efficiency bottleneck. Dominant approaches typically rely on output-level interfaces:

Imagine-then-act: Materializing a full visual future before recovering an action.
Unified World-Action Models: Jointly decoding video and actions, often requiring the generator to run at inference time.
Latent Context: Using internal features or caches from a generative branch, which still often necessitates executing the generative process or relies on frozen encoders that do not expose the generative computation.

The core problem addressed by this work is whether the internal computation that constructs a future—specifically the intermediate states of a world generator that organize appearance, spatial layout, and interaction across abstraction levels—can be internalized into a representation inferred from the present alone. If successful, this would allow a robot to access future-oriented structural knowledge without executing the costly generative branch during the control loop, thereby reducing latency while maintaining predictive capability.

2. Methodology: The Enfold Framework

Enfold proposes a framework that transfers the future-generative computation of a world generator into a current-only predictive representation. The architecture consists of three main components: a predictive encoder ( $U_\phi$ ), a generative world model ( $G_\theta$ ), and lightweight task heads.

2.1 Core Mechanism: Bidirectional Coupling

The framework is trained via two coupled objectives that separate the roles of the generator and the encoder:

Generator-to-Representation (G2R):
- Process: During training, the generator ( $G_\theta$ ) processes an observed future video chunk ( $y$ ) under a teacher-forced pass, exposing its intermediate hidden states ( $H_\theta$ ) across various network depths and corruption timesteps.
- Supervision: A predictive encoder ( $U_\phi$ ), which sees only the current visual context ( $c$ ) and language instruction ( $e$ ), is trained to predict these multi-level generator states.
- Target Construction: The target is not a single layer but a concatenation of selected states from multiple depths ( $A(H_\theta)$ ), processed by a timestep-conditioned head ( $F_\omega$ ). This accounts for the fact that the most informative layer shifts with the corruption level and depth.
- Constraint: The generator states are detached from the task gradients. The encoder learns to predict the structure of the future transition without being shaped by the specific action outcomes.
Representation-to-Generation (R2G):
- Process: The learned representation ( $\bar{z}$ , a stop-gradient copy of the encoder output) is fed back to condition the future generation in the generator ( $G_\theta$ ).
- Objective: The generator is trained to predict the velocity field of the future conditioned on $c$ , $e$ , and $\bar{z}$ .
- Purpose: This acts as a functional test, ensuring the representation $z$ contains sufficient predictive information to improve future modeling, rather than just being a policy-specific summary.
Detached Task Readouts:
- Task heads (e.g., action prediction) read the representation $z$ (via a stop-gradient) to predict actions.
- Crucial Design: Task gradients do not flow back into the encoder $U_\phi$ . This ensures the representation retains the transition structure anticipated from the current state rather than overfitting to nuisance variations in the teacher's specific trajectory.

2.2 Inference and Deployment

At deployment, the system executes only the predictive encoder and the action head. The video generator is not executed for action prediction.

Action Prediction: The action head generates action chunks directly from $z$ .
Future Imagination: If a visual future is explicitly requested (e.g., for visualization), the generator $G_\theta$ is used to sample from $p_\theta(y | c, e, z)$ .
Latency Reduction: By removing the generator from the control path, Enfold achieves significant latency reductions compared to methods that require video rollouts or coupled decoding.

3. Key Contributions

The paper claims the following contributions:

Formulation of Predictive Supervision Space: The authors formulate the multi-level internal computation of a world generator as a predictive supervision space, characterizing how its utility and stability vary across network depth and corruption levels.
The Enfold Framework: Development of a method that distills future-generative computation into a current-only representation, serving as a shared interface for both future generation and stop-gradient task readouts.
Efficient Control without Rollout: Demonstration that this interface supports competitive control in simulated and real-world tasks without executing the generator in the action path, while simultaneously improving future video prediction and substantially reducing action latency.

4. Experimental Results

The framework was evaluated on LIBERO (40 language-conditioned tasks), RoboTwin2.0 (50 dual-arm tasks with domain randomization), and real-world bimanual manipulation (AgileX platform).

4.1 Control Performance and Latency

LIBERO: Enfold achieved 97.8% average success without embodied pre-training, outperforming Fast–WAM (97.6%) and matching or exceeding larger models like $\pi0$ $π 0$ and OpenVLA.
- Latency: Enfold reduced action latency to 134 ms (3.7 $\times$ faster than Fast–WAM). The optimized Enfold-Flash variant reached 49 ms (10.1 $\times$ faster).
RoboTwin2.0: Enfold achieved 91.77% average success (clean and randomized), comparable to Fast–WAM (91.83%) and slightly below LingBot-VA (92.20%), despite using no embodied pre-training.
Real-World: On four bimanual tasks, Enfold achieved an 89.7% in-distribution completion score, significantly outperforming Fast–WAM (77.8%) and $\pi0.5$ (86.1%). Under out-of-distribution (OOD) shifts, Enfold maintained a 76.6% score, outperforming Fast–WAM by 6.6 points.

4.2 Adaptation and Counterfactual Consistency

Intervention Tests: When human intervention altered the scene (e.g., moving an object) while the goal remained fixed, Enfold's executed actions and imagined continuations adapted to the new state.
Significance: This behavior contrasts with fixed trajectory replay, providing qualitative evidence that the representation encodes counterfactual consistency and can re-plan based on the altered present.

4.3 Ablation and Representation Analysis

Supervision Targets: Multi-level generator state supervision outperformed both future-pixel prediction and action-only targets.
R2G Validation: Conditioning the generator on the learned representation improved future video prediction metrics (PSNR, SSIM, LPIPS), confirming the representation's utility beyond just control.
Nuisance Suppression: Analysis showed Enfold suppresses nuisance variations (e.g., illumination changes, generation noise) present in raw generator features while preserving task-relevant structure.
Future-Oriented Encoding: Probes revealed that the representation becomes increasingly predictive of scene regions that change over longer horizons (e.g., moving robot arms and manipulated objects) compared to static visual features.
Token Geometry: Visualization of token similarity showed that Enfold organizes the representation to highlight interaction relations (gripper-object contact) rather than just visual similarity.

5. Significance and Claims

The paper posits that Enfold recasts the role of a world generator from a runtime simulator to a training-time source of structured predictive supervision.

Internalization of Computation: The primary claim is that the "future need not be materialized at every step if its internal structure can be enfolded into the present."
Asymmetry: The framework leverages an asymmetry where the encoder predicts stochastic, future-dependent states without access to the future. This forces the encoder to retain transition structures that are anticipatable from the current state, rather than reproducing every nuisance variation of the teacher.
Efficiency vs. Capability: The results demonstrate that removing the generative branch from the control loop does not sacrifice control quality; instead, it enables real-time performance (sub-50ms latency) while retaining the ability to "imagine" futures when explicitly requested.
Generalization: The method shows robustness to distribution shifts and human intervention, suggesting the learned representation captures a more causal and adaptable understanding of the environment than simple trajectory replay mechanisms.

The authors remain modest regarding the scope, noting that the intervention evidence is qualitative and that robustness under large distribution shifts (compared to models with heavy pre-training) remains an open challenge. However, they establish a broader principle: rich generative computation can be internalized during learning and reused efficiently for control.