💻 computer science

Zeva: In-Context Causal Learning for Generalizable Embodied Manipulation

Zeva is a novel framework that enables generalizable embodied manipulation by extracting causal interactions from a robot's physical experiences into a dual-timescale memory, allowing a frozen policy model to self-evolve and improve its performance across tasks through in-context learning without requiring gradient updates.

Original authors: Fu Chen, Xin Ding, Bingjia Huang, Xiangyu Li, Mingju Wang, Jiawei He, Kun Li, Wei Sun, Yunxin Liu, Hao Wu, Ting Cao

Published 2026-09-01

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Fu Chen, Xin Ding, Bingjia Huang, Xiangyu Li, Mingju Wang, Jiawei He, Kun Li, Wei Sun, Yunxin Liu, Hao Wu, Ting Cao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Robots have long been masters of the factory floor, where they repeat the same motion thousands of times with perfect precision. But step them out of that controlled environment and into a real kitchen or a laboratory, and they often stumble. The physical world is messy; objects shift, surfaces are uneven, and the way a gripper feels a glass of water is different from how it feels a metal cup. For years, the leading approach to teaching robots has been to feed them massive amounts of video and data before they ever leave the lab, hoping they learn enough general rules to handle anything. Yet, when these robots face a situation they have never seen before, they frequently fail because their internal "brain" cannot adapt to the new physics of the moment. They are stuck with the knowledge they were given, unable to learn from their own mistakes as they happen.

A team of researchers at Tsinghua University and Z-Trans AI has proposed a different path, one where a robot learns to think about cause and effect in real time. They introduced a system called Zeva, which allows a robot to improve its own performance without ever changing its underlying software code. Instead of trying to retrain the robot's brain, which is slow and risky, Zeva acts like a highly organized notebook. As the robot tries to perform a task, it records exactly what happened when it moved: it saw a specific object, it moved its arm in a certain way, and the object either stayed put or tipped over. The system extracts the lesson from that single interaction—the causal link between the action and the result—and stores it. When the robot tries the same task again, it looks back at this notebook, finds the relevant lesson, and uses it to adjust its next move. This happens instantly, allowing the robot to get better with every single attempt, even if its core programming remains frozen and unchanged.

The researchers tested this idea in two very different worlds. First, they used a sophisticated computer simulation of a kitchen, a place filled with complex objects like kettles, toasters, and mixers. In this virtual environment, Zeva faced five distinct tasks, such as turning on a microwave or opening a mixer head. The results were striking. While other advanced robot systems managed to succeed about 60 to 72 percent of the time, Zeva reached a success rate of 76.8 percent. More importantly, the system showed a clear pattern of self-improvement. On its very first try at a task, the robot succeeded only about a quarter of the time. But as it was allowed to keep trying the same scenario, using the lessons from its failures to guide its next moves, its success rate climbed steadily. By the fourth attempt, it was succeeding in 73 percent of cases. This proved that the robot was not just getting lucky; it was genuinely learning from the physical feedback of its own actions.

To ensure this was not just a result of the computer simulation, the team took the system into a real-world chemical laboratory. They equipped a physical robot arm with the ability to handle delicate glassware, such as test tubes and beakers, and perform tasks like pouring water or weighing chemicals. This environment was far more unpredictable than the simulation, with real gravity, friction, and the risk of breaking glass. Here, Zeva again outperformed the best existing systems. In the simplest tasks, like picking up a test tube, it succeeded 100 percent of the time. In more complex sequences, like preparing a salt solution, it achieved a success rate of 70 percent, significantly higher than its competitors. The researchers also measured how much of the process the robot completed, not just whether it finished the whole job. Even when a task was difficult, Zeva managed to complete more of the required steps than any other system, showing that its ability to learn from interaction helped it navigate the messy reality of the physical world.

A key part of Zeva's success is how it organizes its memory. The system does not just store a long list of everything it has ever done; that would be too much to process quickly. Instead, it uses two types of memory working together. One is a short-term trace that remembers the last few seconds of action, helping the robot understand what is happening right now. The other is a persistent memory that keeps the most useful lessons from previous attempts, even if those attempts failed. When the robot starts a new try, it searches this persistent memory for a situation that looks similar to the one it is facing right now. It then uses that past experience as a guide. For example, if the robot previously tried to pour water and the glass tipped because it tilted too fast, it remembers that specific cause-and-effect relationship. On the next try, it consults this memory and slows down its tilt, avoiding the same mistake. This process happens without any changes to the robot's main software, meaning the robot can learn and adapt instantly without the slow and dangerous process of retraining its entire brain.

The researchers also discovered that this system could learn from humans, not just from its own mistakes. In one experiment, a human physically guided the robot arm through a single successful attempt of a complex task. The system recorded this human demonstration, extracted the causal lessons, and stored them in its memory. When the robot then tried the task on its own, it used that single human example to jump-start its learning. This "warm-up" allowed the robot to perform much better right from the start, improving its success rate by up to 15 percent compared to when it had to learn entirely from scratch. This suggests that a robot could learn a new skill from a single human demonstration and then refine that skill through its own repeated practice, combining the best of human guidance with the power of self-evolution.

The study also looked at whether the lessons a robot learned in one task could help it in a completely different one. They found that the system could recognize when a physical effect in a new task was similar to one it had seen before. For instance, the motion of tilting a container to pour water was recognized as similar to the motion of tilting a container to mix chemicals, even though the objects and the goals were different. This ability to transfer knowledge across tasks means that a robot does not need to start from zero every time it faces a new challenge. It can draw on a library of physical cause-and-effect relationships it has built up over time, making it more versatile and capable in a wide range of situations.

Despite these successes, the researchers are clear about the limits of their work. The system currently learns only from the attempts it makes while trying to finish a specific job. It does not yet have the ability to wander around and explore the world just to learn new things, a behavior known as active exploration. The authors suggest that future work will focus on teaching the robot to choose its own experiments, deliberately trying out different actions to reduce uncertainty about how the physical world works. Until then, Zeva represents a significant step forward, showing that a robot does not need to be retrained to get smarter. By simply paying attention to the consequences of its own actions and remembering what worked and what did not, a robot can evolve its own capabilities, turning every failure into a lesson for the next attempt.

Technical Summary: Zeva – In-Context Causal Learning for Generalizable Embodied Manipulation

1. Problem Statement

Generalizable embodied manipulation remains a significant challenge for Embodied AI, particularly when deploying pre-trained models (such as Vision-Language-Action (VLA) and World-Action Models (WAMs)) in real-world physical environments. While foundation models have made rapid progress, their deployment is often brittle due to unseen physical conditions absent during pretraining, including novel object geometries, contact dynamics, visual variations, and embodiment-specific execution errors.

Existing adaptation methods face limitations:

Test-time training (e.g., RoboTTT, WAM-TTT): These methods update model weights or fast weights via gradient descent during deployment. This leaves the action-state change structure implicit and often suffers from high latency, preventing adaptation between consecutive attempts.
Standard In-Context Learning (e.g., GEN-1.5, Skild S1): These keep weights fixed but rely on task demonstrations for generalization. They primarily target task generalization rather than on-the-fly learning of physical interactions from the robot's own experience.

The core problem addressed is how to enable a robot to learn the causal relationships between its actions and resulting state changes during real-world deployment, allowing it to adapt and self-evolve without updating the underlying policy model parameters.

2. Methodology: Zeva Framework

The authors propose Zeva, a framework that enables In-Context Causal Learning (ICCL). Zeva keeps the policy model frozen during deployment but extracts causal knowledge from physical interactions to guide subsequent actions. The framework operates through three main stages:

A. Causal Interaction Extraction

Zeva utilizes a Causal Transition Encoder (CTE) to bridge raw pixels and causal reasoning.

Input: At each timestep, the system extracts visual latents ( $s_\tau$ ), action encodings ( $u_\tau$ ), and observed effects ( $d_\tau$ ).
Processing: These signals are integrated into a Causal Interaction State ( $B_\tau$ ) via a gated recurrence (GRU).
Output: The state is projected into two components:
1. A Phase Token ( $p_\tau$ ) representing task progress.
2. A Causal Interaction Signal ( $e_\tau$ ) characterizing the physical dynamics.
Optimization: The CTE is trained with a multi-objective loss including Causal Effect Prediction (grounding signals in physical effects), Task Identity Clustering (ensuring task consistency), and Phase Progression (monotonic task execution).

B. Dual-Timescale Causal Memory

To support adaptation both within and across attempts, Zeva organizes interaction units into two memory streams:

Brief Interaction Trace (BIT): A sliding window of recent interaction signals ( $\{e_j\}$ ) within a single attempt. This provides immediate local context regarding the efficacy of current actions. It is reset at the start of every new attempt.
Persistent Interaction Memory (PIM): Accumulates causal units across multiple attempts within the same episode. It employs similarity-based merging to consolidate useful evidence. A new experience is merged with an existing record only if it matches both the task progress (phase) and physical dynamics (signal) of the existing record. This memory is cleared only at the start of a new episode.

C. In-Context Policy Injection

The final stage transforms stored experiences into a structured Causal Prompt for the frozen foundation policy.

Retrieval: The system queries the PIM using the current phase token ( $p_\tau$ ) to retrieve the most relevant interaction evidence ( $R_\tau$ ) from previous attempts.
Construction: A Memory Context Encoder ( $F_{mem}$ ) fuses the task instruction ( $g$ ), current phase token, the BIT, and the retrieved PIM signals into a single context vector ( $M_\tau$ ).
Inference: This context is injected into the self-attention layers of the frozen foundation policy (specifically the diffusion path in the implementation). The policy performs implicit system identification, inferring latent environmental properties (e.g., object mass, joint constraints) to adapt its action strategy without gradient updates.

3. Key Contributions

Formulation of ICCL: The paper formulates embodied action generalization in real-world deployment as In-Context Causal Learning, where a frozen policy learns from its own physical interaction experiences without test-time weight updates.
The Zeva Framework: A novel architecture combining Causal Interaction Extraction, Dual-timescale Causal Memory, and In-Context Policy Injection to enable gradient-free self-evolution across repeated attempts.
Empirical Validation: Comprehensive evaluation in both simulation (RoboCasa365-Atomic5) and real-world (ChemLab-Evo) settings, demonstrating that the framework achieves state-of-the-art performance and exhibits self-evolution capabilities.

4. Experimental Results

The authors evaluated Zeva on the RoboCasa365-Atomic5 simulation benchmark and the real-world ChemLab-Evo benchmark using an ARX manipulator.

Benchmark Performance:
- RoboCasa365-Atomic5: Zeva achieved the highest average success rate of 76.8%, outperforming frontier models like Fast-WAM (72.4%) and Cosmos3-Nano (64.8%).
- ChemLab-Evo: Zeva achieved the best success rate across all three difficulty levels (Atomic, Short-sequence, Complex). For example, on the "Pick Up Test Tube" task, it achieved 100% success, compared to 80-95% for baselines. It also outperformed baselines in "Process Score" (ordered stage completion) for long-horizon tasks.
Post-Deployment Scaling (Self-Evolution):
- Zeva demonstrated that success rates improve as the robot accumulates interaction experience.
- On RoboCasa365-Atomic5, the cumulative success rate increased from 26% on the first attempt to 73% within four repeated attempts.
- On ChemLab-Evo, similar scaling trends were observed, with success rates rising from 65% to 100% for "Pick Up Test Tube" across evolution milestones.
Human-Guided Experience Injection:
- Zeva can incorporate a single human teleoperation demonstration (one-shot warm-up) to initialize the PIM. This improved performance by up to 20% on the "Place Beaker" task and 15% on "Pour Water" over self-evolution alone, allowing the robot to reproduce demonstrated approaches (e.g., grasp and placement trajectories) without gradient updates.
Cross-Task Generalization:
- Ablation studies confirmed the necessity of both BIT and PIM; removing PIM reduced success rates by 10–20%, while removing BIT caused drops of 15–30%.
- Experiments showed that causal signals can be retrieved across tasks. Replacing task-local signals with nearest cross-task matches preserved most performance (e.g., 95% success on "Pick Up Test Tube" vs. 100% with local signals), whereas random replacement caused significant drops.

5. Significance and Claims

The paper claims that Zeva represents the first framework to enable in-context learning from a robot's own physical interaction experiences while maintaining a frozen policy model.

Self-Evolution: Unlike gradient-based methods that are slow or prone to capability collapse, Zeva enables continuous improvement through repeated attempts by updating only the interaction memory.
Generalization: By explicitly modeling the causality between actions and state changes, Zeva allows robots to adapt to diverse physical conditions and transfer knowledge across tasks without retraining the foundation model.
Efficiency: The approach avoids the computational cost of test-time training, making it suitable for real-time deployment where rapid adaptation is required.

The authors acknowledge a current limitation: Zeva learns causality only from attempts generated for task execution rather than actively collecting interactions for learning. Future work is proposed to investigate causality-driven exploration to reduce uncertainty more efficiently.