💻 computer science

CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation

The paper introduces CaP-X, a comprehensive framework comprising the CaP-Gym environment and CaP-Bench benchmark, which demonstrates that while Code-as-Policy agents initially rely heavily on human-crafted abstractions, their robustness and ability to achieve human-level performance on robot manipulation tasks can be significantly enhanced through test-time scaling strategies and reinforcement learning with verifiable rewards.

Original authors: Max Fu, Justin Yu, Karim El-Refai, Ethan Kou, Haoru Xue, Huang Huang, Wenli Xiao, Guanzhi Wang, Fei-Fei Li, Guanya Shi, Jiajun Wu, Shankar Sastry, Yuke Zhu, Ken Goldberg, Linxi "Jim" Fan

Published 2026-03-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Max Fu, Justin Yu, Karim El-Refai, Ethan Kou, Haoru Xue, Huang Huang, Wenli Xiao, Guanzhi Wang, Fei-Fei Li, Guanya Shi, Jiajun Wu, Shankar Sastry, Yuke Zhu, Ken Goldberg, Linxi "Jim" Fan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you want to teach a robot to make a sandwich. You have two main ways to do it:

The "Black Box" Way (Current Standard): You feed the robot millions of videos of people making sandwiches. The robot learns by pattern matching, like a parrot repeating a song. It works great if the bread is exactly where it expects it, but if you move the bread or ask it to make a different kind of sandwich, it gets confused and freezes. It's like a student who memorized the answers to a specific test but can't solve a new problem.
The "Programmer" Way (This Paper's Idea): Instead of showing videos, you give the robot a brain that knows how to write code. You tell it, "Here is a toolbox of basic moves (grab, lift, move), and here is the goal. Write the instructions yourself." This is the Code-as-Policy approach.

CaP-X is a new "gym" and "exam" created by researchers to test how good these robot-programmer brains really are.

Here is the breakdown of their findings, using some fun analogies:

1. The Problem: The "Training Wheels" Trap

The researchers found that when they gave the robots "training wheels" (high-level, pre-written instructions like stack_objects()), the robots did a great job. But the moment they took the training wheels off and asked the robots to use only the raw, basic tools (like move_arm_to_coordinate(x,y,z)), the robots crashed and burned.

The Analogy: It's like a student who can solve a math problem if the teacher gives them a specific formula to plug numbers into. But if you ask them to derive the formula from scratch, they panic. The robots were relying on the "human scaffolding" rather than actually understanding the task.

2. The Solution: The "Swiss Army Knife" Agent

The paper introduces CaP-Agent0, a framework that teaches these robots to be self-reliant. Instead of just writing one script and hoping for the best, this agent uses a few clever tricks:

The "Visual Translator" (VDM): Robots are bad at looking at a picture and understanding what changed. So, CaP-Agent0 uses a "translator" that looks at the camera feed and writes a simple text report: "The red block is now on the table, but the blue block is still on the floor." This turns a confusing image into clear text the robot can reason with.
The "Skill Library" (Auto-Synthesis): When the robot successfully does something hard (like stacking a block), it saves that specific sequence of code as a new "tool" in its toolbox. Next time, it doesn't have to reinvent the wheel; it just grabs the tool. It's like a carpenter who, after building a perfect chair, adds a "Make Chair" button to their toolbox for next time.
The "Council of Experts" (Ensemble Reasoning): Instead of asking one AI to write the code, CaP-Agent0 asks three different AIs to write three different solutions. Then, a "judge" AI looks at all three, picks the best parts of each, and combines them into one perfect plan. It's like a committee meeting where everyone debates the best way to build a bridge before construction starts.

3. The Results: From "Novice" to "Grandmaster"

By using these tricks, the researchers found that their agent could perform almost as well as a human expert, even without any specific training data for the task.

The "Real World" Test: They tested this on real robots (not just simulations). When asked to find a hidden object under a cup (a "needle in a haystack" task) or solve a math problem using physical blocks, the robot didn't just guess. It looked, thought, wrote code to move its arm, checked if it worked, and if it failed, it wrote new code to try a different angle.
The "Reinforcement Learning" Boost: They also showed that if you let the robot practice in a simulation and give it a "gold star" (reward) every time it succeeds, it gets even better. The best part? The skills it learned in the video game (simulation) transferred perfectly to the real robot without needing to relearn anything.

The Big Picture

CaP-X proves that the future of robotics isn't just about feeding robots more data to memorize. It's about giving them the ability to think, write instructions, and debug their own mistakes.

Think of it this way:

Old Way: Teaching a robot to dance by playing a video of a dancer over and over.
CaP-X Way: Teaching a robot the steps of the dance, then handing it a pen and paper and saying, "You figure out the choreography for this new song."

The paper shows that with the right tools (visual translation, skill libraries, and team reasoning), robots can indeed figure out the choreography themselves, making them much more adaptable to the messy, unpredictable real world.

1. Problem Statement

The paper addresses the limitations of current paradigms for controlling embodied robots:

Classical Control: Relies on explicit, human-engineered code. While robust and interpretable, it suffers from a scalability bottleneck, requiring manual design of abstractions and handling of edge cases for every new task.
Vision-Language-Action (VLA) Models: These data-driven models learn directly from large-scale datasets. While impressive on contact-rich tasks, they often lack interpretability, struggle to generalize to new robot embodiments or long-horizon tasks without retraining, and inherit biases from their training data.
Code-as-Policy (CaP) Gap: Recent work suggests using Large Language Models (LLMs) to generate executable code for robot control. However, prior evaluations often rely on high-level, human-crafted primitives (e.g., stack_objs_in_order()). It remains unclear how much performance stems from the agent's reasoning versus the scaffolding provided by these simplified APIs. Furthermore, there is a lack of systematic benchmarks to evaluate how agents perform when forced to operate over low-level primitives (closer to how humans actually program robots) and how they recover from errors without human intervention.

2. Methodology: The CaP-X Framework

The authors introduce CaP-X, a unified framework comprising four main components: CaP-Gym, CaP-Bench, CaP-Agent0, and CaP-RL.

A. CaP-Gym (The Environment)

An interactive, hierarchical control framework built on the Gymnasium interface.

Architecture: It binds a low-level physics simulator (or real robot) with a stateful code executor loop.
Primitives: It exposes a "Read-Eval-Print Loop" (REPL) where agents generate code to compose perception and control primitives.
Scope: Integrates 187 tasks from standard simulators (RoboSuite, LIBERO-PRO, BEHAVIOR) covering tabletop, bimanual, and mobile manipulation.
Key Feature: It allows agents to control robots by synthesizing programs that call low-level APIs (e.g., inverse kinematics solvers, segmentation models) rather than just high-level macros.

B. CaP-Bench (The Benchmark)

A systematic benchmark designed to evaluate frontier models across three axes:

Abstraction Level:
- High-Level (S1/S2): Uses human-designed macros.
- Low-Level (S3/S4): Uses atomic primitives (e.g., solve_ik(), sam3_text_prompt()). S3 includes in-context usage examples; S4 removes them, forcing the agent to reason solely from function signatures.
Temporal Interaction:
- Single-Turn: Zero-shot generation.
- Multi-Turn: Agents can iterate, observe execution feedback (stdout/stderr), and refine code.
Perceptual Grounding:
- Evaluates how visual feedback impacts code generation, comparing raw RGB interleaving against Visual Differencing Modules (VDM) that convert visual changes into structured text.

C. CaP-Agent0 (The Training-Free Framework)

A novel agentic framework designed to bridge the performance gap between raw models and human experts without task-specific training data. It integrates:

Multi-turn Visual Differencing (VDM): Converts visual observations into structured natural language to mitigate cross-modal alignment gaps found in raw image interleaving.
Auto-Synthesized Skill Library: An iterative process where the agent analyzes successful execution traces to extract and promote recurring low-level utility functions into a persistent, task-agnostic skill library.
Parallel Reasoning: Uses an ensemble of models (e.g., Gemini-3-Pro, GPT-5.2, Claude Opus 4.5) with varied sampling temperatures to generate multiple candidate solutions, which are then synthesized into a final, robust code snippet.

D. CaP-RL (Reinforcement Learning)

Demonstrates that Reinforcement Learning with Verifiable Rewards (RLVR) can be applied directly to the coding agent.

Method: Uses Group Relative Policy Optimization (GRPO) to post-train a base model (Qwen2.5-Coder-7B) using environment rewards from simulation.
Transfer: Because the agent reasons over abstract APIs rather than raw pixels, the learned policy transfers zero-shot to real-world robots with minimal sim-to-real gap.

3. Key Findings and Results

Benchmark Insights (CaP-Bench)

The Abstraction Gap: Performance drops significantly as abstraction levels decrease (from High-Level to Low-Level). Models rely heavily on human scaffolding.
The Reasoning Gap: Even with high-level primitives, frontier models (LLMs/VLMs) lag behind human experts in zero-shot settings.
The Power of Iteration: Multi-turn interaction significantly improves success rates.
- Text-only feedback (M1) helps debugging.
- Raw RGB interleaving (M2) surprisingly degrades performance due to cross-modal alignment issues.
- Visual Differencing (M3), which translates visual changes to text, yields the best results, outperforming both raw images and text-only feedback.
Scaling Test-Time Compute: Agents operating on low-level primitives with multi-turn feedback and visual grounding (M4) can surpass single-turn high-level baselines, approaching human-level reliability.

Performance of CaP-Agent0

Simulation: On 4 out of 7 CaP-Bench tasks, CaP-Agent0 achieves success rates comparable to or exceeding human expert baselines, despite operating solely on low-level primitives.
Generalization (LIBERO-PRO & BEHAVIOR): CaP-Agent0 outperforms state-of-the-art training-based VLA models (OpenVLA, $\pi^0$ , $\pi^{0.5}$ ) on tasks involving instruction perturbations and initial position shifts, demonstrating superior robustness to distribution shifts.
Real-World Deployment: Successfully deployed on Franka Panda and AgiBot G1 robots for zero-shot tasks like "needle in a haystack" (finding a specific object in clutter), mechanical search (lifting cups), and multimodal symbolic reasoning (solving math problems with blocks).

CaP-RL Results

Post-training with GRPO on simulation tasks significantly improves code compilation and strategic robustness.
The RL-trained agent transfers effectively to real robots (Franka Emika), achieving high success rates (e.g., 84% for cube lifting) without needing to retrain on real-world data.

4. Significance and Contributions

Unified Framework (CaP-X): Provides the first principled, open-access platform to systematically evaluate and improve coding agents for robotics, moving beyond high-level abstractions to test true reasoning capabilities.
New Benchmark (CaP-Bench): Establishes a rigorous standard for measuring agentic capability across abstraction levels, interaction modes, and perceptual grounding, revealing that current models are heavily dependent on human scaffolding.
Training-Free High Performance (CaP-Agent0): Proves that with the right agentic scaffolding (visual grounding, skill synthesis, parallel reasoning), LLMs can achieve human-level reliability on complex manipulation tasks without fine-tuning on task-specific data.
Sim-to-Real via RL (CaP-RL): Demonstrates that reinforcement learning on the code generator itself, using verifiable simulation rewards, leads to robust policies that transfer directly to physical robots, bypassing the visual sim-to-real gap.
Paradigm Shift: The work suggests a future where generalist embodied agents are evaluated on their ability to reason over low-level primitives, with robustness synthesized at test-time through iterative reasoning and self-correction, rather than relying on brittle, pre-defined high-level APIs.

Conclusion

CaP-X demonstrates that "Code-as-Policy" is a viable and powerful paradigm for robot control. By systematically removing human scaffolding and equipping agents with iterative reasoning, visual grounding, and self-synthesized skills, the authors show that coding agents can achieve robust, generalizable, and human-competitive performance in both simulation and the real world. This framework offers a clear path toward developing truly general-purpose embodied intelligence.