Demystifying Action Space Design for Robotic Manipulation Policies

This paper presents a large-scale empirical study demonstrating that the design of a robotic manipulation policy's action space significantly impacts learning performance, revealing that predicting delta actions improves results while joint-space and task-space representations offer complementary benefits for stability and generalization, respectively.

The Gist

Imagine you are teaching a robot to do chores, like picking up a cup or stacking blocks. You want the robot to learn by watching you do it (this is called "Imitation Learning").

For years, researchers have been trying to make these robots smarter by giving them more data and bigger brains (better AI models). But they kept ignoring one crucial question: How exactly do we tell the robot what to move?

This paper is like a massive, scientific "taste test" to figure out the best way to give instructions to a robot arm. The authors ran over 13,000 real-world experiments with actual robots to find the "secret sauce" for robot control.

Here is the breakdown of their findings using simple analogies:

1. The Two Big Questions

The researchers realized that giving a robot an instruction is like giving directions to a friend. You have to decide two things:

• The "Map" (Spatial Abstraction): Do you tell the robot, "Move your shoulder and elbow to these specific angles" (Joint Space), or do you say, "Move your hand to these specific X, Y, Z coordinates in the room" (Task Space)?
• The "Step" (Temporal Abstraction): Do you say, "Go to the cup" (Absolute), or do you say, "Move your hand 2 inches to the right" (Delta/Relative)?

2. The "Map" Dilemma: Angles vs. Coordinates

• Joint Space (The Angles): Imagine telling a dancer, "Bend your knee 45 degrees, rotate your hip 30 degrees."
  • Pros: It's very stable. The robot knows exactly how its own body works.
  • Cons: It's hard to learn. The robot has to figure out how those angles translate to moving a cup in the real world. It's like trying to navigate a city by only knowing how many steps to take with each foot, without looking at the street signs.
• Task Space (The Coordinates): Imagine telling the dancer, "Walk to the red chair."
  • Pros: It's intuitive. The robot sees the cup and knows where to go.
  • Cons: It can be shaky. If the math used to convert "red chair" into "knee angles" is slightly off, the robot might miss the target or get stuck.

The Verdict:

• If you are training a robot on one specific arm with lots of data and time, Joint Space (Angles) usually wins. It's like a professional dancer who knows their body perfectly.
• If you want the robot to learn quickly or switch between different robots (like moving from a small arm to a big arm), Task Space (Coordinates) is better. It's like a GPS that works on any car, regardless of the brand.

3. The "Step" Dilemma: Destination vs. Direction

This was the paper's biggest discovery.

• Absolute (Destination): "Go to the cup."
• Delta (Direction): "Move your hand 2 inches toward the cup."

The Analogy:
Imagine you are walking through a foggy forest.

• Absolute (Destination): You try to guess the exact location of the campfire from where you are standing. If you guess wrong by a little bit, you might be miles off. As you try to plan a long path, your errors pile up, and you get lost.
• Delta (Direction): You just take one small step toward the fire. Then you look again and take another small step. Even if you stumble a little, you can correct it on the next step.

The Verdict:
Delta (Direction) is almost always better.
The paper found that telling the robot "move a little bit" (Delta) is much more stable and easier to learn than telling it "go to this exact spot" (Absolute). It prevents the robot from getting confused by small errors.

4. The "Chunking" Secret

Modern robots don't just predict one move; they predict a whole sequence of moves at once (like a video clip of movement). The researchers found that how you link these moves matters.

• Bad Way: Linking moves like a chain reaction (Step-wise). If the first link is slightly wrong, the error gets multiplied down the line, and the robot goes wildly off course.
• Good Way: Linking moves as a single block (Chunk-wise). Every move in the sequence is calculated relative to the start of the sequence. If one part is slightly off, it doesn't ruin the whole plan.

The Final "Cheat Sheet" for Robot Designers

Based on 13,000+ tries, here is the recipe for the best robot policy:

1. Always use "Delta" instructions: Tell the robot to "move a little bit" rather than "go to a specific spot." It's more stable.
2. Use "Chunk-wise" grouping: Predict a whole block of moves at once, but calculate them all relative to the start of that block.
3. Choose your "Map" based on your goal:
   • For maximum performance on one specific robot: Use Joint Space (angles) + Delta (direction). This is the "Power User" combo.
   • For robots that need to work on different machines or learn fast: Use Task Space (coordinates) + Delta (direction). This is the "Generalist" combo.

Why This Matters

Before this paper, people were guessing which method worked best. Some used old methods from 10 years ago; others tried random new tricks. This study provides a clear, scientific rulebook. It tells us that the way we "speak" to robots is just as important as the robot's "brain" itself. By speaking the right language (Delta + Chunking), we can make robots that are more reliable, learn faster, and actually get the job done.

Technical Summary

1. Problem Statement

While recent advancements in robotic manipulation have focused heavily on scaling training data and model capacity (e.g., foundation models), the specification of the action space remains an under-explored yet critical determinant of success.

• The Ambiguity: There is no consensus on the best practices for designing action spaces. Researchers often rely on ad-hoc heuristics or legacy designs (e.g., choosing between joint-space vs. task-space, or absolute vs. delta representations) without a unified understanding of their impact.
• The Consequence: Subtle changes in the action interface can drastically alter the optimization landscape, leading to policies that either fail to generalize or lack deployment stability.
• The Goal: To provide a systematic, large-scale empirical study that dissects action space design to establish principled guidelines for robotic policy learning.

2. Methodology

The authors propose a structured framework to analyze action spaces along two orthogonal axes: Spatial Abstraction and Temporal Abstraction, and investigate their interplay with Action Chunking.

A. Action Abstraction Taxonomy

1. Spatial Abstraction:
   • Joint-Space (Configuration Space): Directly predicts joint positions. Avoids Inverse Kinematics (IK) singularities but requires the policy to learn complex, non-linear kinematic mappings from visual inputs.
   • Task-Space (End-Effector Pose): Predicts the gripper's position/orientation. Geometrically intuitive but relies on IK solvers during deployment, introducing numerical instability and error accumulation.
2. Temporal Abstraction:
   • Absolute (0th-order): Predicts global target states directly.
   • Delta (1st-order): Predicts relative state increments (displacements).
3. Action Chunking:
   • The paper investigates how chunking (predicting a sequence of future actions) interacts with temporal abstraction.
   • Key Distinction: It analyzes Step-wise Delta (relative to the immediately preceding predicted state) vs. Chunk-wise Delta (relative to the robot's state at the start of the chunk).

B. Experimental Setup

• Scale: The study involves 13,000+ real-world rollouts and evaluation of 500+ trained models.
• Platforms:
  • Real-world: Single-arm and bimanual AgileX robots, and AIRBOT.
  • Simulation: RoboTwin-2.0 benchmark (10 tasks).
• Tasks: A curriculum of 4 real-world tasks (Touch Cube, Pick Up Cup, Pick & Place, Bimanual Transfer) ranging from precision checks to complex coordination.
• Models: Evaluated across different architectures including Regression-based (ACT) and Flow-Matching-based (Diffusion Policy) models, as well as foundation model transfer (using $\pi_0$).
• Protocol: Rigorous grid-based spatial coverage to ensure statistical significance and mitigate distribution shifts.

3. Key Contributions & Findings

RQ1: Implementation Nuances are Decisive

• Chunk-wise vs. Step-wise Delta: The paper proves theoretically and empirically that Chunk-wise Delta is superior to Step-wise Delta.
  • Theory: Step-wise integration amplifies prediction noise linearly with the horizon ($O(k)$), whereas Chunk-wise and Absolute actions maintain a constant error bound ($O(1)$).
  • Result: Chunk-wise delta outperforms step-wise by an average of 10% across tasks.
• Horizon Coupling: The optimal execution horizon depends on the abstraction. Absolute actions benefit from longer horizons, while Delta actions peak at shorter horizons due to drift sensitivity.

RQ2: Systematic Trends in Action Abstraction

• Temporal Superiority (Delta > Absolute): Across all platforms, tasks, and model types, Delta representations consistently outperform Absolute representations.
  • Reasoning: Learning a direct mapping to global coordinates is difficult due to variable target distributions. Delta actions provide a more tractable inductive bias by focusing on immediate displacement.
• Spatial Superiority (Joint > Task, with caveats):
  • Joint-space generally provides more robust performance, especially when paired with strong generative modeling (e.g., Flow Matching/Diffusion) which can handle the complex, multi-modal distribution of joint configurations.
  • Task-space is competitive in low-data regimes but often lags in high-capacity settings due to IK instability.

RQ3: Consistency and Scaling

• Scaling Laws: As data volume and training epochs increase, the superiority of Joint-space + Delta becomes even more pronounced for standard imitation learning.
• Generalization Exception: In Cross-Embodiment and Transfer Learning scenarios (e.g., transferring from $\pi_0$ to a new robot), Task-space (EEF) representations show a distinct advantage.
  • Reasoning: Task-space abstracts away robot-specific kinematics, making the policy more embodiment-invariant and better suited for transfer.

4. Practical Guidelines (Takeaways)

The authors summarize their findings into actionable guidelines for future research:

1. Temporal Abstraction: Use Delta (relative) actions as the default for modern policy backbones. They offer superior sample efficiency and stability.
2. Spatial Abstraction:
   • For standard single-embodiment tasks with sufficient data and compute: Use Joint-space control. It offers the highest robustness and performance.
   • For generalization tasks (Cross-embodiment, Transfer Learning): Use Task-space control to leverage embodiment invariance.
3. Implementation Details:
   • Always use Chunk-wise Delta alignment rather than Step-wise.
   • Tune the execution horizon based on the abstraction: Shorter horizons for Delta, longer for Absolute.

5. Significance

• Demystification: This work moves the field away from "ad-hoc" heuristics toward a principled understanding of how action spaces shape the optimization landscape.
• Foundation for Generalists: By clarifying the trade-offs between stability (Joint) and transferability (Task), the paper provides a roadmap for designing foundation models that can scale across different robot morphologies.
• Reproducibility: The release of a large-scale benchmark and the identification of critical implementation details (like chunk-wise alignment) address a major source of irreproducibility in robotic learning literature.

In conclusion, the paper establishes that action space design is not a trivial implementation detail but a fundamental architectural choice that dictates the learnability, stability, and generalizability of robotic policies.