💻 computer science

PrimitiveVLA: Learning Reusable Motion Primitives for Efficient and Generalizable Robotic Manipulation

PrimitiveVLA addresses the data inefficiency and poor generalization of Vision-Language-Action models by introducing a primitive-centric framework that disassembles demonstrations into reusable motion primitives via a shared multimodal representation and reassembles them during inference, thereby enabling more efficient learning and robust zero-shot generalization across complex tasks.

Original authors: Yutai Li, Shaohui Peng, Jiaming Guo, Di Huang, Zihao Zhang, Yuxuan Guo, Yunkai Gao, Siming Lan, Ling Li, Xing Hu, Yunji Chen

Published 2026-07-21

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Yutai Li, Shaohui Peng, Jiaming Guo, Di Huang, Zihao Zhang, Yuxuan Guo, Yunkai Gao, Siming Lan, Ling Li, Xing Hu, Yunji Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where robots aren't just clumsy, pre-programmed machines that can only do exactly what they were told to do yesterday, but rather clever helpers that can figure out how to open a new kind of jar or stack a weirdly shaped block without needing a manual for every single object. This is the dream of "Embodied AI"—giving robots a brain that combines what they see, what they hear, and how they move. Right now, scientists are trying to build these brains using something called Vision-Language-Action (VLA) models. Think of these models as a robot's "brain" that looks at a picture, reads a sentence like "put the cup on the table," and then decides which muscles to twitch to make it happen. The big problem is that these robots are currently terrible at learning new things quickly. They are like students who memorize the exact answer key for a specific test but fail completely if the teacher changes the numbers on the questions. They try to memorize the entire, messy path of a task all at once, which makes them slow to learn and easily confused when the world changes.

This is where a new paper called "PrimitiveVLA" comes in to save the day. The researchers argue that the reason robots are so bad at generalizing is that they are trying to learn whole, giant tasks as one big, unbreakable chunk. Instead, they propose a smarter way: teaching the robot to learn "primitives." Imagine you are learning to cook. Instead of memorizing the exact recipe for "Spaghetti Carbonara" as one giant, confusing list of steps, you learn the basic moves: chop, boil, stir, and fry. Once you master those reusable moves, you can mix and match them to make almost any dish, even ones you've never seen before. The authors suggest that robots should do the same thing. They built a system that breaks down complex robot tasks into these small, reusable "motion primitives" (like grasp, push, or lift) during training. Then, when the robot faces a new, tricky job, it doesn't try to remember the whole thing; it just assembles the right sequence of these basic moves to solve the problem.

The paper shows that this "disassemble and assemble" approach works incredibly well. In their tests, robots trained with this method learned much faster, needing only half the amount of practice data to perform as well as robots trained on full datasets. But the real magic happened when they tested the robots on things they had never seen before. For example, on a set of long, complicated tasks, the standard top-tier robot model only succeeded about 30% of the time. However, the robot using PrimitiveVLA jumped up to an 80% success rate. Even more impressively, when faced with completely new tasks that the robot had never encountered, the new method improved the success rate by six times compared to the old way. The researchers tested this in both computer simulations and on a real physical robot arm, proving that teaching robots to master the basics first, rather than memorizing the whole picture, is the key to making them truly smart and adaptable helpers.

Technical Summary: PrimitiveVLA

Problem Statement

Current Vision-Language-Action (VLA) models, such as RT-2 and OpenVLA, face significant bottlenecks in data efficiency and generalization when adapted to downstream robotic tasks. The authors argue these limitations stem from the prevailing Direct Instruction-to-Control Mapping paradigm. In this conventional approach, models are trained on monolithic task trajectories, forcing them to memorize scene-specific visual patterns and task-context correlations rather than learning reusable, invariant physical motion patterns. Consequently, knowledge gained from one task (e.g., opening a cabinet) fails to transfer to similar tasks (e.g., opening a microwave), leading to brittle performance on novel object-skill combinations and long-horizon tasks.

Methodology: PrimitiveVLA

To address these issues, the authors propose PrimitiveVLA, a framework that shifts the learning paradigm from monolithic trajectory memorization to a Primitive-Centric Disassemble & Assemble approach. The framework operates through two distinct phases unified by a shared Multimodal Canonical Representation (MCR).

1. Fine-tuning Phase: Primitive Disassembly

Instead of training on full trajectories, the framework disassembles demonstrations into a library of reusable motion primitives (e.g., Grasp, Push, Twist). This phase addresses the scarcity of fine-grained labels in public datasets through an automated pipeline:

Primitive Sequence Reasoning: A Vision-Language Model (VLM) analyzes task instructions and video demonstrations to predict the sequence of primitives required, without needing explicit boundary annotations.
State-Based Boundary Segmentation: An LLM generates Python code to define termination criteria for each primitive based on physical state changes (e.g., gripper width, end-effector height). This code automatically segments trajectories into discrete primitive samples.
Learning Objective: The VLA is fine-tuned on these segmented samples to master the underlying motion patterns rather than specific task contexts.

2. Inference Phase: Primitive Assembly

During execution, the system assembles the learned primitives to solve complex or novel tasks:

Primitive Planner: A VLM retrieves similar task-primitive sequences from a Disassembly Library and generates a high-level plan for the current task.
Primitive Switch Controller: An LLM-generated switch module monitors the robot's state in a closed-loop manner. It uses statistical trends from a sliding history window to trigger transitions between primitives, ensuring robust execution flow and preventing premature switching due to sensor noise.

3. Multimodal Canonical Representation (MCR)

To ensure that primitives remain consistent and reusable across different tasks, the framework employs MCR to unify inputs:

Semantic Unification: Task-specific instructions are mapped to canonical primitive instructions (e.g., "grasp the black bowl" becomes "grasp the object with green mask"). This forces the model to learn the action associated with the primitive identity rather than textual variations.
Visual Compatibility: Raw observations are transformed into object-centric masks (using SAM and Cutie). This preserves task-specific context while providing the VLA with a uniform visual feature format, decoupling motion patterns from specific scene details.

Key Contributions

Primitive-Centric Paradigm: The introduction of a "Disassemble & Assemble" paradigm that shifts VLA learning from memorizing indivisible trajectories to mastering a finite set of reusable physical primitives.
PrimitiveVLA Framework: A model-agnostic framework that unifies fine-tuning and inference via MCR. It features an automated segmentation pipeline for disassembly and a hierarchical VLM-LLM mechanism for assembly.
Empirical Validation: Extensive experiments demonstrating significant improvements in data efficiency and zero-shot generalization across simulation and real-world environments.

Experimental Results

The authors evaluated PrimitiveVLA on the Libero suite (including Libero-90, Libero-90-Novel, and Libero-Long) and the RLBench suite, as well as a real-world UR5e robot setup.

Data Efficiency: PrimitiveVLA significantly improves performance with limited data. On Libero-90, it boosted OpenVLA performance by 9.2% and OpenVLA-OFT by 5.0%. Notably, using only 50% of the training data, PrimitiveVLA outperformed the full-data baseline of standard OpenVLA.
Zero-Shot Generalization (OOD): The method demonstrated superior generalization to unseen tasks (Libero-90-Novel). It achieved a 6× improvement over the standard OpenVLA baseline (45.50% vs. 7.38%) and improved the SOTA $\pi0.5$ from 56.00% to 75.50%.
Long-Horizon Tasks: On long-horizon tasks (Libero-Long), where standard models often fail catastrophically (<5% success), PrimitiveVLA elevated the success rate of $\pi0.5$ from 30.50% to 80.25%.
Real-World Performance: In real-world experiments, the method achieved a 90% average success rate on in-distribution tasks and demonstrated robust zero-shot generalization, improving success on unseen objects from 10% to 80% and enabling compositional tasks where the baseline failed (0% to 60-70%).
Ablation Studies: Results confirmed that the MCR is the primary driver for out-of-domain generalization, while Primitive Disassembly is critical for stability in long-horizon execution.

Significance

The paper claims that the disassembly of physical interactions into reusable primitives is essential for scalable Embodied AI. By decoupling invariant motion patterns from specific task contexts, PrimitiveVLA resolves the dual bottlenecks of data inefficiency and poor generalization. The authors posit that this approach allows models to flexibly assemble learned primitives to solve novel, unseen, and long-horizon tasks, moving beyond the limitations of direct instruction-to-control mapping.

Limitations: The authors note that the current primitive set relies on a pre-defined kinematic taxonomy, which may not cover highly specialized dexterous manipulations. Future work aims to explore unsupervised primitive disassembly and end-to-end differentiable planning.