Technical Summary: OneVLA – A Unified Framework for Embodied Tasks

Problem Statement

Current Vision-Language-Action (VLA) models for embodied intelligence face a fundamental architectural limitation: they are typically constrained to single-task domains (either navigation or manipulation) or require task-specific model variants to handle multiple tasks. While cross-task models like UniVLA have attempted to transfer knowledge across different robotic embodiments, they still rely on separate action heads or specific model configurations for different tasks. This fragmentation hinders the development of truly general-purpose robotic agents capable of seamlessly interpreting natural language and interacting physically with their surroundings across diverse domains. The paper identifies two key questions: (1) Can a unified VLA architecture perform both navigation and manipulation without task-specific variants? and (2) Can these two fundamental tasks mutually benefit from joint training to improve overall performance?

Methodology

The authors propose OneVLA, a unified VLA architecture designed to generate both language responses and robotic actions for navigation and manipulation within a single framework.

1. Unified Architecture

• Unified Vision-Language Encoder: The model utilizes a shared backbone comprising a 32-layer Vision Transformer (processing multi-view RGB images) and a 36-layer Transformer language model (using GQA). Visual features are projected into the language model's hidden dimension ($d_h=2048$) to create a unified multimodal representation.
• Token-Based Decoding: To handle dual outputs, the vocabulary is expanded with special tokens <text>...</text> and <action>...</action>. This allows the model to autoregressively generate a text response explaining the task understanding, followed by the action sequence, in a single forward pass.
• Unified Action Head: The core innovation is a single action head that concatenates the action dimensions of navigation and manipulation into an 11-dimensional unified space:
  • Navigation ($a_{navi} \in \mathbb{R}^4$): Discrete probability distribution for commands (forward, turn left, turn right, stop).
  • Manipulation ($a_{mani} \in \mathbb{R}^7$): Continuous 7-DOF end-effector control ($\Delta x, \Delta y, \Delta z, \Delta roll, \Delta pitch, \Delta yaw, g$).
• Flow-Matching Mechanism: The action head employs a flow-matching diffusion process to predict future action sequences. It learns to predict a velocity field by minimizing the flow-matching objective, using an action encoder, a DiT backbone with cross-attention, and an action decoder.
• Task-Specific Weighting: To accommodate heterogeneous action spaces, the model uses task-specific weight masks during training. These masks ensure that only relevant action dimensions contribute to the loss calculation for a given sample, allowing simultaneous learning without architectural switching.

2. Multi-Stage Progressive Training Strategy

To facilitate strong positive transfer and mutual reinforcement, the authors employ a three-stage training strategy:

• Stage 1 (Manipulation Foundation): Trains solely on manipulation datasets and general VQA data to establish robust manipulation skills and visual-language understanding.
• Stage 2 (Cross-Task Transfer): Introduces navigation data into joint training while retaining manipulation and VQA data. This stage leverages the unified action head to learn both tasks simultaneously, enabling knowledge transfer between domains.
• Stage 3 (Reasoning Enhancement): Integrates Chain-of-Thought (CoT) data to refine the model's reasoning capabilities and decision-making quality across both tasks.

3. Loss Function

The model is trained end-to-end using a composite loss function:
$$L_{total} = \lambda_{vlm}L_{vlm} + \lambda_{text}L_{text} + \lambda_{action}L_{action}$$

• $L_{vlm}$: Standard cross-entropy for general vision-language alignment.
• $L_{text}$: Next-token prediction for the reasoning text phase (including boundary tokens).
• $L_{action}$: Flow-matching loss computed with task-specific weight masks to handle heterogeneous outputs.

Key Contributions

1. Unified Architecture: Proposal of OneVLA, the first VLA framework capable of performing both navigation and manipulation tasks without requiring task-specific model variants or separate action heads.
2. Innovative Components: Introduction of a unified action head with flow-matching and a multi-stage progressive training strategy that facilitates effective mutual enhancement between different task domains.
3. Empirical Evidence of Mutual Benefit: Comprehensive experiments demonstrating that joint training on navigation and manipulation leads to significant performance improvements for both tasks compared to single-task training.
4. New Paradigm: Breaking down architectural barriers between independent task domains to establish a new paradigm for general-purpose robotic systems.

Experimental Results

The authors evaluated OneVLA on both simulated and real-world benchmarks, comparing it against specialized single-task models and existing cross-task models.

• Simulated Benchmarks:

  • Navigation (VLN-CE): OneVLA achieved state-of-the-art (SOTA) performance, reaching 68.6% on the R2R dataset and 58.2% on the RxR dataset. This significantly outperformed UniVLA (47.1% and 26.3% respectively) and specialized navigation models like StreamVLN.
  • Manipulation (SimplerEnv): OneVLA achieved an average success rate of 64.5%, outperforming UniVLA-Mani (35.4%) and the SOTA manipulation model $\pi0$-Fast (48.3%).
  • Efficiency: Notably, the 3B parameter OneVLA model outperformed all 7B parameter task-specific models, demonstrating the efficiency of the unified design.

• Real-World Experiments:

  • Using a Franka Research 3 robotic arm and a mobile robot platform, OneVLA achieved 78.8% success in manipulation tasks and 77.5% in navigation tasks.
  • These results represent improvements of 16.3% and 35.0% respectively over UniVLA in real-world settings, confirming the model's ability to generalize beyond simulation.

• Ablation Studies:

  • Multi-Stage Training: Compared to single-stage training, the three-stage strategy improved R2R performance by 12.5% and SimplerEnv performance by 18.3%.
  • Joint Training: Joint training improved performance over single-task training by 7.3% (R2R) and 5.5% (SimplerEnv), validating the hypothesis that shared representations benefit both domains.
  • Action Horizon: An action horizon of 5 time steps was found to provide the optimal balance between temporal modeling and prediction accuracy.

Significance and Claims

The paper claims that OneVLA paves the way for truly general-purpose robotic systems by unifying core embodied capabilities (navigation and manipulation) into a single, cohesive framework. The authors assert that this approach is not only architecturally more elegant—eliminating the need for task-specific variants—but also superior in performance. By demonstrating that joint training leads to mutual performance improvements, the work suggests that a unified approach is essential for advancing embodied intelligence toward Artificial General Intelligence (AGI). The model and source code are publicly released to support further research in this direction.