InternVLA-A1: Unifying Understanding, Generation and Action for Robotic Manipulation

1. Problem Statement

Current Vision-Language-Action (VLA) models, typically built upon Multimodal Large Language Models (MLLMs), excel at semantic understanding but struggle with physical world dynamics. They lack the ability to reason about how states evolve under motion and contact, leading to brittleness in dynamic environments (e.g., industrial conveyor belts).

Conversely, World Models based on video prediction can forecast future states but often suffer from:

• Lack of semantic grounding: They struggle to understand complex instructions.
• Error sensitivity: Small prediction errors in video generation can cascade into catastrophic action failures.
• Real-time limitations: High-fidelity video generation is often too computationally heavy for high-frequency robotic control.

Existing approaches also face data bottlenecks, relying heavily on expensive real-world robot data or struggling with the sim-to-real gap when using synthetic data alone.

2. Methodology: InternVLA-A1

The authors propose InternVLA-A1, a unified framework that integrates scene understanding, visual foresight generation, and action execution into a single architecture.

A. Architecture: Mixture-of-Transformers (MoT)

The model employs a unified MoT architecture with three specialized "experts" that interact via a unified masked self-attention mechanism:

1. Understanding Expert: Based on MLLMs (InternVL3 or Qwen3-VL). It encodes multimodal inputs (images and text instructions) into contextual embeddings ($h_{und}$). This provides the semantic grounding.
2. Generation Expert: A lightweight module designed for real-time inference. Instead of autoregressive token prediction, it uses a non-autoregressive, parallel decoding strategy.
   • Input: 6 images (3 views $\times$ 2 timestamps: $t$ and $t-15$).
   • Tokenization: Uses a COSMOS VAE encoder to compress images into latent grids.
   • Compression: Reduces sequence length via convolutional downsampling (from 1024 tokens/image to 16 tokens/image).
   • Output: Predicts future visual states ($t+15$) in a single forward pass, providing "visual foresight" without the latency of standard video generators.
3. Action Expert: Takes the semantic context ($h_{und}$), predictive dynamics ($h_{gen}$), and proprioceptive state ($q_t$) to predict action chunks. It utilizes Flow Matching to generate continuous control commands, handling multi-modal action distributions better than direct regression.

Information Flow: The architecture enforces a strict causal flow: Understanding $\to$ Generation $\to$ Action. Later blocks can attend to earlier blocks, but not vice versa.

B. Training Strategy & Data Corpus

InternVLA-A1 is trained on a massive, heterogeneous dataset of 692M frames using a two-stage process (Pre-training and Post-training):

1. Synthetic Data (InternData-A1 & RoboTwin): Provides scalability and diversity (rigid, deformable, fluid objects).
2. Real-World Robot Data (AgiBot-World, RoboMind): Grounds the model in physical reality to minimize the sim-to-real gap.
3. Egocentric Human Videos (EgoDex): Enriches visual representations of manipulation skills without requiring robot teleoperation.

Optimization Objectives:

• Visual Foresight Loss: Minimizes the distance between predicted and ground-truth latent future frames.
• Flow Matching Loss: Learns a velocity field to transform noise into expert action trajectories.
• Joint Loss: $L_{total} = \lambda L_{gen} + L_{act}$, ensuring representational consistency across modalities.

3. Key Contributions

1. Unified Architecture: First VLA to tightly couple semantic reasoning (MLLM) with dynamic prediction (World Model) in a single MoT framework, bridging the gap between "what" and "how it moves."
2. Efficient Visual Foresight: Introduces a non-autoregressive, parallel decoding generation expert that achieves real-time inference (~13 Hz) while providing sufficient dynamic guidance, overcoming the latency issues of previous video-prediction-based VLAs.
3. Hybrid Data Recipe: Demonstrates that joint training on synthetic data, real robot data, and human videos significantly outperforms models trained on any single source, effectively closing the sim-to-real gap.
4. Scalable Implementation: Successfully instantiated at 2B and 3B parameter scales, proving that smaller, specialized models can outperform larger, generic baselines when architecture and data are optimized.

4. Experimental Results

The model was evaluated on 12 real-world tasks (static and dynamic) and the RoboTwin 2.0 simulation benchmark.

• Static Manipulation: InternVLA-A1 (3B) achieved a 75.1% average success rate, outperforming the strong baseline $\pi0.5$ (70.7%) by +4.4%. It notably excelled in long-horizon tasks like "Make Sandwich" (93.3% vs 73.3%).
• Dynamic Manipulation: The model showed massive gains in dynamic scenarios where targets are moving.
  • In-motion Ingredient Picking: Achieved 93.3% success vs. 66.7% for $\pi0.5$ (+26.7% improvement).
  • Express Sorting: Achieved 80.0% success vs. 53.3% for $\pi0.5$.
  • Significance: This proves the model's ability to reason about momentum and future states, which previous reactive models failed to do.
• Simulation Benchmark (RoboTwin 2.0): Outperformed $\pi0.5$ by +2.6% in both Easy and Hard (domain randomized) settings.
• Ablation Studies:
  • Removing pre-training caused a 51.6% drop in performance.
  • Removing the Generation Expert caused a 19.4% drop in average success rate, with the most severe impact on dynamic tasks, validating the necessity of visual foresight.

5. Significance

InternVLA-A1 represents a paradigm shift in robotic learning by moving beyond reactive "perception-to-action" mapping toward foresight-aware manipulation. By unifying semantic understanding with dynamic prediction, it enables robots to operate robustly in complex, changing environments (e.g., conveyor belts, moving objects) where traditional VLA models fail. The work also provides a blueprint for efficient training, showing that a hybrid data strategy (Sim + Real + Human) combined with a specialized architecture can achieve state-of-the-art performance with relatively small model sizes (2B-3B parameters).