Technical Summary: Latent Reasoning VLA (LaRA-VLA)

Problem Statement

Vision-Language-Action (VLA) models have demonstrated significant potential for scalable robotic manipulation by mapping multimodal observations and language instructions to continuous control commands. A prevailing strategy to enhance performance involves Chain-of-Thought (CoT) reasoning, which provides structured intermediate steps for task decomposition and planning. However, existing CoT approaches face two fundamental limitations:

1. High Inference Overhead: Text-based CoT methods require generating extensive discrete reasoning tokens during inference. This leads to substantial computational costs, excessive KV-cache usage, and high latency, often reducing control frequencies to below 5 Hz, which is insufficient for real-time robotic control.
2. Representational Mismatch: Most CoT formulations rely on discrete representations (language tokens or discrete visual tokens from VQ-based tokenizers). This creates a mismatch with the inherently continuous nature of robotic perception and action spaces. Constraining reasoning to discrete tokens may hinder the fluidity required for embodied control.

Methodology: LaRA-VLA

To address these challenges, the authors propose Latent Reasoning VLA (LaRA-VLA), a unified framework that internalizes multimodal CoT reasoning into continuous latent representations. Instead of generating explicit reasoning traces at inference time, LaRA-VLA performs reasoning and prediction entirely within a latent space, aligning with continuous perception and control.

1. Data Construction

The authors developed an automated annotation pipeline to construct structured CoT datasets (LIBERO-LaRA and Bridge-LaRA) for both simulation and real-world environments. This pipeline employs an "anchor-first, generate-later" paradigm:

• Semantic Anchors: Extracted using Qwen3-VL to identify manipulated objects from initial frames and instructions.
• Temporal Anchors: Generated by segmenting robot trajectories into atomic stages based on gripper state changes.
• Annotations: The system generates concise subtask descriptions, temporally consistent target object bounding boxes (via GroundingDINO and SAM3), and motion reasoning derived from end-effector trajectories.

2. Model Architecture

LaRA-VLA utilizes Qwen3-VL as the backbone Vision-Language Model (VLM), inheriting its image encoder for consistent visual representations.

• Visual Goal Latents: A dedicated <img next> token is introduced to represent predicted visual latents, enabling explicit supervision and alignment during early training stages.
• Action Expert: In the final stage, action generation is decoupled from autoregressive token decoding. A 16-layer Diffusion Transformer (Diffusion Transformer) acts as an action expert, conditioning on learned latent representations to produce continuous action trajectories.

3. Three-Stage Curriculum Training

The training paradigm follows a progressive transition from explicit reasoning to latent embodied reasoning:

• Stage I: Explicit CoT Fine-Tuning. The model is fine-tuned on datasets with explicit textual CoT annotations and visual goal predictions. It learns to autoregressively generate discrete CoT tokens and align future visual latents ($z_{t+1}$) using an Exponential Moving Average (EMA) target encoder to prevent representation collapse. Action tokens are also trained via an inverse dynamics objective.
• Stage II: Curriculum-Based Replacement. Discrete CoT tokens are progressively masked and replaced by learnable continuous latent representations. The proportion of explicit tokens decreases according to a curriculum schedule, while the model relies increasingly on latent reasoning, guided by visual prediction objectives and action signals.
• Stage III: Action Generation via Flow Matching. Explicit CoT supervision is removed. The model adapts to continuous control using a flow matching objective. The action expert predicts a velocity field conditioned on the multi-modal latent context (current visual/textual inputs, intermediate reasoning latents, and predicted future visual latents) to generate continuous action trajectories without explicit CoT generation.

4. Attention Mechanism

A specialized attention mechanism regulates cross-token information flow. In Stages I and II, future image tokens attend causally to text and current image tokens. In Stage III, action tokens are excluded from the attention computation, and the model operates solely on text and vision tokens under the same constraints, ensuring efficient inference.

Key Contributions

1. Latent Reasoning Paradigm: The paper introduces a paradigm where CoT reasoning is internalized into continuous latent representations across textual and visual modalities, enabling inference-efficient reasoning that aligns with continuous perception and control.
2. LaRA-VLA Framework: A unified VLA model realizing this paradigm through a curriculum-based training strategy, transitioning from explicit multi-modal CoT supervision to latent embodied reasoning, stabilized by EMA-based visual encoders.
3. Structured Datasets: The construction of two structured CoT datasets (LIBERO-LaRA and Bridge-LaRA) featuring multi-modal reasoning annotations (subtask decomposition, movement reasoning, target localization) for embodied manipulation.

Experimental Results

The authors evaluated LaRA-VLA on simulation benchmarks (LIBERO, SimplerEnv) and long-horizon real-robot manipulation tasks.

• Simulation Performance: On the LIBERO benchmark, LaRA-VLA achieved a state-of-the-art average success rate of 97.9%, outperforming both No-CoT baselines (e.g., OpenVLA, $\pi_0$) and explicit CoT methods (Textual and Visual). On SimplerEnv-WidowX, it attained the highest average success rate of 68.8%, demonstrating superior real-to-sim generalization.
• Real-World Performance: In real-world long-horizon tasks (e.g., sorting fruits, stacking bowls), LaRA-VLA achieved the highest average success rate among compared methods (ACT, ECoT, GR00T N1.5), particularly excelling in tasks requiring multi-stage reasoning and temporal coordination.
• Inference Efficiency: LaRA-VLA reduced inference latency by up to 90% compared to explicit CoT-based approaches, achieving a rollout time of 135 ms on an NVIDIA A100 GPU. This enables control frequencies suitable for real-time robotics.
• Robustness: The model demonstrated greater robustness to visual perturbations (Gaussian noise and blur) compared to non-latent baselines, with latent representations remaining semantically consistent under corruption.
• Ablation Studies: Results confirmed that latent CoT supervision provides larger gains than explicit textual CoT. The combination of latent textual and visual CoT yielded the best performance, highlighting the benefit of multimodal alignment.

Significance and Claims

The paper claims that LaRA-VLA demonstrates that structured reasoning for embodied agents can be effectively realized in latent space without explicit chain-of-thought generation at inference time. By internalizing reasoning into continuous latent dynamics, the approach resolves the representational mismatch between discrete tokens and continuous control while significantly reducing computational overhead. The authors posit that this paradigm offers a viable path toward efficient, real-time, and robust robotic manipulation, moving beyond the limitations of verbose, discrete reasoning traces.

The authors acknowledge limitations, noting that latent representations may suffer from collapse without explicit supervision (mitigated here by restricting latent tokens and using EMA) and that the curriculum training process incurs higher costs during the transition phases. However, the work establishes a foundation for efficient, action-oriented reasoning in VLA models.