💻 computer science

Agentic-VLA: Efficient Online Adaptation for Vision-Language-Action Models

Agentic-VLA is an efficient online adaptation framework for Vision-Language-Action models that leverages adaptive reward synthesis, language-guided exploration, and experience memory to significantly improve generalization, sample efficiency, and cross-task transfer on robotic manipulation benchmarks.

Original authors: Ruofan Jin, Zaixi Zhang

Published 2026-05-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ruofan Jin, Zaixi Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to cook a complex meal, like making a specific type of coffee using a stovetop pot. In the past, you had to show the robot exactly how to do it hundreds of times, step-by-step. If the robot dropped the pot or the stove was in a slightly different spot, it would get confused and fail completely. It was like a student who memorized the answers to a specific test but couldn't solve a similar problem if the numbers changed.

The paper introduces Agentic-VLA, a new way to train robots that acts less like a rigid student and more like a smart apprentice with a helpful mentor. Instead of just copying what they see, this system learns how to learn.

Here is how it works, broken down into three simple "superpowers":

1. The Smart Coach (Adaptive Reward Synthesis)

The Problem: Usually, a robot only gets a "Good job!" or "Try again!" at the very end of a task. If the task is long (like cooking), the robot doesn't know which specific step it messed up.
The Solution: Agentic-VLA acts like a coach who breaks a big goal into small, manageable steps.

The Analogy: Imagine learning to ride a bike. A bad coach just says, "You failed the race." A smart coach says, "Great job balancing on the flat ground! Now, let's try turning left. You wobbled a bit, so let's focus on that."
How it works: The system automatically breaks a complex task (like "make coffee") into tiny sub-goals ("find the stove," "turn it on," "find the pot"). It then watches the robot. If the robot is already good at finding the stove, the coach stops giving points for that and focuses the robot's attention on the part it is struggling with (like placing the pot). It creates a custom "curriculum" that gets harder only as the robot gets better.

2. The Helpful Guide (Language-Guided Exploration)

The Problem: When robots try to learn by themselves, they often just flail around randomly, like a toddler banging on piano keys hoping to hit a note. This wastes a lot of time and energy.
The Solution: Instead of random guessing, the robot gets hints in plain English.

The Analogy: Imagine you are trying to find a hidden key in a messy room. A random search means looking under every rug and in every drawer blindly. A "Language-Guided" search is like a friend whispering, "Hey, I think you're looking from the wrong angle; try checking the left side of the table."
How it works: A special "Critic" model looks at what the robot is doing and suggests specific changes in natural language, like "Try approaching the object from the left" or "Grab the center for better stability." This helps the robot discover good strategies much faster than random trial-and-error.

3. The Memory Book (Experience Memory)

The Problem: If a robot learns to open a drawer, it usually has to start from scratch when it learns to open a cabinet, even though the movements are similar.
The Solution: The robot keeps a "library" of its past successes.

The Analogy: Think of this like a chef who has learned to chop onions. When they need to chop garlic, they don't start from zero; they remember, "I know how to hold the knife and the motion is similar."
How it works: When the robot faces a new task, it looks at its memory book to find a similar task it has already learned. It "warms up" by starting with the skills from that similar task, rather than starting with a blank slate. This lets it learn new things much faster.

The Results: What Did They Find?

The researchers tested this new system on a benchmark called LIBERO (a set of robot manipulation tasks) and a dual-arm robot test called RoboTwin. Here is what happened:

Long Tasks: The robot got 12.3% better at completing long, multi-step tasks compared to previous methods.
Learning from One Example: In a "one-shot" test (where the robot only sees the task once before trying to learn), it improved by 28.5%. It could learn much faster with very little data.
Zero to Hero: Without any specific examples for a new task, the robot could still figure out how to do it about 31% of the time, whereas older methods failed 100% of the time.
Speed: The system learned 2.4 times faster than other online learning methods, meaning it needed far fewer attempts to get good at a task.

Summary

Agentic-VLA is a framework that makes robots smarter learners. Instead of just memorizing videos of humans doing tasks, it uses a Smart Coach to break tasks down, a Helpful Guide to give language-based hints, and a Memory Book to reuse past skills. This allows robots to adapt quickly to new environments and learn complex tasks with much less data and time than before.

Technical Summary: Agentic-VLA: Efficient Online Adaptation for Vision-Language-Action Models

1. Problem Statement

Vision-Language-Action (VLA) models have advanced robotic manipulation by leveraging pre-trained vision-language representations to interpret instructions and generate actions. However, current VLA training paradigms face two critical limitations that hinder real-world deployment:

Poor Generalization: Models trained via Supervised Fine-Tuning (SFT) on expert demonstrations tend to memorize specific trajectories rather than learning underlying task semantics. Consequently, they exhibit brittle performance when deployment conditions deviate slightly from training distributions, lacking mechanisms for error recovery or adaptive replanning.
Low Training Efficiency: Current methods require hundreds of demonstrations per task, with data collection costs scaling linearly with the number of tasks. This makes scaling to the diverse range of tasks required for general-purpose robotics impractical.

While recent works have explored online adaptation using Reinforcement Learning (RL), they suffer from reliance on noisy reward signals, inefficient random exploration in high-dimensional action spaces, and an inability to transfer knowledge across related tasks.

2. Methodology: Agentic-VLA Framework

Agentic-VLA proposes an agentic training framework that reimagines the learning process itself as an intelligent, orchestrated system. Instead of treating reward design, exploration, and knowledge transfer as fixed procedures, the framework employs learned agents to actively manage these components based on the current learning state. The framework integrates three synergistic innovations:

2.1 Adaptive Reward Synthesis (ARS)

To address the absence of oracle reward signals, ARS dynamically generates and adjusts reward functions tailored to the VLA's current capabilities.

Task Decomposition: A language model decomposes complex, long-horizon task instructions ( $l_{task}$ ) into a sequence of intermediate sub-goals ( $G = \{g_1, \dots, g_K\}$ ).
Capability-Aware Weighting: The system maintains an estimate of the VLA's proficiency ( $\hat{c}_k$ ) for each sub-goal, calculated as an exponential moving average of recent success rates. The reward weight for a sub-goal is defined as $w_k = 1 - \hat{c}_k$ . This creates a natural curriculum: sub-goals where the agent struggles receive high weights to encourage focused learning, while mastered sub-goals receive diminished weights.
Dense Progress Rewards: A learned critic model ( $C_\phi$ ) provides dense progress estimates ( $\Delta_k$ ) for each sub-goal. The total reward is the weighted sum of these progress signals, ensuring feedback focuses on the agent's current learning frontier.

2.2 Language-Guided Exploration (LGE)

To overcome the inefficiency of random sampling in high-dimensional action spaces, LGE replaces blind trial-and-error with structured exploration.

VLM-Based Critic: A pre-trained Vision-Language Model (VLM) acts as an exploration critic. Given the current visual observation, task instruction, and trajectory context, the VLM generates natural language suggestions (e.g., "approach from the left") to guide behavior.
Prompt-Augmented Generation: These suggestions are concatenated with the task instruction to bias the VLA's action distribution toward promising behaviors while maintaining stochasticity.
Adaptive Frequency: The frequency of suggestions is dynamically adjusted based on learning progress, providing frequent guidance early in training and reducing it as the policy gains autonomy.

2.3 Experience Memory (EM)

To enable cross-task knowledge transfer, EM maintains a memory bank of adapted policy weights indexed by task embeddings.

Warm-Start Retrieval: When encountering a new task, the system computes a task embedding and retrieves the $k$ most similar previously learned tasks.
Weight Interpolation: The initial policy parameters are computed as a weighted combination of the retrieved policy weights. This provides a "warm start" that accelerates learning on related tasks, rather than learning from scratch.
Capacity Management: The memory bank is updated with successful adaptations and managed to prioritize diverse, high-quality entries.

The training loop utilizes GRPO (Group Relative Policy Optimization) for policy updates, integrating these components into a unified online adaptation cycle.

3. Key Contributions

Agentic Framework: The first agentic framework for VLA online adaptation that unifies adaptive reward synthesis, language-guided exploration, and experience memory.
Adaptive Reward Mechanism: A novel mechanism that automatically decomposes complex tasks and adjusts reward granularity based on real-time learning progress, effectively implementing a capability-aware curriculum.
Structured Exploration: Demonstration that language-guided exploration significantly improves sample efficiency by enabling systematic, goal-directed discovery of strategies.
Cross-Task Transfer: Evidence that experience memory enables effective transfer, achieving non-trivial success rates on tasks without any task-specific demonstrations.
State-of-the-Art Performance: Extensive experiments showing substantial improvements across multiple metrics and benchmarks.

4. Experimental Results

The framework was evaluated on the LIBERO benchmark (four task suites: Spatial, Object, Goal, Long) and the dual-arm RoboTwin 2.0 benchmark.

Overall Performance: Agentic-VLA achieved a state-of-the-art average success rate of 97.8% on LIBERO, outperforming the previous best RL method (EVOLVE-VLA) by +2.0% and the SFT baseline by +8.6%.
Long-Horizon Tasks: The most significant improvement was observed on LIBERO-Long tasks, with a +12.3% gain over the SFT baseline, attributed to the adaptive reward synthesis and curriculum learning.
One-Shot Learning: In a low-data regime with only one demonstration per task, Agentic-VLA achieved a 70.5% success rate, a +28.5% improvement over the SFT baseline and +9.2% over EVOLVE-VLA.
Cross-Task Generalization: Without any task-specific demonstrations, Agentic-VLA improved cross-task transfer success rates from 0% (direct transfer) to 31.2% on the LIBERO-Object suite after training on LIBERO-Long.
Training Efficiency: The framework demonstrated 2.4× faster convergence compared to existing online adaptation methods, reaching 90% success rate in 700 iterations compared to 1,680 for EVOLVE-VLA. It required 22.4k rollouts versus 53.8k for the baseline.
RoboTwin 2.0: The framework retained its advantages on the dual-arm RoboTwin 2.0 benchmark, particularly under the randomized "Hard" setting, achieving a subset average of 34.7% compared to 16.3% for $\pi_0$ and 13.7% for RDT.

Ablation Studies confirmed that each component (ARS, LGE, EM) contributes synergistically. Removing any component resulted in performance drops and increased training iterations. Controlled comparisons against alternative curricula (uniform weights, fixed schedules) and exploration bonuses (RND, ICM) confirmed that the specific design choices of Agentic-VLA were responsible for the gains.

5. Significance and Claims

The authors claim that Agentic-VLA represents a significant step toward truly adaptive VLA systems capable of continuous learning in deployment. By shifting from static imitation learning to an agentic, online adaptation process, the framework addresses the fundamental brittleness and data inefficiency of current VLA models.

The paper highlights that the system enables emergent capabilities such as error recovery (autonomously re-adjusting after a slip), adaptive object handling (adjusting trajectories to displaced objects), and novel strategy discovery (finding manipulation strategies not present in demonstrations).

While acknowledging limitations such as potential reward hacking, memory scalability, and the challenges of deploying component-rich systems on physical robots, the authors position Agentic-VLA as a foundational step toward robust, general-purpose robotic learning that can operate effectively in diverse and changing real-world environments.