🤖 AI

ProgressVLA: Progress-Guided Diffusion Policy for Vision-Language Robotic Manipulation

The paper introduces ProgressVLA, a novel vision-language-action model that enhances robotic manipulation in long-horizon tasks by integrating a pre-trained progress estimator with a differentiable guidance mechanism to overcome the limitations of heuristic-based termination and significantly improve success rates and generalization.

Original authors: Hongyu Yan, Qiwei Li, Jiaolong Yang, Yadong Mu

Published 2026-03-31

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Hongyu Yan, Qiwei Li, Jiaolong Yang, Yadong Mu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to make a sandwich. You give it a simple command: "Make a peanut butter and jelly sandwich."

Most current robot brains (called Vision-Language-Action models) are like students who are very good at following instructions step-by-step but have no sense of time or progress. They might grab the bread, then stare at it for a long time, then grab the knife, then put the knife down, then pick it up again, and then... they just keep doing random things until they accidentally finish the sandwich. They don't know if they are 10% done or 90% done. They only know when they are completely done, and they often don't know when to stop, leading to wasted time or broken toast.

This paper introduces ProgressVLA, a new way to teach robots that acts like giving them a smart GPS with an ETA.

Here is how it works, broken down into simple concepts:

1. The Problem: The "Lost in the Kitchen" Robot

Current robots are like a person walking through a dark room with a flashlight. They can see the object right in front of them (the bread, the knife), but they can't see the whole path to the goal. They rely on guesswork. If they get stuck, they might spin in circles or repeat the same motion over and over because they don't have a way to measure, "Hey, I'm actually getting closer to the goal!"

2. The Solution: The "Progress GPS"

The authors built a special tool called a Progress Estimator. Think of this as a smart assistant sitting next to the robot, watching everything it does.

What it does: It looks at the robot's current view and the original instruction.
What it says: It gives the robot a score from 0 to 100%.
- 0%: "You haven't started."
- 50%: "You have the bread and the knife, but no spread yet."
- 95%: "You are just putting the final slice of bread on top!"

This assistant is trained on thousands of videos of humans doing tasks, so it knows what "progress" looks like even if the lighting changes or the objects are different.

3. The Magic: "Steering the Ship" (Classifier Guidance)

This is the coolest part. Usually, robots generate their movements by "dreaming" up possibilities and picking one. It's like rolling a dice to decide what to do next.

With ProgressVLA, the robot doesn't just roll the dice. It asks the Progress GPS for advice before it moves.

The Analogy: Imagine you are trying to find your way out of a maze.
- Old Way: You pick a random hallway, walk down it, hit a dead end, and go back.
- ProgressVLA Way: Every time you are about to pick a hallway, you ask your GPS, "If I go left, how close will I be to the exit?" The GPS says, "Going left gets you 20% closer; going right gets you 5% closer."
- The Result: The robot naturally "steers" its movements toward the path that increases its progress score. It stops wasting time on dead ends.

4. The "World Model": The Robot's Imagination

To ask the GPS for advice, the robot needs to know what will happen next.

The robot has a World Model (a mental simulator).
Before it actually moves its arm, it imagines the move in its head.
It asks the GPS: "If I imagine moving my arm this way, will my progress score go up?"
If the answer is "Yes," it does the move. If the answer is "No," it tries a different imagined move.

This allows the robot to plan ahead without actually crashing into things or wasting time.

5. Real-World Results

The researchers tested this on real robots doing tasks like:

Opening a drawer.
Stacking bowls.
Putting fruit on a plate.

The results were impressive:

Faster: The robots finished tasks in fewer steps (like taking a shortcut instead of walking in circles).
Smarter: They stopped exactly when the job was done, instead of fumbling around afterwards.
Robust: Even when the lights changed or they used a different fruit (like an orange instead of an apple), the "Progress GPS" still worked because it had learned the concept of progress, not just the specific objects.

Summary

ProgressVLA is like giving a robot a comprehensive sense of "how far along" it is. Instead of just reacting to what it sees right now, it constantly checks its progress score and steers its actions to make that score go up. It turns a robot that blindly stumbles through a task into a focused worker that knows exactly how to get to the finish line efficiently.

1. Problem Statement

Current Vision-Language-Action (VLA) models for robotic manipulation face significant challenges in long-horizon tasks:

Lack of Progress Awareness: Most existing models rely on hand-crafted heuristics for task termination or implicit, noisy goal-satisfaction cues. They lack a dense, explicit understanding of how close the robot is to completing a task.
Inefficient Sampling: Generative planners (like diffusion policies) often produce trajectories that are visually plausible but task-irrelevant because they are driven solely by conditioning without a mechanism to steer toward goal completion.
Brittle Termination: Without a principled progress signal, policies often fail to know when to stop, leading to redundant motions or premature termination.
Generalization Issues: Learning progress directly from raw pixels is difficult due to nuisance variations (camera jitter, lighting, distractors), leading to noisy progress signals.

2. Methodology: ProgressVLA

The authors propose ProgressVLA, a framework that integrates a learned progress estimator directly into a diffusion policy via a differentiable pipeline. The system operates in a latent action space to ensure robustness and generalization.

Core Components

Progress Estimator (Vision-Language Evaluator):
- Function: A regressor that takes a language instruction ( $l$ ), an initial observation ( $o_0$ ), and a current observation ( $o_t$ ) to output a normalized scalar progress score $p \in [0, 1]$ .
- Training: Pre-trained on large-scale unsupervised video-text datasets (Open X-Embodiment) using normalized timestep as a proxy for ground-truth progress. It is fine-tuned on specific datasets (LIBERO, CALVIN) and real-world data.
- Architecture: Uses DINOv2 for visual features and CLIP for text, processed through a cross-attention stack to align language with visual state changes.
Action-Oriented World Model:
- Function: An Inverse Dynamics Model (IDM) and Forward Dynamics Model (FDM) operating in a latent visual feature space (not pixel space).
- Mechanism:
  - Inverse Dynamics: Encodes the transition between $o_t$ and $o_{t+N}$ into a latent action $a_z$ .
  - Forward Dynamics: Predicts future visual features $\hat{o}_{t+N}$ given current features and the latent action.
- Purpose: Allows the system to "imagine" the outcome of a candidate action chunk before executing it.
Progress-Guided Diffusion Policy:
- Two-Stage Generation:
  1. Latent Action Expert: A diffusion model generates a candidate chunk of latent actions ( $a_z$ ) representing high-level strategy.
  2. Action Decoder: Decodes $a_z$ into low-level robot commands.
- Classifier Guidance (CG): This is the core innovation. During the diffusion denoising process:
  - The world model predicts the future state $\hat{o}_{t+N}$ for a noisy latent sample $x_\tau$ .
  - The progress estimator evaluates this future state to get a predicted progress score $\hat{p}_{t+N}$ .
  - Gradients of the progress score are backpropagated through the world model to the latent action sample.
  - The update rule is modified: $x_{\tau-1} = \mu_\theta + s \nabla_{x_\tau} \hat{p}_{t+N} + \text{noise}$ .
- Result: The sampling process is steered toward action chunks predicted to maximize task progress.
RL Finetuning:
- The system is further refined using online experience.
- Monotonicity Loss: Penalizes the progress estimator if the predicted progress decreases over time in successful trajectories.
- KL-Regularized Policy Improvement: Adjusts the diffusion policy to increase the likelihood of actions that yield higher progress scores while staying close to the original distribution.

3. Key Contributions

Robust Progress Estimation: A pre-trained, vision-language conditioned progress estimator that achieves low prediction residuals (0.07 on CALVIN, 0.1 on real-world) and generalizes to unseen real-world scenarios via transfer learning.
Differentiable Progress Guidance: A novel pipeline that uses an inverse dynamics world model to map latent actions to future states, enabling the use of progress gradients as classifier guidance to steer diffusion sampling.
End-to-End Framework: Integration of progress estimation, world modeling, and diffusion policies into a unified system that improves both success rates and execution efficiency without requiring dense action labels for the progress signal.

4. Experimental Results

Benchmarks (Simulation)

CALVIN (Long-horizon manipulation):
- ProgressVLA (Full) achieved a 52.0% success rate for completing 5 subtasks in a row, outperforming the baseline Diffusion Policy (41.2%) and strong baselines like GR-MG (64.4% for 4-in-a-row, but lower for 5).
- Progress Alignment: Pearson correlation between predicted progress and linear ramp increased from 0.722 (no guidance) to 0.934 (with guidance).
- Efficiency: Average steps to completion reduced from 90.4 to 77.3.
LIBERO (Knowledge transfer):
- On the challenging LIBERO-LONG split, ProgressVLA (Full) achieved 66.2% success, significantly outperforming OpenVLA (53.7%) and Dita (63.8%).
- Overall average success rate across all subsets reached 84.5%.

Real-World Deployment

Setup: ARX AC-One dual-arm robot with RGB-D cameras.
Tasks: 5 tasks including "Pick up peach," "Open drawer," and "Stack bowls."
Results:
- Success Rate: Improved from 23% (Octo baseline) to 76% (ProgressVLA with guidance).
- Efficiency: Reduced average end-effector travel distance from 1.30m to 0.81m and steps from 187.9 to 53.3.
- Generalization: The pre-trained + finetuned progress estimator maintained high accuracy (Pearson > 0.95) under lighting shifts and novel objects, whereas models trained from scratch failed (Pearson ~0.81, Stop reliability ~37%).

5. Significance and Impact

Solving Long-Horizon Planning: ProgressVLA addresses the critical gap in long-horizon tasks where traditional VLA models struggle with termination and goal-directedness.
Data Efficiency: By leveraging large-scale pre-training for the progress estimator, the method achieves strong performance with minimal real-world fine-tuning data.
Reliable Termination: The explicit progress signal provides a principled, threshold-based termination criterion, eliminating the need for brittle, hand-crafted heuristics.
Generalization: The use of a latent action space and object-centric visual features allows the system to handle domain shifts (lighting, new objects) effectively, making it a viable candidate for real-world robotic deployment.

In summary, ProgressVLA demonstrates that explicitly modeling and guiding task progress within a generative diffusion framework significantly enhances the reliability, efficiency, and success rate of robotic manipulation in complex, long-horizon scenarios.