🤖 AI

Recurrent Reasoning with Vision-Language Models for Estimating Long-Horizon Embodied Task Progress

The paper proposes $\text{R}^2$ VLM, a recurrent reasoning framework that iteratively processes local video snippets while maintaining an evolving Chain of Thought to efficiently and accurately estimate long-horizon embodied task progress, achieving state-of-the-art performance without the computational cost of processing full video trajectories.

Original authors: Yuelin Zhang, Sijie Cheng, Chen Li, Zongzhao Li, Yuxin Huang, Yang Liu, Wenbing Huang

Published 2026-03-19

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yuelin Zhang, Sijie Cheng, Chen Li, Zongzhao Li, Yuxin Huang, Yang Liu, Wenbing Huang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to cook a complex meal, like "Make a cup of coffee." The robot doesn't just need to see the video; it needs to understand where it is in the process. Is it done? Is it halfway? Did it forget to get the milk?

This paper introduces a new AI system called R2VLM (Recurrent Reasoning Vision-Language Model) that acts like a super-smart, patient cooking instructor sitting next to the robot. Its job is to constantly answer the question: "How much of the task have we actually finished?"

Here is how it works, broken down into simple concepts:

1. The Problem: The "Long Video" Nightmare

Imagine trying to watch a 2-hour movie to understand what happened in the first 5 minutes. If you tried to re-watch the entire movie every time a new scene happened, you would get exhausted, and your brain would get confused.

Old AI methods tried to do exactly this. They looked at the entire video history every single second to guess the progress. This is slow, expensive, and the AI often gets "hallucinations" (confused about what happened earlier) because the video is too long.
The Challenge: Real-world tasks (like cleaning a house or cooking) take a long time and have many steps. If the AI forgets that it already picked up the coffee mug, it might try to pick it up again, or worse, think it's done when it hasn't even started.

2. The Solution: The "Running Notebook" (Recurrent Reasoning)

Instead of re-watching the whole movie, R2VLM uses a clever trick: The Running Notebook.

Think of the AI as a detective solving a mystery.

The Old Way: Every time a new clue arrives, the detective re-reads the entire 500-page case file from page 1.
The R2VLM Way: The detective keeps a notebook (called a "Chain of Thought" or CoT).
1. The detective looks at the new 2-second video clip (the latest clue).
2. They open their notebook, which already says: "I found the coffee beans, but I haven't ground them yet."
3. They look at the new clip, see the robot grinding the beans, and update the notebook: "Okay, beans are ground. Now I need to get the water."
4. They write down the new progress percentage (e.g., "40% done").

By only looking at the newest clip and the notebook, the AI stays fast, doesn't get confused, and remembers the big picture perfectly.

3. The Training: "The Distractor Game"

To teach this AI to be smart, the researchers didn't just show it videos of people doing tasks. They played a tricky game with it.

Imagine you are teaching a student to identify a "Red Car." If you only show them red cars, they might just guess "Red" without looking at the shape.

The Trick: The researchers showed the AI a video of someone making coffee, but told it, "This is a video of someone making a sandwich."
The Result: The AI had to look closely. It realized, "Wait, they are pouring coffee, not putting cheese on bread. The task description doesn't match the video!"
This forced the AI to stop guessing based on shortcuts and actually reason about the steps, ensuring it truly understands the task structure.

4. Why This Matters: The "Coach" for Robots

This isn't just about guessing numbers; it's about helping robots learn and help us. The paper shows three cool ways this "Coach" can be used:

The Personal Trainer (Policy Learning): If a robot is learning to walk or cook, the Coach tells it, "You're doing great, you've finished 3 out of 5 steps!" This helps the robot learn faster than if it just got a "Yes/No" at the very end.
The Referee (Reward Modeling): In video games, you get points for good moves. In the real world, robots don't have points. R2VLM acts as the referee, giving the robot a "score" (reward) every time it completes a small step, guiding it to do the right thing.
The Helpful Assistant (Proactive Help): Imagine you are an elderly person trying to assemble furniture. The AI watches you via a camera. It sees you struggling with a screw and says, "You've finished the legs, but you haven't attached the top yet. Here is the next step." It doesn't just watch; it understands the flow of your work.

Summary

R2VLM is like a smart, attentive assistant that keeps a running diary of what a robot is doing. Instead of getting overwhelmed by long videos, it updates its diary step-by-step. This allows it to accurately tell us how much of a task is done, helping robots learn faster, make fewer mistakes, and assist humans more effectively in our daily lives.

1. Problem Statement

Accurately estimating the progress of long-horizon, multi-step embodied tasks (e.g., "make a cup of coffee" or "clean the kitchen") is critical for agents to plan effectively, recover from errors, and interact proactively with humans. However, existing approaches face two major challenges:

Reasoning Limitations: Current Vision-Language Models (VLMs) primarily rely on video understanding and large context windows but underutilize their complex reasoning potential. They often fail to align visual observations with complex temporal dependencies between subtasks.
Computational Inefficiency: Processing full, long video trajectories with VLMs is computationally prohibitive for real-time, real-world deployment due to the high cost of processing massive numbers of visual tokens and context length limits.

2. Methodology: R2VLM

The authors propose R2VLM (Recurrent Reasoning Vision-Language Model), a framework designed to estimate task progress by combining iterative local video processing with a global reasoning memory.

Core Architecture

Recurrent Reasoning Framework: Instead of feeding the entire video history into the model at once, R2VLM processes the video as a sequence of short, local snippets (e.g., 2–4 seconds).
Evolving Chain of Thought (CoT): The model maintains a global context through an evolving CoT. At each time step $t$ , the model takes:
1. The current video snippet ( $v_t$ ).
2. The historical CoT from the previous iteration ( $c_{t-1}$ ).
3. The task description ( $\tau$ ).
It then updates the CoT ( $c_t$ ) and outputs the current progress estimate ( $p_t$ ).

The CoT Mechanism

The CoT serves as a form of historical memory and explicitly records:

Task Decomposition: Breaking down the long-horizon task into key subtasks.
Status Tracking: Identifying which steps are completed, pending, or incomplete based on visual evidence.
Dynamic Adjustment: The model can merge, split, or reorder steps if the initial decomposition conflicts with actual visual observations (e.g., if a user skips a step or performs them out of order).
Progress Calculation: The final progress is estimated based on the proportion of completed steps relative to the total required steps.

Training Strategy

The model is trained on large-scale, automatically generated datasets from ALFRED (simulation) and Ego4D (real-world). The training involves two stages:

Supervised Fine-Tuning (SFT): The model learns to generate structured CoTs and progress estimates using high-quality data generated by an automated pipeline. This pipeline creates "distractor" task descriptions to prevent the model from relying on superficial visual cues.
Reinforcement Learning (RL): Using Proximal Policy Optimization (PPO), the model is further refined with a multi-component reward function:
- Format Reward: Ensures output follows the CoT + Answer structure.
- Bin Reward: Checks if the progress falls within the correct interval based on the number of completed steps.
- MAE Reward: Encourages precise numerical accuracy.
- Improvement Reward: Specifically designed for multi-turn RL, it rewards the model for reducing the error compared to the previous turn, fostering self-correction.
- Finish Reward: Ensures accurate detection of task completion (100%).

3. Key Contributions

Novel Recurrent Reasoning Paradigm: R2VLM is the first to apply a recurrent CoT framework to embodied task progress estimation, allowing the model to maintain global context while processing only local video snippets. This avoids the computational overhead of full-video processing.
Automated Dataset Construction: The authors developed a pipeline to generate large-scale, multi-turn training data with CoT annotations and distractor tasks from existing datasets (ALFRED and Ego4D), addressing the lack of high-quality reasoning data for this specific task.
Comprehensive Reward Design: The introduction of the "Improvement Reward" and "Bin Reward" in the RL stage effectively guides the model to refine its reasoning iteratively and understand step-wise completion.
Versatile Downstream Applications: The model is not just a progress estimator but a general-purpose tool for:
- Progress-Enhanced Policy Learning: Improving agent performance by providing semantic progress signals (superior to time-based signals).
- Reward Modeling: Generating dense, accurate rewards for online reinforcement learning, solving the sparsity problem in long-horizon tasks.
- Proactive Assistance: Enabling real-time, step-wise tracking for human-in-the-loop guidance.

4. Experimental Results

The model was evaluated on ALFRED and Ego4D benchmarks against strong baselines, including closed-source APIs (GPT-5, Gemini-2.5-Pro) and open-source VLMs (Qwen2.5-VL, InternVL3).

State-of-the-Art Performance: R2VLM achieved the best results across all four metrics:
- pMAE (Progress Mean Absolute Error): Significantly lower than baselines (e.g., 2.19 on ALFRED vs. 27.87 for the base Qwen2.5-VL-7B).
- $\Delta$ pMAE: High accuracy in predicting progress increments (crucial for reward modeling).
- Bin Accuracy: High accuracy in identifying the correct completion stage.
- Acc: High accuracy in detecting full task completion.
Efficiency: By using video snippets + CoT, R2VLM maintains constant inference time regardless of video length, whereas full-video baselines suffer from linearly increasing latency and attention dispersion.
Generalization: The model showed strong cross-domain generalization, performing well when trained on one dataset and tested on another, indicating it learned reasoning strategies rather than dataset-specific patterns.
Ablation Studies: Removing the "Improvement Reward" caused a significant drop in performance, confirming its necessity for iterative reasoning refinement.

5. Significance

This work bridges the gap between the reasoning capabilities of large VLMs and the practical constraints of embodied AI. By decoupling global context (via CoT) from local visual processing (via snippets), R2VLM offers a scalable, efficient, and highly accurate solution for long-horizon task monitoring. Its ability to generate dense, semantic rewards makes it a foundational component for training more capable embodied agents via reinforcement learning, moving beyond sparse, binary success signals. The public release of models and benchmarks further accelerates research in embodied AI progress estimation.