Technical Summary: ReVision – Scaling Computer-Use Agents via Temporal Visual Redundancy Reduction

Problem Statement

Computer-use agents (CUAs) rely on multimodal large language models (MLLMs) to interact with graphical user interfaces (GUIs) by processing sequences of screenshots. A critical bottleneck in scaling these agents to long-horizon tasks is the rapid increase in visual token costs. As interaction trajectories grow, appending past screenshots to the model context introduces hundreds or thousands of visual tokens per image.

The authors identify that consecutive GUI screenshots exhibit substantial temporal redundancy. In most steps, only small regions of the interface change (e.g., a button click or text update), while the majority of the screen remains static. Standard MLLMs process each image independently, encoding nearly identical visual tokens repeatedly. This inefficiency exhausts context budgets, forcing agents to discard useful historical information or limiting the length of trajectories they can process. Consequently, performance often saturates or even degrades when adding more history, not because past information is useless, but because the token representation is inefficient.

Methodology: ReVision

To address this, the authors introduce ReVision, a redundancy-aware training framework designed to reduce unnecessary visual tokens while preserving the spatial structure and task-relevant information required for reasoning. The approach consists of two primary components:

1. ReVision Token Selection (RTS)

At the core of the framework is a lightweight, learned patch selector (a three-layer MLP) trained to identify redundant visual patches across consecutive screenshots.

• Mechanism: For a trajectory step $t$, the RTS module compares the patch-level embeddings of the current screenshot ($I_t$) with the corresponding patches of the previous screenshot ($I_{t-1}$).
• Training Supervision: The model is trained using labels generated by OmniParserV2, which segments screenshots into semantic regions. Patches are labeled as redundant if their corresponding regions in consecutive images have high Intersection over Union (IoU), indicating unchanged content. This region-level matching is preferred over raw pixel similarity to handle minor variations like rendering noise or cursor movement.
• Output: The RTS produces a binary mask $m_t$ for each step, filtering out patches that are visually redundant relative to the previous frame.

2. Training with Filtered Trajectories

Rather than treating token reduction as a post-processing step, ReVision integrates the filtering mechanism directly into the training pipeline.

• Data Construction: The model is fine-tuned on AgentNet trajectories where redundant patches are removed from all images in the history window except the first one (which is kept intact to preserve spatial alignment).
• Inference Consistency: During inference, the same RTS filtering is applied. The model conditions on the full textual context (reasoning and actions from previous steps) but receives a compressed set of visual tokens $\{V'_1, \dots, V'_t\}$.
• Spatial Preservation: Crucially, the filtering process preserves the original positional indices of the retained tokens. This ensures that the model's attention mechanisms and positional encodings (e.g., m-ROPE) remain valid, allowing the model to reason over the compact visual history without losing spatial coherence.

Key Contributions

The paper outlines three main contributions:

1. Quantification of Redundancy: The authors demonstrate that a significant fraction of visual tokens in GUI trajectories are redundant. Across multiple benchmarks (OSWorld, AgentNetBench, WebTailBench, etc.), an average of 45.4% of patches remain unchanged between consecutive steps, with some environments showing over 56% redundancy.
2. The ReVision Framework: They introduce a method to train MLLMs on trajectories where redundant patches are dynamically removed. This enables the model to reason over compact visual histories without modifying the underlying architecture, relying instead on temporally distributed evidence.
3. Re-evaluation of History Scaling: The work challenges the observation that visual history performance saturates early. The authors show that saturation is a consequence of inefficient token representations; when redundancy is removed, performance continues to improve as more past observations are incorporated.

Experimental Results

The authors evaluated ReVision using Qwen2.5-VL-7B across three benchmarks: OSWorld, WebTailBench, and AgentNetBench.

• Token Efficiency: When processing trajectories with 5 history screenshots, ReVision reduces token usage by approximately 46% on average compared to a baseline that retains all patches.
• Performance Gains: Despite the reduced token count, ReVision improves the success rate by 3% over the "no-drop" baseline. In specific benchmarks like WebTailBench, the improvement reaches up to 14 percentage points.
• Scaling Behavior:
  • Baseline: Standard models show diminishing returns or performance saturation as history length increases (e.g., peaking around 7 images) due to context budget exhaustion.
  • ReVision: By removing redundant tokens, the saturation point is delayed (up to 11 images), allowing the model to utilize longer histories effectively within the same compute budget.
• Trajectory Length: ReVision agents solve tasks more efficiently, requiring fewer interaction steps to complete tasks. For instance, on OSWorld, the average trajectory length was reduced by 4 steps while achieving higher success rates.
• Comparison to Other Strategies: ReVision outperforms naive token dropping strategies (random, spiral) and simple similarity-based filtering (pixel or embedding cosine similarity). While region-based methods using OmniParser at inference time yield slightly higher performance, they incur prohibitive latency (>550ms). ReVision achieves comparable gains with low latency (~22ms).
• Generalization: The method generalizes across different model families (Qwen2.5-VL-7B and Qwen3-VL-8B) and does not degrade performance in single-image settings, confirming that the training distribution shift does not harm standard grounding abilities.

Significance and Claims

The paper posits that the primary barrier to scaling computer-use agents is not the lack of historical information, but the inefficiency of visual token representations. By explicitly modeling and removing temporal redundancy, ReVision shifts the efficiency frontier, enabling agents to process longer, more informative trajectories without increasing computational costs.

The authors conclude that the commonly observed saturation in visual history performance is an artifact of context budget constraints rather than a limitation of the information's utility. By freeing up context budget through redundancy reduction, agents can incorporate longer histories, leading to better long-horizon reasoning and decision-making. The work suggests that future scaling of visual reasoning should focus on preserving useful information within limited budgets rather than simply accumulating more raw data.