🤖 AI

SCALECUA: Scaling Computer Use Agents with Verifiable Task Synthesis and Efficient Online RL

ScaleCUA is a unified framework that advances open-source computer use agents to state-of-the-art performance by addressing data scarcity and training inefficiency through VeriGen for verifiable task synthesis, Frontier Sampling for optimized sample allocation, and Visual Context Segmentation for accelerated training.

Original authors: Bowen Lv, Xiao Liu, Yanyu Ren, Hanyu Lai, Bohao Jing, Hanchen Zhang, Yanxiao Zhao, Shuntian Yao, Jie Tang, Yuxiao Dong

Published 2026-07-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Bowen Lv, Xiao Liu, Yanyu Ren, Hanyu Lai, Bohao Jing, Hanchen Zhang, Yanxiao Zhao, Shuntian Yao, Jie Tang, Yuxiao Dong

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you're trying to teach a robot to use a computer just like a human does. The robot has to look at the screen, click buttons, type in terminals, and figure out how to install software or organize files. This is the world of Computer Use Agents (CUAs).

For a long time, teaching these robots has been like trying to learn a new video game without a scoreboard. You can play, but you don't know if you actually won until a human watches the whole thing and says, "Good job!" or "Try again." This makes learning super slow and expensive because you can't just run thousands of games at once if you need a human to grade every single one.

Enter SCALECUA, a new framework that acts like a super-charged, self-correcting training camp for these robot agents. It solves two big problems: finding enough practice tasks that have a clear "win" condition, and making the training process fast enough to actually work.

The Scoreboard Problem: Enter VERIGEN

The first hurdle was that most computer tasks don't have a built-in "Game Over, You Win" signal. To fix this, the researchers built VERIGEN.

Think of VERIGEN as a team of three robot coaches working in a giant, infinite playground of virtual computers (Docker containers).

The Proposer suggests a task, like "Install this specific theme for your computer."
The Judger checks if the instructions make sense and if the "winning condition" is logical.
The Checker actually tries to do the task in the virtual computer to see if it's possible and if the "winning condition" actually triggers.

If the task fails any of these checks, it gets tossed out. If it passes, it becomes a verified training task. The coolest part? This team of coaches works in parallel. They can run 100+ of these coaches at the same time, interacting with 100+ virtual computers simultaneously.

The result? They generated over 24,000 verified tasks and nearly 3,000 high-quality tasks specifically for training. This is a massive jump compared to previous methods, which often relied on tiny, manually written lists of tasks.

The Training Strategy: Frontier Sampling

Once you have a mountain of tasks, how do you pick which ones to practice? A common mistake is picking tasks randomly. But imagine if you were learning to ride a bike:

If you pick a task that's too easy (riding on flat ground), you learn nothing.
If you pick a task that's too hard (riding down a steep cliff), you crash and get discouraged.
The sweet spot is the "learning frontier"—the tasks you are just starting to figure out.

SCALECUA uses a smart strategy called Frontier Sampling. It keeps a running score of how well the robot is doing on every single task. It then focuses its practice time on the tasks where the robot is hovering right around a 50% success rate. This ensures the robot is always pushing its limits without getting stuck on impossible challenges or bored by easy ones.

The Speed Trick: Visual Context Segmentation

Here's the tricky part: Computer tasks can take a long time. A single task might involve 47 steps (like the example in the paper where the robot installs a theme). If you try to remember every screenshot from step 1 all the way to step 47, your brain (or the computer's memory) gets overwhelmed, and the training slows to a crawl.

The researchers solved this with Visual Context Segmentation. Imagine you are reading a long story, but instead of keeping the whole book open, you only keep the last few pages visible on your desk. You remember the plot from earlier pages as a summary, but you only look at the recent pages to make your next decision.

This technique keeps a "sliding window" of the most recent screenshots (about 5 to 8 images) while summarizing the older ones as text. This keeps the robot's memory fresh and fast. The paper found this made the training process 2.83 times faster than the old way of trying to remember every single step at once, without making the robot forget how to solve the problem.

The Results: Did It Work?

The team tested this new system on a model called Qwen3.5-9B (a 9-billion-parameter vision-language model).

On OSWorld (a benchmark for general desktop tasks), the robot achieved a success rate of 68.7%.
On ScienceBoard (a benchmark for complex scientific software), it hit 54.0%.

These numbers are significant. The paper notes that this 9-billion-parameter model beat other open-source models that were four times larger (like the 32-billion-parameter EvoCUA). It also outperformed some proprietary models, suggesting that the method of training matters just as much as the size of the robot's brain.

What This Doesn't Do

It's important to know what this paper doesn't claim.

It's not a magic wand for all computers yet. The tests were run on Ubuntu-based desktops (a specific type of Linux computer). The authors admit they haven't tested this on Windows or macOS yet, so we don't know if it works there.
It has a time limit. The training episodes were capped at 50 interaction turns. While this covers most tasks, the authors suggest it might not be enough for ultra-long, complex workflows that take hundreds of steps.
It's not a solved problem. The paper presents this as a new framework that establishes a "state-of-the-art" among open-source agents, but it acknowledges that generalization to other systems and scales is still future work.

In short, SCALECUA is a clever recipe for teaching robots to use computers by generating its own practice tests, focusing on the "just right" difficulty level, and remembering just enough of the past to stay fast. It shows that with the right training tools, even a smaller robot brain can outperform much larger ones.

Technical Summary: SCALECUA

Problem Statement

Computer Use Agents (CUAs) represent a transformative approach to automating digital workflows by operating directly on screen pixels and UI elements rather than structured APIs. While recent advancements in online reinforcement learning with verifiable rewards (RLVR) show promise for scaling CUA capabilities, the field faces two critical bottlenecks:

Verifiable Data Scarcity: Unlike math or code tasks, GUI tasks lack built-in mechanisms to score final states deterministically. Generating tasks that are both verifiable (via executable judges) and suitably difficult (neither trivial nor out-of-reach) at scale is fundamentally challenging.
Online RL Inefficiency: Current RL training for multi-turn GUI interactions suffers from inefficiencies in both sampling and training. Uniform task selection often yields rollout groups where all tasks either succeed or fail, providing no learning signal. Furthermore, training strategies face a trade-off: trajectory-level training accumulates excessive visual tokens, slowing inference, while step-wise decomposition inflates the number of training items, slowing parameter updates.

Methodology

The authors introduce SCALECUA, a unified framework designed to scale online RL for CUAs by addressing data generation and training efficiency simultaneously. The framework consists of three core components:

1. VERIGEN: Verifiable Task Synthesis

To overcome data scarcity, the authors propose VERIGEN, an end-to-end framework for generating verifiable RL tasks.

Multi-Agent Feedback Loop: VERIGEN employs three independent LLM agents (Proposer, Judger, and Checker) interacting with live Docker environments. The Proposer generates tasks and verifiers; the Judger performs static semantic reviews; the Checker executes "dry runs" to dynamically verify task reachability and judge correctness.
Trajectory-Guided Synthesis: The system iteratively refines the task pool by analyzing model rollout trajectories. Failed tasks are decomposed into smaller subtasks based on the earliest deviation point, while successful tasks are clustered and composed into harder multi-objective tasks.
Scalability: A shared Docker interaction probe allows 100+ concurrent agent workers to interact with 100+ Docker environments, producing over 24,000 verifiable tasks and nearly 3,000 high-quality RL tasks.

2. Frontier Sampling: Efficient Task Allocation

To maximize sample efficiency, SCALECUA introduces Frontier Sampling, a lightweight sampler that dynamically allocates rollouts based on the model's current capability.

Mechanism: The system maintains an Exponential Moving Average (EMA) of success rates for each task. It applies a Gaussian kernel distribution to compute sampling weights, peaking at a target success rate (e.g., 0.5).
Goal: This directs rollouts toward tasks at the "learning frontier"—those the model is currently capable of solving with some probability—avoiding wasted computation on tasks that are too easy or too hard.

3. Visual Context Segmentation: Efficient Multi-Turn Training

To resolve the rollout-training trade-off in long-horizon tasks, the authors propose Visual Context Segmentation.

Sliding Window Strategy: Instead of retaining the full visual history (which burdens the rollout engine) or splitting every turn (which burdens the training engine), the system maintains a sliding window of recent visual context (e.g., $K$ screenshots).
Token-ID Stream: When the window overflows, the oldest screenshots are dropped and replaced by a text summary, while the token-ID stream is re-tokenized. This bounds visual-token accumulation while preserving textual continuity and cross-step reasoning.

Key Contributions

VERIGEN Framework: An end-to-end pipeline that scales verifiable task generation to 100+ parallel workers, yielding a dataset of 24K+ verifiable tasks and nearly 3K high-quality RL tasks, significantly larger than prior desktop CUA methods.
Frontier Sampling: A novel sampling strategy that tracks per-task capability via EMA success rates, ensuring rollouts are concentrated on the most informative tasks to accelerate convergence.
Visual Context Segmentation: A training optimization technique that balances visual-token load and training item count, achieving a 2.83× end-to-end training speedup over step-wise decomposition without sacrificing reasoning quality.

Experimental Results

The authors evaluated SCALECUA on three vision-language models (GLM-4.6V-Flash, Qwen3-VL-8B-Thinking, and Qwen3.5-9B) using the OSWorld and ScienceBoard benchmarks.

OSWorld Performance: Using the Qwen3.5-9B backbone, SCALECUA achieved a 68.7% success rate. This surpasses the strongest open-source baselines, including Kimi K2.5 (63.3%) and EvoCUA-32B (56.7%), as well as proprietary models like Claude Sonnet 4.5 (62.9%). The model showed particularly strong gains in long-horizon "Professional" (89.5%) and "Workflow" (48.1%) categories.
ScienceBoard Performance: SCALECUA-Qwen3.5-9B reached 54.0% on ScienceBoard, establishing a new state-of-the-art among open-source agents and surpassing Claude Opus 4.6 (52.7%). Notable improvements were observed in scientific document authoring (TeXstudio) and theorem proving.
Ablation Studies: Removing any of the three core components (VERIGEN, Frontier Sampling, or Visual Context Segmentation) resulted in significant performance drops, confirming their complementary roles. Specifically, removing VERIGEN dropped performance to the base model level (43.9%).

Significance

SCALECUA demonstrates that scaling computer use agents requires a holistic approach that integrates verifiable data synthesis with efficient training algorithms. By solving the bottleneck of verifiable data scarcity through iterative, live-environment interaction and addressing training inefficiencies through frontier-aware sampling and visual context management, the framework establishes new state-of-the-art performance for open-source CUAs. The authors posit that this unified approach offers a reusable recipe for training agents with verifiable rewards, accelerating progress in the field of automated digital workflows.