Qwen-CUA: Native Computer Use for (almost) Everything
The paper introduces Qwen-CUA, a native computer-use agent built on a 397B mixture-of-experts backbone that operates solely through screenshots and input events without relying on DOM trees, achieving state-of-the-art performance on computer-use benchmarks through large-scale cloud rollout training, verifiable reward optimization, and scalable architecture.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine a world where your computer doesn't just wait for you to click buttons, but actually watches what you do and learns to do it for you. This is the exciting frontier of "AI agents," a branch of science where computers are taught to act like humans in the digital world. For a long time, these agents were like brilliant students who could only read textbooks (code) or follow strict instruction manuals (APIs). They were great at math and logic, but if you asked them to use a messy, old-fashioned program or a website that changes every time you refresh it, they got lost. They couldn't "see" the screen the way a human does; they needed special blueprints of the website's structure to know where the buttons were. But most of our digital lives happen on screens that don't have those blueprints. This paper tackles the challenge of teaching an AI to be a true digital apprentice: one that looks at a screen, figures out what to do, and clicks or types just like a person would, without needing any special shortcuts or hidden maps.
The team behind this research introduces Qwen-CUA, a new kind of AI agent designed to be a "native" computer user. Think of it like teaching a robot to drive a car. Instead of giving the robot a perfect digital map of every road and traffic light (which doesn't exist for many old or complex software programs), they taught the robot to look out the window, see the road, and turn the steering wheel based on what it sees. Qwen-CUA only "sees" screenshots of the computer screen and only "acts" by sending keyboard and mouse commands. It doesn't peek under the hood at the code or ask the software for help; it just watches and learns.
To make this robot smart enough to handle real-life tasks, the researchers had to build a massive training gym. They created a cloud-based fleet with nearly 100,000 virtual CPUs (imagine a city of thousands of computers working at once) and set up about 40,000 different tasks for the AI to practice. These tasks ranged from simple things like organizing files to complex, multi-step workflows in professional software. The AI didn't just get a "good job" or "try again" at the very end; the system was designed to verify if the task was actually done correctly, allowing the AI to learn from its mistakes over and over again.
The results are impressive. In tests across eight different benchmarks, Qwen-CUA consistently beat its previous version, Qwen3.7, and even held its own against some of the most powerful, expensive AI systems available today. For example, on a test called OSWorld-Verified, which measures how well an AI can handle everyday desktop tasks, Qwen-CUA scored 86.2, while the previous model scored 73.3. When they scaled this up to an even bigger model called Qwen-CUA-Max (with over one trillion parameters), the score climbed to 87.6.
One of the coolest things about this paper is how it handles long, complicated tasks. Imagine trying to remember a story you read a week ago while writing a new chapter. The AI had to remember what the screen looked like many steps ago to know where it was in its task. The researchers solved this by giving the AI a "visual memory" that holds the last 20 screenshots and cleverly "folds" older ones into a summary, so it doesn't get overwhelmed. This allowed the AI to keep its cool during long workflows without forgetting the beginning of the story.
The paper also looked at safety. They tested the AI against "RedTeamCUA," a challenge where the AI is tricked by fake instructions hidden in the screen. The new model was much better at ignoring these tricks, dropping the success rate of attacks from 36.6% down to 16.4%, while still getting its own tasks done.
Finally, the researchers explored a "hybrid" approach. They found that if they let the AI use a command-line tool (like a text-based magic wand for file operations) alongside its mouse and keyboard skills, it could finish tasks faster. However, the AI sometimes switched tools at the wrong time, so while it was faster, it wasn't always perfect yet. The authors suggest that with more training, this mix of visual watching and text-based commands could be the key to making AI agents truly efficient.
In short, this paper shows that we don't need special bridges to connect AI to our computers. By teaching the AI to simply look at the screen and act like a human, we can build agents that are ready to tackle almost any software, from the latest apps to the oldest programs, all while learning from massive amounts of real-world practice.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.