Efficient Agent Training for Computer Use

The paper introduces PC Agent-E, an efficient training framework that synthesizes diverse action decisions using Claude 3.7 Sonnet to augment a small set of 312 human trajectories, resulting in a model that significantly outperforms both human-only training and direct distillation on the WindowsAgentArena-V2 benchmark.

The Gist

Imagine you want to teach a robot to use a computer just like a human does—clicking buttons, typing in forms, and navigating menus. The biggest problem has always been that teaching a robot requires showing it thousands of examples of humans doing these tasks. It's like trying to teach a child to drive by only letting them watch one person drive for a few minutes; they won't get the hang of it.

This paper introduces PC Agent-E, a new way to train these computer-using robots that is incredibly efficient. Here is the story of how they did it, explained simply.

The Problem: The "Data Famine"

Usually, to make a smart AI, you need a massive library of "human demonstrations" (videos or logs of people using computers). But getting thousands of high-quality examples is expensive, slow, and hard to do. The authors started with a tiny library: just 312 examples recorded by two people in a single day.

The Solution: The "Smart Tutor" Method

Instead of just memorizing those 312 examples, the authors used a clever trick they call Trajectory Boost. Think of it like this:

1. The Raw Footage (Human Data): They recorded 312 times when humans successfully used a computer. This is the "trunk" of the tree.
2. The Inner Monologue (Thought Completion): Humans don't usually say out loud why they clicked a button. The AI took those raw recordings and used a super-smart AI (Claude 3.7 Sonnet) to write down the "inner thoughts" the human must have had. Now, the robot knows not just what the human did, but why.
3. The "What If" Game (Trajectory Boost): This is the magic step. The authors asked the super-smart AI: "Okay, we know the human clicked 'Save' here. But what are 9 other valid ways to solve this problem? Maybe they could have clicked 'File' then 'Save', or used a keyboard shortcut?"
   • The AI generated 9 alternative paths for every single step the human took.
   • Suddenly, those 312 examples turned into 27,000 training examples.

The Result: A Super-Student

They trained their robot (PC Agent-E) on this massive, enriched dataset. The results were shocking:

• The Base Model: A standard AI model (Qwen2.5-VL-72B) could only solve about 15% of the computer tasks.
• The New Robot: PC Agent-E solved 36% of the tasks.
• Beating the Teacher: Even more impressively, this small, open-source robot performed better than the giant, expensive "teacher" AI (Claude 3.7 Sonnet) that was used to generate the extra data.

Why This is a Big Deal

The authors compared their method to two other ways of training:

1. Just watching humans: This gave a tiny improvement.
2. Direct Distillation (Copying the Teacher): This is where you just ask the smart AI to do the whole task and copy its answers. This is slow, expensive, and prone to errors (like copying a mistake).

The PC Agent-E method was 300 times faster than the "Direct Distillation" method. Instead of asking the smart AI to actually do the tasks on a real computer (which takes hours), they just asked it to imagine the steps offline. It's the difference between hiring a master chef to cook 3,000 meals for you (expensive and slow) versus hiring them to write down 3,000 variations of a recipe (fast and cheap), and then teaching your apprentice from those recipes.

The New "Driving Test" (WindowsAgentArena-V2)

The authors also realized that the existing tests for these robots were flawed. Some tests were impossible to pass (like trying to use a feature that doesn't exist), and robots could "cheat" by just saying "I failed" to get a perfect score. They built a new, fairer test called WindowsAgentArena-V2 to ensure robots are actually learning to use computers, not just gaming the system.

The Bottom Line

You don't need a million human videos to teach a robot to use a computer. If you have a small, high-quality set of human examples and a smart AI to help you imagine all the different ways to solve a problem, you can train a robot that is smarter than the AI you used to teach it.

It's like teaching a student not just by showing them one way to solve a math problem, but by having a genius tutor explain the logic and then brainstorming ten different ways to solve it, ensuring the student truly understands the concept rather than just memorizing the answer.

Technical Summary

Here is a detailed technical summary of the paper "Efficient Agent Training for Computer Use" (PC Agent-E), published as a conference paper at ICLR 2026.

1. Problem Statement

The development of autonomous computer-use agents (agents that can operate Graphical User Interfaces like humans) is currently bottlenecked by the extreme scarcity of high-quality trajectory data.

• Data Scarcity: Collecting human-annotated computer-use trajectories is expensive, time-consuming, and difficult to scale.
• Performance Gap: Open-source models significantly lag behind proprietary leaders (e.g., Claude 3.7 Sonnet) in computer-use tasks.
• Inefficiency of Current Methods: Existing approaches either rely on small amounts of human data (limiting performance) or attempt to distill knowledge from large models via end-to-end trajectory generation, which is computationally expensive, prone to error accumulation, and slow.

2. Methodology: PC Agent-E Framework

The authors propose PC Agent-E, an efficient training framework that integrates human expertise with AI automation to maximize data efficiency. The framework consists of four key stages:

A. Trajectory Collection

• Data Source: The authors collected 312 real-world human computer-use trajectories using a tool called PC Tracker.
• Annotation: Two human annotators completed tasks on Windows machines in a single day.
• Quality Control: A rigorous decontamination process was applied, filtering out erroneous steps and ensuring no overlap with the evaluation benchmark tasks (using n-gram and semantic similarity metrics).
• Output: A dataset of 312 trajectories containing task descriptions, screenshots, and human keyboard/mouse actions.

B. Thought Completion

• Goal: Raw human trajectories often lack the implicit "thought process" (reasoning) behind an action.
• Process: Using Claude 3.7 Sonnet, the authors reconstructed the latent thought process for every step in the human trajectories.
• Input: Task description, history of actions/thoughts, current action, and screenshot.
• Output: Enhanced trajectories where every action is paired with a reconstructed reasoning step, creating a "Human Trajectory with Thoughts."

C. Trajectory Boost (Core Innovation)

This is the novel data synthesis method designed to diversify the training data.

• Motivation: Computer tasks can often be solved via multiple valid pathways. Relying solely on the single path taken by a human limits the agent's ability to learn alternative strategies.
• Mechanism:
  1. The system treats each step of the human trajectory as an environment snapshot (Task + State + History).
  2. Claude 3.7 Sonnet is prompted to generate diverse alternative action decisions for that specific snapshot, rather than re-generating the whole trajectory.
  3. The model samples 9 alternative action decisions per step.
• Result: A "Traj Tree" is formed where the human trajectory is the main trunk, and the AI-generated alternatives branch off as leaf nodes. This creates a rich dataset of 27,000 training samples from the original 312 trajectories.

D. Agent Training

• Model: The framework trains a native agent model based on Qwen2.5-VL-72B.
• Architecture: A simple end-to-end scaffold using the ReAct (Reasoning + Acting) paradigm.
• Input: Screenshot, task description, and history of thoughts/actions.
• Output: A thought process followed by a single action (e.g., click, type, hotkey).
• Training: Supervised Fine-Tuning (SFT) on the 27k augmented samples.

3. Key Contributions

1. Trajectory Boost Method: A data synthesis technique that augments a small set of human demonstrations with diverse, high-quality alternative actions generated by a frontier model. It proves significantly more effective than using human data alone or direct distillation.
2. WindowsAgentArena-V2: The authors released an improved benchmark to address limitations in the original WindowsAgentArena, specifically:
   • Evaluation Dependency: Implemented VM snapshot restoration to ensure independent and identically distributed (i.i.d.) task evaluation.
   • Infeasible Hacking: Removed "infeasible tasks" (tasks impossible to complete) that allowed weak models to "hack" the score by simply outputting a failure command.
   • Stability: Added validation frameworks to ensure VM initial states are stable and free of bugs.
3. PC Agent-E Model: An open-source computer-use agent that achieves state-of-the-art (SOTA) performance on Windows tasks with minimal data (312 augmented trajectories).

4. Experimental Results

The model was evaluated on WindowsAgentArena-V2 (141 tasks across 11 applications) and OSWorld (Linux).

• Performance vs. Base Model: PC Agent-E achieved a 141% relative improvement over the base Qwen2.5-VL-72B model.
• Performance vs. Proprietary Models: PC Agent-E surpassed Claude 3.7 Sonnet (the teacher model used for synthesis) by 10% in relative terms (36.0% success rate vs. 32.6%). It even matched the performance of Claude 3.7 Sonnet with "extended thinking" enabled.
• Data Efficiency:
  • vs. Human Data Only: Training on 312 human trajectories alone only improved the base model from 14.9% to 17.2%.
  • vs. Direct Distillation: Directly distilling 3,120 end-to-end trajectories from Claude took ~900 hours and yielded lower performance (26.2%) compared to PC Agent-E's 3-hour synthesis process (36.0%).
• Generalization: The model showed a 34% relative improvement on Linux (OSWorld) despite being trained exclusively on Windows data, demonstrating strong cross-platform generalization.
• Test-Time Scaling: Performance improved as the allowed step limit increased from 15 to 30, indicating enhanced planning capabilities.

5. Significance and Conclusion

• Paradigm Shift: The paper demonstrates that high-quality data synthesis (augmenting human data with diverse AI alternatives) is far more efficient and effective than simply scaling up human annotation or performing direct end-to-end distillation.
• Bridging the Gap: It successfully bridges the performance gap between open-source and proprietary computer-use agents, proving that open-source models can exceed proprietary counterparts with minimal human effort if the data augmentation strategy is correct.
• Practical Impact: The method reduces the cost of training computer-use agents from months of annotation to a single day of human work plus a few hours of offline AI synthesis.
• Benchmark Integrity: By releasing WindowsAgentArena-V2, the authors provide a more robust and fair evaluation standard for the community, eliminating "hacking" vulnerabilities present in previous benchmarks.

In summary, PC Agent-E establishes a new standard for efficient agent training, showing that combining a small amount of verified human expertise with the reasoning capabilities of frontier models to generate diverse alternatives is the key to unlocking high-performance, open-source computer automation.