Technical Summary: LEARNWEAK – Learn from Weaknesses for Automated Domain Specialization

Problem Statement

Computer-use agents (CUAs) have advanced significantly, yet a substantial performance gap remains between large proprietary models and small, open-source models fine-tuned for computer-use tasks. While small models offer advantages in cost, latency, and privacy for edge deployment, they struggle in domain-specific software environments (e.g., LibreOffice, VSCode) due to unique conventions and workflows.

Existing approaches to bridge this gap face two primary challenges:

1. Data Generation: Collecting human trajectories is costly. Automated trajectory generation methods often produce data that is "weakness-agnostic," failing to target the specific deficiencies of the student model, leading to inefficient training.
2. Training Objectives: Naive fine-tuning on broad datasets can cause catastrophic forgetting or distort the agent's native reasoning patterns. Furthermore, standard training often treats all errors uniformly, failing to distinguish between planning errors (incorrect action selection) and execution errors (incorrect parameters or coordinates), which require different corrective signals.

Methodology: LEARNWEAK

The authors propose LEARNWEAK, a fully automated, annotation-free framework for domain specialization that operates in two coupled stages: Weakness-Aware Data Generation (LEARNWEAK-GEN) and Error-Aware Agent Training (LEARNWEAK-DPO).

1. Weakness-Aware Data Generation (LEARNWEAK-GEN)

This stage constructs a targeted training dataset by iteratively identifying and addressing the student model's specific failures without human annotation.

• Teacher-Student Comparison: A stronger "teacher" policy ($\pi_T$) and the "student" policy ($\pi_S$) execute tasks in the same environment. A verifier identifies tasks where the teacher succeeds but the student fails ($v_T=1, v_S=0$).
• Weakness Summarization: The verifier's diagnostics for student failures are summarized into a weakness report ($R_d$), capturing recurring failure modes (e.g., incorrect element localization, invalid arguments).
• Screenshot-Guided Query Synthesis: A task generator synthesizes new queries for the next iteration based on:
  • Weakness-focused synthesis: Tasks conditioned on the weakness report to target identified deficiencies.
  • Exploration-focused synthesis: Tasks conditioned only on representative screenshots to ensure coverage of unexplored functionalities.
• Iterative Process: This cycle repeats for $N$ iterations, shifting the task distribution toward unresolved weaknesses while maintaining diversity.

2. Error-Aware Agent Training (LEARNWEAK-DPO)

This stage specializes the student using Direct Preference Optimization (DPO) with a novel error-aware masking strategy.

• Step-Level Preference Construction: Instead of treating entire trajectories as single units, the method replays the teacher's successful trajectory step-by-step. At each step, the student policy is queried with the teacher's context. Preference pairs are constructed only for steps where the student's tool execution differs from the teacher's.
• Error-Type Disentanglement: Failures are classified into two types:
  • Planning Errors ($\epsilon_{PLAN}$): The student chooses a different function type ($f_t$) than the teacher.
  • Execution Errors ($\epsilon_{EXEC}$): The student chooses the same function type but with incorrect parameters ($p_t$).
• Masked DPO Objective: The training loss applies a binary mask to the token positions. Crucially, for planning errors, the loss is applied to both reasoning and action tokens; for execution errors, it focuses on the action parameters. This ensures updates are behaviorally precise, targeting the specific error type rather than relearning the entire sequence.
• Modular Scalability: Domain specialization is achieved by freezing the base student and training lightweight LoRA adapters ($\Delta_d$) for each domain, allowing a single base model to serve multiple specialized agents.

Key Contributions

1. Annotation-Free Specialization Framework: LEARNWEAK eliminates the need for human trajectory annotation by using a teacher-student loop to automatically synthesize targeted training data.
2. Student-Aware Data Generation: Unlike prior autonomous generation methods that focus on volume or diversity, LEARNWEAK explicitly conditions query generation on the student's observed failure patterns, ensuring high informational value per sample.
3. Error-Aware Training Objective: The paper introduces a DPO variant that disentangles planning and execution errors, applying selective supervision masks to update only the relevant behavioral spans.
4. Modular Domain Adaptation: The framework demonstrates a scalable approach to handling multiple domains via LoRA adapters, avoiding the need for separate full-model fine-tunes.

Experimental Results

The framework was evaluated on 8 domains within the OSWorld benchmark using EvoCUA-8B and OpenCUA-7B as base student models.

• Performance Gains: LEARNWEAK achieved average success rate improvements of 11.6% (EvoCUA-8B) and 11.1% (OpenCUA-7B) across all domains.
• Surpassing the Teacher: In several domains (Gimp, Thunderbird, VSCode), the specialized small student outperformed the 32B teacher model, suggesting that targeted corrective supervision can be more effective than simple imitation of a larger model.
• Comparison with Baselines:
  • LEARNWEAK outperformed existing human-validated datasets (AgentNet) and zero-shot baselines.
  • It significantly outperformed other zero-annotation generation pipelines (AgentSynth, OS-Genesis, ZeroGUI) and filtering methods (WebSTAR), confirming that targeting weaknesses is superior to generic data expansion.
  • Ablation studies showed that removing the weakness report or the iterative generation process resulted in lower performance, validating the necessity of student-aware guidance.
  • The error-aware DPO objective outperformed standard SFT and standard DPO by a significant margin (9.62 percentage points on average).

Significance and Claims

The paper argues that efficient domain specialization for small CUAs depends not on scaling synthetic data volume, but on identifying and repairing student-specific weaknesses. By shifting the focus from broad generalization to targeted correction, LEARNWEAK provides a principled path to narrowing the performance gap between small open models and large proprietary agents.

The authors claim that this approach makes small, specialized agents a more practical deployment option for real-world software environments, particularly where privacy and cost are constraints. The work highlights the importance of "student-awareness" in both data synthesis and training objectives, suggesting that future research should prioritize adaptive, failure-driven learning over static dataset scaling.

Limitations Acknowledged by the Authors:

• The framework assumes the availability of a reasonably reliable teacher model; if the teacher is unstable or biased, the supervision may inherit these errors.
• It assumes the student base model already possesses foundational computer-use skills (visual grounding, action generation). It is not designed to teach these capabilities from scratch to models that lack them.