Technical Summary: UI-MOPD

Problem Statement

The paper addresses the challenge of training unified multi-platform GUI agents capable of operating across heterogeneous environments (specifically desktop and mobile) while preserving the distinct interaction conventions of each platform.

Current approaches to cross-platform learning typically rely on:

1. Data Aggregation: Mixing heterogeneous supervision signals from different platforms.
2. Model Merging: Averaging parameters of specialized models.
3. Sequential Training: Training on one platform then the other.

The authors argue that these methods often fail because desktop and mobile platforms, while sharing foundational capabilities (visual understanding, planning), possess conflicting action semantics and interaction conventions (e.g., "back" navigation differs between OS). Mixing supervision or averaging parameters tends to blur native behaviors, leading to capability conflation, imbalanced performance, and the degradation of platform-specific priors. Furthermore, high-quality, executable cross-platform interaction data has been scarce, hindering the development of robust unified agents.

Methodology: UI-MOPD

To address these challenges, the authors propose UI-MOPD (Multi-Platform On-Policy Distillation), a framework that integrates specialist knowledge without averaging distinct interaction conventions. The approach consists of two main stages:

1. Data Construction: Uni-GUI

The authors first constructed Uni-GUI, a high-quality dataset containing nearly 10,000 executable cross-platform interaction trajectories.

• Collection: Built using a unified desktop-mobile harness, collecting ~110K computer and ~50K mobile interaction steps.
• Processing: Rigorous filtering, trajectory validation, and quality control were applied to ensure executable actions and accurate state-action alignment.

2. Training Framework: Multi-Teacher On-Policy Distillation (MOPD)

The training pipeline involves two stages:

• Stage 1 (Teacher Training): Supervised Fine-Tuning (SFT) of a vision-language foundation model (Qwen3-VL-32B) on the platform-specific partitions of Uni-GUI to create two expert teachers: a Desktop Teacher ($\pi^d_{ref}$) and a Mobile Teacher ($\pi^m_{ref}$).
• Stage 2 (Student Training): A shared student policy ($\pi_\theta$, initialized from an 8B model) is trained using Multi-Teacher On-Policy Distillation.
  • On-Policy Rollouts: The student samples rollouts from its current policy in mixed platform environments.
  • Platform-Conditioned Routing: Each rollout is routed to the corresponding platform-specialized teacher based on its environment label.
  • Behavioral Anchoring: The teacher provides guidance only at states actually visited by the student. This acts as a platform-conditioned behavioral anchor, preventing the student from overwriting native interaction patterns with signals from the other platform.
  • Loss Function: The training objective combines clipped policy optimization (PPO-style) with a Reverse KL divergence penalty. The KL term is computed between the student and the routed teacher, not a unified teacher.
  • Adaptive KL Masking: A group-level mask ($\mu$) is applied to remove teacher penalties when the task feedback (reward) is already sufficient, preventing unnecessary restriction of exploration on high-performing rollouts.

Key Contributions

1. Uni-GUI Dataset: The introduction of a high-quality, unified dataset of ~10K cross-platform interaction trajectories, addressing the scarcity of executable multi-platform data.
2. UI-MOPD Framework: The first application of Multi-Teacher On-Policy Distillation to unified GUI agent training. It uniquely integrates complementary desktop and mobile expertise by using platform-conditioned teachers as behavioral anchors rather than averaging them.
3. Systematic Evaluation: Comprehensive experiments demonstrating that UI-MOPD achieves balanced cross-platform performance, outperforming parameter-matched integration strategies (mixed SFT/RL, model merging, sequential training) while preserving general GUI grounding capabilities.

Experimental Results

The authors evaluated UI-MOPD on OSWorld (desktop) and MobileWorld (mobile) benchmarks, using an 8B parameter student model.

• Performance:
  • OSWorld: Achieved a 38.2% task success rate (a 12.7% relative improvement over the base model).
  • MobileWorld: Achieved a 12.0% task success rate (a 55.8% relative improvement over the base model).
  • Balance: UI-MOPD achieved the highest Harmonic Mean (H-Mean) of 18.3% among all 8B integration strategies, matching the performance of a much larger 235B parameter model in terms of balanced capability.
• Comparisons:
  • Outperformed strategies like Mixed-SFT, Mixed-RL, Model Merging (Weight Averaging/TIES), and Sequential-RL.
  • Demonstrated that using a single "Unified Teacher" (trained on mixed data) or single-platform teachers was insufficient compared to the routed multi-teacher approach.
• Generalization:
  • The method preserved general GUI grounding capabilities, showing improvements or negligible degradation on static benchmarks like AndroidControl, ScreenSpot-Pro, and OSWorld-G, whereas other integration strategies often caused broader degradation.

Significance and Claims

The paper claims that Multi-Teacher On-Policy Distillation provides an effective approach to building unified cross-platform GUI agents. The core significance lies in the finding that:

• Specialist Knowledge Integration: Platform-specific teachers provide complementary, not redundant, supervision. They act as stable anchors that allow a shared policy to improve task completion without erasing platform-specific behavioral priors.
• Resolution of Conflict: By routing rollouts to specific teachers, the method resolves the conflict between shared capabilities and divergent interaction conventions, avoiding the "averaging" effect that plagues other integration methods.
• Efficiency: The approach enables a single 8B model to achieve balanced performance across platforms that typically requires either much larger models or separate specialized agents, without sacrificing general GUI understanding.

The authors conclude that this framework establishes a new paradigm for training unified agents in heterogeneous environments, moving beyond simple data mixing toward structured, condition-aware knowledge integration.