🤖 AI

Beyond Success and Failure: Length-Aware Contrastive Learning for GUI Agents

This paper proposes LACL-GUI, a length-aware contrastive learning framework that enhances GUI agent training by incorporating fine-grained trajectory-level quality signals to overcome the reward-gradient misalignment and outcome-level supervision limitations of existing methods.

Original authors: Chengyang Gu, Le Zhang, Jingbo Zhou, Yize Chen, Yu Shi, Siqi Bao, Zheng-Fan Wu, Hua Wu, Hui Xiong

Published 2026-08-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Chengyang Gu, Le Zhang, Jingbo Zhou, Yize Chen, Yu Shi, Siqi Bao, Zheng-Fan Wu, Hua Wu, Hui Xiong

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the digital world, computers are increasingly being asked to do more than just follow simple commands; they are being asked to act as autonomous assistants that can navigate complex software, click the right buttons, and fill out forms just as a human would. These digital workers, known as GUI agents, rely on advanced artificial intelligence models that can see a computer screen and understand what is happening on it. For years, the most effective way to teach these agents has been through a method called reinforcement learning. In this process, the agent tries to complete a task, receives a simple "yes" or "no" signal at the very end indicating whether it succeeded or failed, and then adjusts its behavior to try again. While this trial-and-error approach has produced impressive results, researchers have discovered a flaw in how the computer learns from those final yes-or-no answers. The standard methods often get confused about which specific actions led to the success or the failure, causing the learning process to become unstable or inefficient. It is a bit like a student trying to learn a complex skill from a teacher who only says "good job" or "try again" at the very end of a long exercise, without pointing out exactly which steps were helpful and which were wasteful.

A team of researchers from Hong Kong, Beijing, and Edmonton has developed a new approach to fix this problem, called Length-Aware Contrastive Learning for GUI Agents, or LACL-GUI. Instead of treating every successful attempt as equally good and every failed attempt as equally bad, their method looks closer at the details of how the task was performed. They realized that not all successes are created equal: one agent might solve a problem in five quick, precise clicks, while another might take twenty steps, wandering around the screen and making unnecessary moves before finally getting it right. Similarly, a failure is not always a total disaster; one agent might get stuck on the very first step, while another might follow the correct path for most of the way before making a single mistake at the end. The researchers' new system teaches the agent to prefer the shorter, more efficient successful paths and to recognize that the "near-miss" failures are actually more valuable learning opportunities than the ones that went wrong immediately.

To achieve this, the researchers built a system that compares different attempts at the same task side by side. When the agent tries to complete a task, the system collects a group of attempts, some that worked and some that didn't. For the successful attempts, the system identifies the shortest one and uses it as a reference. It then gently encourages the agent to be more like that efficient example, rewarding it for taking fewer steps and penalizing it for redundant movements. For the failed attempts, the system looks at how far the agent got before it went off track. If an agent followed the correct path for a long time before making a mistake, the system treats that as a "near-miss" and gives it a softer penalty, recognizing that it learned something useful. If an agent went off track immediately, it receives a stronger penalty. This creates a much more nuanced learning signal than the simple pass-or-fail grading used in previous methods. The system also maintains a special memory bank of successful attempts, allowing it to compare new failures against the best examples of success it has seen so far, ensuring the agent always has a clear target to aim for.

The team tested this new method on a comprehensive benchmark called OSWorld, which involves a wide variety of real-world computer tasks, such as editing documents, managing email, and using image editing software. They used powerful AI models as the backbone for their agents and trained them to solve hundreds of different tasks. The results showed that the new method consistently outperformed the previous best techniques. On the larger AI models, the new approach improved the success rate by nearly three percentage points compared to the standard methods, a significant gain in this field. More importantly, the agents trained with this new method didn't just succeed more often; they also became more efficient. When they did succeed, they took fewer steps to get there, cutting down on unnecessary actions. When they failed, the system was better at distinguishing between a total loss and a near-win, allowing the agent to learn more effectively from its mistakes.

This work suggests that the key to building better digital assistants lies in looking beyond the final outcome. By teaching the AI to appreciate the quality of the journey, not just the destination, the researchers have created a more stable and effective way for these agents to learn. The method does not require the agent to have a human teacher watching every move; it simply uses the structure of the attempts themselves to guide the learning. This approach could lead to more reliable and efficient AI agents in the future, capable of handling complex digital tasks with a level of precision and economy that was previously difficult to achieve. The findings indicate that even in a world of binary success and failure, there is a rich landscape of quality and nuance that, if properly understood, can dramatically improve how machines learn to act.

Technical Summary: Length-Aware Contrastive Learning for GUI Agents (LACL-GUI)

1. Problem Statement

Graphical User Interface (GUI) agents powered by Multimodal Large Language Models (MLLMs) aim to automate tasks across diverse digital environments. While Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a dominant training paradigm, existing methods face significant limitations in optimization efficiency and stability.

Specifically, widely used algorithms like Group Relative Policy Optimization (GRPO) suffer from reward–gradient misalignment. Recent theoretical analyses (e.g., Zhai et al., 2026) identify two gradient pathologies in GRPO:

Gradient Misassignment in Positives: High-probability tokens in successful trajectories receive disproportionately large updates, while low-probability tokens in the same successful trajectories receive weak gradients.
Gradient Domination in Negatives: Gradient magnitudes concentrate excessively on a small number of high-probability negative tokens, allowing them to dominate the optimization process.

To mitigate these issues, recent works (e.g., REAL, CLIPO) have reformulated RLVR as contrastive or classification-based objectives. However, these methods typically rely on outcome-level supervision (binary success/failure labels). This coarse-grained approach treats all trajectories within the same outcome category as equivalent, failing to capture fine-grained differences in:

Efficiency: Successful trajectories may vary significantly in length and redundancy.
Failure Quality: Failed trajectories may differ in how close they come to task completion (e.g., diverging early vs. diverging late).

The core challenge addressed by this paper is how to incorporate trajectory-level quality signals (specifically length and divergence) into contrastive RLVR frameworks without compromising the optimization stability provided by classification-based objectives.

2. Methodology: LACL-GUI

The authors propose Length-Aware Contrastive Learning for GUI Agents (LACL-GUI), a framework that introduces structured preferences within both successful and failed trajectory groups while preserving the underlying success–failure decision boundary.

2.1 Architecture

LACL-GUI employs an asynchronous training architecture consisting of four modules:

Rollout Service: Collects trajectories from multiple GUI environments in parallel.
Data Manager: Filters trajectory groups, discarding those where all samples share the same outcome (all success or all failure) to ensure discriminative supervision.
Success Buffer: Maintains a repository of successful trajectories to serve as reference samples for divergence estimation and to augment training when current rollouts yield few successes.
Trainer: Asynchronously optimizes the policy using filtered trajectory groups, decoupling data collection from policy updates.

2.2 The LACL-GUI Objective

Building upon the REAL (Rewards as Labels) objective, which formulates RLVR as a binary classification problem using a contrastive loss, LACL-GUI introduces logit perturbations to encode fine-grained preferences.

Positive Group Perturbation (Efficiency Preference)

For successful trajectories ( $T^+$ ), the framework encourages concise executions. It computes a weighted mean trajectory length ( $\mu_w$ ) based on current logits and perturbs the logits ( $\tilde{s}_\theta$ ) as follows:
$\tilde{s}_\theta(\tau^j) = \bar{s}_\theta(\tau^j) + \epsilon (|\tau^j| - \mu_w)$
where $\epsilon$ is a perturbation coefficient. This adjustment down-weights longer successful trajectories (likely containing redundant actions) and up-weights shorter ones, without altering the binary success label.

Negative Group Perturbation (Divergence-Aware Refinement)

For failed trajectories ( $T^-$ ), the framework distinguishes between "informative" near-success failures and fundamentally incorrect attempts.

Reference Selection: A reference successful trajectory ( $\tau^+_{ref}$ ) is selected (typically the shortest successful trajectory from the buffer or current batch).
Divergence Estimation: The system compares the failed trajectory step-by-step with the reference using Structural Similarity Index (SSIM) to identify the divergence length ( $d_j$ ), defined as the number of steps before the trajectory deviates significantly from the successful path.
Error Severity: The number of erroneous steps is calculated as $E_j = |\tau^j| - d_j + 1$ .
Logit Perturbation: Similar to the positive group, the logits are perturbed based on the deviation of erroneous steps ( $E_j$ ) from the weighted mean ( $\mu_e$ ):
$\hat{s}_\theta(\tau^j) = \bar{s}_\theta(\tau^j) + \epsilon (E_j - \mu_e)$
This results in stronger suppression for failures that diverge early (high $E_j$ ) and gentler penalties for failures that remain aligned with the successful path for longer (low $E_j$ ).

2.3 Theoretical Properties

The authors prove a First-Order Invariance Theorem, demonstrating that LACL-GUI satisfies:
$L_{LACL-GUI} = L_{REAL} + O(\epsilon^2)$
This indicates that the proposed perturbations preserve the first-order optimization objective of the original REAL method. Consequently, the framework refines gradient allocation and local optimization geometry within outcome groups without altering the fundamental success–failure classification boundary, ensuring stability.

3. Key Contributions

Trajectory-Level Supervision in Contrastive RLVR: The paper introduces a novel mechanism to incorporate fine-grained quality signals (length and divergence) into contrastive RLVR, addressing the "blindness" of existing methods to intra-group trajectory differences.
Structured Preference Modeling: It proposes specific logit perturbation strategies for both successful (length-aware) and failed (divergence-aware) trajectories, encouraging efficiency and distinguishing informative failures.
Stable Optimization Framework: By utilizing zero-mean logit modulations and stop-gradient operations on group statistics, the method achieves fine-grained guidance while maintaining the gradient safety and stability of classification-based objectives.
Asynchronous Training Infrastructure: The implementation includes a decoupled architecture with a Success Buffer and trajectory filtering, specifically designed to handle the sparse rewards and long-horizon nature of GUI tasks.

4. Experimental Results

The authors evaluated LACL-GUI on the OSWorld benchmark using Qwen3-VL-Thinking models (4B and 8B parameters) as backbones.

Overall Performance: LACL-GUI-8B achieved an overall task success rate of 50.0%, outperforming the base Qwen3-VL-8B-Thinking model (+7.3%), the GRPO baseline (+2.8%), and the REAL baseline (+2.7%).
Comparison with SOTA: The model surpassed several strong open-source and closed-source baselines, including DART-GUI-7B (46.7%) and DeepMiner-Mano-7B (47.9%).
Efficiency Gains: In ablation studies focusing on tasks where all methods achieved the same outcome, LACL-GUI reduced the average length of successful trajectories by 10.3% (from 9.13 steps in REAL to 8.19 steps) and reduced the average length of failed trajectories by 10.5%.
Robustness: Sensitivity analysis showed that the method maintains stable performance across different similarity thresholds ( $\Theta$ ) for divergence estimation, indicating that the relative ordering of failures is robust to parameter variations.

5. Significance and Claims

The paper claims that LACL-GUI demonstrates the value of trajectory-level supervision within contrastive RLVR frameworks. By moving beyond binary outcome labels, the method provides more informative learning signals that guide agents toward more efficient executions and help them learn from near-miss failures.

The authors emphasize that their approach does not require additional models or task-specific supervision (unlike milestone-based rewards) but derives quality signals directly from the interaction data within a unified contrastive framework. The results suggest that refining gradient allocation based on trajectory quality is a viable path to improving the stability and efficiency of training GUI agents, particularly in environments with sparse rewards and long horizons.