🤖 AI

Task-Adaptive Rubrics for GUI Reward Modeling

The paper introduces AdaptRubric, a coarse-to-fine framework that enhances GUI reward modeling by dynamically constructing task-adaptive judging criteria through category-level retrieval and instance-level refinement, thereby significantly improving both reward evaluation accuracy and task success rates compared to existing methods.

Original authors: Tao Xiong, Xavier Hu, Wenkai Wang, Qinzhuo Wu, Changqiao Wu, Pengzhi Gao, Wei Liu, Jian Luan, Shengyu Zhang

Published 2026-08-26

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Tao Xiong, Xavier Hu, Wenkai Wang, Qinzhuo Wu, Changqiao Wu, Pengzhi Gao, Wei Liu, Jian Luan, Shengyu Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a computer program that can look at a screen, understand a human's request, and click the right buttons to get the job done. This is the promise of the modern GUI agent, a digital worker capable of navigating the complex interfaces of our phones and computers. But for these agents to learn and improve, they need a teacher. In the world of artificial intelligence, this teacher is a reward system that watches the agent's actions and decides if it succeeded or failed. If the agent successfully sets a timer or sends an email, the teacher gives a "good job" signal. If the agent clicks the wrong button or leaves a setting unchanged, the teacher must say "try again." The quality of this feedback is everything; a teacher that is too vague or too strict will confuse the student, preventing it from ever learning the correct way to behave.

For a long time, these digital teachers have struggled with a specific problem: they often miss the fine details of what a human actually asked for. They might see that a number was changed on a screen and assume the task is complete, even if the wrong number was changed, or they might ignore a specific instruction about where to place a piece of text. The researchers behind a new study, led by Tao Xiong and Shengyu Zhang, realized that the way these teachers judge success was too rigid. They were either using a generic checklist that applied to every task, or they were relying on the computer's own guesswork about what mattered. Both approaches led to mistakes where the computer thought it had succeeded when it had actually failed, or vice versa.

To solve this, the team developed a new method called ADAPTRUBRIC. Think of it as a two-step process for creating a custom grading rubric for every single task. First, the system looks at the user's request and figures out what broad category of work it belongs to, such as "sending a message," "changing a setting," or "deleting a file." For each of these categories, the system has a pre-written set of rules that covers the common pitfalls and standard requirements. This is the "coarse" stage, which provides a solid foundation and ensures the teacher knows the general boundaries of the task.

But the system does not stop there. In the second step, the "fine" stage, it looks closely at the specific details of the current request. If a user asks to change a setting to exactly fifty, the system adds a specific rule to check that number. If a user asks to move a file to a specific folder, the system adds a rule to verify the destination. This allows the teacher to adapt instantly to the unique constraints of the moment. The researchers tested this new approach on a wide variety of tasks across different operating systems, including Windows, macOS, Android, and the web. They found that by combining the broad category rules with the specific, task-adaptive details, the system became significantly better at judging success.

The results were clear and measurable. When tested on a benchmark of over 1,400 different task attempts, the new method correctly identified whether a task was a success or a failure 86.7% of the time. This was a noticeable improvement over previous methods, which often missed the mark by a wider margin. More importantly, the researchers used this smarter teacher to train an AI agent in a live environment. When the agent learned from the feedback provided by ADAPTRUBRIC, it became much better at completing tasks on its own, achieving a success rate that was 4.23 percentage points higher than when it was trained with older, less precise reward systems.

The study also highlighted exactly where the old methods failed. In one specific example, a user asked an agent to add a greeting to the very top of a digital note. The agent did add the greeting, but it placed it at the bottom of the text instead. Older teachers, looking only at whether the text appeared, marked this as a success. The new system, however, recognized that the instruction specifically demanded the text be at the top, and correctly marked the attempt as a failure. This ability to distinguish between "something happened" and "the right thing happened in the right way" is what makes the difference. By building a judging system that is both broad enough to cover general tasks and sharp enough to catch specific details, the researchers have provided a more reliable way for AI agents to learn from their mistakes and master the digital world.

Technical Summary: Task-Adaptive Rubrics for GUI Reward Modeling

1. Problem Statement

Recent advancements in Graphical User Interface (GUI) agents rely heavily on Outcome Reward Models (ORMs) to evaluate whether an executed trajectory satisfies user instructions. However, existing reward verifiers suffer from a critical limitation: under-specified judging criteria.

Current approaches generally fall into two categories, both of which lack task adaptability:

Static Templates: These use fixed rubric structures. While stable, they are too generic. They may verify if a task is "completed" but fail to adapt checks to specific targets, required values, operation scopes, or output formats defined in the current instruction.
Implicit Reasoning: These rely on the model's internal reasoning without explicit boundaries. This offers flexibility but risks overlooking specific instruction details or introducing constraints (scope creep) that were not part of the original task.

The core problem is that a verifier must derive explicit, task-adaptive judging criteria from the user instruction before assigning a reward. Without this, verifiers may incorrectly judge a trajectory as successful (e.g., verifying content presence but missing a specific placement constraint) or overly strict by enforcing unstated requirements.

2. Methodology: ADAPTRUBRIC

The authors propose ADAPTRUBRIC, a Coarse-to-Fine Rubrics Framework designed to construct task-adaptive judging criteria through a two-stage process. The framework fuses a category-level coarse rubric with an instance-level fine rubric to guide a Visual Language Model (VLM) verifier.

2.1 Stage 1: Category-Level Coarse Rubric Retrieval

This stage establishes the verification boundary based on the task family.

Taxonomy Construction: The authors define a taxonomy of 8 GUI task categories (e.g., info_query, create_modify, delete_cleanup, communication, transfer, state_navigation, composite_workflow, general).
Rubric Bank: A fixed bank of rubrics is constructed offline using an LLM-assisted induction procedure on a development trajectory pool. Each category entry ( $E_c$ $E_{c}$ ) contains:
- Verification Steps: Standard procedures for that category.
- Common Pitfalls: Typical failure modes.
- Special Rules: Domain-specific constraints.
- Output Format: Structured response requirements.
Routing: At inference, a task router ( $g_\phi$ ) predicts the category $c$ from the instruction $I$ . The system retrieves the corresponding coarse rubric ( $R_c$ ) from the bank. If no specific category is identified, it falls back to a general rubric.

2.2 Stage 2: Instance-Level Fine Rubric Generation

This stage adapts the coarse rubric to the specific constraints of the current instruction.

Generator: A fine rubric generator ( $h_\psi$ ) takes the instruction $I$ , the predicted category $c$ , and the coarse rubric $R_c$ as input.
Constraints: The generator is constrained to produce a compact list of at most two items ( $|R_f| \le 2$ $∣ R_{f} ∣ \leq 2$ ).
- Instruction Grounding: Every fine item must be supported by an explicit phrase or value in the user instruction.
- Abstention: The generator may return an empty set ( $R_f = \emptyset$ ) if no additional specific cues are needed, preventing the introduction of hallucinated constraints.
Output: The result is a set of compact, instance-level cues ( $R_f$ ) highlighting concrete values, scopes, or constraints (e.g., "add to the top of the note" vs. just "add text").

2.3 Criterion Fusion and Reward Verification

Fusion: The final judging criterion $R$ is formed by appending the fine rubric to the coarse rubric: $R = R_c \oplus R_f$ . This preserves the category-level boundary while adding instruction-specific details.
Context Selection: A context selector ( $\sigma$ ) extracts a compact trajectory context $\tau'$ (e.g., initial state, final state, and high-signal action frames) from the full trajectory.
Verification: The fused criterion $R$ , instruction $I$ , and context $\tau'$ are passed to a VLM verifier ( $f_\theta$ ) to predict a binary outcome reward ( $\hat{r} \in \{0, 1\}$ ).

3. Key Contributions

Formulation: The paper formulates the problem of task-adaptive criterion construction for GUI outcome reward modeling, emphasizing the need to derive explicit criteria from instructions prior to reward assignment.
Framework (ADAPTRUBRIC): Introduction of a coarse-to-fine framework that combines reusable category-level rubrics with compact, instruction-grounded instance-level rubrics.
Empirical Validation: Comprehensive evaluation demonstrating that explicitly constructing task-adaptive criteria improves both offline reward discrimination and downstream online reinforcement learning (RL) performance.

4. Experimental Results

4.1 Offline Reward Discrimination (OGRBench)

Evaluated on the OmniGUIRewardBench (OGRBench) across five platforms (Ubuntu, Mobile, Windows, macOS, Web) using various Qwen and Gemini backbones.

Performance: ADAPTRUBRIC achieved 86.7% Accuracy and 86.6 F1.
Improvement: It improved the F1 score by 3.6 points over the baseline average (ZeroGUI and OS-Themis) under a matched 10-screenshot image budget.
Recall Gain: The primary improvement came from Recall (increasing from 80.2% to 86.5%), indicating the method successfully recovers more true positive trajectories without sacrificing precision.
Consistency: The method outperformed baselines across all platforms and judge backbones.

4.2 Online Reinforcement Learning

Evaluated on the MobileWorld environment using the ClawGUI framework with GRPO optimization.

Setup: Replaced the reward agent in the training loop with ADAPTRUBRIC while keeping the policy backbone (MAI-UI-8B) and training protocol fixed.
Result: Using ADAPTRUBRIC as the reward verifier yielded a 4.23-point absolute gain in task success rate (rising from 19.70% to 23.93%), outperforming all other compared reward agents (DigiRL, ZeroGUI, OS-Themis).

4.3 Reward-Guided Test-Time Scaling

Evaluated on a heterogeneous pool of 113 AndroidWorld tasks with 10 candidate trajectories per task.

Selection Protocols: Tested EarlyStop (low-latency) and BestOfN (batch) selection.
Results: ADAPTRUBRIC achieved the largest gains over Random selection: +11.88 points for EarlyStop@7 and +13.28 points for BestOfN@8.
Efficiency: It maintained a low False Positive Rate (11.17%), comparable to the conservative OS-Themis baseline, while significantly outperforming others in accuracy and F1.

4.4 Ablation and Efficiency

Ablation: Removing either the Coarse or Fine component reduced F1 by 2.0 points. Removing both (relying only on images and instructions) reduced F1 by 5.9 points, confirming the complementary nature of the two stages.
Efficiency: ADAPTRUBRIC provided the best quality-cost trade-off, using significantly fewer tokens and runtime than OS-Themis and ZeroGUI while achieving higher accuracy.

5. Significance and Claims

The paper claims that explicitly constructing task-adaptive criteria is essential for reliable GUI reward modeling.

Motivation: The authors argue that generic templates or implicit reasoning are insufficient because they cannot simultaneously preserve the verification boundaries of task categories and adapt to the concrete constraints of specific instructions.
Impact: By combining category-level boundaries (via Coarse Rubrics) with instance-specific cues (via Fine Rubrics), ADAPTRUBRIC enables verifiers to correctly identify failures that baselines miss (e.g., verifying content but missing placement constraints, as shown in the "Hello, World!" case study).
Downstream Value: The improved reward signal directly translates to better policy optimization in online RL, suggesting that the quality of the reward criterion is a bottleneck in current GUI agent training.

The authors remain modest regarding limitations, noting that the taxonomy is fixed offline and may not capture every emerging application or domain-specific workflow, and that the evaluation, while cross-platform, does not exhaust all possible interface designs.