🤖 AI

LiteGUI: Distilling Compact GUI Agents with Reinforcement Learning

This paper proposes LiteGUI, a novel SFT-free training paradigm that combines Guided On-policy Distillation and a Multi-solution Dual-level GRPO framework to significantly enhance the performance of lightweight, on-device GUI agents, enabling 2B/3B scale models to achieve state-of-the-art results that rival much larger models.

Original authors: Yubin Wu, Zicheng Cai, Liping Ning, Hua Wang, Zhi Chen, Yaohua Tang, Hao Chen

Published 2026-05-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yubin Wu, Zicheng Cai, Liping Ning, Hua Wang, Zhi Chen, Yaohua Tang, Hao Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Pocket-Sized" Computer Butler

Imagine you want a computer assistant that can click buttons, type text, and navigate menus for you. Usually, to make a smart assistant, you need a giant, super-powerful brain (a massive AI model) that lives in a data center. It's like hiring a PhD professor to do your grocery shopping. It works great, but it's expensive, slow, and you can't carry it in your pocket.

The authors of this paper asked: "Can we make a tiny, lightweight assistant (a '2B' or '3B' parameter model) that fits on your phone or laptop but is just as smart as the giant professors?"

The answer is yes, but not by teaching it the old-fashioned way. They built a new training system called LiteGUI.

The Problem: The "Rigid Robot" Trap

Usually, to teach a small AI, we use a method called Supervised Fine-Tuning (SFT). Think of this like a strict teacher showing a student a single, perfect path to solve a puzzle.

The Issue: If the student (the small AI) tries to take a slightly different path, the teacher scolds them.
The Result: The student becomes a "rigid robot." They memorize the exact path but panic if the situation changes even a little. They also tend to "forget" everything they knew before (like how to type or click) because they are so focused on the new, narrow rules. This is called catastrophic forgetting.

The Solution: A New Training Camp

The authors propose a two-stage training camp that skips the rigid "SFT" teacher entirely. Instead, they use a mix of Guided Distillation and Reinforcement Learning.

Stage 1: Guided On-Policy Distillation (The "Smart Shadow")

Imagine a student (the small AI) trying to solve a maze.

Old Way: The teacher just says, "You did it wrong," and shows the one correct path.
LiteGUI Way: The student tries to solve the maze. The teacher (a giant, smart AI) watches. But here's the trick: The teacher doesn't just look at the student's messy attempt. The teacher also looks at a cheat sheet of all the possible correct moves for that specific spot in the maze.

The teacher then says, "Okay, you tried to go left. That's actually a valid move! Here is a 'shadow' version of your move that is slightly better."

The Analogy: It's like a coach who doesn't just say "Wrong!" but instead says, "You're on the right track, but try this specific variation of your move." This helps the student learn without getting confused by the teacher's "hallucinations" (mistakes) and without forcing them to copy just one single path.

Stage 2: Multi-Solution Dual-Level GRPO (The "Strategy Game")

Once the student has learned the basics, they enter a video game mode called Reinforcement Learning.

The Problem with Old Games: In many AI games, if you take a valid path that isn't the exact one the game designer wrote down, you get a "Game Over" (a negative score). This stops the AI from being creative.
The LiteGUI Fix: They introduced a Multi-Solution rule.
- Analogy: Imagine you need to get to the kitchen. You can walk through the front door, the back door, or the window. In the old system, only the front door was "correct." In LiteGUI, all three doors are valid. If you pick the window, you still get points! This encourages the AI to explore different ways to solve problems.

They also added a Dual-Level scorecard:

Micro-Level (The Step): Did you click the right button right now?
Macro-Level (The Plan): Are you actually moving toward the final goal, or are you just clicking buttons randomly?

The Result: The AI learns not just how to click, but why it is clicking, helping it plan long, complex tasks without getting lost.

The Toolkit: Building the Training Data

To make this work, the authors couldn't just use existing data. They built a factory (an automated pipeline) to create their own training data.

They used a giant AI to generate thousands of computer tasks.
Then, they had humans verify these tasks and, crucially, annotate multiple correct ways to do each step.
They released this data (called Lite-Dataset) and a new test suite (called Lite-Bench) so others can test their own tiny AI agents.

The Results: Small but Mighty

When they tested their new "LiteGUI" agents:

Performance: The tiny 2-billion-parameter model (LiteGUI-2B) became the champion among small models.
The Shock: It didn't just beat other small models; it performed almost as well as models that are 10 to 20 times larger.
Efficiency: It achieved this using far less training data than previous methods, proving that the way you teach the AI matters more than just throwing more data at it.

Summary

The paper claims that by stopping the practice of forcing small AI models to copy a single "perfect" path, and instead teaching them to recognize multiple valid paths and plan ahead using a smart "shadow" teacher, we can create tiny, on-device computer agents that are surprisingly powerful, flexible, and capable of handling complex tasks without needing a supercomputer.

Technical Summary: LiteGUI - Distilling Compact GUI Agents with Reinforcement Learning

Problem Statement

The development of lightweight, on-device vision-language GUI agents is critical for efficient cross-platform automated interaction. However, current on-device agents (typically 2B–3B parameter scales) face significant performance bottlenecks. Traditional Supervised Fine-Tuning (SFT) on small-scale models often leads to overfitting, catastrophic forgetting of foundational capabilities, and policy rigidity, particularly in dynamic UI environments requiring long-horizon planning. Furthermore, existing Reinforcement Learning (RL) approaches often rely on single-solution rewards and naive coordinate matching, which suffer from sparsity, false negatives, and an inability to handle the multi-solution nature of GUI tasks. While large-scale models (e.g., GPT-4o, Claude 3.5) demonstrate strong performance, their computational overhead prohibits localized deployment.

Methodology

The authors propose LiteGUI, a novel SFT-free training paradigm that combines Guided On-policy Distillation (Guided-OPD) and Multi-solution Dual-level GRPO (MD-GRPO). The framework operates in two stages:

1. Guided On-policy Distillation (Guided-OPD)

To address the instability of distilling from teacher models in complex GUI settings, the authors introduce a mechanism that conditions the teacher model on human-verified valid action sets ( $A^*_t$ ) during training.

Privileged Guidance: The teacher model receives the student's current output alongside a "guidance" signal derived from ground-truth valid actions, while the student model receives only the standard input. This follows the "Learning Using Privileged Information" paradigm.
Dynamic Retrieval: Recognizing that GUI tasks often have multiple valid solutions, the method employs a Most-Matched-GT strategy. Instead of randomly selecting a single ground truth or providing all valid actions (which may dilute the signal), the system dynamically retrieves the valid action from the pool that best matches the student's current on-policy behavior using a unified matching function ( $\phi_{gui}$ ). This reduces hallucinations and cognitive misalignment.
Objective: The student is optimized using a reverse-KL divergence objective to align with the teacher's distribution under this privileged context.

2. Multi-solution Dual-level GRPO (MD-GRPO)

To overcome the limitations of single-solution rewards in long-horizon tasks, the authors propose a dual-level reward framework integrated into Group Relative Policy Optimization (GRPO):

Multi-solution Action Reward ( $R_{ms}$ ): Instead of penalizing deviations from a single annotated action, the reward is calculated as the maximum matching score between the model's output and any valid action in the ground-truth set ( $A^*_t$ ). This mitigates "false negative" penalties for valid alternative paths.
Long-horizon Planning Reward ( $R_{sub}$ ): A subtask planning reward evaluates the agent's intermediate planning quality. A strong VLM judge (Qwen3-VL-32B) assesses the generated subtask list based on task relevance, visual grounding, decomposition granularity, and state consistency. Crucially, if the agent is stuck or in an error state, the plan must explicitly propose a corrective step to receive a high score.
Optimization: The total reward combines these two signals, normalized within a group of sampled trajectories to compute relative advantages for policy updates.

Data Infrastructure

To support this training, the authors constructed:

Lite-Dataset: A vision-only GUI dataset containing 30K trajectories and 11K human-verified multi-solution annotations across diverse scenarios (File System, Web, Terminal).
Lite-Bench: A new benchmark with 160 diverse tasks designed to evaluate long-horizon planning and error recovery in realistic environments, strictly prohibiting terminal usage for file tasks to enforce GUI interaction.
Automated Pipeline: A "generation-verification-correction" loop using a strong teacher model to synthesize trajectories, followed by human auditing to create the multi-solution annotations.

Key Contributions

New Training Paradigm: The paper proposes the first SFT-free training approach for lightweight GUI agents, demonstrating that combining on-policy distillation with RL can unlock the full potential of 2B–3B scale models.
Guided On-policy Distillation: The introduction of oracle-guided references and dynamic trajectory retrieval to reduce hallucinations and align with the multi-solution nature of GUI tasks.
Multi-solution Dual-level GRPO: A novel RL framework that jointly optimizes macro-level subtask planning and micro-level execution matching, effectively addressing exploration bottlenecks in long-horizon tasks.
Resources: The release of Lite-Dataset (30K trajectories, 11K multi-solution annotations) and Lite-Bench (160 evaluation instances) to address the scarcity of high-quality, multi-solution GUI data.

Experimental Results

Extensive evaluations were conducted on ScreenSpot-Pro, OS-World, and the new Lite-Bench.

Performance: The Lite-GUI-2B model achieved a new state-of-the-art (SOTA) average accuracy of 46.86% on ScreenSpot-Pro, surpassing significantly larger specialized models like UI-TARS-72B (35.7%) and JEDI-7B (39.5%).
Scalability: The Lite-GUI-30B-A3B model achieved 58.95% on ScreenSpot-Pro, outperforming the Qwen3-VL-32B teacher model (58.57%) and other large-scale specialized models.
Long-horizon Tasks: On OS-World, Lite-GUI-2B achieved a 13.24% success rate, more than doubling the performance of its vanilla backbone (6.04%). On Lite-Bench, it reached a 61.76% success rate, nearly double the baseline.
Data Efficiency: Notably, Lite-GUI-2B achieved these results using only 8K samples from ShowUI-Desktop for ScreenSpot-Pro, highlighting the data efficiency of the proposed framework.

Significance and Claims

The paper claims that the proposed method successfully overcomes the limitations of traditional SFT and naive RL for small-scale models. By bypassing SFT entirely and utilizing structured on-policy distillation combined with multi-solution dual-level exploration, LiteGUI enables on-device agents to achieve performance competitive with much larger-scale models. The authors assert that their approach fully unlocks the capabilities of 2B–3B scale agents, surpassing the performance limits of conventional imitation learning and providing a viable path for efficient, localized GUI automation. The work emphasizes that the combination of guided distillation and dual-level RL is essential for handling the complexity, multi-solution nature, and long-horizon requirements of real-world GUI tasks.