GUI agent: Guided Exploration of User-Sensitive Screens

Technical Summary: GUI Agent: Guided Exploration of User-Sensitive Screens

Problem Statement
Large Language Model (LLM) agents are increasingly deployed to automate tasks within open Graphical User Interface (GUI) environments. However, these agents inevitably encounter screens containing user-sensitive information (e.g., financial data, private messages), where autonomous execution poses risks to privacy and safety. Current state-of-the-art LLM-driven agents are typically fine-tuned to complete tasks regardless of safety implications, making their real-world deployment difficult and reducing reliability. Furthermore, GUI agent workflows often operate in closed loops where actions are irreversible (e.g., deleting files, sending emails), and a single error can lead to cascading failures. There is a critical need to identify and categorize user-sensitive states and define "user-sensitive queries" to enable agents to recognize when to request human handover. Existing exploration methods often lack rigorous coverage of the solution space or fail to specifically target user-sensitive states.

Methodology
The paper proposes a framework centered on an Explorer Agent designed to systematically explore the query space of a GUI application, starting from a single demonstrated task, to identify queries that lead to user-sensitive states. The framework utilizes two models:

1. Native LM: Determines actions during execution (e.g., M3A).
2. Explorer LM: The core contribution, iteratively trained with the objective of pure exploration. Its policy determines which nodes to explore based on rewards derived from the novelty and user sensitivity of the nodes.

The methodology follows an iterative training loop (Algorithm 1) comprising three main stages:

• Query Selection, Expansion, and Saturation:

  • The Explorer LM generates batches of queries ($N \times M$ episodes) based on a base trajectory ($T_0$) and an instruction.
  • A Query Selection Algorithm ($A_{sel}$) filters these queries. It computes vectorized representations and uses cosine similarity to retain only queries that are novel (low similarity to the existing pool and within the batch) and distinct from the initial trajectory.
  • A Saturation Check Algorithm ($A_{sat}$) monitors the generation process. It fits linear and polynomial curves to the number of retained queries per batch to detect an inflection point where the model stops generating novel queries, signaling saturation.

• Roll-out with Emulator:

  • Selected queries are executed in an Android emulator (using the SPABench setup).
  • The Native LM executes actions, and the system captures screenshots and action logs.
  • This process generates a "memory bank" of episodes containing queries, steps, screen descriptions, and embeddings.

• Training (Backpropagation):

  • Instead of standard MCTS backpropagation, the Explorer LM is trained using Group Relative Policy Optimization (GRPO).
  • The training objective is dual: (1) generate accurate steps (via Supervised Fine-Tuning on prompt-response pairs) and (2) generate novel queries leading to novel user-sensitive states (via GRPO with a custom reward model).
  • Reward Modeling: The reward function ($R$) is composed of three novelty scores:
    1. Query Novelty: Based on K-means clustering inertia and cluster indices.
    2. Step Novelty: Based on the cosine similarity of "actionPurpose" steps against the memory bank.
    3. Category Novelty: Based on the frequency of screen categories (specifically flagging "critic" or sensitive categories).

Key Contributions

1. Explorer Agent Framework: A novel algorithm to isolate user-sensitive queries and screens starting from a single user-defined trajectory, utilizing experience distillation from previous iterations.
2. Novelty-Driven Reward System: A composite reward function that quantifies the novelty of queries, steps, and screen categories to guide the agent toward unexplored, potentially sensitive states.
3. Saturation Detection: An algorithmic approach to determine when the exploration space has been sufficiently covered, preventing redundant training iterations.
4. Dataset Generation Potential: The method aims to produce a dataset of user-sensitive queries to assist engineers in recognizing critical scenarios and requesting user handover.

Results
The authors conducted three training rounds using the Qwen-2.5-32B-Instruct model as the Explorer LM, with smaller models (Llama3.1-8B, Qwen2.5-3B) failing to proceed beyond the initial screen.

• Convergence: The total reward decreased progressively by orders of magnitude (from $10^{-2}$ to $10^{-5}$) across rounds, indicating a shrinking of the query and screen space as the agent saturated the environment.
• Saturation: The number of queries required to reach saturation decreased significantly across rounds (160 queries in Round 1, 100 in Round 2, 70 in Round 3).
• Step Accuracy: The standard deviation of step novelty scores dropped significantly from Round 1 (0.0686) to Rounds 2 and 3 (0.0024 and ~0.0068), suggesting that Supervised Fine-Tuning (SFT) improved the accuracy of step predictions.
• Instruction Tuning: Novelty scores decreased as batch numbers increased within a round, demonstrating that instruction tuning enables the generation of novel queries in subsequent batches.

Significance and Claims
The paper claims to provide a systematic method for autonomously exploring GUI query spaces to identify user-sensitive states. It argues that by iteratively optimizing exploration policies, the framework can prioritize unvisited states and improve screen diversity coverage. The authors posit that the shrinking of the query space in subsequent rounds serves as evidence of coverage. They suggest that this approach could help engineers design training datasets for application-specific purposes and improve the safety of LLM agents by enabling them to recognize when to stop and request user intervention. The paper concludes modestly, noting that while the method shows promise, further investigation is needed to handle high-similarity queries (e.g., "turn on notification" vs. "turn off notification") and to achieve more aggressive search strategies.