💬 NLP

OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflows

This paper introduces OS-Sentinel, a hybrid safety detection framework that combines a formal verifier and a VLM-based contextual judge to address safety risks in mobile GUI agents, supported by the new MobileRisk-Live benchmark and dynamic sandbox environment.

Original authors: Qiushi Sun, Mukai Li, Zhoumianze Liu, Zhihui Xie, Fangzhi Xu, Zhangyue Yin, Kanzhi Cheng, Zehao Li, Zichen Ding, Qi Liu, Zhiyong Wu, Zhuosheng Zhang, Ben Kao, Lingpeng Kong

Published 2026-07-28

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Qiushi Sun, Mukai Li, Zhoumianze Liu, Zhihui Xie, Fangzhi Xu, Zhangyue Yin, Kanzhi Cheng, Zehao Li, Zichen Ding, Qi Liu, Zhiyong Wu, Zhuosheng Zhang, Ben Kao, Lingpeng Kong

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where your phone isn't just a tool you tap on, but a digital butler that actually does things for you. You tell it, "Send a sticker to the group chat to see who's free for lunch," and it opens the app, finds the contacts, types the message, and hits send. This is the promise of "computer-using agents" powered by super-smart AI models that can see your screen and understand your commands, much like a human would. But here's the catch: just because a robot can follow instructions doesn't mean it won't accidentally cause a mess. If you ask a human to "send a sticker," they won't suddenly decide to delete your bank account or leak your private photos. But an AI, trying its best to be helpful, might misinterpret a command and wander into dangerous territory, like opening a sketchy app or sharing sensitive data it wasn't supposed to. This is the big question researchers are wrestling with: How do we make sure these digital butlers stay safe, reliable, and don't accidentally turn our phones into digital disaster zones?

Enter OS-Sentinel, a new safety system designed to be the ultimate "guard dog" for these mobile phone agents. The researchers behind this paper realized that existing safety checks were like trying to catch a thief with a blindfold: some checks were too rigid (like a rulebook that only knows specific words) and missed subtle dangers, while others were too vague (like a human judge who might get distracted). To fix this, the team built a special training ground called MobileRisk-Live. Think of this as a "simulated phone" where they can let AI agents run wild in a safe, virtual environment. They recorded thousands of these digital adventures, noting exactly what the agent saw, what it clicked, and what happened in the background of the phone's operating system. They turned these recordings into a massive test bank called MobileRisk, which is packed with realistic scenarios ranging from harmless tasks to dangerous privacy leaks.

Using this test bank, they built OS-Sentinel, a hybrid safety detector that works like a two-person security team. The first member is the Formal Verifier, a strict, rule-following robot that checks the phone's "under-the-hood" system logs. It looks for concrete, undeniable red flags, like if the agent tried to install a suspicious app or change a system setting it shouldn't touch. It's like a security guard checking a list of forbidden items. The second member is the Contextual Judge, a super-smart AI that looks at the screen and the agent's actions to understand the story. It asks, "Is this agent trying to share a secret password? Is it sending an offensive meme?" This part is flexible and understands nuance, catching dangers that a simple rulebook would miss.

When they put OS-Sentinel to the test, it was a clear winner. The researchers found that this two-part team caught safety issues 10% to 30% better than previous methods. It was great at spotting both the obvious system crashes and the sneaky, context-heavy mistakes. The paper suggests that by combining hard system checks with smart, context-aware observation, we can finally start building mobile agents that are not only helpful but also trustworthy. While the system is currently tested on Android-like environments (and might need tweaking for other types of phones), the approach offers a promising new path forward. It proves that to keep our digital butlers safe, we need a safety net that is both rigid enough to catch technical glitches and smart enough to understand the messy, real-world context of our daily lives.

Technical Summary: OS-Sentinel

Problem Statement

While Vision-Language Models (VLMs) have enabled autonomous agents to operate mobile Graphical User Interfaces (GUIs), ensuring their safety remains a critical, underexplored challenge. Unlike desktop or web environments, mobile GUIs present unique risks due to their diverse applications, sensitive user data, and dynamic interaction contexts. Existing safety research faces three primary limitations:

Infrastructure Gaps: Current environments focus on desktop or web platforms, lacking mobile-specific capabilities to capture full system states (e.g., runtime processes) necessary for detecting deep safety issues.
Detection Limitations: Deterministic rule-based methods struggle with scalability and context, while model-based approaches often target narrow scenarios or rely on generic paradigms that fail to establish strict safety boundaries.
Evaluation Scope: Most studies emphasize step-level detection, disconnected from realistic multi-action trajectories and system-state transitions, making it difficult to assess risks that emerge over time.

The core problem is the inability to reliably detect multifaceted safety threats—ranging from explicit system compromises (e.g., unauthorized file access) to implicit contextual violations (e.g., privacy leakage or offensive content)—within the complex, dynamic operational space of mobile agents.

Methodology

The paper proposes a two-pronged approach: the construction of a new evaluation infrastructure and a novel hybrid detection framework.

1. Infrastructure: MobileRisk-Live and MobileRisk

To address the lack of realistic mobile testing environments, the authors introduce:

MobileRisk-Live: A dynamic sandbox environment built on Android emulators. It supports real-time agent execution and provides a unified interface to record:
- GUI Observations ( $o_t$ ): Screenshots and accessibility trees (a11ytree).
- Agent Actions ( $a_t$ ): User interactions.
- System State Trace ( $T_{sys}$ ): Underlying system metadata (e.g., file operations, permission changes, background processes) captured via Android UIAutomator2. This trace is critical for detecting invisible risks not visible on the GUI.
MobileRisk: A benchmark derived from MobileRisk-Live containing 204 realistic trajectories (102 safe, 102 unsafe).
- Data Schema: Each trajectory includes GUI observations, system state traces (lightweight hash representations of file metadata and screen text), and fine-grained safety annotations.
- Taxonomy: Risks are categorized into ten fine-grained types, split into User-side risks (Malicious Use, Prompt Injection, Harmful Content) and Agent-side risks (Destructive Operations, Privacy Violations, Security Evasion, UI Manipulation, Resource Abuse, Compliance Violations, Over-Automation).
- Construction: Unsafe trajectories are generated by executing curated risk-prone instructions, while safe trajectories are either generated via safe instructions or constructed as corrected counterparts to unsafe ones to minimize false positives.

2. Framework: OS-Sentinel

OS-Sentinel is a hybrid safety detection framework that synergizes two distinct components to cover both explicit system-level violations and implicit contextual risks:

Formal Verifier (Deterministic): A rule-based system checker that analyzes the System State Trace ( $T_{sys}$ ). It performs three specific checks:
1. System State Integrity Monitoring: Computes cryptographic hashes of file system metadata to detect unauthorized modifications or privilege escalations.
2. Sensitive Keyword Detection: Matches visible text against a curated lexicon of financial, personal, and security terms (based on OWASP guidelines).
3. Sensitive Pattern Matching: Uses regular expressions to identify structured data like credit card numbers or emails.
Contextual Judge (Model-based): A VLM-powered semantic analyzer that processes multimodal inputs (screenshots, accessibility trees, and actions).
- It operates at the Step Level ( $J_\theta(o_t, a_t)$ ) for real-time intervention.
- It operates at the Trajectory Level via Consecutive (window-based) or Sampled modes to assess holistic behavioral intent.
Hybrid Verdict: The final safety decision is an aggregation of both components. The framework supports a Strict Mode (logical OR), where a flag from either component triggers an unsafe verdict, ensuring zero-tolerance for risk.

Key Results

Experiments were conducted on the MobileRisk benchmark using various backbone models (GPT-4o, Claude-3.7/4.5, Qwen2.5-VL, etc.) compared against Rule-based Evaluators and standard VLM/LLM-as-a-Judge baselines.

Performance Gains: OS-Sentinel consistently outperformed existing approaches across multiple metrics. It achieved 10%–30% improvements in accuracy and F1 scores over baselines at both trajectory and step levels.
Robustness: The hybrid approach demonstrated superior capability in detecting diverse risk categories. While baselines often excelled in specific categories but failed in others, OS-Sentinel provided balanced detection across the full taxonomy of unsafe behaviors.
Efficiency: The framework is model-agnostic, achieving strong results even with a 7B-scale backbone (suitable for local mobile deployment). For API-based models, the average detection latency was approximately 66ms, indicating practical viability for real-world deployment.
Environment Validity: Comparisons between the dynamic MobileRisk-Live environment and the frozen MobileRisk dataset showed high consistency (closeness), validating the use of frozen trajectories for reproducible safety research.

Significance and Claims

The paper claims to establish a new paradigm for safeguarding mobile GUI agents through the following contributions:

Foundational Infrastructure: By introducing MobileRisk-Live and MobileRisk, the work provides the first dynamic sandbox and fine-grained benchmark specifically designed for systematic safety studies on mobile agents, addressing the gap in mobile-specific evaluation infrastructure.
Methodological Advancement: OS-Sentinel demonstrates that a hybrid approach, combining deterministic formal verification with model-based contextual judgment, is necessary to overcome the limitations of relying solely on either rules or models. This approach effectively captures both "invisible" system-level changes and "nuanced" semantic violations.
Empirical Insights: The study provides critical analysis on the components of safety detection, showing that integrating system state traces significantly enhances detection capabilities. It also highlights the importance of trajectory-level evaluation over isolated step-level checks.

The authors position this work as a step toward more reliable and trustworthy autonomous mobile agents, offering a reproducible foundation for future research in mobile agent safety. They acknowledge limitations regarding the dependency on open platforms (like Android) for system state traces and the inherent differences between simulated and real-world network conditions, but assert that the proposed framework offers a robust baseline for the field.