Technical Summary: Protecting On-Device AI Inference

Problem Statement

The rapid adoption of Artificial Intelligence (AI) and Machine Learning (ML) on edge and mobile devices has shifted inference from cloud-based servers to local execution to satisfy strict requirements regarding privacy, latency, and bandwidth. While this "on-device" paradigm offers significant benefits, such as direct user control over data and reduced network exposure, it introduces unique security and privacy vulnerabilities distinct from cloud or distributed edge training scenarios.

Existing literature largely focuses on "edge intelligence," which encompasses distributed training (e.g., Federated Learning) and broader edge computing contexts. These surveys often overlook the specific threats inherent to running pre-trained models entirely on a single, resource-constrained device. In on-device settings, threats such as the physical theft of the entire model, side-channel extraction of weights, and local adversarial manipulation differ significantly from cloud-based threats. Furthermore, current research reveals a critical imbalance: while numerous attacks targeting on-device inference have been demonstrated, corresponding defense mechanisms are often fragmented, lacking a comprehensive framework, and disproportionately focused on Intellectual Property (IP) protection rather than other critical threat vectors like adversarial attacks or availability.

Methodology

To address these gaps, the authors conducted a systematic literature review following established guidelines for software engineering reviews.

• Search Strategy: A systematic search was performed on the Scopus database covering peer-reviewed journal articles and conference proceedings from January 2019 to December 2025. Two complementary query strings were used to capture attack-focused and defense-focused literature, targeting keywords related to on-device inference, mobile AI, TinyML, and specific security terms (e.g., "model extraction," "TEE," "obfuscation").
• Selection Process: The initial pool of 260 unique publications was screened through a three-stage process:
  1. Title/Abstract Screening: Reduced the pool by ~65%, retaining 90 candidates.
  2. Full-Text Review: Applied strict inclusion and exclusion criteria. Papers were excluded if they focused solely on cloud infrastructure, federated learning training, or lacked empirical evaluation.
  3. Quality Assessment: Final selection required clear threat definitions, technical reproducibility, and quantitative results on real or benchmark systems.
• Final Corpus: The review resulted in a final corpus of 61 studies: 17 focusing on attack techniques and 44 on defense mechanisms.
• Analysis Framework: The authors developed multi-dimensional taxonomies to categorize attacks based on objectives and exploitation techniques, and defense mechanisms based on their primary technological approach.

Key Contributions

The paper makes three primary contributions to the field of AI security:

1. Survey Assessment: The authors analyzed existing reviews on edge and on-device AI, identifying a significant gap in coverage specifically dedicated to the security and privacy of on-device inference as distinct from general edge intelligence or training.
2. Attack Taxonomy: A comprehensive taxonomy of six major attack families targeting on-device AI was established:
   • Physical Attacks: Exploiting hardware side-channels (timing, power, electromagnetic) and memory access.
   • Intellectual Property (IP) Attacks: Model stealing, extraction, and reverse engineering of compiled artifacts.
   • Adversarial Attacks: Manipulating inputs to cause misclassification (white-box and black-box).
   • Data Privacy Attacks: Membership inference, model inversion, and training data exposure.
   • Integrity Attacks: Model tampering, backdoors, and pipeline manipulation.
   • Availability Attacks: Resource exhaustion (DoS) and energy-latency attacks.
3. Defense Taxonomy and Mapping: A multidimensional mapping of state-of-the-art defense mechanisms was provided, categorized into:
   • Model Segmentation: Partitioning models between secure (TEE) and unsecure (REE) environments.
   • Access Control: Authentication, licensing, and weight permutation.
   • Obfuscation: Encoding weights and parameters to hinder reverse engineering.
   • TEE Optimisation: Enhancing Trusted Execution Environments for better performance and resource management.
   • TEE Extensions: Extending secure regions to GPUs or multi-enclave architectures.

Results and Findings

The analysis of the 61 selected papers reveals several critical insights regarding the current state of on-device AI security:

• Asymmetry Between Attacks and Defenses: There is a marked imbalance in the literature. Approximately one-quarter of the surveyed attack papers focus on Intellectual Property (IP) theft, yet half of the defense solutions address this specific issue. Conversely, Adversarial Attacks constitute roughly one-third of the attack literature, yet no defense papers were found that explicitly target these attacks. Similarly, Availability Attacks have no associated defense papers in the reviewed corpus.
• Attack Effectiveness: The surveyed attacks demonstrate high practical feasibility.
  • Physical Attacks: Achieved success rates up to 100% (e.g., GPU timing attacks).
  • IP Attacks: Success rates ranged from 33.83% to 99%, with ML-based reconstruction techniques showing near-perfect recovery of compiled models.
  • Adversarial Attacks: Success rates varied widely (10.23%–99.12%), heavily dependent on query efficiency.
  • Integrity Attacks: Tools like DeepPayload achieved 93.5% success in injecting backdoors into deployed models.
• Defense Trends:
  • Model Segmentation is the most prominent defense strategy (15 works), primarily focusing on protecting DNN/DL models.
  • Target Models: The majority of research (25 works) focuses on DNN/DL models. Defenses for Large Language Models (LLMs) and TinyML are emerging but scarce (5 and 2 works, respectively).
  • Temporal Growth: Research interest has surged, with 2024 seeing the highest concentration of publications (8 attack papers, 14 defense papers).

Significance and Claims

The paper positions itself as the first comprehensive review specifically dedicated to the security threats and defense mechanisms of on-device AI inference. Its significance lies in:

• Clarifying the Scope: It distinguishes on-device inference from broader edge intelligence, highlighting that threats like full-model theft are unique to the single-device context where the entire model resides locally.
• Highlighting Critical Gaps: The authors explicitly claim that the current research landscape is unbalanced. The lack of defenses for adversarial and availability attacks, despite their prevalence in the attack literature, represents a "clear opportunity for future research."
• Call for Holistic Security: The paper argues that securing on-device inference requires shifting from a narrow focus on IP protection to a holistic approach that addresses physical exposure, heterogeneous hardware constraints, and the specific integrity and availability needs of local inference.
• Future Directions: It suggests that future work should explore hardware-aware Neural Architecture Search (NAS) for secure models, machine unlearning for privacy compliance, and the development of standardized security evaluation frameworks for on-device AI.

The authors conclude that while the threat landscape is broad and practically exploitable, the defensive ecosystem is currently insufficient to address the full spectrum of risks, particularly those related to model robustness and service availability.