Technical Summary: Lifecycle-Aware Dynamic Analysis for Secure ML Model Execution

Problem Statement

The rapid adoption of pre-trained Machine Learning (ML) models has created a software supply chain vulnerability where malicious behavior can be embedded within model artifacts (e.g., .pt, .keras, .h5). Unlike traditional software, these artifacts are often loaded and executed directly by ML frameworks, bypassing standard code inspection.

Current defense mechanisms face significant limitations:

• Static Analysis: Existing scanners rely on static, format-specific rules or known attack signatures (e.g., specific Pickle opcodes or Lambda layers). These approaches struggle to generalize across different file formats, ML frameworks (PyTorch, TensorFlow, Keras), and novel exploitation techniques.
• Reactive Nature: Defenses are largely reactive, detecting known patterns while failing to identify new attacks that exploit previously unexplored framework features or abuse legitimate APIs.
• Precision-Recall Trade-off: Static scanners often suffer from high false-positive rates (misclassifying benign models with complex structures) or false negatives (missing novel exploits).
• Scope Limitations: Many solutions focus narrowly on the loading phase or specific serialization formats (like Pickle), ignoring threats that manifest during inference or training phases.

Methodology: MOAT and RE-MOAT

The authors propose MOAT (Model-Oriented Action Tracing), a lifecycle-aware dynamic analysis approach, and its reference implementation, RE-MOAT. The core intuition is that while ML models vary in parameters, their execution is constrained by well-defined lifecycle phases (loading, inference, training) and the specific framework used. Consequently, interactions with the host system during these phases are highly structured and predictable.

Core Concepts

1. Execution Boundaries: Instead of looking for specific attack signatures, the system defines "execution boundaries" (allowlists) for each framework–phase pair. These boundaries specify the set of expected security-relevant actions (interactions with files, network, processes, and devices) a benign model should perform.
2. Action Abstraction: Low-level system calls are abstracted into high-level security actions. For example, a mmap system call is interpreted based on its arguments as either memory allocation or file access.
3. Anomaly Detection: Any action performed by the model that falls outside the defined execution boundary for its specific lifecycle phase is flagged as suspicious (unexpected).

System Architecture (RE-MOAT)

RE-MOAT is implemented on Linux x86-64 and consists of four components:

1. Action Abstraction: Maps system calls to abstract security actions.
2. Execution Boundaries Definition: An offline process where benign models are executed to profile expected behaviors. These profiles are manually validated to ensure they cannot be abused.
3. Orchestrator: Manages the execution of the ML model, isolating specific lifecycle phases (e.g., loading vs. inference) to ensure the correct boundary is applied. It injects signals to mark the start and end of the monitored window, filtering out initialization noise.
4. Tracer: A runtime component (built on libdebug/ptrace) that intercepts system calls, maps them to actions, and checks them against the active execution boundary. Violations trigger alerts.

Key Contributions

• Foundational Intuitions: The paper establishes that ML model execution exhibits low heterogeneity across instances within a specific framework and lifecycle phase, enabling the definition of precise execution boundaries.
• MOAT Framework: A conceptual design for securing ML execution by enforcing boundaries over host interactions rather than analyzing artifact structure.
• RE-MOAT Implementation: A syscall-based reference implementation supporting multiple frameworks (PyTorch, Keras, TensorFlow) and formats (Pickle, .keras, .h5, SavedModel).
• Empirical Validation: A comprehensive evaluation across 77,974 real-world models, 31 Proofs-of-Concept (PoCs) from CVEs and novel attacks, and a comparison against state-of-the-art scanners.

Experimental Results

The evaluation was conducted under a threat model where the attacker aims to compromise the host system (e.g., arbitrary code execution, data exfiltration) during loading, inference, or training.

• Unseen Exploits (RQ1): RE-MOAT successfully detected all 31 PoCs, including attacks exploiting novel vulnerabilities (e.g., CVE-2025-1550, CVE-2025-9905) and the TensorAbuse technique. Detection was achieved without hardcoding vulnerability-specific rules, relying solely on the deviation from expected host interactions.
• Large-Scale Analysis (RQ2): Analyzing 77,974 models from the Hugging Face Hub, RE-MOAT flagged 23 models as suspicious. Manual inspection confirmed all 23 were malicious (ranging from command execution to reverse shells), resulting in a 0% false-positive rate.
• State-of-the-Art Comparison (RQ3): Compared against tools like MODELSCAN, PICKLEBALL, and Fickling on a dataset of 334 models:
  • Under the paper's threat model (focusing on host compromise), RE-MOAT achieved 0% false positives and 0% false negatives.
  • Competing tools exhibited significant false positives (up to 42.5%) or false negatives, often due to reliance on static signatures or limited threat assumptions (e.g., ignoring inference-time attacks).
  • RE-MOAT remained effective regardless of the underlying model format, whereas tools like PICKLEBALL are restricted to Pickle artifacts.

Significance and Claims

The paper claims that lifecycle-aware dynamic analysis offers a robust and generalizable defense for ML model execution. By shifting the focus from "how an attack is encoded" to "what effects the execution has on the host," the approach:

• Generalizes: It works across diverse frameworks, serialization formats, and attack vectors (code injection, framework abuse, code reuse).
• Reduces Maintenance: Unlike signature-based scanners that require constant updates for new CVEs, execution boundaries are derived from the stable behavior of the framework itself.
• Complements Static Analysis: The authors envision a hybrid approach where static analysis filters known bad patterns, while dynamic analysis (MOAT) catches novel attacks and framework abuses that static tools miss.

The work validates the intuition that the structured nature of ML model execution allows for the creation of tight execution boundaries, making deviations a strong signal of malicious behavior with minimal false positives.