On the (In)Security of Loading Machine Learning Models

1. Problem Statement

The rapid adoption of Machine Learning (ML) has led to a proliferation of model-sharing ecosystems (e.g., Hugging Face, Kaggle, TensorFlow Hub). While these platforms lower entry barriers, they introduce significant security risks often overlooked by practitioners. Unlike traditional software supply chains, the security implications of loading pre-trained ML models remain underexplored.

The core problem is the misalignment between the perceived security of model-sharing mechanisms and their actual implementation. Many frameworks and hubs promote "secure" or "safe" modes (often relying on data-based formats like JSON) to mitigate the risks of arbitrary code execution (ACE) associated with code-based formats like Python's pickle. However, the authors hypothesize that these security narratives are misleading, creating a false sense of safety that leaves users vulnerable to 0-day vulnerabilities and unscanned threats.

2. Methodology

The authors employed a multi-faceted approach to evaluate the security landscape, addressing three Research Questions (RQs):

• RQ1: Security Posture Analysis: The team systematically analyzed the five most widely used ML frameworks (TensorFlow, Keras, PyTorch, scikit-learn, XGBoost) and five major model hubs (Hugging Face, Kaggle, PyTorch Hub, TensorFlow Hub, Keras Hub). They categorized sharing mechanisms based on:

  • Content: Self-contained (full model) vs. Non-self-contained (weights only).
  • Representation: Code-based (e.g., pickle) vs. Data-based (e.g., JSON, Protobuf).
  • Security Claims: Whether the framework/hub explicitly claims security features (e.g., safe_mode, scanning).

• RQ2: Vulnerability Assessment (0-Day Discovery): Focusing on mechanisms claiming security while offering full model sharing, the authors performed manual reverse engineering on the open-source codebases of Keras and Skops (a scikit-learn library). They specifically targeted the logic handling "safe" modes and data-based formats to identify logic flaws that could lead to ACE.

• RQ3: User Perception Survey: A survey of 62 ML practitioners (53 with model-sharing experience) was conducted to measure how security narratives (e.g., safe_mode=True, hub scanning labels) influence user trust and risk perception.

• Hub Scanner Evaluation: The authors tested their discovered exploits against the integrated scanning tools of Hugging Face (ClamAV, PickleScan, Protect AI, JFrog) to determine if hub-level defenses could catch framework-level 0-days.

3. Key Contributions

The paper makes four primary contributions:

1. Systematization of ML Model Sharing Security: A comprehensive taxonomy of sharing mechanisms across major frameworks and hubs, highlighting the gap between "security-oriented" claims and actual implementation.
2. Discovery of 0-Day Vulnerabilities: The identification of six previously undisclosed vulnerabilities (all assigned CVEs) in mechanisms explicitly marketed as secure. These vulnerabilities allow arbitrary code execution during model loading.
3. Challenge to "Data-Based" Security Assumptions: The paper demonstrates that data-based formats (like JSON in Keras and Skops) are not inherently secure if the deserialization logic reconstructs executable code objects without strict validation.
4. Survey on Security Narratives: Empirical evidence showing that security-oriented terminology significantly inflates user confidence, leading to a dangerous disconnect between perceived and actual safety.

4. Key Results

A. Vulnerability Findings (RQ2)

The authors uncovered critical flaws in "secure" mechanisms:

• Keras (safe_mode):
  • KV.1 (CVE-2025-1550): Insecure Module Resolution. Attackers could craft a config.json to import and execute arbitrary Python modules (e.g., subprocess) by exploiting insufficient validation in layer resolution.
  • KV.2 (CVE-2025-8747, CVE-2025-9906): Code Reuse via Lambda Layers. Attackers could bypass safe_mode by reusing legitimate internal Keras functions (gadgets) to disable security checks or execute code.
  • KV.3 (CVE-2025-9905): Silent Bypass. Loading legacy HDF5 files with safe_mode=True silently ignored the flag, allowing unrestricted code execution without warnings.
• Skops (Secure Persistence for scikit-learn):
  • SV.1 (CVE-2025-54413): MethodNode Abuse. Attackers could traverse object graphs to access dangerous internals (e.g., __builtins__) via MethodNode, bypassing type validation.
  • SV.2 (CVE-2025-54412): OperatorFuncNode Bypass. A mismatch between validated fields and internal usage allowed attackers to invoke arbitrary functions from the operator module.
  • SV.3 (CVE-2025-54886): Silent Fallback. If a model card file was not a valid ZIP, Skops silently fell back to joblib (insecure pickle), bypassing all Skops security checks.

B. Hub Scanner Effectiveness

Testing against Hugging Face's integrated scanners revealed significant gaps:

• False Negatives: The scanners failed to detect the Keras and Skops exploits (SV.1, SV.2) because they were not designed to analyze these specific "secure" formats. Consequently, malicious models were labeled "Safe."
• False Positives: Benign models using Lambda layers were frequently flagged as "Unsafe" by tools like JFrog and Protect AI, leading to alert fatigue.
• Format Blindness: Scanners often labeled unsupported formats (like .skops) as "not a model" or "No issue," providing a false sense of security.

C. User Perception (RQ3)

• The "Safe" Illusion: Users reported significantly higher comfort levels (7.8/10 vs. 4.5/10) when safe_mode=True was enabled in Keras, despite the existence of the 0-days.
• Hub Trust: 73.6% of participants stated that loading models from hubs with scanning features increased their comfort.
• Cybersecurity Gap: Non-security experts were disproportionately likely to trust these "safe" flags, often failing to recognize the risk of arbitrary code execution.

5. Significance and Implications

This paper fundamentally challenges the current paradigm of ML model security:

1. Models are Code: The authors argue that "data-based" formats are a fallacy for security. If a format allows the reconstruction of executable logic (even via JSON), it is effectively code and must be treated as such. The Google Security Team's response to the authors confirmed this: "Loading untrusted models is like running untrusted code—models are code."
2. The Danger of "Safe" Labels: The paper highlights that labeling a mechanism as "safe" or "secure" without rigorous, proven guarantees creates a false sense of security that is more dangerous than having no security at all.
3. Legacy Compatibility vs. Security: The persistence of legacy formats (HDF5, pickle) and the slow adoption of patched versions (Keras 3.x) create a systemic vulnerability where users remain exposed to known risks.
4. Ineffective Defenses: Current hub-level scanners are insufficient as a primary defense. They rely on static signatures and specific format support, missing novel logic flaws and 0-days.

Conclusion: Securely loading ML models is not a solved problem. The community must move away from relying on file formats or "safe" flags as guarantees. Instead, security requires a shift in mindset: treating all shared models as untrusted code, enforcing strict allowlists, and acknowledging that security hardening often comes at the cost of flexibility.