🤖 machine learning

One RNG to Rule Them All: How Randomness Becomes an Attack Vector in Machine Learning

This paper explores how vulnerabilities in pseudorandom number generators (PRNGs) across machine learning frameworks can be exploited as covert attack vectors and proposes **RNGGuard**, a tool that statically analyzes and dynamically replaces insecure random functions with secure implementations to protect machine learning systems.

Original authors: Kotekar Annapoorna Prabhu, Andrew Gan, Zahra Ghodsi

Published 2026-02-11

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Kotekar Annapoorna Prabhu, Andrew Gan, Zahra Ghodsi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Secret Ingredient: Why "Randomness" is the Achilles' Heel of AI

Imagine you are a master chef. To make your signature dish perfect, you don't just follow a recipe; you rely on "controlled chaos." You toss a handful of salt "just enough," you stir the pot in a random pattern to ensure even heat, and you pick a few secret spices at random to keep the flavor unique.

In the world of Artificial Intelligence (AI), randomness is that secret ingredient.

AI models use randomness for almost everything: picking which data to learn from, deciding how to "shuffle" information, and even adding "noise" to protect people's privacy. If the randomness is good, the AI is smart and fair. But if the randomness is "broken" or predictable, the whole kitchen falls apart.

The Problem: The "Rigged Dice" Scenario

The researchers at Purdue University discovered a scary reality: The "dice" that AI uses to make these random decisions are often rigged—sometimes by accident, and sometimes by design.

Think of it like this: Imagine a casino where the roulette wheel looks perfectly normal, but a clever thief has slightly weighted the ball so it always lands on red. To a casual observer, the game looks fair. But to the thief, the outcome is 100% predictable.

In AI, this happens because different software frameworks (the "toolkits" used to build AI) use different methods to generate random numbers. Some use "weak" randomness (like using the current time as a seed, which is easy to guess), and some use "insecure" randomness that isn't strong enough to protect private data.

If an attacker can predict the "random" numbers, they can:

Cheat the Privacy: They can "undo" the noise meant to hide your personal data, essentially seeing through a digital mask.
Sabotage the Brain: They can manipulate how the AI learns, making it biased, unfair, or causing it to fail at specific tasks.
Create Backdoors: They can hide "trapdoors" in the AI that only trigger when they want them to.

The Solution: RNGGUARD (The Digital Security Guard)

The researchers created a new system called RNGGUARD. Think of RNGGUARD as a high-tech, invisible security guard that stands over the AI chef's shoulder.

RNGGUARD works in two clever ways:

The Blueprint Check (Static Analysis): Before the cooking even starts, RNGGUARD scans the entire recipe book (the code). It looks for any part of the process where a "cheap" or "unreliable" dice might be used. If it finds a weak spot, it marks it for replacement.
The Live Auditor (Dynamic Analysis): While the AI is "cooking" (training), RNGGUARD performs live spot-checks. It’s like a food critic who occasionally grabs a spoonful of the sauce to check if the seasoning is actually random or if it’s being manipulated. If the "sauce" tastes suspicious (meaning the numbers aren't statistically random), RNGGUARD sounds an alarm and swaps in a high-quality, "cryptographically secure" version of the randomness.

Why This Matters

As we move toward a world where AI makes decisions about our medical diagnoses, our bank loans, and our privacy, we can't afford to have "rigged dice" in the system.

This paper proves that we need to stop treating randomness as a background detail and start treating it as a critical security feature. RNGGUARD is a step toward ensuring that when an AI says a decision is "random," it actually is.

Technical Summary: One RNG to Rule Them All

Title: One RNG to Rule Them All: How Randomness Becomes an Attack Vector in Machine Learning
Authors: Kotekar Annapoorna Prabhu, Andrew Gan, Zahra Ghodsi (Purdue University)

1. The Problem: The "Invisible" Attack Vector

Machine Learning (ML) relies heavily on randomness for critical processes, including data splitting, augmentation, weight initialization, optimization (SGD), and privacy-preserving mechanisms like Differential Privacy (DP) and randomized smoothing.

The authors identify a significant security gap: while randomness is fundamental, its implementation is often overlooked from a security perspective. Most ML frameworks use Pseudorandom Number Generators (PRNGs), which are deterministic algorithms. The authors highlight three core vulnerabilities:

Inconsistent Implementations: Variations in PRNG design across frameworks (PyTorch, TensorFlow, JAX), software libraries (NumPy), and hardware backends (CPU vs. GPU).
Insecure Seeding: Many systems use predictable seeds, such as system time or fixed strings, which allow attackers to reconstruct the entire sequence of "random" numbers.
Lack of Statistical Validation: Frameworks rarely perform rigorous statistical tests to ensure that transformations (e.g., turning a uniform integer into a Gaussian distribution) maintain the required mathematical properties.

An attacker exploiting these weaknesses can covertly degrade model performance, introduce backdoors, compromise privacy (by "denoising" DP gradients), or create false robustness certifications.

2. Methodology

The researchers employed a multi-stage approach to study and mitigate these risks:

Ecosystem Study: They performed a comprehensive audit of major ML frameworks (PyTorch, TensorFlow, JAX, Keras), DP libraries (Opacus, TFPrivacy), and Federated Learning platforms (FedML-AI, TFFederated). They analyzed their PRNG algorithms, seeding practices, and dependency chains.
Attack Taxonomy: They developed a new taxonomy classifying attacks based on the target (Seed Source, Library Dependencies, or the ML Framework) and the attacker's objective (Performance, Privacy, Fairness, Integrity, or Robustness).
System Design (RNGGUARD): To address these gaps, they developed RNGGUARD, a two-phase security system:
1. Static Phase: Uses CodeQL to perform semantic analysis of a library's source code. It identifies all functions that transitively depend on core PRNG primitives.
2. Dynamic Phase: Operates at runtime using Python’s dynamic nature to intercept and replace insecure function calls. It supports two modes:
  - Static Mode: Replaces insecure calls with secure implementations (e.g., injecting a /dev/urandom seed or using a Cryptographically Secure PRNG (CSPRNG) for DP).
  - Dynamic Mode: Performs real-time statistical auditing using the Kolmogorov-Smirnov (KS) test (for continuous distributions) and Pearson’s $\chi^2$ test (for discrete distributions).

3. Key Contributions

Comprehensive Ecosystem Audit: Revealed that many DP-ML frameworks default to insecure PRNGs and that seeding practices (like using system time) are highly vulnerable to brute-force attacks.
Novel Attack Taxonomy: Provided a structured way to understand how randomness manipulation can be used as a covert attack vector.
RNGGUARD System: A practical tool for ML engineers to secure randomness with low effort, capable of both enforcing policies and detecting distributional deviations.
Optimization Techniques: Introduced Asynchronous (ASN) and Randomized Asynchronous (RASN) modes to mitigate the high runtime overhead of real-time statistical testing.

4. Results and Evaluation

Effectiveness: RNGGUARD successfully detected and corrected a simulated attack where a weight initialization function was tampered with to return uniform instead of normal distributions.
DP Auditing: The authors demonstrated that existing DP auditing tools (like DeltaSiege) fail to detect privacy violations caused specifically by insecure PRNGs, proving the necessity of RNGGUARD.
Performance Overhead:
- Static Mode: Introduced moderate overhead (approx. 47% in non-DP settings).
- Dynamic Mode (Unoptimized): Introduced massive overhead (up to 650% in DP settings) due to the cost of statistical testing.
- Optimized Dynamic Mode (RASN): By running tests asynchronously and only auditing a subset of random sequences (every 10th sequence), the authors significantly reduced the overhead, making it practical for real-world training.

5. Significance

This paper shifts the paradigm of ML security from focusing solely on model inputs (evasion/poisoning) to focusing on the stochastic foundations of the training process itself. By demonstrating that randomness is a viable and covert attack vector, the authors provide a roadmap for building more resilient and trustworthy ML systems. RNGGUARD offers a scalable way to bridge the gap between the high-performance requirements of ML and the rigorous security requirements of cryptography.