Detecting Hidden ML Training With Zero-Overhead Telemetry

Technical Summary: Detecting Hidden ML Training With Zero-Overhead Telemetry

Problem Statement
Effective AI compute governance relies on the ability to verify when and where large-scale model training occurs. Current proposals for regulation (e.g., FLOP thresholds, registration, and audits) assume that regulators can reliably distinguish training workloads from inference or non-ML tasks. However, these schemes are vulnerable if developers can evade detection by disguising training as other workloads. While previous work has shown that ML and non-ML computing can be distinguished in non-adversarial settings, there has been no systematic evaluation of whether detection remains robust when an adversary actively attempts to obfuscate training signals. Furthermore, existing profiling-based monitoring solutions often incur prohibitive runtime overheads or require access to sensitive model data, raising privacy and operational concerns.

Methodology
The authors investigate the adversarial robustness of GPU workload classifiers using only zero-overhead, privacy-preserving telemetry from the NVIDIA Management Library (NVML).

• Data Collection: The study utilized 9 NVIDIA GPU models spanning four architecture generations (Pascal to Blackwell), including consumer (RTX 4050, 5080, 5090) and datacenter (A100, H100, H200, B200) hardware. The dataset comprises 1,404 runs of 162 distinct workloads, totaling approximately 117 GPU-hours and over 445,000 telemetry readings. Workloads included 93 ML training tasks (supervised, distributed, LoRA), 40 inference tasks, and 16 non-ML tasks (HPC, mining, rendering).
• Telemetry Signals: The classifier operates on nine raw NVML counters sampled at 1 Hz: GPU utilization, memory utilization, memory used, power draw, temperature, SM clock, memory clock, and PCIe TX/RX bandwidth. These signals are content-agnostic, observing only physical effects of computation without accessing weights, data, or hyperparameters.
• Classifier Architecture: A Random Forest classifier was selected as the optimal model. It employs a two-stage feature extraction pipeline:
  1. Structural Layer: Identifies workloads in a training-like operating region based on arithmetic intensity (mean power vs. mean utilization).
  2. Temporal Layer: Resolves ambiguities using time-domain features (autocorrelation, coefficient of variation, periodicity).
• Adversarial Evaluation (Red-Teaming): The authors conducted a five-round "monitor–evader" iteration. In each round, attackers introduced new evasion strategies targeting the classifier's weaknesses, and defenders retrained the classifier to harden against them. This process generated 20 families of evasion strategies, including utilization modulation, memory minimization, workload mimicry (e.g., FFT, mining), training-as-inference dilution, multi-GPU structuring, and white-box attacks where the adversary had access to the classifier's internal mechanics.

Key Contributions

1. Zero-Overhead Detection: The paper demonstrates that high-accuracy classification of training vs. non-training workloads is achievable using only standard, always-available NVML telemetry, eliminating the need for expensive profiling tools like Nsight Compute.
2. Adversarial Robustness Benchmark: Through five rounds of iteration, the authors stress-tested the classifier against a comprehensive suite of 20 evasion strategy families, establishing a benchmark for the "cat-and-mouse" dynamic in AI governance.
3. Hardware Generalization: The study validates that a classifier trained on a diverse set of hardware can generalize to unseen GPU models (Leave-One-GPU-Out evaluation) without re-fitting, supporting site-agnostic deployment.
4. Privacy-Preserving Mechanism: The approach confirms that governance-relevant questions can be answered without inspecting proprietary model weights or training data, addressing key legal and ethical barriers to cloud-based monitoring.

Results

• Baseline Performance: On non-adversarial workloads across all 9 GPU models, the classifier achieved 98.2% binary accuracy (macro-F1 = 0.982) in distinguishing training from non-training.
• Adversarial Performance:
  • Multi-GPU: The classifier detected all multi-GPU evasion strategies with ≥99.6% accuracy.
  • Single-GPU: Against the most challenging single-GPU evasion strategies (excluding white-box), detection rates remained high. In a "leave-one-strategy-out" evaluation, the classifier achieved a mean accuracy of 95.2% on unseen external strategies.
  • White-Box Attacks: Even when adversaries had access to the classifier's internals (Round 5), the hardened classifier maintained 43–87% accuracy against unseen white-box strategies. Only one specific evasion type (white-box LoRA) dropped below a 50% detection rate.
• Feature Importance: Physics-based features, specifically cumulative energy, power–memory correlation, and pre-allocation memory-plateau features, proved critical for detecting the most sophisticated evasion attempts. The authors note that the fundamental physical constraint of training (requiring ~2× the FLOPs and energy of inference) provides a signal that is difficult to suppress without aborting the training process.

Significance and Claims
The paper claims that the proposed zero-overhead telemetry approach provides a viable, empirically realizable foundation for AI compute governance. The authors argue that defenders are structurally favored because the physical energy requirements of the backward pass impose a fundamental constraint that evasion strategies cannot fully eliminate without sacrificing the ability to train the model.

The significance of the work lies in shifting the governance challenge from a purely machine learning problem to a hardware security problem. The authors assert that while their software-level classifier is robust, its real-world efficacy is conditional on the implementation of Hardware-Enabled Mechanisms (HEMs) that ensure the integrity of the telemetry data (e.g., tamper-resistant read paths and authenticated channels). If these hardware guarantees are met, the paper concludes that the economic and technical costs of evasion (such as dilution and idle-padding) become unviable at frontier scales, making the detection of unauthorized training feasible for regulators and cloud providers.