💻 computer science

Request Segmentation and Metadata Leakage in Low-Rate GPUPower Telemetry: A Single-Platform Audi

This paper audits the feasibility of inferring GPT-2 request boundaries from low-rate GPU power telemetry on a single NVIDIA RTX 2080 Ti, revealing that while oracle-window classifiers can achieve high accuracy, standard passive detection struggles with prefill requests and yields low overall coverage, highlighting the gap between experimental conditions and real-world attack scenarios.

Original authors: Ali Hamza, Chih-Yi Chiu, Aziz Ahmad, Shezad Arian, Hassan iftikhar

Published 2026-10-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ali Hamza, Chih-Yi Chiu, Aziz Ahmad, Shezad Arian, Hassan iftikhar

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Modern computers are often monitored by software that tracks how much electricity they use, how hot they get, and how busy their processors are. This data, known as telemetry, is essential for keeping systems running smoothly and efficiently. However, just as a person's heartbeat can reveal whether they are running or sleeping, the electrical activity of a computer's graphics processor can inadvertently reveal what tasks it is performing. This creates a potential security risk: if someone can read these power measurements, they might be able to guess what information a computer is processing, even if they are not allowed to see the data directly. This field of study, which looks for hidden information leaking through physical signals like power consumption, is called a side-channel attack. The question researchers ask is whether the power readings available to ordinary software are detailed enough to expose sensitive details about the work a computer is doing.

A team of researchers set out to test this specific risk using a common graphics card found in many high-end computers. They focused on a scenario where an attacker could continuously watch the power usage of the card while it ran two different versions of a popular artificial intelligence model designed to generate text. The core challenge they investigated was not just whether the power usage changed when the model worked, but whether an observer could actually pinpoint exactly when a specific request started and ended. In a controlled experiment, the researchers knew exactly when the computer began and finished a task, but they wanted to see if a passive observer, who only saw a continuous stream of power numbers without any start or stop signals, could figure out the same boundaries. They also wanted to know if, once a request was found, the observer could tell which of the two models was being used or how long the computer spent generating a response.

The researchers conducted their study on a single graphics card, recording power data at a very high speed while the card processed hundreds of requests. They divided their data into three groups: one for building their detection tools, one for testing those tools, and a final group that remained hidden until the very end to serve as a strict test. Their first major finding was that identifying the exact moments a task started and stopped was extremely difficult. Out of nearly six hundred requests the system processed, their best automated detector could only match up with sixty of them. Furthermore, the detector completely missed a specific type of short task, failing to find any of the 117 instances where the computer only prepared data without generating new text. This meant that for the vast majority of requests, the observer was effectively blind to the fact that work was even happening.

However, the story changed when the researchers simulated a scenario where the observer already knew exactly when the tasks occurred. In this "oracle" scenario, where the start and end times were provided as a gift, the ability to identify which model was running jumped dramatically. When the researchers gave the system the correct time windows, it could correctly identify the model in nearly 90 percent of cases. This stark difference highlighted a crucial gap: the power signal itself contains enough information to identify the work, but the ability to find the work in the first place is the real bottleneck. The researchers found that the power readings changed relatively slowly, with the value shifting only about ten times every second, even though the computer was being asked to do thousands of things in that same time. This slow pace made it hard to catch the brief moments when the computer was working on short tasks.

The team also tested whether changing how they looked at the data could improve their success. They tried different methods to spot the start and end of tasks, including looking for sudden jumps in power or changes in how the computer's internal clock behaved. One new method managed to find more requests, increasing the success rate from a very low number to a slightly higher one, but this came with a significant downside: it also created many more false alarms, flagging periods of inactivity as if they were active work. The researchers concluded that while the power data does leak information about what the computer is doing, the practical ability to exploit this leak is severely limited by the difficulty of finding the requests in the first place. They demonstrated that simply having a high success rate on the few requests you happen to find is not the same as having a working attack that can monitor a system continuously.

Ultimately, this study serves as a careful audit of a specific security claim. It shows that while the power consumption of a graphics card does carry a signature of the work it is performing, the low speed of the available measurements and the difficulty of separating real work from background noise make it hard to turn this into a reliable spying tool. The researchers found that the signal is there, but it is buried under a mountain of missed opportunities and false alarms. They emphasized that security evaluations must report not just how well a system can guess the right answer when it finds the right moment, but also how often it misses the moment entirely. In this specific setup, the combination of missed requests and false alarms suggests that the risk of leaking detailed information through these power readings is lower than it might appear if one only looked at the successful guesses, but the risk is not zero. The work underscores the need to look at the entire process of detection and identification together, rather than focusing on just the final step of guessing the answer.

Technical Summary: Request Segmentation and Metadata Leakage in Low-Rate GPU Power Telemetry

Problem Statement
Software-accessible graphics processing unit (GPU) power telemetry, such as that provided by the NVIDIA Management Library (NVML), can potentially expose workload metadata to an observer. However, a critical distinction exists between experimental conditions and real-world attacks: in experiments, request boundaries (start and end times) are often known or provided by the victim, whereas an attacker must passively detect these boundaries within a continuous trace. This paper audits the impact of this distinction. It investigates whether the difficulty of request segmentation (localizing requests in a continuous stream) significantly limits the ability to infer metadata (specifically model identity and generated token count) compared to scenarios where boundaries are known (oracle windows). The study focuses on a single NVIDIA RTX 2080 Ti platform running two GPT-2 model sizes.

Methodology
The authors conducted a reanalysis of an archived dataset comprising 36 sessions (18 development, 8 validation, 10 test) containing 1,044 development, 465 validation, and 583 test monitored requests. The dataset includes GPT-2 Small and Large models under various conditions (preheat settings, CPU/GPU contention, background loads).

Threat Model: The observer is a passive, offline entity with access to continuous device-level telemetry (power, utilization, streaming-multiprocessor clock, temperature) via NVML. The observer has no access to victim timestamps, model labels, or output counts during the segmentation phase.
Detection Pipeline: An archived "power-and-time" detector was used to identify request boundaries. It combines normalized power, utilization, and clock signals ( $s = 0.55z_P + 0.30z_U + 0.15z_C$ ) with specific thresholds and gap-merging rules.
Classification: Random forest classifiers were trained on development sessions to predict model identity (Large vs. Small) and token counts.
Comparative Protocols:
1. Detected Windows: The classifier is trained and tested only on windows successfully matched by the passive detector.
2. Controlled Oracle: The classifier is trained on the exact same 144 development event IDs as the detected model but uses ground-truth boundaries (oracle windows) for both training and testing. This isolates the effect of boundary detection from training data quantity.
3. Full-Data Oracle: The classifier uses all observable development events with oracle boundaries.
Metrics: The primary metric is the "class-balanced all-request identification score" ( $A_{all}$ ), which accounts for missed requests (misses) in the denominator. False alarm rates and conditional balanced accuracy (on matched requests only) are also reported.
Ablations: The study tested feature subsets (time-only, amplitude-only), alternative detectors (power hysteresis, candidate-change heuristics), and normalization controls to assess their impact on detection coverage and inference accuracy.
Hardware Cadence Check: New physical measurements were taken to determine the rate of power value changes versus API call frequency.

Key Results

Detection Coverage is the Primary Bottleneck: The archived passive detector matched only 60 out of 583 test requests (10.29% coverage). All matches were generation requests; zero prefill-only requests were detected due to duration and overlap rules.
Inference Accuracy Disparity:
- All-Request Score: The detected pipeline achieved a class-balanced identification score of 0.076 over all requests.
- Oracle Impact: When using oracle boundaries with the same training events (144 IDs), the score jumped to 0.764. Using all available oracle training data raised this to 0.887.
- Conditional Accuracy: On the 60 requests that were successfully matched, the conditional balanced accuracy was 0.688 for the detected model and 0.694 for the controlled oracle. This indicates that once a request is located, the inference quality is similar; the massive gap in overall scores is driven by the inability to locate the requests, not poor classification of located ones.
Feature Contributions: Both amplitude (power distribution) and timing features retained conditional signal. Amplitude-only and timing-only ablations showed that neither feature group alone is sufficient to explain the signal, and they are correlated.
Alternative Detectors: A "candidate-change" detector increased the all-request score to 0.127 but at the cost of a significantly higher false alarm rate (1,280 per hour vs. 488 per hour).
Hardware Cadence: Despite >1,000 interface calls per second, the hardware exhibited only ~9.8 power-value changes per second, with a median interval between changes of ~100 ms. This low cadence limits the resolution for detecting brief operations (like prefill).
Token Prediction: For generation requests, the oracle model achieved a high $R^2$ (0.922), while the detected model achieved a much lower $R^2$ (0.300), again highlighting the coverage deficit.

Significance and Claims
The paper explicitly avoids claiming a general security guarantee or a new architecture-extraction attack. Instead, its significance lies in methodological rigor and transparent accounting:

Distinction of Stages: The study establishes that "low-rate power telemetry does not leak" is an insufficient claim. Leakage must be evaluated by combining request localization coverage, training supervision, inference quality, and false alarm rates.
Limitations of Single-Platform Audits: The results are specific to the RTX 2080 Ti, driver version, and workload configurations tested. They do not generalize to all GPU generations or cloud environments.
Evaluation Protocol: The paper advocates for reporting "misses" and "false alarms" alongside inference scores. It demonstrates that high conditional accuracy on matched requests can coexist with low overall security risk (due to low coverage) or high risk (if a better detector is found), depending on the segmentation capability.
No New Attack: The authors state they are not reproducing previous attacks (like DeepTheft or Energon) nor refuting their accuracy. The novelty is treating request localization as a separately measured attack stage and controlling for training event identities to isolate the impact of boundary detection.

In conclusion, the paper argues that while passive detectors on this specific platform miss most requests, the successfully located requests retain significant metadata signal. Therefore, the limitations observed are likely due to the specific detector implementation and hardware cadence rather than an inherent absence of leakage in the telemetry data itself.

Technical Summary: Request Segmentation and Metadata Leakage in Low-Rate GPU Power Telemetry

More like this