🤖 machine learning

Fingerprinting Inference Systems of Large Language Models

This paper reveals that subtle numerical deviations caused by specific inference system components (such as engines, backends, and hardware) propagate to LLM outputs, enabling a new fingerprinting method to reliably identify these underlying systems and demonstrating that completely preventing such identification is fundamentally difficult.

Original authors: Anna Wimbauer, Jonas Möller, Erik Imgrund, Konrad Rieck

Published 2026-05-29

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Anna Wimbauer, Jonas Möller, Erik Imgrund, Konrad Rieck

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: The "Digital Fingerprint" of a Computer

Imagine you order the exact same meal at three different restaurants. Even if they use the same recipe, the food might taste slightly different. One chef chops the onions a tiny bit finer, another uses a slightly hotter pan, and a third stirs the pot in a different order. To a casual eater, the meal looks the same. But to a professional food critic, those tiny differences reveal exactly which kitchen cooked the meal.

This paper argues that Large Language Models (LLMs) work the same way.

When you ask an AI a question, the answer isn't just generated by the "brain" (the model). It is also shaped by the "kitchen" (the inference system) where the cooking happens. This kitchen includes:

The Engine: The software that manages the conversation (like vLLM or SGLang).
The Backend: The specific math tools used to process attention (like FlashAttention).
The Hardware: The physical computer chips (like an Nvidia A100 vs. an H100 GPU).

Even though the math is supposed to be the same, computers don't always calculate numbers in the exact same order. Because of this, tiny, invisible errors (called numerical deviations) happen. The paper shows that these tiny errors are unique to the specific combination of software and hardware used.

How the Attack Works: The "Detective's Quiz"

The researchers created a method to "fingerprint" these systems. They act like a detective who doesn't have access to the kitchen, but can only ask the chef questions and read the menu.

The Strategy:
The detective uses four specific types of "traps" (prompts) to force the computer to reveal its secrets:

The Rare Token Trap: Asking the AI to recall a weird, strange symbol hidden in a sentence. Tiny math errors can make the AI guess the wrong symbol.
The Yes/No Trap: Asking a simple question that requires only one step of thinking. This isolates how the computer reads the question before answering.
The Long Story Trap: Giving the AI a very long text and asking for a specific number inside it. This forces the computer to split the work into chunks, revealing how it handles large tasks.
The Repetition Trap: Asking the AI to repeat a sentence 100 times. This checks how the computer remembers what it just said (using a "cache").

By feeding these traps to the AI and analyzing the tiny differences in the answers, the researchers built a classifier (a smart sorting machine) that can say with high accuracy: "This answer was cooked on an A100 chip using the vLLM engine."

What They Found

The researchers tested this on many different combinations of software and hardware.

Perfect Accuracy (The "Zero Temperature" Case): When the AI is set to be very strict and deterministic (no randomness), the fingerprinting works 100% of the time. It's like a perfect match.
Good Accuracy (The "Chat" Case): When the AI is allowed to be a bit creative (randomness is turned on), the fingerprinting is still very effective, getting it right 76% to 80% of the time.
Robustness: The method works even if the AI is asked to handle different amounts of data at once or if the system prompt (the AI's instructions) changes slightly.

Why Should We Care? (The Security Risk)

The paper highlights a serious security problem.

Imagine a hacker wants to break into a bank. They need to know if the bank uses a specific type of lock (a specific software engine) to know which key to pick.

The Vulnerability: If an attacker can identify the "kitchen" (the inference engine and hardware) just by talking to the AI, they can look up known weaknesses (bugs) in that specific software.
The Result: They can then launch a targeted attack. The paper notes that critical vulnerabilities have already been found in popular engines like vLLM and SGLang. Identifying the system is the first step to exploiting it.

Can We Stop It?

The paper concludes that stopping this is very hard.

The Dilemma: To stop the fingerprinting, you would have to make every computer calculate numbers in the exact same way, no matter the hardware or software. But this would destroy the benefits of having different, specialized tools (like faster chips or optimized software).
The Trade-off: You could add "noise" (randomness) to the answers to hide the fingerprint, but this would make the AI less reliable for tasks that need perfect precision, like coding or math.
The Best Defense: The easiest defense is simply rate limiting. If you limit how many questions a user can ask, it becomes too slow and expensive for a hacker to collect enough data to build a fingerprint.

Summary

This paper reveals that the "kitchen" where an AI cooks its answers leaves a unique, detectable taste in the food. By analyzing these tiny tastes, attackers can identify exactly what hardware and software the AI is running, potentially allowing them to target specific security weaknesses. While we can't easily fix the root cause without sacrificing performance, we can make it harder for attackers to gather the data they need.

Technical Summary: Fingerprinting Inference Systems of Large Language Models

Problem Statement

The behavior of Large Language Models (LLMs) is not determined solely by the model weights. The inference system—comprising the inference engine, attention backend, and hardware platform—introduces subtle numerical deviations due to the non-associative nature of floating-point arithmetic. While these deviations are typically treated as harmless artifacts hindering reproducibility, this paper posits that they leak critical information about the underlying infrastructure.

Specifically, an adversary who can observe the prompt-response behavior of an LLM (without access to internal states or log-probabilities) can infer the specific components of the inference stack. This leakage has direct security implications: knowing the specific engine (e.g., vLLM, SGLang) allows an attacker to target known critical vulnerabilities in those implementations, while identifying the hardware platform (e.g., A100 vs. H100) enables verification of service claims or targeted hardware-specific attacks.

Methodology

The authors propose a fingerprinting method that treats the inference system as a black box, analyzing only the textual input and output. The approach consists of two main stages:

1. Threat Model

The attacker is assumed to have:

Chat access: The ability to submit arbitrary prompts and observe text responses.
Knowledge of the model: The specific LLM and system prompt are known (recoverable via existing model fingerprinting or prompt extraction attacks).
Offline reference access: The ability to instantiate and query a set of reference systems covering the candidate components to collect training data.
No access to logits: Internal probability distributions or hidden states are inaccessible.

2. Prompt Set Construction

To isolate numerical deviations at different computational stages, the authors construct four distinct prompt sets ( $P_1$ to $P_4$ ):

$P_1$ (Rare Token Prompts): Contains rare tokens (e.g., non-standard Unicode) embedded in text. These test autoregressive decoding, where minor deviations in low-probability token selection can shift the final prediction.
$P_2$ (Binary Decision Prompts): Designed to elicit a single "Yes" or "No" token. This isolates the prefill stage, as there is no autoregressive generation to mask the initial processing differences.
$P_3$ (Long-Context Prompts): Long texts requiring the model to extract a specific number. These trigger chunked prefill, revealing deviations introduced by splitting long contexts across multiple passes.
$P_4$ (Repetition Prompts): Requests to repeat a sentence multiple times. This triggers KV-cache prefix reuse, exposing how cached states are managed and loaded.

3. Feature Extraction and Classification

Embedding: Responses are converted into feature vectors using scoring functions. For example, $P_1$ and $P_3$ use indicator functions (1 if the expected token/number is present, 0 otherwise), while $P_4$ normalizes the repetition count.
Classification: The resulting feature vectors are concatenated and fed into Random Forest classifiers trained to identify three target labels: the Inference Engine, the Attention Backend, and the Hardware Platform (GPU).
Stochastic Handling: For non-zero temperature settings, the method aggregates predictions from multiple samples ( $k$ ) per prompt using majority voting to mitigate sampling noise.

Key Results

The authors evaluated their approach on 30 distinct system configurations combining 4 inference engines, 6 attention backends, and 3 hardware devices (Nvidia A100, H100, L4) across three models (Llama-3.2-3B, Qwen2.5-7B, Qwen3-4B).

Deterministic Decoding (Temperature = 0)

Perfect Accuracy: The method achieved 100% accuracy in identifying all three components (engine, backend, GPU) across all models.
Prompt Efficiency: Perfect identification was possible with as few as 113 prompts total, though the optimal composition of prompts varied by model. Inference engines were identified fastest (reliable with ~~10 prompts), while backends and GPUs required more data (~~100 prompts).

Stochastic Decoding (Temperature > 0)

Robustness: Accuracy decreased as temperature increased due to sampling noise, but remained significant.
- Inference Engine: Averaged ~80% accuracy across temperatures ( $T \in \{0.3, 0.6, 0.9\}$ ).
- Attention Backend & GPU: Accuracy ranged from 46% to 78% depending on the model and temperature.
Aggregation: Performance saturated after aggregating 10–20 samples per prompt, confirming that majority voting effectively recovers the signal.

Robustness Analysis

The fingerprinting remained effective under realistic deployment variations:

System Prompts: Accuracy remained high (~99.7%) even when using application-specific system prompts instead of defaults.
Batch Size: The signal generalized well across different batch sizes (training on 32/64/128, testing on 256), with only marginal accuracy drops.
Proprietary Components: When a component (e.g., a specific GPU) was unseen during training, identification of the other components (engine, backend) remained robust, though identifying the unseen component itself dropped to random chance.
Temperature Mismatch: If the training and deployment temperatures differed significantly, accuracy dropped substantially, suggesting attackers should align their training temperature with the target.

Significance and Claims

The paper claims to be the first method to fingerprint LLM inference system components (engine, backend, hardware) using only prompt-response pairs without access to logits.

The authors emphasize that:

Security Implications: This leakage is not merely a curiosity; it provides adversaries with the precise knowledge required to exploit known vulnerabilities in widely used inference engines like vLLM and SGLang.
Fundamental Hardness of Defense: Preventing this fingerprinting is "fundamentally hard" because it would require eliminating numerical differences between diverse hardware and software stacks. Unifying the stack is unrealistic for production, and adding noise to mask the signal degrades utility (e.g., breaking deterministic decoding required for code generation).
Partial Mitigations: The paper suggests that rate limiting is the most practical defense, though it can be evaded via Sybil attacks. Adding noise is possible but comes at a high cost to application utility.

The work concludes that machine learning outputs are inextricably tied to their execution environment, and ignoring this link leads to information leakage that compromises system integrity and security.