Technical Summary: Continuous Discovery of Vulnerabilities in LLM Serving Systems with Fuzzing

Problem Statement

Large Language Model (LLM) inference and serving systems (e.g., vLLM, SGLang) have become critical infrastructure for modern AI applications. These systems employ aggressive optimizations such as KV caching, dynamic batching, prefix sharing, speculative decoding, and multi-tenant scheduling to manage the compute and memory demands of large models. However, this complexity creates a distinct and under-tested vulnerability surface within the serving layer.

Current security testing primarily focuses on model behavior (e.g., jailbreaks, hallucinations) or individual API compliance. These approaches fail to capture failures arising from the software execution logic of valid inference requests when processed under realistic concurrent workloads. Recent empirical analysis indicates that over 35% of bugs in these systems manifest as non-crash anomalies rather than outright crashes. Specifically, faults in the serving stack can lead to:

• Silent state corruption: Cross-request contamination where one request's output is corrupted by another's cached state.
• Performance pathologies: "Noisy-neighbor" denial of service where valid requests cause severe latency amplification or throughput collapse for unrelated tenants without crashing the server.
• Delayed liveness failures: Crashes triggered only by the specific concurrent composition of valid requests, violating scheduler invariants.

Existing testing methods (model-level evaluations, standard API tests, and conventional fuzzers) do not systematically exercise the timing, overlap, and shared-state interactions that determine whether a serving system preserves isolation.

Methodology: GRIEF

To address this gap, the authors present GRIEF (GReybox Inference Engine Fuzzer), a greybox fuzzer designed specifically for LLM inference and serving systems.

Core Abstraction: Timed Request Traces

Unlike traditional fuzzers that mutate byte strings or isolated prompts, GRIEF treats timed multi-request traces as first-class inputs. A trace is a sequence of timestamped client-side events capturing:

• Request Content: The prompt and parameters.
• Scheduling Structure: When requests overlap, how they compete for shared state, and their lifecycle events (SEND, CANCEL, DISCONNECT, WAIT).
• Semantic Identity: Requests are grouped by prompt_family_id to allow GRIEF to vary timing and concurrency while holding prompt content fixed, isolating serving-layer divergence from prompt drift.

Fuzzing Loop and Mutations

GRIEF operates a greybox loop that generates, mutates, and executes traces against live servers, using runtime feedback (latency, resource usage, KV-cache events) to guide exploration. It employs three classes of trace-level mutations:

1. Timing Mutations: Perturbing event offsets to change co-batching, prefill overlap, or cache eviction timing.
2. Event Mutations: Inserting, deleting, or modifying lifecycle events to exercise cleanup, retry, and teardown paths.
3. Splicing Mutations: Combining segments from parent traces, including directed splicing that aligns cache-warming phases with high-pressure windows to expose race conditions.

Oracle and Confirmation Pipeline

Because LLM serving is noisy and partially non-deterministic, GRIEF uses a staged pipeline to distinguish bugs from infrastructure noise:

1. Behavioral Oracles: Detect external anomalies (e.g., severe latency regression, corrupted outputs, lifecycle violations).
2. Logprob-Assisted Relational Confirmation: Suspicious traces are replayed with deterministic decoding. GRIEF compares the original and replayed outputs. If they diverge, it checks the log-probability of the first divergent token. If the original token is not a near-tied candidate under the replay distribution, it confirms a serving-layer divergence (e.g., stale KV state) rather than benign decoding ambiguity.
3. Structural KV Forensics: For high-confidence attribution, GRIEF observes out-of-band KV-cache block lifecycle events to detect structural anomalies like cross-adapter block reuse or hash-content conflicts.

Key Contributions

1. Identification of a New Attack Surface: The paper identifies concurrent LLM inference serving as a security-relevant boundary where valid requests can trigger isolation, performance, and liveness failures.
2. GRIEF System: The design and implementation of a greybox fuzzer that treats timed multi-request traces as inputs, mutating request timing, lifecycle events, and serving-mode parameters.
3. Empirical Evaluation: The evaluation of GRIEF on vLLM and SGLang, discovering 15 potential vulnerabilities, 10 of which were confirmed by engine developers. This includes 2 assigned CVEs (e.g., CVE-2026-7141) and additional pending requests.

Results and Findings

The study categorizes discovered failures into three impact classes:

1. State Corruption and Isolation Failures

GRIEF discovered bugs where KV-cache isolation breaks, allowing cross-request output contamination.

• Symptoms: "Confident value pollution" (outputting a wrong value copied from another request), "reasoning-chain disturbance" (infinite loops or extended CoT), and "answer-first reasoning confusion" (generating an answer before reasoning).
• Mechanism: These failures occur without malformed inputs. The trigger request perturbs the serving state, causing the victim's output to be determined by the vulnerable schedule position rather than the victim's prompt alone.
• Verification: Controlled replay confirmed that corrupted outputs were bit-identical across different trigger prompts, ruling out prompt injection and confirming state reuse issues.

2. Performance Pathologies (Noisy-Neighbor DoS)

GRIEF identified a bug in vLLM where a single API-valid request shape causes severe cross-request interference.

• Impact: Victim time-to-first-token (TTFT) increased by 1,361× (from ~12ms to ~18s), and useful throughput collapsed to near zero.
• Mechanism: Unlike standard head-of-line blocking, eBPF traces revealed that the shared EngineCore coroutine was repeatedly descheduled by the OS. This halted the engine-driving path for all co-scheduled requests, effectively starving unrelated tenants while the server remained alive and error-free.

3. Crash and Liveness Failures

GRIEF found workloads that violate scheduler invariants only when valid requests are composed concurrently.

• Case Study: An SGLang LoRA-serving crash triggered by a specific combination of high KV-cache pressure, mixed prompt shapes, and bursty adapter arrivals.
• Mechanism: The scheduler's view of active adapters diverged from the loaded set, creating an invalid batch that triggered an assertion error. The crash was delayed, occurring after normal prefill and successful responses, making it difficult to diagnose without the full trace.

Significance and Claims

The paper argues that concurrency, caching, and scheduler behavior must be treated as first-class security and reliability boundaries for LLM infrastructure. The findings demonstrate that:

• Standard model safety tests and API compliance checks are insufficient for securing LLM serving systems.
• Valid, documented API requests can induce silent output corruption, cross-request interference, and availability loss without explicit server errors.
• The "serving layer" itself is a critical attack surface where bugs can manifest as subtle, non-crash anomalies that are invisible to traditional monitoring.

The authors conclude that future LLM infrastructure must integrate serving-layer fuzzing into routine testing and hardening processes to mitigate these risks before they are encountered in production. The work is presented as an early warning and a methodological shift toward treating the inference engine as a security-critical component, not merely a performance substrate.