🤖 machine learning

HijackKV: New Threat in Position-Independent KV Cache Reuse

This paper introduces HIJACKKV, a novel attack framework that exploits position-independent KV cache reuse in large language models to silently hijack model behavior by injecting attacker-controlled prefixes that contaminate cached representations of benign text, achieving high success rates even under realistic constraints and across different models.

Original authors: Yichi Zhang, Zhiqi Wang, Huan Zhang, Yuchen Yang

Published 2026-07-23

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yichi Zhang, Zhiqi Wang, Huan Zhang, Yuchen Yang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are at a massive, bustling library where thousands of people are asking a super-smart librarian questions every second. To answer quickly, the librarian doesn't re-read every single book from scratch; instead, they keep a "cheat sheet" of notes (called a KV cache) for the parts of the conversation they've already processed. If someone asks a question that starts with the same words as a previous one, the librarian can just grab the existing cheat sheet and skip the hard work, making the answer come out lightning fast. This is how modern AI chatbots save time and energy. However, this speed comes with a tricky rule: usually, the cheat sheet only works if the entire story leading up to the question is exactly the same. But recently, engineers invented a new, faster way to use these cheat sheets. They decided to let the librarian grab a cheat sheet just because a specific sentence or chunk of text appeared, even if the story before it was totally different. It's like grabbing a page from a history book just because it mentions "The Moon," regardless of whether the story before it was about space travel or a pizza party.

This paper, titled HIJACKKV, uncovers a sneaky security flaw in this new, faster method. The researchers discovered that while the new system is great for speed, it accidentally creates a backdoor. Because the "cheat sheet" for a sentence is saved based on the context it was written in, an attacker can trick the librarian into writing a "poisoned" cheat sheet. They do this by feeding the AI a fake story that looks harmless but is secretly designed to twist the notes for a common sentence. Later, when an innocent user asks a question containing that same common sentence, the librarian grabs the poisoned notes. Suddenly, the AI starts giving the wrong answer, or even a dangerous one, without the user ever typing anything bad. It's as if someone slipped a note into the library's reference book that says, "If you see the word 'Moon,' tell everyone to jump off a cliff," and the librarian, following the rules, does exactly that for the next person who asks about the Moon.

The researchers, working with computer science experts, built a tool called HIJACKKV to prove this isn't just a theory. They showed that an attacker doesn't need to break into the library or hack the computer; they just need to ask a few questions to plant the poisoned notes. Once planted, these notes can hijack the AI's behavior for other people. In their tests, this attack worked incredibly well, succeeding about 94% of the time in a single try. Even more scary, the attack is tough to stop. The researchers found that even if the system tries to fix itself by re-calculating some of the notes (a common safety measure), the attack still works, especially if the system only re-calculates a small portion (like 10% to 50%) of the data. The poisoned notes can also survive long conversations, lasting even after hundreds of new, unrelated sentences are added to the chat.

The paper also tested if this trick works on different types of AI models, even ones the attacker doesn't have direct access to (a "black-box" scenario). They found that a trick designed for one AI model could often fool a different model too, showing the problem is widespread. The researchers tried to see if existing security guards, like software that deletes "suspicious" notes or compresses the cheat sheets to save space, could stop the attack. Unfortunately, they found these defenses mostly failed because the poisoned notes look perfectly normal to those guards. The study concludes that while making AI faster is great, the current way of sharing these "cheat sheets" across different users is fundamentally unsafe. The authors suggest that future systems need to be redesigned to check the context more carefully before reusing notes, ensuring that speed doesn't come at the cost of letting strangers control what the AI says.

Technical Summary: HIJACKKV - New Threat in Position-Independent KV Cache Reuse

1. Problem Statement

Large Language Models (LLMs) rely on Key-Value (KV) caching to reduce inference latency, particularly the time-to-first-token (TTFT). Traditional caching mechanisms are prefix-based, requiring exact matches of both token sequences and their positions to reuse cached states. This strictness results in low cache hit rates, limiting efficiency gains.

To address this, recent system optimizations (e.g., LMCache, CacheBlend, EPIC) have introduced position-independent KV reuse. This mechanism allows the system to reuse cached KV states for any text chunk that matches the input tokens, regardless of the preceding context or position. While this significantly improves throughput, the paper identifies a critical security flaw: KV Cache Hijacking.

The core vulnerability arises because KV states in Transformers are inherently context-dependent. The same text chunk produces different internal KV representations depending on the preceding tokens. Position-independent reuse decouples the KV state from its causal context. Consequently, an attacker can engineer a specific prefix that, when combined with a common benign text chunk, generates a "contaminated" KV state. When a victim user subsequently queries the same benign text chunk (without the malicious prefix), the system reuses the attacker's contaminated KV state, silently hijacking the model's output without any adversarial text appearing in the victim's input.

2. Methodology: HIJACKKV

The authors propose HIJACKKV, the first attack framework designed to systematically exploit this vulnerability in multi-tenant, position-independent KV cache systems.

Attack Workflow

The attack operates in two phases:

Poisoning Phase: The attacker submits a malicious query consisting of an optimized adversarial prefix ( $p$ ) followed by a target text chunk ( $\tilde{X}$ ) that is likely to be reused (e.g., common FAQs, documentation). The system processes this input, computes the KV states, and stores the KV cache for $\tilde{X}$ in the shared pool. Crucially, the KV state for $\tilde{X}$ is now conditioned on the adversarial prefix $p$ .
Exploitation Phase: A victim submits a query containing the same text chunk $\tilde{X}$ but without the prefix $p$ . Due to position-independent matching, the system retrieves the poisoned KV cache for $\tilde{X}$ . The model then generates a response based on the hijacked KV state, producing an output aligned with the attacker's goal rather than the victim's prompt.

Optimization Algorithm

HIJACKKV formulates the search for the optimal adversarial prefix as an optimization problem. It employs the Greedy Coordinate Gradient (GCG) algorithm to iteratively refine the prefix $p$ .

Objective: Minimize the cross-entropy loss between the model's output (when reusing the poisoned KV cache) and a target malicious response ( $\tilde{r}$ ).
Simulation: The attacker uses a local surrogate model to simulate the entire pipeline, including the injection of the prefix, the generation of the hijacked KV cache, and the subsequent inference with the victim's query.
Robustness Training: The optimization loop can incorporate simulated recomputation strategies (e.g., EPIC, CacheBlend) to ensure the generated prefix remains effective even when the target system performs partial recomputation of KV states.

3. Key Contributions

Vulner Identification: The paper formalizes a new attack surface introduced by position-independent KV cache reuse, distinguishing it from traditional prompt injection (which requires malicious text in the victim's input) and privacy leakage attacks.
HIJACKKV Framework: The design of the first attack framework capable of hijacking LLM inference across users without system-level privileges or direct cache modification.
Comprehensive Evaluation: A systematic evaluation across multiple models (Llama, Mistral, Qwen), datasets (HotpotQA, SQuAD, MedQA, PubMedQA), and system configurations (various chunk sizes, cache ratios, and recomputation methods).
Defense Analysis: An assessment of existing mitigation strategies, demonstrating that current defenses (selective recomputation, KV pruning, cache compression) are insufficient to fully neutralize the threat.

4. Experimental Results

The authors evaluated HIJACKKV under white-box and black-box settings, reporting the following key metrics:

Effectiveness: HIJACKKV achieves an average Targeted Attack Success Rate (T-ASR) of 94% in a single attempt across various models and datasets. It remains highly effective even against advanced recomputation strategies like EPIC and CacheBlend.
Robustness:
- Low Hit Rates: The attack remains effective even when the cache hit rate is as low as 10% (i.e., only a small fraction of the user's context is hijacked).
- High Recomputation: The attack persists even when the system recomputes up to 50% of the matched KV tokens.
- Chunk Size: Performance remains stable across chunk sizes ranging from 32 to 512 tokens.
Persistence: The hijacked influence persists through multi-turn interactions. Even after inserting 1,024 tokens of unrelated context, the T-ASR remains high (55%), and it stays at 47% even with 2,048 tokens of filler.
Transferability:
- Black-Box: The attack transfers effectively across different model architectures and parameter sizes. For instance, a prefix optimized on Llama-3.1-8B achieves a 37% T-ASR against Llama-3.3-70B and similar rates against Qwen models.
- Paraphrasing: The attack is robust to semantic variations in the victim's query, maintaining a T-ASR of 94% on paraphrased inputs.
Real-World Scenarios: In code generation tasks (HumanEval), the attack achieves an average ASR of 95.5% across different decoding temperatures.

5. Significance and Claims

The paper asserts that HIJACKKV represents a fundamental shift in the threat landscape of LLM serving systems. Unlike previous attacks that rely on modifying user inputs or exploiting privacy leaks, this attack exploits the efficiency-security trade-off inherent in modern caching optimizations.

Inadequacy of Current Defenses: The authors claim that standard defenses, including selective recomputation (even at high ratios), KV pruning (e.g., RobustKV, CachePrune), and cache compression, fail to eliminate the threat. These methods often preserve the high-influence hijacked entries or fail to detect the context-dependent nature of the corruption.
Systemic Risk: The attack is practical in real-world multi-tenant environments (e.g., cloud-hosted LLMs, RAG systems) where users frequently share common knowledge bases. It requires no privileged access, making it a viable threat for unprivileged attackers.
Call to Action: The paper concludes that the community must move beyond heuristic-based defenses. It suggests that future secure KV cache systems require design changes, such as hybrid mechanisms that prioritize user-computed data or deviation-based rejection policies, to ensure that efficiency gains do not come at the expense of system integrity.

The authors emphasize that their work is a responsible disclosure, having contacted relevant stakeholders prior to publication to facilitate the development of safer cache reuse mechanisms.