🤖 AI

LaCache: Robust Semantic Caching for LLM Serving

LaCache is a novel semantic caching scheme for LLM serving that enhances security against cache-collision attacks and improves response relevance by verifying the cache hits of both user queries and their first k speculatively decoded tokens.

Original authors: Jiacheng Liang, Yuhui Wang, Tanqiu Jiang, Ting Wang

Published 2026-08-04

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jiacheng Liang, Yuhui Wang, Tanqiu Jiang, Ting Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the internet as a massive, bustling library where a super-smart robot librarian (an AI) answers millions of questions every day. To keep things running fast and cheap, the librarian doesn't re-read the entire encyclopedia for every single question. Instead, they use a "Semantic Cache." Think of this as a sticky-note system: if you ask, "How do I bake a cake?" and someone else just asked, "What's the recipe for a cake?", the librarian grabs the answer from the sticky note instead of doing the work again. This saves time and money, making AI feel instant.

However, just like a sticky note on a public fridge, this system has a weak spot. A mischievous prankster could sneak in and write a fake note that looks almost exactly like a real one. If the librarian's system is too trusting, they might hand out the prankster's fake answer to the next person who asks a similar question. This is called a "cache-collision attack." The bad news is that current safety guards are like bouncers checking only the person's face at the door; a clever prankster can wear a perfect mask to get past them. The good news is that a new study proposes a smarter way to check the notes themselves, turning a potential disaster into a solved puzzle.

The Paper's Big Idea: LACACHE

The researchers behind this paper, from Stony Brook University, have built a new security system called LACACHE (LookAhead Cache). Their goal was to stop those sneaky pranksters from poisoning the AI's memory without slowing down the library.

Here is the clever trick they discovered: While a prankster has total control over the question they ask (they can write it however they want to trick the system), they have almost no control over the answer the AI generates. The AI is like a strict storyteller; once it starts a story, it has to follow the rules of grammar and logic to finish it. If the prankster tries to force the AI to tell a dangerous lie, the first few words of that lie will still sound weird or different from a normal, helpful answer.

How LACACHE Works (The "Look-Ahead" Trick)

Imagine you are checking a library book.

The Old Way: You just check the title. If the title looks similar to a book you've seen before, you grab it. A prankster can easily write a title that looks just like the real one.
The LACACHE Way: You check the title, and then you peek at the first few sentences of the story inside.

LACACHE does exactly this. When a request comes in, it doesn't just ask, "Does this question match a saved one?" It also asks, "Does the beginning of the saved answer match what the AI would naturally say for this question?"

If a prankster tries to trick the system, they might succeed in making the question look similar enough to get a match. But when the system peeks at the first 20 words of the answer, it sees a mismatch. The prankster's fake answer starts with something weird or dangerous, while the real answer starts with something helpful. LACACHE spots this difference immediately, throws away the fake note, and asks the AI to write a fresh, safe answer.

What They Found

The team tested this idea on different AI models and found some impressive results:

Stopping the Attack: In their tests, LACACHE reduced the success rate of these attacks to nearly zero. The pranksters couldn't trick the system anymore.
Keeping it Fast: Usually, adding extra security slows things down. But because LACACHE only checks the first 20 words (a tiny peek) and uses a lightweight helper AI to do the peeking, it barely slowed anything down. It kept over 90% of the speed benefits of the original system.
Beating the Competition: They compared LACACHE to other safety methods, like checking how "confused" the AI is or asking a second AI to judge if a question is bad. Those old methods failed against clever, adaptive attacks. LACACHE, however, worked against all of them because it relies on a rule the prankster simply cannot break: the AI's answer must make sense.

Why It Matters

The paper proves that you can have both speed and safety. By shifting the focus from checking the question (which the bad guy controls) to checking the answer (which the AI controls), LACACHE creates a "provable" shield. It's like realizing that while a thief can wear any mask, they can't change the fact that their hands are shaking when they try to open the safe. The researchers showed that this method is mathematically strong and works in the real world, offering a promising new direction for keeping our AI libraries safe from pranksters.

Technical Summary: LACACHE – Robust Semantic Caching for LLM Serving

1. Problem Statement

Semantic caching has become a standard mechanism for reducing latency and inference costs in Large Language Model (LLM) serving by reusing responses for semantically similar queries. However, this efficiency introduces a critical vulnerability: cache-collision attacks. In this attack vector, an adversary injects crafted queries ( $x^*$ ) designed to collide with legitimate victim queries ( $x$ ) in the semantic cache (i.e., $h(x) = h(x^*)$ ). Once the cache is poisoned with the adversary's chosen response ( $y^*$ ), subsequent legitimate requests for $x$ retrieve the malicious $y^*$ instead of the correct response.

Existing defenses are fundamentally reactive and query-centric. They attempt to filter adversarial inputs using signals like perplexity, query-response consistency, or LLM-based classification. These approaches are inherently limited because the adversary has full control over the adversarial query $x^*$ and can optimize it (e.g., via gradient-guided methods like GCG) to evade detection while maintaining low perplexity and fluency.

2. Methodology: LACACHE

The paper proposes LACACHE (LookAhead Cache), a principled redesign of semantic caching that shifts the integrity check from the query to the response. The core insight is an asymmetry in control: while the adversary controls the input query, they have significantly less control over the LLM-generated response, which must satisfy competing constraints of semantic coherence and autoregressive plausibility.

Core Mechanism:
Instead of indexing solely on the query embedding, LACACHE augments the cache entry with a lookahead key derived from the first $k$ tokens of the response. A cached response is served only if two conditions are met:

Cache-Key Check: The incoming query $x$ collides with a cached query $x'$ ( $h(x) = h(x')$ ).
Lookahead Check: The first $k$ tokens of the response for the incoming query ( $f_k(x)$ ) match the cached response's prefix ( $f_k(x')$ ) within a semantic threshold ( $h(f_k(x)) = h(f_k(x'))$ ).

Implementation & Optimization:

Draft LLM: To avoid the computational cost of generating the $k$ -token prefix using the main (expensive) LLM, LACACHE employs a lightweight "draft" LLM ( $g$ ) to generate the prefix.
Speculative Decoding: If the cache-key check passes but the lookahead check fails, the system falls back to full decoding. To minimize overhead, it uses speculative decoding to verify the draft tokens against the main LLM, accepting tokens where the main model's probability is high enough.
Ensemble Defense: Theoretical analysis suggests detection probability can be amplified to $1 - p_2^m$ by using an ensemble of $m$ independent indexing functions for the lookahead check.

Theoretical Guarantee:
The authors prove that under two empirically validated assumptions—Semantic Separability (adversarial responses diverge significantly from genuine ones) and Prefix Coherency (the response prefix reflects the full response's divergence)—LACACHE detects any cache-collision attack producing a semantically distinct response with probability at least $1 - p_2$ . This guarantee holds for any query-optimizing attack, including white-box scenarios.

3. Key Contributions

Shift in Integrity Verification: The paper introduces the first defense that moves the security check from the adversarial query (which is fully controllable by the attacker) to the LLM response prefix (which is constrained by the model's generation logic).
Provable Security: It provides a formal proof that LACACHE detects attacks with high probability, provided the adversarial response is semantically distinct from the genuine one.
Efficiency Preservation: The method integrates seamlessly with existing serving pipelines using draft models and speculative decoding, adding negligible latency.

4. Experimental Results

The authors evaluated LACACHE across diverse LLMs (Qwen3, Gemma3) and embedding models (BGE, MiniLM) against two state-of-the-art attacks: Key-Collision Attack (KCA) and Semantic Cache Poisoning (SCP).

Security Efficacy: LACACHE reduced the Attack Hit Rate (AHR) to near zero (0.00–0.04) across all configurations, effectively neutralizing both KCA and SCP-P (prompt injection with optimized prefix). In contrast, baseline defenses (Perplexity filtering, Query-Response consistency, LLM-as-Judge) failed to suppress all attack variants simultaneously, often leaving AHRs above 40–90%.
Utility Preservation: LACACHE preserved over 90% of the Benign Hit Rate (BHR), demonstrating that the lookahead check does not significantly degrade the utility of the cache for legitimate paraphrased queries.
Adaptive Attacks: Even when adversaries were granted full white-box access to the embedding model, the LLM, and the LACACHE defense parameters (optimizing for both cache collision and lookahead similarity), the attack success rate remained negligible (e.g., 0% for KCA, 2% for SCP-P). The authors attribute this to the inherent conflict between maximizing lookahead similarity and preserving the semantic intent of the malicious response.
Overhead: With optimizations (draft LLM and speculative decoding), the per-query latency was reduced by 1.6× (from ~108.5 ms without optimization to ~68.6 ms with optimization), while maintaining defense guarantees.

5. Significance and Claims

The paper claims to be the first work to exploit the inherent constraints on adversarial responses in cache-collision attacks to provide provable security. By leveraging the structural tension between the adversary's need to manipulate the query and the LLM's requirement to generate a coherent response, LACACHE offers a robust defense that is not brittle to query-side optimization.

The authors position this work as a promising direction for building robust semantic caching, noting that while the current guarantee applies to attacks producing semantically distinct responses (excluding subtle misinformation attacks that preserve semantic proximity), the structural approach provides a foundation for future unified integrity and privacy defenses.