Technical Summary: Attacks on Approximate Caches in Text-to-Image Diffusion Models

Problem Statement

Text-to-image diffusion models are computationally intensive, requiring iterative denoising steps to generate high-quality images. To mitigate this overhead, recent industry and academic work (e.g., NIRVANA by Adobe) has adopted approximate caching. This technique reuses intermediate states from previous generations when a new prompt is semantically similar to a cached one, allowing the system to skip denoising steps.

While efficient, this optimization introduces a critical security vulnerability: it breaks the isolation between users. By storing and reusing intermediate states based on prompt similarity, the system creates a shared state that can be exploited. This paper posits that existing security research on diffusion models has largely focused on the model weights or training data, leaving the vulnerabilities of the serving system—specifically the approximate cache—underexplored.

Methodology

The authors replicate the state-of-the-art NIRVANA approximate cache system and deploy it with two leading text-to-image models: FLUX and Stable Diffusion 3 (SD3). They utilize real-world prompt datasets (DiffusionDB and Lexica) and a cloud-based GPU environment (H100 and A100) to simulate a realistic serving system.

The research identifies two fundamental Attack Primitives inherent to approximate caching:

1. Timing Channel (Attack Primitive 1): Cache hits result in significantly lower latency than cache misses because denoising steps are skipped. The magnitude of latency reduction correlates with the semantic similarity between the new prompt and the cached prompt.
2. Structural Similarity (Attack Primitive 2): Images generated from the same cached intermediate state retain higher structural similarity (measured by SSIM) compared to images generated from different states, even if the final prompts differ slightly.

Based on these primitives, the authors demonstrate three distinct attacks under a threat model where the attacker has no prior knowledge of cached prompts and only possesses standard user access to the serving system.

1. Remote Covert Channel

• Mechanism: A sender and receiver exchange messages asynchronously. The sender injects prompts containing specific, rare keywords (e.g., "Apricity") into the cache. The receiver probes the cache using the same keywords.
• Detection: The receiver uses a two-level detection process:
  • Latency Classifier: Determines if a probe hit any cached prompt.
  • Content Detector: Uses object detection (DINOv2) to verify if the output image contains a specific "marker" (e.g., a dog or McDonald's logo) embedded by the sender.
• Encoding: A cache hit with the marker encodes a '1'; a miss or a hit without the marker encodes a '0'.

2. Prompt Stealing Attack (CacheExposer)

• Goal: To recover a victim's original prompt without access to their generated images.
• Mechanism: The attacker sends a large volume of "probing" prompts to the system.
• Clustering: Using the timing channel, the attacker identifies which probes hit the cache. A structural classifier then groups these probes based on the similarity of their generated images, assuming probes in the same group hit the same underlying cached prompt.
• Recovery: An Embedding Predictor (using SGD) generates a target embedding that maximizes similarity to the grouped probes. A Prompt Recoverer (a GPT-2 based model) then converts this embedding back into text, reconstructing the victim's prompt.

3. Image Poisoning Attack (CachePoison)

• Goal: To inject the attacker's content (e.g., logos) into images generated by other users.
• Mechanism: The attacker first steals prompts using CacheExposer. They then use a Logo Injector (an attention-based model) to embed a logo description into the stolen prompt's embedding space.
• Execution: The attacker converts this poisoned embedding back to text and injects it into the cache. When a victim's prompt hits this poisoned cache entry, the system reuses the intermediate state, causing the victim's generated image to inadvertently include the attacker's logo, even though the victim never requested it.

Key Results

Covert Channel Performance

• Accuracy: The channel achieved 97.8% accuracy on FLUX and 95.8% on SD3 when combining latency and content detection.
• Stealth: Injected prompts remained in the cache for up to 44 hours (depending on cache size and replacement policies like LCBFU), allowing for long-term, asynchronous communication.
• Bandwidth: While lower bandwidth than synchronous channels, the asynchronous nature and high stealth make it a viable threat.

Prompt Stealing (CacheExposer) Performance

• Semantic Similarity: Recovered prompts achieved an average cosine similarity of 0.78 with the original prompts, significantly outperforming a naive baseline (0.67).
• Image Fidelity: Images generated from stolen prompts were nearly identical to the originals, with PSNR scores exceeding 25 and high SSIM scores.
• Efficiency: The attack required a median of 4,385 probing prompts per recovery, but this cost is amortized as multiple prompts can be recovered from a single probing session.

Poisoning (CachePoison) Performance

• Hit Rate: CachePoison achieved significantly higher hit rates than a naive direct-injection baseline (e.g., 3,646 hits vs. 2,624 for FLUX on DiffusionDB).
• Render Success: The attack successfully rendered logos in ~57% of hits for FLUX (vs. ~40% for naive injection). SD3 showed lower success rates due to weaker text-image alignment in early denoising steps.
• Persistence: The attack remained effective across different cache sizes and replacement policies, though smaller caches and FIFO/LRU policies increased the hit count by evicting competing entries faster.

Significance and Claims

The paper claims to be the first work to exploit security vulnerabilities specifically within the approximate caching mechanisms of text-to-image diffusion models. Its significance lies in:

1. Shifting the Attack Surface: It demonstrates that efficiency optimizations in serving systems can create severe security risks (covert channels, data theft, and content poisoning) that are independent of model architecture or training data poisoning.
2. Practicality: The attacks are performed remotely against a realistic cloud serving system, requiring no model access or training data manipulation.
3. Stealth and Persistence: The asynchronous nature of the covert channel and the long lifetime of cached entries make these attacks difficult to detect and mitigate.
4. Urgency for Defense: The authors argue that as approximate caching becomes standard in commercial deployments (e.g., by Adobe, Google, OpenAI), these vulnerabilities must be addressed before widespread adoption.

The paper concludes that while approximate caching improves performance, it fundamentally compromises user isolation, necessitating new defense mechanisms such as random cache selection, content filtering, and malicious behavior detection.