💻 computer science

Memorization In Stable Diffusion Is Unexpectedly Driven by CLIP Embeddings

This paper reveals that memorization in Stable Diffusion is unexpectedly driven by the structural duplication of the end-of-text embedding in padding tokens, which amplifies its influence, and proposes simple inference-time strategies to mitigate this issue without degrading image quality.

Original authors: Bumjun Kim, Albert No

Published 2026-05-06

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Bumjun Kim, Albert No

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The Model is "Stuck" on Memories

Imagine a talented artist (Stable Diffusion) who has been trained on millions of photos. Sometimes, when you ask this artist to draw something new, they accidentally copy a specific photo from their training set exactly, pixel-for-pixel. This is called memorization. It's a privacy and copyright risk because the model isn't creating something new; it's just regurgitating old data.

Scientists have been trying to figure out why this happens. Most theories focused on the words you type (the prompt) or the internal math of the model. This paper, however, found a surprising culprit hiding in the "padding" of the instructions.

The Setup: How the Artist Reads Instructions

To understand the discovery, we need to look at how the artist reads your instructions.

The Prompt: You type a sentence like "A cat on a mat."
The Limit: The artist can only read instructions up to a certain length (77 "tokens" or word-parts).
The Filler: If your sentence is short (e.g., only 10 words), the computer has to fill the remaining 67 empty spots to reach the limit of 77.

The Old Way (Stable Diffusion v1.4):
The computer fills those empty spots with a special "End of Text" token (let's call it <eot>).

The Analogy: Imagine you are reading a story, but the last page is blank. Instead of leaving it blank, the printer keeps stamping the words "THE END" over and over again until the page is full.
So, for a short prompt, the artist sees: [Your Words] + [THE END] + [THE END] + [THE END]... (repeated 60+ times).

The Discovery: The "Echo Chamber" Effect

The paper's authors discovered that the artist relies heavily on these repeated "THE END" stamps to decide what to draw, especially when memorizing images.

Here is the chain of events they found:

Training Mismatch: The system that translates words into math (called CLIP) was trained to treat the first "THE END" token as the most important summary of the whole sentence. It learned to ignore the actual words and the filler tokens.
The Glitch: Because the computer fills the empty space with more "THE END" tokens, the artist suddenly sees the "THE END" token repeated dozens of times.
The Amplification: The artist thinks, "Wow, there are so many 'THE END' tokens! This must be the most important part of the instruction!"
The Result: The model over-focuses on this repeated token. If that specific "THE END" pattern was associated with a specific copyrighted image during training, the model gets stuck in a loop and reproduces that exact image, ignoring your actual request.

The Metaphor:
Imagine a choir where the conductor (the model) is supposed to listen to the soloist (your prompt). But, the choir members are all shouting "STOP!" (the padding token) over and over. The conductor gets so overwhelmed by the shouting that they stop listening to the soloist and just start marching in a circle based on the "STOP!" commands. If the "STOP!" command was previously linked to a specific dance move, the choir just repeats that dance, ignoring the music.

The Surprising Twist: The Words Don't Matter as Much

The authors tested this by removing the actual words from the prompt and replacing them with the "End" token.

Result: The model could still generate a recognizable image.
Conclusion: The actual words you typed (the prompt) contribute very little to the memorization part of the process. The "filler" tokens are doing the heavy lifting for the copying behavior.

The Solution: Two Simple Fixes

The paper proposes two easy ways to stop this without retraining the whole model or slowing it down. Both are "inference-time" fixes, meaning you can apply them just before the image is generated.

Fix 1: Change the Filler Token

The Action: Instead of filling the empty space with "THE END" (<eot>), replace it with a neutral symbol like an exclamation mark (!).
Why it works: This breaks the echo chamber. The model no longer sees 60 copies of the "End" token. It sees a neutral symbol that doesn't carry the same heavy weight.
Bonus: They also hide (mask) the single original "End" token to be extra safe.

Fix 2: Mute the Filler

The Action: Keep the tokens as they are, but simply turn down the volume (mask) on the repeated "End" tokens so the model pays less attention to them.
Why it works: It reduces the influence of the "echo" without changing the tokenizer.

The Proof: Why Version 2.1 is Better

The authors noticed that Stable Diffusion v2.1 (a newer version) doesn't have this memorization problem as badly.

Why? It turns out the creators of v2.1 accidentally fixed this issue. They switched to a different text system (OpenCLIP) that uses a neutral symbol for padding instead of repeating "THE END."
The Takeaway: The fact that v2.1 naturally stopped memorizing so much proves that the "repeated End token" was indeed the main cause of the problem in v1.4.

Summary

The Problem: Stable Diffusion sometimes copies training images exactly.
The Cause: Short prompts get filled with repeated "End of Text" tokens. The model mistakes these repeated tokens for the most important part of the instruction, causing it to "overfit" and memorize specific images.
The Fix: Change the filler token to something neutral (like !) or mute the repeated tokens. This stops the memorization while keeping the image quality high.
The Benefit: You can use this fix immediately without needing to retrain the AI or detect which images are risky beforehand.

Technical Summary: Memorization In Stable Diffusion Is Unexpectedly Driven by CLIP Embeddings

Problem Statement

Text-to-image diffusion models, particularly Stable Diffusion, exhibit a critical privacy risk known as memorization, where the model reproduces specific images from its training set in response to certain prompts. This raises significant concerns regarding copyright violations and the leakage of private data. While prior research has investigated memorization through dataset duplication, trigger tokens, and cross-attention patterns, the role of the text embedding space—specifically how different components of the embedding sequence contribute to memorization—remains underexplored. This paper posits that the root cause of memorization in Stable Diffusion v1.4 lies not merely in the tokens themselves, but in a structural misalignment between the training dynamics of the CLIP text encoder and the inference mechanisms of the diffusion model.

Methodology

The authors conduct a detailed investigation into the embedding vectors of Stable Diffusion, categorizing input tokens into four types: <startoftext> ( $v_{sot}$ ), <prompt> tokens ( $v_{pr}$ ), <endoftext> ( $v_{eot}$ ), and <pad> tokens ( $v_{pad}$ ).

1. Empirical Analysis of Embedding Contributions

The study employs controlled inference-time interventions to quantify the influence of each embedding component on memorized prompts. The experiments involve:

Masking and Replacement: Systematically masking or replacing specific embedding vectors (e.g., setting $v_{pr}$ or $v_{pad}$ to zero vectors, or replacing them with $v_{eot}$ ).
Metric Evaluation: Using SSCD (Self-Supervised Contrastive Distance) to measure memorization (similarity to the original training image), CLIPScore for text-image alignment, and Aesthetic Score for visual quality.
Dataset Curation: The authors curated a robust set of 458 "Matching Verbatim" (MV) prompts from Webster [35] and Membench [13], filtering out unstable or degraded cases to ensure reliable analysis.

2. Comparative Analysis (v1.4 vs. v2.1)

The paper compares Stable Diffusion v1.4 (which uses the original CLIP text encoder) with v2.1 (which uses OpenCLIP). The authors hypothesize that differences in how these models handle padding tokens explain the variance in memorization rates between versions.

3. Mitigation Strategies

Based on the findings, two inference-time mitigation strategies are proposed:

Tokenizer-Level Fix: Replace the default <pad> token (which is <eot> in v1.4) with a semantically neutral token (e.g., !) before embedding, and mask the single $v_{eot}$ .
Partial Masking: Partially mask the $v_{pad}$ embeddings (specifically 70%) to reduce redundancy without collapsing generation quality.

Key Findings and Contributions

1. The Dominance of Padding Embeddings

Contrary to the assumption that prompt tokens ( $v_{pr}$ ) are the primary drivers of generation, the authors find that $v_{pr}$ contribute minimally to memorization. In fact, masking $v_{pr}$ or replacing them with $v_{eot}$ preserves the generated image structure. Conversely, padding embeddings ( $v_{pad}$ ) play a dominant role. When $v_{pad}$ are masked, generation collapses, indicating they are critical for reproducing memorized content.

2. Structural Duplication of $v_{eot}$

The core mechanism identified is the structural duplication of the <endoftext> embedding.

In Stable Diffusion v1.4, prompts shorter than the fixed length (77 tokens) are padded by repeating the <eot> token.
Since CLIP is trained via contrastive learning to optimize $v_{eot}$ as the representative of sentence-level semantics, the repeated <eot> tokens result in multiple near-identical $v_{eot}$ vectors (as $v_{pad}$ ).
This unintentional duplication amplifies the influence of $v_{eot}$ during the diffusion process, causing the model to over-rely on this specific embedding and leading to memorization.

3. Explanation of Stable Diffusion v2.1 Behavior

The paper notes that Stable Diffusion v2.1 exhibits significantly lower memorization rates. The authors attribute this not to dataset de-duplication, but to the use of OpenCLIP, which utilizes a neutral ! token for padding instead of duplicating <eot>. This structural change removes the amplification of $v_{eot}$ , thereby reducing memorization.

4. Proposed Mitigation Strategies

The authors propose two effective, plug-and-play strategies that do not require model retraining:

Strategy 1: Replace the tokenizer's default <pad> with a neutral token (!) and mask the original $v_{eot}$ . This decouples $v_{pad}$ from $v_{eot}$ .
Strategy 2: Partially mask 70% of the $v_{pad}$ embeddings.
Both methods significantly reduce SSCD scores (indicating reduced memorization) while maintaining high CLIPScore and Aesthetic Scores, effectively restoring diversity across random seeds.

Results

Memorization Suppression: The proposed methods reduce SSCD scores from ~0.5 (indicating memorization) to ~0.08–0.10, effectively preventing the reproduction of training images.
Quality Preservation: Unlike prior methods (e.g., Ren et al. [24], Wen et al. [36]) that often degrade visual quality or fail to suppress memorization consistently, the proposed strategies maintain prompt alignment and image quality.
Efficiency: The mitigation adds negligible computational overhead, with inference times nearly identical to standard generation.
Robustness: The methods perform consistently across memorized and non-memorized prompts, showing no significant degradation in general generation quality.

Significance and Claims

The paper claims to shift the perspective on memorization from model internals (e.g., cross-attention patterns) to the structural properties of text embeddings. It argues that the mismatch between CLIP's training objective (optimizing only $v_{eot}$ ) and the diffusion model's inference (conditioning on the full sequence including duplicated $v_{eot}$ ) creates a structural bias that drives memorization.

The significance lies in identifying a previously overlooked mechanism: the unintended amplification of the <endoftext> embedding due to tokenizer design. By addressing this structural flaw, the authors provide simple, effective, and deployable solutions that mitigate memorization without the need for dataset filtering, prompt tuning, or model retraining. The findings suggest that future diffusion models should carefully consider padding token strategies to avoid similar structural biases.