Technical Summary: Broken Memories: Detecting and Mitigating Memorization in Diffusion Models with Degraded Generations

Problem Statement

Diffusion models have achieved remarkable success in high-quality image generation but face a critical privacy and copyright challenge: memorization. These models inadvertently replicate or produce images highly resembling specific training data, leading to potential data leakage. Existing mitigation strategies suffer from two primary limitations:

1. Dependence on Re-generation: Most current methods operate on a "detect-then-retry" basis. Upon detecting memorization, they abort the current generation and restart with modified inputs (e.g., altered prompts or embeddings), significantly increasing latency and computational cost.
2. Heuristic or Static Interventions: Mitigation often relies on fixed rules or static adjustments (e.g., fixed guidance scale reductions) that fail to adapt to the dynamic nature of the generation trajectory, often compromising semantic fidelity or image quality.

Furthermore, while some methods detect memorization via internal signals (e.g., attention weights or guidance magnitudes), they generally lack a principled theoretical link between these signals and the visual degradation often observed in memorized outputs.

Methodology

The authors propose a principled framework for on-the-fly detection and adaptive mitigation based on the observation that memorization induces numerical instability in the latent dynamics of diffusion models.

1. Theoretical Insight: Numerical Instability

The paper identifies that when generating memorized samples, the denoising trajectory often deviates from the model's stable behavior, manifesting visually as "broken" artifacts (e.g., distorted details). Drawing from numerical analysis, the authors posit that:

• Memorization as Instability: Memorized prompts cause the model to overfit to specific data points, resulting in excessively large gradients in the classifier-free guidance (CFG) mechanism.
• Latent Update Anomalies: In high-order solvers like PNDM (used in Stable Diffusion v1), these large gradients cause the latent update step ($\delta_t = z_t - z_{t+1}$) to exceed the bounds of a stable manifold, leading to oscillation or divergence in the trajectory.
• Sampler Agnosticism: This instability is not limited to PNDM; even with deterministic samplers like DDIM where final images may appear visually plausible, the internal latent trajectory still exhibits significant deviations from stability.

2. Empirical Stability Regions

To quantify this behavior without computing intractable Jacobian eigenvalues, the authors define Empirical Stability Regions:

• Latent Update Norm ($\|\delta_t\|$): A statistical region $R^{(t)}_\delta = [\mu_t - \gamma\sigma_t, \mu_t + \gamma\sigma_t]$ is constructed using the mean ($\mu_t$) and standard deviation ($\sigma_t$) of latent update norms from a small set of non-memorized (normal) prompts.
• Reconstructed Initial Latent ($\|\hat{z}^{(t)}_0\|$): Similarly, a stability region is defined for the reconstructed initial latent in DDIM to serve as a semantic stability proxy.

3. Detection Mechanism

The system monitors the generation trajectory step-by-step. At each step $t$, it computes a Z-score ($s_t$) measuring the deviation of the current latent update norm from the empirical stability region:
$$s_t = \frac{|\|\delta_t\| - \mu_t|}{\sigma_t}$$
A high $s_t$ indicates an anomalous update, signaling potential memorization. The maximum Z-score over the initial steps serves as the detection metric.

4. Adaptive Mitigation Strategy

Unlike static methods, this framework intervenes on-the-fly without restarting the generation. It employs a two-level adaptive strategy based on the severity of the detected instability:

• Mild Memorization: If the deviation is moderate, the method applies latent-magnitude constraints (rescaling $\hat{z}^{(t)}_0$ to the empirical mean) primarily in the early steps to regularize the trajectory.
• Strong Memorization: If the deviation is severe, stricter joint constraints are applied throughout the trajectory, enforcing both the latent norm ($\|z_t\|$) and the update norm ($\|\delta_t\|$) to remain within their respective stability regions.

This process acts as a streaming filter, projecting the latent state back into the stable manifold immediately upon detecting instability, ensuring seamless generation.

Key Contributions

1. Systematic Analysis: The first work to systematically link degraded memorization generation to numerical instability in diffusion sampling, revealing distinctive anomalies in latent update magnitudes.
2. Empirical Stability Regions: Introduction of a concept grounded in numerical stability theory to characterize and diagnose normal versus memorized generation, applicable across different samplers (PNDM, DDIM).
3. On-the-Fly Framework: Development of a diagnosis-driven framework that adaptively constrains generation trajectories only when instability is detected, surpassing existing heuristic or static methods.
4. Performance: Achievement of superior detection accuracy and complete elimination of memorized generations while preserving image quality with negligible overhead.

Experimental Results

Experiments were conducted on Stable Diffusion 1.4 (and extended to 1.5 and 2.1 in appendices) using datasets of memorized prompts (from LAION) and non-memorized prompts.

• Detection Performance: The method achieves an AUC > 0.999 and a high True Positive Rate at 1% False Positive Rate (TPR@1%FPR), outperforming state-of-the-art baselines (e.g., Ren et al., Wen et al., Jeon et al.) across various seed settings and scheduler types (PNDM and DDIM).
• Mitigation Performance:
  • Memorization Rate: The method reduces the memorization rate to 0.0% on the pretrained SD 1.4 model.
  • Image Quality: It preserves high semantic fidelity (CLIP Score) and image realism (FID), outperforming baselines that often suffer from over-mitigation (semantic drift) or under-mitigation (retaining artifacts).
  • Efficiency: The approach incurs negligible computational overhead (≈0.01s per image), avoiding the latency penalties associated with re-generation or iterative optimization found in other methods.

Significance and Claims

The paper claims that its primary significance lies in shifting the perspective of memorization mitigation from heuristic signal suppression to principled numerical stability. By grounding the detection in the statistical laws of normal image generation dynamics, the method provides a robust, model-agnostic, and time-invariant standard for identifying anomalies.

The authors emphasize that their approach uniquely combines detection, on-the-fly mitigation, and adaptivity in a single framework. This eliminates the need for costly re-generation and allows for seamless intervention that preserves the semantic intent and visual quality of the output. The work suggests that viewing memorization as a trajectory deviation problem offers a new, principled direction for building trustworthy generative models.