🤖 AI

Fingerprinting Text-to-Image Diffusion Models via Collapsed Generation

This paper introduces a non-invasive framework for verifying ownership of text-to-image diffusion models by leveraging "collapsed generation," a model-specific phenomenon where certain input conditions consistently produce identical images across stochastic seeds, enabling reliable ownership detection in both white-box and black-box settings without invasive watermarks.

Original authors: Yuanmin Huang, Chen Chen, Geng Hong, Xiaoyu You, Hui Xue, Zhenxing Qian, Mi Zhang, Min Yang

Published 2026-08-13

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Yuanmin Huang, Chen Chen, Geng Hong, Xiaoyu You, Hui Xue, Zhenxing Qian, Mi Zhang, Min Yang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where artists don't just paint with brushes, but with magic words. You type a sentence like "a cyberpunk cat eating pizza," and a computer instantly conjures a brand-new, high-definition picture. This is the realm of Text-to-Image Diffusion Models, the digital wizards behind the latest AI art craze. These models work by starting with a screen full of static noise (like an old TV with no signal) and slowly, step-by-step, cleaning it up until a clear image emerges. Because these models are so powerful, companies are selling them as services or sharing the "recipe" (the model file) for others to use. But this creates a tricky problem: if someone steals a company's secret recipe or tweaks it slightly and sells it as their own, how does the original owner prove it's theirs? They can't just look inside the code if the thief hides it, and they can't always change the model to plant a secret digital watermark without messing up the art. They need a way to catch the thief just by watching what the model does.

This paper introduces a clever new detective trick called "Collapsed Generation." Think of it like a personality test for AI. Usually, if you ask an AI to draw a picture five times using the same words but a different random starting point (like rolling dice), you get five totally different, unique images. It's like asking a chef to make a burger five times; you'd expect five slightly different burgers. However, the researchers discovered that for certain specific words, some AI models get "stuck" in a loop. No matter how many times you roll the dice, the model produces almost the exact same image every time. It's as if the chef, when asked for a "burger," suddenly forgets how to vary the recipe and serves the identical burger five times in a row. The paper suggests that this weird, repetitive behavior is a unique fingerprint of that specific model's training. If you find a suspect model that produces these "identical burger" images when you ask the right questions, you've caught it using the original owner's secret recipe, even if the thief tried to disguise it.

The Detective's New Tool: Catching AI by Its Stutter

The researchers, a team from Fudan University and others, realized that this "stuttering" or collapsed generation isn't a bug; it's a feature of how these models learn. When a model memorizes certain patterns from its training data, it sometimes loses its ability to be creative for those specific prompts. Instead of exploring a wide range of possibilities, it collapses into a single, highly consistent outcome. The team figured out that this behavior is like a unique voice print. Just as two people might say the same sentence but with different accents, two different AI models trained on different data will react differently to the same prompt. One might produce a chaotic, diverse set of images, while the "stolen" one might produce a boringly identical set.

The paper proposes a two-step game of cat and mouse to prove ownership. First, the original owner (the detective) needs to find the "magic words" or "magic settings" that trigger this stuttering on their own model. They do this in two ways, depending on how much access they have to the suspect:

The "White-Box" Approach (The Insider): If the detective can see the model's internal gears (like if the thief leaked the code), they can use a computer to mathematically hunt for the perfect "embedding" (a digital version of a prompt) that forces the model to stutter. They don't need to wait for the final picture; they can see the model getting stuck early in the process, which saves a ton of computer power.
The "Black-Box" Approach (The Outsider): If the suspect model is hidden behind a website or an app where you can only type text and get a picture back, the detective can't tweak the internal gears. Instead, they mine through the original training data to find real, natural sentences that already cause the model to stutter. They found that these "stuttering prompts" often correspond to images the model learned very easily (low "loss" during training). By testing these natural sentences, they can see if the suspect model also stutters on them.

The Verdict: Is It a Copycat?

Once the detective has their list of "stuttering triggers," they test the suspect model. They ask the suspect model to generate images using these triggers, rolling the dice (changing the random seed) many times. If the suspect model is a copycat, it will produce a stack of nearly identical images, just like the original. If it's a totally different, unrelated model, it will produce a messy pile of different images, showing it doesn't share the same "memory" of those specific prompts.

The researchers tested this idea on several different types of AI models, including ones based on older "UNet" designs and newer "Transformer" designs. They found that this method works really well. It can tell the difference between two models that were trained separately, even if they look similar on the surface. More importantly, the fingerprint is tough to break. Even if the thief tries to hide their tracks by:

Fine-tuning (teaching the model new tricks),
Pruning (cutting out parts of the model to make it smaller),
Quantization (simplifying the numbers the model uses),
Or even trying to obfuscate the results,

...the "stuttering" behavior often survives. The paper shows that you don't need to ask millions of questions to catch a thief; a modest number of tests is enough to get a statistically strong answer.

What This Means (and What It Doesn't)

The paper is careful to say that this isn't a magic wand that solves every copyright problem instantly. It doesn't claim to stop theft from happening, nor does it suggest that every model has these fingerprints (though they found them in the ones they tested). It also notes that while the "white-box" method is very efficient, the "black-box" method relies on finding the right natural prompts, which can be a bit like searching for a needle in a haystack, though their method of looking at "low-loss" training samples makes it much faster.

Crucially, the authors argue against the idea that you need to plant a secret watermark inside the model to prove ownership. They show that the model's own natural behavior—its tendency to get stuck in a loop for certain inputs—is enough evidence. This is a big deal because it means you don't have to change the model to protect it. You can just watch how it behaves.

In the end, this research suggests that collapsed generation is a reliable, non-invasive way to verify who owns an AI art model. It turns the model's own quirks into a signature. If a model produces the same picture over and over again when it should be being creative, it might just be singing the original owner's song, no matter how hard the thief tries to change the lyrics.

Technical Summary: Fingerprinting Text-to-Image Diffusion Models via Collapsed Generation

Problem Statement

The rapid proliferation of proprietary text-to-image (T2I) diffusion models, distributed as hosted services or downloadable checkpoints, has intensified intellectual property (IP) disputes regarding model leakage, unauthorized copying, and derivative ownership. Existing IP protection mechanisms face significant limitations:

Invasive Watermarking: Methods that embed signals into model parameters or training objectives often introduce computational overhead, compromise generation quality, and cannot be applied retrospectively to already trained models.
Non-Invasive Fingerprinting: Current non-invasive approaches are often restricted by access assumptions. White-box methods relying on internal representations or denoising trajectories fail when the suspect model is a black-box API. Conversely, black-box methods relying on optimized prompt suffixes often produce linguistically abnormal queries that are easily detected or filtered, and their evidence is tied to specific semantic deviations rather than intrinsic model behavior.

There is a critical need for a non-invasive fingerprinting framework that works across both white-box (checkpoint) and black-box (API) settings, relies on intrinsic model behaviors rather than modified parameters, and remains robust against model obfuscations (e.g., fine-tuning, pruning, quantization).

Methodology

The authors propose a framework based on Collapsed Generation, a phenomenon where specific input conditions cause a diffusion model to produce highly consistent images across multiple stochastic seeds, suppressing the usual diversity of generation.

Core Concept: Collapsed Generation

Under normal conditions, independent stochastic samplings from a diffusion model yield diverse images. However, under certain model-dependent input conditions, the sampling stochasticity is suppressed, causing different generation trajectories to converge to visually similar outputs. The authors formalize this via cross-seed consistency, measured by the perceptual similarity (using SSCD scores) of $K$ images generated from the same condition but different initial noise seeds.

Fingerprint Construction

The framework constructs fingerprints differently based on the available access to the suspect model, but both rely on the same verification principle:

White-Box (Pipeline Access) – Continuous-Embedding Synthesis:
- Goal: Synthesize continuous conditioning embeddings that induce collapse.
- Challenge: Directly optimizing for final-image consistency is computationally expensive due to the need to differentiate through the entire denoising process.
- Solution: The authors observe that the stable structure of collapsed generations emerges early in the denoising process. They employ a truncated optimization strategy that minimizes the dispersion among intermediate latents after a few denoising steps ( $t_{trunc}$ ). This surrogate objective significantly reduces computational and memory costs while effectively synthesizing fingerprint embeddings ( $C_f$ ).
Black-Box (API-Only) – Natural-Prompt Mining:
- Goal: Identify natural language prompts that induce collapse without exhaustive search.
- Solution: The authors exploit the correlation between collapsed generation and low training loss. They filter the source model's training data to identify samples with unusually low denoising losses. These low-loss samples are highly enriched with collapsed prompts. The framework mines these prompts, evaluates their cross-seed consistency on the source model, and selects the top candidates to form a natural prompt fingerprint set ( $P_f$ ). This ensures queries appear as ordinary service interactions, enhancing stealthiness.

Verification Protocol

Process: The verifier applies the fingerprint conditions (embeddings or prompts) to the suspect model and generates $K$ images per condition.
Metric: Cross-seed consistency is calculated for each condition. The results are aggregated across the fingerprint set.
Decision: A right-tailed predictive t-test compares the aggregate consistency of the suspect model against a reference distribution of normal generation behavior established on the source model. A statistically significant high consistency ( $p_{val} < \tau_f$ ) indicates ownership.

Key Contributions

Identification of Collapsed Generation: The paper identifies collapsed generation as a model-intrinsic behavioral signature. It demonstrates that this behavior is specific to the learned generation process (influenced by architecture, data, and optimization dynamics) and is difficult for independently trained models to reproduce.
Unified Framework: A single framework bridges white-box and black-box verification. It synthesizes continuous embeddings for controllable pipelines and mines natural prompts for API-only services, both verified via the same cross-seed consistency metric.
Efficiency and Stealth:
- The truncated optimization strategy for white-box fingerprinting avoids the cost of full-denoising differentiation.
- The natural-prompt mining strategy for black-box verification avoids the linguistic abnormalities of adversarial suffixes, making queries indistinguishable from normal user interactions.
Robustness: The fingerprints remain verifiable in fine-tuned derivatives and under common obfuscations (quantization, pruning) and adaptive query-level interventions.

Results

Extensive experiments were conducted across UNet- and transformer-based diffusion models (including Stable Diffusion variants).

Discrimination: The fingerprints successfully distinguish between different source models with low confusion. Matched prompt-model pairs exhibit significantly higher cross-seed consistency than mismatched pairs.
Robustness: The method remains effective even after the suspect model undergoes fine-tuning, pruning, or quantization.
Budget: Verification requires only a modest query budget, making it practical for real-world deployment.

Significance and Claims

The paper claims to establish collapsed generation as a reliable, intrinsic evidence source for non-invasive diffusion model ownership verification. By leveraging a phenomenon that is inherent to the model's learned data distribution and optimization dynamics, the framework avoids the pitfalls of invasive watermarking (quality degradation, training overhead) and the limitations of existing non-invasive methods (access constraints, detectable queries).

The authors position their work as a solution to the growing tension in IP disputes, providing a mechanism that:

Does not require modifying the suspect model.
Works regardless of whether the model is exposed as a checkpoint or a hosted API.
Provides statistically rigorous evidence of ownership that is robust against common adversarial obfuscations.

The significance lies in shifting the paradigm from "embedding signals" to "eliciting intrinsic behavioral signatures," offering a versatile tool for protecting the intellectual property of generative AI models in diverse deployment scenarios.