🤖 AI

CSF: Black-box Fingerprinting via Compositional Semantics for Text-to-Image Models

This paper introduces Compositional Semantic Fingerprinting (CSF), a novel black-box method that attributes fine-tuned text-to-image models to their protected lineages by probing them with rare compositional prompts, thereby enabling IP enforcement in commercial API deployments without requiring internal model access or pre-deployment watermarking.

Original authors: Junhoo Lee, Mijin Koo, Nojun Kwak

Published 2026-04-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Junhoo Lee, Mijin Koo, Nojun Kwak

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you buy a famous, expensive recipe book from a chef. The book says, "Do not sell copies of this." But then, a thief steals the book, tweaks a few ingredients, and starts selling "their own" version of the dish through a secret online kitchen. You can't see their kitchen, you can't check their recipe book, and you can't taste the food before they sell it. You only get to order one dish at a time.

How do you prove that their dish actually came from your original recipe book?

This is the problem the paper CSF (Compositional Semantic Fingerprinting) solves for Text-to-Image AI models.

Here is the simple breakdown of how they did it, using some everyday analogies.

1. The Problem: The "Black Box" Kitchen

Currently, most AI image generators (like Stable Diffusion or Flux) are sold as "APIs." This means you can only send a text prompt (like "a cat") and get an image back. You cannot see the code, the weights, or the internal math.

Old Way (Watermarking): Imagine the chef tries to hide a tiny, invisible speck of gold dust in every cake they bake. If you find the gold dust, you know it's theirs.
- The Flaw: The thief can just filter the gold dust out, or the chef can't put the dust in if the cake is already baked (the model is already deployed).
Old Way (Fingerprinting): Imagine trying to identify the chef by looking at the microscopic texture of the flour in the cake.
- The Flaw: The thief changes the flour type, the oven temperature, and the mixing speed. The texture changes so much that you can't tell who made it anymore.

2. The Solution: The "Riddle" Test (CSF)

The authors realized that while a thief can change the style of the cake (making it look like a cartoon or a photo), they cannot easily change the deep-seated habits of the original chef's brain.

They treat the AI not as a picture-maker, but as a mind that interprets vague riddles.

The Analogy: The "Vague Riddle"

Imagine you ask a group of people: "Draw a dangerous animal."

Person A (The Chef): Might immediately think of a Lion because their brain is trained on specific wildlife books.
Person B (The Thief): Might think of a Shark or a Wolf because they read different books.

If you ask them a very specific but still vague riddle, like "Draw a dangerous animal in a dimly lit studio," the Chef's brain will still lean heavily toward the Lion, while the Thief's brain might lean toward a Wolf.

CSF works like this:

The Prompt: They send the AI a "compositional" prompt. This is a sentence that combines a broad category (Animal) with weird, specific, and rare details (Dangerous + Dimly lit studio).
The Rarity: They make sure these specific combinations are so rare that the thief likely never saw them while training their stolen model.
The Test: They ask the AI to generate 30 images for this one riddle.
The Fingerprint: They don't look at the pictures. They ask a smart classifier (a robot judge): "What kind of animal is this?"
- If the AI is the original Chef's model, 80% of the time it will say "Lion."
- If it's the Thief's model, it might say "Wolf" 80% of the time.

3. Why This is a "Superpower" for the Owner

The paper highlights a clever Asymmetry (a one-sided advantage):

The Defender (Owner): Can invent infinite new riddles. If the thief blocks "Lion in a studio," the owner just asks for "Lion in a swamp." The owner can keep generating new, rare combinations on the fly.
The Attacker (Thief): To stop the owner, the thief would have to guess every single possible combination of words and "unlearn" them. Since the combinations are infinite, the thief can never win. It's like trying to memorize every possible sentence in a language to stop someone from testing you; you can't do it.

4. The Results: It Works!

The researchers tested this on 6 major families of AI models (like Stable Diffusion, Flux, Kandinsky) and 13 different "stolen" versions (some changed to look like anime, some to look like photos, some merged together).

The Outcome: Even when the thief completely changed the visual style (e.g., turning a realistic model into an anime model), the deep semantic habits remained.
The Proof: When they ran the "riddle test," the system could identify the original "parent" model with over 90% accuracy, even when the thief tried to hide it.

Summary

Think of CSF as a DNA test for AI minds.

Instead of looking at the surface (the pictures), they ask the AI a series of weird, specific riddles. The way the AI thinks about these riddles reveals its true origin, even if the thief tries to paint over the AI's face with a new style. It's a way for creators to prove, "This AI is my child," without ever needing to see the AI's internal code.

1. Problem Definition

The paper addresses the critical challenge of Intellectual Property (IP) protection for Text-to-Image (T2I) models in commercial settings.

The Threat: Malicious actors often fine-tune protected base models (e.g., Stable Diffusion, Flux) and distribute them exclusively via APIs (black-box access). This allows them to bypass restrictive licenses (e.g., non-commercial use) while obscuring the model's lineage.
The Limitation of Existing Methods:
- Watermarking: Requires pre-deployment injection of triggers. It degrades model quality and can be removed or diluted by fine-tuning.
- Traditional Fingerprinting: Often relies on white-box or gray-box access (weights, activations, gradients), which is unavailable in commercial API scenarios.
- Naive Black-box Approaches: Direct image matching or visual feature clustering (e.g., using CLIP embeddings) fails because fine-tuning drastically alters visual styles (e.g., photorealistic vs. anime), causing models to cluster by style rather than by their underlying base model lineage.

Goal: Develop a black-box, query-only method to statistically attribute a suspect fine-tuned model to its protected base model lineage without access to internal weights or pre-deployment modifications.

2. Methodology: Compositional Semantic Fingerprinting (CSF)

CSF shifts the attribution problem from visual space (pixels/style) to semantic space (interpretation biases). It treats T2I models not as pixel synthesizers, but as semantic category generators.

Core Concept: Compositional Underspecification

The method exploits the fact that while fine-tuning overwrites common behaviors, it rarely touches rare, compositional conditions that were absent from the fine-tuning dataset.

Prompt Construction: CSF uses prompts that combine a superordinate category (e.g., "animal") with multiple semantic attributes (e.g., "dangerous," "dimmed studio").
- Template: "A photo of [Attribute 1] [Attribute 2] ... [Superordinate Category] [Context]."
- Rationale: While individual components (e.g., "animal") are common, their specific combinations are exponentially rare. Fine-tuning data rarely contains these specific joint conditions, meaning the model's response to them remains governed by the pre-training priors of the base model.
Asymmetry Advantage: Defenders can generate infinite new compositional prompts post-deployment. Attackers cannot anticipate or suppress the entire combinatorial space of these rare prompts.

Technical Pipeline

Query Generation: For a suspect model, CSF generates $N=30$ images for each of 42 distinct compositional prompts using different random seeds.
Semantic Extraction: Each generated image is passed through a CLIP zero-shot classifier to determine its probability distribution over a predefined set of semantic categories (e.g., distinguishing between "tiger," "lion," or "wolf" for the prompt "a dangerous animal").
Distribution Modeling: The model's fingerprint is defined as the empirical distribution of these categorical vectors across the 30 samples. This captures the model's inherent semantic bias.
Distance Metric: The method uses the Wasserstein Distance (Earth Mover's Distance) to compare the suspect model's distribution against reference distributions of known base models. Wasserstein is chosen over Jensen-Shannon Divergence (JSD) because it better captures the geometric structure of the probability simplex.
Bayesian Attribution:
- The attribution is framed as a probabilistic inference problem.
- A Beta-Binomial model aggregates evidence across the 42 prompts.
- Decision Criteria:
  - Significance Test: Is the attribution accuracy significantly better than random chance ( $1/K$ )?
  - Dominance Test: Is the candidate base model more probable than all other candidates combined (Posterior Mean > 0.5)?

3. Key Contributions

First Black-Box Fingerprinting: The first method capable of attributing fine-tuned T2I models to their lineage using only API access (text-in, image-out), requiring no internal model access or pre-deployment watermarking.
Compositional Semantics: Introduces a novel mechanism using underspecified compositional prompts to isolate persistent semantic biases that survive aggressive fine-tuning, style transfer, and model merging.
Statistical Rigor: Provides a Bayesian framework with controlled risk, offering confidence intervals and dominance tests rather than heuristic similarity scores.
Training-Free: The method requires no additional training data or model modification; it operates purely on inference.

4. Experimental Results

The authors evaluated CSF on 6 model families (Flux, Kandinsky, SD1.5, SD2.1, SD3, SDXL) and 13 fine-tuned variants (including LoRA, DPO, model merging, and distillation).

High Accuracy: All 13 fine-tuned variants satisfied the Dominance Test (Posterior Mean > 0.5), correctly identifying their base lineage with high confidence.
- Example: Flux-LoRA was correctly attributed to Flux-Base with a posterior mean of 0.932.
- Example: SD3-Reality-Mix (heavily retrained) was correctly attributed to SD3-Medium-Base with 0.705.
Robustness: The method remained effective against:
- Style Transfer: (e.g., Anime vs. Realistic).
- Model Merging: (e.g., "Cocktail" models blending multiple weights).
- Distillation: (e.g., SDXL-Lightning, which reduces steps to 4).
- Adversarial Removal: Even when specific concepts were erased using Unified Concept Editing (UCE), the broad semantic biases persisted.
Human Validation: In a "Name That Dataset" user study, participants using CSF prompts achieved 71% accuracy in identifying base models, compared to 18% with naive prompts (below random chance for 3-way classification).
Cost Efficiency: The cost to fingerprint a single model is approximately $50 (based on commercial API pricing), making it economically viable.

5. Significance and Impact

Enforceability of Licenses: CSF provides the missing link for enforcing restrictive licenses on commercial APIs. It allows IP owners to detect unauthorized derivatives even when they are distributed as "black boxes."
Paradigm Shift: It reverses the asymmetry of watermarking. Instead of attackers trying to remove a static trigger, defenders can dynamically generate new semantic probes that are intractable for attackers to suppress.
Future of IP Protection: As generative AI moves toward API-only distribution, CSF offers a practical, scalable, and mathematically sound solution for forensic attribution, ensuring that the provenance of AI models can be verified without compromising user privacy or requiring model transparency.

In summary, CSF transforms the problem of model fingerprinting from a visual pattern-matching task into a semantic bias detection task, leveraging the "frozen" nature of pre-training priors under rare compositional conditions to robustly identify model lineage in the most restrictive black-box settings.