💻 computer science

Vision-Encoder Behavioral Fingerprints of Image-to-Image Generative Models: A Training-Paradigm-Driven Taxonomy of Six Commercial APIs

This paper proposes a training-paradigm-driven taxonomy of six commercial image-to-image generative models, demonstrating that they partition into two distinct behavioral clusters—edit-trained versus sampling-time adapted—based on their responses to content-adaptive sub-JND adversarial perturbations measured via frozen DINOv2 token distances.

Original authors: Hunter Hill

Published 2026-06-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Hunter Hill

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very special, invisible ink that you can paint onto a photo. This ink is so faint that the human eye can't see it, but it leaves a unique "fingerprint" on the digital DNA of the image.

This paper is about what happens when you take photos painted with this invisible ink and run them through six different popular AI tools that are designed to edit or redraw images (like turning a sketch into a photo, or changing the style of a picture).

Here is the simple breakdown of what the researchers found:

1. The Two "Personality" Groups

The researchers tested six different commercial AI systems. They expected the results to depend on how the AI was built (its architecture). Instead, they found that the AIs sorted themselves into two distinct "personality groups" based entirely on how they were trained:

The "Careful Editors" (The Tight Band): These AIs (like Flux Kontext, Qwen Edit, and Gemini) were trained specifically to follow instructions to edit an existing photo. When you give them a photo, they try to keep it as close to the original as possible.
- The Analogy: Think of these like a photographer using Photoshop. They tweak the lighting or colors, but they keep the original photo's structure and details intact. They preserve your invisible ink fingerprint almost perfectly.
The "Heavy Regenerators" (The Drift Band): These AIs (like SDXL, SD3, and gpt-image-1) were originally trained to create images from scratch (text-to-image) and were just adapted to edit photos later. When you give them a photo, they tend to "re-imagine" it from scratch based on their training.
- The Analogy: Think of these like a painter looking at a photo and painting a new version of it. They might capture the general vibe, but they change the brushstrokes and details significantly. In doing so, they wash away your invisible ink fingerprint and replace it with their own "painter's signature."

2. The Training Matters More Than the Brand

The most surprising finding is that the "brand" or the specific type of technology (like whether it's a diffusion model or a language model) didn't matter as much as the training method.

Even though two AIs might use the same underlying technology, if one was trained to "edit carefully" and the other to "regenerate heavily," they ended up in completely different groups.
The researchers found that knowing which AI processed the image explained 70% of the behavior, while the type of image (a face vs. a landscape) explained almost nothing (0.2%).

3. The "Fingerprint" Test

To figure out which AI did the work, the researchers used a simple test:

They took the original photo (the clean reference).
They took the AI-edited photo.
They used a smart computer vision tool (DINOv2) to measure the tiny differences between the two.

The Result:

If the AI was a "Careful Editor," the differences were tiny and consistent.
If the AI was a "Heavy Regenerator," the differences were large and chaotic.

Using just this one measurement, they could correctly guess which of the six AIs processed a photo about 51% of the time. Since there were six options, random guessing would only get you 16% right. So, 51% is a significant improvement.

4. Why "Blind" Detection Failed

The researchers also tried to identify the AI without looking at the original photo (a "blind" test), which is how most current AI detectors work.

The Result: The blind detectors failed miserably on the "Careful Editors." Because these AIs kept the photo so close to the original, the blind detectors couldn't tell them apart from the original image or from each other. They were essentially guessing.
The Lesson: The "original photo" is the key. Without it, you can't tell who did the careful editing. With it, you can.

5. What Happens to the "Invisible Ink"?

The paper also measured how much of their special "invisible ink" survived the trip through the AI:

Gemini: Kept about 98% of the ink.
Flux Kontext: Kept about 80% of the ink (even though the image looked almost identical to the human eye).
gpt-image-1: Wiped the ink out completely and replaced it with its own noise.

The Bottom Line

This paper suggests that we should stop thinking of these invisible ink tools just as "defense" (trying to stop AI from stealing art). Instead, they are better used as forensic tools.

If you have the original photo and a suspect AI-edited photo, you can use this method to prove which AI touched the image. However, this only works if you have the original photo to compare against. If you don't have the original, and the AI was a "Careful Editor," current technology cannot tell you which AI made the change.

Technical Summary: Vision-Encoder Behavioral Fingerprints of Image-to-Image Generative Models

Problem Statement

As commercial image-to-image (img2img) generative AI systems proliferate, there is a critical need to understand how these black-box systems interact with pixel-domain adversarial perturbations (e.g., Glaze, Mist, PhotoGuard). Existing attribution literature primarily focuses on text-to-image (T2I) outputs in blind settings where no reference image exists. However, img2img attribution presents a distinct methodological challenge: a clean reference image is available by construction. The paper addresses three core questions:

How do commercial img2img systems interact with sub-just-noticeable-difference (sub-JND) adversarial perturbations?
Can specific AI systems be attributed from their output behavior when a clean reference is available?
What behavioral signatures do these systems leave, and are they driven by architecture or training paradigm?

Methodology

The study employs a reference-anchored framework to analyze six production img2img APIs: gpt-image-1, Gemini 2.5 Flash Image, Flux Kontext, SDXL img2img, SD3 img2img, and Qwen Image Edit.

Data Corpus

Scale: 3,588 valid API calls (after 99.7% success rate) derived from a 200-image corpus.
Domains: COCO val2017 photographs, CelebA-HQ-256 portraits, and Flux-generated images.
Repetition: Three repetitions per (image, AI) cell.
Perturbation: A custom content-adaptive pipeline applies sub-JND adversarial perturbations (L∞ ≤ 0.10, JND budget 2.0) using 120 PGD steps against an ensemble of frozen vision encoders.

Feature Space

For each output image $O$ and clean reference $R$ (resized to 518×518), the authors compute a 2D behavioral feature vector using a frozen DINOv2 ViT-B/14 encoder:

patch_mean: The average per-patch cosine distance between the last-layer token embeddings of $O$ and $R$ over a 37×37 grid. This captures semantic-level modification.
ssim_clean: The luminance structural similarity between $O$ and $R$ , capturing pixel-level visual fidelity preservation.

Classification & Analysis

Band Classification: Outputs are categorized into "tight" (low patch_mean, ≤ 0.25) or "drift" (high patch_mean, > 0.25) bands based on the empirical distribution.
Attribution: A leave-one-out (LOO) nearest-centroid classifier operates on the 2D feature plane without learned parameters, classifying held-out points based on Euclidean distance to the mean centroid of each AI.
Statistical Rigor: Confidence intervals are derived via 2,000-iteration nonparametric cluster bootstrapping at the image level to account for within-image correlation.

Key Results

1. A Two-Band Training-Paradigm Taxonomy

The six systems partition into two distinct, image-invariant behavioral bands on the 2D plane:

Tight Band: Flux Kontext, Qwen Image Edit, and Gemini. These models preserve the input structure closely.
Drift Band: SDXL, SD3, and gpt-image-1. These models exhibit significant semantic drift from the input.

Crucial Finding: The discriminating variable is training paradigm, not architecture family.

Models trained on edit data (paired clean-to-edited supervision) cluster in the tight band, regardless of whether they are diffusion-based (Flux) or multimodal autoregressive (Qwen, Gemini).
Models that are T2I-base architectures adapted at sampling time (starting denoising from a partially noised input) cluster in the drift band, regardless of whether they are diffusion (SDXL/SD3) or autoregressive (gpt-image-1).

Variance Decomposition: AI identity explains 69.5% of behavioral variance, while image domain explains only 0.2%.

2. Attribution Accuracy

Six-Way LOO Accuracy: The reference-anchored framework achieves 51.4% [49.3, 53.4] accuracy across six AIs, significantly outperforming the 16.7% chance baseline.
Pilot Accuracy: On a focused 3-AI pilot, per-image attribution (averaging three repetitions) reaches 82.2% [73.3, 90.0].
Blind Baseline Comparison: When tested on the same corpus, blind attribution methods (AEROBLADE and a PRISM-style DINOv2 embedder) achieve only 0.222 and 0.373 accuracy, respectively. Blind methods perform at chance within the "tight" band (where outputs are near-faithful copies), whereas the reference-anchored method succeeds.

3. Differential Perturbation Survival

The study quantifies how much adversarial signal survives the img2img pipeline:

Gemini: ~98% of the perturbation signal survives (nearly intact).
Flux Kontext: ~80% survives, despite achieving SSIM 0.99 visual fidelity. The diffusion process acts as a selective filter, attenuating high-frequency perturbation content while preserving human-perceptible details.
gpt-image-1: The perturbation is effectively overwritten by the model's own generative variance.

Significance and Claims

The paper reframes pixel-domain perturbation pipelines from purely defensive tools into forensic primitives for reference-anchored AI-processing attribution.

Training Paradigm over Architecture: The primary contribution is demonstrating that behavioral fingerprints are determined by whether a model was trained for editing (tight band) or adapted from T2I at sampling time (drift band), rather than by the underlying architecture family (Diffusion vs. AR).
The Necessity of Reference: The study empirically validates that the clean reference image is the "load-bearing" signal for attribution. Blind methods fail to distinguish models that preserve inputs (the tight band), whereas the reference-anchored approach succeeds by measuring deviation from the known original.
Systematic Measurement: This work provides the first systematic measurement of the "diffusion-purification effect" across deployed commercial img2img systems, showing that survival rates vary systematically with training paradigm.

The authors conclude that while the scheme requires a retained original image (limiting its blind applicability), it offers a deployment-ready threshold and a validated behavioral feature space for attributing near-faithful reproductions generated by commercial AI, a regime where blind detection currently fails.