Technical Summary: Data Provenance for Image Auto-Regressive Generation

Problem Statement

Image Autoregressive Models (IARs) have achieved remarkable photorealistic quality by adapting the next-token prediction paradigm from Large Language Models (LLMs). However, as these models generate content that is visually indistinguishable from natural images, they pose significant risks regarding misinformation, fraud, and the "pollution" of training datasets (leading to model collapse). Existing provenance methods, such as watermarking and fingerprinting, typically require integration into the model's training or generation process. These methods are often inapplicable to content already published without marks or to models that do not natively support watermarking. Furthermore, many existing reconstruction-based approaches fail to handle the stochastic nature of IARs or require specific model access that is not always available.

The core challenge addressed by this work is post-hoc data provenance: the ability to robustly trace an image back to its generating IAR without modifying the model, the generation process, or the image itself, and without requiring auxiliary information like text prompts or class labels.

Methodology

The authors propose a model-agnostic, post-hoc framework that leverages inherent artifacts introduced by the vector-quantized tokenization process unique to IARs. The framework relies on two complementary signals: QuantLoss and EncLoss.

1. Core Observation: Quantization Artifacts

IARs encode images as sequences of discrete tokens from a fixed codebook (vocabulary). The authors observe that token representations of IAR-generated images are consistently closer to the codebook entries than those of natural images. This is because generated images are constructed directly from these discrete entries, whereas natural images must be approximated by them.

2. The Framework Components

The framework operates in three main stages:

• Decoder Inversion ($D^{-1}$):
  Standard encoders ($E$) trained on natural images are not optimal for inverting the decoder ($D$) of generated images. The authors train an inverse decoder ($D^{-1}$) by initializing it with the original encoder weights and fine-tuning it on images generated by the target IAR. This model learns to map a generated image back to its latent feature map ($f_Z$) with high fidelity.

• QuantLoss (Codebook Distance):
  This signal measures the distance between the feature map recovered by the inverse decoder ($f$) and its re-quantized version ($f_Z$).

  • Process: $x \xrightarrow{D^{-1}} f \xrightarrow{Q} t \xrightarrow{Q^{-1}} f_Z$.
  • Signal: $L_{Quant} = \|f - f_Z\|^2$.
  • Logic: For images generated by the target IAR, $f$ aligns closely with codebook entries, resulting in minimal quantization error. For natural images or images from other models, the alignment is poor, resulting in high error.
  • Multi-scale Optimization: For multi-scale IARs (e.g., VAR), the authors introduce an optimized token-search algorithm using gradient descent to find the token combination across scales that minimizes reconstruction error, further distinguishing generated features from natural ones.

• EncLoss (Reconstruction Loss Ratio):
  This signal captures the information loss when compressing an image to the latent space and back.

  • Process: $Rec(x) = D(D^{-1}(x))$.
  • Signal: To address false positives from low-complexity natural images, the authors propose a calibrated loss inspired by AEDR: $L_{Cal}^{Enc} = \frac{\|Rec(x) - x\|^2}{\|Rec(Rec(x)) - Rec(x)\|^2}$.
  • Logic: Generated images exhibit feature consistency, leading to low loss ratios, while natural images often show higher ratios due to information compression artifacts.

• Combined Signal:
  The final provenance signal is the product of the two: $L_{Comb} = L_{Quant} \times L_{Cal}^{Enc}$.

Key Contributions

1. First Post-Hoc Framework for IARs: The paper introduces the first method capable of attributing IAR-generated images to their source model without requiring modifications to the model's training or generation pipeline. It is applicable to already published, unmarked content.
2. Novel Provenance Signals: The work identifies and formalizes QuantLoss (based on codebook proximity) and EncLoss (based on reconstruction consistency) as robust indicators of IAR generation. It demonstrates that combining these signals yields near-perfect detection.
3. Comprehensive Evaluation: The framework is evaluated on a diverse set of state-of-the-art IARs (VAR, RAR, LlamaGen, Taming, Infinity) and a vector-quantized diffusion model (VQ-Diffusion).

Experimental Results

The authors report extensive empirical evaluations with the following findings:

• Detection Performance: The method achieves nearly 100% True Positive Rate (TPR) at 1% False Positive Rate (FPR) across all tested IARs and datasets (ImageNet, LAION, MS-COCO, and images generated by other models). This significantly outperforms baselines such as naive reconstruction loss, LatentTracer, and AEDR, which often fail to distinguish between different IARs or natural images.
• Robustness: The framework demonstrates strong resilience against common image post-processing techniques, including JPEG compression, resizing, noise, and brightness/contrast adjustments. When the inverse decoder is fine-tuned with data augmentations, robustness improves further, maintaining high TPR even under strong attacks.
• Efficiency: The method is computationally efficient. After a one-time fine-tuning of the inverse decoder (which is a small overhead compared to model pre-training), the attribution process takes less than 10 milliseconds per image for most models, making it faster than existing reconstruction-based baselines.
• Adaptive Attacks: Even under adaptive adversarial attacks where an adversary attempts to perturb images to evade detection, the augmentation-trained framework maintains significantly higher robustness than baseline methods.

Significance and Claims

The paper claims to provide a practical and scalable solution for responsible deployment and auditing of autoregressive generative models. By exploiting the inherent "fingerprint" left by the discrete tokenization process, the method enables reliable source attribution without compromising image quality or requiring access to the generation process.

The authors emphasize that this approach is critical for:

• Preventing Model Collapse: By reliably identifying synthetic data, it helps prevent the contamination of training datasets with generated content.
• Combating Misinformation: It allows for the attribution of harmful or misleading content to specific model families.
• Post-Hoc Auditing: It operates on content that has already been published, filling a gap left by watermarking methods that require pre-integration.

The work concludes that while adaptive attacks can degrade performance, the proposed framework offers a robust defense mechanism that is broadly applicable to existing and future IAR architectures.