💻 computer science

SEED: A Large-Scale Benchmark for Provenance Tracing in Sequential Deepfake Facial Edits

This paper introduces SEED, a large-scale benchmark of over 90,000 images with sequential facial edits, to address the gap in provenance tracing for multi-step deepfakes, and proposes FAITH, a frequency-aware Transformer model that effectively aggregates spatial and frequency-domain cues to identify and order latent editing events.

Original authors: Mengieong Hoi, Zhedong Zheng, Ping Liu, Wei Liu

Published 2026-04-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mengieong Hoi, Zhedong Zheng, Ping Liu, Wei Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are looking at a photo of a friend. At first glance, it looks real. But then you notice something odd: their hair is bright blue, they're wearing a hat that doesn't quite fit, and their eyes are a different color than usual.

In the past, spotting a fake photo was like finding a single typo in a sentence. You just looked for one mistake. But today, AI tools (like "diffusion models") are so good that they can edit a photo step-by-step, like a chef adding ingredients one by one to a soup. First, they change the hair. Then, they add a hat. Then, they tweak the eyes. By the time the photo is done, it looks incredibly realistic, but it's actually the result of a long, hidden chain of events.

This paper introduces a new tool called SEED to help us figure out exactly how that soup was made, step-by-step.

Here is the breakdown of the paper in simple terms:

1. The Problem: The "Digital Layer Cake"

Think of a deepfake image as a layer cake.

Old Deepfakes: Were like a cake with just one weird layer (e.g., someone swapped a face). It was easy to spot the "fake" layer.
New Deepfakes (The SEED problem): Are like a cake where someone added a layer of frosting, then sprinkled some sugar, then added a cherry, then changed the color of the frosting. Each new step hides the evidence of the previous steps. The final cake looks delicious, but if you try to guess the recipe just by looking at the top, you might get it wrong.

Current tools are great at saying, "This cake is fake!" but they are terrible at saying, "First they added frosting, then they added sugar, then they changed the color."

2. The Solution: The SEED Benchmark (The "Recipe Book")

The authors created a massive new dataset called SEED (Sequential Editing in Diffusion).

What it is: They took 90,000 real faces and used AI to edit them in a specific order (1 to 4 steps).
The Secret Sauce: Unlike other datasets that just say "Fake," SEED keeps a detailed diary for every single image. It knows:
- Step 1: Changed lips to red.
- Step 2: Added a hat.
- Step 3: Changed eye color.
- Step 4: Added glasses.
Why it matters: This allows researchers to train AI to not just spot a fake, but to reverse-engineer the editing history. It's like giving a detective a list of suspects and asking them to figure out the exact order in which the suspects entered the room.

3. The Challenge: Why is this so hard?

The paper tested many existing AI detectives on this new SEED dataset, and they mostly failed.

The Analogy: Imagine trying to find footprints in a room where someone has been walking back and forth, sweeping the floor, and then walking again. The footprints get smudged, covered, and mixed up.
The Result: Standard AI tools that just look at the "picture" (spatial features) got confused. They couldn't tell if the hat was added before or after the glasses because the later edits erased the "footprints" of the earlier ones.

4. The New Detective: FAITH (The "X-Ray Glasses")

To solve this, the authors built a new AI model called FAITH.

How it works: Instead of just looking at the picture like a human does (spatial), FAITH puts on X-Ray glasses that look at the frequencies (the hidden patterns of light and dark).
The Metaphor: Think of an image like a song.
- Spatial view: You hear the melody (the face, the hat).
- Frequency view: You hear the background static and the specific vibrations of the instruments.
- Even when the melody changes (the hat is added), the "static" or the specific "vibration" of the AI's editing process leaves a tiny, unique fingerprint that doesn't get wiped out easily.
The Magic: FAITH uses a mathematical tool called a Wavelet Transform (think of it as a super-precise microscope) to find these tiny, high-frequency fingerprints. It combines the "picture" view with the "vibration" view to reconstruct the editing history.

5. The Results: Does it work?

The Test: They tested FAITH on photos that had been compressed (like when you send a photo on WhatsApp) or had noise added (like a grainy photo).
The Outcome: FAITH was much better than the old tools. Even when the photo was "dirty" or compressed, FAITH could still hear the "vibrations" of the editing steps and correctly guess the order: "First the lips, then the hat, then the eyes."
The Catch: It's still hard when there are too many steps (4 or more). It's like trying to remember a recipe after someone has added 10 ingredients; eventually, the clues get too faint.

Summary

SEED is a giant training gym where AI learns to spot the history of a fake photo, not just the fact that it's fake.
FAITH is the new champion in that gym. It uses a special "frequency vision" to see the hidden footprints of AI edits that other tools miss.
Why we need this: As AI gets better at making perfect fakes, we need better tools to not just say "That's fake," but to say "Here is exactly how it was faked, and in what order." This is crucial for stopping misinformation and protecting our trust in what we see online.

1. Problem Statement

The proliferation of diffusion models has enabled highly realistic, multi-step facial editing (e.g., changing hair, then eyes, then adding glasses). While existing deepfake detection research focuses on single-step manipulations or global authenticity labels, real-world forgery often involves sequential editing pipelines.

The Gap: Current datasets (e.g., FaceForensics++, ForgeryNet) do not model the history of edits. They treat an image as a single manipulation outcome.
The Challenge: Recovering the "provenance" (the specific order and nature of edits) is difficult because:
- Later edits can overwrite or obscure earlier artifacts.
- Evidence becomes non-stationary and distributed across the image.
- Existing sequential benchmarks (e.g., Seq-DeepFake) rely on GAN-based pipelines, which do not capture the subtle, high-frequency artifacts characteristic of modern diffusion models.

2. Methodology: The SEED Benchmark

The authors introduce SEED (Sequential Editing in Diffusion), the first large-scale benchmark specifically designed for diffusion-based sequential provenance tracing.

Dataset Construction

Scale: 91,526 facial images.
Source: High-quality real faces from FFHQ and CelebAMask-HQ.
Process: Images undergo 1 to 4 sequential attribute edits using diffusion editors (LEdits, SDXL, SD3 with UltraEdit).
Attributes: Includes intrinsic features (Eyes, Lips, Hair, Eyebrows) and accessories (Glasses, Hats).
Metadata: Every sample includes fine-grained, step-wise annotations:
- Edit Order: The sequence of attributes modified.
- Textual Instructions: Prompts used for each step.
- Manipulation Masks: Spatial regions affected.
- Model Identity: Which diffusion model was used at each step.

Data Distribution & Protocol

Sequence Length: Balanced distribution across lengths $L=1, 2, 3, 4$ .
Split Strategy: Identity-disjoint splits (training, validation, and testing sets contain no overlapping identities) to prevent models from memorizing subjects rather than learning provenance cues.
Tasks Supported:
1. Authenticity Analysis: Distinguishing real images from sequentially edited ones.
2. Editing Trace Analysis: Predicting the edited attributes and their specific temporal order.
3. Spatial Evidence Analysis: Localizing manipulated regions using masks.

3. Proposed Model: FAITH

To benchmark the difficulty of SEED, the authors propose FAITH (Frequency-Aware Identification Transformer for Sequential Editing Detection).

Motivation: Spatial-only approaches struggle with subtle diffusion artifacts, especially as edit chains lengthen. The authors hypothesize that frequency-domain signals (specifically high-frequency components) remain informative even when spatial traces are overwritten.
Architecture:
- Backbone: ResNet-50 for spatial feature extraction.
- Frequency Branch: Uses Discrete Wavelet Transform (DWT) to extract high-frequency details (specifically the HH sub-band). These features are processed by a lightweight CNN.
- Transformer Encoder-Decoder:
  - The encoder refines spatial features.
  - The decoder predicts the edit sequence autoregressively (token-by-token).
  - Key Innovation: The frequency features are converted into an additive guidance bias ( $M^f$ ) injected into the decoder's cross-attention mechanism. This guides the model to focus on high-frequency suspicious regions during sequence generation.
Training Objective: Standard cross-entropy loss over the full token sequence (including start/end tokens), treating provenance tracing as an image-to-sequence prediction problem.

4. Key Results

The paper evaluates FAITH against adapted single-step detectors and a sequential baseline (SeqFakeFormer) on the SEED test set.

Performance Metrics

Three metrics were used:

Fixed-Acc: Token-level accuracy under a fixed horizon.
Adaptive-Acc: Overlap-only prefix accuracy.
Full-Acc: Strict exact match of the entire edit sequence (length and order).

Findings

Difficulty of Sequential Tracing: Performance degrades significantly as the sequence length ( $L$ $L$ ) increases.
- At $L=1$ , models achieve ~98% accuracy.
- At $L=4$ , Full-Acc drops to ~24.5% for the best model (FAITH with DWT), highlighting the extreme difficulty of recovering long edit chains.
Frequency Cues are Critical:
- FAITH consistently outperforms spatial-only baselines (SeqFakeFormer).
- Among frequency transforms, DWT (Discrete Wavelet Transform) yields the best results, outperforming DCT and FFT. This suggests that localized high-frequency decomposition is superior for preserving edge/texture changes caused by edits.
Robustness:
- Under JPEG compression and Gaussian noise, FAITH (DWT) maintains higher Full-Acc than baselines.
- Baselines often suffer from "premature termination" (stopping the sequence early) or "order drift" (swapping the order of edits), whereas FAITH maintains more complete and ordered predictions.

Error Analysis

As sequence length increases, the dominant error mode shifts from attribute confusion to order errors (Swap-only).
For $L=4$ , ~68% of errors are due to incorrect ordering, confirming that later edits obscure the temporal cues of earlier ones.

5. Key Contributions

SEED Benchmark: The first large-scale, diffusion-based dataset for sequential provenance tracing, featuring 90K+ images with step-wise metadata (order, prompts, masks).
Systematic Benchmarking: Demonstrates that single-step detectors fail to generalize to sequential tasks and that performance drops sharply with chain length.
FAITH Baseline: A frequency-aware Transformer that proves high-frequency signals (via DWT) are essential for recovering edit order and identity in diffusion-based forgeries, offering a new direction for forensic analysis beyond spatial features.

6. Significance

Forensic Evolution: Moves the field from simple "Real vs. Fake" classification to complex "History Recovery," which is crucial for legal and platform moderation contexts where understanding the evolution of an image is necessary.
Diffusion Era Adaptation: Addresses the specific limitations of GAN-based benchmarks, providing a testbed for the artifacts unique to diffusion models.
Frequency Domain Focus: Establishes that frequency-domain analysis is not just a supplementary tool but a critical component for robust deepfake detection in multi-step scenarios.

In conclusion, SEED and FAITH highlight that while diffusion models create highly realistic images, they leave subtle, frequency-based footprints that can be aggregated to reconstruct editing histories, provided the analysis moves beyond purely spatial features.