💻 computer science

Efficient Document Tampering Localization with Multi-Level Discrepancy Features and Unified DCT-Quantization Embedding

The paper proposes DiffNet, an efficient RGB-DCT early-fusion architecture featuring multi-level discrepancy features and a unified DCT-quantization embedding, which achieves state-of-the-art performance in localizing document tampering across domains with significantly higher throughput than previous methods.

Original authors: Mohamed Dhouib, Ye Zhu, Sonia Vanier, Aymen Shabou

Published 2026-06-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mohamed Dhouib, Ye Zhu, Sonia Vanier, Aymen Shabou

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to spot a fake ID card or a doctored bank statement. The problem is that modern forgers are incredibly skilled. They don't just paste a new name over an old one; they use sophisticated tools to make the new text look like it was printed on the paper years ago, matching the font, the ink texture, and even the slight imperfections of the original document. To the naked eye, the document looks perfect.

This paper introduces a new detective tool called DiffNet. Instead of trying to be a "super-smart" AI that learns every possible way to forge a document, DiffNet uses two clever tricks to spot the tiny, invisible cracks in the forgery.

Here is how it works, explained simply:

1. The "Noise-Canceling" Glasses (Multi-Level Discrepancy)

Imagine you are looking at a painting. If someone tries to paint over a small part of it, the brushstrokes might look slightly different, or the texture might be off. But if you look at the whole painting, the background scenery (like a tree or a mountain) is so loud and detailed that it drowns out those tiny differences.

Most AI models try to look at the whole "painting" (the document content) to find the fake. They get distracted by the text and the layout.

DiffNet puts on a pair of "noise-canceling glasses."

How it works: Before the AI tries to make a decision, it runs the image through a special filter. This filter ignores the actual content (the words, the pictures) and only looks for differences or inconsistencies.
The Analogy: Think of it like listening to a song in a noisy room. If you turn up the volume on the singer, you can't hear the background noise. But if you use noise-canceling headphones that subtract the background music, you can suddenly hear the tiny scratch on the record. DiffNet subtracts the "content" so the "scratches" (the tampering) stand out clearly. It does this at every level of the image, from the big picture down to the tiny pixels.

2. The "Frequency Translator" (DCT–Quantization Embedding)

Digital documents are often saved as JPEGs. When a computer saves a JPEG, it doesn't store every single color dot. Instead, it breaks the image into tiny 8x8 blocks and translates them into a mathematical code called DCT (Discrete Cosine Transform). It's like translating a book from English into a secret code of numbers.

When a forger edits a document, they often have to re-save it as a JPEG. This re-saving process leaves a unique "fingerprint" in that secret code, even if the picture looks the same to us.

DiffNet has a special translator for this code.

How it works: Previous AI models tried to read this secret code by using very heavy, complex machinery that slowed them down. DiffNet uses a lightweight, efficient translator. It looks at the relationship between the numbers in the code and the "rules" used to compress the image (the quantization table).
The Analogy: Imagine a forger trying to forge a banknote. They might get the ink color right, but they might use the wrong type of paper. A heavy-handed detector would weigh the whole banknote to check the paper. DiffNet, however, has a special scanner that instantly checks the texture of the paper fibers. It knows exactly which "fingerprint" belongs to a real document and which belongs to a fake, without needing to carry around a heavy backpack of extra tools.

The Result: Faster and Smarter

By combining these two tricks, DiffNet achieves two major things:

It sees what others miss: When tested on real-world forgeries made by humans (not just computer simulations), DiffNet found about 30% more fakes than the previous best models. It is much better at spotting forgeries that look perfect to the human eye.
It is incredibly fast: Because it doesn't use heavy, unnecessary machinery, it runs 7 times faster than the previous top model. It's like switching from a slow, heavy truck to a sleek, high-speed sports car that gets the same job done in a fraction of the time.

Why This Matters

The paper notes that many AI models are trained on "fake" data generated by computers. These models get good at spotting the specific "glitches" that the computer generator makes, but they fail when they see a real human forger.

DiffNet is designed to ignore those specific computer glitches and focus on the fundamental inconsistencies that any tampering creates. It doesn't try to be a "super-genius" that learns every trick; instead, it uses simple, smart filters to strip away the distraction and reveal the truth.

In short: DiffNet is a fast, efficient detective that ignores the "story" of the document to focus entirely on the "cracks" in the evidence, making it much harder for forgers to get away with their tricks.

Technical Summary: Efficient Document Tampering Localization with Multi-Level Discrepancy Features and Unified DCT–Quantization Embedding

Problem Statement

Document tampering localization is a critical task for preventing fraud in financial and administrative workflows, yet it remains highly challenging. Manipulations are often crafted to be visually consistent, leaving only subtle traces invisible to the human eye. Current state-of-the-art methods face two primary limitations:

Poor Cross-Domain Generalization: Models trained on synthetic benchmarks often fail to generalize to human-made forgeries or cross-domain scenarios where source documents and tampering pipelines differ significantly. This is largely attributed to "shortcut learning," where models rely on pipeline-specific artifacts (e.g., specific font rendering or texture patterns) rather than the fundamental inconsistencies introduced by the manipulation.
Inefficient DCT Backbone Design: While frequency-domain analysis (specifically Discrete Cosine Transform or DCT) has become standard, recent architectures (e.g., those utilizing the Frequency Perception Head) have focused on downstream fusion strategies while leaving the encoder backbone largely unchanged. These existing backbones often introduce significant computational overhead due to early high-dimensional feature expansions and auxiliary blocks.

Methodology: DiffNet

The authors propose DiffNet, a relatively simple yet effective RGB–DCT early-fusion architecture designed to improve cross-domain performance and computational efficiency. The architecture consists of two primary innovations:

1. Multi-Level Discrepancy Transformation

To prevent the decoder from operating on raw, content-heavy activations and to force the model to focus on inconsistency cues, the authors introduce a lightweight transformation applied at the output of every backbone stage (before multi-level fusion).

Mechanism: This transformation converts feature maps into magnitude-only responses of learned zero-sum filters.
Filter Design: The system learns two complementary types of filters:
- Free zero-sum filters: Enforce a zero-sum constraint ( $\sum k = 0$ ) to suppress bias and emphasize discrepancy.
- Center-anchored zero-sum filters: Constrain the filter to act as a neighborhood-based center predictor, where the center weight is the negative sum of non-negative neighbor weights.
Sign-Invariance: By taking the absolute value of the filter responses ( $|u_{c,m}(p)|$ ), the representation captures the strength of the mismatch independent of its sign, creating a sign-invariant discrepancy representation across the feature pyramid.
Implementation: To ensure efficiency, the authors implement a custom CUDA kernel that avoids materializing dense constrained kernels during training, instead using a compact center-tied representation.

2. Unified DCT–Quantization Embedding

The authors redesign the DCT-domain backbone to replace the heavy Frequency Perception Head (FPH) with a more efficient approach.

Joint Embedding: Since both quantized DCT coefficients and quantization table entries are discrete integers, the method leverages embedding layers to map each coefficient–quantization pair $(\hat{C}_{p,k}, Q_k)$ to a compact joint embedding.
Components: The embedding includes:
- A local quantized coefficient embedding ( $v_{p,k}$ ).
- Per-frequency modulation parameters ( $\gamma_k, \beta_k$ ) derived from the quantization table entry ( $Q_k$ ), applied in a FiLM-like manner.
- A learned frequency-index embedding ( $f_k$ ) to identify the DCT position.
- A global table bias ( $t$ ) computed from the full quantization table to capture broader patterns.
Efficiency: This design injects frequency index information explicitly without early high-dimensional expansions or auxiliary blocks, resulting in a lightweight frequency branch.

Architecture Overview

DiffNet follows a two-branch early-fusion design:

RGB Branch: Processes the input image using a four-stage ConvNeXt-V2 backbone.
DCT Branch: Encodes quantized DCT coefficients and the quantization table using the proposed joint embedding and a compact ConvNeXt-V2 stack.
Fusion: DCT features are injected into the RGB stream after Stage 1.
Discrepancy Application: The multi-level discrepancy transformation is applied to the output of each backbone stage.
Prediction: A multi-scale fusion module (FPN-style) aggregates the transformed features, followed by refinement blocks and a lightweight segmentation head to predict the tampering mask.

Key Contributions

Multi-Level Discrepancy Transformation: A novel mechanism applied at every backbone stage that converts features into sign-invariant discrepancy representations using learned zero-sum filters, shifting the decoder's focus from content to inconsistency.
Efficient DCT Backbone: A lightweight frequency-index-aware joint embedding for quantized DCT coefficients and quantization tables, providing a strong frequency branch without the overhead of previous FPH-based designs.
Extensive Ablation Studies: Comprehensive validation confirming the contribution of each component, including the necessity of the zero-sum constraint, the specific design of the DCT embedding, and the superiority of the proposed filters over fixed residual modalities (e.g., SRM, Bayar convolutions).
State-of-the-Art Performance: The integration of these components yields a new state-of-the-art for human-made document tampering detection and localization.

Experimental Results

The authors evaluated DiffNet under two distinct protocols: the Doc Protocol (cross-domain evaluation on synthetic training and diverse test sets) and the Syn2Real-TDoc protocol (training on large-scale synthetic data and testing on human-made datasets).

Cross-Domain Performance (Doc Protocol): DiffNet achieved an average F1 score of 0.515 across cross-domain test sets (T-SROIE, OSTF, TPIC-13, RTM), outperforming the previous best (ADCD-Net) by approximately 30% (0.450 vs. 0.515).
Human-Made Forgery Performance (Syn2Real-TDoc): On human-made datasets (RTM, FindIt, FindItAgain), DiffNet achieved an average image-level F1 of 0.640 and pixel-level F1 of 0.304, improving over the previous best baseline by +0.017 (image) and +0.034 (pixel).
Efficiency: The proposed architecture is significantly faster than prior methods. It achieves an inference throughput of 40.1 it/s and training throughput of 41.9 it/s (at 768×768 resolution), representing up to a 7× speedup over ADCD-Net and outperforming other high-performing baselines like FFDN.

Significance and Claims

The paper claims that DiffNet addresses the critical gap in cross-domain performance by constraining the feature pyramid to rely on inconsistency cues rather than pipeline-specific artifacts. By revisiting the DCT backbone design and introducing a discrepancy-driven feature transformation, the authors demonstrate that architectural simplicity and targeted design choices can outperform complex, heavy models.

The authors acknowledge that while their method provides a consistent boost, localization performance on human-made tampering remains a challenge due to the scarcity of cues in heavily concealed edits and the high proportion of pristine regions in test datasets. They position their work as a step toward more robust generalization, noting that future improvements may require better synthetic data generation that mimics advanced concealment strategies without introducing new shortcuts. The code and pretrained weights are made available to support further research in document forensics.