Field-Localized Forgery Detection for Digital Identity Documents

Technical Summary: Field-Localized Forgery Detection for Digital Identity Documents

Problem Statement
Digital identity verification systems, increasingly relied upon for remote onboarding in banking, fintech, and telecommunications, are vulnerable to localized manipulations of identity documents. Unlike natural images, identity documents possess structured layouts containing semantically meaningful fields (e.g., facial photographs, textual identity data, security textures). Existing forgery detection methods, primarily developed for natural-image forensics (e.g., object insertion, generative synthesis), often fail when applied to these structured documents. They lack the domain specificity to handle the unique characteristics of identity fields and often degrade in performance when processing full-document images. Furthermore, the reliance on proprietary, closed-source datasets in industry limits reproducibility and the development of open, efficient, domain-specific solutions.

Methodology: The FLiD Framework
The authors propose FLiD (Field-Localized Identity Document Forgery Detection), a lightweight framework designed to treat identity documents as structured collections of semantic fields rather than homogeneous image grids. The pipeline consists of three primary stages:

1. Identity Field Localization:

   • A fine-tuned YOLOv8 object detector is employed to automatically localize and extract critical identity regions: the facial photograph and textual identity fields.
   • This step addresses the structural variability of documents from different countries and issuing authorities, ensuring consistent extraction of semantically relevant regions regardless of the document layout.
   • The detector was trained on manually annotated bounding boxes from the dataset to ensure robustness against domain-specific variations.

2. Field-Level Feature Encoding:

   • Extracted face and text regions are resized to 224×224 and passed through a frozen MobileNetV3-Small backbone (pretrained on ImageNet-1K).
   • The backbone generates compact 576-dimensional embeddings for each field. By freezing the backbone weights, the model avoids overfitting on the limited identity-document data while leveraging rich visual representations learned from large-scale pretraining.
   • This approach reduces the trainable parameter count significantly compared to training a full backbone from scratch.

3. Multi-Field Fusion and Classification:

   • For documents potentially containing multi-field manipulations, the face and text embeddings are concatenated to form a unified multi-field representation.
   • A lightweight feed-forward neural network classifier (5 fully connected layers with ReLU activations and dropout) processes these concatenated features to classify the document as bona fide or forged.
   • The entire classification head contains only 191K trainable parameters, optimized using binary cross-entropy loss with class balancing.

Key Contributions

• Identity-Aware Field-Level Framework: A novel approach that explicitly models identity documents as structured compositions of semantic fields, enabling targeted forgery analysis distinct from general natural-image detectors.
• Semantic Localization Module: The introduction of a fine-tuned YOLOv8 module to automatically detect and extract critical identity regions (face and text), facilitating consistent preprocessing across heterogeneous document formats.
• Lightweight Representation Learning: An efficient embedding strategy using a frozen MobileNetV3-Small backbone that achieves strong detection performance with minimal computational cost (191K parameters and 21× fewer FLOPs than the baseline).

Experimental Results
The framework was evaluated on the FantasyID dataset, a benchmark containing 362 unique ID designs with 1,086 bona fide images and various forgery types (face-only, text-only, and combined attacks). Performance was assessed using 5-fold stratified cross-validation and ISO/IEC 30107-3 PAD metrics.

• Performance Gains: FLiD achieved Area Under the Curve (AUC) scores of 0.880 (face), 0.954 (text), and 0.923 (both-field attacks).
• Error Reduction: The method reduced the Equal Error Rate (EER) by 29.40% for face attacks, 35.65% for text attacks, and 35.34% for combined attacks compared to a baseline MobileNetV2 trained from scratch on full-document images.
• Operational Efficiency: FLiD consistently outperformed general-purpose manipulation detectors (TruFor, MMFusion, UniVAD) across all scenarios while requiring 13× fewer parameters and 21× fewer FLOPs.
• Robustness: The model demonstrated superior score separability and ranking reliability, particularly in text-only attacks where the baseline performed near random chance (AUC ~0.54).

Significance and Claims
The paper argues that the structural nature of identity documents necessitates a fundamentally different detection strategy than that used for natural images. By focusing on semantically critical fields rather than processing the entire document indiscriminately, FLiD achieves a superior trade-off between detection accuracy and computational efficiency. The authors claim that domain-specific modeling is essential for digital identity forgery detection, as general-purpose models fail to generalize to structured document scenarios. The proposed framework offers a reproducible, open-source, and computationally efficient solution suitable for real-world deployment in digital identity verification systems, particularly in edge environments where latency and resource constraints are critical.

Limitations and Future Directions
The authors acknowledge that the current system relies on visual feature representations and does not utilize Optical Character Recognition (OCR) or language-specific priors. While effective against current manipulation techniques that introduce visual artifacts (e.g., font inconsistencies, blending boundaries), future work aims to explore multimodal visual-linguistic fusion to detect semantically implausible but visually seamless edits. Additionally, future directions include extending the system to produce pixel-level tampering maps for localization and integrating explainability techniques (e.g., Grad-CAM) to improve operational trust.