💻 computer science

Face2Parts: Exploring Coarse-to-Fine Inter-Regional Facial Dependencies for Generalized Deepfake Detection

The paper proposes Face2Parts, a novel hybrid deepfake detection method that leverages hierarchical feature representation to capture coarse-to-fine inter-regional dependencies among facial components (frame, face, lips, eyes, and nose) via channel attention and deep triplet learning, achieving superior generalization and performance across multiple benchmark datasets.

Original authors: Kutub Uddin, Nusrat Tasnim, Byung Tae Oh

Published 2026-03-30

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Kutub Uddin, Nusrat Tasnim, Byung Tae Oh

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to spot a fake ID card.

Most traditional detectives (existing AI methods) only look at the photo on the card. They check if the face looks real. But what if the forger was really good at copying the face, but messed up the background, or the way the person's eyes blink, or how their lips move when they speak? If you only look at the face, you might get fooled.

This paper introduces a new detective team called Face2Parts. Instead of just looking at the whole face, this team breaks the investigation down into three levels of detail, like zooming in with a magnifying glass.

The Three Levels of Investigation

Think of the Face2Parts method as a team of three specialists working together:

The Wide-Angle Specialist (Level 1 - The Frame):
This person looks at the entire picture, including the background and the lighting. They ask: "Does the whole scene look natural? Is the lighting consistent?" This is the "coarse" view.
The Portrait Specialist (Level 2 - The Face):
This person zooms in on just the face. They check the skin texture and the general shape. This is the "medium" view.
The Micro-Inspector (Level 3 - The Parts):
This is the most detailed part. This specialist uses a microscope to look at specific, tiny parts: the left eye, the right eye, the lips, and the nose. They check for tiny glitches, like a lip that doesn't quite sync with the voice, or an eye that reflects light strangely. This is the "fine" view.

How They Work Together: The "Team Huddle"

In the past, detectives worked alone. The Face2Parts method uses a special Team Huddle (called a Channel-Attention Mechanism).

Imagine the three specialists are in a room. The Micro-Inspector might say, "Hey, the lips look suspicious!" The Wide-Angle Specialist might say, "But the background lighting is perfect!"

Instead of ignoring each other, they share their notes. The system weighs who is right based on the situation. If the lips are the giveaway, the system listens to the Micro-Inspector. If the background is the giveaway, it listens to the Wide-Angle Specialist. They combine their opinions to make a final decision.

They also use a training game called Triplet Learning. Imagine showing the team three photos:

Photo A: A real person (The Anchor).
Photo B: Another real person (The Positive).
Photo C: A fake deepfake (The Negative).

The team learns to say, "A and B are friends (both real), but C is a stranger (fake)." By practicing this game thousands of times, they get really good at spotting the subtle differences that humans miss.

Why This Matters

The paper tested this new detective team on many different types of "fake IDs" (datasets) created by different hackers.

The Result: The Face2Parts team was much harder to fool than the old detectives.
The Score: They got a near-perfect score (98%+) on many tests, and even 100% on some difficult ones where other methods failed.

The Bottom Line

Deepfakes are getting better at faking just the face. To catch them, we can't just look at the face anymore. We need to look at the whole picture, the face, and the tiny details all at the same time.

Face2Parts is like upgrading from a single security guard at the door to a full security team with cameras, microscopes, and a central command center that talks to each other. This makes it much harder for a fake to slip through the cracks.

1. Problem Statement

The rapid advancement of Generative Adversarial Networks (GANs) and AI has made it increasingly easy to create realistic "deepfakes" (synthetic media), posing significant threats to security, privacy, and public trust. While numerous forensic methods exist to detect these fakes, they suffer from critical limitations:

Lack of Generalization: Most State-of-the-Art (SoTA) methods focus exclusively on the face region (medium-grained features). They often fail to generalize when tested on unseen datasets or different manipulation types (e.g., a model trained on face-swapping may fail on lip-sync manipulation).
Localized Manipulations: Deepfake artifacts are often localized to specific facial parts (e.g., eyes, lips, nose) or the surrounding frame. Relying solely on the central face region misses these subtle, region-specific traces.
Inter-Regional Dependencies: Existing methods rarely analyze the complex relationships and dependencies between different facial regions (e.g., the synchronization between lip movement and eye blinking).

2. Methodology: Face2Parts

The authors propose Face2Parts, a novel hybrid approach based on Hierarchical Feature Representation (HFR). The core philosophy is to move from a "face-only" analysis to a "coarse-to-fine" multi-level analysis that captures inter-regional dependencies.

A. Hierarchical Feature Extraction

The framework extracts features from three distinct levels of granularity:

Level-1 (Coarse): The entire video frame ( $I_{Frame}$ ), capturing global context and background artifacts.
Level-2 (Medium): The cropped face region ( $I_{Face}$ ), focusing on the primary subject.
Level-3 (Fine): Specific key facial parts: Left Eye, Right Eye, Lips, and Nose ( $I_{LeftEye}, I_{RightEye}, I_{Lips}, I_{Nose}$ ). These are extracted using Dlib with 81 facial landmarks.

All regions are resized to $224 \times 224$ and processed through pre-trained encoders (specifically CLIP models: ViT-B16, ViT-B32, and ViT-L14) to generate feature vectors.

B. Deep Triplet Learning with Channel Attention

To exploit the relationships between these diverse features, the authors employ a Deep Triplet Learning framework combined with a Channel-Attention-based Multi-Layer Perceptron (CA-MLP).

Triplet Learning: The model processes triplets of samples: an Anchor (A), a Positive (P) from the same class (Real/Real or Fake/Fake), and a Negative (N) from the opposite class. The goal is to minimize the distance between A and P while maximizing the distance between A and N.
CA-MLP Mechanism: Before the triplet loss is applied, a channel-attention mechanism is used. It performs global average and max pooling on the concatenated features to generate attention weights. This allows the model to dynamically focus on the most discriminative features (e.g., prioritizing "lips" for lip-sync fakes or "frame" for compression artifacts).
Loss Function: The model uses a margin ranking loss to learn discriminative embeddings. The authors found that a margin of 0 yielded the best generalization, as foundation models already provide well-separated embeddings.

C. Classification

The final discriminative embeddings (256-dimensional) are passed to a classifier. Extensive testing showed that a simple Fully Connected (FC) layer outperformed traditional ML models (SVM, Random Forest, etc.) and other deep classifiers.

3. Key Contributions

Novel HFR Architecture: Introduction of a hierarchical framework that integrates coarse (frame), medium (face), and fine-grained (eyes, lips, nose) features, addressing the limitation of face-only detectors.
Inter-Regional Dependency Modeling: The use of a channel-attention mechanism within a triplet learning framework to explicitly model the relationships between different facial parts, rather than treating them as isolated inputs.
Superior Generalization: The method is designed to handle "unseen" datasets and manipulation types by learning robust, transferable features across different granularities.
Comprehensive Evaluation: Extensive testing on 8 benchmark datasets (FF++, Celeb-DF, DFD, DFDC, DTIM, PDD, WLDR) covering intra-dataset, inter-dataset, and inter-manipulation scenarios.

4. Experimental Results

The proposed method was evaluated against numerous SoTA methods (e.g., Xception, EfficientNet, F3Net, DSM) using the Area Under the Curve (AUC) metric.

Intra-Dataset Performance: Face2Parts achieved near-perfect or perfect scores on several datasets.
- WLDR: 100% AUC.
- DTIM: 100% AUC.
- FF++ (FaceSwap): 99.70% AUC.
Inter-Dataset Performance (Generalization): When trained on FF++ and tested on unseen datasets, Face2Parts significantly outperformed competitors.
- CDF1: 79.80% (vs. ~77% for others).
- CDF2: 85.34% (vs. ~74% for others).
- DFD: 89.41%.
- DFDC: 84.07%.
- PDD (Lip-sync): 80.76% (a significant improvement over face-only methods which struggled with lip manipulations).
Inter-Manipulation Performance: The model demonstrated strong robustness when trained on one manipulation type (e.g., FaceSwap) and tested on others (e.g., NeuralTextures), consistently outperforming specialized models.
Temporal Encoders: The method was also tested with temporal encoders (MViT, TVC, ViViT), showing that while spatial encoders (CLIP ViT) are highly effective, temporal models offer complementary benefits for specific manipulation types like lip-sync.

5. Significance and Conclusion

Face2Parts represents a paradigm shift in deepfake detection by moving away from single-region analysis to a holistic, multi-scale approach.

Robustness: By leveraging coarse-to-fine information, the model captures both global inconsistencies (frame level) and local artifacts (fine-grained parts), making it resilient to diverse generation techniques.
Generalization: The results prove that focusing on inter-regional dependencies allows the model to generalize effectively to unseen datasets and manipulation types, a critical requirement for real-world deployment.
Future Work: The authors acknowledge the computational overhead of extracting multiple regions and plan to optimize the architecture for efficiency. They also aim to test the model under adversarial conditions (compression, filtering) and explore multi-modal approaches.

In summary, Face2Parts establishes a new benchmark for generalized deepfake detection, demonstrating that a hierarchical, attention-driven analysis of facial parts is superior to traditional face-centric approaches.