💻 computer science

Beyond Deepfake vs Real: Facial Deepfake Detection in the Open-Set Paradigm

This paper proposes a supervised contrastive learning-based open-set deepfake detection framework that moves beyond the limitations of closed-set paradigms by effectively identifying both known forgeries and flagging unseen, unknown manipulation methods as distinct from real images, achieving state-of-the-art performance in handling emerging forgery techniques.

Original authors: Nadarasar Bahavan, Sachith Seneviratne, Sanjay Saha, Ken Chen, Sanka Rasnayaka, Saman Halgamuge

Published 2026-03-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Nadarasar Bahavan, Sachith Seneviratne, Sanjay Saha, Ken Chen, Sanka Rasnayaka, Saman Halgamuge

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a security guard at a very exclusive club. Your job is to check IDs at the door.

The Old Way (The "Closed-Set" Problem)
In the past, security guards were trained with a photo album of known fake IDs. They learned to spot the specific smudges, fonts, or paper types used by a few known forgers.

The Flaw: If a criminal showed up with a brand-new type of fake ID that the guard had never seen before, the guard would look at it, say, "I don't recognize this specific fake," and assume, "Well, if it's not in my 'fake' book, it must be real!"
The Result: The new fake gets in. This is exactly how current AI deepfake detectors work. They are great at catching fakes they've seen before, but they get fooled by new, clever fakes because they assume anything they don't recognize is the truth.

The New Way (The "Open-Set" Paradigm)
This paper proposes a smarter security guard. Instead of just memorizing a list of "known fakes," this guard learns the feeling of what a real person looks like.

The Shift: When someone approaches the door, the guard asks: "Does this look like a real person? Does it look like one of the specific fakes I know?"
The "Unknown" Category: If the person looks suspicious but doesn't match any specific fake ID in the book, the guard doesn't guess. Instead, they raise a red flag and say, "I don't know who this is, and I don't trust it." They treat the unknown as "suspicious" rather than "innocent."

The Secret Sauce: The "Group Hug" Analogy

How did the authors teach the AI to do this? They used a technique called Supervised Contrastive Learning, but let's call it the "Group Hug" strategy.

Imagine a dance floor (the AI's brain):

Real People: In normal training, "Real" photos are scattered all over the dance floor, like people standing randomly. They don't stick together.
The Fix: The authors told the AI: "Hey, all the 'Real' people need to hold hands and form a tight, compact circle in the middle of the room."
The Fakes: The "Fake" photos (Deepfakes) are told to stand in their own separate groups, far away from the Real circle.

Why does this help?
If a new, unknown fake tries to enter the dance floor, it won't fit into the tight "Real" circle (because it's not real), and it won't fit into the known "Fake" groups (because it's a new type). It ends up standing alone in the empty space between the groups.

Old AI: "You aren't in the Fake group, so you must be Real." (Mistake!)
New AI: "You aren't in the Real circle, and you aren't in any Fake group. You are Unknown. Stop right there!"

What Did They Test?

They tested this new security guard on the FaceForensics++ dataset, which is like a massive library of deepfake videos created by four different "hackers" (algorithms).

They set up three scenarios:

The "Unknown" Test: They trained the AI on three hackers, then tested it on the fourth one (which it had never seen).
- Result: The new AI was much better at spotting this stranger than the old AI. It didn't get tricked into thinking the new fake was real.
The "Known" Test: They tested it on the hackers it did know.
- Result: It was just as good as the best existing security guards, proving it didn't lose its skills on old tricks.
The "General" Test: They asked, "Can you just tell me if anything is fake, even if I don't know the method?"
- Result: It won the race, beating almost every other method out there.

The Bottom Line

This paper is a game-changer because it stops deepfake detectors from being "overconfident."

In the real world, bad actors are constantly inventing new ways to fake faces. The old detectors would say, "I've never seen this before, so it's safe," and let the misinformation spread. This new method says, "I've never seen this before, so I'm not taking the risk."

It's the difference between a guard who only knows the faces of three specific burglars and a guard who knows exactly what a "normal" person looks like, so they can instantly spot anyone who doesn't fit the mold, no matter how they try to disguise themselves.

1. Problem Statement

Current deepfake detection systems predominantly operate under a closed-set paradigm. This means they are trained to distinguish between "Real" images and a specific set of "Known Fake" methods (e.g., DeepFakes, FaceSwap) present in the training data.

The Limitation: In real-world scenarios, new forgery techniques emerge constantly. A closed-set model, when encountering a deepfake generated by an unknown technique (one not seen during training), often fails to identify it as fake. Instead, it frequently misclassifies these unknown forgeries as "Real" because they do not match the specific patterns of the known fakes.
The Goal: The authors propose shifting to an Open-Set Paradigm. In this framework, the model must not only classify known fakes and real images but also explicitly identify and flag samples generated by unknown methods as "Unknown," rather than forcing them into the "Real" or "Known Fake" categories.

2. Methodology

The proposed framework is a three-stage pipeline designed to learn robust feature representations and establish decision boundaries that can reject unknown classes.

Stage 1: Weighted Supervised Contrastive Learning (Representation Learning)

The core innovation lies in modifying Supervised Contrastive Learning (SupCon) to address the specific challenge of deepfake detection:

The Challenge: In standard contrastive learning, real data often forms scattered representations in the latent space, while fake data forms tight clusters. This makes it difficult to distinguish real from fake, especially when unknown fakes appear.
The Solution: The authors introduce a Weighted Supervised Contrastive Loss.
- They assign a higher weight ( $\alpha$ ) to pairs of Real samples to encourage them to form a tight, compact cluster in the latent space.
- They assign a weight of 1 to pairs of Fake samples (regardless of the specific forgery type) to group them together but distinct from real.
- This forces the model to learn a feature space where "Real" is a distinct, dense cluster, and "Fake" (both known and unknown) occupies the surrounding space, making it easier to detect anomalies.
Architecture: Uses a DenseNet encoder followed by a projection head to generate 128-dimensional embeddings.

Stage 2: Supervised Fine-Tuning

The encoder weights are frozen (using Stochastic Weight Averaging for better generalization).
A classifier head is trained on top of the frozen encoder using standard Cross-Entropy Loss to predict the known classes (Real, Fake Type A, Fake Type B, etc.).

Stage 3: Open-Set Detection Algorithm

Threshold Estimation: The model calculates confidence scores (softmax probabilities) for correctly classified training samples. For each class, a rejection threshold ( $\epsilon_i$ ) is set based on a specific percentile of these scores (e.g., the 95th percentile).
Inference: When a test sample arrives:
1. The model calculates the probability of the sample belonging to each known class.
2. If the maximum probability is below the pre-determined threshold for that class, the sample is rejected and labeled as "Unknown."
3. This allows the system to flag emerging deepfake techniques without needing retraining.

3. Key Contributions

First Open-Set Deepfake Classification: This is the first work to explicitly investigate the open-set classification problem for deepfakes, moving beyond binary (Real vs. Fake) or multi-class (Real vs. Known Fakes) detection.
Weighted SupCon for Deepfakes: A novel adaptation of Supervised Contrastive Learning that prioritizes the compactness of the "Real" class, significantly improving the separation between real data and both known and unknown forgeries.
Robust Unknown Detection: The method achieves state-of-the-art (SOTA) performance in identifying unknown deepfake techniques, treating them as "Unknown" rather than misclassifying them as "Real."
Competitive Known Detection: Despite the open-set nature, the model maintains competitive performance in detecting known deepfakes compared to specialized closed-set detectors.

4. Experimental Results

The method was evaluated on the FaceForensics++ (FF++) dataset and the CelebDF dataset using three scenarios:

Scenario A: Detecting Unknown Deepfakes (Cross-Manipulation)
- Setup: Train on 3 forgery types + Real; Test on the 4th forgery type (Unknown).
- Result: The proposed model significantly outperformed the Xception baseline. For example, when detecting unknown NeuralTextures (NT), the proposed model achieved an AUROC of 0.8233, compared to Xception's 0.5985.
- Significance: Demonstrates the model's ability to recognize that a new forgery technique is not real, even if it doesn't know what technique it is.
Scenario B: Detecting Known Deepfakes
- Setup: Train on all known fakes; Test on known fakes.
- Result: The model achieved competitive AUROC scores (e.g., 99.43% for FaceSwap) against closed-set SOTA methods, proving that the open-set approach does not sacrifice accuracy on known threats.
Scenario C: General Deepfake Detection (Real vs. Any Fake)
- Setup: Treat all known and unknown fakes as a single "Fake" class vs. "Real."
- Metric: True Open-Set Classification (TOSC) Accuracy.
- Result: The proposed method achieved SOTA results, outperforming previous open-set approaches like Fusion and Xception. For instance, it achieved 83.95% accuracy on unknown DeepFakes, surpassing the previous best of 83.99% (Fusion) in specific configurations and significantly outperforming others.
Cross-Dataset Evaluation:
- Tested on CelebDF after training on FF++. While performance dropped due to domain shift (a known challenge), the proposed method remained competitive, outperforming several established detectors like Meso4 and Face X-ray.

5. Significance and Impact

Proactive Defense: By identifying "Unknown" forgeries, this system allows forensic experts and platforms to flag new threats immediately, rather than waiting for a new detector to be trained on the specific new technique.
Reduced False Negatives: It drastically reduces the risk of malicious deepfakes being misclassified as "Real" (False Negatives), which is the primary failure mode of current closed-set systems.
Forensic Utility: The ability to distinguish between "Known Fake," "Real," and "Unknown" provides a more nuanced and reliable tool for content moderation and legal forensics.
Future Direction: The paper highlights that while effective, the method is sensitive to domain shifts in "Real" data (e.g., different lighting or compression in CelebDF vs. FF++), suggesting future work should focus on video-level analysis and better handling of real-data distribution shifts.