💻 computer science

CUPID: Reconstructing UV Texture Maps for Interpretable Person-of-Interest Deepfake Detection

This paper introduces CUPID, a robust, efficient, and interpretable Person-of-Interest deepfake detector that leverages UV texture maps and Masked Autoencoders to identify forged videos without requiring deepfake data or specific target identities during training, while providing visual residual maps to highlight manipulated facial regions.

Original authors: Giovanni Affatato, Sara Mandelli, Edoardo Daniele Cannas, Paolo Bestagini, Stefano Tubaro

Published 2026-07-13

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Giovanni Affatato, Sara Mandelli, Edoardo Daniele Cannas, Paolo Bestagini, Stefano Tubaro

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to spot a fake photo of a famous celebrity. Usually, you might squint at their eyes or check if their teeth look weird. But what if the fake is so good that even a human can't tell the difference? That's where CUPID comes in. It's a new digital detective designed to catch "Person-of-Interest" (POI) deepfakes—those scary videos where bad actors swap a real person's face onto a fake body or make them say things they never said.

Here's the twist: CUPID doesn't need to see a single fake video to learn how to catch them. In fact, the paper explicitly argues against the old way of doing things, where you had to train a separate, special robot for every single celebrity using thousands of fake examples. The authors show that this old method is slow, requires too much data, and often fails when the video gets compressed or resized (like when you watch a video on a shaky phone connection). Instead, CUPID learns from only real videos of many different people to understand what a "genuine" face looks like, and then uses that knowledge to spot fakes of anyone, even those it has never met before.

The Magic Map: Unfolding the Face

So, how does it work? Imagine a celebrity's face is a 3D clay sculpture. If you take a photo, the nose might look different depending on the angle, or the lighting might make the cheek look dark. This makes it hard for computers to compare faces.

CUPID uses a clever trick called a UV Texture Map. Think of this as taking that 3D clay face, peeling the skin off like a sticker, and laying it flat on a table. On this flat map, the nose is always in the same spot, the eyes are always in the same spot, and the mouth is always in the same spot, no matter how the person turns their head or smiles. This "flat map" removes the confusion of angles and lighting, giving the computer a clean, standardized blueprint to work with.

The "Fill-in-the-Blanks" Game

Once the face is flattened into this map, CUPID plays a game called "Masked Autoencoding." Imagine you have a puzzle, but someone covers 50% of the pieces with black tape. CUPID looks at the visible pieces and tries to guess what's under the tape.

To get really good at this, CUPID trains on a massive library of real videos (from a dataset called VoxCeleb2 with over 6,000 different people). It learns that if it sees a left eye, the nose should be a certain distance away, and the mouth should have a specific shape. It learns the "rules" of how real human faces are built. The paper proves that by mastering these rules using only real data, the system becomes an expert at spotting when a face breaks those rules.

Catching the Imposter

When a new video arrives—say, a clip of a politician that might be fake—CUPID peels that face into a UV map and tries to fit it into the "real face" blueprint it learned.

If it's real: The face fits perfectly into the blueprint. The computer says, "Yep, this matches the pattern of a real human."
If it's a deepfake: The AI tries to fill in the gaps, but the fake face has weird glitches or impossible shapes. The computer says, "Whoa, the nose is in the wrong spot for this blueprint! This is a fake."

The paper shows that this method is extremely robust. Even when videos are shrunk down to tiny sizes or squashed with heavy compression (like when you download a video from social media), CUPID keeps working. In tests on four different datasets, it outperformed other top detectors, especially in these "low-quality" conditions where others failed. For example, on the "FakeAVCeleb" dataset, when videos were compressed, CUPID maintained a high accuracy score (around 87.5%), while other methods dropped significantly.

The "Why" Behind the Verdict

One of the coolest features is that CUPID is interpretable. Most AI detectors just give you a "Yes/No" answer. CUPID, however, can show you why it thinks a video is fake. It generates a "heat map" that highlights exactly which part of the face is suspicious.

If the deepfake swapped the mouth, the heat map glows red around the lips.
If the eyes look weird, the glow moves to the eyes.
The paper demonstrates this by showing that for fake videos, these "glow spots" are much brighter and more concentrated than for real videos, giving investigators a visual clue to trust.

Speed and Efficiency

Finally, CUPID is a speedster. While other methods might scan every single frame of a long video (which takes forever), CUPID only needs to look at 10 frames from the whole clip to make a decision. This makes it 32 times faster than the previous best method for celebrity detection. The authors measured this on over 2,000 videos, finding that CUPID took an average of just 0.155 seconds per video, compared to nearly 5 seconds for the competitor.

What It Doesn't Do (Yet)

It's important to know what CUPID isn't perfect at. The paper notes that it struggled a bit more with a dataset called KoDF, which features Korean subjects. The authors suggest this is because their training data didn't have enough people from that specific demographic, showing that the system can still be sensitive to differences in ethnicity. Also, while it's great at spotting face swaps, it doesn't explicitly model how a person moves over time (like lip-syncing), so it might not catch every single type of audio-driven fake yet.

In short, CUPID is a new kind of lie detector that doesn't need to see a lie to know what one looks like. By flattening faces into maps and playing "fill-in-the-blanks" with real data, it builds a super-sharp sense of what a real human looks like, allowing it to spot fakes quickly, even when the video quality is terrible.

Technical Summary: CUPID – Reconstructing UV Texture Maps for Interpretable Person-of-Interest Deepfake Detection

Problem Statement

The proliferation of generative AI has enabled the creation of highly realistic deepfakes targeting specific high-profile individuals, known as Persons-of-Interest (POI). These synthetic media pose significant threats to democratic processes and societal trust. While the forensics community has developed specialized methods to detect POI deepfakes, existing solutions face a critical triad of challenges:

Robustness: Many methods struggle to maintain performance under common post-processing operations like compression and downscaling.
Efficiency: Current detectors often require computationally expensive inference, particularly those utilizing sliding-window strategies.
Interpretability: Few methods provide intuitive visual explanations for why a video is flagged as fake, limiting their utility in forensic analysis.

Furthermore, existing POI-specific approaches often require training a separate model for each target identity (requiring large amounts of specific data) or fail to generalize well to unseen identities without extensive calibration.

Methodology

The authors propose CUPID (ReConstructing UV texture maps for Person-of-Interest Deepfake detection), a visual-only detector that operates without requiring any deepfake samples during training and remains POI-agnostic during the training phase. The pipeline consists of three main stages:

1. UV Texture Map Extraction

Instead of processing raw video frames, CUPID converts facial appearance into UV texture maps.

Process: For each frame, a 3D Morphable Model (3DMM) is fitted to the face. The model projects the facial texture onto a canonical 2D UV space.
Advantage: This representation establishes dense semantic correspondence; specific pixel coordinates in the UV map correspond to the same facial region (e.g., nose tip, eyes) across all identities, poses, and expressions. This disentangles identity features from pose and expression variations, providing a standardized input that enhances robustness to post-processing.

2. Masked Autoencoder (MAE) Training

A Vision Transformer (ViT)-based Masked Autoencoder is trained exclusively on real video data (VoxCeleb2) to learn a general latent space of facial identity.

Architecture: The encoder processes visible patches of the UV texture map, while a lightweight decoder reconstructs the masked patches.
Training Objective: The model is optimized using a joint loss function comprising:
- Masked Reconstruction Loss ( $L_{rec}$ ): Per-pixel MSE on masked patches to learn rich texture representations.
- Contrastive Loss ( $L_{cont}$ ): A multi-layer NT-Xent loss that clusters embeddings of the same identity and separates different identities across multiple encoder depths.
- Perceptual Loss ( $L_{perc}$ ): A VGG-16 based loss to ensure perceptually coherent reconstructions and suppress grid artifacts.
Outcome: The resulting encoder produces identity-discriminative embeddings that generalize to unseen subjects.

3. Inference and Detection

During the testing phase, the system operates in an open-set scenario:

Reference Set Construction: A set of pristine reference videos for the target POI is encoded into the latent space to form a reference distribution (represented by the centroid of CLS embeddings).
Similarity Scoring: A test video is encoded, and its embeddings are compared against the reference distribution using a pairwise similarity function (maximum inner product).
Decision: If the similarity score falls below a threshold $\gamma$ , the video is classified as fake.

4. Interpretability Mechanism

CUPID leverages the structured nature of UV maps and the latent space to generate residual maps:

The system decodes the latent displacement between the test video and the reference centroid.
By subtracting the "natural" variability observed in the reference set, the method produces a map highlighting specific facial regions where the test video deviates most significantly from the POI's genuine identity. This provides visual evidence of manipulation (e.g., anomalies around the mouth or eyes).

Key Contributions

Novel POI Detector: Introduction of CUPID, which combines UV texture maps with a MAE trained solely on real data. It learns a general identity-aware representation applicable to unseen POIs at inference time.
Interpretability Framework: A procedure that utilizes the UV space and latent representations to generate subject-level residual maps, offering intuitive visual explanations for detection decisions.
Comprehensive Evaluation: Extensive benchmarking against state-of-the-art general detectors and a POI-specific baseline across four datasets (DF-TIMIT, FakeAVCeleb, KoDF, DeepSpeak) under both High Quality (HQ) and Low Quality (LQ) settings.

Experimental Results

Experiments demonstrate that CUPID achieves superior performance in several key areas:

Robustness: CUPID outperforms current state-of-the-art methods, particularly under strong compression and downscaling (LQ settings). While general baselines suffer an average performance drop of ~24 points in LQ settings, CUPID's performance degrades by only ~3.2 points.
Performance: It achieves the best overall robustness and competitive accuracy in HQ settings, often surpassing the POI-specific baseline (POI-Forensics) across most datasets.
Efficiency: CUPID is significantly faster, achieving inference speeds 32 times faster than POI-Forensics. This is attributed to its strategy of sampling a fixed, small number of frames (10) per video, independent of clip duration, whereas competitors use sliding-window approaches that scale with video length.
Threshold Stability: The method exhibits a more compact distribution of optimal decision thresholds across subjects, indicating that it requires less per-subject calibration compared to existing POI detectors.
Interpretability: Qualitative analysis of residual maps confirms that fake videos produce higher activation values in specific facial regions consistent with the manipulation techniques used (e.g., blending artifacts in face-swaps).

Significance and Limitations

The paper posits that CUPID demonstrates that identity-contextual reconstruction and similarity cues derived from UV facial representations provide a strong signal for deepfake detection, even without exposure to fake data during training. This suggests that the structured nature of UV maps effectively isolates identity features from manipulation artifacts.

The authors acknowledge two primary limitations:

Temporal Dynamics: The current video-only representation is identity-centric and does not explicitly model temporal expressivity, potentially limiting effectiveness against LipSync manipulations.
Ethnicity Bias: The method showed slightly weaker results on the KoDF dataset (focused on Korean subjects), suggesting sensitivity to ethnicity-related distribution shifts due to the training data (VoxCeleb2) lacking sufficient demographic diversity.

The authors conclude that future work will focus on incorporating subject-specific dynamics and training on more ethnically diverse datasets to improve fairness and generalization.