🤖 AI

AI-Powered Facial Mask Removal Is Not Suitable For Biometric Identification

This paper presents a large-scale analysis demonstrating that AI-powered facial mask removal, often used in crowd-sourced investigations, fails to produce reliable biometric matches and poses significant risks of misidentification.

Original authors: Emily A Cooper, Hany Farid

Published 2026-03-31

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Emily A Cooper, Hany Farid

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

🕵️‍♀️ The Big Idea: Why "AI Unmasking" is a Bad Idea for Police Work

Imagine you are trying to identify a suspect in a crime. You have a grainy photo of their face, but they are wearing a mask or the photo is blurry. Someone suggests, "Let's use AI to guess what their face looks like underneath!"

This paper says: Stop. That is a terrible idea.

The authors, researchers from UC Berkeley, ran a massive experiment to prove that when you use AI to "fill in the blanks" of a masked face, the result is not the real person. It's more like a creative artist's guess than a forensic fact. Using these AI-generated faces to identify someone could lead to innocent people being wrongly accused.

🎨 The Experiment: The "Fill-in-the-Blanks" Game

To test this, the researchers played a game with four different groups of faces:

Strangers: 400 random people.
Twins: 100 people, each with two photos (same person, different day).
Politicians: 91 US Senators wearing masks in one photo and no mask in another.
Lookalikes: 63 pairs of celebrities who look very similar but aren't related.

They took these faces, covered the bottom half (like a mask), and asked three famous AI tools (ChatGPT, Gemini, and GrokAI) to "draw" the missing mouth and chin.

The Analogy:
Think of it like a "Connect the Dots" puzzle where half the dots are missing.

The Real Goal: You want the AI to draw the exact missing dots so the picture matches the original person perfectly.
What Actually Happened: The AI didn't look up the person's file. Instead, it looked at millions of other faces in its training data and said, "Hmm, most people with this nose shape have a mouth like this." It drew a statistically average mouth, not the specific mouth of the person in the photo.

📊 The Results: The "Guess" vs. The "Real Thing"

The researchers used a high-tech "face scanner" (a biometric system) to compare the Original Photo against the AI-Generated Photo.

Here is how they scored it (on a scale of -1 to 1, where 1 is a perfect match):

Same Person (Real Photos): Score of 0.71. (This is what we expect when comparing a person to themselves).
Total Strangers: Score of 0.04. (These look nothing alike).
Famous Lookalikes: Score of 0.15. (They look a bit similar, but not the same).
AI-Generated Faces: The scores landed right in the middle, around 0.34 to 0.52.

The Takeaway:
The AI-generated faces were not the real person. They were better than a random stranger, but they were nowhere near close enough to be the same person.

The Analogy:
Imagine you are trying to identify your best friend, "Bob."

Real Bob: You see him, and you say, "That's Bob!" (100% confidence).
AI Bob: The AI draws a picture of a guy who looks a bit like Bob. He has Bob's nose and hair, but the mouth is wrong. You might say, "Hey, that looks like Bob's cousin," but you wouldn't arrest that guy thinking it's Bob.
The Danger: If a crowd of people on the internet sees the AI drawing and says, "That's definitely Bob!" they might start accusing the wrong person.

🎲 The "Roll of the Dice" Problem

The researchers also discovered that AI is non-deterministic. This is a fancy way of saying: If you ask the AI the same question twice, it gives you two different answers.

They took one photo of a Senator and asked ChatGPT to unmask it 100 times.

Result: They got 100 different faces.
Some looked a little like the Senator; some looked very different.
The Analogy: It's like asking a chef to "make a burger" 100 times. Sometimes they give you a cheeseburger, sometimes a veggie burger, sometimes a burger with extra pickles. They are all "burgers," but they aren't the same burger. You can't use a random burger to prove who ordered it.

⚖️ Why This Matters (The "GrokAI" Incident)

The paper mentions a real-world disaster that sparked this research. A masked ICE agent killed a civilian. Someone used AI (GrokAI) to remove the mask. The AI drew a face that looked like a random guy named "Steve Grove."

Because the AI's guess looked "realistic" enough, people on social media convinced themselves that Steve Grove was the killer. It was a complete misidentification caused by the AI hallucinating a face that didn't exist.

🏁 The Final Verdict

Generative AI is a great artist, but a terrible detective.

What it does well: It creates beautiful, realistic-looking images based on what is likely to be there.
What it fails at: It cannot reconstruct the exact truth of a specific person's face.

The Bottom Line:
If you see an "AI-unmasked" photo of a criminal, do not trust it. It is a work of fiction, not a piece of evidence. Using these images for biometric identification (like matching a face to a database) is scientifically unsound and dangerous.

1. Problem Statement

The paper addresses the growing trend of "crowd-sourced online investigations" where non-experts utilize Generative AI to enhance low-quality visual evidence, specifically to remove facial masks or occlusions.

Context: High-profile incidents (e.g., the misidentification of a masked ICE agent as Steve Grove, and confusion regarding a surveillance photo of a shooter) have demonstrated that AI-generated "unmasked" images can lead to severe misidentifications.
Core Question: Can commercial AI tools reliably reconstruct a masked face such that the resulting image can be biometrically matched to the true identity of the individual?
Hypothesis: The authors posit that because Generative AI relies on statistical inference rather than factual reconstruction, the resulting images are "hallucinations" that lack the specific biometric fidelity required for identification.

2. Methodology

The study employed a large-scale, controlled experimental design using four distinct datasets and three commercial AI models.

A. Datasets

Different-IDs: 400 distinct identities (balanced by gender, race, and age) with one image each.
Same-IDs: 100 identities, each with two images in different environments/lighting/poses (from VGGFace2).
US-Senators: 91 members of the 116th US Senate, featuring pairs of a masked photo (COVID era) and an official unmasked portrait.
Doppelgangers: 63 pairs of celebrity lookalikes (excluded non-frontal or mismatched gender/race pairs).

B. AI Unmasking Process

The authors tested three major AI models: OpenAI's ChatGPT (gpt-image-1.5), Google's Gemini (gemini-2.5-flash-image), and X's GrokAI.

Simulation 1 (Lower-Half Removal): For the Different-IDs dataset, the lower half of the face was digitally obscured (simulating a mask). The prompt was: "Given this image of the top half of a face, return an image showing the full face."
Simulation 2 (Real Mask Removal): For the US-Senators dataset, existing COVID masks were removed. The prompt was: "Create a realistic reconstruction of the facial features that would be beneath this medical mask."
Control: Prompts included "This is for academic research" to bypass content moderation filters.

C. Biometric Evaluation

Metric: ArcFace, a deep neural network-based facial recognition system.
Scoring: Cosine similarity between 512-D embeddings, yielding a score between -1 (mismatch) and 1 (perfect match).
Calibration: Scores were benchmarked against:
- Same-ID pairs (Ground truth for match).
- Different-ID pairs (Ground truth for mismatch).
- Doppelganger pairs (Ground truth for high-similarity false positives).

3. Key Results

A. Biometric Similarity Scores

The study found that AI-generated faces consistently failed to reach the similarity threshold of the true identity.

Baseline (Same-ID): Mean similarity = 0.71.
Baseline (Different-ID): Mean similarity = 0.04.
Baseline (Doppelgangers): Mean similarity = 0.15.
AI Unmasking Results:
- ChatGPT: Mean = 0.34 (SD 0.10).
- Gemini: Mean = 0.52 (SD 0.10).
- GrokAI: Mean = 0.51 (SD 0.10).

Analysis: While Gemini and GrokAI performed better than ChatGPT, all scores were significantly lower than the true identity baseline (Cohen's D ranging from -2.1 to -4.1). However, they were significantly higher than random mismatches, falling into a "danger zone" where they resemble the person more than a stranger but not enough to be a reliable match.

B. Real Mask vs. Simulated Removal

Removing real COVID masks from US Senators yielded even lower scores (ChatGPT: 0.41, Gemini: 0.38) compared to simulated lower-half removal.

Reasoning: Real-world photos often differ in pose, lighting, and facial hair from the reference portrait, compounding the AI's difficulty.

C. Variability and Non-Determinism

Intra-model Variability: Generating 100 unmasked images of a single Senator using ChatGPT resulted in scores ranging from 0.29 to 0.54 (Mean 0.42). This high variance proves the output is stochastic, not deterministic.
Inter-model Variability: The correlation between ChatGPT and Gemini scores was low ( $R^2 = 0.18$ ), indicating different models hallucinate different facial features.

D. Demographic Bias

Statistical analysis (2-way ANOVA) revealed significant biases:

Gender: Both models produced significantly lower similarity scores for females than males.
Race:
- ChatGPT: Performed best on South Asian faces.
- Gemini: Performed worst on East Asian faces.

4. Key Contributions

Empirical Evidence of Failure: Provides the first large-scale quantitative analysis proving that current Generative AI cannot reliably reconstruct masked faces for biometric identification.
Quantification of "Hallucination": Demonstrates that AI unmasking produces images that are statistically distinct from the true identity, effectively creating a "new" face that shares some features but fails biometric verification.
Bias Identification: Highlights that these failures are not uniform; they vary significantly by gender and race, potentially exacerbating existing biases in surveillance and identification.
Risk Assessment: Directly links the technical limitations of AI to real-world legal and social risks (e.g., wrongful accusations based on misidentified AI-enhanced photos).

5. Significance and Conclusion

The paper concludes that AI-powered facial mask removal is unsuitable for biometric identification.

Theoretical Limitation: Generative AI models operate on statistical inference (predicting the most likely appearance of a face given a prompt) rather than veridical reconstruction (recovering the actual hidden pixels). Therefore, an unmasked image is a "good guess" at best, not a factual representation.
Implications:
- Legal/Judicial: AI-enhanced images should not be admitted as evidence for identification purposes in criminal investigations.
- Security: Relying on these tools for security screening or identity verification is dangerous due to high false-positive rates and demographic biases.
- Future Outlook: Even as models improve, unless they move beyond statistical inference to factual reconstruction (which is currently impossible without the underlying data), they will remain unreliable for biometric matching.

The authors warn that the "photo-realistic" nature of these images creates a dangerous illusion of truth, leading to misidentifications that can have severe consequences for civil liberties and justice.