The Deepfakes We Missed: We Built Detectors for a Threat That Didn't Arrive
This position paper argues that the machine learning community's decade-long focus on detecting public-figure deepfakes has misaligned with reality, as the actual dominant threats have shifted to non-consensual intimate imagery, voice-clone scams, and emotional fraud, creating a critical bottleneck in real-world defense that requires an immediate rebalancing of research agendas.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
The Big Picture: Building the Wrong Shield
Imagine a group of engineers spends ten years designing the ultimate fire extinguisher. They spend all their time and money testing it against massive, roaring forest fires that they think are going to happen. They build huge tanks, create complex spray patterns, and write manuals on how to fight a blaze that could burn down a whole city.
But when the "fire season" finally arrives, the massive forest fires never happen. Instead, the actual danger turns out to be thousands of small, hidden kitchen fires starting in people's homes, and gas leaks in their basements.
The engineers are still standing there with their giant forest-fire extinguishers, ready to save the city, while the real damage is happening in the kitchens and basements. The paper argues that the field of AI deepfake detection is exactly in this situation.
The "Forest Fire" That Never Happened (T1)
For the last decade, researchers have been obsessed with one specific type of deepfake: fake videos of famous people (like politicians or movie stars) saying things they never said.
- The Fear: Everyone was terrified that a fake video of a President declaring war or a celebrity endorsing a scam would cause chaos in elections and destroy trust in video evidence.
- The Reality: In the 2024 global elections, this specific disaster did not happen. While some fake videos of politicians did appear, regular people, journalists, and fact-checkers spotted them. They didn't need a super-computer AI detector to find them; human eyes did the job.
- The Research: Despite this, 71% of all research papers and datasets are still focused entirely on catching these "famous person" videos. It's like the engineers are still polishing their forest-fire extinguishers.
The "Kitchen Fires" That Actually Happened (T2, T3, T5)
While researchers were looking at the sky for the big fire, the real damage was happening in three specific areas that got very little attention:
The "Kitchen Fire" (Peer-Generated NCII):
- What it is: Regular people (often students or acquaintances) using AI to create fake, intimate images of other regular people without their permission.
- The Scale: This is exploding. In 2025, reports showed a 260-fold increase in AI-generated child sexual abuse material and non-consensual intimate images.
- The Gap: Researchers have almost zero detectors for this. It's like having a fire extinguisher for forests but no smoke detectors for homes.
The "Gas Leak" (Voice-Clone Scams):
- What it is: Scammers using AI to clone the voices of family members or bosses to trick people into sending money.
- The Scale: In 2025, these scams caused nearly $900 million in losses. A famous case involved a finance worker sending $25 million because they thought they were on a video call with their boss (who was actually an AI avatar).
- The Gap: Most research looks at pre-recorded audio in a studio. But these scams happen in real-time phone calls with bad connection quality. The current detectors are too slow and don't work on phone lines.
The "Hidden Pipes" (Messaging Apps):
- What it is: Fake content spreading through private group chats (like WhatsApp or Telegram) where encryption hides the content from public view.
- The Scale: This is how the "kitchen fires" (the intimate image scams) spread.
- The Gap: Current detectors are built for the "open web" (like YouTube or Twitter). They can't see inside private, encrypted messages.
Why Are We Still Looking at the Wrong Thing?
The paper asks: If the big forest fires didn't happen, why are we still building forest-fire extinguishers?
It identifies three reasons why the research community is stuck:
- The "Test Score" Trap: Researchers need to publish papers to get jobs and funding. To do this, they use standard "test sets" (like FaceForensics++). These tests only have famous people in them. If you try to test a detector for voice scams or private images, there is no standard test to use. It's like trying to get a driver's license, but the only test available is driving a race car on a track, even though you need to drive a minivan in the city.
- The "Privacy Wall": It is very easy to get pictures of celebrities to train AI. It is illegal and unethical to collect private photos of victims or recordings of scam calls. Because the data is hard to get, researchers avoid working on it.
- The "Headline" Effect: A fake video of a President makes the nightly news. A fake video of a teenager being bullied in a group chat does not. Researchers and funders follow the headlines, not the actual statistics of who is getting hurt.
The Solution: Change the Map
The paper doesn't say we should stop studying famous-person deepfakes entirely. Instead, it argues we need to rebalance our efforts.
It proposes three new research agendas:
- Real-Time Phone Detectors: Build AI that can listen to a live phone call and say "Wait, that voice sounds fake" in less than a second, even with bad phone connection.
- Private "Home" Detectors: Create tools that run on your own phone to check if a photo is fake before you share it, without sending your private photos to a central server (to protect privacy).
- Chat-App Defenses: Develop ways to detect fake content inside private group chats without breaking encryption.
The Bottom Line
The paper concludes that the biggest problem in fighting deepfakes isn't that our AI isn't smart enough. The problem is that we are using a map from 2017 to navigate a 2026 world. We built detectors for a threat that didn't arrive, while the threats that did arrive are growing unchecked. It's time to put down the forest-fire extinguisher and start building smoke detectors for the homes where the real danger is.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.