💻 computer science

Towards Unified Multimodal Misinformation Detection in Social Media: A Benchmark Dataset and Baseline

This paper introduces OmniFake, a large-scale benchmark dataset combining human-crafted misinformation and AI-generated content, along with the UMFDet framework that utilizes a category-aware mixture-of-experts architecture to unify and outperform specialized models in detecting diverse types of multimodal deception on social media.

Original authors: Haiyang Li, Yaxiong Wang, Shengeng Tang, Yuchen Zhang, Lianwei Wu, Lechao Cheng, Liu Liu, Chaofeng Dong, Zhun Zhong

Published 2026-07-31

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Haiyang Li, Yaxiong Wang, Shengeng Tang, Yuchen Zhang, Lianwei Wu, Lechao Cheng, Liu Liu, Chaofeng Dong, Zhun Zhong

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the internet as a giant, bustling town square where everyone is shouting stories, sharing photos, and posting news. In this square, there are two main kinds of troublemakers. The first kind is the "Human Trickster"—a person who writes a fake story or edits a photo just to get attention, spread a rumor, or make people angry. The second kind is the "Robot Artist"—a powerful computer program that can paint a picture of a flying dog or write a convincing news headline about an event that never happened. For a long time, the people trying to keep the town square safe (the researchers) built two different security guards. One guard was an expert at spotting human lies, and the other was an expert at spotting robot fakes. But in the real world, a tricky post might come from a human, a robot, or a mix of both, and the security guards didn't know which one to call. They were like having a fire department that only knows how to put out wood fires and a police force that only knows how to catch bank robbers; when a fire starts in a bank, neither team knows what to do. This paper steps into that chaos to build a single, super-smart detective that can handle any kind of fake news, no matter who made it.

The authors of this paper, a team from universities and tech companies in China, realized that the old way of separating "human lies" from "AI fakes" was too slow and clunky for the real world. So, they created a massive new training ground called OmniFake. Think of this dataset as a giant library of 98,592 examples—almost 100,000 posts!—that includes real news, human-written rumors, and AI-generated tricks. They didn't just collect these; they made them. They took real photos and used AI to swap faces, change backgrounds, or even generate entirely new images of things that don't exist. They also took real headlines and used AI to rewrite them into misleading stories. They even mixed these techniques together to create "Mixed Manipulation," where a post has a real photo but a fake story, or a fake photo with a real story. To test how tricky this is, they asked eight human experts to look at 600 of these samples. The humans only got about 40% right, proving that these fakes are incredibly hard to spot.

To solve this, the team built a new detective system called UMFDet. Instead of hiring two separate guards, they built one smart brain with a special "team of experts" inside it. Imagine a central hub that receives a post (a picture and some text) and then asks, "Who is the best expert to look at this?" The system has three specialized experts: one for Real news, one for Human-crafted lies, and one for AI-synthesized fakes. This is called a "Category-aware Mixture-of-Experts." When a post comes in, the system routes it to the right expert. But here's the clever part: they made sure these experts don't get confused. They used a special training rule (called "Expert-wise Discriminative Regularization") to force the "Human Expert" to stay very close to other human lies and far away from AI fakes, and vice versa. They also added a "Cross-modal Consistency" check, which acts like a fact-checker that asks, "Does the picture actually match the story?" If the text says "a flying dog" but the picture shows a cat, this check flags it as suspicious.

The results show that this new detective is much better than the old specialists. When tested on their new 98,000-sample library, UMFDet correctly identified the type of misinformation about 82% of the time on average. This is a big jump compared to other methods. For example, when looking at "Text Manipulation" (fake stories), the new system improved the best previous method by over 13%. It also proved it could handle "Mixed Manipulation" (a mix of real and fake) better than anyone else. The team even tested their system on other existing datasets, like DGM4 and MMFakeBench, and it still came out on top, beating specialized models that were designed for just one type of fake.

The paper suggests that by unifying these tasks, we can build a more practical and powerful tool for social media safety. The authors don't claim to have solved the problem of fake news forever, but they have shown that treating human lies and AI fakes as a single, unified challenge works much better than keeping them apart. Their system, along with their massive new dataset, is now available for others to use and improve upon, offering a promising step toward a town square where the truth is easier to find, no matter who is trying to hide it.

Technical Summary: Towards Unified Multimodal Misinformation Detection in Social Media

1. Problem Definition

The proliferation of deceptive multimodal content on social media has evolved into a dual-threat landscape comprising human-crafted misinformation (e.g., rumors, misleading posts) and AI-generated content (e.g., deepfakes, synthetic media from vision-language models). Historically, these two categories have been treated as separate research tasks, leading to specialized models that excel in one domain but fail in the other. In real-world deployment, the source of an incoming post is typically unknown, rendering specialized, siloed systems impractical. Furthermore, existing approaches struggle with the fine-grained diversity of manipulation types and the interaction between human and AI-generated cues, often resulting in performance degradation when faced with mixed or novel deception strategies.

2. Methodology

2.1 The OmniFake Benchmark

To address the lack of unified evaluation, the authors constructed OmniFake, a large-scale benchmark dataset containing 98,592 image-text pairs. The dataset is curated from existing resources (r/Fakeddit, VisualNews) and newly generated synthetic data. It covers five distinct categories:

Real: Authentic news posts.
Human-crafted: Misleading posts created by humans (e.g., rumors).
Vision Manipulation: AI-generated or edited images (face swapping, attribute editing, full image generation, inpainting).
Text Manipulation: AI-generated or rewritten text (pure fake generation, keyword distortion).
Mixed Manipulation: Combinations of the above.

The dataset construction pipeline involves using Large Multimodal Models (LMMs) like Qwen3-VL and FLUX to generate realistic synthetic images and texts, ensuring high fidelity and diverse manipulation patterns across various domains. Human evaluation on a subset of 600 samples revealed that humans achieve only 40% accuracy, validating the dataset's difficulty and the deceptive quality of the AI-generated samples.

2.2 The UMFDet Framework

The authors propose Unified Multimodal Fake Content Detection (UMFDet), a framework designed to handle all deception types simultaneously. The architecture is built upon a Vision-Language Model (VLM) backbone (specifically Florence-2-0.7B) and incorporates three key innovations:

Category-aware Mixture-of-Experts (CMoE) Adapter:
Instead of a single classifier, UMFDet employs a router that dynamically selects one of three specialized experts based on the input:
- Reality Expert ( $E_R$ ): Specialized for authentic content.
- Deception Expert ( $E_D$ ): Specialized for human-crafted misinformation.
- Synthesis Expert ( $E_S$ ): Specialized for AI-synthesized content.
  This allows the model to leverage category-specific cues rather than forcing a single representation to handle all deception types.
Expert-wise Discriminative Regularization (EDR):
To prevent expert collapse and ensure specialization, EDR enforces intra-expert compactness and inter-expert separation. It calculates class prototypes within the routed sample sets for the two most active experts in a batch, minimizing intra-class variance while maximizing the distance between different expert prototypes.
Cross-modal Consistency Alignment (CCA):
Recognizing that human-crafted misinformation often pairs unrelated images with misleading text (weak semantic alignment), while real news exhibits strong consistency, CCA is introduced. It aligns the visual and textual embeddings of samples routed to the "Reality" and "Deception" experts, enforcing a prior where real news has high cross-modal similarity and deceptive human-crafted news has lower similarity.

The total training objective is a joint optimization of the language modeling loss, EDR, and CCA.

3. Key Contributions

OmniFake Dataset: A comprehensive, large-scale benchmark unifying human-crafted and AI-generated misinformation into a single evaluation framework with five fine-grained categories.
UMFDet Framework: A unified detection model that integrates a VLM backbone with a CMoE adapter, EDR, and CCA to simultaneously detect diverse deception types without prior knowledge of the source.
Extensive Evaluation: Comprehensive experiments demonstrating that UMFDet outperforms specialized baselines and state-of-the-art methods across multiple datasets (OmniFake, DGM4, MMFakeBench, PHEME) and generalizes well to open-world scenarios.

4. Experimental Results

OmniFake Performance: UMFDet achieved an average F1-score of 82.23% across the five categories, significantly outperforming the best baseline (HAMMER, 70.81% F1). Notably, it improved the F1-score for Text Manipulation by 13.07% over the strongest baseline.
Generalization:
- On DGM4, UMFDet achieved 83.20% F1, surpassing HAMMER by 2.39%.
- On MMFakeBench, it achieved 61.74% F1, outperforming standard 7B models (52.5% F1) and MMD-Agent enhanced 13B models (50.2% F1).
- On PHEME, it reached 74.3% F1, beating the best LLM-based variant by 9.7%.
Ablation Studies: The study confirmed that each component contributes to performance. The full model (CMoE + EDR + CCA) outperformed the base CMoE model by 6.24% in F1. One MoE layer was found optimal, and three experts yielded the best results compared to 2, 4, or 5.
Open-World Test: In a test with 100 real-time news samples, UMFDet achieved 51.04% F1, outperforming HAMMER (46.26%) and FKA-OWL (41.37%).

5. Significance and Claims

The paper claims that the primary significance of this work lies in shifting the paradigm from specialized, siloed detection to unified multimodal misinformation detection. By acknowledging that human-crafted and AI-generated content coexist and interact in the wild, the authors argue that a unified framework is necessary for practical deployment.

The authors position OmniFake as a critical resource for the community to evaluate models under realistic, mixed-source conditions, moving beyond binary "real vs. fake" classification to fine-grained recognition. They assert that UMFDet provides a practical solution that not only matches but exceeds specialized systems in their respective domains while maintaining robustness across diverse manipulation types, thereby addressing the limitations of current systems that fail when the deception source is unknown.