MALicious INTent Dataset and Inoculating LLMs for Enhanced Disinformation Detection

This paper introduces MALINT, the first human-annotated English corpus of disinformation with malicious intent labels, and proposes an intent-augmented reasoning framework that leverages this dataset to significantly improve zero-shot disinformation detection across multiple languages and models.

The Gist

Imagine the internet as a giant, bustling town square. In this square, people share news, opinions, and stories. But lately, a group of troublemakers has been sneaking in, not just to tell lies, but to tell lies with a specific, malicious plan. They aren't just spreading rumors; they are trying to break the town's trust, start fights between neighbors, or convince everyone that the police are corrupt.

This paper is about building a better "lie detector" for this town square, one that doesn't just ask, "Is this story true or false?" but also asks, "What is the liar trying to achieve?"

Here is the breakdown of their work using simple analogies:

1. The Problem: The "Why" Matters More Than the "What"

Most previous tools for spotting fake news are like a security guard who only checks if a person's ID photo looks real. They check the facts. But they miss the intent.

• The Analogy: Imagine two people handing you a flyer.
  • Person A accidentally printed a flyer with a typo. (Mistake)
  • Person B printed a flyer designed to make you hate your neighbor. (Malice)
  • Old detectors might flag both as "bad." This paper argues we need to understand why Person B did it. Is it to change your vote? To make you distrust the government? To sell you fake medicine?

2. The New Tool: The "MALINT" Map

The researchers created a new dataset called MALINT. Think of this as a training manual for detectives.

• They gathered 1,600 news articles and had real-life fact-checking experts (the "Sherlocks" of the internet) label them.
• They didn't just say "Fake." They categorized the evil plan behind the fake news into five specific buckets:
  1. Undermining Trust: Trying to make you think the government or hospitals are lying.
  2. Changing Politics: Trying to make you hate one political side and love the other.
  3. Breaking Alliances: Trying to make countries hate each other (like making NATO look like a villain).
  4. Creating Hate: Trying to make groups of people (like refugees or minorities) look dangerous.
  5. Anti-Science: Trying to convince you that science is a scam (e.g., "vaccines are poison").

3. The Experiment: Testing the "Detectives"

They tested 12 different AI "detectives" (ranging from small, fast ones to huge, smart ones like Llama 3 and GPT-4) to see how good they were at spotting these specific evil plans.

• The Result: The big, smart AIs were okay at spotting the lies, but the smaller, specialized AIs were actually better at figuring out the specific type of evil plan when they were trained on this new map.

4. The Big Idea: "Vaccinating" the AI

This is the most creative part of the paper. The authors used a concept from psychology called Inoculation Theory.

• The Analogy: Think of a biological vaccine. You give a person a tiny, weakened version of a virus so their immune system learns to fight it. If they meet the real virus later, they are ready.
• The AI Version: The researchers "vaccinated" the AI against disinformation.
  1. The Threat: They told the AI, "Hey, this text might be trying to trick you with a specific evil plan."
  2. The Pre-emption: They gave the AI a "cheat sheet" explaining what those evil plans look like (the MALINT categories).
  3. The Result: The AI didn't just read the text; it analyzed the text through the lens of the evil plan. It's like a security guard who, instead of just checking your ID, asks, "Wait, why are you trying to sneak into the bank? Are you trying to rob it or just steal a pen?"

5. The Outcome: A Stronger Defense

When they used this "vaccinated" approach (which they call Intent-Based Inoculation):

• The AI got significantly better at spotting fake news, even on news it had never seen before.
• It worked in English and even in languages the AI wasn't super good at (like Estonian or Polish).
• It worked especially well on long articles, where the "evil plan" is usually hidden in the details.

Summary

The paper says: "Don't just teach AI to spot lies; teach it to spot the liar's motive."

By giving AI a "vaccine" that teaches it to recognize the intent behind the lies (like trying to break a country's trust or sell fake cures), the AI becomes much harder to fool. It's like upgrading a security system from just checking for broken windows to understanding the motive of the burglar.

Technical Summary

1. Problem Statement

The proliferation of online disinformation poses a severe threat to public discourse. While existing research and datasets (e.g., LIAR, FakeNewsNet) focus on detecting false content, they largely ignore the intentionality behind the creation of such content. Current definitions of disinformation emphasize that it is "designed, presented, and promoted to intentionally cause public harm or for profit." However, there is a lack of high-quality, human-annotated English datasets that explicitly categorize the malicious intents of disinformation agents. Furthermore, it remains unclear whether incorporating knowledge of these specific intents can improve the zero-shot disinformation detection capabilities of Large Language Models (LLMs).

2. Methodology

The paper proposes a two-pronged methodology: the creation of a novel dataset (MALINT) and the development of an Intent-Based Inoculation (IBI) framework.

A. The MALINT Dataset

• Source & Collection: The dataset consists of 1,600 news articles collected from ~50 sources (mainstream, alternative, and biased). Sources were pre-classified by fact-checking experts into Reliable, Unreliable, or Mixed/Biased.
• Annotation Process: A rigorous, multi-stage workflow was employed involving fact-checking experts from IFCN-accredited organizations.
  1. Credibility Assessment: Articles were labeled as Credible, Disinformation, or Hard-to-say.
  2. Malicious Intent Annotation: For articles deemed disinformative, annotators identified the underlying malicious intent. This was a multilabel task.
• Intent Taxonomy: Five specific categories of malicious intent were defined:
  1. UCPI: Undermining the Credibility of Public Institutions.
  2. CPV: Changing Political Views.
  3. UIOA: Undermining International Organizations and Alliances.
  4. PSSA: Promoting Social Stereotypes/Antagonisms.
  5. PASV: Promoting Anti-scientific Views.
• Quality Control: Independent annotation by a primary annotator and a supervisor, with consensus resolution and expert review for ambiguous cases. Inter-annotator agreement reached >95% after consensus.

B. Intent Classification Experiments

The authors benchmarked 12 Language Models (LMs) on the MALINT dataset:

• Small Language Models (SLMs): BERT, RoBERTa, DeBERTa V3, DistilBERT (fine-tuned).
• Large Language Models (LLMs): GPT-4o Mini, GPT-4.1 Mini, Gemini 2.0 Flash, Gemma 3 27B, Llama 3.3 70B (zero-shot).
• Tasks: Binary detection per class (is intent X present?) and Multilabel detection (identifying all present intents).

C. Intent-Based Inoculation (IBI) Framework

Inspired by Inoculation Theory (psychology), which suggests that exposing individuals to weakened forms of a threat builds resistance, the authors propose a similar approach for LLMs.

• Concept: "Vaccinate" the LLM against disinformation by explicitly analyzing the malicious intent before making a final judgment.
• Mechanism:
  1. Threat: The prompt warns the model that the input may contain hidden malicious intent.
  2. Refutational Preemption: The model first generates a structured analysis of the text's potential malicious intents (using the MALINT taxonomy).
  3. Detection: The model then uses this generated intent analysis as context to perform the final binary disinformation detection.
• Evaluation: Tested on MALINT and five other datasets (ISOT, CoAID, EUDisinfo, ECTF, EUvsDisinfo) across different genres (articles vs. social posts), temporal splits (pre/post-knowledge cutoff), and six languages (English, German, French, Polish, Estonian, Russian).

3. Key Contributions

1. MALINT Dataset: The first human-annotated English corpus that explicitly labels disinformation and categorizes the malicious intent of the agents behind it, featuring comprehensive stepwise annotations.
2. Benchmarking: The first systematic evaluation of intent classification capabilities across 12 different LMs (both SLMs and LLMs) in binary and multilabel settings.
3. Intent-Augmented Reasoning (IBI): A novel framework demonstrating that integrating intent analysis into the reasoning process significantly boosts zero-shot disinformation detection.
4. Cross-Lingual Generalization: Evidence that intent-based reasoning improves detection performance not only in English but also in low-resource languages (e.g., Estonian, Polish).

4. Key Results

Intent Classification Performance

• SLMs vs. LLMs: In multilabel detection, fine-tuned SLMs (specifically DeBERTa V3 Large and RoBERTa) outperformed LLMs, achieving a weighted F1 of 82.1%.
• Binary Detection: In binary detection per class, LLMs (e.g., GPT-4.1 Mini, Llama 3.3) outperformed fine-tuned SLMs on three of the five intent categories.
• Baseline: Simple logistic regression baselines performed poorly compared to both SLMs and LLMs, highlighting the complexity of intent recognition.

Disinformation Detection via IBI

• Performance Gain: The IBI approach consistently outperformed competitive baselines (VaN, Z-CoT, DeF-SpeC).
  • English: Average improvement of 9% in F1 score across five datasets and five LLMs.
  • Multilingual: Gains were even larger in other languages, with an average improvement of 20%. The highest gain was observed in Estonian (+25%).
• Robustness: IBI improved performance across:
  • Genres: Articles (long-form) saw the greatest gains, likely due to more context for intent reasoning.
  • Temporal Splits: IBI improved detection on texts published after the models' knowledge cutoffs, proving its utility on unseen data.
  • Statistical Significance: McNemar's tests confirmed that IBI significantly outperformed baselines in ~90% of evaluation scenarios (p < 0.01).
• Impact of Accuracy: While correct intent predictions generally boosted detection F1 scores, the system remained robust even when intent predictions were imperfect, suggesting that the process of reasoning about intent is beneficial even if the specific label is slightly off.

5. Significance and Implications

• Shift in Paradigm: The paper argues that detecting disinformation requires understanding the why (intent) as much as the what (falsehood). By explicitly modeling intent, detection systems become more robust.
• Inoculation Theory in NLP: This work successfully translates a psychological theory (inoculation) into a prompt engineering strategy, showing that "pre-bunking" via intent analysis strengthens LLM defenses against manipulation.
• Resource Availability: The release of MALINT, along with prompts and code, fills a critical gap in English-language disinformation research, providing a benchmark for intent-aware detection.
• Global Applicability: The success in low-resource languages suggests that intent-based reasoning is a transferable skill that does not rely solely on massive pre-training data in the target language, making it valuable for global disinformation monitoring.

In conclusion, the authors demonstrate that intent-aware reasoning is a powerful mechanism for enhancing LLMs' ability to detect disinformation, offering a significant improvement over standard zero-shot prompting methods.