💬 NLP

When Hate Meets Facts: LLMs-in-the-Loop for Check-worthiness Detection in Hate Speech

This paper introduces WSF-ARG+, the first dataset combining hate speech with check-worthiness information, and proposes an LLM-in-the-loop framework that reduces human annotation effort while demonstrating that incorporating check-worthiness labels significantly improves hate speech detection performance.

Original authors: Nicolás Benjamín Ocampo, Tommaso Caselli, Davide Ceolin

Published 2026-03-27

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Nicolás Benjamín Ocampo, Tommaso Caselli, Davide Ceolin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the internet as a giant, chaotic town square. In this square, there are two main problems: people shouting hateful slurs (Hate Speech) and people spreading lies or half-truths (Misinformation).

For a long time, researchers treated these as two separate problems. They built tools to catch the shouters and separate tools to catch the liars. But this paper argues that in the real world, these two problems often wear the same mask. Hate speech is increasingly being disguised as "facts" or "news."

Here is a simple breakdown of what the authors did, using some everyday analogies.

1. The Problem: The "Wolf in Sheep's Clothing"

Imagine a person trying to start a riot. Instead of just screaming insults (which is easy to spot), they stand on a soapbox and say, "Did you know that 90% of [Group X] are actually aliens?"

This is a fact-like claim. It sounds like a news report, but it's a lie designed to make people hate a specific group.

The old way: Moderators might miss this because it doesn't look like a standard insult.
The new way: If we can spot that this sentence is a "claim" that needs fact-checking, we can catch the hate speech hiding inside it.

The authors call this intersection "Check-worthiness." It asks: "Is this statement a claim that matters enough to be fact-checked?"

2. The Solution: The "Human-LLM Team"

To study this, the researchers needed a massive dataset of these tricky messages. But labeling them is hard work. It's like trying to grade 1,000 essays where the answers are subjective. You need experts, and experts are expensive and tired.

So, they invented a new workflow called "LLM-in-the-Loop."

Think of this like a construction site:

The LLM (Large Language Model) is the Power Drill. It's fast, strong, and can do the heavy lifting. It scans thousands of messages and says, "This looks like a claim!" or "This is just an opinion!"
The Human is the Foreman. They don't do every single screw. Instead, they watch the Power Drill.
- If the Drill and the Foreman agree, they move on.
- If the Drill gets confused or disagrees with the Foreman, the Foreman steps in to make the final call.

The Result: They used 12 different "Power Drills" (AI models) to help them build a new dataset called WSF-ARG+. This dataset is special because it contains both hateful and non-hateful messages, and every single "fact" inside them has been labeled as either "Check-worthy" or "Not Check-worthy."

3. The Discovery: Lies Make Hate Worse

Once they had their new dataset, they ran some experiments and found two surprising things:

A. The "Poisoned Chalice" Effect
They found that hate speech containing "check-worthy" claims (lies presented as facts) is more harmful than simple insults.

Analogy: A simple insult is like a paper cut; it stings but heals fast. A lie wrapped in hate is like a poisoned dart; it spreads fear and causes deeper, longer-lasting damage to the community. The data showed these messages were rated significantly higher for "harassment" and "hate."

B. The "Super-Helper" Effect
They tested if giving AI models a "cheat sheet" (the check-worthiness labels) helped them detect hate speech better.

Analogy: Imagine trying to find a needle in a haystack. If you tell the AI, "The needle is wrapped in red tape," it finds it much faster.
When they gave the AI the "check-worthiness" labels, the AI's ability to spot hate speech improved significantly, especially for the larger, smarter models. It went from being a decent detective to a master detective.

4. Why This Matters

This paper is a blueprint for the future of internet safety.

We have a new map: They released the WSF-ARG+ dataset, which is the first map of where hate speech and fake facts overlap.
We have a better tool: They proved you don't need to hire 100 humans to label data. You can use a smart AI as a "junior assistant" and have a human "manager" fix the mistakes. This saves time and money without losing quality.
We understand the enemy better: We now know that when hate speech tries to sound like "science" or "news," it becomes more dangerous. By focusing on the facts inside the hate, we can dismantle the argument more effectively than just blocking the insult.

In short: The authors built a smarter way to catch the liars who are also the haters, using a team of humans and AI working together, and they proved that catching these specific types of lies makes the internet a safer place.

1. Problem Statement

The paper addresses the intersection of Hate Speech (HS) and Misinformation. Online hate is increasingly expressed not just through slurs, but through "fact-like" claims that may be false or misleading (e.g., false statistics about specific ethnic groups).

The Challenge: Current content moderation systems often treat hate speech and misinformation as separate phenomena. However, when hate speech relies on verifiable claims, moderators must assess both harmfulness and veracity, significantly increasing the cognitive load and effort required for annotation.
The Gap: Existing datasets for hate speech typically contain short, explicit slurs without complex arguments, while misinformation datasets lack hate speech annotations. There is a lack of resources to study how "check-worthy" claims (claims that warrant fact-checking) function within hateful narratives.

2. Methodology

A. The LLM-in-the-Loop Framework

To address the high cost of expert annotation for subjective tasks like check-worthiness, the authors propose a novel LLM-in-the-loop framework. Unlike "Human-in-the-loop" (where humans verify automated outputs), this framework integrates the LLM as an auxiliary annotator alongside a human expert.

Workflow:
1. Parallel Annotation: A human expert and an LLM annotate the same dataset instance.
2. Consensus Check: The LLM generates three independent predictions per instance (at temperature 1.0) to establish a majority vote.
3. Conflict Resolution:
  - If the LLM's majority vote matches the human label, the label is accepted.
  - If they disagree, or if the LLM's three predictions are inconsistent, a Judge Annotator (a second expert) reviews the case.
  - The Judge is "blind" to the source of the labels (human vs. LLM) to prevent anchor bias and makes the final decision.
Mitigation Strategies: The framework employs specific strategies to reduce LLM errors, including majority voting, blind adjudication, and comparing zero-shot vs. one-shot prompting.

B. Dataset Construction: WSF-ARG+

The authors introduce WSF-ARG+, the first dataset combining hate speech with check-worthiness annotations.

Source: Based on the White Supremacy Forum Argumentative (WSF-ARG) dataset, which contains 227 hateful messages from the Stormfront forum.
Expansion: The dataset was extended with 294 non-hateful messages (long, argumentative posts) to allow for comparative analysis.
Granularity: Claims are extracted from messages. Each claim is annotated for Check-worthiness into three classes:
1. CFS (Checkworthy Factual Statement): Important, verifiable claims with public impact.
2. UFS (Unimportant Factual Statement): Verifiable but low-interest claims.
3. NFS (Non-Factual Statement): Opinions, beliefs, or rhetorical questions.

C. Experimental Setup

Models: 12 open-weight LLMs across four families (Mistral, Llama3, Qwen2.5, OLMo2, Command-r) ranging from 7B to 104B parameters.
Evaluation: The framework was validated against a "Platinum" standard (full human annotation by three experts) to measure agreement (Cohen's $\kappa$ ) and Inter-Annotator Agreement (IAA).

3. Key Contributions

WSF-ARG+ Dataset: A new resource containing 521 messages (227 HS, 294 Non-HS) with claim-level check-worthiness annotations (CFS, UFS, NFS). It includes both "Gold" labels (LLM-in-the-loop) and "Platinum" labels (Full Human).
LLM-in-the-Loop Framework: A validated methodology that reduces human effort by 46.69% (claims requiring a judge) while maintaining annotation quality comparable to full human annotation.
Empirical Insights:
- Harm Correlation: Hate speech messages containing check-worthy claims are significantly more harassing and hateful than those without.
- Detection Improvement: Incorporating check-worthiness labels as input features improves Hate Speech detection performance in Large Language Models (LLMs).

4. Results

Annotation Quality & Efficiency

Human Effort Reduction: The framework reduced the number of claims requiring human adjudication from 100% to 46.69%.
Agreement: The best-performing model (OLMo2-32B in zero-shot) achieved a Cohen's $\kappa$ of 0.483 (binary) against the first human annotator, comparable to human-human agreement in similar tasks.
Validation: When comparing the LLM-in-the-loop "Gold" labels against the "Platinum" (Full Human) labels, the framework achieved a Cohen's $\kappa$ of 0.800 (3-class) and 0.815 (binary), demonstrating high fidelity.

Impact on Hate Speech Detection

The authors tested if providing check-worthiness labels to LLMs improves their ability to detect hate speech.

Performance Gains: Large models (≥70B parameters) showed significant improvements in Macro-F1 scores when check-worthiness labels were included in the prompt.
- Average Improvement: Large models improved by 0.154 Macro-F1 on average.
- Specific Example: The Qwen2.5-72B model improved from an F1 of 0.701 (without labels) to 0.766 (with labels).
Statistical Significance: Messages with check-worthy claims had significantly higher harassment and hate scores (OpenAI Moderation API) compared to those without ( $p < 0.05$ ).

5. Significance and Future Work

Theoretical Impact: The study confirms that misinformation is a potent vehicle for hate speech. By identifying and fact-checking specific claims within hateful narratives, platforms can more effectively disrupt the "misinformation-hate nexus."
Practical Application: The LLM-in-the-loop framework offers a scalable, cost-effective solution for annotating complex, subjective datasets, making high-quality training data more accessible.
Counter-Speech: The dataset enables the development of targeted counter-speech that debunks specific false premises rather than just flagging general toxicity.
Limitations: The framework was tested primarily on white supremacist content; generalizability to other domains and languages requires further study. Additionally, the authors advocate for counter-speech generation over censorship to mitigate risks of speech policing.

Conclusion: The paper demonstrates that hate speech often relies on "fact-like" claims to legitimize prejudice. By integrating LLMs into the annotation loop, researchers can efficiently create high-quality datasets that link hate speech with misinformation, ultimately leading to more robust detection systems and targeted interventions.