💬 NLP

Truth with a Twist: The Rhetoric of Persuasion in Professional vs. Community-Authored Fact-Checks

This study compares the rhetorical techniques used in community-authored versus professional fact-checks, finding that community-produced notes do not rely more heavily on persuasion than professional ones and that while certain persuasive elements can increase helpfulness ratings, crowd raters are capable of penalizing problematic language.

Original authors: Olesya Razuvayevskaya, Kalina Bontcheva

Published 2026-02-11

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Olesya Razuvayevskaya, Kalina Bontcheva

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Verdict: Are "Crowd-Sourced" Truth-Tellers More Manipulative?

Imagine there are two types of referees in a high-stakes soccer match.

The first type is the "Professional Referee"—they went to specialized school, follow a strict rulebook, and work for a major league. They are expected to be calm, clinical, and strictly objective.

The second type is the "Community Referee"—they are fans from the stands who have been given a whistle and the power to step onto the field to correct a bad call. Because they aren't "professionals," people often assume they might be more emotional, biased, or prone to using "tricks" to get their way.

This research paper set out to see if that assumption is true. The researchers compared Community Notes on X (formerly Twitter)—the "Community Referees"—against professional fact-checkers (the "Professional Referees") to see who uses more "persuasion tricks" (rhetoric) to win an argument.

The Big Reveal: The "Fair Play" Surprise

Most people expected the Community Notes to be full of "spin"—using emotional language or logical shortcuts to force people to believe them.

But the data showed something different: The community-written notes weren't actually more "persuasive" or "manipulative" than the professionals. In fact, on average, they used about the same amount of rhetorical tricks. The "fans in the stands" were playing much more by the rules than everyone thought.

The Nuance: Different Tools for Different Jobs

While the amount of persuasion was similar, the flavor of the persuasion changed depending on who was writing.

The Specialized Experts (EUvsDisinfo): These are like surgeons. They focus on a very specific, intense area (geopolitical disinformation). Because their job is so narrow and strict, they use very little "fluff."
The Global Crowd (Community Notes): These are like general practitioners. They cover everything from celebrity gossip to local politics. Because they cover so much ground, they occasionally use different rhetorical tools than the specialists.

The "Bulls**t Detector": How the Crowd Reacts

The researchers also looked at how the community "polices" itself. If a Community Note uses a "trick," do people call it out?

Think of it like a social immune system. The researchers found that the crowd has a built-in "bulls**t detector" for specific types of bad behavior:

The "My Team is Better" Trick (Flag Waving): If a note uses overly patriotic or "us vs. them" language, the crowd tends to rate it as less helpful.
The "Guilt by Association" Trick: If a note tries to make someone look bad just because of who they hang out with, the crowd penalizes it.
The "Straw Man" Trick: If a note misrepresents what someone actually said just to make them look silly, the crowd catches on and lowers its rating.

However, there is a funny quirk: some "tricks"—like simplifying a complex issue to make it easier to understand—actually make people rate the note as more helpful. It turns out, humans sometimes prefer a "simple truth" over a "complicated reality."

The Bottom Line

The study suggests that crowdsourced fact-checking isn't the "wild west" of misinformation we might fear. While it isn't perfect, the community has a surprisingly strong ability to self-regulate. They aren't just shouting opinions; they are participating in a massive, global attempt to keep the facts straight, and they are actually quite good at spotting when someone is trying to "twist" the truth.

Technical Summary: Truth with a Twist

Title: Truth with a Twist: The Rhetoric of Persuasion in Professional vs. Community-Authored Fact-Checks
Authors: Olesya Razuvayevskaya and Kalina Bontcheva (The University of Sheffield)
Venue: Proceedings of the ACM Web Conference 2026 (WWW ’26)

1. Problem Statement

There is an ongoing debate regarding the objectivity of crowdsourced content moderation versus professional fact-checking. Specifically, researchers have hypothesized that community-driven efforts—such as X’s Community Notes (CNs)—might be more susceptible to bias and more reliant on persuasive, manipulative, or emotionally charged linguistic strategies (e.g., appeals to authority or insults) compared to professional organizations. This study seeks to quantify whether community-authored debunks actually employ more persuasive language than professional ones and whether the community's self-governing mechanism is capable of penalizing such rhetoric.

2. Methodology

The study employs a large-scale comparative analysis using three distinct datasets:

Community Notes (CN): A crowd-authored dataset (approx. 2.01 million debunks).
EUvsDisinfo: A professional dataset focusing on pro-Kremlin disinformation (8,522 English debunks).
Database of Known Fakes (DBKF): A diverse, multilingual professional dataset (681,346 debunks).

Technical Approach:

Persuasion Detection: The authors used a state-of-the-art multilingual transformer-based classifier (Wu et al.) to detect various persuasion techniques, applying a high confidence threshold of 0.8.
RQ1 (Self-Governance): To test if the crowd penalizes persuasion, the authors correlated the number of persuasion techniques in a note with its "helpfulness" rating (mapped to a 1–3 scale). They used Spearman correlation for overall trends and the Mann–Whitney U test with Cliff’s delta ( $\delta$ ) to measure the effect size of individual techniques on helpfulness.
RQ2 (Comparative Rhetoric): To compare the prevalence of techniques across datasets, the authors used the Mann–Whitney U test for average counts and one-sided proportions z-tests to compare the presence of specific techniques between CNs and the professional datasets.

3. Key Contributions

First Large-Scale Comparison: It provides the first systematic, quantitative comparison of rhetorical strategies between crowdsourced and professional fact-checking ecosystems.
Nuanced Understanding of Rhetoric: It moves beyond the binary "objective vs. subjective" debate to show how different institutional norms (e.g., geopolitical niche vs. broad topical coverage) shape language.
Evaluation of Crowdsourced Moderation: It offers empirical evidence on how community raters react to specific linguistic "twists" in content moderation.

4. Results

RQ1 (Crowd Reaction):
- Contrary to the hypothesis, there is a slight positive correlation ( $\rho \approx 0.039$ ) between the number of persuasion techniques and helpfulness ratings.
- However, the crowd is selective: certain techniques are penalized (associated with lower helpfulness), specifically Flag Waving ( $\delta = -0.142$ ), Guilt by Association ( $\delta = -0.088$ ), and Straw Man ( $\delta = -0.068$ ).
- Other techniques, such as Consequential Oversimplification ( $\delta = 0.255$ ), actually correlate with higher helpfulness.
RQ2 (Professional vs. Community):
- The study found no evidence that CNs contain a higher average number of persuasion techniques than professional fact-checks.
- The comparison revealed context-dependency: CNs were less persuasive than the DBKF in most dimensions but showed higher prevalence of certain techniques (e.g., Appeal to Hypocrisy, Red Herring, Whataboutism) when compared to the highly specialized EUvsDisinfo dataset.

5. Significance

The findings challenge the assumption that crowdsourced moderation is inherently more "biased" or "manipulative" in its linguistic style than professional moderation. The study demonstrates that while community-driven notes are not more persuasive on average, the community's self-governing mechanism is effective at identifying and devaluing specific problematic rhetorical tactics (like logical fallacies and identity-based attacks). This suggests that the "bridging" and rating mechanisms in CNs act as a functional check against the most egregious forms of manipulative language, even if they do not eliminate persuasion entirely.