From Birdwatch to Community Notes, from Twitter to X: four years of community-based content moderation
This paper presents a comprehensive four-year descriptive analysis of X's Community Notes (formerly Birdwatch) program, examining its linguistic diversity, sourcing practices, contributor activity, and interaction networks while releasing a curated dataset and code to advance research on community-based content moderation.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine social media as a massive, chaotic town square where millions of people shout out their thoughts every second. Sometimes, people shout lies, rumors, or confusing stories. In the past, a small group of "town guards" (experts) or a "robot scanner" (AI) tried to listen to everyone and flag the bad stuff. But there are too many people shouting for the guards to keep up, and the robot scanner often makes mistakes because it was trained on biased data.
To solve this, X (formerly Twitter) launched a new idea called Community Notes (originally "Birdwatch"). Think of this as a neighborhood watch program where regular citizens are invited to help police the square.
Here is what this paper tells us about how that neighborhood watch has worked over its first four years (2021–2025):
1. The Setup: How the Watch Works
In this system, anyone who has been a good citizen for six months can become a "Contributor."
- The Job: If you see a post that seems misleading, you can write a "Note" to add missing context or facts.
- The Vote: Other Contributors read your Note and vote on whether it is "Helpful," "Somewhat Helpful," or "Not Helpful."
- The Rule: A Note only gets shown to the public if people from different political sides agree it is helpful. This is like a bridge; if only people from one side of the river agree the bridge is safe, it doesn't get built. The goal is to find truth that everyone can agree on.
2. Who is Doing the Work? (The "Super-Contributors")
The paper found that the work isn't shared equally. It's like a neighborhood where a few dedicated volunteers do almost all the sweeping and trash collection.
- The Heavy Lifters: A tiny number of people write the vast majority of the Notes. One person (who seems to be a bot) wrote over 33,000 notes, mostly about crypto scams.
- The Rating Gap: Similarly, a small group of people does almost all the voting.
- The Result: Because so few people are doing the heavy lifting, many posts that need a Note never get one, or the Notes they get never get enough votes to be shown.
3. What Are They Talking About? (The Topics)
The "town square" is noisy, but the volunteers focus on specific types of noise:
- Politics is King: The biggest chunk of work (about 30–45%) is about politics and government.
- Health: Early on, there was a huge wave of notes about vaccines and the pandemic.
- War & Crypto: As time went on, more notes appeared about wars (like Ukraine and Gaza) and cryptocurrency scams.
- The "NNN" Phenomenon: Sometimes, volunteers write a note saying "Note Not Needed" (NNN). This is like a neighbor saying, "Hey, that story isn't actually a lie, stop trying to fact-check it!" The paper notes that this has turned into a way for people to debate whether something needs correcting, not just what is wrong.
4. The Language Barrier
Even though the system is global, the "neighborhoods" are mostly separate.
- One Language, One Group: Most people only write in one language. Even if someone speaks five languages, they usually stick to just one when writing notes.
- English Dominance: English is the main language of the town square. The researchers focused their deep analysis on English notes because that's where most of the activity happens.
5. The Speed Problem: The "Too Little, Too Late" Issue
This is the most critical finding. The system is slow.
- The Wait: On average, it takes 26 hours for a Note to go from being written to being shown to the public.
- The Missed Window: By the time a "Helpful" Note finally appears, the lie has usually already traveled around the world. The paper notes that by the time a note is published, the post has already reached 80% of its audience.
- The Success Rate: Out of every 100 posts that get a Note proposed, only about 13 actually get a "Helpful" Note shown to the public. Most Notes get stuck in a waiting room, never getting enough diverse votes to pass.
6. Where Do They Get Their Facts?
When writing a Note, contributors are encouraged to bring proof.
- The Source: Most Notes include a link to a source.
- The Favorites: The most popular sources are Wikipedia, YouTube, and X itself.
- The Bias Check: The researchers checked the political bias of the websites used. Surprisingly, most of the top sources are considered "neutral," though there is a slight lean toward the left. However, the paper found one suspicious case where one user kept linking to a specific antivirus site for crypto scams, suggesting that some "volunteers" might be automated bots pushing specific agendas.
The Bottom Line
Community Notes is a brave experiment in letting the crowd police itself. It has successfully built a massive library of context and facts. However, the paper concludes that the system is bottlenecked.
- It relies too heavily on a few super-volunteers.
- It is too slow to stop lies before they spread.
- It struggles to get enough diverse people to agree on a note before the news cycle moves on.
The authors have released all their data and code (like handing over the blueprints of the neighborhood watch) so other researchers can study how to make this system faster and fairer in the future.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.