💻 computer science

Assessing How Hate, Counterspeech, and Toxicity Affect Hate Group Newcomers

This study analyzes 16,513 newcomers across 104 hate subreddits and finds that while counterspeech effectively reduces the likelihood of new users continuing to post in hate communities, the presence of toxic counterspeech increases the probability of continued hostility from hate users within the same discussion.

Original authors: Daniel Hickey, Matheus Schmitz, Daniel M. T. Fessler, Paul E. Smaldino, Kristina Lerman, Goran Murić, Keith Burghardt

Published 2026-04-13

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Daniel Hickey, Matheus Schmitz, Daniel M. T. Fessler, Paul E. Smaldino, Kristina Lerman, Goran Murić, Keith Burghardt

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the internet as a giant, chaotic town square. In this square, there are specific, walled-off clubs where people gather to shout hateful things about others. Let's call these "Hate Clubs."

This paper is like a group of social scientists acting as detectives, trying to figure out what happens when a newbie (someone who just joined the club for the first time) starts shouting hate, and someone from the outside tries to talk back to them.

Here is the breakdown of their investigation, using some simple analogies:

1. The Big Question: Does "Talking Back" Work?

For a long time, people have thought that the best way to stop hate is to use Counterspeech. This is like shouting, "Hey, that's not true!" or "That's mean!" instead of just ignoring the hater or banning them.

The researchers wanted to know: Does shouting back actually make the new hater leave the club, or does it just make them angrier and stick around?

2. The Surprise: The "Toxic" Shout-Back

The researchers looked at thousands of new people joining 104 different hate clubs on Reddit. They noticed something weird about the people shouting back (the counterspeakers).

The Analogy: Imagine a bully in a schoolyard. A bystander steps in to stop the bully. You'd expect the bystander to be calm and reasonable.
The Reality: The study found that the bystanders were often just as rude and aggressive as the bully.
The Data: While the hate speech was the most toxic, the "fighting back" speech was almost twice as toxic as normal conversation. It was like the person trying to stop the fire was throwing gasoline on it, even if they were trying to put it out.

3. The Main Discovery: Newbies Actually Leave

Here is the most surprising part. The researchers thought that if you shout back aggressively at a new hater, they would get defensive, feel attacked, and dig their heels in (a concept called "backfire").

But that didn't happen.

The Finding: When a new hater got a reply (even a rude one), they were less likely to come back to the hate club.
The Analogy: Think of the new hater as a kid who just joined a gang to look tough. If a stranger yells at them, the kid doesn't get mad and stay to prove a point. Instead, the kid thinks, "Wow, this place is too much drama. I'm not cool enough here," and they walk away.
The Conclusion: These newcomers might not be deep-down, committed evil geniuses. They might just be "toying" with hate ideas. A little push (even a rude one) is enough to make them lose interest and leave.

4. The Catch: The "Toxic" Side Effect

So, if rude counterspeech works to make newbies leave, should we all just be rude? No.

The study found a nasty side effect:

The Chain Reaction: When a counterspeaker was toxic, the other members of the hate club got even more toxic in response.
The Analogy: It's like a rowdy sports game. If one fan yells a curse word at the other team, the whole stadium starts screaming and throwing things. The "fighting back" didn't just stop the new guy; it made the whole room more hostile.

5. The Takeaway for the Real World

The researchers are basically saying:

Newbies are fragile: People just starting to dabble in hate are easy to scare off. They aren't usually deeply committed yet.
Rudeness works (sort of): Even mean replies can make newbies quit the group.
But it's dangerous: Using mean replies makes the whole environment worse for everyone else, turning the discussion into a screaming match.

The Final Advice:
If you see a newbie posting hate, you don't need to be a saint, but you also shouldn't be a jerk. The goal is to stop them from joining the club without turning the whole club into a war zone. The best strategy is to realize that these new people are often just testing the waters, and a firm "No" (even if it's a bit blunt) is often enough to make them swim away, but we should try to keep the water from boiling over for everyone else.

In short: Shouting back at new haters makes them leave, but if you shout too meanly, you ruin the party for everyone else.

1. Problem Statement

Online hate speech is pervasive and contagious, often leading to offline hate crimes and the radicalization of individuals. While content moderation (bans, quarantines) is a common strategy, it faces limitations such as user migration to "alt-tech" platforms and accusations of restricting free speech. Consequently, counterspeech (speech opposing hate) has emerged as an alternative intervention.

However, critical gaps remain in the literature:

Context: Most studies focus on mainstream platforms (e.g., X/Twitter) rather than dedicated hate communities.
Outcome: Prior research focuses on the reduction of hate speech production rather than downstream behavioral outcomes like user retention (whether users stay in the community).
Toxicity: The role of toxicity in counterspeech is poorly understood. It is unclear if hostile counterspeech (e.g., aggressive replies) effectively dissuades hate adherents or if it hardens their positions and escalates conflict ("flame wars").
Target Audience: It is unknown if counterspeech affects newcomers (who may be tentatively exploring hate ideologies) differently than long-term adherents.

2. Methodology

The authors conducted a large-scale observational study using causal inference methods on Reddit data.

Data Collection

Dataset: 168 hate subreddits (forums) sourced from Pushshift.
Filtering: After filtering for subreddits with sufficient hate/counterspeech pairs, the final dataset included 104 hate subreddits and 16,513 matched triplets of newcomers.
Control Group: 104 matched non-hate subreddits were used for comparison.
Subjects: Newcomers defined as users posting hate speech for the first time in a specific subreddit.

Counterspeech Detection (LLM-Based)

To overcome the limitations of existing supervised models trained on specific domains, the authors developed a novel LLM-based detection approach:

Distillation: Used GPT-4o to label 5,500 comment pairs (newcomer + reply) as Counterspeech, Hate Speech, or Other.
Fine-tuning: Fine-tuned a RoBERTa model on the GPT-4o labeled data.
Direct Inference: Used Llama 3.3 70B (open-weight) to label the entire corpus. To ensure high precision (minimizing false positives), they employed a conservative thresholding strategy and used temperature sampling (prompting the model 5 times per comment) to gauge confidence.
Validation: Manually annotated 336 pairs for validation (Fleiss Kappa = 0.52) and tested precision at different confidence thresholds. The LLaMa 3.3 70B approach achieved a mean precision of 0.85, outperforming traditional RoBERTa models trained on prior datasets (which had precisions of 0.26–0.34).

Causal Matching & Experimental Design

Treatment Groups: Newcomers were categorized into three groups based on their first post's reply:
1. No reply.
2. Reply containing Counterspeech.
3. Reply containing Other speech (including hate speech from other users).
Matching: Used categorical matching (Nattino et al. 2021) to create triplets of users across the three groups.
Covariates: Matched on 13 features including semantic similarity (MPNet embeddings), comment nest level, sentiment (VADER), upvote scores, timestamp, thread activity, and user history.
Subreddit Matching: Matched hate subreddits with non-hate subreddits using Mahalanobis distance, controlling for ban dates and activity rates to simulate a fair comparison.

Statistical Analysis

Model: Mixed-effect logistic regression with subreddits as random effects.
Outcome Variable: Binary indicator of User Retention (did the user post/comment in a different thread within the same subreddit after the initial interaction?).
Toxicity Measurement: Used the Perspective API to score toxicity (probability of a comment being rude/disrespectful).
Secondary Analysis: Measured the probability of receiving a toxic follow-up reply based on the toxicity of the initial counterspeech.

3. Key Contributions

Novel Detection Method: Developed an LLM-based counterspeech detection pipeline that significantly outperforms specialized models trained on existing datasets, demonstrating the superiority of LLMs for generalization in computational social science.
Focus on Newcomers: Shifted the focus from established hate adherents to newcomers, identifying them as a critical intervention point.
Toxicity Analysis: Systematically evaluated the impact of toxic vs. non-toxic counterspeech on user retention and subsequent discussion toxicity.
Causal Evidence: Provided causal estimates on how counterspeech affects user retention in dedicated hate communities, a context previously under-studied.

4. Key Results

RQ1: Toxicity Levels

Counterspeech is Highly Toxic: While counterspeech is less toxic than hate speech, it is 88% more toxic than non-hate/non-counterspeech replies within hate subreddits and 330% more toxic than replies in non-hate subreddits.
Robustness: This high toxicity persists even after removing quoted slurs, indicating it is not merely an artifact of quoting hate speech.

RQ2: Impact on User Retention

Counterspeech Dissuades Newcomers: Receiving counterspeech significantly reduces the likelihood of a newcomer continuing to engage in the hate subreddit (Average Treatment Effect = -2.0%, $p=0.001$ ).
Hate Speech Encourages Engagement: Conversely, receiving hate speech replies increases engagement (ATE = +3.1%, $p=0.001$ ).
Role of Toxicity in Retention:
- The toxicity level of the counterspeech did not significantly moderate its effect on user retention. Both toxic and non-toxic counterspeech reduced engagement.
- However, when isolating toxic counterspeech, the negative effect on retention became statistically insignificant (likely due to smaller sample size), whereas non-toxic counterspeech remained highly effective at reducing engagement.
Escalation Effect: While toxicity didn't change retention rates, toxic counterspeech significantly increased the probability of receiving a toxic follow-up reply from hate users, fueling further hostility in the discussion thread.

5. Significance and Implications

Theoretical Implications

Newcomer Susceptibility: The findings suggest that newcomers to hate groups are not yet deeply committed to the ideology. They may be "toying" with hateful beliefs or engaging in low-commitment exploration. Counterspeech effectively interrupts this early-stage socialization.
Toxicity Paradox: While toxic counterspeech fails to deter users more than non-toxic counterspeech, it actively degrades the discussion environment by triggering more toxicity from hate adherents.

Practical Implications

Platform Design: Platforms should not rely solely on moderation. They should develop tools to assist counterspeakers in composing constructive, non-toxic responses (e.g., pre-reply prompts, reframing suggestions) to maximize the deterrent effect while minimizing escalation.
Intervention Strategy: Interventions should target newcomers specifically, as they are the most responsive to opposing views.
Policy: Moderation and counterspeech are complementary. Moderation handles clear rule violations, while well-designed counterspeech tools can reduce the "social reinforcement" of hate communities by discouraging new participation.

Limitations

Belief Change vs. Behavior: The study measures behavioral disengagement (stopping posts), not necessarily a change in internal beliefs. Users may simply migrate to other platforms.
Model Bias: The Perspective API may misclassify certain dialects or contexts as toxic.
Scope: The study is limited to Reddit; results may vary on platforms with different architectures (e.g., TikTok, alt-tech).

Conclusion: The study demonstrates that counterspeech is an effective tool for reducing participation in hate communities, particularly among newcomers. However, the high prevalence of toxicity in counterspeech poses a risk of escalating conflict. Future efforts should focus on empowering users to deliver non-toxic counterspeech to maximize deterrence without fueling further hostility.