💻 computer science

Real-Time Toxicity Filtering for Open-Source Code Reviews

This paper introduces ToxiShield, a real-time browser extension that combines a high-accuracy BERT-based toxicity detector, an LLM-based multiclass classifier, and a fine-tuned Llama 3.2 model to automatically identify and detoxify toxic code reviews, thereby fostering a more inclusive open-source development environment.

Original authors: Md Awsaf Alam Anindya, Showvik Biswas, Anindya Iqbal, Jaydeb Sarker, Amiangshu Bosu

Published 2026-04-13

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Md Awsaf Alam Anindya, Showvik Biswas, Anindya Iqbal, Jaydeb Sarker, Amiangshu Bosu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are part of a massive, global team building a giant Lego castle. Everyone is welcome to add a brick, but sometimes, people get frustrated and start yelling, insulting each other, or throwing bricks at the wall. This is what happens in Open Source Software (like GitHub), where developers collaborate to build code.

The paper you shared introduces a new tool called ToxiShield. Think of it as a super-smart, real-time "Communication Coach" and "Editor" that lives right inside your web browser. Its job is to stop the yelling before it happens and help people say what they mean without being mean.

Here is how it works, broken down into three simple steps:

1. The Security Guard (Toxicity Filter)

Imagine a security guard standing at the door of a party. Before you can say anything, the guard checks your words.

What it does: It scans every comment a developer writes to see if it's "toxic" (rude, aggressive, or insulting).
How it works: The researchers taught a computer brain (an AI model) by showing it over 38,000 real examples of code reviews. It learned to spot the difference between a helpful critique ("This code is buggy") and a toxic insult ("This code is garbage and you are stupid").
The Result: It's incredibly good at this, catching 97% of the bad stuff without raising false alarms.

2. The Detective (Communication Coach)

If the Security Guard flags a comment as toxic, the comment doesn't just get deleted. Instead, it gets passed to a Detective.

What it does: The Detective asks, "Okay, why is this toxic?" Is it because of a swear word? Is it mocking someone? Is it just being rude?
How it works: The AI looks at the comment and categorizes the specific type of rudeness. It then explains to the writer, "Hey, you used a word that sounds like an insult," or "You are being sarcastic, which might hurt feelings."
The Result: It helps the writer understand what they did wrong, not just that they did something wrong.

3. The Diplomat (The Reframer)

This is the most magical part. Once the Detective explains the problem, a Diplomat steps in to rewrite the message.

What it does: It takes the angry, toxic comment and rephrases it into a polite, professional version that keeps the exact same technical meaning.
The Analogy: Imagine someone says, "This code is disgusting and you're an idiot for writing it." The Diplomat instantly rewrites it to: "This code has some serious issues that need fixing, and the current approach seems ineffective. Let's discuss a better way."
The Result: The technical feedback remains 100% the same, but the tone is now friendly and constructive. The AI is so good at this that it keeps the meaning intact 67% of the time while sounding 97% fluent and natural.

Why Does This Matter?

The researchers tested this tool with 10 professional software developers. They found that the tool was easy to use and made people feel happier about collaborating.

The Big Picture:
Open source projects are like giant community gardens. If people start yelling and throwing rocks, the garden dies. ToxiShield is like a friendly gardener who gently catches the rocks before they hit the flowers, explains why throwing them is bad, and hands the person a nice watering can instead. It helps keep the community inclusive and productive, ensuring that the best ideas win, not the loudest voices.

In short: ToxiShield is a browser extension that acts as a real-time filter, a teacher, and a translator, turning toxic code reviews into helpful conversations.

1. Problem Statement

Open-source software (OSS) development relies heavily on collaboration via platforms like GitHub. However, this collaboration often devolves into toxic interactions (e.g., sarcasm, insults, passive-aggressiveness), which hinder productivity, create barriers for newcomers, and degrade software quality.

Limitations of Current Solutions: Existing approaches primarily rely on post hoc text classification, intervening only after toxic comments have already caused harm. They often lack explainability and real-time integration into the developer workflow.
Goal: The authors propose ToxiShield, a real-time browser extension designed to proactively detect, explain, and detoxify toxic code reviews before they are posted, fostering a more inclusive environment.

2. Methodology: The ToxiShield Framework

ToxiShield operates through a three-stage pipeline (Figure 1 in the paper), integrating specialized modules for detection, classification, and remediation.

Module 1: Toxicity Filter (Binary Classification)

Objective: Determine if a code review comment is toxic or non-toxic.
Data: A curated dataset of 38,761 pull-request comments (10,120 toxic, 28,641 non-toxic) derived from filtering 15 million comments from a larger corpus of 101 million OSS comments.
Models:
- Fine-tuned BERT-base-uncased and Xtreme-DistilBERT.
- Zero-shot prompting with GPT-4o (used as a baseline).
Technique: 8:1:1 train/validation/test split with 10-fold stratified cross-validation.
Outcome: The fine-tuned BERT model was selected as the optimal classifier.

Module 2: Communication Coach (Multiclass Classification & Explanation)

Objective: Once toxicity is detected, categorize the comment into specific sub-categories (based on Sarker et al.'s taxonomy of 11 types) and provide an explanation.
Data: A multi-label, multi-class dataset of 1,200 samples (600 manually annotated toxic samples + 600 non-toxic samples).
Models:
- Claude 3.5 Sonnet and Llama 3.3 70B.
- GPT-4o used for prompt optimization.
Technique: Due to data scarcity preventing traditional fine-tuning, the authors employed Prompt Engineering with In-Context Learning. They developed a 5-element prompt strategy (role, task, definitions, guidelines, XML output) and refined it iteratively to condition the model to act as an experienced OSS maintainer. Deterministic decoding (temperature = 0) was used for reproducibility.

Module 3: The Reframer (Text Detoxification)

Objective: Rewrite toxic comments into constructive, semantically equivalent, professional alternatives while preserving technical accuracy.
Approach: Formulated as a Text Style Transfer (TST) task.
Data Synthesis: Since no parallel corpus (toxic vs. non-toxic pairs) existed for the SE domain, the authors synthesized one:
- Source: 10,120 toxic comments from Module 1.
- Generation: Used GPT-4o, Qwen 2.5 Instruct 7B, and GPT-4o mini to rewrite toxic comments into respectful versions using Chain-of-Thought (CoT) reasoning.
- Selection: The dataset generated by GPT-4o-05-13 (highest J-score of 88.14%) was selected as the training corpus.
Models: Fine-tuned Llama 3.2 3B (distilled student model).
Evaluation: Measured using Style Transfer Accuracy, Fluency, Content Preservation, and the J-score (a composite metric).

3. Key Results

4. User Evaluation

Setup: The three modules were integrated into a browser extension.
Participants: 10 professional software developers.
Duration: 2 weeks.
Framework: Technology Acceptance Model (TAM).
Findings: The tool was well-received, particularly regarding ease of use and satisfaction. Developers acknowledged its utility, though improvements in task efficiency and error management are needed for broader adoption.

5. Significance and Contributions

First Real-Time Tool: This is the first study to provide a real-time detoxification tool specifically for code reviews that includes explanations for why a comment is flagged.
Holistic Approach: Unlike previous works that only flag toxicity, ToxiShield offers a complete workflow: Detection $\rightarrow$ Explanation $\rightarrow$ Remediation.
Data Contribution: The authors synthesized a novel parallel corpus for toxic-to-civil code review translation and made the dataset, code, and tool publicly available (DOI: 10.5281/zenodo.19490337).
Community Impact: By integrating directly into the developer workflow (GitHub PRs), ToxiShield aims to prevent toxic interactions before they occur, thereby reducing barriers for newcomers and improving the overall health of the open-source ecosystem.