💬 NLP

ContiGuard: A Framework for Continual Toxicity Detection Against Evolving Evasive Perturbations

This paper introduces ContiGuard, a novel framework for continual toxicity detection that leverages an LLM-powered semantic enriching strategy and a discriminability-driven feature learning approach to dynamically update detection capabilities and maintain resilience against evolving, semantics-distorting evasive perturbations.

Original authors: Hankun Kang, Xin Miao, Jianhao Chen, Jintao Wen, Mayi Xu, Weiyu Zhang, Wenpeng Lu, Tieyun Qian

Published 2026-03-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Hankun Kang, Xin Miao, Jianhao Chen, Jintao Wen, Mayi Xu, Weiyu Zhang, Wenpeng Lu, Tieyun Qian

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The Never-Ending Game of "Hide and Seek"

Imagine the internet as a giant, bustling town square. In this square, there are Toxicity Detectors (like security guards) whose job is to spot mean, hateful, or dangerous comments and stop them.

For a long time, these guards were good at spotting obvious insults like "You are an idiot." But then, the "bad guys" (malicious users) started playing a game of Hide and Seek. They realized that if they just changed the spelling of "idiot" to "i-d-1-0-t" or "iiiddioot," the guards couldn't recognize it anymore.

The problem? The guards were static. They were like security guards who only knew how to spot one specific type of disguise. As soon as the bad guys invented a new disguise (a new "perturbation"), the guards failed.

ContiGuard is a new, super-smart security system designed to learn on the fly. It doesn't just memorize old tricks; it evolves every time the bad guys try something new.

The Three Superpowers of ContiGuard

The paper proposes three main strategies to make this security guard unbeatable. Think of them as three special tools in the guard's toolkit:

1. The "Sherlock Holmes" Lens (LLM Powered Semantic Enriching)

The Problem: When bad guys scramble text (e.g., "id10t"), it looks like gibberish. The security guard gets confused and can't see the anger underneath the mess.
The Solution: ContiGuard brings in a Sherlock Holmes (an AI called an LLM).

How it works: When the guard sees a scrambled word like "id10t," it asks Sherlock, "What does this really mean?" Sherlock looks at the context and says, "Ah, even though it's scrambled, the user is clearly calling someone an idiot with an aggressive tone."
The Magic: ContiGuard takes Sherlock's insight and adds it back into the scrambled text. It's like putting a pair of glasses on the guard that reveals the hidden meaning. Now, the guard sees "idiot" clearly, even if the text is written as "id10t."

2. The "Gold Filter" (Discriminability Driven Feature Learning)

The Problem: Scrambled text is full of noise. It's like trying to find a needle in a haystack, but the haystack is also on fire. The guard gets distracted by the fire (the random letters added to hide the word) and forgets to look for the needle (the actual hate).
The Solution: ContiGuard uses a Gold Filter.

How it works: It analyzes every part of the text and asks, "Is this part actually important for deciding if this is hate speech?"
- If the text has a random extra "z" or a weird symbol, the filter says, "Ignore that, it's just noise."
- If the text has a strong insult, the filter says, "Focus on this! This is critical!"
The Magic: It forces the guard to ignore the distractions and only learn from the "gold" (the real clues), making the guard much harder to trick.

3. The "Memory Book" (Historical Capability Replay)

The Problem: In the world of AI, there is a phenomenon called "Catastrophic Forgetting." Imagine a student who studies for a math test today. When they study for a history test tomorrow, they suddenly forget how to do math. This happens to AI too. As the guard learns to spot new disguises, it often forgets how to spot the old ones.
The Solution: ContiGuard keeps a Memory Book.

How it works: The system saves a few examples of the "old" disguises it has already learned. Every time it learns something new, it quickly flips through the book to review the old stuff.
The Magic: This ensures the guard doesn't lose its past skills. It stays sharp on all types of disguises, from the very first one it saw to the newest one.

Why This Matters (The Results)

The researchers tested ContiGuard against:

Old Guards: Standard detectors that get easily fooled.
Static Trainers: Guards that were trained on specific tricks but couldn't adapt to new ones.
Other Learning Systems: AI that tries to learn continuously but forgets too much.

The Result: ContiGuard won hands down.

It handled the "scrambled" text much better than anyone else.
It didn't forget old tricks while learning new ones.
It even worked well on real-world data collected from the wild, not just lab tests.

The Takeaway

Think of ContiGuard as a chameleon security guard.

When the bad guys change their colors (text), the guard changes its strategy.
It uses a detective (LLM) to understand the disguise.
It uses a filter to ignore the junk.
It uses a diary to remember everything it has ever learned.

This framework ensures that as long as bad actors try to hide their hate speech, the internet's security system will be one step ahead, evolving right alongside them to keep our online spaces safe.

1. Problem Definition

The paper addresses the critical challenge of Continual Toxicity Detection in the face of evolving evasive perturbations.

Context: Malicious users continuously develop new text perturbations (e.g., character repetition, homoglyph substitution, slang replacement) to disguise toxic content and bypass static toxicity detectors.
Limitation of Existing Methods:
- Static Detectors: Traditional models trained on clean or specific perturbed text fail to generalize to new types of perturbations appearing over time.
- Standard Continual Learning (CL): Existing CL methods struggle in this domain because:
  1. Semantic Corruption: Perturbations distort text semantics, making it difficult for models to comprehend hidden toxicity.
  2. Feature Noise: Perturbations introduce redundant, non-critical features (e.g., specific manipulation patterns) that distract the model from learning core toxic features.
  3. Catastrophic Forgetting: The heterogeneity between different perturbation domains causes the model to forget previously learned detection capabilities as it adapts to new ones.

The authors formulate this as a domain-incremental learning problem, where the detector must sequentially learn from distinct perturbation domains ( $p_1, p_2, ..., p_T$ ) over time without forgetting previous knowledge.

2. Methodology: The ContiGuard Framework

ContiGuard is the first framework designed specifically for continual toxicity detection against evolving perturbations. It integrates three core strategies:

A. LLM-Powered Semantic Enriching Strategy

To combat semantic corruption and improve text comprehension:

Mechanism: The framework leverages a Large Language Model (LLM) to extract auxiliary information (possible original meanings and toxicity-related clues) from the perturbed text.
Process:
1. How-Why-What Interrogation: The LLM is prompted using a structured logic (inspired by the Golden Circle) to analyze how to mine clues, why the analysis is needed, and what the specific clues are.
2. Dynamic Cooperation: The LLM-generated auxiliary features are dynamically weighted and fused with the perturbed text features.
3. Optimization Benefit: The auxiliary information acts as a momentum factor during optimization. Since LLM-generated insights are less noisy than the perturbed text, they smooth the gradient landscape, reducing the risk of the model getting trapped in local optima caused by perturbation noise.

B. Discriminability-Driven Feature Learning Strategy

To mitigate the interference of non-critical (noisy) features and focus on robust classification:

Attribution Analysis: The framework uses Integrated Gradients to measure the contribution of each feature dimension to the toxicity prediction.
Differentiation: Features are categorized based on their discriminative power:
- Discriminative Features: Those that contribute differently to toxic vs. non-toxic classes.
- Less-Discriminative Features: Those that contribute similarly to both classes (often noise introduced by perturbations).
Learning Mechanism:
- Enhancement: Discriminative features are strengthened via global flipping (a low-complexity rotation) to decouple classification boundaries from absolute feature directions.
- Suppression: Less-discriminative features are "unlearned" by forcing the model to predict a uniform distribution for them, effectively resetting them to neutrality. This forces the model to rely only on core toxic features.

C. Historical Capability Replay Strategy

To prevent catastrophic forgetting across different perturbation domains:

Mechanism: The framework maintains a memory buffer of representative samples from previous perturbation domains.
Feature Alignment: Instead of just replaying raw data, ContiGuard aligns the feature representations of these memory samples between the old (frozen) detector and the current detector using cosine similarity loss.
Goal: This ensures that the new model retains the ability to recognize previously seen perturbation types by preserving the critical feature space learned in the past.

3. Key Contributions

Problem Formulation: The paper identifies and formalizes "Continual Toxicity Detection" as a distinct, overlooked problem where detectors must adapt to time-evolving, diverse perturbation domains.
Novel Framework (ContiGuard): Proposes the first end-to-end framework combining LLM-based semantic enrichment, discriminability-driven feature selection, and historical capability replay.
Dataset Construction (DynEscape): Introduces DynEscape, a large-scale, perturbation-wise dataset containing 9 distinct types of evasive perturbations (Insert, Repeat, Swap, Remove, Homoglyph, Mask, Abbr, Auth, Distract), specifically designed to benchmark continual learning in this domain.
Comprehensive Evaluation: Extensive experiments demonstrate that ContiGuard outperforms both static detectors and state-of-the-art continual learning baselines.

4. Experimental Results

Datasets: Evaluated on DynEscape (9 perturbation types) and NoisyHate (real-world human-written perturbations).
Baselines: Compared against commercial APIs (PerspectiveAPI, OpenAI), fine-tuned models (ToxicBERT, Detoxify), and 8 standard Continual Learning methods (EWC, GEM, LWF, etc.).
Key Findings:
- Superior Performance: ContiGuard achieved an average accuracy of 86.47% on DynEscape, significantly outperforming the next best method (Joint training at 85.88%) and existing CL methods (which often dropped below 80%).
- Robustness: While static detectors suffered relative performance drops of 14.4% to 35.8% on perturbed text, ContiGuard only dropped by 3.0% compared to ordinary text.
- Ablation Studies: Removing any of the three core strategies (LLM enrichment, discriminability learning, or historical replay) resulted in significant performance degradation, confirming the necessity of each component.
- Efficiency: The framework can operate effectively with reduced training data (20-60% sampling) and can function without the LLM during inference (ContiGuard $_{Eco}$ ) while maintaining high detection speeds (~220 items/s) and strong generalization.

5. Significance

Practical Impact: ContiGuard offers a viable solution for real-world content moderation systems that face a "moving target" of adversarial attacks. It moves beyond static training to a dynamic, self-updating defense mechanism.
Theoretical Advancement: It bridges the gap between Large Language Models (for reasoning/semantics) and Continual Learning (for adaptation), demonstrating how LLMs can act as "momentum" to stabilize training in noisy environments.
Resource Availability: The authors release the DynEscape dataset and the ContiGuard code, establishing a new benchmark for future research in robust and continual toxicity detection.

In conclusion, ContiGuard represents a significant leap forward in securing online environments, providing a robust mechanism to detect toxic content even as attackers continuously evolve their evasion tactics.