ChildGuard: A Specialized Dataset for Combatting Child-Targeted Hate Speech

Technical Summary: ChildGuard: A Specialized Dataset for Combatting Child-Targeted Hate Speech

Problem Statement
The mental health industry faces increasing concerns regarding hate speech directed at children on social media. Exposure to such content during critical developmental stages can lead to adverse psychological outcomes, including emotional distress, anxiety, social isolation, and reduced self-esteem. However, existing hate speech detection systems and datasets are predominantly adult-centric. They typically model the relationship between textual content ($X$) and hate speech labels ($Y \in \{0, 1\}$) without accounting for age-specific characteristics ($A$). Consequently, these systems lack the dedicated representations necessary to effectively detect hate speech targeting children, whose emotional resilience and cognitive maturity differ significantly from adults.

Methodology
The authors introduce ChildGuard, a large-scale English dataset designed to bridge this gap. The methodology encompasses data collection, filtration, and annotation:

• Data Collection: Approximately 580,000 raw text instances were collected from three major platforms where children actively participate: Reddit, X (formerly Twitter), and YouTube. Collection utilized public APIs (PRAW, Tweepy, YouTube Data API v3) and, where necessary, custom scripts (BeautifulSoup, Selenium) to navigate thread depth restrictions. A lexicon of child-related keywords (e.g., "kid," "teenager," "diaper," "school") was used to filter relevant content.
• Filtration: A multi-stage pipeline removed non-English text, spam, duplicates (via SHA-256 hashing), and instances with fewer than 10 tokens or excessive emojis/URLs. This process retained 469,910 instances for annotation.
• Annotation: Three experienced annotators manually labeled the retained instances. Each instance received:
  • A hate speech label ($Y \in \{0, 1\}$).
  • An age-group label ($A \in \{\text{younger child}, \text{pre-teen}, \text{teen}\}$), corresponding to under 11, 11–12, and 13–17 years, respectively.
  • A subset category ($S \in \{\text{lexical}, \text{contextual}\}$). The lexical subset (194K instances) focuses on vocabulary and direct insults, while the contextual subset (157K instances) captures discourse-level characteristics, sarcasm, and indirect references.
• Privacy: Prior to annotation, all Personally Identifiable Information (PII) was anonymized using spaCy's NER model and regex rules, replacing identifiers with SHA-256 hashed tokens.

Key Contributions

1. ChildGuard Dataset: The introduction of a large-scale dataset containing 351,877 annotated instances, which is the first to combine child-focused content with specific age-group annotations and a distinction between lexical and contextual hate speech.
2. Empirical Benchmarking: The paper provides a comprehensive evaluation of recent transformer-based models (DeBERTa-v3-large, ModernBERT, RoBERTa-large, HateBERT) and Large Language Models (GPT-4o, GPT-4.5, Claude-3.7 Sonnet, Gemini-2.5 Pro, DeepSeek-V3) on this specific domain.

Results
Evaluation on the held-out test set reveals significant challenges in detecting child-targeted hate speech:

• Overall Performance: The best-performing model, GPT-4.5, achieved a Macro-F1 score of 82.07% on the full dataset.
• Age-Group Variance: Performance consistently declined as the target age decreased. GPT-4.5 scores dropped to 79.41% for pre-teens and 79.24% for younger children, indicating that hate speech targeting younger demographics contains more subtle and context-dependent characteristics.
• Lexical vs. Contextual: Models performed significantly better on the lexical subset (85.11% Macro-F1 for GPT-4.5) compared to the contextual subset (79.24%), highlighting the difficulty of detecting indirect hostility and discourse-level abuse.
• Error Analysis: The primary sources of misclassification were implicit hate (31.4%), context dependence (25.8%), and age ambiguity (18.9%).
• Cross-Subset Generalization: Models struggled to generalize across subsets; when transferring from lexical to contextual instances, GPT-4.5's performance dropped to 74.88%.

Significance and Claims
The paper claims that ChildGuard addresses a critical scarcity in resources for the mental health industry, providing a necessary foundation for developing child-focused hate speech detection systems. The authors emphasize that current state-of-the-art systems, while achieving moderate success, remain insufficient for this specific application due to their inability to handle age-specific nuances and implicit context. The dataset serves as a benchmark to motivate the development of models that can better account for the relationship between textual content, age-group information, and hate speech labels, moving beyond traditional $P(Y|X)$ formulations to more complex, age-aware frameworks.

Limitations
The authors note that the dataset is limited to English content from three specific platforms. Age-group annotations are inferred from textual cues rather than verified demographic data, and some instances involving implicit hate remain inherently ambiguous. Furthermore, the benchmark results are restricted to the evaluated models, leaving broader architectures and multilingual settings for future research.