Causality Guided Representation Learning for Cross-Style Hate Speech Detection

Technical Summary: Causality Guided Representation Learning for Cross-Style Hate Speech Detection (CADET)

Problem Definition
The proliferation of online hate speech presents a significant challenge for content moderation, particularly regarding the detection of implicit hate speech. While explicit hate speech is characterized by overt slurs and direct attacks, implicit hate relies on sarcasm, irony, stereotypes, and coded language, making it difficult for models trained on surface-level linguistic cues to detect. Current detection systems suffer from poor generalization across diverse stylistic variations and platforms. Models often rely on spurious correlations between specific stylistic features (e.g., specific slurs or platform-specific syntax) and hate labels, failing when confronted with implicit expressions or when deployed on platforms with different demographic and moderation contexts. The core problem is distinguishing the genuine hate intent (the causal factor) from the stylistic manifestation (the confounding factor) which varies across domains.

Methodology: The CADET Framework
The authors propose CADET (Causality Guided Representation Learning for Cross-Style Hate Speech Detection), a framework grounded in a hypothesized causal graph of hate speech generation.

1. Causal Hypothesis: The generation of hate speech is modeled as a process involving:

   • Contextual Environment ($U$): An unobserved confounder (e.g., platform policies, demographics) that influences the creator's motivation, target selection, and stylistic choices.
   • Latent Factors:
     • Creator Motivation ($M$): The intent to cause harm.
     • Target ($T$): The group or individual targeted.
     • Style ($S$): The linguistic expression (explicit vs. implicit).
   • True Motivation ($M^*$): A domain-invariant latent variable representing the genuine hate intent, independent of the contextual environment $U$.
   • Label ($Y$): Determined by the true motivation $M^*$ and the context $U$.

2. Architecture Components:

   • Causally-Aligned Disentanglement: The model uses a pre-trained encoder (RoBERTa) to map text into a latent space. It then disentangles this representation into four latent factors: continuous variables for motivation ($z_m$) and confounder ($z_u$), and discrete variables for target ($z_t$) and style ($z_s$) using Gumbel-Softmax.
   • Confounder Mitigation: To prevent the latent factors from encoding spurious correlations with the context, the framework employs:
     • Adversarial Confounder Reconstruction: A gradient reversal layer (GRL) is used where discriminators attempt to recover the confounder $U$ from the merged latent factors. The encoder is trained to minimize this recovery, effectively removing $U$ from the causal factors.
     • Orthogonality Constraints: Projections ensure that the latent factors ($M, T, S, U$) are mutually orthogonal, enforcing statistical independence.
     • Input Reconstruction: A decoder reconstructs the original text from the fused latent factors to ensure the disentanglement preserves semantic meaning.
   • Latent Counterfactual Reasoning: To enforce style invariance, the model intervenes on the style latent variable ($S$) within the latent space. A "style flipping network" inverts the style (e.g., explicit to implicit) while keeping motivation and target fixed. The model is trained to ensure the hate prediction remains consistent regardless of the style intervention (Counterfactual Consistency Loss).

3. Training Objective: The model is optimized using a multi-objective loss function combining task loss (hate classification), target/style classification, orthogonality, adversarial deconfounding, reconstruction, and counterfactual consistency losses. A staged training curriculum is employed to stabilize the optimization of these competing objectives.

Key Contributions

• Principled Causal Graph: The paper introduces a formal causal model that explicitly separates the generative process of hate speech into interpretable latent factors (motivation, target, style, context), unifying explicit and implicit hate detection.
• CADET Framework: The implementation of a representation learning framework that operationalizes this graph, utilizing counterfactual reasoning and adversarial deconfounding to isolate hate intent from stylistic artifacts.
• Empirical Validation: Extensive experiments on four real-world datasets (IsHate, IHC, AbuseEval, DynaHate) demonstrating superior performance in cross-style generalization tasks compared to state-of-the-art baselines, including traditional NLP models, specialized hate speech detectors, and large language models.

Results

• Cross-Style Generalization: CADET achieves an average macro-F1 of 0.81 in explicit-to-implicit transfer tasks, representing a 13% relative improvement over the strongest baseline. In implicit-to-explicit transfer, it maintains a strong average F1 of 0.885.
• Robustness: Unlike conventional models that exhibit high recall but low precision in implicit settings (or vice versa), CADET maintains balanced performance. It significantly outperforms causal baselines (PEACE, HateWATCH) and LLM-based detectors (ShieldGemma, Llama Prompt Guard), which show erratic performance or failure in cross-style scenarios.
• Ablation Studies: Removing the counterfactual loss ($L_{cf}$) results in the most significant performance drop (24-point F1 decrease), confirming the critical role of style intervention in learning invariant representations.
• Visualization: t-SNE visualizations confirm that the motivation latent space ($z_m$) clearly separates hateful from non-hateful posts, while target, style, and confounder spaces do not, validating the successful disentanglement of the causal factors.

Significance
The paper claims that CADET offers a principled solution to the challenge of style-invariant hate speech detection. By shifting focus from surface-level linguistic patterns to underlying causal intent, the framework addresses the brittleness of current models against implicit hate and domain shifts. The work highlights the potential of causal disentanglement and counterfactual reasoning in creating more robust, interpretable, and generalizable content moderation systems, providing a pathway toward safer online environments that can adapt to evolving forms of hate speech.