Technical Summary: Leveraging LLMs for Context-Aware Implicit Textual and Multimodal Hate Speech Detection

Problem Statement

Hate Speech Detection (HSD) in social media is increasingly challenged by the prevalence of implicit hate speech, where hostility is veiled through irony, euphemism, or coded language, often requiring external world knowledge to interpret. Traditional HSD models typically operate in zero-context settings, analyzing raw posts in isolation. While effective for explicit hate, these models struggle with implicit intent.

Previous attempts to incorporate context have relied on:

1. Conversational Context: Using surrounding replies or thread history, which is often unavailable in isolated datasets.
2. Background Context via Entity Linking: Augmenting posts with external knowledge (e.g., Wikipedia, ConceptNet) based on extracted named entities. However, these static approaches often suffer from poor entity detection, irrelevant data linking, or noise that degrades performance, particularly in fine-grained multi-class tasks.

This research addresses the limitations of static entity-linking by investigating Large Language Models (LLMs) as dynamic knowledge bases. The core problem is determining whether LLMs can generate superior, post-specific background context and, crucially, how to best incorporate this generated context into a downstream classifier to improve detection of implicit and multimodal hate.

Methodology

The proposed framework consists of two primary stages: Context Generation and Context Incorporation, evaluated across textual and multimodal domains.

1. Context Generation

Instead of extracting entities and retrieving static data, the authors prompt an LLM (Gemini 2.0 Flash) to generate auxiliary background context based on the full text of each post.

• Prompting Strategy: The LLM is tasked with producing objective background knowledge (relevant entities, events, implicit meanings, rhetorical devices) without being informed of the classification task, thereby preventing label leakage.
• Goal: To allow the LLM to dynamically determine relevance and resolve ambiguous references that entity-level retrieval might miss.

2. Datasets

• Latent Hatred (Textual): 21,480 tweets focusing on implicit hate speech, including binary (hate/non-hate) and six-way multi-class classification (e.g., incitement, stereotyping, irony).
• MAMI (Multimodal): 10,995 misogynous memes, classified via binary (misogynous/non-misogynous) and multi-label subtypes (shaming, stereotyping, objectification, violence). Multimodal inputs are textualized by combining extracted image text and LLM-generated image descriptions.

3. Context Incorporation Strategies

Four distinct methods were tested to integrate the generated context into a Sentence BERT (SBERT) encoder followed by a Multi-Layer Perceptron (MLP) classifier:

1. Append & Embed: Concatenating the generated context directly to the post text (separated by [SEP]) before encoding.
2. Embed & Concat: Encoding the post and the context separately into vectors, then concatenating the resulting embeddings.
3. Context-Embed: A hierarchical fusion where the context is encoded via SBERT, projected to match token dimensions, and prepended to the post's token embedding sequence before passing through the remaining encoder layers.
4. LLM Enhance: Prompting the LLM to rewrite the original post, integrating the context into a single, modified text before encoding.

4. Baselines

The study compares the proposed methods against:

• Zero-Context: Raw post input only.
• REL (Entity Linking): Replicating Lin (2022) using the Radboud Entity Linker to append Wikipedia summaries.
• ConceptNet: Replicating ElSherief et al. (2021) using ConceptNet embeddings for extracted entities.
• LLM Classifier: Using the generative LLM directly for prediction (zero-shot) without a downstream classifier.

Key Contributions

1. Shift from Entity-Centric to Post-Centric Context: The paper proposes and validates a methodology where LLMs generate context based on the holistic post content rather than isolated named entities, addressing the limitations of static knowledge retrieval.
2. Systematic Evaluation of Incorporation Methods: The study provides a comparative analysis of four distinct strategies for fusing LLM-generated context with classifier inputs, identifying that Embed & Concat (separate encoding followed by vector concatenation) is the most effective approach.
3. Multimodal Extension: The framework is adapted for multimodal hate speech (memes), demonstrating the generalizability of context-enhanced detection across domains.
4. Trade-off Analysis: The research contrasts the utility of using LLMs as context generators versus direct classifiers, highlighting the benefits of a modular pipeline.

Results

Textual Setting (Latent Hatred)

• Performance Gain: The Embed & Concat strategy achieved the highest performance, improving the Macro F1 score by 3 points (from 0.72 to 0.75) in binary classification and 2 points (0.51 to 0.53) in multi-class classification over the zero-context baseline.
• Comparison to Baselines: The proposed LLM-generated context outperformed both the REL and ConceptNet baselines. Notably, the REL approach (static entity linking) resulted in a 4-point drop in multi-class F1 compared to the zero-context model, confirming the limitations of static retrieval in noisy data.
• LLM Classifier: Direct LLM classification performed comparably to baselines in binary tasks but significantly underperformed in multi-class tasks, suggesting that fine-tuned encoders with context augmentation are superior for nuanced categorization.

Multimodal Setting (MAMI)

• Performance Gain: Context incorporation improved performance across all methods. The Embed & Concat approach yielded the highest gains, increasing binary Macro F1 by 6 points (0.79 to 0.85) and multi-label Macro F1 by 4 points (0.59 to 0.63) over the zero-context baseline.
• Baseline Performance: The REL baseline underperformed the zero-context model, likely due to poor named entity coverage in memes. The LLM Classifier performed strongly in binary classification (0.87 F1) but remained competitive in multi-label tasks.

Error Analysis

The study identified specific failure modes:

• False Negatives: Adding lengthy context to explicitly hateful posts sometimes "neutralized" the hate, causing the model to interpret the combined input as descriptive rather than hostile.
• False Positives: The LLM occasionally invented hateful context for benign posts (e.g., associating a warning about conspiracy groups with hate speech), leading to over-classification.
• Multimodal Ambiguity: In memes, generated context sometimes failed to recognize specific visual references (e.g., Bill Cosby), leading to missed implications of sexual assault or misogyny.

Significance and Claims

The paper claims that generating background context from the full post content is a more effective strategy for implicit HSD than traditional entity-linking. The primary significance lies in demonstrating that:

1. Context Quality Matters: The method of context generation (LLM vs. static retrieval) significantly impacts downstream performance.
2. Separation is Key: The method of incorporation is critical; preserving a strict distinction between the original post and the generated context (via Embed & Concat) prevents semantic interference and yields better results than text concatenation or hierarchical fusion.
3. Modular Utility: Using LLMs as dynamic knowledge bases offers a viable alternative to direct LLM classification, providing a lightweight, modular framework that enhances traditional classifiers without sacrificing the interpretability or efficiency of encoder-based models.

The authors conclude that while LLM-generated context improves HSD performance, the challenge extends beyond generation to validation and integration, ensuring that context complements rather than overrides the original meaning of the post. The work advocates for a shift in contextualized HSD from static, entity-centric retrieval to dynamic, post-specific context generation.