💬 NLP

STEMTOX: From Social Tags to Fine-Grained Toxic Meme Detection via Entropy-Guided Multi-Task Learning

This paper introduces the TOXICTAGS dataset of 6,300 annotated memes enriched with social tags and proposes STEMTOX, an entropy-guided multi-task learning framework that leverages these tags to significantly improve fine-grained toxic meme detection using vision-language models.

Original authors: Subhankar Swain, Naquee Rizwan, Vishwa Gangadhar S, Nayandeep Deb, Animesh Mukherjee

Published 2026-03-17

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Subhankar Swain, Naquee Rizwan, Vishwa Gangadhar S, Nayandeep Deb, Animesh Mukherjee

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the internet as a giant, chaotic digital town square. In this square, people don't just talk; they share "memes"—pictures with funny (or sometimes mean) captions that spread like wildfire. While many are harmless jokes, some are like hidden landmines: they look like jokes but are actually spreading hate, danger, or toxicity.

The paper STEMTOX is a new toolkit designed to help the "town guards" (content moderators) spot these dangerous landmines more accurately. Here is how they built it, explained simply:

1. The Problem: The "Joke" Trap

Detecting toxic memes is hard because they are tricky. A picture might show a cartoon character, but the text underneath is a coded insult. Current computer programs (called AI models) often miss these nuances because they are trained on old, simple data or only look at the picture without understanding the "vibe" or context.

2. The New Map: The TOXICTAGS Dataset

To fix this, the researchers built a new, high-quality map of the internet called TOXICTAGS.

Real-World Fuel: Instead of making up fake examples, they collected 6,300 real memes from the internet.
Two-Step Sorting: They didn't just ask, "Is this bad?" They used a two-step process:
1. Step One: Is this post toxic or normal? (Like a security guard checking a bag).
2. Step Two: If it is toxic, what kind is it? Is it Hateful (attacking a group), Dangerous (promoting violence or self-harm), or just Offensive (rude but not necessarily dangerous)?
The Secret Ingredient (Tags): The biggest innovation here is that every meme came with a list of "social tags" (like #Hitler, #SchoolShooting, #SesameStreet). Think of these tags as the context clues or the "whispered gossip" around the meme. They tell the AI what the meme is really about, even if the picture looks innocent.

3. The New Detective: STEMTOX

The researchers built a new AI framework called STEMTOX. You can think of it as a detective who uses a special trick called "Entropy-Guided Multi-Task Learning."

The "Entropy" Trick (Finding the Calmest Voice): Imagine a room full of people shouting different theories about a crime. Some are confused, some are guessing, and some are very sure. "Entropy" is a way to measure how confused or "noisy" the AI is. STEMTOX looks at the AI's internal brain layers and picks the moment where the AI is least confused (lowest entropy) to make its final decision. It ignores the noise and listens to the most confident signal.
The "Multi-Task" Trick (Doing Two Jobs at Once): Usually, AI is trained to do one thing: "Is this toxic?" STEMTOX is trained to do two things at the same time:
1. Classify: Decide if it's toxic.
2. Generate: Invent the social tags (like #Hitler or #9/11) that explain why it's toxic.
The Analogy: It's like training a student not just to pass a test, but to also write the study guide. By forcing the AI to "write the tags" (explain the context), it learns to understand the meme much deeper, which makes it better at spotting the toxicity.

4. The Results: Smarter Guards

When they tested STEMTOX:

It got better at spotting the bad stuff: By using the tags and the "two-job" training, the AI became much better at distinguishing between a harmless joke and a hateful one compared to previous models.
It works on old maps too: Even when they tested it on other existing datasets (that didn't have tags), it still performed better than other top methods. This suggests that the "training to explain" method makes the AI smarter generally.
It found the hidden patterns: The analysis showed that toxic memes often use specific combinations of tags (like mixing "Hitler" with "9/11" or using "Sesame Street" characters in dark contexts) to hide their true intent. STEMTOX learned to recognize these patterns.

Summary

In short, the authors realized that to catch toxic memes, you can't just look at the picture; you need to understand the context. They built a massive new library of real memes with context clues (tags) and taught a new AI detective to explain the meme while judging it. This "teach-by-explaining" method made the AI significantly sharper at spotting harmful content that tries to hide behind a joke.

Technical Summary: STEMTOX – From Social Tags to Fine-Grained Toxic Meme Detection via Entropy-Guided Multi-Task Learning

1. Problem Statement

Memes have evolved from lighthearted humor into potent vehicles for spreading harmful content, including hate speech, misinformation, and toxic ideologies. Detecting such content is uniquely challenging due to the subtle, context-dependent nature of memes, which often rely on cultural references, sarcasm, and coded language. While Vision-Language Models (VLMs) have emerged as powerful tools for content moderation, recent studies indicate they struggle to accurately detect hateful memes, particularly when lacking deep contextual understanding.

A critical bottleneck in advancing this field is data scarcity. Existing datasets are often manually curated with limited scope (focusing on specific events or targets), lack fine-grained toxicity labels beyond binary classification, and frequently omit the auxiliary metadata (such as social tags) that provides crucial context. Furthermore, current approaches often fail to leverage the semantic richness of social tags to guide classification, resulting in performance gaps in robust toxicity detection.

2. Methodology

A. The TOXICTAGS Dataset
To address data limitations, the authors introduce TOXICTAGS, a novel, diverse, real-world dataset comprising 6,300 meme-based posts sourced from Imgflip. Unlike prior datasets, TOXICTAGS is event-independent and enriched with auxiliary metadata, including titles and socially relevant tags.

The dataset is constructed through a rigorous two-stage human annotation pipeline:

Stage I (Binary Classification): Memes are classified as either toxic or normal.
Stage II (Fine-Grained Classification): Toxic memes are further categorized into three nuanced classes: hateful, dangerous, or offensive.
Tagging: Each post is annotated with a set of social tags that contextually ground the meme. The dataset includes 7,209 unique tags after filtering noise.
Quality Control: The process involved 25 annotators, with three independent annotations per sample to minimize bias. The final inter-annotator agreement scores were 0.753 (Stage I) and 0.793 (Stage II), measured by Fleiss' $\kappa$ .

B. The STEMTOX Framework
The authors propose STEMTOX, a novel entropy-guided multi-task learning framework designed to jointly perform toxic meme classification and socially grounded tag generation.

Architecture: STEMTOX utilizes a multimodal VLM backbone (tested with PALIGEMMA and LLAVA) that processes image and text inputs.
Entropy-Guided Layer Selection: Instead of relying solely on the final hidden layer, STEMTOX dynamically selects the most confident token embeddings across intermediate layers. It computes the Shannon entropy of token representations at each layer $l$ . The layer with the minimum entropy (highest confidence) is selected to form the effective sequence representation ( $h_{eff}$ ). This mechanism allows the model to extract features from the most certain internal representations, mitigating representational drift often seen in terminal layers.
Multi-Task Objective: The framework is trained with a combined loss function:
$L_{total} = L_{gen} + L_{cls}$
Where $L_{gen}$ is the generative loss for predicting social tags, and $L_{cls}$ is the classification loss for toxicity detection. The generative task acts as a semantic regularizer, shaping the shared representation space to be more discriminative for toxicity classification.
Training Strategy: The model employs LoRA adapters for efficient fine-tuning and uses weighted cross-entropy to handle class imbalance.

3. Key Contributions

TOXICTAGS Dataset: The first real-world meme dataset featuring fine-grained toxicity labels (hateful, dangerous, offensive, normal) alongside rich, human-annotated social tags. It moves beyond the traditional binary toxic/normal paradigm.
Two-Stage Annotation Pipeline: A validated process that reduces annotation errors and captures the nuance of meme toxicity, demonstrating that a four-class taxonomy is effective for moderation.
STEMTOX Framework: A novel multi-task architecture that integrates tag generation with toxicity classification. It introduces an entropy-guided mechanism to select optimal layer representations, significantly improving classification performance by leveraging the semantic supervision of tag generation.
Empirical Validation: Comprehensive evaluation showing that incorporating contextual tags and using entropy-guided selection substantially enhances state-of-the-art VLM performance.

4. Experimental Results

Performance on TOXICTAGS:

Toxicity Detection: STEMTOX with multi-task learning consistently outperformed uni-task baselines. Using the PALIGEMMA backbone, STEMTOX achieved a Macro-F1 score of 72.55 for Stage I and 64.66 for Stage II. This represents a significant improvement over uni-task training (60.85 and 42.78, respectively).
Comparison with Baselines: STEMTOX outperformed powerful few-shot GPT-4O setups (where examples were selected based on tag similarity) and models relying on Last Hidden Layer (LHL) features.
Tag Generation: In terms of tag generation quality (measured by chrF, METEOR, and Semantic Similarity), STEMTOX (PALIGEMMA) achieved scores of 0.4872, 0.265, and 0.626, respectively. This significantly surpassed state-of-the-art tag generators like RAM++ and TAG2TEXT, which struggled to capture the implicit social context and meme-specific vocabulary.

Generalizability:

The framework was tested on two external benchmark datasets, FHM (Hateful Memes) and MAMI (Misogynistic Memes), which lack native tags. Tags were inferred using a model fine-tuned on TOXICTAGS.
STEMTOX achieved superior results on these benchmarks compared to five competing baselines (PromptHate, ProCap, ModHate, Few-Shot, M2KE), reaching a Macro-F1 of 78.11 on FHM and 79.67 on MAMI.

Error Analysis:

Analysis revealed that models struggle most with "offensive" categories (highest uncertainty) and specific tags related to humor/slang (for PALIGEMMA) or death/religion (for LLAVA).
The entropy-guided selection was found to be crucial, as models relying on the final layer (LHL) showed lower performance.

5. Significance and Claims

The paper claims that STEMTOX offers a novel and scalable foundation for improved content moderation in multimodal online environments. By introducing a dataset enriched with social tags and a framework that leverages these tags through multi-task learning, the authors demonstrate that:

Context is Key: Auxiliary metadata (tags) is not merely supplementary but essential for robust toxicity detection.
Generative Supervision: Using a generative task (tagging) to supervise a discriminative task (classification) leads to richer, more meaningful internal representations.
Entropy Guidance: Dynamically selecting representations based on confidence (minimum entropy) is a more effective strategy than relying on fixed terminal layers for complex multimodal tasks.

The authors position this work as a step toward building more accurate, context-aware, and socially responsible moderation systems, addressing the gap between current VLM capabilities and the nuanced reality of online hate speech.

6. Limitations

The authors acknowledge several limitations:

Modality: The dataset and framework are currently limited to image-based memes, excluding video and audio formats.
Psychological Impact: The study does not examine the psychological or emotional impact of hateful memes on viewers.
Real-World Consequences: The link between online toxicity and offline violence or criminal behavior is not investigated.