💬 NLP

BioSentinel at EXIST 2026: Soft-Label Optimization with XLM-RoBERTa for Sexism Intent Classification in Memes

The BioSentinel team's paper details their participation in the EXIST 2026 Task 2.2, where they employed an XLM-RoBERTa-based model trained with a composite loss function to effectively balance soft-label and hard-label predictions for classifying sexist intent in memes, ultimately achieving specific rankings in the CLEF 2026 evaluation.

Original authors: Chandru Munisamy, Karthikeya Raguveer, Alapan Kuila

Published 2026-07-28

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Chandru Munisamy, Karthikeya Raguveer, Alapan Kuila

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the internet as a giant, chaotic town square where everyone is shouting, whispering, and drawing pictures on the walls. Sometimes, these pictures (called "memes") are just funny jokes, but other times, they hide mean-spirited comments about gender that are hard to spot. This is the world of sexism detection, a branch of computer science where we teach machines to read the room and understand if a joke is harmless or hurtful.

But here's the tricky part: humans don't always agree on what's mean. One person might see a meme as a direct insult, while another thinks it's just a subtle, ironic joke, and a third might not see anything wrong at all. This is called disagreement. In the past, computers were taught to pick just one "right" answer, like a teacher grading a test with a single correct key. However, this paper argues that when humans can't agree, the computer shouldn't just guess one answer; it should learn to understand the spread of opinions, kind of like a weather forecast that says "there's a 60% chance of rain" instead of just shouting "It's raining!" This approach is called Learning with Disagreement, and it helps machines become more nuanced and less overconfident.

The BioSentinel Team's Mission: Reading the Text, Ignoring the Picture

Meet the BioSentinel team, a group of researchers who entered a high-stakes competition called EXIST 2026. Their challenge? To build a robot that could look at memes and decide if the creator was being direct (obviously sexist), judgemental (using irony or subtle hints), or no (not sexist at all).

The twist? The competition didn't just want the robot to pick one label. It wanted the robot to spit out a probability distribution—a fancy way of saying, "I'm 70% sure this is direct, 20% sure it's judgemental, and 10% sure it's harmless." This mimics how a group of human judges might vote differently on the same meme.

The Strategy: A Text-Only Detective

The team, led by Chandru Munisamy and friends from India, decided to play a very specific game. Even though memes are pictures with text, they chose to ignore the pictures entirely. They acted like a detective who only reads the transcript of a conversation, ignoring the facial expressions and body language.

Why? They wanted to see if they could get really good at understanding the words and the uncertainty of the labels without getting distracted by the messy visual parts. They used a powerful language brain called XLM-RoBERTa (a model with 270 million parameters that speaks both English and Spanish) and taught it a special trick.

Instead of just telling the model, "This is the answer," they gave it two types of homework:

The Soft Homework: "Here is how the humans voted. Try to match that exact mix of opinions." They used a math tool called KL Divergence to make the model's guesses look like the human voting pattern.
The Hard Homework: "Here is the official winner of the vote. Make sure you get this right too." They used a standard tool called Weighted Cross-Entropy to ensure the model didn't get too confused.

By mixing these two lessons (70% soft, 30% hard), they hoped to create a model that was both accurate and aware of human disagreement.

The Results: A Solid, But Not Perfect, Performance

When the BioSentinel team tested their system on the official exam (the test set), here is what happened:

Soft-Label Score: They achieved a score of 0.3229 (called ICM-Soft-Norm). This was better than a basic "guess the most common answer" robot, which only got 0.2835.
Hard-Label Score: For picking the single best answer, they scored 0.4236 (F1-score).
Ranking: Out of 118 teams who submitted soft-label predictions, they came in 40th. Out of 187 teams submitting hard-label predictions, they were 49th.

The team found that their "mix-and-match" training method worked. When they removed the "soft" lesson (the KL divergence), their ability to match human opinions dropped drastically (from 0.312 down to 0.142). When they removed the "hard" lesson, their accuracy on single answers dropped. The combination was the key to their success.

The "Judgemental" Struggle

However, the robot had a blind spot. It was terrible at spotting judgemental sexism (the subtle, ironic kind). Its score for this specific category was a very low 0.071.

Why? The team realized that without the picture, the robot was missing the punchline. Many of these memes rely on a visual joke—like a caricature or a specific facial expression—that the text alone couldn't explain. For example, a text saying "Sitting at your family table..." might seem innocent, but if the picture shows a father getting the biggest fish and the mother getting the smallest, the joke is clear. Without the image, the robot just saw a neutral sentence.

The team also noticed something funny about how the robot learned. For the first two days of training, it didn't even guess "judgemental" once. It only started to understand this tricky category after it had already learned to tell the difference between "harmless" and "direct." This suggests that the subtle stuff is the hardest to learn and might need more time or special training.

What They Didn't Do (And Why It Matters)

The paper is very honest about what it didn't do. They didn't try to use the brainwave data (EEG) or eye-tracking data that the competition organizers provided. Why? Because the data didn't match up between the training and testing sets. They also didn't try to use bigger, more complex models because, in their specific setup, those models became unstable and confused.

They also tested a "temperature" setting (a knob that makes the robot's guesses more or less confident). On their practice test, turning this knob to 0.7 made the robot slightly better at matching human opinions (raising the score from 0.317 to 0.326) and made its confidence more realistic. However, for the final official test, they stuck to the standard settings to be safe.

The Takeaway

The BioSentinel team showed that you can build a decent sexism detector using just text, as long as you teach it to respect human disagreement. By using a mix of "soft" and "hard" lessons, they created a model that understands the gray areas of human opinion better than older methods.

But the paper also warns us: if you want to catch the sneaky, ironic jokes, you can't just read the text. You need to see the picture. The robot's struggle with the "judgemental" category proves that sometimes, the visual context is the only thing that makes the joke (or the insult) make sense. The team suggests that future robots should try to look at both the words and the pictures to get the full story.

Technical Summary: BioSentinel at EXIST 2026

Problem Definition
This paper addresses EXIST 2026 Task 2.2, "Source Intention in Memes," a challenge focused on classifying the communicative intent behind memes containing potentially sexist content. The task operates under a Learning with Disagreement (Le-Wi-Di) paradigm, requiring models to predict not only a single hard label (direct, judgemental, or no) but also a soft probability distribution reflecting the variance in human annotator opinions. The dataset consists of 3,984 training and 1,053 test instances in English and Spanish, each annotated by six human raters. A key challenge is the inherent subjectivity of "judgemental" (implicit/ironic) sexism, where annotators frequently disagree, and the class imbalance, with the "judgemental" category representing only 14.2% of resolvable instances.

Methodology
The BioSentinel team proposes a text-centric approach utilizing the xlm-roberta-base transformer (270M parameters). The system deliberately excludes image and physiological sensor data (EEG, eye tracking) to isolate the efficacy of soft-label optimization on text, citing issues with missing features and synchronization in the multimodal data.

The core innovation lies in the composite loss function designed to handle the Le-Wi-Di paradigm:
$\mathcal{L}_{train} = \alpha \cdot \mathcal{L}_{KL} + \beta \cdot \mathcal{L}_{CE}$

KL Divergence ( $\mathcal{L}_{KL}$ ): Minimizes the divergence between the model's predicted distribution and the soft gold distribution derived from annotator votes. This term ( $\alpha=0.7$ ) is the primary driver for aligning predictions with human uncertainty.
Weighted Cross-Entropy ( $\mathcal{L}_{CE}$ ): Applied to instances with official hard labels, using fixed normalized class weights to penalize the under-represented "judgemental" class more heavily ( $\beta=0.3$ ).
Architecture: The model processes OCR-extracted text through XLM-RoBERTa, followed by a classification head (768 $\to$ 256 $\to$ 3).
Training Configuration: Trained for 6 epochs on NVIDIA A100 GPUs using AdamW with a learning rate of $2 \times 10^{-5}$ . The final checkpoint (Epoch 4) was selected based on minimum validation KL divergence.
Temperature Analysis: While official test predictions used raw softmax ( $T=1.0$ ), a post-hoc analysis on the validation set indicated that a temperature of $T=0.7$ improved soft-label metrics and reduced Expected Calibration Error (ECE).

Key Results
On the official test set, the system achieved the following performance:

ICM-Soft-Norm: 0.3229 (Ranking 40th out of 118 submissions).
ICM-Norm: 0.3778.
Hard Macro-F1: 0.4236 (Ranking 49th out of 187 submissions).

The system significantly outperformed the majority-class hard baseline (F1 = 0.1676) and the prior-distribution soft baseline (ICM-Soft-Norm = 0.2835).

Ablation and Analysis

Loss Contribution: Ablation studies confirmed that KL divergence is critical for soft-label performance (ICM-Soft-Norm dropped from 0.312 to 0.142 when removed), while Cross-Entropy is essential for hard-label accuracy (F1 dropped from 0.348 to 0.294 when removed).
Class Performance: The model struggled significantly with the "judgemental" class (F1 = 0.071). Error analysis suggests this is due to the reliance on OCR text alone, which misses visual context, irony, and cultural tropes essential for identifying implicit sexism.
Cold-Start Behavior: The model exhibited a "cold-start" phenomenon, producing zero "judgemental" predictions in the first two epochs, indicating a difficulty in learning subtle patterns without extended training or specific augmentation.
Architecture Stability: Exploratory runs with larger models (Gemma 2 2B, DeBERTa-v3-large, XLM-RoBERTa-large) were less stable than the base model under the same hyperparameters, often diverging or converging to majority-class predictions.

Significance and Claims
The authors position this work as a demonstration that soft-label optimization via KL divergence effectively captures annotator disagreement in subjective NLP tasks, improving upon standard majority-vote approaches. The paper claims that combining KL divergence with weighted cross-entropy allows a lightweight, text-only model to balance distributional alignment with hard-label accuracy.

The authors maintain a modest stance regarding their findings:

Limitations: They acknowledge that the text-only approach inherently fails to capture visual-textual interactions central to meme interpretation, particularly for the "judgemental" class.
Scope: The stability issues with larger models are presented as exploratory observations rather than definitive conclusions about model scale, noting that a dedicated hyperparameter search would be required for generalization.
Ethics: The paper explicitly warns against deploying such systems for automated content moderation without human oversight due to risks of unjust censorship and the propagation of harmful content via false negatives.

In conclusion, the BioSentinel system validates the utility of the Le-Wi-Di paradigm and composite loss functions for sexism detection in memes, while highlighting the specific challenges of modeling implicit intent without multimodal context.