💬 NLP

Leveraging LaBSE with Progressive Curriculum Learning for Multicultural Polarization

This paper addresses the challenge of detecting multilingual and multicultural online polarization by proposing a novel architecture that leverages LaBSE embeddings for enhanced cross-lingual learning in low-resource languages and evaluates diverse Qwen encoder models within a retrieval-based prompting framework.

Original authors: Sachin Sundar, Sandeep Kumar, Mothish M

Published 2026-06-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sachin Sundar, Sandeep Kumar, Mothish M

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the internet as a massive, bustling global town square. In this square, people from every culture and language are chatting, arguing, and sharing opinions. Sometimes, these conversations turn toxic, with groups attacking each other based on politics, religion, race, or gender. The challenge for computer scientists is to build a "digital referee" that can spot this toxic polarization instantly, no matter what language is being spoken.

This paper, titled "PolarMind at SemEval-2026 Task 9," describes how a team of researchers built a smarter referee to handle this chaotic town square. Here is the story of how they did it, using simple analogies.

The Problem: The "Lost in Translation" Gap

The researchers faced a tricky problem: most computer brains (AI models) are very good at understanding English but struggle with "low-resource" languages (languages with less data available, like Amharic or Urdu).

Think of it like a teacher who is an expert in English literature but has never read a book in Urdu. If you ask the teacher to spot an angry argument in a Urdu text, they might miss it because they don't know the specific words or cultural context. The researchers needed a way to help the AI understand that a sentence might be harmless in English but highly inflammatory in another culture.

The Solution: The "Universal Translator" (LaBSE)

Instead of building a separate referee for every single language, the team used a special tool called LaBSE.

The Analogy: Imagine a universal translator that doesn't just translate words, but translates feelings. If someone says "I hate this" in English and "I despise this" in Hindi, LaBSE recognizes that these two sentences live in the same "emotional neighborhood."
The Innovation: The team took this translator and gave it a special upgrade. They didn't just look at the final sentence; they looked at the "layers" of understanding the AI had built, mixing them together like a chef blending spices to get the perfect flavor. They also created a Hybrid Pooling system:
- Mean Pooling: Taking a "big picture" average of the whole conversation.
- Attention Pooling: Zooming in on the specific, loud, angry words that really matter.
- The Result: By combining the big picture with the specific details, the AI became much better at spotting the difference between a normal comment and a polarized attack.

The Training Method: "The School Curriculum"

The team didn't just throw all the data at the AI at once. They used a strategy called Progressive Curriculum Learning.

The Analogy: Imagine teaching a student. You wouldn't start them with a PhD thesis on complex politics. You would start with easy stories, then medium-difficulty essays, and finally the hard stuff.
How it worked: The AI first learned from "easy" examples (clear-cut cases of polarization). Once it mastered those, it moved to "medium" and then "hard" examples. This helped the AI build a strong foundation before tackling the tricky, confusing cases.

The "Big Brain" Experiment: Testing LLMs

The researchers also tested a very powerful, modern AI called Qwen-2.5 (a Large Language Model). They tried two different ways to teach it:

The "Show and Tell" Method (Retrieval): They gave the AI a few examples of polarized text from different languages to look at before asking it to judge a new text.
The "Phonetic" Method: They tried converting text into sounds (phonemes), thinking that hearing the sound of the words might help the AI understand the culture better.

The Surprise: The "Big Brain" (Qwen) didn't perform as well as the specialized "Translator" (LaBSE). The researchers found that adding sound information (phonemes) actually confused the AI a bit. They suspect that for this specific task, the AI already understood the cultural nuances through the written text, and turning it into sound actually stripped away important visual clues (like emojis or specific script styles) that signaled anger.

The Results: A Winning Team

The team tested their system on 22 different languages.

The Score: Their custom "Translator" system beat the standard baseline models by a significant margin. In some difficult languages, their score jumped by 0.2 points (which is a huge deal in AI competitions).
The Ranking: They finished in the top 10 for five languages in the first challenge (detecting if something is polarized) and the top 5 for five languages in the second challenge (detecting what kind of polarization it is, like political or racial).

The Bottom Line

The paper concludes that for spotting online hate and division across many cultures, you don't necessarily need the biggest, most expensive AI. Instead, you need a smart, specialized system that:

Understands how different languages share the same "emotional DNA."
Learns in a step-by-step curriculum (easy to hard).
Knows how to balance looking at the whole picture while focusing on the most important words.

Their "PolarMind" system proved that with the right architecture, we can build digital referees that understand the world's diverse voices, not just the loudest ones.

Technical Summary: PolarMind at SemEval-2026 Task 9

Problem Statement

The paper addresses the critical challenge of detecting online polarization in multilingual and multicultural contexts, specifically within the scope of SemEval-2026 Task 9. The task involves identifying polarized speech across 22 languages, categorized into two subtasks:

Subtask 1: Binary classification (polarized vs. non-polarized).
Subtask 2: Multi-class classification of polarization types (Political, Racial, Gender, Ethnic, Other).

The authors highlight that polarization is culturally and linguistically dependent; a sentence may be neutral in one language (e.g., English) but polarized when translated to another (e.g., Urdu). A primary difficulty lies in the scarcity of training data for low-resource languages, which often leads to significant performance gaps between high-resource (e.g., English) and low-resource languages in state-of-the-art models.

Methodology

The authors propose PolarMind, a system centered on a customized LaBSE (Language-agnostic BERT Sentence Embedding) architecture, augmented by Proxy-Guided Curriculum Learning.

Core Architecture: LaBSE with Hybrid Pooling

Unlike standard multilingual encoders (e.g., mBERT, XLM-R), LaBSE is optimized to map semantically similar cross-lingual sentences into the exact same vector space. The authors hypothesize that this strict alignment allows the model to project scarce low-resource polarized phrases into robust semantic neighborhoods established by high-resource languages.

The architecture incorporates two key modifications:

Weighted Layer Aggregation: Instead of using only the final layer, the model aggregates the final four layers ( $L_{n-4}$ to $L_n$ ) using learnable parameters ( $w_1, w_2, w_3, w_4$ ). The weights for the earlier layers are initialized to be smaller ( $\sim 0.2w_1$ ) to prioritize deeper semantic features.
Hybrid Pooling: The final representation combines Global Mean Pooling (capturing overall semantic context) and Additive Attention Pooling (highlighting high-impact tokens).
- Mean Pooling: Calculates the arithmetic mean of hidden states.
- Attention Pooling: Uses an additive attention mechanism (Bahdanau et al., 2016) to assign weights to specific tokens.
- Fusion: The final embedding is $e_{hybrid} = e_{mean} + \lambda e_{attn}$ . For Subtask 2, independent attention mechanisms are maintained for each classification head to prevent conflation of different polarization types (e.g., gender vs. ethnicity).

Training Strategy: Proxy-Guided Curriculum Learning

To address data imbalance and difficulty variation, the authors employ a curriculum learning strategy:

Difficulty Ranking: A proxy model (XLM-RoBERTa-base) ranks the dataset based on a difficulty score $R(x) = \alpha \cdot P(y|x) + \beta \cdot \frac{1}{len(x)}$ , where $P(y|x)$ is the proxy confidence and the inverse length term promotes shorter, high-resource examples.
Training Phases: Models are trained in stages: easy samples first, followed by easy+medium, medium+hard, and finally the full dataset.

Ablation Study and Alternative Models

The paper evaluates the core LaBSE architecture against several alternatives:

RemBERT: Tested with and without continual pretraining on a corpus derived from Subtask 3 data to update the model's knowledge of recent events.
EuroBERT: A state-of-the-art encoder optimized for European languages with a more recent knowledge cutoff.
Qwen-2.5-14B (LLM): Evaluated using a retrieval-based prompting framework.
- Retrieval: LaBSE retrieves top- $k$ semantically similar sentences from training data.
- Diversity: Maximal Marginal Relevance (MMR) ensures retrieved examples are both relevant to the query and diverse from each other.
- Prompting: Few-shot prompting (7 shots) was tested with raw text and text augmented with phonemes (using the Epitran library) to analyze if phonetic information aids low-resource performance.

Key Contributions

LaBSE-based Architecture: Introduction of a weighted layer aggregation and hybrid pooling mechanism specifically designed for cross-lingual polarization detection, leveraging LaBSE's strict cross-lingual alignment.
Proxy-Guided Curriculum Learning: A strategy to mitigate multilingual data imbalance by training on data sorted by difficulty.
Comprehensive Model Evaluation: A comparative analysis of encoder models (LaBSE, RemBERT, EuroBERT) and Large Language Models (Qwen-2.5) within a retrieval-based framework.
Empirical Analysis of LLM Limitations: Investigation into the efficacy of phoneme-based prompting and few-shot learning for low-resource polarization tasks.

Results

The system was evaluated on the development set and the official test set across 22 languages.

Subtask 1 (Binary Classification):
- The proposed LaBSE architecture with hybrid pooling consistently outperformed the official POLAR baseline (fine-tuned LaBSE-base) by an average of 0.04 Macro F1 across all 22 languages.
- In low-resource languages, the system achieved competitive results, securing top-10 rankings in five languages. For instance, in Amharic (amh), the system achieved a score of 0.771, a 5.6% improvement over the baseline.
- Continual pretraining on RemBERT yielded only marginal gains (+0.005 Macro F1), suggesting the hybrid pooling strategy offered a more significant boost than model scaling alone.
- The Qwen-2.5 decoder model did not outperform the encoder models, likely due to tokenization mismatches and weaker cross-lingual alignment compared to LaBSE.
Subtask 2 (Type Detection):
- The LaBSE-based model again outperformed RemBERT and EuroBERT.
- Language-specific threshold calibration improved the average Macro-F1 on the development set from 0.5216 to 0.5441 (+2.25%), highlighting the sensitivity of multi-label detection to threshold selection.
- Qwen-2.5 (7-shot) performed on par with the encoder model, leveraging prior knowledge to generalize sparse patterns (e.g., gender/ethnicity) where the encoder struggled with limited samples.
- Phoneme Prompting: The addition of phoneme information (text + IPA) did not yield significant improvements. The authors hypothesize that for this task, phonetic layers offer redundant information, and the IPA transformation may inadvertently filter out emojis or script-specific markers that carry significant polarized weight.

Significance and Claims

The paper claims that the optimal approach to multicultural polarization detection involves specialized systems that account for linguistic and cultural nuances, rather than relying solely on general-purpose models.

The primary significance of the work lies in demonstrating that LaBSE, typically reserved for retrieval tasks, can be effectively repurposed for classification in low-resource settings. By combining LaBSE's strict cross-lingual alignment with hybrid pooling and curriculum learning, the system bridges the performance gap between high-resource and low-resource languages, achieving up to a 0.2 macro F1 improvement in specific low-resource scenarios compared to standard baselines.

The authors modestly conclude that while LLMs show promise via few-shot learning, they currently face limitations in cross-lingual alignment and tokenization for this specific task, and that phonetic augmentation does not currently provide a conclusive advantage for polarization detection. The work establishes a robust baseline for future research in multilingual, multicultural NLP.