Technical Summary: StanceNakba 2026 Shared Task

Problem Definition

The StanceNakba 2026 shared task addresses the challenge of automatically detecting political stances in polarized social media discourse surrounding the Palestinian-Israeli conflict. While stance detection (identifying whether an author is in favor of, against, or neutral toward a target) is a well-established NLP task, existing resources and benchmarks have largely focused on English and politically neutral topics. This task aims to fill the gap in conflict-related, multilingual, and highly polarized discourse analysis.

The task is structured around two distinct subtasks designed to investigate different dimensions of political alignment:

• Subtask A (Actor-Level Stance Detection): An English-language task requiring models to infer an author's overarching political orientation toward the conflict as a whole. The goal is to classify posts into three categories: Pro-Palestine, Pro-Israel, or Neutral. This formulation assumes that political orientation is often expressed indirectly through framing choices rather than explicit opinion markers.
• Subtask B (Cross-Topic Stance Detection): An Arabic-language task requiring models to detect stances toward two specific, conflict-related policy topics: "Normalization with Israel" and "Refugee/Immigrant Presence in Jordan." The goal is to classify posts into Favor, Against, or Neither. This subtask tests the ability of models to generalize stance representations across different topics within the same conflict domain.

Methodology and Dataset

The shared task is built upon an annotated dataset of 2,606 social media posts, split between English and Arabic.

Dataset Construction:

• Subtask A (English): Comprises 1,401 posts collected from X (formerly Twitter). Posts were retrieved using specific keyword queries (e.g., "I stand with Palestine" for Pro-Palestine) and balanced to 467 samples per class. A rigorous preprocessing pipeline removed label-leaking keywords (e.g., the seed phrases used for retrieval), URLs, mentions, and emojis, retaining only alphanumeric text. The data was split into training (70%), development (15%), and test (15%) sets.
• Subtask B (Arabic): Utilizes a subset of the MARASTA corpus, focusing on 1,205 samples across two topics. The data covers multiple Arabic dialects (Gulf, Levantine) and Modern Standard Arabic (MSA). Annotations were performed by multiple annotators with high inter-annotator agreement (Cohen's Kappa of 0.84). The dataset is balanced across the three stance classes for each topic.

Participating Systems:
A total of 7 teams participated in Subtask A and 6 teams in Subtask B. The dominant methodology across all participating systems was the fine-tuning of pre-trained transformer-based models.

• Model Architectures: The most common backbones included Arabic-specific models (MARBERT, ARBERT, CAMeLBERT) and multilingual models (DeBERTa-v3, XLM-RoBERTa, mDeBERTa).
• Techniques: Teams employed various strategies to handle class imbalance and improve generalization, including:
  • Ensemble Methods: Combining predictions from multiple models or cross-validation folds.
  • Loss Functions: Weighted cross-entropy, label smoothing, and contrastive learning (e.g., Prototype Contrastive Learning).
  • Input Formulations: Topic-conditioned inputs, Natural Language Inference (NLI) reformulation, and "Statement Tuning" (using cloze templates with [MASK] tokens).
  • Data Augmentation: Back-translation and LLM-generated paraphrasing (specifically for Subtask B).
  • Regularization: R-Drop consistency regularization and dropout adjustments.

Key Results

The evaluation metric for both subtasks was the Macro F1 score, which treats all classes equally, thereby penalizing poor performance on the often underrepresented "Neutral" or "Neither" classes.

• Subtask A (Actor-Level): The top-performing system, Shroukgbr, achieved a Macro F1 of 0.9620. This team utilized a single BERT-based transformer with weighted cross-entropy loss and stratified 5-fold cross-validation. The second and third places were achieved by teams using MARBERT/ARBERT and a Mixture-of-Experts (MoE) BERT architecture, respectively.
• Subtask B (Cross-Topic): The top-performing system, Viva_Palestine, achieved a Macro F1 of 0.8724. This system fine-tuned MARBERT on a merged training and validation set with topic-sentence pair inputs. The second and third places were achieved by systems using AraBERT and MARBERTv2 with dialect-aware data augmentation.

The results indicate that transformer-based approaches are highly effective for conflict-domain stance detection. However, the lower scores in Subtask B compared to Subtask A highlight the persistent challenges in cross-topic generalization and the difficulty of predicting the neutral class.

Significance and Contributions

The paper positions StanceNakba 2026 as a significant step forward in Arabic NLP and political discourse analysis by:

1. Introducing a Dual-Framework: Distinguishing between actor-level political alignment (Subtask A) and cross-topic stance patterns (Subtask B), allowing for a more nuanced investigation of how general alignments relate to specific policy issues.
2. Providing a Conflict-Aware Resource: Offering a high-quality, annotated dataset specifically for the Palestinian-Israeli conflict, addressing the scarcity of resources for polarized, multilingual discourse compared to English-centric benchmarks.
3. Establishing Baselines: Demonstrating that fine-tuned Arabic and multilingual transformers (particularly MARBERT and DeBERTa variants) serve as strong baselines for this domain, while also revealing the limitations of current models in handling cross-topic shifts and neutral stance prediction.

The authors conclude that while the task has attracted broad participation and demonstrated the feasibility of large-scale Arabic stance detection, future work must address the challenges of class imbalance, dialectal variation, and the inherent ambiguity of the neutral stance class. The paper also emphasizes the ethical necessity of using these tools for scientific analysis rather than surveillance or political profiling.