Technical Summary: DimStance: Multilingual Datasets for Dimensional Stance Analysis

Problem Statement

Traditional stance detection is a well-established task that classifies an author's attitude toward a specific target into discrete categorical labels (e.g., Favor, Against, Neutral). However, this categorical approach fails to capture the nuanced affective states underlying stance expressions. Stances are rarely purely "for" or "against"; even expressions sharing the same categorical label can vary significantly in emotional tone and intensity. Existing resources lack human-annotated dimensional representations, limiting the ability to analyze why opinions harden or to mitigate polarization in debates regarding complex issues like climate change and politics. Furthermore, there is a scarcity of stance resources for low-resource languages.

Methodology

The authors introduce DIMSTANCE, the first dimensional stance resource annotated with continuous Valence-Arousal (VA) scores. The methodology encompasses dataset construction, task formulation, and model evaluation.

Dataset Construction

• Scope: The dataset comprises 11,746 target aspects across 7,365 texts in five languages: English, German, Chinese, Nigerian Pidgin, and Swahili.
• Domains: Data covers politics and environmental protection.
• Sources:
  • English: EZ-STANCE and Reddit (Environmental Protection).
  • German: Wahl-O-Mat Archive (Politics).
  • Chinese: Threads platform (Environmental Protection).
  • Nigerian Pidgin & Swahili: X platform (Politics).
• Annotation Process:
  • Target aspects were extracted using LLMs and verified via majority vote by five native speakers per language.
  • Annotators rated each target-text pair on two scales: Valence (1 = highly negative to 9 = highly positive) and Arousal (1 = highly unaroused to 9 = highly aroused).
  • Final gold labels are the average of five annotators' ratings.
• Quality Control: Reliability was measured using Root Mean Squared Error (RMSE). Results indicate that arousal annotations are generally less reliable than valence annotations, consistent with prior studies.

Task Formulation

The paper introduces Dimensional Stance Regression, a task requiring systems to predict continuous VA scores for a given target aspect within a text, rather than selecting a discrete class.

Experimental Setup

The authors evaluated three groups of models:

1. PLM Regressors: Multilingual pre-trained models (RemBERT, LaBSE, XLM-R) with a regression head, optimized via Mean Squared Error (MSE).
2. Closed LLMs (Prompting): Zero-shot and few-shot prompting of closed-source models (GPT-5 mini, Gemini 2.5 Flash, Kimi K2) to generate numerical scores.
3. LLM Regressors (Fine-tuning): Open-source LLMs (Mistral-3, Phi-4, Llama-3.3, Qwen-2.5) adapted via 4-bit QLoRA with a regression head.

Performance was measured using RMSE in the VA space, calculated as the Euclidean distance between predicted and gold (V, A) pairs.

Key Results

Dataset Analysis

• VA Distribution: Across all languages, the data exhibits a U-shaped relationship between valence and arousal; arousal tends to be lowest near neutral valence and increases toward both negative and positive extremes.
• Cross-Lingual Variations:
  • English (Env. Protection): Balanced valence distribution; high arousal associated with both positive and negative stances.
  • German (Politics): Skewed toward positive valence (69%), likely reflecting party-generated content.
  • Chinese (Env. Protection): Compact distribution with mild expressions; targets often cluster around mid-range valence.
  • Nigerian Pidgin & Swahili (Politics): Dominated by negative valence (approx. 63-64%) and high arousal, reflecting the contentious nature of electoral discourse. Swahili data contains a notable number of toxic examples and exhibits the highest mean arousal.

Model Performance

• Fine-tuned LLMs: Generally outperformed both prompted LLMs and PLM regressors, particularly larger 70B-scale models.
• Prompted LLMs: Achieved competitive results, serving as a data-efficient alternative. However, they produced "grid-like" (discrete) VA distributions due to token-based generation, which struggled with compact distributions (e.g., Chinese).
• Language Performance:
  • Chinese: Consistently achieved the best performance across models, attributed to its concentrated VA distribution around the neutral region.
  • Swahili: Showed the lowest performance, highlighting challenges in low-resource languages.
  • Nigerian Pidgin: Performed better than Swahili, possibly due to its English-like characteristics.
• Prompt Sensitivity: The performance of closed LLMs was highly sensitive to prompt phrasing. Increasing the number of few-shot examples (up to 16 shots) generally improved performance, with Kimi K2 showing the strongest overall results among prompted models.

Significance and Contributions

The paper claims the following contributions:

1. First Dimensional Resource: The introduction of DIMSTANCE, the first multilingual dataset for stance analysis with human-annotated continuous VA scores, covering both high- and low-resource languages.
2. New Task Framework: The formulation of dimensional stance regression, integrating affective science frameworks into stance detection to enable fine-grained analysis.
3. Benchmarking: Provision of baseline results across PLM regressors, prompted LLMs, and fine-tuned LLM regressors, establishing a foundation for future research.

The authors emphasize that this resource provides a pivotal foundation for multilingual, emotion-aware stance analysis. By moving beyond categorical labels, the work facilitates a deeper understanding of the affective states driving public opinion. However, the paper modestly notes limitations, including the inherent difficulty of annotating arousal, potential cultural variations in VA interpretation, and the risk of bias in datasets with skewed valence distributions (e.g., Nigerian Pidgin). The authors strictly prohibit commercial or government use of the data to prevent potential misuse, such as the persecution of individuals based on inferred stances.