💬 NLP

Zero-Shot Stance Detection in the Wild: Dynamic Target Generation and Multi-Target Adaptation

This paper proposes a novel zero-shot stance detection task called DGTA that identifies multiple dynamic targets and their stances without prior knowledge, demonstrating that fine-tuned large language models, particularly through two-stage and integrated strategies, achieve superior performance on a newly constructed Chinese social media dataset.

Original authors: Aohua Li, Yuanshuo Zhang, Ge Gao, Bo Chen, Xiaobing Zhao

Published 2026-02-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Aohua Li, Yuanshuo Zhang, Ge Gao, Bo Chen, Xiaobing Zhao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are walking through a bustling, chaotic marketplace (like a social media feed). In this market, people are shouting opinions about everything: politicians, new phones, sports teams, and abstract ideas like "freedom" or "corporate greed."

The Problem: The "Blind" Detective
Traditional computer programs designed to understand these opinions (called "Stance Detection") are like detectives who only work when given a specific "Wanted Poster." If you tell them, "Find out what people think about Tesla," they can do a great job. But if someone walks up and says, "Tesla is great, but BYD is better, and NIO is awkward," the traditional detective freezes. They don't know to look for BYD or NIO because they weren't on the original list. They are stuck waiting for a pre-defined target.

The Solution: The "Wild" Detective
The authors of this paper propose a new kind of detective: the DGTA (Dynamic Target Generation and Multi-Target Adaptation) model. Instead of needing a "Wanted Poster," this detective is trained to walk into the wild marketplace, listen to the chatter, and say:

"Hey, I hear someone talking about Tesla."
"Oh, and they're also mentioning BYD."
"And NIO too."
"Now, let me figure out if they like or dislike each one."

This is called Zero-Shot Stance Detection in the Wild. The computer has to invent the list of targets on the fly and figure out the opinion for each one, all at once.

How They Built the Training Ground
To teach this new detective, the researchers couldn't just use old textbooks. They had to build a massive, new training ground using real Chinese social media posts (Weibo).

The Crowd: They collected over 125,000 posts from 240 different users.
The Clean-up: They scrubbed away emojis, links, and usernames to leave just the pure text.
The Teachers: They didn't just hire one human to label the data. They used a "committee" of three different AI models to label the posts. If two out of three agreed on what the target was and what the opinion was, they kept it. If they disagreed, they threw it out. Then, human experts double-checked the work.
The Result: A high-quality dataset of about 71,000 posts, covering everything from single-target rants to complex arguments involving three or four different topics in one sentence.

The Training Methods: One-Step vs. Two-Step
The researchers tried two different ways to train their "Large Language Models" (super-smart AI brains) to do this job:

The "All-in-One" Chef (Integrated Strategy): This model is told, "Read this text, find all the targets, and tell me the opinion for each, all in one go." It's like a chef who chops, cooks, and plates the meal simultaneously.
The Assembly Line (Two-Stage Strategy): This splits the job.
- Station 1: A model reads the text and just lists the targets (like a scanner reading barcodes).
- Station 2: A different model takes that list and the original text to decide if the opinion is positive, negative, or neutral. It's like having one person find the ingredients and a second person cook them.

The Results: Who Won?
They tested these models against older, simpler programs and models that just read instructions without being specially trained (prompted models).

The Winners: The specially trained (fine-tuned) AI models crushed the competition.
The "All-in-One" Winner: A model called DeepSeek-R1 (which has been trained to "think" before answering) became the best at figuring out the opinions. It got a score of 79.26% on accuracy.
The "Two-Step" Winner: A model called Qwen2.5 was the best at just finding the targets, scoring 66.99%.
The Lesson: Training the AI specifically for this messy, real-world job made it much better than just giving it a generic instruction. Also, models that were taught to "reason" (think step-by-step) did better than those that just guessed.

The Challenges (Where the Detective Stumbles)
Even the best models had trouble in specific situations:

Too Many Targets: When a sentence had three or four different topics, the models got confused and started mixing them up.
Hidden Meanings: If someone used sarcasm or a metaphor (like saying "Apple marrying a cowherd" to mean a bad partnership), the models sometimes missed the negative tone and thought it was neutral.
Vague Targets: If the target wasn't a clear name (like "Trump") but an abstract idea (like "the unfairness of the system"), the models struggled to pin it down exactly, though they were good at finding that something was being discussed.

In Summary
This paper introduces a new way to teach computers to understand social media arguments without needing a pre-made list of topics. By building a massive, carefully checked dataset and training AI models to either "think it all through" or "work in an assembly line," they created a system that can automatically spot who people are talking about and what they think of them, even in the chaotic, unpredictable "wild" of the internet.

Technical Summary: Zero-Shot Stance Detection in the Wild with Dynamic Target Generation and Multi-Target Adaptation (DGTA)

1. Problem Definition

Current stance detection research predominantly operates under a constrained setting where targets are predefined, static, and known prior to inference. However, real-world social media environments present a more complex reality: targets are neither predefined nor static. They are dynamic, often multiple per text, and can be implicit or abstract (e.g., specific events, actions, or conceptual states). Existing approaches, such as unsupervised target adaptation or single-target extraction tasks, rely on candidate target lists or fail to handle the simultaneous identification of multiple unknown targets within a single post.

To bridge this gap, the authors propose a novel task: Zero-Shot Stance Detection in the Wild with Dynamic Target Generation and Multi-Target Adaptation (DGTA). This task requires a model to:

Automatically identify all stance targets in a text without prior knowledge of the target label space.
Determine the stance (Support, Against, or Neutral) for each identified target.
Output a set of dynamic target-stance pairs, handling variable numbers of targets per instance.

2. Methodology

2.1 Dataset Construction

The authors constructed a high-quality, multi-domain Chinese social media stance detection dataset comprising 70,931 annotated samples.

Data Collection: Posts were collected from 240 diverse Weibo users, resulting in an initial pool of 125,176 entries.
Annotation Pipeline: A rigorous workflow was employed involving:
- Collaborative LLM Annotation: Three mainstream LLMs (GLM4-9B, Qwen2.5-7B, Llama3-8B) performed cascaded target identification and stance determination.
- Cross-Validation: Samples were only accepted if at least two models agreed on both the target entity and the stance.
- Scoring and Correction: A secondary model (DeepSeek-V3) scored valid samples, with low-scoring instances flagged for review.
- Human Verification: Eight professional annotators verified all automatically annotated samples.
- Cleaning: 36,379 low-quality texts (containing logical contradictions or ambiguous references) were removed.
Statistics: The dataset covers single-target, dual-target, triple-target, and multi-target (>3) scenarios, with a diverse distribution of stances.

2.2 Evaluation Metrics

Recognizing that traditional metrics fail for open-ended target generation, the authors designed a multi-dimensional evaluation framework:

Target Identification: A comprehensive score (C-Score) combining semantic similarity and surface matching:
$C\text{-Score} = (\alpha \times \text{BERTScore} + \beta \times \text{BLEU} + \gamma \times \text{ROUGE-L}) \times \text{Recall}$
(Weights: $\alpha=0.6, \beta=0.2, \gamma=0.2$ ).
Stance Detection: Evaluated only on samples where target identification metrics exceeded specific thresholds (e.g., C-Score $\ge 0.3$ ), using standard Precision, Recall, and F1 scores.

2.3 Fine-Tuning Strategies

The authors explored two distinct strategies for fine-tuning Large Language Models (LLMs) using LoRA:

Integrated Fine-Tuning: An end-to-end approach where the model receives an instruction to simultaneously identify targets and classify stances, outputting a sequence of target-stance pairs.
Two-Stage Fine-Tuning: A decoupled approach where:
- Stage 1: A model is fine-tuned solely for target identification.
- Stage 2: A separate model is fine-tuned to classify stances based on the text and the targets identified in Stage 1.

3. Key Contributions

Task Proposal: Defined the DGTA task, addressing the critical gap of zero-shot stance detection in open-world scenarios with dynamic, multi-target generation.
Dataset: Released the first high-quality Chinese multi-domain social media stance detection dataset specifically for this task, featuring rigorous multi-LLM and human verification.
Evaluation Framework: Introduced unified, comprehensive metrics (C-Score) that account for both the semantic accuracy of generated targets and the correctness of stance classification.
Methodological Baselines: Established strong baselines by comparing integrated and two-stage fine-tuning strategies against pre-trained models and instruction-prompted LLMs.

4. Experimental Results

Experiments were conducted on a stratified test set using various models, including fine-tuned pre-trained models (mT5, BERT, RoBERTa), instruction-prompted LLMs (GPT-4o, DeepSeek-V3, GLM4-9B, Llama3-8B), and fine-tuned LLMs (Qwen2.5-7B, DeepSeek-R1-Distill-Qwen-7B).

Key Findings:

Superiority of Fine-Tuned LLMs: Fine-tuned LLMs significantly outperformed both pre-trained models and zero-shot prompted LLMs.
- Target Identification: The two-stage fine-tuned Qwen2.5-7B achieved the highest comprehensive target recognition score (66.99%).
- Stance Detection: The integrated fine-tuned DeepSeek-R1-Distill-Qwen-7B achieved the highest stance detection F1 score (79.26%).
Strategy Comparison:
- The two-stage approach showed advantages in target identification, likely due to focused learning on entity extraction.
- The integrated approach excelled in stance detection, suggesting that modeling target-stance relationships jointly improves reasoning in complex semantic contexts.
Reasoning Capabilities: Models with reasoning capabilities (e.g., DeepSeek-R1) generally outperformed those without, achieving higher precision in identifying implicit targets and nuanced stances.
Chain-of-Thought (CoT): Introducing CoT in prompting improved performance across all tested LLMs, particularly for GLM4-9B (C-Score +7%) and Qwen2.5-7B (F1 +4%).
Target Complexity: Performance degraded as the number of targets increased. Dual-target texts yielded the best semantic alignment, while multi-target texts (>3) showed significant drops in C-Score due to semantic complexity and overlapping references.
Explicit vs. Implicit: Models performed significantly better on explicit targets. Implicit targets (abstract concepts) showed higher recall but lower precision, highlighting a challenge in semantic boundary definition.

5. Significance and Claims

The paper claims that the proposed DGTA framework and dataset provide a necessary foundation for advancing stance detection from controlled, predefined settings to the "wild" of real-world social media. By demonstrating that fine-tuned LLMs can effectively handle dynamic target generation and multi-target adaptation, the work establishes a robust baseline for future research. The authors emphasize that their approach addresses the limitations of previous methods which rely on predefined target lists, thereby enabling models to operate in scenarios where targets are unknown, implicit, and numerous. The availability of the dataset and models is intended to facilitate further exploration into open-world stance detection.