Technical Summary: KIRP - Zero-Shot Tweet-Level Stance Detection

Problem Definition

Zero-shot tweet-level stance detection (ZSSD) aims to identify a text's attitude toward a specific target (Favor, Against, Neutral, or Irrelevant) without task-specific training data. The paper identifies three primary challenges in this domain:

1. Context Sparsity: Tweets are short and often lack sufficient contextual information.
2. Implicit Targets: Texts frequently contain implicit targets that are difficult to link to the stance without external knowledge.
3. Label Differentiation: Current models struggle to distinguish between "Neutral" (relevant but no clear stance) and "Irrelevant" (unrelated to the target), a critical distinction in four-class classification.

Existing methods often rely heavily on external knowledge injection but neglect intrinsic semantic cues within intra-textual entities. Furthermore, they exhibit limited reasoning capabilities to validate the relevance of unseen targets, leading to poor differentiation between neutral and irrelevant labels.

Methodology: The KIRP Framework

The authors propose KIRP (Knowledge-enhanced, Iterative, Reflective Prompting), a framework integrating external knowledge, data augmentation, and reflective reasoning. The architecture consists of three core components:

1. Entity Reorganization and Augmentation

To address context sparsity and implicit targets, the framework utilizes external knowledge sources (Wikipedia and news reports) to construct a knowledge graph ($G$).

• Subgraph Extraction: For each input, a localized subgraph ($G'$) is extracted based on $k$-hop neighbors of entities mentioned in the text.
• Graph Autoencoder: A Graph Attention Network (GAT) encodes entities, and a DistMult scoring function reconstructs potential missing links. This process identifies equivalent entities (e.g., "golf course" and "football field" sharing the relation instance_of with sports facility).
• Augmentation: The framework substitutes original key entities in the text with these equivalent entities. This generates semantically diverse but structurally aligned $\langle \text{text}, \text{target} \rangle$ pairs, effectively expanding the training data and making implicit targets explicit.

2. Reflective Chain-of-Thought (CoT) Prompting

The framework employs a three-stage prompting chain to enhance reasoning and generate rich contextual semantics:

• Stage 1 (Criteria Generation): The Large Language Model (LLM) generates hypothetical explanations for the text's stance toward the target under four hypothetical labels (Favor, Against, Neutral, Irrelevant), even if the explanation is initially incorrect.
• Stage 2 (Reflection): The LLM reflects on the generated explanations to identify logical flaws and judge the validity of the hypothetical stance.
• Stage 3 (Correction): The LLM proposes modifications to the text or target to make the hypothetical stance valid.
• Output: These stages produce three text outputs ($T_{cri}$, $T_{ref}$, $T_{cor}$). $T_{cri}$ serves as supplementary criteria added to the original text, while $T_{ref}$ and $T_{cor}$ help generate the final stance label ($SL$) used for training the smaller model. This process provides rich logical reasoning cues for short texts.

3. Stance Contrastive Learning and Iterative Prototype Network

• Stance Contrastive Learning: The model concatenates the augmented data with the LLM-generated arguments and encodes them using BERT. A contrastive loss function is applied to pull samples of the same stance closer and push different stances apart in the feature space, optimizing the representation for discriminative features.
• Iterative Prototype Network: To solve the four-class classification problem (specifically distinguishing "Irrelevant" and "Neutral"), the framework uses a three-layer iterative prototypical network:
  1. Layer 1 (FAN/I): Distinguishes between topic-relevant (Favor, Against, Neutral) and Irrelevant samples.
  2. Layer 2 (FA/N): Distinguishes between samples with stance polarity (Favor, Against) and Neutral samples.
  3. Layer 3 (F/A): Distinguishes between Favor and Against stances.
     This hierarchical approach simplifies the classification task and addresses the difficulty of separating "Irrelevant" from "Neutral" in a single step.

Key Contributions

1. KIRP-D Dataset: The construction of KIRP-D, the first Japanese tweet-level dataset for zero-shot stance detection. It is a four-class, multi-topic dataset covering social, political, diplomatic, and economic domains, designed to bridge the gap in multilingual stance detection research.
2. Entity Reorganization Augmentation: A novel data augmentation method that aligns textual entities with equivalent knowledge from external sources to generate diverse target representations, uncovering implicit targets and expanding the LLM's reasoning scope.
3. Reflective CoT Framework: The integration of a three-stage reflective prompting chain (Criteria, Reflection, Correction) with a prototypical network. This combines knowledge injection with reorganization-based augmentation to generate rich semantic representations and optimize the feature space via stance-aware contrastive learning.
4. State-of-the-Art Performance: Extensive validation demonstrating that KIRP achieves superior performance across multiple benchmarks, particularly in distinguishing "Neutral" from "Irrelevant" and handling cross-lingual, data-scarce scenarios.

Experimental Results

The model was evaluated on three datasets: SemEval-2016 (English, 3-class), WT-WT (English, 4-class), and the new KIRP-D (Japanese, 4-class).

• Overall Performance: KIRP achieved state-of-the-art (SOTA) results across all datasets.
  • SemEval-2016: 84.05% F1-score (3-class).
  • WT-WT: 84.99% F1-score (4-class).
  • KIRP-D: 79.18% F1-score (4-class).
• Comparison: KIRP outperformed the best LLM-based baselines (e.g., DeepSeek-V4-ToC) and traditional contrastive learning models (e.g., JointCL). The performance gap was most significant on the Japanese KIRP-D dataset (4.68 points higher than the best LLM baseline), highlighting the framework's effectiveness in cross-lingual, data-scarce environments.
• Ablation Studies:
  • Removing the LLM prompting chain ($M\setminus LLM$) caused the most significant performance drop, confirming the critical role of deep semantic reasoning.
  • Removing the prototypical network ($M\setminus PN$) significantly reduced the ability to distinguish "Irrelevant" from "Neutral."
  • The three-stage prompting chain (Criteria, Reflection, Correction) was shown to be superior to standard CoT, with the Criteria generation stage being the most impactful.
• Robustness: KIRP demonstrated strong robustness against adversarial perturbations and topic drift, maintaining high F1 scores during cross-target transfer tasks compared to other baselines.

Significance and Claims

The paper claims that KIRP provides a comprehensive solution to the challenges of zero-shot tweet-level stance detection. Its significance lies in:

• Addressing Data Scarcity: By leveraging external knowledge and LLM reasoning, the framework effectively handles the lack of labeled data and context in short texts.
• Refining Label Granularity: The iterative prototype network specifically targets the long-standing difficulty of differentiating "Neutral" and "Irrelevant" labels, a limitation in many existing three-class datasets and models.
• Multilingual Expansion: The introduction of KIRP-D breaks the dominance of English and Chinese datasets, providing a critical resource for Japanese stance detection research.
• Generalizability: The framework demonstrates robust cross-lingual and cross-domain generalization, proving that integrating knowledge-based augmentation with reflective reasoning yields more robust stance detection than pure LLM prompting or traditional feature-based methods alone.