Technical Summary: SMART-D

Problem Statement

Stance detection involves identifying an author's position (e.g., favor, against, none) toward a specific target, such as a public figure, policy, or ideological concept. This task is particularly challenging in short-form online texts where stance is often conveyed implicitly through sarcasm, rhetorical framing, indirect references, or attacks on related entities rather than explicit statements. While Large Language Models (LLMs) demonstrate strong few-shot and zero-shot performance, single-pass prompting often yields brittle predictions when multiple interpretations are plausible. Existing aggregation strategies, such as majority voting or self-consistency, improve robustness by combining final labels but discard the intermediate reasoning necessary to resolve conflicting interpretations. Furthermore, many current multi-agent or reasoning-oriented approaches utilize fixed reasoning structures that do not adapt the computational effort to the complexity of individual instances.

Methodology: SMART-D

The authors propose SMART-D (Structured Multi-Agent Reasoning for Dynamic allocation), a framework that shifts aggregation from the label level to the reasoning level. The system employs a Manager–Worker architecture designed to synthesize intermediate explanations rather than simply voting on final outputs.

Core Components

1. Manager Agent:

   • Planning Stage: Upon receiving an input text ($x$), a target ($t$), and a dataset identifier ($d$), the Manager estimates the reasoning complexity of the instance. It dynamically selects a variable number of Workers ($N \in \{3, \dots, 7\}$) and assigns each a distinct analytical role (e.g., analyzing explicit cues, sarcasm/irony, emotional tone, indirect stance via related entities, or policy framing).
   • Decision Stage: The Manager receives reasoning-only explanations from the Workers. It synthesizes these explanations by comparing their consistency, relevance, and evidential support to resolve conflicts and produce a single final stance label.

2. Worker Agents:

   • Each Worker analyzes the same input from its assigned complementary perspective.
   • Constraint: Workers are explicitly prohibited from emitting stance labels. They must produce short, evidence-grounded reasoning explanations only. This prevents premature commitment to a label and preserves intermediate evidence for the Manager's synthesis.

3. Dynamic Worker Allocation:

   • Instead of using a fixed number of reasoning paths for all inputs, SMART-D treats reasoning depth as an adaptive resource.
   • The Manager assigns more Workers to ambiguous, sarcastic, or rhetorically complex cases and fewer Workers to instances with explicit stance cues. This aims to balance reasoning diversity against aggregation noise and computational cost.

Implementation

• Backbones: The framework was evaluated using three LLMs: Gemini-3-Flash-preview, Llama-3.3 70B, and Mistral-3.2 24B. In all experiments, the Manager and Workers utilized the same backbone model, differing only in role-specific prompts and decoding temperatures (Workers: $T=0.7$ for diversity; Manager: $T=0.3$ for stability).
• Setting: All experiments were conducted in a zero-shot prompting setting without task-specific fine-tuning.

Experimental Results

The framework was evaluated on three benchmarks: SemEval-2016, P-Stance, and COVID-19 Stance.

• COVID-19 Stance: SMART-D achieved the most significant gains on this dataset, which contains many implicit and context-dependent expressions.
  • Using Gemini, SMART-D achieved a Macro-F1 of 86.07, outperforming the single-model baseline (+3.99 points) and strong supervised baselines by over 6 points.
  • Using Llama, the framework improved the average Macro-F1 from 69.78 to 81.31, notably resolving performance drops on the "Stay-at-Home" target where the baseline struggled with indirect policy references.
• SemEval-2016: SMART-D improved performance across all backbones, with Llama reaching 82.90 Macro-F1 (a +2.67 point improvement over its baseline).
• P-Stance: Results were more model-dependent. While Mistral saw a substantial improvement (81.86 to 85.48), Gemini and Llama showed slight decreases relative to their single-model baselines. This suggests that additional reasoning is less beneficial when stance cues are already explicit.

Ablation Studies

• Reasoning-Level Synthesis: The primary performance gain came from Worker Reasoning (WR). Aggregating intermediate explanations significantly outperformed aggregating final labels (Worker Voting). Adding Manager Reasoning (MR) provided smaller, incremental gains.
• Dynamic vs. Fixed Allocation: Performance did not increase monotonically with the number of Workers. Dynamic allocation provided a strong compromise between performance and cost, particularly on the complex COVID-19 dataset.
• Cross-Model Workers: Mixing different LLM backbones for Workers did not consistently improve performance. Stronger Managers (Gemini, Llama) performed better with homogeneous Worker pools, while the weaker Mistral backbone benefited from heterogeneous Workers.

Key Contributions

1. Framework Introduction: The authors introduce SMART-D, a Manager–Worker framework that shifts stance aggregation from final-label voting to reasoning-level synthesis.
2. Adaptive Mechanism: The proposal of dynamic Worker allocation, allowing the model to adapt reasoning depth to input complexity rather than applying a fixed budget to all instances.
3. Empirical Evidence: Comprehensive evaluation across three benchmarks and three LLM backbones demonstrating that adaptive reasoning-level aggregation is most beneficial for implicit, ambiguous, and rhetorically complex stance cases.

Significance and Claims

The paper claims that adaptive reasoning-level aggregation is most effective when stance cannot be reliably inferred from surface cues alone. The results suggest that while single-pass prompting is brittle for implicit stance, structured multi-agent reasoning can resolve conflicting interpretations by synthesizing complementary perspectives.

The authors position SMART-D not as a uniformly superior inference strategy for all instances, but as an adaptive framework specifically designed for complex stance cases. They note that on datasets with explicit cues (like P-Stance), the benefits are model-dependent and sometimes negligible. The work highlights that the value of additional reasoning lies in its ability to handle rhetorical framing, indirect references, and target-specific interpretations, rather than simply increasing computational budget indiscriminately.

Limitations

The authors acknowledge several limitations:

• Experiments are restricted to English-language Twitter/X datasets; effectiveness on other languages or platforms is untested.
• The multi-agent architecture introduces computational overhead, though dynamic allocation mitigates this by avoiding unnecessary reasoning for simple cases.
• Benefits are condition-dependent, with the most pronounced gains on datasets with implicit stance (COVID-19) and more modest gains on explicit datasets.
• Potential biases in social media data may be reflected in the models.