Compact Prompting in Instruction-tuned LLMs for Joint Argumentative Component Detection

1. Problem Definition

The paper addresses Argumentative Component Detection (ACD), a core subtask of Argumentation Mining (AM). The goal of ACD is to simultaneously:

1. Delimit argumentative spans within raw text (identifying where an argument begins and ends).
2. Classify these spans into specific components, primarily Claims (positions/opinions) and Premises (supporting evidence/reasoning).

Current Limitations:

• Pipeline Approaches: Most existing methods treat ACD as a multi-stage pipeline (segmentation followed by classification) or a simplified sequence labeling problem (e.g., BIO tagging) applied to pre-segmented text.
• Dependency on Pre-segmentation: These approaches often assume that argumentative boundaries are already known or explicitly marked, which is rarely the case in real-world, unstructured discourse.
• Error Propagation: Separating segmentation and classification leads to cumulative errors; if the boundary is detected incorrectly, the classification is inevitably wrong.

2. Methodology

The authors propose a novel Generative Framework that reframes ACD as a single, unified text-to-structure generation task using instruction-tuned Large Language Models (LLMs).

A. Task Formulation

Instead of token-level classification, the model is tasked with reproducing the original plain text while inserting explicit XML-style tags to demarcate components.

• Input: Raw, unsegmented text.
• Output: The same text with <claim>...</claim> and <premise>...</premise> tags inserted at the correct boundaries.
• Goal: Jointly learn boundary detection and component classification in a single forward pass.

B. Data Preparation

• Datasets: Three diverse benchmarks were used:
  1. USElecDeb60To16: Spoken political debates (highly implicit, conversational).
  2. Persuasive Essays (PE): Structured written essays (explicit discourse markers).
  3. Web Discourse: Informal, noisy online comments.
• Conversion: Original datasets annotated with BIO tags were converted into XML-tagged text to serve as input-output pairs for instruction tuning.

C. Models and Training

• Models: The study fine-tuned several open-weight instruction-tuned LLMs (GPT-2-XL, OPT-1.3B/6.7B, Mistral-7B, Llama-3-8B-Instruct).
• Baselines: Compared against traditional encoder-based models (RoBERTa, DeBERTa) trained on standard BIO sequence labeling tasks.
• Prompting Strategy: The approach uses compact, instruction-based prompts designed to guide the model to perform both segmentation and classification simultaneously without relying on pre-segmented components.
• Decoding: To ensure determinism, decoding was configured with a very low temperature (0.01) and restrictive nucleus sampling (top-p = 0.1).

3. Key Contributions

1. Reconceptualization of ACD: This is one of the first works to fully model ACD as a generative task rather than a classification or sequence labeling problem. It eliminates the need for pre-segmented inputs.
2. Joint Modeling: By treating delimitation and classification as a single generation process, the approach avoids error propagation inherent in multi-stage pipelines.
3. Compact Prompting: Demonstrates that concise, instruction-based prompts are sufficient to guide LLMs to perform complex structural extraction tasks effectively.
4. Open-Weight Focus: The study relies exclusively on open-weight models to ensure reproducibility and transparency, avoiding proprietary black-box APIs.

4. Experimental Results

Experiments were conducted on the Persuasive Essays (PE) dataset and a merged dataset (combining all three sources).

• Performance on PE Dataset:

  • The fine-tuned Llama-3-8B model achieved a Macro-F1 score of 0.8778.
  • This outperformed all state-of-the-art baselines, including the strongest feature-engineered CRF model (0.8670) and encoder-based transformers (RoBERTa: 0.6933, DeBERTa: 0.7112).
  • The result is within 0.0082 of the human upper bound (0.8860), indicating near-human performance.

• Performance on Merged Dataset:

  • Performance dropped due to increased heterogeneity (from 0.87 to ~0.78), but the generative models (specifically OPT-6.7B at 0.7822) still significantly outperformed encoder-based baselines (which dropped to ~0.48–0.49).
  • This suggests generative models are more robust to cross-domain variability and diverse discourse styles.

• Qualitative Findings:

  • Argument Refinement: Models sometimes corrected ambiguous gold labels with more contextually appropriate classifications.
  • Discovery: Models identified implicit argumentative components that were missing from the original annotations.
  • Hallucination: A minor limitation where models occasionally altered the original text (e.g., changing "doing this" to "doing this job") despite instructions to preserve verbatim text, leading to span misalignment.

5. Significance and Future Directions

• Paradigm Shift: The paper challenges the traditional "segment-then-classify" pipeline in Argumentation Mining, proving that generative LLMs can handle complex structural prediction tasks end-to-end.
• Robustness: The approach demonstrates superior generalization across different text genres (spoken debates vs. essays vs. web comments) compared to traditional sequence labeling models.
• Limitations & Future Work:
  • Hallucination: The tendency of LLMs to modify input text remains a challenge for strict span alignment metrics. Future work may require constrained decoding or hybrid extractive-generative approaches.
  • Scope: The current work focuses only on Claims and Premises. Future research should extend this generative framework to include argument relation identification, stance detection, and full argument graph construction.
  • Ethical Considerations: The authors note that argument boundaries are inherently interpretative; models may amplify annotation biases or present subjective analyses as objective facts.

In conclusion, this work establishes that instruction-tuned LLMs with compact prompting offer a powerful, flexible, and high-performing alternative to traditional methods for Argumentative Component Detection, effectively bridging the gap between raw text and structured argumentative analysis.