Technical Summary: Are LLMs Effective Negotiators?

Problem Statement

Successful negotiation is a complex cognitive process requiring a synthesis of conversation comprehension, Theory-of-Mind (ToM) capabilities to infer partner motives, strategic reasoning, and effective communication. While Large Language Models (LLMs) have demonstrated remarkable performance in various NLP tasks, their specific capabilities within the domain of automated negotiation remain unexplored. Prior research utilizing LLMs as negotiation agents lacks a systematic evaluation of the diverse abilities required for effective negotiation. This gap hinders the development of advanced AI negotiators and limits the application of LLMs in designing dialogue systems, providing pedagogical feedback, and scaling data collection practices.

Methodology

The authors propose a comprehensive framework to systematically analyze the multifaceted capabilities of LLMs across 35 distinct tasks derived from four human-human negotiation datasets (CRA, DND, CA, and JI). These datasets are based on the Multi-Issue Bargaining Task (MIBT) framework, where agents negotiate to divide fixed quantities of items to maximize total value based on predefined preferences.

The evaluation framework categorizes tasks along three axes:

1. Objectivity: Distinguishing between objective tasks (fact-based, e.g., item counts, deal specifics) and subjective tasks (involving human perception, e.g., satisfaction, likability).
2. Time Stage: Assessing capabilities at the Start (context only), During (partial dialogue), and End (full dialogue) of a negotiation.
3. Task Type:
   • Comprehension: Understanding context, outcomes, and priorities.
   • Annotation: Identifying dialogue acts and linguistic strategies.
   • Partner Modeling: Inferring the partner's priorities and subjective states (ToM).
   • Generation: Producing contextually appropriate and strategically advantageous responses.

The study evaluates a range of proprietary (GPT-3.5, GPT-4) and open-source models (Mistral-7B, Wizard-13B, Vicuna-13B/33B) using zero-shot prompting. Baselines include a Majority-label voting model and a task-specific fine-tuned Flan-T5 (base). Evaluation metrics include Accuracy, Macro-F1, Pearson Correlation Coefficient (PCC) for subjective tasks, and BLEU/ROUGE/BERTScore alongside human expert ratings for Coherence and Strategy in generation tasks.

Key Contributions

1. Novel Evaluation Methodology: The paper introduces a structured methodology to decompose negotiation into 35 specific tasks, enabling a granular analysis of LLM capabilities beyond end-to-end performance.
2. Systematic Zero-Shot Evaluation: The authors evaluate out-of-the-box LLM capabilities across diverse scenarios, revealing that GPT-4 often outperforms even fine-tuned baselines (Flan-T5) in zero-shot settings.
3. Human-LLM Comparison: Through expert human annotation, the study compares model generation against human references, identifying specific areas of parity and failure.
4. Prompting Strategy Analysis: The work investigates the efficacy of Chain-of-Thought (CoT) and few-shot prompting, as well as the impact of varying the number of seen utterances on model performance.

Results

Comprehension and Partner Modeling

• Objective Tasks: GPT-4 demonstrates superior performance in comprehension tasks, particularly at the Start stage (81.7% accuracy) and End stage (65.3% accuracy), outperforming both open-source models and the fine-tuned Flan-T5 baseline.
• Subjective Tasks: All models, including GPT-4, show poor correlation with human judgments regarding subjective assessments (satisfaction and likability). GPT-4's PCC for satisfaction ratings is only 0.3, and it frequently misclassifies deal outcomes in directions opposite to human perception.
• Partner Modeling: GPT-4 exhibits strong ToM abilities in inferring partner priorities (objective) but struggles to capture the degree of positivity or negativity in partner subjective states.

Annotation and Generation

• Annotation: GPT-3.5 and GPT-4 outperform the fine-tuned Flan-T5 on dialogue act annotation (Macro-F1 of 62.4% and 52.5% respectively), though performance drops on strategy detection due to the subtlety of linguistic strategies.
• Generation:
  • Automatic Metrics: Fine-tuned Flan-T5 achieves higher BLEU and ROUGE scores due to dataset-specific training, but GPT models show better semantic similarity (BERTScore) on diverse datasets.
  • Human Evaluation: GPT-4 achieves Coherence scores comparable to human references. However, in terms of Strategy, all models significantly underperform compared to humans. Models struggle to generate responses that are strategically advantageous, often failing to incorporate shared information or adhering to their own priorities.

Prompting and Robustness

• CoT Prompting: Applying Chain-of-Thought prompting to GPT-4 significantly improves performance on multi-hop arithmetic reasoning tasks (e.g., calculating total deal points), achieving near 100% accuracy.
• Prediction Balance: GPT-4 demonstrates superior robustness and prediction balance (low KL-Divergence) compared to other models, particularly in imbalanced task scenarios.
• Context Sensitivity: Increasing the number of utterances seen by the model improves partner modeling (inferring others) but degrades comprehension of the agent's own context, suggesting a recency bias.

Significance and Claims

The paper concludes that while LLMs, particularly GPT-4, possess the foundational capabilities to assist in negotiation research—such as designing dialogue systems, providing pedagogical feedback, and automating data annotation—they are not yet fully effective as autonomous strategic agents.

• Current Utility: LLMs are effective for tasks requiring comprehension of context and objective facts. They can serve as efficient tools for annotating dialogue acts and strategies, potentially scaling up data collection practices.
• Limitations: Significant gaps remain in strategic reasoning and the ability to capture subjective human states. Models tend to be overly agreeable, fail to utilize shared information strategically, and struggle with coherence in complex, multi-turn interactions.
• Future Directions: The authors suggest that future work should focus on combining LLMs with Reinforcement Learning (RL) policies to control strategic behavior and personality. Additionally, improving the handling of long contexts and developing structured reasoning approaches (breaking generation into subgoals) are necessary to advance AI negotiation agents.

The study emphasizes that while proprietary models like GPT-4 show promise, smaller open-source models currently perform comparably to trivial baselines in key negotiation tasks, highlighting the need for further development in open-source architectures for this domain.