Arguments that Alter Minds: LLM Rationales Sway Human (and LLM) Notions of Plausibility

Technical Summary: Arguments that Alter Minds: LLM Rationales Sway Human (and LLM) Notions of Plausibility

Problem Statement
Commonsense reasoning benchmarks (e.g., Social IQA, CommonsenseQA) typically evaluate models by asking them to select the most plausible answer from multiple choices. Unlike logical or mathematical problems, these answers exist on a continuum of plausibility rather than being definitively true or false. While human annotators are traditionally considered "experts" in this domain, the paper investigates whether Large Language Models (LLMs) can generate arguments (rationales) that systematically alter human and LLM judgments regarding the plausibility of these answer choices. Specifically, the authors ask if LLM-generated "PRO" (supporting) and "CON" (refuting) rationales can sway human plausibility ratings, even in domains where humans possess strong intuitive knowledge.

Methodology
The study builds upon a prior plausibility rating framework (Palta et al., 2024) using two datasets: Social IQA (SIQA) and CommonsenseQA (CQA).

1. Data Construction: The authors sampled 100 questions (50 from each dataset). For each question, they selected the gold label (correct answer) and a randomly selected distractor (incorrect answer).
2. Rationale Generation: Using GPT-4o (selected via a preference study for its ability to generate convincing arguments), the authors generated three types of rationales for each (question, answer) pair:
   • PRO: An argument supporting the answer's plausibility.
   • CON: An argument supporting the answer's implausibility.
   • PRO+CON: Both arguments presented together.
3. Human Evaluation: The authors collected 3,000 new human plausibility judgments (using a 1–5 Likert scale: Impossible to Very Likely) across four conditions: No Rationale, PRO, CON, and PRO+CON.
4. LLM Evaluation: In parallel, 17 different LLMs (divided into OpenAI and Non-OpenAI families) were prompted to rate the plausibility of the same items under the same four conditions, generating 13,600 judgments.
5. Analysis: The study compared mean plausibility ratings across conditions and performed Ordinary Least Squares (OLS) regression to identify factors (e.g., initial rating, rationale type) driving changes in scores.

Key Results

• Swaying Effect: Both human and LLM plausibility judgments are significantly influenced by the presence of rationales. PRO rationales generally increase mean ratings, while CON rationales decrease them.
• Bimodal Human Response to PRO Rationales: A critical finding is the divergent reaction to PRO rationales based on answer type. For distractor (incorrect) answers, PRO rationales successfully raised human mean ratings. However, for gold (correct) answers, PRO rationales surprisingly lowered human mean ratings. The authors suggest this may be due to Grice's Maxim of Quantity, where humans interpret the need for an argument as a signal that the fact is not self-evident, thereby "underselling" the obviousness of the correct answer.
• LLM Sensitivity: LLMs also showed significant shifts in ratings. Unlike humans, LLMs generally increased their ratings for gold labels when presented with PRO rationales. OpenAI models appeared more sensitive to the rationales (generated by GPT-4o) than Non-OpenAI models, potentially indicating a self-preference bias.
• Anchoring Effect: Regression analysis revealed a strong anchoring effect: the higher the initial plausibility rating of an answer, the smaller the subsequent change after a rationale is introduced. This effect was more pronounced for distractors than for gold labels, suggesting initial judgments on incorrect answers are more rigid.
• CON Dominance: CON rationales exerted a stronger negative influence on ratings than PRO rationales exerted positive influence. When PRO and CON were presented together, the opposing component tended to dominate, often resulting in a net decrease in plausibility ratings.
• Rationale Length: The length of the rationale (word count) showed no significant linear relationship with the magnitude of rating changes.

Significance and Claims
The paper claims to demonstrate a novel use of LLMs for studying aspects of human cognition, specifically how external arguments influence belief formation in commonsense reasoning. The authors highlight several implications:

• Vulnerability of Human Judgment: Even in domains where humans are considered "experts" (everyday common sense), LLM-generated arguments can significantly sway human beliefs. This raises concerns about the potential for persuasive manipulation, as users may prioritize the plausibility of an LLM's argument over their own knowledge.
• Alignment Challenges: The study underscores the difficulty in aligning AI-generated judgments with human intuition, particularly in subjective reasoning tasks where humans and LLMs react differently to the same stimuli (e.g., the bimodal effect on gold labels).
• Safety and Collaboration: The findings suggest that LLMs can effectively steer human judgment toward more implausible inferences. This highlights a critical vulnerability in human-AI collaboration, necessitating safeguards such as transparency and bias mitigation to prevent harmful manipulative uses.
• Methodological Contribution: The work provides empirical evidence that LLM rationales can alter the perceived plausibility of commonsense scenarios, challenging the assumption that human intuition in these domains is immune to algorithmic influence.

The authors remain modest regarding the scope, noting that their findings are specific to English-language commonsense datasets and that the persuasive power may vary across different models, languages, and more constrained domains (e.g., legal or scientific reasoning).