Technical Summary: Human Decision-Making with Persuasive and Narrative LLM Explanations

Problem Statement

While Large Language Models (LLMs) possess the capacity to generate cogent, narrative explanations for AI predictions, the impact of these explanations on objective human decision-making performance remains under-explored. Prior research indicates that narrative explanations improve user understanding, trust, and subjective agreement with AI, yet evidence suggests they often fail to improve objective accuracy compared to predictions alone. Furthermore, the specific influence of persuasiveness within these narratives—ranging from neutral to highly persuasive or manipulative—on human reliance and decision accuracy is a critical gap in the literature. This study addresses whether varying the persuasiveness of LLM-generated narratives alters human decision accuracy, reliance rates, and confidence calibration in joint human/AI decision-making tasks.

Methodology

The authors conducted a large-scale, pre-registered human behavioral experiment involving 320 participants recruited via Prolific. The study utilized a 4 (Explanation Condition) × 2 (Dataset) factorial design.

• Datasets: Two real-world classification tasks were used: the Census Income dataset (predicting income >$50k) and the Student Performance dataset (predicting pass/fail status). These were selected for their accessibility to non-expert participants.
• Stimuli Generation: Using OpenAI GPT-4o (temperature 0.1), the authors generated predictions and corresponding narrative explanations for 52 test instances per participant. Three explanation conditions were created via prompt engineering:
  1. Neutral: Standard, non-persuasive explanations.
  2. Low Persuasion: Explanations using a "persuasive style."
  3. Extreme Persuasion: Explanations using "propaganda and logical fallacies" styles.
  • A Prediction Alone condition served as the baseline.
• Experimental Task: Participants viewed tabular data, an AI prediction (Pass/Fail or Yes/No), and, in explanation conditions, a narrative explanation presented word-by-word (typewriter effect) to encourage reading. Participants then made a binary choice, rated their confidence (1–7 Likert scale), and received feedback.
• Metrics:
  • Decision Accuracy: Comparison of human choices against ground truth.
  • AI Reliance: The proportion of trials where the human followed the AI's prediction.
  • Confidence Calibration: The relationship between reported confidence and actual accuracy.
  • Response Times: Time taken to make a decision.
  • Outperformance: The rate at which humans exceeded the AI's standalone accuracy.

Key Contributions

1. Evaluation of Persuasiveness: This work is among the first to systematically evaluate the impact of varying levels of persuasiveness in LLM-generated narrative explanations on objective decision metrics, moving beyond subjective trust or understanding.
2. Decoupling Reliance from Accuracy: The study provides empirical evidence that increased reliance on AI (driven by narrative explanations) does not necessarily translate to improved decision accuracy, particularly when the AI is incorrect.
3. Trade-off Identification: The research highlights potential trade-offs where narrative explanations, especially highly persuasive ones, may increase decision response times and reduce the human ability to discriminate between correct and incorrect AI predictions.

Results

The study yielded the following primary findings:

• Decision Accuracy: There was no meaningful difference in decision accuracy across the four conditions (Prediction Alone, Neutral, Low Persuasion, Extreme Persuasion). The degree of persuasiveness did not improve or degrade overall accuracy compared to the AI prediction alone.
• AI Reliance: Narrative explanations increased reliance on AI predictions compared to the Prediction Alone condition. However, this increase occurred regardless of whether the AI prediction was correct or incorrect.
  • Crucial Finding: In the Prediction Alone condition, participants showed discernment, relying more on correct AI predictions than incorrect ones. In explanation conditions, this discernment diminished; reliance on incorrect AI predictions increased significantly, effectively canceling out the benefits of relying on correct predictions.
• Confidence Calibration: Persuasiveness did not significantly impact confidence calibration. Calibration was better in the Census dataset than the Student dataset, but the explanation condition had no substantial effect.
• Response Times: The Extreme Persuasion condition resulted in significantly slower response times and higher variability compared to other conditions, suggesting that highly persuasive narratives may induce cognitive load or hesitation.
• Outperformance: Fewer participants in the persuasive conditions (Low and Extreme) outperformed the AI's standalone accuracy compared to the Prediction Alone and Neutral conditions, though these differences were not statistically significant.

Significance and Claims

The authors conclude that including narrative explanations with AI predictions involves tradeoffs for decision-making performance. While narratives may increase the rate at which humans follow AI, they do not inherently improve objective decision accuracy. In fact, the study suggests that more persuasive narratives may have detrimental effects, such as:

• Reducing the human ability to detect when the AI is wrong (reduced discernment).
• Increasing decision latency (response times).
• Potentially lowering the rate of human-AI collaboration outperforming the AI alone.

The paper posits that interventions focusing on subjective measures like trust or user experience may have limited practical relevance if they do not translate to improved decision accuracy. The authors emphasize that while LLMs can generate highly persuasive arguments, this capability does not automatically make them better aids for human decision-making and may, in some contexts, be counterproductive. The work calls for further research to determine the specific contexts and mechanisms by which narrative explanations impact human-AI collaboration, rather than assuming that more "convincing" explanations are inherently beneficial.