Technical Summary: Think Thrice Before You Speak (TTBYS)

1. Problem Definition

Persuasive dialogue systems require the ability to infer and reason about a user's latent mental states, a capability known as Theory of Mind (ToM). While Large Language Models (LLMs) have advanced in this domain, existing approaches often rely on simple prompting strategies and lack sufficient ToM knowledge. Consequently, they frequently fail to capture the intrinsic dependencies among mental states (Beliefs, Desires, and Intentions), leading to fragmented representations and unstable reasoning in multi-turn interactions.

Current benchmarks (e.g., NegotiationToM, ToMATO, PersuasiveToM) indicate that LLMs struggle with reliable mental-state inference due to a lack of prior knowledge. Furthermore, most existing methods treat mental states as independent variables rather than modeling their sequential evolution, which limits interpretability and long-term effectiveness in complex persuasive scenarios.

2. Methodology

2.1 The ToM-PD Task and BDI Framework

The authors formalize the ToM-based Persuasive Dialogue (ToM-PD) task, grounded in the Belief–Desire–Intention (BDI) framework. Unlike the forward evolution of self-states (Belief $\to$ Desire $\to$ Intention $\to$ Action), the ToM-PD task is modeled as a reverse, stepwise inference process. The agent observes the persuadee's external utterances and must iteratively infer:

1. Intention ($i_t$): The immediate goal of the utterance.
2. Desire ($d_t$): The user's attitude toward the persuasion target (modeled as -1: unwilling, 0: hesitant, 1: willing).
3. Belief ($b_t$): The underlying views or concerns driving the desire.
4. Strategy ($s_t$): The optimal persuasive response based on the inferred states.

2.2 The TTBYS Framework

To address the challenges of instability and lack of knowledge, the paper proposes Think Thrice Before You Speak (TTBYS), a knowledge-enhanced stepwise reasoning framework. TTBYS leverages both explicit and implicit prior experiences to guide LLMs through three distinct reasoning stages:

• First Think (Desire Inference):

  • The current dialogue is summarized into a high-level intent ($i$).
  • This summary is used to retrieve top-$N$ semantically similar historical experiences from a knowledge base.
  • A probability distribution of desires is constructed from the retrieved experiences ($P_{ex}$).
  • This is linearly fused with the LLM's intuitive probability distribution ($P_{ag}$) to determine the final desire prediction: $d^*_t = \arg \max (\alpha \cdot P_{ag} + (1-\alpha) \cdot P_{ex})$.

• Second Think (Belief Inference):

  • Using the inferred desire ($d^*$) and the dialogue summary, the system retrieves relevant experiences containing belief information.
  • These retrieved experiences are injected directly into the LLM's context as explicit knowledge.
  • The LLM generates the belief description ($b^*$) conditioned on the dialogue, the inferred desire, and the retrieved examples.

• Third Think (Strategy Prediction):

  • The system performs a joint retrieval using the dialogue summary and the inferred belief.
  • Similar to the First Think, an experience-driven strategy distribution is fused with the LLM's distribution using a coefficient $\beta$ to select the optimal strategy.

2.3 The ToM-BPD Dataset

To facilitate research, the authors constructed ToM-based Broad Persuasive Dialogues (ToM-BPD), a large-scale annotated dataset derived from the PersuasiveToM benchmark.

• Annotations: Includes fine-grained labels for the persuadee's desire (discrete values -1, 0, 1) and belief (natural language statements), as well as the persuader's strategy.
• Taxonomy: Strategies are categorized into three groups (Socio-emotional, Cognitive, Interactive) and nine fine-grained techniques based on the Elaboration Likelihood Model (ELM) and Motivational Interviewing.
• Quality Control: The dataset underwent rigorous annotation with semi-automatic pre-annotation, multi-round human verification, and qualification tests for annotators.

3. Key Contributions

1. Task Formulation: Introduction of the ToM-PD task, which explicitly models the sequential dependencies among mental states (Intention, Desire, Belief) to enhance interpretability.
2. Framework Proposal: Development of TTBYS, a knowledge-enhanced framework that integrates structured ToM experiences to improve the robustness and consistency of LLM reasoning in persuasive dialogues.
3. Dataset Construction: Release of ToM-BPD, a multi-domain dataset with fine-grained mental state annotations, validated through extensive experiments and case studies.

4. Experimental Results

The authors evaluated TTBYS across six frontier LLMs (including LLaMA-3.1-8B, Qwen-3-8B, Mixtral-7B, and closed-source models like GPT-5 and Gemini-3-Pro).

• Static Evaluation:

  • Performance: Qwen-3-8B equipped with TTBYS outperformed GPT-5 by 1.20% in desire prediction, 22.80% in belief prediction, and 16.97% in strategy prediction.
  • Comparison: TTBYS consistently outperformed vanilla zero-shot prompting, Chain-of-Thought (CoT), and other ToM-enhanced baselines (Hypothetical Minds, Metamind) across all open-source backbones.
  • Ablation: The study confirmed that a balanced fusion of experience-based and LLM-based signals (via coefficients $\alpha$ and $\beta$) yields optimal results. Increasing the quantity of retrieved experiences generally improved performance, particularly in the First and Third Think stages.

• Interactive Evaluation:

  • Simulated interactions in product purchase, community activities, and empathetic dialogue scenarios showed that TTBYS (using Qwen-3-8B) significantly outperformed GPT-5 baselines in Identification (uncovering mental states) and Persuasion effectiveness.
  • While GPT-5+CoT showed slightly higher consistency, TTBYS demonstrated superior ability to adapt strategies based on accurate mental state inference.

• Case Studies:

  • Qualitative analysis demonstrated that TTBYS could capture nuanced beliefs (e.g., uncertainty about the relevance of online privacy to rural revitalization) that baseline models missed, leading to more accurate desire predictions and appropriate strategy selection.

5. Significance and Claims

The paper claims that TTBYS significantly advances the state of the art in ToM-driven persuasive dialogue by:

• Enhancing Interpretability: By decomposing the persuasion process into explicit steps (Desire $\to$ Belief $\to$ Strategy) and utilizing structured reasoning, the system's decision-making becomes more transparent and traceable.
• Improving Robustness: The integration of prior experiences mitigates the instability of LLMs in inferring latent mental states, allowing smaller open-source models to match or surpass larger closed-source models.
• Providing a Foundation: The ToM-PD task and ToM-BPD dataset offer a standardized benchmark for future research in modeling dynamic mental states in social interactions.

The authors acknowledge limitations, noting that the dataset size is relatively small and that the approach has not yet been validated in a fully deployed, real-world persuasive system. They emphasize that while the framework is designed for pro-social applications, safeguards are necessary to prevent potential misuse in manipulative contexts.