💬 NLP

ToMAP: Training Opponent-Aware LLM Persuaders with Theory of Mind

The paper introduces ToMAP, a novel 3B-parameter framework that enhances LLM persuaders by integrating Theory of Mind modules to dynamically model opponents' mental states, enabling the model to outperform significantly larger baselines like GPT-4o through more diverse, logical, and opponent-aware arguments.

Original authors: Peixuan Han, Zijia Liu, Jiaxuan You

Published 2026-06-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Peixuan Han, Zijia Liu, Jiaxuan You

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: Teaching AI to "Read the Room"

Imagine you are in a debate club. There are two types of debaters:

The Robot: It has a script. No matter what you say, it just repeats its main point louder and louder. If you say, "I hate vegetables," it says, "But vegetables are healthy!" If you say, "I hate cooking," it says, "But vegetables are healthy!" It never changes its tune.
The Human: They listen to you. If you say, "I hate vegetables," they think, "Ah, they don't like the taste." So, they switch tactics: "Okay, what if we roasted them with garlic? They taste like candy then." If you say, "I hate cooking," they say, "What if you just buy pre-cut ones?"

The paper argues that current Large Language Models (LLMs) act mostly like The Robot. They are good at generating text, but they are terrible at Theory of Mind—the human ability to understand what someone else is thinking, feeling, or believing.

ToMAP (Theory of Mind Augmented Persuader) is a new training method designed to turn the Robot into a Human-like debater. It teaches a small AI (only 3 billion parameters, which is tiny compared to giants like GPT-4) how to "read the room" and adapt its arguments dynamically.

How ToMAP Works: The Two "Superpowers"

To give the AI this human-like intuition, the researchers gave it two special tools (modules) that act like a mental cheat sheet during a conversation.

1. The "Crystal Ball" (Counterclaim Predictor)

The Problem: Humans naturally think, "If I say X, my opponent might think Y." Current AIs often miss this. They just stick to their own script.
The Solution: Before the AI speaks, it uses a "Crystal Ball" to simulate the opponent's mind. It asks itself: "What are three reasons my opponent might disagree with me?"
The Analogy: Imagine playing chess. A grandmaster doesn't just look at their next move; they look at the board and think, "If I move here, my opponent will likely move there." The Crystal Ball lets the AI do the same thing with words. It anticipates objections before they even happen.

2. The "Emotion Radar" (Opponent Attitude Predictor)

The Problem: Even if the AI guesses the objection, it doesn't know how strongly the opponent feels about it. Is the opponent just mildly annoyed, or are they furious?
The Solution: This tool acts like a radar. It analyzes the conversation history and the predicted objections to give the AI a "score" of how much the opponent agrees or disagrees with specific points.
The Analogy: Think of a salesperson. A bad salesperson ignores the customer's face. A good salesperson watches the customer's face. If the customer looks skeptical, the salesperson slows down and explains more. If the customer looks interested, they speed up. The "Emotion Radar" tells the AI exactly how to adjust its speed and tone.

The Training Method: Learning by Doing (Reinforcement Learning)

You can't just tell an AI, "Be more empathetic," and expect it to work. The paper uses Reinforcement Learning (RL), which is like training a dog with treats.

The Game: The AI plays a debate game against another AI.
The Goal: The goal isn't just to talk; it's to change the opponent's mind.
The Reward: If the opponent's opinion shifts even a tiny bit toward the AI's side, the AI gets a "treat" (a reward score). If the AI repeats itself or gets stuck in a loop, it gets a "scolding" (a penalty).
The Result: Through thousands of games, the AI learns that the only way to get treats is to stop repeating itself and start using the "Crystal Ball" and "Emotion Radar" to craft unique, tailored arguments.

The Surprising Results

The most exciting part of the paper is the performance.

Small vs. Giant: The ToMAP model is tiny (3 billion parameters). The "giants" it beat include GPT-4o and models with 70 billion or even 671 billion parameters.
The Score: ToMAP outperformed these massive models by a huge margin (about 39% better in some tests).
Why? The giant models are like libraries with millions of books but no librarian to organize them. They know a lot of facts but don't know how to use them strategically against a specific person. ToMAP is a smaller library with a brilliant librarian who knows exactly which book to pull out for this specific customer.

Key Takeaways from the Experiments

Less Repetition: Without ToMAP, AIs tend to repeat the same 8 words over and over (like a broken record). ToMAP stops this because it's constantly checking the opponent's mental state and finding new angles.
Long Conversations: In short chats, big models are okay. But in long, 10-turn conversations, the big models get tired and repetitive. ToMAP gets better as the conversation goes on because it keeps adapting its strategy.
Logic over Emotion: The paper found that ToMAP stopped using "fluff" (like emotional appeals or shouting) and started using logic, evidence, and finding common ground. It became a more sophisticated debater.

Summary

ToMAP is a framework that teaches AI to stop being a "broken record" and start being a "mind reader." By giving the AI tools to predict what the opponent is thinking and how they feel, and then training it with rewards for actually changing their mind, the researchers created a small, efficient AI that is more persuasive than the world's largest and most expensive AI models.

It proves that strategy and awareness are more important than just having a massive brain.

Technical Summary: ToMAP – Training Opponent-Aware LLM Persuaders with Theory of Mind

1. Problem Statement

While Large Language Models (LLMs) have demonstrated potential in persuasion tasks, current LLM-based persuaders face significant limitations in modeling the interactive and dynamic nature of human debate. Human persuaders naturally construct a "cognitive graph" of related propositions, proactively anticipating an opponent's objections and tailoring arguments to the opponent's mental state—a capability known as Theory of Mind (ToM).

In contrast, existing LLM persuaders often:

Lack Opponent Awareness: They focus narrowly on the central claim, ignoring the interconnected structure of related claims and the opponent's perspective.
Suffer from Repetition: They tend to repeat the same arguments in consecutive turns rather than generating diverse insights from a broader cognitive graph.
Exhibit Rigidity: They are often self-centered, failing to dynamically adapt strategies based on the persuadee's evolving beliefs.

The paper posits that the deficiency in ToM reasoning is the primary bottleneck preventing LLMs from achieving human-level persuasive flexibility and diversity.

2. Methodology: ToMAP Framework

The authors propose ToMAP (Theory of Mind Augmented Persuader), a training framework that integrates two dedicated ToM modules with Reinforcement Learning (RL) to enhance a persuader's ability to model and respond to an opponent's mental state.

2.1. Core Components

The framework consists of three main stages:

Counterclaim Predictor:
- Function: This module prompts the persuader to proactively generate potential counterarguments ( $\neg q_{1...k}$ ) against the target claim ( $Q$ ) that the opponent might hold.
- Mechanism: It forces the model to explicitly conceptualize the debate topic's associated claims, thereby anticipating objections before they are raised.
Opponent Attitude Predictor:
- Function: This module estimates the persuadee's current level of agreement ( $s'$ ) with the generated counterclaims.
- Mechanism: Unlike zero-shot prompting, which yields unsatisfactory results, ToMAP employs an external classifier consisting of a pre-trained text encoder (BGE-M3) and a Multi-Layer Perceptron (MLP). It encodes the conversation history and claims separately, concatenates them, and predicts a 5-point agreement score. This provides a dynamic, quantitative assessment of the opponent's mental state.
Reinforcement Learning (RL) Integration:
- Training Objective: The persuader is trained using Proximal Policy Optimization (PPO). The reward function is designed to maximize the attitude shift of the persuadee.
- Reward Calculation: The reward is derived from the difference in the persuadee's agreement score before and after the persuader's turn, normalized to account for the difficulty of shifting already high agreement scores.
- Augmentation: During training and inference, the ToM information (predicted counterclaims and their predicted attitude scores) is prepended to the persuader's prompt as an augmentation of the input context. This allows the RL agent to learn how to leverage these insights to formulate strategies.

2.2. Training Setup

Base Models: Qwen2.5-3B-Instruct and LLaMa3.2-3B-Instruct.
Persuadee Model: Qwen2.5-7B-Instruct (used during training).
Datasets: Cornell CMV (Change My View), Anthropic Persuasion Dataset, and args.me corpus.
Evaluation Metric: Agreement shift (percentage change in the persuadee's stance from the start to the end of the conversation).

3. Key Contributions

Novel Framework: Introduction of ToMAP, the first framework to explicitly combine ToM modules (counterclaim and attitude prediction) with RL for training LLM persuaders.
Efficiency: Demonstrates that a small 3B parameter model, when trained with ToMAP, can outperform significantly larger state-of-the-art models (e.g., GPT-4o, LLaMa3.1-70B, Deepseek-R1) in persuasion tasks.
Ablation Insights: Proves that ToM modules alone do not improve zero-shot performance; the combination of ToM information and RL training is necessary for the persuader to effectively utilize opponent modeling.
Behavioral Analysis: Shows that ToMAP reduces argument repetition and increases reasoning complexity (longer thought chains) compared to baselines.

4. Experimental Results

Performance Gains: ToMAP (3B parameters) achieved a 39.4% relative gain over baseline configurations across multiple persuadee models and diverse corpora.
Comparison with SOTA: ToMAP outperformed much larger baselines, including GPT-4o and Deepseek-R1 (671B parameters), in terms of agreement shift.
Generalization: The model demonstrated stable performance across out-of-domain persuadees (e.g., LLaMa3.1, Phi-4), suggesting the learned strategies are robust.
Long-Form Conversations: In extended conversations (up to 10 turns), ToMAP maintained steady persuasion gains, whereas baseline RL models plateaued or declined due to repetition and lack of adaptation.
Strategy Analysis: ToMAP adopted more opponent-aware strategies, such as "Common Ground," "Preemptive Rebuttal," and "Concession," while relying less on rhetorical or emotional appeals compared to base models.

5. Significance and Claims

The paper claims that ToMAP represents a significant step toward developing more effective AI persuaders by integrating Theory of Mind and opponent modeling.

Scalability: The results suggest that small models can be highly persuasive if equipped with the right training framework (ToM + RL), challenging the notion that only massive models can handle complex social reasoning.
Mechanism of Improvement: The authors argue that the success of ToMAP stems from its ability to generate complex reasoning chains and reduce repetition, leading to more diverse and logically sound arguments.
Safety and Ethics: The authors emphasize the potential risks of persuasive AI in manipulating opinions. They advocate for transparency, explicit user awareness of AI interaction, and controlled release with usage guidelines to prevent misuse.

The work concludes that incorporating ToM capabilities is a promising direction for building language agents that can engage in dynamic, adaptive, and effective social interactions.