💻 computer science

Cognitive World Models for Process-Level Social Influence Evaluation

This paper introduces CogWM, an LLM-based cognitive world model trained via a novel SaA annotation pipeline to evaluate social influence dialogues by tracking the evolution of users' internal cognitive states (beliefs, desires, intentions, and emotions) rather than relying on surface-level text metrics.

Original authors: Minghui Ma, Bin Guo, Han Wang, Mengqi Chen, Jingqi Liu, Yan Liu, Zhiwen Yu

Published 2026-06-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Minghui Ma, Bin Guo, Han Wang, Mengqi Chen, Jingqi Liu, Yan Liu, Zhiwen Yu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are watching a friend try to convince you to do something, like start a new hobby, donate to a charity, or just feel better after a bad day.

The Old Way of Measuring Success
Until now, if we wanted to know if a computer program (an AI) was good at having these conversations, we used two main methods:

The "Grammar Police": Checking if the AI's words sounded smooth or matched the user's words closely (like checking if two sentences rhyme).
The "Final Grade": Asking a human or a smart computer to give the whole conversation a single score at the very end, like a teacher grading a final exam.

The Problem: These methods miss the point. A conversation can sound perfect and get a high grade, but the user might still feel exactly the same (or worse) than when they started. It's like a movie with a perfect script and a happy ending, but the main character never actually learned anything or changed their mind. We couldn't see how the user's mind changed during the chat.

The New Solution: The "Cognitive World Model" (CogWM)
The researchers at Northwestern Polytechnical University and Harbin Engineering University built a new kind of AI called CogWM. Think of it as a "Mind-Reading Simulator."

Instead of just listening to what the user says, CogWM tries to guess what the user is thinking and feeling inside. It tracks four specific things, which they call BDI/E:

Beliefs: What the user thinks is true (e.g., "I am bad at math").
Desires: What the user wants (e.g., "I want to pass this class").
Intentions: What the user plans to do (e.g., "I will study for 10 minutes").
Emotions: How the user feels right now (e.g., "Anxious" or "Hopeful").

How It Works (The "Wind Tunnel" Analogy)
The authors compare CogWM to a wind tunnel for airplanes. Before building a real plane, engineers test a model in a wind tunnel to see how the air pushes against it.

The Airplane: The AI chatbot trying to influence the user.
The Wind Tunnel: CogWM.
The Air: The conversation.

When you test a chatbot, you connect it to CogWM. CogWM plays the role of the user. As the chatbot talks, CogWM updates its internal "scorecard" of the user's beliefs, desires, and emotions turn-by-turn.

The Three-Layer Report Card
Instead of giving one final grade, CogWM generates a detailed report with three layers:

The Turn Level: Did the chatbot say something that made sense for this specific moment?
The Journey Level: Did the user's internal state move in a good direction over time? (e.g., Did their anxiety go down? Did their belief that "I can do this" go up?)
The Goal Level: Did the conversation actually achieve the desired outcome (like making a donation or feeling better)?

What They Found

Better than the Experts: They trained CogWM on over 150,000 conversation snippets. When they tested it against other top AI models, CogWM was much better at guessing the user's internal feelings and thoughts (about 2 times more accurate than the best existing models).
The "Fake" vs. "Real" Change: They tested six famous AI chatbots. Some chatbots were great at getting a "Good Outcome" (like getting a user to say "Yes, I'll donate"). But when they looked at the journey, they saw that some chatbots just pressured the user into saying "Yes" without actually changing their mind.
- Analogy: One chatbot was like a Sprinter (got a quick result but the user's mind didn't really change). Another was like a Steady Hiker (slowly guided the user's mind to a better place).
The "Black Box" is Open: With CogWM, we can finally see why a chatbot worked or failed. We can see exactly which turn of the conversation made the user feel hopeful, or which turn made them feel pressured.

In Short
This paper introduces a tool that stops judging conversations by their "final score" and starts judging them by the story of how the user's mind changed. It turns the invisible process of "influence" into a visible, measurable map, showing us which AI assistants are truly helpful and which are just good at talking.

Technical Summary: Cognitive World Models for Process-Level Social Influence Evaluation

1. Problem Definition

Social Influence Dialogue (SID) systems aim to alter a user's internal cognitive states—specifically beliefs, desires, intentions, and emotions—rather than merely completing external tasks. While Task-Oriented Dialogues focus on goal completion and Non-Cooperative Dialogues on conflict resolution, SID requires measuring the process of cognitive change.

Current evaluation methods fail to capture this process-oriented criterion:

Surface Text Metrics (BLEU/ROUGE): Ignore the social influence objective entirely.
LLM-as-Judge: Provide terminal, black-box scores that reveal nothing about the trajectory of cognitive state evolution.
Human Evaluation: Expensive, unscalable, and unable to track granular, turn-level cognitive shifts.
Existing User Simulators: Predict user utterances but fail to model the underlying "why" (internal BDI/E states), leaving beliefs, desires, intentions, and emotions unobserved.

The central challenge is to develop an evaluation framework that moves from terminal judgment to process tracking, quantifying how an agent measurably changes a user's internal cognitive state over the course of a conversation.

2. Methodology: The Cognitive World Model (CogWM)

The authors propose CogWM, an LLM-based user model that reframes dialogue evaluation from "what did the user say" to "how did the user's internal cognitive state evolve?"

2.1 Core Framework

CogWM operates as a joint predictor of User Utterances ( $U_t$ ) and BDI/E Cognitive States ( $S_t$ ).

BDI/E Dimensions:
- Belief: What the user believes, knows, or judges.
- Desire: What the user wants, needs, or prefers.
- Intention: What the user plans, decides, or commits to doing.
- Emotion: The user's current affective state and intensity.
Input/Output: Given a user profile ( $P$ ), task context ( $C$ ), and dialogue history ( $H_{<t}$ ), CogWM predicts the next state $S_t$ and utterance $U_t$ .
Dual Role: It functions as both a User Simulator (generating realistic responses and states) and an Evaluation Platform (quantifying the influence of an agent on the simulated user).

2.2 Data Construction: Summarize-and-Allocate (SaA)

To train CogWM, the authors constructed a dataset of 150,454 user-turn samples across four scenarios: Emotional Support (ESConv), Charitable Donation Persuasion (P4G), Conversational Recommendation (DuRecDial), and Open-Domain (DailyDialog).

Annotation Pipeline: A two-stage method was developed:
1. Global Summarization: Extracts global BDI items and scores from the full dialogue.
2. Temporal Allocation: Assigns per-turn BDI/E states with evidence grounding and links them to the global summary.
Quality: Human validation showed a global BDI recall of 0.79 and emotion Cohen's $\kappa$ of 0.69.

2.3 Evaluation Framework: Three-Tier System

The evaluation is structured into three hierarchical tiers:

Turn-Level Fidelity: Assesses single-response quality using 8 automatic metrics (e.g., semantic similarity, MAE) and 5 LLM-as-Judge semantic metrics.
Trajectory-Level State Dynamics: Models multi-turn evolution using four indicators adapted from financial time-series analysis:
- AUC: Cumulative performance (Area Under Curve).
- $\Delta$ : Net change between initial and final states.
- PSR: Improvement frequency (Positive Step Ratio).
- Mono: Directional consistency.
Task-Level Composite Scoring: Aggregates trajectory metrics into two scores:
- CTS (Composite Trajectory Score): A weighted sum ( $0.40 \cdot AUC + 0.30 \cdot PSR + 0.30 \cdot \Delta$ ) measuring global cognitive influence.
- GO (GoodOutcome): A binary indicator of whether the dialogue achieved a specific behavioral outcome (e.g., donation, acceptance).

3. Key Contributions

Three-Tier BDI/E Evaluation Framework: Adapts the classical BDI model to the LLM era, enabling decomposable evaluation from turn-level fidelity to global trajectory impact.
SaA Annotation Pipeline: A scalable method for generating 150K+ annotated samples with explicit, interpretable BDI/E states across four distinct social influence scenarios.
CogWM Model: A jointly trained model (based on Qwen3-14B) that predicts both cognitive states and text. It achieves 77.6% emotion accuracy, a 2.1 $\times$ improvement over GPT-5.5, and significantly outperforms baselines in tracking beliefs, desires, and intentions.
Empirical Validation of Process vs. Outcome: Through 3,600 multi-agent discrimination trials, the study demonstrates that cognitive trajectory improvement (CTS) and behavioral outcomes (GO) are not equivalent. An agent can achieve a positive behavioral outcome (GO=true) while failing to produce meaningful cognitive change (flat trajectory), and vice versa.

4. Experimental Results

4.1 Simulation Fidelity

State Prediction: CogWM-14B achieved 77.6% emotion accuracy, outperforming GPT-5.5 (36.4%) and DeepSeek-V4-Pro (31.0%). It also led in BDI semantic similarity and quantitative score accuracy (MAE).
Utterance Generation: CogWM achieved a Length Ratio (LenR) of 1.01 (near-perfect match to ground truth), whereas API baselines tended to generate overly long responses (LenR > 1.8).
Joint Training Necessity: Ablation studies confirmed that cognitive state tracking cannot be acquired via shared input distribution alone; explicit joint optimization of state prediction and text generation is required.

4.2 Agent Discrimination

In trials comparing six commercial agents (GPT-5.5, Llama-4-Scout, DeepSeek-V4-Pro, etc.):

Ranking: Llama-4-Scout ranked first with a CTS of +0.233, demonstrating superior ability to drive positive cognitive trajectories.
Typology: Agents were categorized into "All-round" (Llama), "Sprint" (Qwen: high net change but lower cumulative gain), "Steady" (Claude), and "Hollow" (Gemini: low performance across all metrics).
CoT Impact: Chain-of-Thought prompting asymmetrically affected agents, significantly improving GPT-5.5's emotional trajectory but offering marginal gains for already strong agents.
CTS vs. GO Divergence: The study highlighted that high GO scores (task success) do not guarantee high CTS scores (cognitive improvement). For instance, in emotional support, some agents achieved GO=true via polite closure without driving actual cognitive shifts, a nuance captured only by CTS.

5. Significance and Claims

The paper claims to shift the paradigm of Social Influence Dialogue evaluation from terminal judgment to process tracking. By making the user's internal cognitive state explicit and auditable, CogWM provides:

Interpretability: Evaluation conclusions can be traced to specific turns and BDI/E dimensions.
Granularity: It captures subtle cognitive shifts that surface metrics miss.
Scalability: It enables low-cost, automated evaluation of complex social influence scenarios.

The authors position CogWM not just as a metric, but as a social AI infrastructure (analogous to wind tunnels for aircraft) that allows developers to observe the cognitive impact of agents before deployment, thereby offering a concrete tool for preventing manipulative AI and ensuring responsible social influence. The work emphasizes that true social influence requires measurable changes in the user's internal state, not just the completion of a transaction.