💬 NLP

StoryMI: Steerable Multi-Agent Therapeutic Dialogue Generation

This paper introduces StoryMI, a multi-agent framework that leverages situational storytelling and dynamic strategy control to generate clinically grounded, steerable motivational interviewing dialogues, validated through a comprehensive evaluation protocol and a new dataset of 6,000 simulated sessions.

Original authors: Qingyu Meng, Min Chen, Dingming Liu, Yifan Mo, Yue Su, Xin Sun, Koen Hindriks, Jiahuan Pei

Published 2026-05-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Qingyu Meng, Min Chen, Dingming Liu, Yifan Mo, Yue Su, Xin Sun, Koen Hindriks, Jiahuan Pei

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to be a therapist. You want it to use a specific, gentle style called Motivational Interviewing (MI). This style isn't about giving orders or fixing problems immediately; it's about asking the right questions and reflecting feelings to help a person find their own motivation to change.

The paper introduces a new system called StoryMI. Think of StoryMI not as a single robot, but as a tiny, highly organized film production crew working together to create a realistic therapy session.

Here is how they do it, broken down into simple parts:

1. The Problem: Robots are Too "Flat"

Previous attempts to make AI therapists had three main glitches:

No Backstory: They knew a client was "sad" (a score of 3 out of 5), but they didn't know why or what happened. It's like an actor knowing their character is "angry" but not knowing they just got fired. The conversation felt generic and fake.
No Director: The AI would chat fluently, but it didn't follow a plan. It might ask too many questions or give too much advice, missing the specific "rules" of good therapy.
Bad Grading: We could tell if the robot spoke clearly (like checking for spelling errors), but we couldn't easily tell if it was actually good at therapy.

2. The Solution: The StoryMI "Production Crew"

The authors built a system with three distinct AI agents (robots) that act like a movie crew:

The "Biographer" (Questionnaire Agent):
Instead of just giving a robot a list of symptoms, this agent takes a standard medical questionnaire and turns it into a rich, short story.
- Analogy: Imagine a questionnaire asks, "Do you have trouble sleeping?" The Biographer doesn't just say "Yes." It writes a 200-word story about a man lying awake at 3 AM, listening to the rain, worrying about his job. This gives the robot a real "world" to live in.
The "Director" (Interaction Agent):
This is the most important new piece. While the "Therapist Robot" and "Client Robot" talk, the Director watches closely.
- Analogy: Think of a film director shouting, "Okay, the client just said something sad; now the therapist needs to use a 'Reflection' technique, not a 'Question'!" The Director ensures the conversation follows the strict rules of Motivational Interviewing (e.g., "We need two reflections for every one question").
The "Actors" (Therapist & Client Agents):
These two robots have a conversation. The Client acts out the story created by the Biographer, and the Therapist responds based on the Director's instructions.

3. The Result: A Better Script

The team created 6,000 simulated conversations using this method. They tested it against six different large language models (the "brains" behind the robots).

What they found:

Context is King: When they removed the "Biographer" (the story part), the conversations became generic and boring. When they removed the "Director" (the strategy control), the robots stopped following therapy rules and started giving random advice.
The "Director" Works: The system successfully forced the robots to stick to the complex rules of therapy (like using more reflections than questions), even when the robots tried to take shortcuts.
Human vs. Robot Judges: They asked both human experts and other AI models to grade the conversations.
- The Twist: The AI judges were good at spotting if the conversation made sense logically (Coherence) or if it moved forward (Progress).
- The Gap: However, AI judges struggled to tell the difference between a "robotic" conversation and a "human-like" one. They also missed subtle signs of empathy. The human experts were much better at spotting the "soul" of the conversation.

4. The Takeaway

The paper concludes that to make AI therapy work, you can't just rely on a smart robot chatting. You need a structured workflow:

Give the robot a real story to ground the conversation.
Have a manager (the Director) constantly check that the robot is using the right techniques.
Use human experts to check the final product, because AI judges still miss the subtle "human touch" of empathy and naturalness.

Important Note from the Paper:
The authors are very clear that this is a research tool for training and studying therapy techniques. They explicitly state that this system is not ready to be used as a real therapist for actual patients. It is a simulation to help us understand how to build better tools in the future.

Technical Summary: StoryMI: Steerable Multi-Agent Therapeutic Dialogue Generation

Problem Statement

While Large Language Models (LLMs) demonstrate fluency in generating dialogue, prior research in Motivational Interviewing (MI) faces three critical barriers:

Lack of Situational Grounding: Existing methods often rely on abstract Likert-scale ratings or isolated turn-level generation, failing to capture the rich narrative context and personal intentions central to MI.
Insufficient Dynamic Controllability: Current approaches struggle to enforce macro-level counseling strategies (e.g., specific ratios of reflections to questions) across multi-turn conversations, often resulting in static or script-dependent interactions.
Evaluation Fidelity: Standard lexical metrics and generic LLM-as-judge rubrics fail to capture clinically meaningful distinctions, such as the alignment with macro-level therapeutic strategies and the authenticity of the therapeutic stance.

Methodology: The StoryMI Framework

StoryMI is a multi-LLM agent framework designed to generate controllable, situationally grounded MI dialogues. It operates through a three-stage workflow involving three distinct agents: a Client Agent, a Therapist Agent, and an Interaction Management Agent.

1. Questionnaire-Based Profiling

The system begins by generating structured client profiles using standardized DSM-5 questionnaires. An LLM generates severity scores (0–4) and brief first-person rationales for 23 items across 13 symptom domains (e.g., depression, anger, sleep). This creates a psychologically consistent and interpretable control signal ( $P$ ) for the dialogue.

2. Situational Story Contextualization

To bridge the gap between abstract scores and naturalistic dialogue, the system expands the client profile $P$ into a situational story ( $N$ ) of approximately 200 words. This narrative grounds the client's symptoms in concrete life experiences, specific scenes, and emotional textures, ensuring persona consistency across multi-turn interactions without manual curation.

3. Controllable MI Dialogue Simulation

The core simulation involves a coordinated loop:

Client Agent: Generates an utterance paired with a client MI code (Change, Sustain, or Neutral) based on the situational story and dialogue history.
Interaction Management Agent: Acts as the coordinator. It classifies the client's motivational orientation, selects an appropriate therapist strategy ( $\ell^*_T$ ) from the MI code set (Reflection, Question, Input), and updates the shared dialogue state. This agent enforces macro-level constraints, such as maintaining a Reflection-to-Question ratio > 2.0.
Therapist Agent: Generates a response conditioned on the selected strategy and the current dialogue state.

The process iterates until a termination condition is met (e.g., natural closure cues or a maximum turn limit), resulting in a fully simulated, MI-consistent conversation.

Key Contributions

Multi-Agent Workflow: The design of a structured workflow that integrates situational story contextualization with dynamically controllable MI-coded interactions, benchmarked across six representative LLMs.
MI-Specific Evaluation Protocol: The proposal of a two-level evaluation system combining standard lexical metrics with six MI-specific behavioral metrics (e.g., Strategy Adherence, Complex Reflection Ratio, Open Question Ratio) and a six-dimensional rubric for LLM and human judges.
Dataset Construction: The creation of a dataset comprising 6,000 simulated multi-turn dialogues grounded in 1,000 unique questionnaire-story pairs, covering 12 MI codes and 13 symptom domains.

Experimental Results

The study benchmarked six LLMs (GPT-5-Nano, LLaMA 3.1-8B, Phi-4-14B, OpenChat-7B, Gemma-7B, Qwen 2.5-7B) using the StoryMI framework.

Strategy Adherence: The framework successfully enabled macro-level control. Models like GPT, OpenChat, Gemma, and Qwen achieved Strategy Adherence scores >80%, closely aligning with the target MI distribution (50% reflections, 25% questions, 20% input). All models exceeded the recommended threshold for Complex Reflection Ratio (>50%).
Lexical vs. Clinical Quality: Lexical metrics (Perplexity, Distinct-2) alone failed to predict therapeutic quality. For instance, Phi-4-14B showed high lexical diversity but low Strategy Adherence, whereas Qwen demonstrated high Reflection Depth and Complex Reflection Ratios despite moderate lexical scores.
Ablation Studies: Removing MI codes caused a significant drop in Strategy Adherence (from 80.9% to 64.2%), confirming that interaction-level coding is essential for macro-control. Removing situational stories reduced Code Entropy, indicating that narrative grounding enriches behavioral diversity.
Human Alignment: Human experts and LLM judges showed strong agreement on "Depth" and "Progress" but significant divergence on "Naturalness," "Empathy," and "Adherence." Notably, models with higher Open Question Ratios received higher human Empathy ratings, validating the MI principle that open questions signal an accepting therapeutic stance.

Significance and Limitations

The paper posits that structured multi-agent coordination enables the generation of therapeutically coherent dialogues that are both situationally grounded and strategically controllable. The work demonstrates that MI-style quality can be shaped by controlling dialogue strategies rather than relying solely on model scale.

The authors emphasize that their evaluation framework provides a foundation for future research, particularly in extending controllable generation to multi-modality therapies and validating systems in real-world clinical training settings. They acknowledge limitations, including the reliance on DSM-5 Western diagnostic norms, the lack of interaction with actual clients, and the need for longitudinal studies to establish causal links between specific MI strategies and therapeutic outcomes. The work is presented as a research tool for simulation and training, not a direct replacement for qualified mental health professionals.