Technical Summary: DeepSAGE

Problem Statement

While Large Language Models (LLMs) have demonstrated fluency and supportiveness in generating counseling responses, they frequently lack the structured, goal-directed progression necessary for coherent therapeutic sessions. Existing systems often rely on rule-based scripts, shallow NLP pipelines, or client-led interactions that fail to guide users through a psychologically grounded process. Although Cognitive Behavioral Therapy (CBT) offers a structured methodology, current LLM-based CBT systems typically focus on individual dialogue turns or isolated components rather than managing an entire session as a structured, stage-based process. Consequently, reliable, session-level therapeutic guidance remains largely unexplored.

Methodology

The authors propose DeepSAGE (Strategic AI Guidance Engine), a hybrid framework combining LLMs with Deep Reinforcement Learning (DRL) to enable stage-aware counseling dialogue grounded in the first session of CBT.

1. Stage-Structured Framework

The counseling session is modeled as a sequence of eleven distinct stages derived from clinical literature (Beck, 2020), each with explicit therapeutic objectives:

1. Greet
2. Set Agenda
3. Mood Check
4. Obtain Updates
5. Discuss Diagnosis
6. Identify Problems and Purposes
7. Educate About Cognitive Model
8. Apply Cognitive Model to a Client Problem
9. Elicit Summary
10. Review Homework
11. Elicit Feedback

2. Stage Transition Mechanism

An external controller determines when to advance to the next stage ($S_{i+1}$) based on a quantitative success criterion $G(S_i, r_t)$. This score is computed after each client reply ($r_t$) and combines:

• Semantic Similarity: Cosine similarity between the client's reply and the stage goal description.
• NLI Entailment: Probability that the reply entails the stage goal, ensuring the client actually satisfies the objective rather than just talking around it.
  A stage is considered complete if $G(S_i, r_t) \geq \tau$ (threshold) or if the maximum number of turns ($T_{max}$) for that stage is reached.

3. Hybrid LLM-DRL Architecture

When a stage is not yet complete, a DRL agent selects a therapeutic intention to guide the LLM's response generation.

• State ($s_t$): Composed of a semantic embedding of recent dialogue history ($h_t$) and a contextual vector ($c_t$) encoding the current stage index and turn count.
• Action Space: Seven therapeutic intentions adapted from Hill and O'Grady (2001): Support, Encourage Catharsis, Clarify, Focus, Identify Feelings, Identify Maladaptive Cognitions, and Normalize Experience.
• Reward Function ($R_t$): A modular function balancing:
  • Relevance ($Rel_t$): Coherence with client input and alignment with stage goals.
  • Self-Disclosure Count ($SDC_t$): Encouraging client engagement via first-person pronouns and personal entity mentions.
  • Stage Turn Penalty ($-\lambda T_t$): Discouraging excessive looping to promote timely session completion.
• Training: The policy is optimized using Proximal Policy Optimization (PPO).

4. Evaluation Setup

The system was evaluated using LLM-simulated clients (Anxiety Disorder and Major Depressive Disorder) in a controlled environment. Baselines included retrieval-augmented bots, standard LLMs, and various prompting strategies (LLM4CBT, Full-Protocol, Stage-Prompt).

Key Contributions

1. First Stage-Structured Framework: The authors propose a framework modeling an initial CBT session as eleven stages with explicit objectives and automated stage detection for dialogue management.
2. Hybrid Architecture: Development of an LLM-DRL architecture where a DRL policy selects therapeutic intentions to guide LLM generation, enabling structured progress while preserving conversational flexibility.
3. Therapeutically Grounded Optimization: Design of an action space and reward function that jointly optimize client engagement, semantic relevance, stage progression, and completion.
4. Empirical Demonstration: Evidence that DeepSAGE outperforms retrieval, prompting, and non-DRL stage-structured alternatives in simulated engagement, self-disclosure, and session efficiency.

Results

In simulated evaluations involving 100 sessions per condition:

• Engagement and Disclosure: DeepSAGE achieved the highest User Utterance Length (UL) and Self-Disclosure Count (SDC) for both Anxiety Disorder (AD) and Major Depressive Disorder (MDD) clients. For MDD, it improved UL by ~59.9% and SDC by ~80.8% compared to the strongest non-DeepSAGE baseline (LLM4CBT).
• Emotional Intensity Drop (EID): DeepSAGE demonstrated strong emotional relief, achieving the highest EID for AD clients and a balanced outcome for MDD (combining distress reduction with high engagement, unlike standard LLMs which showed high EID but low engagement).
• Session Efficiency: DeepSAGE achieved the highest Session Success Rate (SSR) while requiring the fewest counselor utterances. Compared to the ablation variant DeepSAGE_R (random action selection), DeepSAGE improved SSR by ~10.9% (AD) and ~8.6% (MDD) while reducing counselor turns by ~9.3% and ~7.0%, respectively.
• Expert Review: Domain experts rated the generated conversations as broadly plausible with recognizable CBT processes, though some noted occasional over-reliance on questions or premature solutioning.

Significance and Limitations

The paper claims that combining stage-structured dialogue with learned strategy selection is a promising approach for AI counseling, demonstrating comparative improvements in dialogue control, engagement, and session efficiency. The results suggest that a learned policy is superior to static prompting or random action selection in navigating the complex dynamics of a CBT session.

However, the authors maintain a modest stance regarding clinical validity:

• Simulated Nature: The evaluation relies primarily on simulated clients and model-based metrics. The findings demonstrate comparative dialogue-control improvements rather than proven clinical effectiveness.
• Safety Constraints: The current action space lacks dedicated crisis escalation or referral mechanisms. Safety stress tests indicate the system selects "Identify Maladaptive Cognitions" for high-risk inputs, which is not an appropriate clinical response for acute crises.
• Deployment Status: DeepSAGE is a research prototype, not a licensed clinical tool. The authors explicitly state it should not be deployed with real users without licensed clinician oversight, dedicated crisis detection, and rigorous human evaluation of safety and efficacy.

The work positions DeepSAGE as a step toward human-AI workflows where structured dialogue support is combined with clinician oversight, rather than a replacement for mental health professionals.