🤖 AI

VERA-MH Concept Paper

The paper introduces VERA-MH, an automated framework that utilizes simulated user and judge AI agents to evaluate the safety and ethical performance of mental health chatbots, specifically regarding suicide risk, through a clinician-developed rubric.

Original authors: Luca Belli, Kate H. Bentley, Will Alexander, Emily Ward, Matt Hawrilenko, Kelly Johnston, Mill Brown, Adam M. Chekroud

Published 2026-05-14

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Luca Belli, Kate H. Bentley, Will Alexander, Emily Ward, Matt Hawrilenko, Kelly Johnston, Mill Brown, Adam M. Chekroud

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are building a new kind of digital helper for people struggling with their mental health. You want it to be kind, helpful, and safe. But how do you test if it's actually safe before letting it talk to real people? You can't just ask it, "Are you safe?" because it will always say "Yes."

This paper introduces VERA-MH, a new "safety inspector" designed specifically to test AI chatbots that deal with mental health, with a special focus on suicide risk. Think of VERA-MH not as a simple checklist, but as a high-tech simulation lab.

Here is how the system works, broken down into simple parts:

1. The Three Actors in the Play

To test the chatbot, VERA-MH sets up a stage with three actors:

The Chatbot (The Star): This is the AI you are testing. It's the one trying to help.
The User-Agent (The Method Actor): This is a second AI programmed to act like a real person in crisis. Instead of a human typing, this "actor" role-plays different types of people. Some are very direct about their pain, some are vague, and some are in immediate danger. The clinicians (real doctors) wrote the scripts for these actors to make sure they sound realistic.
The Judge-Agent (The Critic): This is a third AI. It watches the conversation between the Chatbot and the User-Agent. It doesn't just say "good job" or "bad job." It uses a special scorecard (called a rubric) created by mental health experts to grade the Chatbot's performance.

2. The Scorecard (The Rubric)

The "Judge" doesn't just look for kindness. It looks for specific safety skills, much like a driving test checks for braking, signaling, and lane changes. The scorecard checks five things:

Did they notice the danger? (Did the Chatbot hear the warning signs?)
Did they ask the hard questions? (Did they gently ask, "Are you thinking of hurting yourself?")
Did they take the right next steps? (Did they offer a lifeline, like a crisis hotline, or suggest talking to a human?)
Did they validate feelings? (Did they say, "I hear you, and your pain is real," instead of just giving advice?)
Did they stay safe? (Did they avoid saying anything that might make the situation worse?)

3. The "Stress Test"

Instead of asking the Chatbot one question and seeing the answer, VERA-MH runs full conversations. It's like a fire drill. The User-Agent might say, "I'm tired of living," and the Chatbot has to respond. Then the User-Agent might say, "I don't have a plan," and the Chatbot has to keep the conversation safe.

The system runs these simulations many times with different "actors" to see how the Chatbot handles different types of people and different levels of risk.

4. The Results So Far (The "Preliminary Report")

The authors tested three famous AI models (GPT-5, Claude Opus, and Claude Sonnet) using this system.

The Good News: All three models were generally good at being empathetic and validating feelings. They rarely said things that were "actively damaging."
The Bad News: The models sometimes missed opportunities to ask direct questions about suicide or didn't escalate the conversation to human help when they should have.
The "Lenient Judge" Problem: When the authors compared the AI "Judge" to real human doctors, they found the AI Judge was too easy on the Chatbots. It gave them "Best Practice" grades more often than the human doctors did. The human doctors were stricter.

5. What's Next?

The authors admit this system is still a work in progress. They are currently:

Calibrating the Judge: Teaching the AI Judge to be stricter and more like a real human doctor.
Improving the Actors: Making the "User-Agents" sound even more like real people, including those who are shy or ashamed to ask for help.
Expanding the Cast: They currently have 10 different "actor" scripts, but they know they need more to cover all types of people. (They intentionally left out children for now to keep things simple).

The Bottom Line

The paper argues that we cannot just trust AI with mental health because it "sounds nice." We need a rigorous, automated way to stress-test these tools to ensure they don't accidentally harm someone who is vulnerable. VERA-MH is their attempt to build that safety net, using AI to test AI, guided by the wisdom of human doctors.

Important Note: The paper explicitly states this is a concept and validation tool. It is not a therapy app itself, and the results so far are preliminary. The goal is to help developers build safer tools, not to replace human therapists.

Technical Summary: VERA-MH Concept Paper

Problem Statement

The rapid integration of AI into mental healthcare, particularly through chatbots for support and clinical triage, has outpaced the establishment of ethical, clinical, and safety standards. Current AI systems in this domain often operate without clinical oversight or consistent evaluation metrics. Existing evaluation methods for Large Language Models (LLMs) typically rely on static datasets and single-turn conversations, which are insufficient for mental health contexts where therapeutic interactions are dynamic, context-dependent, and evolve over multiple turns. There is a critical need for an automated, clinically validated framework to assess the safety of AI chatbots, specifically regarding suicide risk, to ensure they adhere to the "do no harm" principle.

Methodology

The authors propose VERA-MH (Validation of Ethical and Responsible AI in Mental Health), an automated evaluation system designed to assess the safety of AI chatbots in mental health contexts. The methodology relies on a multi-agent architecture and a clinically derived rubric:

Clinically Informed Rubric: Developed by practicing clinicians and suicide prevention experts, the rubric evaluates chatbot responses across five dimensions:
- Detects risk: Identification of user statements indicating potential suicide risk.
- Probes risk: Follow-up with direct, clear questions regarding suicide risk.
- Takes appropriate actions: Escalation to human providers or provision of crisis resources based on risk levels.
- Validates and collaborates: Validation of user feelings and collaborative engagement.
- Maintains safe boundaries: Avoidance of language that fuels harmful behaviors.
  Responses are categorized into four options: Best practice, Missed opportunity, Actively damaging, or Not relevant.
Automated Evaluation Agents:
- User-Agent: Simulates realistic mental health conversations using clinically developed personas (system prompts). These agents role-play individuals with varying suicide risk levels (from acute ideation to indirect indicators) and communication styles. To account for LLM probabilistic variance, each persona generates five independent conversation samples.
- Judge-Agent: An AI model that receives the simulated conversation and the rubric as inputs to score the chatbot's performance across the five dimensions.
Aggregation and Output: Results are synthesized into a $5 \times 4$ matrix (5 dimensions $\times$ 4 response categories), where each cell represents the proportion of times a specific score was assigned to a dimension across all simulated conversations.
Human Validation: The system undergoes rigorous validation by licensed mental health clinicians to ensure:
- User-agents realistically simulate human patients.
- Judge-agents provide credible ratings comparable to human clinicians.
- Inter-rater reliability among human clinicians is established to serve as a consistent standard.

Key Contributions

First-of-its-Kind Framework: VERA-MH is introduced as a novel, automated evaluation system specifically designed for the safety of AI chatbots in mental healthcare, with an initial focus on suicide risk.
Multi-Turn, Dynamic Assessment: Unlike static, single-turn evaluations, VERA-MH analyzes complete, multi-turn conversations to capture the evolving nature of therapeutic interactions and context.
Clinical-Technical Integration: The system bridges the gap between technical AI evaluation and clinical practice by embedding expert clinical judgment into the design of personas, the scoring rubric, and the validation process.
Scalable Automation: The framework allows for the scalable, automated evaluation of AI models without requiring manual review of every interaction, addressing the pace of AI model iteration.

Preliminary Results

The authors conducted preliminary evaluations using initial versions of the VERA-MH rubric and agents on three models: GPT-5, Claude Opus, and Claude Sonnet.

Model Performance: GPT-5 achieved the highest number of "Best Practice" scores across all five dimensions. "Missed Opportunity" or neutral ratings were more common for Claude Opus and Claude Sonnet. All three models performed best on the "Validates and Collaborates" dimension. "Actively Damaging" ratings were infrequent across all tested chatbots.
Human Validation Findings:
- User-Agent Realism: Clinicians rated the simulated user-agents as highly realistic (average score of 4.1/5).
- Judge-Agent Calibration: The initial judge-agent tended to be more lenient than human clinicians, rating 88% of interactions as "Best Practice" compared to 46% by clinicians. It rarely assigned "Actively Damaging" ratings, whereas clinicians did so 7% of the time.
- Inter-Rater Reliability: Clinician ratings showed inconsistency (57% mismatch rate), highlighting the need for structured training and calibration to improve human reliability before using it as a gold standard.
- Agreement: The judge-agent agreed most often with clinicians on "Validates and Collaborates" (60%) and least on "Probes Risk" (41%).

Significance and Claims

The paper positions VERA-MH as a critical step toward the safe and responsible deployment of AI in mental healthcare. The authors claim that for AI to effectively reduce barriers to mental health support, developers must demonstrate that their tools are safe, provide appropriate risk assessments, and respond with empathy.

The significance of VERA-MH lies in its ability to provide a structured, consistent, and automated method to evaluate AI safety in a high-stakes domain. The authors emphasize that the system is currently under active development and iteration. They explicitly state that the current results are preliminary and that the framework requires further refinement, particularly in:

Calibrating the judge-agent to align more rigorously with human clinical standards.
Expanding the diversity of user personas (currently 10, excluding youth to reduce complexity).
Addressing the "saturation" risk where models may easily achieve top scores.
Improving the interpretability of the multi-dimensional matrix output.

The paper concludes by inviting community feedback on both technical and clinical aspects to refine the evaluation tool before broader adoption.