💬 NLP

PsychoSafe: Eliciting Psychologically-Informed Refusals in Large Language Models

The paper introduces PsychoSafe, a psychologically-informed framework that improves LLM refusal quality by reframing non-compliance as structured, supportive communication grounded in evidence-based intervention strategies, achieving significant gains in resource referral and psychological grounding while maintaining task relevance.

Original authors: Gianluca Barmina, Federico Torrielli, Sven Harms, Jacob Nielsen, Felix Mächtle, Stine Lyngsø Beltoft, Peter Schneider-Kamp, Thomas Eisenbarth, Lukas Galke Poech, Anne Lauscher

Published 2026-06-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Gianluca Barmina, Federico Torrielli, Sven Harms, Jacob Nielsen, Felix Mächtle, Stine Lyngsø Beltoft, Peter Schneider-Kamp, Thomas Eisenbarth, Lukas Galke Poech, Anne Lauscher

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart, helpful robot assistant. Usually, when someone asks this robot for something dangerous—like "How do I make a bomb?" or "I want to hurt myself"—the robot's safety programming kicks in. It simply says, "No. I cannot do that."

While this is safe, it can feel cold, blunt, and lonely, especially if the person asking is in a crisis. It's like a doctor telling a patient, "No surgery," and walking away without offering any comfort or next steps.

The paper PSYCHOSAFE proposes a new way for these robots to say "No." Instead of just slamming the door, the robot learns to hold the door open just a crack, offer a warm hand, and point the person toward a lifeline.

Here is the breakdown of their approach, using simple analogies:

1. The Problem: The "Brute Force" Refusal

Currently, AI safety is like a bouncer at a club who only has one move: The Block. If you look suspicious, you get stopped. It works to keep bad things out, but it doesn't help the person who is actually in trouble. If someone is in a crisis, a blunt "No" might stop them from getting immediate harm, but it doesn't stop the pain or guide them to help.

2. The Solution: The "Crisis Counselor" Mode

The researchers built a framework called PSYCHOSAFE. Think of this as giving the robot a new "personality" based on real human psychology. Instead of just blocking, the robot is trained to act like a crisis counselor who has been taught specific, proven techniques (like "Psychological First Aid" or "Motivational Interviewing").

When a user asks a dangerous question, the robot doesn't just say "No." It follows a four-step script, like a recipe:

The Warm Hug: It acknowledges the person's feelings ("I hear you are in pain") without agreeing to the dangerous request.
The Gentle Nudge: It offers a small, safe step the person can take right now (like "Take a deep breath" or "Try to change your location").
The Map: It points to real-world help (like a suicide hotline or a drug rehab center).
The Hopeful Goodbye: It ends with a message of hope, reminding the person they matter.

3. How They Taught the Robot

To teach the robot this new way of speaking, the team did two things:

Created a New Textbook: They wrote 8,019 examples of these "helpful refusals." They covered five specific "danger zones": Suicide/Self-harm, Sexual Crimes, Drug Use, Weapons, and Violence. They made sure every example was grounded in real psychological science.
Two Ways to Learn:
- The Cheat Sheet (Prompting): They gave the robot a long, detailed instruction manual (a "system prompt") every time it started talking. This told it, "Remember, if someone is in crisis, use the 4-step counselor script."
- The Brain Surgery (Fine-Tuning): They actually rewired the robot's brain by training it on their new textbook. This way, the robot became the counselor naturally, without needing the instruction manual every time.

4. The Results: Did It Work?

They tested the robot on 500 difficult questions and had a "judge" (another AI and human experts) grade the answers.

The "Cheat Sheet" worked best overall: When they used the instruction manual, the robot's answers became 28% better at being helpful and safe. It got much better at pointing people to real resources (up 46%) and sounding psychologically grounded (up 34%).
The "Brain Surgery" made it a safety machine: The trained robot refused dangerous requests almost 100% of the time and always offered resources. However, it sometimes got a bit too robotic and generic, forgetting to listen to the specific details of the person's story.
It didn't break the robot: The robot didn't get "dumber" at other tasks. It could still write stories, solve math problems, and answer general questions just as well as before.

5. The Catch (Limitations)

The paper warns that this new "counselor" mode is very good at what it was trained for, but it's not a magic wand.

It's specialized: It works great for the five specific danger zones they trained it on (like suicide or violence). If you ask it about something outside those areas, it might not know how to apply the same gentle logic.
It's not a real doctor: The robot is still an AI. It can offer comfort and resources, but it cannot diagnose mental illness or provide actual therapy. The paper emphasizes that this is a bridge to human help, not a replacement for it.

The Bottom Line

PSYCHOSAFE shows that we don't have to choose between a robot that is "safe" and a robot that is "helpful." By teaching AI to use the same gentle, supportive techniques that human counselors use, we can make AI refusals feel less like a wall and more like a hand reaching out to help.

Technical Summary: PSYCHOSAFE

Problem Statement

Large Language Models (LLMs) face a fundamental tension between maximizing helpfulness and preventing harm. Current safety mechanisms typically treat refusal as a binary classification problem: a model either complies with a request or issues a blunt, non-compliant denial. While this prevents direct harm, the authors argue that such "blunt non-compliance" often fails to support the underlying needs of users in high-risk situations involving crisis, coercion, or escalating intent. A simple refusal may stop immediate harm but can leave the user without support, potentially exacerbating distress. The paper posits that refusal itself can be a form of helpful communication if it is grounded in psychological theory, transforming a denial into a structured, supportive intervention.

Methodology

1. The PSYCHOSAFE Framework

The authors operationalize "psychologically-informed refusals" by integrating established principles from counseling psychology, motivational interviewing, and crisis communication. The framework is built on three core stages:

Risk Harmonization: The authors aggregated 16 public safety benchmarks and harmonized their heterogeneous taxonomies using semantic clustering (via sentence-t5-large embeddings and HDBSCAN). This process identified five specific risk clusters where psychological intervention is most appropriate:
1. Suicide and Self-Harm: Requiring de-escalation and crisis resource referral.
2. Sexual Crimes: Requiring emphasis on consent and victim protection.
3. Substance Use: Aiming to encourage reflection and help-seeking.
4. Weapons: Prioritizing interruption of harmful action and consequence reflection.
5. Violence: Focusing on de-escalation and redirection.
Intervention Principles: The framework adapts evidence-based strategies (e.g., Psychological First Aid, Motivational Interviewing, Safety Planning) into a reusable refusal structure. Every response follows a four-part template:
1. Warm acknowledgment paired with a gentle refusal.
2. A personalized self-help step grounded in the specific risk cluster's intervention strategy.
3. Referral to relevant external professional resources (e.g., helplines).
4. A brief, hopeful closing tied to the user's context.
Dataset Construction: The authors constructed the PSYCHOSAFE dataset, comprising 8,019 prompt–response pairs. These pairs map prompts from the five risk clusters to responses generated by human experts following the intervention template. The dataset includes metadata regarding the risk cluster and provenance.

2. Elicitation Strategies

The study evaluates two methods for inducing this behavior in the Qwen 3.5 27B model:

In-Context Learning (Prompting): A dedicated system prompt (v1) encodes the full PSYCHOSAFE framework, instructing the model to follow the four-part structure, personalize responses based on user cues, and prioritize acute risks (e.g., suicide) if multiple clusters are detected.
Supervised Fine-Tuning (SFT): The model is fine-tuned on the PSYCHOSAFE dataset augmented with "reasoning traces." These traces were generated by a larger model (GPT-OSS-120B) to simulate the intermediate reasoning process before the final refusal. The SFT model is trained to internalize the refusal behavior and the associated reasoning, aiming to reproduce the behavior without relying on a long system prompt.

3. Evaluation

The authors evaluated both approaches on a held-out validation set of 500 prompts (100 per risk cluster) using an LLM-as-a-judge pipeline (selected as Qwen 3.5 397B-A17B based on highest agreement with human raters, $\kappa = 0.61$ ). The evaluation criteria were:

Refusal: Binary check for non-compliance.
Psychological Grounding: Score (1–5) on the use of empathetic validation and intervention strategies.
External References: Binary check for resource referral.
Relevance: Score (1–5) on how well the response is tailored to the specific user context.

Additional evaluations were conducted on SORRY-Bench (adversarial robustness), XSTest (over-refusal and adversarial safety), and general capability benchmarks (MMLU, HellaSwag).

Key Results

Refusal Quality

Prompting (v1): The PSYCHOSAFE prompting strategy improved overall refusal quality by 28.1% over a generic baseline (v0). The most significant gains were in External Resource Referral (+46.8%) and Psychological Grounding (+34.8%).
Fine-Tuning (SFT): The fine-tuned model achieved near-perfect refusal and resource-referral rates (approaching 100%). However, it showed a significant drop in Relevance (-25% to -28%) compared to the prompting approach. Qualitative analysis indicated that the SFT model often applied a generic crisis-intervention template too broadly, failing to adapt to specific user contexts or misinterpreting benign prompts as acute crises.
Stability: The fine-tuned model was nearly indifferent to the system prompt version, whereas the base model's performance varied significantly between v0 and v1.

Safety and Robustness

SORRY-Bench: The PSYCHOSAFE fine-tuned model achieved near-zero compliance (0.0–0.1%) across all prompt conditions and 19 linguistic mutations, significantly outperforming the base model.
XSTest (Over-refusal): The fine-tuned model maintained low over-refusal rates on benign prompts (3.6%–9.2%), ruling out indiscriminate refusal. However, its safety rate on out-of-domain adversarial prompts was lower than the base model (17.0% vs. 59.0%), suggesting limited generalization beyond the five training domains.
General Capabilities: Performance on MMLU and HellaSwag remained largely stable, with only modest decreases (approx. 4–5 percentage points) observed in some configurations, which the authors deem acceptable given the safety gains.

Significance and Claims

The paper claims that helpfulness and harm prevention are not necessarily opposing objectives. By drawing on established human intervention practices, LLMs can refuse unsafe requests while simultaneously providing supportive, constructive responses.

Theoretical Contribution: The work bridges AI alignment research with established psychological intervention strategies, arguing that refusal design should be grounded in psychological theory to ensure generalizability and coherence as models evolve.
Practical Contribution: It demonstrates that psychologically-informed refusals can be elicited via prompting or embedded into model weights via fine-tuning.
Limitations and Scope: The authors modestly claim that PSYCHOSAFE is best suited for domain-specific deployments (e.g., mental health or crisis-intervention platforms) where risks are known and high-quality, supportive refusals are critical. They acknowledge that the current approach lacks broad out-of-domain generalization and that the fine-tuned model sometimes lacks the pragmatic judgment to apply interventions selectively. The authors explicitly state that PSYCHOSAFE is not a substitute for professional mental health intervention and emphasize the need for extensive pre-deployment stress testing.