Perception of Safety in Behavioral Health Crisis Units among Patients and Care Partners versus Artificial Intelligence (AI): A Multimethod Study

This multimethod study reveals that while perceived safety significantly influences patient and care partner facility selection in behavioral health crisis units, there are notable discrepancies between human and AI-identified environmental risks, underscoring the value of integrating AI tools to support safer decision-making while acknowledging their current limitations in capturing nuanced human perceptions.

The Gist

Imagine you are looking for a new home for a family member who is going through a very tough emotional storm. You want a place that feels safe, calm, and secure. Now, imagine that instead of just looking at a house, you are looking at a specialized "emotional shelter" called a Behavioral Health Crisis Unit (BHCU). These are places designed to help people when they are in deep crisis, but they come with their own unique dangers—like hidden spots where someone could accidentally hurt themselves.

This study is like a detective story comparing two different ways of judging how safe these shelters are: Human Intuition (what patients and their families feel) versus Artificial Intelligence (a super-smart computer program).

Here is the breakdown of the story:

1. The Setup: The "Safety Menu"

The researchers created a giant online game. They showed people pictures of different crisis units, like a menu of options. Some pictures looked very safe and open; others had hidden dangers, like sharp corners or places where a rope could be tied (called "ligature points").

They asked two groups of people:

• The Humans: Patients and their care partners (families/friends).
• The AI: A computer program trained by safety experts to spot dangerous spots in photos.

2. The Human Choice: "I Don't Like That Look"

When the humans were asked to pick a place, they acted like sensitive smoke detectors. If they saw even a few "danger signs" in the picture, they immediately crossed that option off their list.

• The Metaphor: Think of safety risks like pebbles in a shoe. One pebble might be annoying, but a handful of pebbles makes the shoe impossible to wear. The study found that the more "pebbles" (risks) they saw in the photo, the less likely they were to choose that facility. It didn't matter how good the staff was; if the room looked dangerous, they walked away.

3. The AI vs. Human Showdown: The "Flashlight" Test

Next, the researchers asked both the humans and the AI to point out exactly where the dangers were in the pictures. They used a "heatmap" (a glowing map showing where people looked) to compare notes.

• The Good News: The AI and the humans were often on the same page. Like two friends looking at a map, they both spotted the big, obvious traps.
• The Twist: The humans saw things the AI missed.
  • The Metaphor: Imagine the AI is a high-tech metal detector. It beeps loudly when it finds metal (obvious dangers). But the humans are like experienced hikers. They don't just look for metal; they look at the texture of the ground. They noticed that a certain type of shiny plastic felt unsafe, or that a door handle looked like it could be used in a weird way, even if the AI's "metal detector" didn't beep. The humans were picking up on "vibes" and subtle details that the computer hadn't been taught to see yet.

4. The Big Lesson: Teamwork Makes the Dream Work

The study concluded that while the AI is a powerful tool, it isn't perfect. It's like having a brilliant navigator who knows the map perfectly but doesn't know how the car feels to drive.

The researchers suggest that we shouldn't choose between the human and the AI. Instead, we should mix them together.

• The Analogy: Think of designing a safe room like building a fort. The AI is the engineer who checks the blueprints for structural flaws. The human is the resident who says, "This wall feels cold and scary," or "That window looks like it could be broken."

In short: People choose safe places based on how safe they feel. Computers are getting better at spotting physical dangers, but they still need human intuition to catch the subtle, "gut feeling" risks that make a place truly safe for everyone.

Technical Summary

Technical Summary: Perception of Safety in Behavioral Health Crisis Units (BHCUs)

1. Problem Statement

Behavioral Health Crisis Units (BHCUs) are critical environments where patients in acute distress are treated. A primary challenge in these facilities is environmental safety, specifically the presence of ligature points and other hazards that can lead to self-harm or injury to others.

Despite the known physical risks, there is a significant research gap regarding:

• How perceived safety (subjective user experience) influences facility selection among patients and their care partners.
• The degree of alignment between human perception of safety risks and Artificial Intelligence (AI)-driven objective risk assessments in these environments.
• Whether current AI tools can fully capture the nuances of safety that influence human decision-making.

2. Methodology

The study employed a multimethod approach combining quantitative survey data with computer vision analysis.

A. Data Collection: Discrete Choice Experiment (DCE)

• Format: A nationwide online survey utilizing image-based scenarios of BHCU environments.
• Participants: Patients and care partners.
• Task: Participants were presented with facility images varying in attributes (including environmental safety risks like ligature points) and asked to select their preferred facility.
• Annotation Task: Participants were asked to identify and mark specific safety risks within the provided images.

B. AI-Driven Risk Assessment

• Tool: An AI-driven tool, trained and validated by domain experts, was used to detect environmental risks in the same survey images.
• Output: The AI generated risk detection outputs and heatmaps indicating the location and probability of hazards.

C. Analytical Framework

1. Quantitative Analysis: Conditional logit models were used to determine the statistical influence of specific attributes (e.g., frequency of safety risks) on the probability of facility selection.
2. Spatial Analysis: A comparative analysis was conducted between the heatmaps generated by human participants (via their annotations) and the heatmaps generated by the AI to assess alignment and divergence in risk perception.

3. Key Contributions

• Integration of Human and AI Perception: This study is among the first to directly compare subjective safety perceptions of non-expert users (patients/care partners) against objective AI-driven risk assessments in a clinical setting.
• Quantification of Safety Impact: It provides empirical evidence quantifying how specific environmental risk frequencies directly correlate with facility preference.
• Identification of AI Limitations: The research highlights specific gaps where AI fails to capture human safety concerns, such as material types and "out-of-label" risks not explicitly defined in the training data.

4. Key Results

• Impact of Perceived Safety on Choice: There is a statistically significant negative correlation between the frequency of safety risks in an image and the likelihood of that facility being selected.
  • Statistical Finding: Higher frequency of safety risks reduced selection likelihood with p < .001 and an Odds Ratio (OR) of approximately 1.28.
• Human-AI Alignment:
  • Convergence: There was notable alignment between the heatmaps generated by participants and the AI, suggesting AI can effectively identify common, high-visibility hazards (e.g., obvious ligature points).
  • Divergence: Significant differences emerged where human perception flagged risks that AI missed. These included:
    • Specific material types (e.g., texture or finish that feels unsafe).
    • Unknown or out-of-label risks that were not part of the AI's training schema but were intuitively recognized by humans.

5. Significance and Implications

• Decision Support: The findings validate the utility of integrating AI-driven assistive tools into the decision-making process for patients and care partners. These tools can help non-experts identify potential risks they might otherwise overlook.
• Design and Policy: The study suggests that while AI is a powerful tool for objective risk detection, it cannot fully replace human intuition regarding environmental safety. Facility design must account for both objective hazards and the subjective feeling of safety (e.g., material choices).
• Future AI Development: To be effective in clinical settings, AI models must evolve to include "out-of-label" risk detection and better understand contextual nuances like material safety, moving beyond simple object detection to holistic environmental assessment.

In conclusion, the study demonstrates that while AI can effectively map known environmental risks, the perception of safety is a complex human construct influenced by factors beyond current algorithmic capabilities. A hybrid approach, leveraging AI for data-driven insights while respecting human perceptual nuances, is essential for creating safer BHCU environments.