Runtime Anomaly Detection and Assurance Framework for AI-Driven Nurse Call Systems

This research proposes a reproducible, lightweight, and interpretable runtime anomaly detection framework for AI-driven Nurse Call Systems that utilizes an Isolation Forest model to ensure operational safety and reliability in resource-constrained medical environments without relying on complex deep learning models.

The Gist

🏥 The Problem: The "Smart" Nurse Call System is Getting Tired

Imagine a hospital where every patient has a button to call a nurse. In the old days, it was just a simple light and a buzzer. But now, hospitals are using Artificial Intelligence (AI) to manage these calls. The AI tries to be smart: it decides which nurse is closest, prioritizes urgent calls, and balances the workload so no one gets overwhelmed.

But here's the catch: What happens when the AI gets confused, glitches, or starts making mistakes?

If the AI suddenly decides to ignore a call, or if it takes 10 minutes to alert a nurse for a heart attack, that's a disaster. The paper argues that while we are building these "smart" systems, we aren't building a good enough security guard to watch them while they work. We need a way to catch the AI when it starts acting weird in real-time, before a patient gets hurt.

🕵️‍♂️ The Solution: The "Shadow Cop" Framework

The researchers (Yuanyuan Liu and David Concepcion) built a Runtime Anomaly Detection Framework.

Think of the AI Nurse Call System as a busy chef in a kitchen.

• The Goal: The chef needs to cook meals fast and perfectly.
• The Risk: Sometimes the chef gets distracted, drops a pan, or forgets an order.
• The Solution: They placed a "Shadow Cop" (their framework) in the kitchen. This cop doesn't cook; they just watch. If the chef starts dropping pans too often or takes too long to answer the phone, the Shadow Cop immediately raises a red flag.

🛠️ How It Works (The Magic Trick)

The researchers didn't try to build a new super-smart AI to replace the old one. Instead, they built a lightweight, simple watchdog. Here is how they tested it:

1. The Simulation (The Dress Rehearsal):
   Since they couldn't risk messing up a real hospital, they created a video game version of a hospital. They generated thousands of fake nurse calls.

   • Analogy: Imagine a flight simulator. They didn't crash a real plane; they crashed the simulator to see if the alarm system worked.

2. Injecting the Chaos (The "What Ifs"):
   They deliberately broke the simulation to see if the watchdog would catch it. They created specific "glitches":

   • The Slowpoke: A call that takes 90 seconds to get answered (when it should take 10).
   • The Stutter: A patient pressing the button 10 times in 30 seconds because no one came.
   • The Ghost: A call that disappears from the log entirely.
   • The Panic: 50 calls coming in at once (an emergency).

3. The Detective (Isolation Forest):
   To catch these glitches, they used a specific tool called an Isolation Forest.

   • Analogy: Imagine a forest of trees. Most trees are normal. But if you have a tree that is shaped like a question mark, or is floating in the air, it's easy to spot because it's "isolated" from the rest. The Isolation Forest algorithm does exactly this: it looks for the "question mark trees" (the weird data) in the forest of normal calls.

4. The Dashboard (The Control Panel):
   The results aren't hidden in a computer code. They are shown on a live, interactive dashboard (like a car's dashboard). If the system detects a problem, the dashboard lights up, showing exactly what is wrong and how bad it is.

🏆 Why This is Special (The "Secret Sauce")

Most AI safety systems are like heavy tanks: they are powerful, but they are slow, expensive, and need massive amounts of data to learn. You can't put a tank on a small, battery-powered device in a hospital room.

This framework is like a smartwatch:

• Lightweight: It runs easily on small computers (edge devices).
• Interpretable: It doesn't just say "Error." It says, "Hey, the response time is too slow." Doctors and nurses can actually understand why it flagged the issue.
• No Secret Data: It doesn't need to see private patient records to learn what "normal" looks like. It learns from the patterns of the calls themselves.

📊 The Results: Did It Work?

They tested their "Shadow Cop" against the "glitches" they created:

• It caught the delays: When calls took too long, the system sounded the alarm almost every time (High Recall).
• It didn't cry wolf too much: It was pretty good at ignoring normal busy times and only flagging real problems (Good Precision).
• It's ready for the real world: Because it's simple and fast, it can be installed on the actual hardware hospitals use right now without slowing anything down.

🔮 What's Next?

The researchers admit this is just the beginning.

• Current Limitation: They used fake data (the video game hospital).
• Future Goal: They want to test this in a real hospital with real nurses and real patients. They also want to make the system even smarter by adding "context" (e.g., "It's 3 AM and it's raining, so calls are usually slower, so don't panic yet").

💡 The Big Takeaway

This paper isn't about building a smarter AI doctor. It's about building a reliable safety net for the AI doctors we already have. It's about making sure that when the technology is working, we know it's working, and when it starts to fail, we know immediately so we can step in and save the day.

In short: They built a simple, fast, and explainable alarm system to make sure our AI-powered hospitals don't accidentally drop the ball.

Technical Summary

1. Problem Definition

Context: Nurse Call Systems (NCS) are critical safety infrastructure in hospitals, facilitating communication between patients and clinical staff. Modern NCS increasingly integrate Artificial Intelligence (AI) and Internet of Things (IoT) for features like automated prioritization and workload balancing.
The Gap: While AI improves efficiency, there is a lack of runtime assurance mechanisms for these systems once deployed. Existing systems often fail to detect safety-relevant deviations (e.g., excessive response delays, missed calls, or silent failures) in real-time.
Constraints: Traditional assurance methods often rely on complex deep learning models, large labeled datasets, or offline validation, which are unsuitable for resource-constrained, real-time clinical environments that require low latency, interpretability, and transparency.
Research Question: How can lightweight, interpretable runtime monitoring and anomaly detection be used to improve the safety and reliability of AI-enabled NCS during live operation without relying on proprietary data or complex models?

2. Methodology

The authors propose a reproducible, lightweight runtime assurance framework consisting of four modular components:

• Data Generation & Simulation:
  • Instead of using sensitive proprietary clinical data, the study generates simulated nurse call logs based on cleaned NYC 311 service request data (over 10,000 records).
  • Anomaly Injection: Clinically motivated anomalies are systematically injected into the logs to simulate real-world safety risks. These include:
    • Delayed Response: Response time > 90 seconds.
    • Repeated Calls: Multiple calls from the same room within 30 seconds.
    • Missing Responses: Removal of expected log entries.
    • Invalid Timestamps: Inconsistent event ordering.
    • Unusual Call Volume: Bursts of 10+ calls per minute.
• Anomaly Detection Models:
  • Primary Model: Isolation Forest (Unsupervised). Selected for its computational efficiency, ability to handle tabular data, and suitability for detecting outliers without labeled training data.
  • Baselines for Comparison:
    • Rule-based: Flags delays exceeding a fixed threshold.
    • Supervised: Random Forest (RF) and One-Class SVM (OCSVM).
    • Deep Learning: Autoencoder (AE) for comparison.
• Evaluation Strategy:
  • Metrics: Precision, Recall, F1-score, and Precision-Recall (PR) curves (preferred over ROC due to class imbalance in safety alerts).
  • Operational Metrics: Detection delay and hourly alarm frequency to assess "alert fatigue."
  • Interpretability: SHAP (SHapley Additive exPlanations) values are used to analyze feature contributions.
  • Threshold Sensitivity: A sweep of anomaly score thresholds is performed to identify optimal operating points balancing detection sensitivity and false alarms.

3. Key Contributions

1. Operational Assurance Framework: A novel framework specifically tailored for AI-enabled NCS that prioritizes runtime monitoring over offline model training.
2. Lightweight & Interpretable: Demonstrates that high-reliability assurance can be achieved using simple, unsupervised models (Isolation Forest) rather than complex deep learning, ensuring the system is deployable on edge devices and understandable by clinical staff.
3. Reproducibility & Open Science: The authors released all code, cleaned datasets, experiment scripts, and an interactive Streamlit dashboard (allowing users to upload custom CSVs) as open research artifacts (Zenodo DOI provided).
4. Clinically Motivated Simulation: A robust simulation methodology that maps specific injected anomalies to real-world clinical scenarios (e.g., staffing overload, device faults) without compromising patient privacy.

4. Experimental Results

• Model Performance:
  • Random Forest (Supervised): Achieved the highest performance (Precision ≈ 1.00, Recall ≈ 0.98) but is limited by the need for labeled data.
  • Isolation Forest (Unsupervised): Demonstrated robust performance with a favorable balance between precision and recall, making it the most suitable candidate for real-world deployment where labeled data is scarce.
  • One-Class SVM & Autoencoder: Showed lower accuracy or higher complexity/tuning overhead compared to Isolation Forest.
• Feature Importance (SHAP): Analysis confirmed that response time and temporal features (hour of day, day of week) are the dominant predictors of anomalies, aligning with operational intuition regarding workload fluctuations.
• Threshold Analysis: Sensitivity analysis identified an optimal operating point (threshold ≈ -0.08) that maximizes the F1-score, providing an evidence-based method for configuring alarm sensitivity to mitigate alert fatigue.
• Recall on Critical Anomalies: The framework showed a high recall rate specifically for delay-type anomalies, which are critical for patient safety.

5. Significance and Future Work

Significance:

• Safety-Critical Deployment: The work addresses a critical gap in healthcare AI by shifting focus from "model accuracy" to "runtime safety assurance."
• Trust & Auditability: By using interpretable models and providing clear visualizations, the framework supports clinical trust and regulatory auditability.
• Resource Efficiency: The low computational overhead makes it feasible for deployment on edge devices within hospital infrastructure.

Limitations & Future Directions:

• Data Source: Current evaluation relies on simulated data derived from public service requests; real-world validation with actual hospital NCS data is needed.
• Feature Scope: The current feature set is intentionally limited for deployability. Future work will incorporate temporal context (sliding windows, historical patterns) and unit-level workload indicators.
• Explainability: Further integration of advanced explainability modules to support deeper clinical interpretation and audit trails.

In conclusion, this paper provides a practical, reproducible blueprint for ensuring the safety of AI-driven healthcare systems, proving that lightweight, interpretable anomaly detection is a viable and necessary component of modern clinical infrastructure.