Human-supervised, large language model-based clinical decision support aligned to national newborn protocols in Kenya: a pragmatic, early-stage evaluation

This study presents a pragmatic evaluation of AIFYA, a human-supervised large language model-based clinical decision support system aligned with Kenya's national newborn protocols, demonstrating successful implementation, high expert-rated accuracy, and strong user adoption in low-resource public health facilities.

The Gist

Imagine a busy, understaffed hospital nursery in rural Kenya. The nurses and doctors are heroes, working hard to save tiny, fragile newborns. But they face a massive challenge: they have to make life-or-death decisions quickly, often while juggling dozens of patients at once. To get it right, they need to follow thick, complex rulebooks (national medical guidelines) that are hundreds of pages long. Trying to flip through those pages during an emergency is like trying to find a specific needle in a haystack while running a marathon.

This is the problem a team of researchers in Kenya tried to solve with a new tool called AIFYA.

Here is a simple breakdown of what they did, how it works, and what they found, using some everyday analogies.

1. The Problem: The "Overwhelmed Librarian"

Think of the national medical guidelines as a massive, high-tech library. In a perfect world, a doctor could walk in, ask for the book on "Newborn Jaundice," read the chapter, and know exactly what to do.

But in a real-world emergency room, the "librarian" (the doctor) is too busy to run to the library. They might guess, or rely on memory, which can lead to mistakes. The researchers wanted to build a tool that brings the library to the doctor's desk, instantly.

2. The Solution: AIFYA (The "Smart Co-Pilot")

The team built an app called AIFYA. Think of it as a super-smart, hyper-organized co-pilot sitting next to the doctor on a tablet.

• How it works: The doctor types in what they see (e.g., "Baby is 2 days old, yellow skin, fever").
• What it does: The app, powered by Artificial Intelligence (specifically a Large Language Model), instantly suggests a plan: "Check for infection, give this specific medicine, keep baby warm."
• The Safety Catch (Human-in-the-Loop): This is the most important part. The AI is not the boss. It's like a GPS that suggests a route, but the driver (the doctor) still holds the steering wheel. The doctor must read the suggestion, check it, and press "Accept" before doing anything. The AI never acts on its own.
• The "Citation" Feature: Every time the AI suggests something, it doesn't just say "Do this." It says, "Do this, and here is the exact page in the official government rulebook that says so." It's like a student who not only gives you the answer but also points to the textbook line where the answer is written. This builds trust.

3. The Experiment: A "Practice Run" in Real Life

From late 2024 to mid-2025, the team tested this system in three hospitals in Bungoma County, Kenya.

• The Setup: They trained 50 healthcare workers (nurses and doctors) on how to use the app.
• The Challenge: Internet in rural areas can be spotty. So, they built the app to work offline-first. It's like a smartphone that saves your maps and notes even when you have no signal, and only uploads them when you get back to Wi-Fi.
• The Volume: Over 10 months, they used the app to manage 550 newborn cases.

4. The Results: Did it Work?

The researchers asked three main questions:

A. Did people use it?
Yes! The doctors and nurses loved it. 92% said it was useful. It became part of their daily routine, even with staff changes and busy shifts.

B. Was the advice correct?
They hired two expert baby doctors (neonatologists) who didn't know which cases were AI-generated to grade the app's suggestions.

• The Score: 75% of the time, the AI was 100% correct. Another 15% was "mostly correct" (safe, but maybe missing a tiny detail). Only 10% were wrong.
• The "Citation" Score: 96% of the time, the AI pointed to the exact right page in the rulebook. This is huge because it means the AI isn't just guessing; it's quoting the rules.

C. Did it slow things down?
No. The time it took to go from "patient arrives" to "doctor makes a decision" stayed steady at about 23 minutes. The app didn't add extra steps; it actually helped organize the thinking process.

5. The Big Takeaway

This study is like a successful test drive for a new kind of car. It proved that:

1. AI can be safe in a hospital if a human is always in the driver's seat.
2. AI can be trusted if it shows its work (the citations).
3. AI can work even in places with poor internet and limited resources.

The researchers aren't saying the AI is perfect yet. They found some tricky spots (like calculating medicine for extremely tiny, premature babies) and are already fixing those. But the main message is clear: When you combine smart technology with human wisdom and local rules, you can save more babies.

The next step? They plan to run a bigger, more rigorous test to see if this system actually reduces the number of babies who get sick or die. But for now, this "practice run" shows that the future of healthcare in low-resource settings looks bright, smart, and safe.

Technical Summary

1. Problem Statement

• High Neonatal Mortality: Over two million newborns die annually, primarily in low- and middle-income countries (LMICs) due to preventable causes.
• Implementation Gap: While Kenya has robust national guidelines (Comprehensive Newborn Care Protocols), translating these extensive documents into bedside practice is hindered by high cognitive loads and urgent clinical environments.
• AI Risks: The deployment of "black-box" AI in healthcare raises concerns regarding safety, accountability, algorithmic bias, and the potential erosion of clinical autonomy. There is a lack of early-stage, transparent evaluations of Large Language Models (LLMs) specifically designed for low-resource settings with human-in-the-loop safeguards.

2. Methodology

Study Design & Setting:

• Type: Prospective, mixed-methods, early-stage implementation study.
• Framework: Guided by the DECIDE-AI reporting framework for early-stage AI evaluation.
• Location: Three public health facilities in Bungoma County, Kenya (one Level 5 referral hospital and two Level 4 sub-county hospitals).
• Timeline: September 1, 2024 – June 30, 2025.
• Participants: 50 credentialed Healthcare Workers (HCWs) including nurses, clinical officers, and medical officers.

The Intervention (AIFYA System):

• Architecture: A tablet-based Clinical Decision Support System (CDSS) utilizing a fine-tuned GPT-4 based LLM.
• Knowledge Base: Meticulously aligned with the 2022 Kenya Consolidated Newborn Clinical Protocols (CNCP) and Basic Paediatric Protocols.
• Key Technical Features:
  • Human-in-the-Loop (HITL): Mandatory workflow where clinicians must review, acknowledge, modify, or reject AI recommendations before proceeding. All actions are logged for audit.
  • Explainability: Every recommendation includes a direct hyperlink to the specific page/section of the source protocol.
  • Offline-First: Data is stored locally and synchronized to the cloud only when connectivity is available, ensuring functionality in low-connectivity rural areas.
  • Safety Guardrails: Embedded weight-based dose calculators with range checks and automated alerts for contraindications and "red-flag" signs.

Evaluation Metrics:

• Primary Outcomes: Adoption (cases managed), Training Reach (HCWs credentialed), and Guideline/Citation Concordance (via blinded expert review).
• Secondary Outcomes: Protocol adherence rates, triage-to-decision time, and user perceptions.
• Expert Review: 118 de-identified clinical scenarios were independently reviewed by two blinded neonatologists (rated on correctness and citation accuracy), with discrepancies adjudicated by a third.

3. Key Contributions

• Novel "Citation Correctness" Metric: Introduced a specific evaluation metric for LLM-based CDSS that measures not just clinical accuracy, but the accuracy of the reference citation to the source guideline (96% accuracy achieved).
• Context-Aware AI Design: Demonstrated a scalable model for integrating generative AI into low-resource settings by combining offline-first architecture with a facility-based coaching model to sustain adoption despite staff turnover.
• Transparent Workflow: Established a "human-supervised" paradigm where the AI acts strictly as a decision-support tool, retaining final clinical authority with the human provider, thereby addressing ethical concerns regarding autonomy.
• DECIDE-AI Compliance: Provided a rigorous, early-stage evaluation pathway that prioritizes feasibility, safety, and workflow integration before moving to large-scale clinical trials.

4. Results

• Adoption & Reach:
  • 550 neonatal cases were managed over 10 months.
  • 50 HCWs were trained and credentialed; 33 participated in the endline survey.
  • 92% of users agreed that AIFYA was a useful tool in daily work.
• Clinical Concordance (Expert Review):
  • Correctness: 75% of recommendations were rated Correct, 15% Partially Correct, and 10% Incorrect.
  • Agreement: Strong inter-rater reliability with a weighted Cohen's kappa of 0.85 (95% CI: 0.79–0.91).
  • Citation Accuracy: 96% of references correctly supported the recommendation.
  • Complex Scenarios: In 40 complex dosing cases, 75% were rated correct; errors primarily involved weight-based adjustments for extremely preterm infants, leading to algorithm refinements.
• Process Performance:
  • Protocol Adherence: Increased by 5% per month (OR = 1.05; p = 0.004), indicating successful integration into routine practice.
  • Efficiency: Median triage-to-decision time was 23 minutes (IQR 18–31), remaining stable throughout the study.
  • Resilience: 35% of clinical sessions were conducted fully offline due to intermittent internet.
• User Perception:
  • 79% of HCWs rated human oversight as "extremely" or "very" important.
  • Top concerns were potential clinical over-dependence (51.5%) and accuracy of recommendations (30.3%).

5. Significance

• Feasibility & Safety: The study proves that a human-supervised, guideline-anchored LLM can be safely deployed in rural, low-resource neonatal units without compromising clinical efficiency.
• Trust Building: The high citation accuracy and transparent "explainable correctness" model are critical for building clinical trust and enabling auditable AI, which is often a barrier to adoption.
• Scalability: The combination of offline-first technology and a peer-coaching implementation strategy offers a replicable model for other LMICs facing similar infrastructure and workforce challenges.
• Future Direction: These findings provide the necessary evidence base to justify progression to multi-site, cluster-randomized controlled trials to definitively evaluate the system's impact on neonatal morbidity and mortality and to inform national digital health policy.