A Hybrid Knowledge-Grounded Framework for Safety and Traceability in Prescription Verification

This paper introduces PharmGraph-Auditor, a hybrid framework that combines a Virtual Knowledge Graph-based pharmaceutical knowledge base with a Chain of Verification reasoning paradigm to enable reliable, evidence-grounded, and traceable prescription auditing by transforming Large Language Models into transparent reasoning engines.

The Gist

🏥 The Problem: The Overwhelmed Pharmacist

Imagine a pharmacist as a highly skilled air traffic controller. Their job is to look at a flight plan (a prescription) and make sure the plane (the patient) won't crash into another plane (a drug interaction) or run out of fuel (wrong dosage).

The problem? The sky is getting crowded. There are thousands of new flight rules, complex weather patterns (patient history), and the controller is tired. If they miss one detail, people get hurt.

Enter Artificial Intelligence (AI). Everyone wants to use AI to help. But current AI (Large Language Models or LLMs) is like a confident but forgetful intern.

• The Intern's Flaw: If you ask the intern, "Can this patient take this drug?" they might guess a "Yes" or "No" based on what they think they know. They might make up facts (hallucinate) because they can't look up the exact rulebook.
• The Risk: In medicine, a "maybe" or a "made-up fact" is dangerous. We need 100% certainty, and we need to know exactly where that certainty came from.

🛠️ The Solution: PharmGraph-Auditor

The authors built a new system called PharmGraph-Auditor. Think of this not as a "smart intern," but as a super-organized Librarian with a magnifying glass and a calculator.

The system solves the AI's problems by splitting the job into two distinct parts, using a "Hybrid" approach.

1. The Two-Part Brain (The Hybrid Knowledge Base)

Pharmaceutical knowledge is weird. Some of it is strict math (numbers), and some of it is a messy web of connections (concepts).

• The Calculator (Relational Database):
  • What it does: Handles strict rules like "If the patient is over 65, the dose must be 50mg."
  • The Analogy: This is like a spreadsheet. It's perfect for checking numbers, ranges, and hard limits. It's fast and precise.
• The Web (Graph Database):
  • What it does: Handles connections like "Drug A causes an allergy in people who are allergic to Penicillin."
  • The Analogy: This is like a family tree or a subway map. It's great for tracing paths. If Drug A is related to Drug B, and Drug B is related to Drug C, the system can trace that path instantly to find hidden dangers.

Why mix them? If you try to do math on a subway map, it's slow. If you try to trace a family tree on a spreadsheet, it's impossible. PharmGraph-Auditor uses the Calculator for numbers and the Web for connections, giving it the best of both worlds.

2. Building the Library (Iterative Schema Refinement)

Before the system can work, it needs to learn the rules from thousands of messy medical documents (PDFs).

• The Old Way: Trying to read a whole book and guess the chapters.
• The New Way (ISR): The system uses a team of AI agents (like a construction crew) that read the documents section by section.
  • One agent looks at the "Dosage" section.
  • Another looks at "Side Effects."
  • They work together with human experts to build the perfect filing system. If they find a new type of rule, they ask the expert, "Should we add a new folder for this?" This ensures the library is built perfectly before it ever sees a real patient.

3. The Audit Process: The "Chain of Verification" (CoV)

This is the most important part. When a prescription comes in, the system doesn't just "guess" the answer. It follows a strict 4-Step Detective Process:

1. The Planner (Decomposition): The AI breaks the big question ("Is this safe?") into tiny, specific questions ("Check the dose," "Check for allergies," "Check for drug interactions").
2. The Researcher (Query Generation): Instead of guessing, the system writes specific search commands.
   • If it needs a number, it asks the Calculator (SQL query).
   • If it needs a connection, it asks the Web (Cypher query).
3. The Filter (Evidence Selection): The search might return 50 results. The system uses a Patient Profile Filter to throw away everything that doesn't apply to this specific patient. It keeps only the one rule that matters.
4. The Reporter (Synthesis): The AI writes the final report. Crucially, it must cite its source.
   • Bad AI: "This is dangerous."
   • PharmGraph-Auditor: "This is dangerous because Page 12 of the Drug Manual says 'Do not mix with Drug X'."

If the system doesn't have enough info (e.g., it doesn't know the patient's kidney function), it doesn't guess. It raises a red flag saying, "I need more data to be sure." This prevents dangerous guesses.

📊 The Results: Why It Matters

The researchers tested this system against:

1. Human Experts: The best pharmacists in the world.
2. Old Computer Systems: The rigid, rule-based software hospitals use today.

The Outcome:

• Human Experts: Very accurate when they say "Yes," but they miss about 54% of the hidden dangers because they get tired or forget a specific rule.
• Old Systems: Catch more errors, but they scream "DANGER!" for everything, even when it's safe. This causes "Alert Fatigue," where pharmacists start ignoring the alarms.
• PharmGraph-Auditor: It caught more errors than the humans (better safety) but made fewer false alarms than the old computers (less annoyance).

🚀 The Big Picture

This paper isn't just about a new computer program. It's about changing how we trust AI in medicine.

Instead of asking AI to be a magician that pulls answers out of thin air, this system treats AI as a transparent assistant. It forces the AI to show its work, check its sources, and admit when it doesn't know something.

In short: It turns the "black box" of AI into a "glass box" where every decision is backed by evidence, making the pharmacy counter a safer place for everyone.

Technical Summary

Here is a detailed technical summary of the paper "A Hybrid Knowledge-Grounded Framework for Safety and Traceability in Prescription Verification" (PharmGraph-Auditor).

1. Problem Statement

Medication errors pose a severe threat to patient safety, making Pharmacist Verification (PV) a critical final safeguard. However, manual verification is increasingly strained by the complexity of modern pharmaceutical evidence, leading to cognitive overload and overlooked details.

While Large Language Models (LLMs) offer potential for automating these tasks, their direct application in zero-tolerance clinical settings is currently untenable due to three fundamental limitations:

1. Factual Unreliability: LLMs are prone to "hallucinations," generating plausible but incorrect information.
2. Lack of Traceability: Knowledge is encoded opaquely within model parameters, making it impossible to trace conclusions back to source documents, violating evidence-based medicine principles.
3. Weakness in Complex Reasoning: Auditing requires multi-hop reasoning (connecting patient history, drug properties, and guidelines) and strict numerical constraint satisfaction, which LLMs struggle to perform reliably without a factual scaffold.

Existing solutions often focus on data entry standardization or simple fact-checking, failing to address the rigorous, evidence-grounded requirements of the final verification stage.

2. Methodology

The authors propose PharmGraph-Auditor, a system built on a Virtual Knowledge Graph (VKG) paradigm that unifies relational and graph data models.

A. Theoretical Foundation: Hybrid Pharmaceutical Knowledge Base (HPKB)

The system models pharmaceutical knowledge as a tuple $H = \langle R, G, \phi \rangle$, stratifying data based on its logical nature:

• Relational Component (R): Stores atomic, numerical, and conditional facts (e.g., dosage limits, renal function thresholds). It utilizes Set Constraint Satisfaction logic, optimized for $O(\log N)$ range lookups via SQL indices.
• Graph Component (G): Stores associative, hierarchical, and transitive facts (e.g., drug interactions, allergy hierarchies). It utilizes Topological Traversal logic, optimized for $O(1)$ relationship traversal via index-free adjacency.
• Mapping Function ($\phi$): A bijective function linking graph vertices to relational tuples, ensuring a unified identity across both storage modalities.

B. Knowledge Construction: Iterative Schema Refinement (ISR)

To build the HPKB from unstructured medical texts, the authors propose the ISR algorithm, a semi-automated, expert-supported loop:

1. Stratified Sampling: Selects diverse therapeutic areas (e.g., oncology, cardiology) to ensure schema generalization.
2. Human-AI Synergy:
   • LLM as Gap Detector: Identifies "Schema Gaps" (missing information) in documents relative to the current schema.
   • Expert as Architect: Reviews proposals to prevent "Schema Fragmentation," abstracting specific gaps into generalized structures (e.g., a generic Constraint node for various organ impairments).
3. Section-Aware Multi-Agent Framework: Once the schema stabilizes, specialized agents (e.g., Dosage Agent, Interaction Agent) extract facts from specific document sections. Crucially, every extracted fact is tagged with provenance metadata (source text, document ID, section) to ensure traceability.

C. Application: KB-Grounded Chain of Verification (CoV)

Instead of asking an LLM to generate a verdict directly, CoV decomposes the audit into a transparent, four-stage pipeline:

1. Task Decomposition: An LLM agent breaks the prescription audit into specific, verifiable sub-tasks (e.g., "Check Dosage," "Check Allergies").
2. Hybrid Query Generation: A deterministic rule-based engine generates precise queries based on the task type:
   • SQL for relational constraints (e.g., SELECT * FROM DosageRules WHERE CrCl < 30).
   • Cypher for graph topology (e.g., traversing drug interaction paths).
3. Evidence Retrieval & Curation (P-EST): The Patient Profile-driven Evidence Selection Tree prunes retrieved data. It performs exact matches first, then hierarchical fallbacks, ensuring the LLM receives only the single most relevant rule, reducing noise and cognitive load.
4. Evidence-Grounded Synthesis: A final LLM agent synthesizes the curated evidence into an audit report. It explicitly flags Information Gaps (missing patient data) rather than hallucinating a verdict, ensuring safety.

3. Key Contributions

1. Hybrid VKG Architecture: A novel framework that mathematically justifies the separation of pharmaceutical knowledge into relational (constraints) and graph (topology) components to solve the trade-off between numerical rigor and semantic reasoning.
2. Iterative Schema Refinement (ISR): A dynamic algorithm for co-evolving graph and relational schemas from unstructured texts, balancing information recall with structural compactness.
3. Chain of Verification (CoV): A reasoning paradigm that transforms the LLM from an opaque generator into a transparent reasoning engine by forcing it to execute verifiable hybrid queries and cite specific evidence.
4. Traceability and Safety: The system explicitly handles missing data by flagging information gaps, prioritizing safety over generating a potentially hallucinated conclusion.

4. Experimental Results

The system was evaluated on real-world inpatient prescriptions annotated by clinical experts.

• Knowledge Construction (RQ1):
  • PharmGraph-Auditor achieved an F1-score of 0.8374 (using GPT-4o), significantly outperforming Zero-shot OpenIE (F1: 0.6148) and One-shot Schema-guided agents (F1: 0.7863).
  • It maintained high Recall (>0.84) to ensure risk coverage and high Precision (>0.82) to suppress hallucinations.
• Prescription Auditing (RQ2):
  • vs. Human Experts: The system achieved a Recall of 70.3%, significantly surpassing the human baseline (45.9%), indicating it detects latent risks humans miss due to cognitive overload.
  • vs. Traditional CDSS: The system achieved a Precision of 74.3%, drastically reducing the False Positive rate compared to rule-based CDSS (52.1%), thereby mitigating "alert fatigue."
  • Overall Performance: The proposed method achieved an F1-score of 72.2%, representing a +13.4% improvement over traditional rule-based CDSS.
• Ablation Study: Removing the CoV framework or the external Knowledge Base resulted in significant drops in F1-score (down to 0.4487), confirming the necessity of both structured retrieval and the verification pipeline.

5. Significance

• Clinical Impact: PharmGraph-Auditor offers a practical solution to augment pharmacist expertise, significantly improving the detection of complex medication errors (especially in special populations like renal impairment) while reducing the noise that leads to alert fatigue.
• AI Safety Paradigm: The work demonstrates a shift from "black-box" LLM generation to "white-box," evidence-grounded reasoning. It provides a blueprint for deploying LLMs in high-stakes domains where traceability and factual accuracy are non-negotiable.
• Scalability: The hybrid architecture is adaptable to other complex domains requiring both rigorous constraint satisfaction and advanced semantic reasoning, moving beyond the limitations of pure vector-based RAG or pure rule-based systems.