Retrieval-Augmented Generation for Reducing Hallucinations in Drug-Related Question Answering Systems

Technical Summary: Retrieval-Augmented Generation for Reducing Hallucinations in Drug-Related Question Answering Systems

Problem Statement
Large Language Models (LLMs) deployed in pharmaceutical question-answering applications frequently generate "hallucinations"—factually incorrect outputs that appear plausible but are clinically dangerous. In the context of drug interactions, dosage thresholds, and contraindications, these errors are not merely informational; they pose serious patient-safety risks, potentially precipitating adverse drug events. While general-purpose LLMs excel at fluent language generation, they lack the rigorous factual grounding required for medical reliability. Current literature lacks systematic, reproducible evaluations of hallucination-mitigation strategies specifically tailored to the conversational pharmaceutical domain, often relying on general benchmarks or structured clinical tasks that do not reflect end-user scenarios.

Methodology
The authors propose and evaluate a Retrieval-Augmented Generation (RAG) pipeline designed to ground LLM responses in a curated, citation-verified pharmaceutical knowledge base. The system architecture follows five sequential stages:

1. Knowledge Base Construction: A database was assembled from authoritative sources, including the FDA Drugs@FDA database (2,847 approved drugs), ASHP clinical practice guidelines, and peer-reviewed PubMed articles (2015–2024). Documents were segmented into 512-token overlapping chunks, resulting in 184,329 total chunks.
2. Embedding & Indexing: Chunks were encoded using Sentence-BERT (all-MiniLM-L6-v2) to generate 384-dimensional dense vectors. These were indexed using FAISS with an IVF-PQ configuration (nlist=256, m=64) to enable sub-millisecond approximate nearest-neighbor search.
3. Retrieval & Re-ranking: At inference, user queries are encoded, and the top-20 candidates are retrieved via L2 distance. A cross-encoder (ms-marco-MiniLM-L-6-v2) then re-scores these candidates, selecting the top-5 passages for the final prompt.
4. Prompt Augmentation: Retrieved passages are concatenated with a system prompt instructing the LLM (Llama 3.2) to answer only from the provided context, cite supporting passages, and explicitly state if information is insufficient.
5. Generation: The LLM generates the final response based on the augmented context.

Key Contributions
The paper makes four primary contributions:

• Dataset Construction: Creation of a specialized pharmaceutical QA dataset comprising 150 question–answer pairs covering ten commonly prescribed medications across ten medical categories (e.g., Cardiovascular, Oncology, Endocrinology).
• Pipeline Implementation: Development and evaluation of a full RAG pipeline integrating Sentence-BERT, FAISS, cross-encoder re-ranking, and Llama 3.2.
• Quantitative Evaluation: A rigorous comparison across five metrics: Hallucination Rate, Faithfulness, Answer Relevance, Factual Accuracy, and Context Precision.
• Clinical Case Study: A qualitative demonstration of the system's efficacy using a high-stakes query regarding the warfarin–ibuprofen interaction.

Experimental Results
Evaluated on the 150-question dataset, the RAG system demonstrated statistically significant improvements over an LLM-only baseline:

• Hallucination Rate: Reduced from 47.8% to 12.3%, representing a 74.3% relative reduction. The authors note that residual errors in the RAG system shifted from "commission" (false assertions) to "omission" (retrieval failures), which is considered a safer failure mode in clinical settings.
• Faithfulness: Improved from 0.52 to 0.87.
• Factual Accuracy: Improved from 0.54 to 0.89.
• Answer Relevance: Improved from 0.61 to 0.83.
• Latency: The RAG pipeline increased inference time from 1.1 seconds (baseline) to 3.2 seconds.

The system showed the greatest benefit in high-risk categories such as Oncology (44.8 percentage point reduction in hallucinations) and Pain Management (40.3 percentage point reduction).

Case Study Findings
In a specific case study regarding the safety of taking ibuprofen with warfarin, the baseline model provided dangerous misinformation, incorrectly suggesting the combination was generally safe and advising specific dosages. In contrast, the RAG system correctly identified the contraindication, explained the mechanism (CYP2C9 inhibition increasing bleeding risk), and provided authoritative citations (FDA and ASHP sources), thereby preventing potential life-threatening hemorrhagic events.

Significance and Claims
The paper claims that retrieval-augmented architectures are the preferred approach for safety-critical pharmaceutical AI. By reducing hallucination rates by over 70% and shifting error types toward safer omissions, the proposed framework addresses a critical gap in consumer-facing medical AI. The authors emphasize that their work provides an open, reproducible evaluation framework, including the release of the 150-question dataset and source code, to facilitate further research in pharmaceutical NLP and patient safety. The study concludes that while limitations exist (such as the static nature of the knowledge base and English-only corpus), the RAG approach effectively transforms a system that frequently fabricates information into one that provides verifiable, citation-grounded guidance.