📊 statistics

Errors in AI-Assisted Retrieval of Medical Literature: A Comparative Study

This study quantitatively reveals that widely used free-version large language models exhibit significant variability and a high rate of complete failure (47.8%) when retrieving accurate medical literature references, underscoring the critical need for human verification of bibliographic data generated by AI.

Original authors: Jenny Gao (College of Arts and Science, New York University, New York, NY), Yongfeng Zhang (Department of Computer Sciences, School of Arts & Sciences, Rutgers University, Piscataway, NJ), Mary L Disi

Published 2026-03-25

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jenny Gao (College of Arts and Science, New York University, New York, NY), Yongfeng Zhang (Department of Computer Sciences, School of Arts & Sciences, Rutgers University, Piscataway, NJ), Mary L Disis (UW Medicine Cancer Vaccine Institute University of Washington, Seattle, WA), Lanjing Zhang (Department of Chemical Biology, Ernest Mario School of Pharmacy, Rutgers University, Piscataway, NJ, Department of Pathology, Princeton Medical Center, Plainsboro, NJ, Rutgers Cancer Institute, New Brunswick, NJ)

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a student trying to write a research paper. You ask a super-smart, all-knowing robot librarian for a list of 10 books to help you. The robot hands you a list with titles, authors, and library call numbers. You look happy and start reading. But then, you realize: half the books don't actually exist. Some have the right title but the wrong author. Others have the right author but the wrong year. Some call numbers lead to empty shelves.

This is exactly what a new study by Jenny Gao and Lanjing Zhang discovered about AI "librarians" (called Large Language Models or LLMs) when they try to find medical research papers.

Here is the breakdown of their findings, translated into everyday language:

The Big Experiment: The "Robot Librarian" Test

The researchers decided to put five popular free AI tools to the test: Grok, ChatGPT, Google Gemini, Perplexity, and DeepSeek.

They picked 40 real, high-quality medical articles from four famous journals (like the New England Journal of Medicine and JAMA). They asked each AI: "Here is a summary of this article. Please find me 10 other important papers related to this topic, and give me the exact details (like the DOI, PubMed ID, and links) so I can find them."

The AI tools generated 2,000 references in total. The researchers then acted like strict librarians, checking every single one to see if it was real and correct.

The Shocking Results: The "Hallucination" Problem

The results were a bit scary for anyone trusting AI to do their homework:

The "Complete Fail" Rate: Nearly 48% of the time, the AI gave a reference that was a total disaster. It was either a fake paper that never existed, or the details were so wrong you couldn't find the paper at all. It's like the robot handing you a map to a treasure chest that is buried in a field that doesn't exist.
The "Average" Score: If you gave the AI a score from 0 to 1 (where 1 is perfect), the average score was only 0.29. That's a failing grade.
The Winners and Losers:
- Grok was the "honor student," getting the most correct details (though it still made mistakes).
- Google Gemini was the "struggling student," getting the details wrong most of the time.
- ChatGPT and Perplexity were in the middle, but still far from perfect.

Why Did This Happen?

The study found two main reasons why the robots got confused:

The "Journal" Factor: It was harder for the AI to find correct details for papers from the New England Journal of Medicine (NEJM) compared to the British Medical Journal (BMJ). The researchers suspect this is because NEJM abstracts are shorter and have less information for the AI to "grab onto" to find the right paper. It's like trying to find a specific person in a crowd when you only have their first name versus having their full name and address.
The "Confidence" Trap: The AI tools were very confident. They would give you a perfect-looking list with titles, dates, and links. But often, the links were broken or the dates were wrong. This is called "Referential Hallucination." The AI is so eager to please that it makes up facts that sound true but are actually fiction.

The "Double-Check" Didn't Work

The researchers tried a clever trick. After the AI gave the list, they asked it: "Please double-check these numbers and links to make sure they are real."

Did it help? No. The AI just confidently double-checked its own mistakes. It's like asking a person who made up a story to verify their own story; they will just repeat the lie with more confidence.

The Takeaway: Use AI as a Guide, Not a Guru

So, what does this mean for you?

AI is a great brainstorming partner: It can help you think of topics or find general ideas.
AI is a terrible fact-checker: You cannot trust it to give you the "receipts" (the actual links, dates, and IDs) for medical or scientific research.
The Golden Rule: If an AI gives you a reference, you must go find the paper yourself and verify that it exists. Do not just copy-paste the AI's list into your work.

In short: Large Language Models are like enthusiastic tour guides who know a lot about the world but sometimes invent landmarks that aren't there. If you follow their map blindly, you might end up lost in a field that doesn't exist. Always bring your own compass (human verification) when navigating medical research.

1. Problem Statement

Large Language Models (LLMs) are increasingly used in biomedicine for literature synthesis and reference retrieval. However, they are prone to "referential hallucinations," where they generate plausible but fabricated or inaccurate citations, including incorrect titles, Digital Object Identifiers (DOIs), PubMed IDs, and URLs. While anecdotal evidence and smaller studies suggest high error rates, there is a lack of rigorous, quantitative comparative data on the performance of contemporary, widely used free-version LLMs in retrieving accurate bibliographic metadata for high-stakes medical research.

2. Methodology

The study employed a quantitative, comparative experimental design to evaluate the retrieval accuracy of five popular free-version LLM platforms.

Dataset Selection:
- Source: Four leading medical journals: British Medical Journal (BMJ), Journal of the American Medical Association (JAMA), The Lancet, and The New England Journal of Medicine (NEJM).
- Sample: 40 original research articles were randomly selected (10 per journal), split between publications from 2024 and early 2025 (January–July).
- Total References: 2,000 references were generated (10 references per article × 40 articles × 5 LLMs).
LLM Platforms Evaluated:
1. Grok-2
2. ChatGPT (GPT-4.1)
3. Google Gemini (Flash 2.5)
4. Perplexity AI
5. DeepSeek (GPT-4)
Prompting Strategy:
- Initial Prompt: Provided the article abstract and requested a list of 10 key references with specific metadata (Title, Year, DOI, PubMed ID, Google Scholar URL).
- Verification Prompt: A secondary prompt instructed the LLM to "confirm" the DOI, PubMed ID, and Google Scholar link before finalizing the output.
Validation Metrics:
Four manual validation metrics were applied to each retrieved reference:
1. DOI Validity: Matched via doi.org.
2. PubMed ID Validity: Matched via pubmed.gov.
3. Google Scholar Link Validity: Matched via search.
4. Relevance Score: Determined if the reference appeared in the index article's reference list (1 point) or was the index article itself (2 points).
Scoring System:
- Score Ratio: A composite metric calculated as (Total Points Earned) / (Score Cap). The Score Cap adjusted for "not applicable" metrics (e.g., engineering papers lacking a PubMed ID) and included a bonus for the index article.
- Complete Miss Rate: Defined as references where the total score was zero (no valid metadata or relevance found).
Statistical Analysis:
- Multivariable regression was used to identify independent associations between LLM platforms, journals, publication years, and the outcome metrics (Score Ratio and Complete Miss Rate).
- Significance was set at $p < 0.05$ .

3. Key Contributions

Quantitative Benchmarking: Provides the first rigorous, large-scale (2,000 references) quantitative comparison of five major free LLM platforms specifically for medical literature retrieval.
Multimetric Framework: Introduces a composite scoring system that accounts for both bibliographic accuracy (DOIs, IDs) and topical relevance, adjusting for non-applicable metrics to ensure fair comparison.
Factor Analysis: Isolates the independent effects of the LLM platform and the target journal on retrieval accuracy, moving beyond simple aggregate error rates.
Prompt Limitation Insight: Demonstrates that explicit prompts asking the model to "confirm" data do not significantly mitigate hallucination errors.

4. Key Results

Overall Performance: LLMs performed poorly overall. The average Score Ratio across all platforms was 0.29 (SD 0.35), indicating low accuracy.
Complete Miss Rate: LLMs failed to retrieve any correct data (complete miss) in 47.8% of cases.
Platform Variability:
- Best Performer: Grok-2 achieved the highest Score Ratio (0.57) and the lowest Complete Miss Rate (11.2%).
- Worst Performer: Google Gemini had the lowest Score Ratio (0.11) and the highest Complete Miss Rate (78.5%).
- Other Platforms: ChatGPT, Perplexity, and DeepSeek fell between these extremes but generally underperformed compared to Grok.
Journal Variability:
- Articles from NEJM had the lowest retrieval accuracy (Score Ratio 0.25) and highest complete miss rate (56.0%).
- BMJ articles had the highest accuracy (Score Ratio 0.33).
- Hypothesis: The authors suggest NEJM's shorter abstracts (250 words vs. 300–350 words for others) may provide insufficient context for accurate retrieval.
Individual Metrics:
- Performance varied significantly by journal and publication year for DOI, PubMed ID, and Google Scholar links.
- Relevance scores were less variable across years but still showed significant differences by journal.
- Gemini retrieved accurate bibliographic data almost never for BMJ (0/100 for PubMed/Google Scholar) and NEJM.

5. Significance and Implications

Reliability Warning: The study concludes that current LLMs are unreliable for autonomous medical literature retrieval. The high rate of "plausible but wrong" references poses a significant risk to scholarly integrity and medical research.
Necessity of Human Verification: Researchers must manually verify all metadata (DOIs, IDs) generated by LLMs. Relying on a single platform is insufficient; cross-checking across multiple models is recommended.
Prompt Engineering Limitations: The study found that asking LLMs to self-correct or confirm data did not improve accuracy, suggesting that current prompting strategies are inadequate for mitigating hallucinations in this domain.
Journal Abstract Optimization: The correlation between abstract length and retrieval accuracy suggests that journal editors and authors should consider providing more detailed abstracts to improve AI-assisted search capabilities.
Future Directions: The authors call for the development of robust hallucination mitigation systems and verification frameworks before LLMs can be trusted as authoritative tools in biomedical scholarship.

Conclusion: While LLMs offer potential for accelerating literature discovery, their current performance in retrieving accurate medical references is highly variable and frequently erroneous. Until verification systems are improved, they should be used strictly as assistive tools requiring rigorous human oversight.