💻 computer science

Can Large Language Models Detect Contradicted Biomedical Evidence? A Dual-Annotator Benchmark Audit

This study demonstrates that current large language models and retrieval-augmented generation systems significantly fail to detect contradictions between generated biomedical answers and retrieved evidence, often misclassifying them as insufficient, thereby highlighting that such automated verifiers are not yet ready for clinical autonomous screening.

Original authors: xinzheng Chen

Published 2026-07-06

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: xinzheng Chen

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a strict editor at a medical journal. You have a team of AI writers (Large Language Models) who are trying to answer complex medical questions. To make sure their answers are trustworthy, you give them a stack of medical research papers (evidence) to read first. This is called Retrieval-Augmented Generation (RAG).

The big question this paper asks is: If the AI writer gives an answer that actually contradicts the research papers it just read, can a second AI "checker" spot the lie?

Here is a breakdown of what the researchers found, using simple analogies.

1. The Setup: The "Fact-Checker" Experiment

The researchers set up a test with 150 medical questions. For each question, they had:

The Question: A medical query.
The Evidence: A snippet of a medical abstract.
The Answer: An answer generated by an AI.
The Truth: A human expert labeled the answer as Supported (the paper backs it up), Insufficient (the paper is vague), or Contradicted (the paper says the opposite!).

They then asked various AI "checkers" (including top models like GPT-5.5, DeepSeek, and specialized classifiers) to look at the Evidence and the Answer and say: "Does this match?"

2. The Main Discovery: The "Blind Spot"

The results were surprising and a bit worrying. The AI checkers were terrible at spotting contradictions.

The Analogy: Imagine a security guard whose job is to catch people trying to sneak a forbidden item into a building. In this test, there were 23 people trying to sneak in forbidden items (contradictions). The best security guard (the top AI model) only caught 5 of them. The others were let right through.
The Mistake: When the AI checkers missed a contradiction, they didn't usually say "This is wrong." Instead, they said, "This is Insufficient" (meaning: "I can't tell if it's right or wrong, so I'll play it safe"). They treated a clear lie as just a "vague" statement.
The Scale: Even the smartest models only detected about 17% to 22% of the actual contradictions. The rest were missed.

3. The "Human" Factor: Are Humans Better?

The researchers also had a second human editor check the work.

Blind Test: When the human editor looked at a random batch of questions without knowing which ones were tricky, they missed zero contradictions. They were just as "blind" as the AI.
Focused Test: However, when the human editor was told, "Hey, look specifically at these 23 questions; I know they contain lies," they found 13 out of 23.
The Lesson: Detecting a contradiction isn't just about being smart; it's about how you look. If you aren't specifically hunting for lies, you tend to miss them. Both humans and AI have a "conservative bias"—they prefer to say "I don't know" rather than "That's a lie."

4. The "Knowledge Graph" Experiment

The researchers also tried a fancy trick. They built a "Knowledge Graph" (a giant map connecting medical terms like a subway map) to help the AI find the best research papers to read. They hoped this would make the answers more accurate.

The Result: It didn't work. Using this fancy map didn't make the answers any better than just using standard search methods. It was like giving a driver a GPS that was just as good as their own sense of direction, but no better.

5. The Bottom Line

Current AI Checkers are not ready for prime time. If you rely on these AI systems to automatically catch medical lies in a hospital setting, they will miss most of them.
They are good at finding "Support," but bad at finding "Lies." They are great at saying, "Yes, this paper supports that answer," but they are very poor at saying, "No, this paper actually says the opposite."
The "Insufficient" Trap: The biggest problem is that the AI confuses "contradiction" with "not enough info." It's safer for the AI to say "I'm not sure" than to risk saying "That's wrong," but in a medical context, missing a contradiction is dangerous.

In short: The paper concludes that while these AI tools are useful for organizing information, they are not yet reliable enough to act as autonomous police for catching medical contradictions. They need human oversight to catch the errors they are currently missing.

Technical Summary: Can Large Language Models Detect Contradicted Biomedical Evidence?

Problem Statement
Biomedical question answering (QA) systems increasingly rely on Retrieval-Augmented Generation (RAG) to ground answers in retrieved literature. However, a critical failure mode exists where generated answers are either unsupported by or actively contradict the retrieved evidence. This study addresses two specific gaps in current evaluation: (1) whether integrating Knowledge Graph (KG) signals into retrieval improves answer accuracy, and (2) whether automated verifiers (LLMs and classifiers) can reliably detect when evidence contradicts a generated answer. The authors note that while answer-level accuracy is a standard metric, it is insufficient for assessing system trustworthiness if the underlying evidence does not support the conclusion.

Methodology
The study employed a dual-pronged experimental design using the PubMedQA dataset:

KG-Guided Retrieval Audit (KG-RRAG):
- Setup: A reranking module (KG-RRAG) was compared against standard vector-similarity RAG on 300 stratified PubMedQA questions.
- Mechanism: KG-RRAG constructed query-specific biomedical graphs (linking entities to UMLS concepts) and fused three signals for reranking: vector similarity ( $\alpha$ ), entity overlap ( $\beta$ ), and graph structure centrality ( $\gamma$ ).
- Generation: Both systems used glm-5.1 as the generator. The study controlled for generation variables, isolating the impact of evidence ranking.
Evidence-Answer Verification Benchmark:
- Dataset: A held-out set (Set B, $n=150$ ) was constructed with human-labeled gold standards: Supported (81), Insufficient (46), and Contradicted (23).
- Verifiers: The study evaluated a spectrum of architectures:
  - LLM-based: GPT-5.5, DeepSeek-v4-pro, and glm-5.1 using two prompt strategies: a simple direct classification (E1) and a "contradiction-first" structured prompt (V2.1).
  - Classifier-based: DistilBERT-MNLI (general domain) and PubMedBERT-MedNLI (biomedical domain).
  - Baselines: Rule-based heuristics (V1.6), hybrid rule/LLM fusion (V3.1), and a majority-class proxy.
- Human Calibration: A second human annotator (Human2) performed two protocols: a blind first-pass on a 30-item sample and a focused review of all 23 known contradicted cases to assess protocol sensitivity.

Key Results

KG-RRAG Null Result: The integration of KG signals did not improve answer accuracy. KG-RRAG achieved 0.500 accuracy versus 0.510 for standard RAG ( $\Delta = -0.010$ , McNemar $p=0.766$ ). The 95% confidence interval for the difference was $[-0.053, +0.033]$ , indicating no statistically significant benefit.
Systemic Contradiction Under-Detection: All automated verifiers struggled significantly with the "Contradicted" class.
- Best Automated Performance: The E1-glm-5.1 configuration detected only 5 of 23 gold contradictions (Recall $\approx$ 21.7%).
- Other Models: GPT-5.5 detected 4/23; DeepSeek variants detected 1–3/23; specialized NLI classifiers (PubMedBERT-MedNLI) detected only 2/23.
- Failure Mode: The dominant error pattern was absorbing contradicted cases into the "Insufficient" category rather than identifying them as active conflicts.
Protocol Sensitivity of Human Annotation:
- In a blind first-pass, Human2 detected 0/7 contradicted cases in a sample.
- In a focused review where the annotator knew the items were gold-contradicted, Human2 detected 13/23 (Recall $\approx$ 56.5%).
- This indicates that contradiction detection is not spontaneous but requires explicit, criterion-driven protocols.
Model vs. Prompt Effects: Prompt engineering effects were model-dependent. A prompt that performed better on glm-5.1 performed worse on DeepSeek-v4-pro, suggesting that model selection is a more significant factor than prompt design within the tested range.

Key Contributions

Controlled Null Result for KG-RAG: The study provides empirical evidence that adding KG-derived signals (entity overlap and graph centrality) to standard vector retrieval did not improve answer accuracy in the PubMedQA setting, challenging the assumption that structured knowledge always enhances RAG performance.
Audit of Contradiction Under-Detection: It establishes that current state-of-the-art LLMs and NLI classifiers fail to reliably detect contradictions in biomedical evidence-answer pairs, frequently misclassifying them as "insufficient."
Protocol-Sensitivity Evidence: The paper demonstrates that contradiction detection is highly sensitive to annotation protocols. The discrepancy between blind and focused human review suggests that "ground truth" in contradiction detection is difficult to establish without explicit, targeted criteria.

Significance and Claims
The authors modestly claim that their work serves as an audit of current capabilities rather than a demonstration of a solved problem.

Clinical Readiness: The results indicate that current LLM-based verification is not ready for autonomous contradiction screening in high-risk biomedical contexts. The failure to detect active contradictions poses a safety risk.
Evaluation Shift: The study argues that answer accuracy alone is an incomplete metric for biomedical RAG. Systems may generate correct answers from incorrect or unsupported evidence; therefore, evidence-answer consistency must be evaluated separately.
Practical Utility: While not ready for autonomous deployment, LLM-based verification may still serve as a tool for offline auditing and system comparison, provided that the high rate of false negatives (missed contradictions) is acknowledged.

The paper concludes that the difficulty of contradiction detection persists across model scales, architectures, and prompting strategies, likely due to the inherent ambiguity of biomedical text, specialized terminology, and the distinction between "absence of evidence" and "evidence of absence."