💬 NLP

PanCanBench: A Comprehensive Benchmark for Evaluating Large Language Models in Pancreatic Oncology

PanCanBench is a comprehensive benchmark comprising 282 authentic pancreatic cancer patient questions and 3,130 expert-derived criteria that reveals significant variations in the factual accuracy and completeness of 22 large language models, highlighting that higher rubric scores and web-search integration do not necessarily guarantee clinical safety or reduced hallucinations.

Original authors: Yimin Zhao, Sheela R. Damle, Simone E. Dekker, Scott Geng, Karly Williams Silva, Jesse J Hubbard, Manuel F Fernandez, Fatima Zelada-Arenas, Alejandra Alvarez, Brianne Flores, Alexis Rodriguez, Stephen

Published 2026-03-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yimin Zhao, Sheela R. Damle, Simone E. Dekker, Scott Geng, Karly Williams Silva, Jesse J Hubbard, Manuel F Fernandez, Fatima Zelada-Arenas, Alejandra Alvarez, Brianne Flores, Alexis Rodriguez, Stephen Salerno, Carrie Wright, Zihao Wang, Pang Wei Koh, Jeffrey T. Leek

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart, super-fast robot librarian who has read every medical book ever written. You ask it, "My uncle has pancreatic cancer; what should we do?" The robot gives you a long, confident answer.

The Big Problem: Just because the robot sounds confident and uses big words doesn't mean it's telling the truth. In the world of medicine, a confident lie can be dangerous.

This paper, called PanCanBench, is like a "final exam" designed specifically to test these robot librarians on the tricky, scary topic of pancreatic cancer. Here is how they did it, explained simply:

1. The "Real World" Test (No Fake Questions)

Most tests for AI use fake questions made up by computers (like "What is the capital of France?"). But real patients ask messy, emotional, and complex questions like, "My doctor said it's stage 4, but the biopsy wasn't clear. What now?"

The researchers went to a real help line for pancreatic cancer patients and grabbed 282 real questions from real people. They didn't just ask the AI to answer; they asked: "Is this answer safe and helpful?"

2. The "Human Grader" vs. The "Robot Judge"

To grade the answers, the researchers didn't just use a computer. They hired oncology fellows (doctors in training who specialize in cancer) to create a "grading rubric."

The Rubric is like a Recipe: Imagine a recipe for a perfect cake. It says: "Must have flour (5 points), must have sugar (5 points), must NOT have salt (-10 points)."
The doctors wrote these "recipes" for every single question.
Then, they used a super-smart AI (GPT-5) to act as the Judge, checking the other robots' answers against these human-written recipes.

The Result: The AI Judge was almost as good as the human doctors at grading. This means we can use AI to grade AI answers in the future, saving time and money!

3. The Shocking Findings

When they ran the test, here is what happened:

Confidence $\neq$ Correctness: Some of the newest, "smartest" reasoning models (like o3) got the highest scores on the "recipe" test. They sounded perfect! BUT, they were also the ones making the most dangerous factual mistakes. It's like a student who writes a beautiful essay but gets the math wrong.
The "Hallucination" Rate: This is when the AI makes things up.
- The best models (like GPT-5 and Gemini) only made up facts about 6% of the time.
- Some open-source models (free models anyone can download) made up facts in 54% of their answers! That's like flipping a coin to decide if you should take your medicine.
The "Google Search" Trap: The researchers turned on the "Web Search" button for the robots, hoping they would look up the facts.
- Surprise: It didn't always help! Sometimes, when the robot started searching the internet, it forgot the good information it already knew. It's like a chef who stops cooking to read a recipe book and burns the soup because they got distracted.
- Also, the robots sometimes found bad links (like a random blog) instead of reliable medical journals.

4. The "AI vs. Human" Rubric Experiment

The researchers asked: "Can we just let an AI write the grading recipe (rubric) instead of hiring doctors?"

The Answer: No.
When AI wrote the grading recipes, the robots got much higher scores (inflated by nearly 18 points).
It's like if a student wrote their own test questions and then graded their own answers. They would get an A+, but they wouldn't actually know the material. Human experts are still needed to set the rules.

The Bottom Line

This paper is a warning label for the future of AI in healthcare.

Don't trust the "smartest" sounding AI: The model that sounds the most confident isn't always the most accurate.
Human experts are irreplaceable: We need doctors to help design the tests and check the facts.
Open-source models need work: The free models are currently too risky for serious medical advice because they lie too often.

In short: PanCanBench is a reality check. It shows that while AI is getting better at talking like a doctor, it still needs a human supervisor to make sure it's not acting like a dangerous charlatan.

1. Problem Statement

While Large Language Models (LLMs) have achieved expert-level performance on standardized medical multiple-choice exams (e.g., USMLE via MedQA), their performance on real-world clinical inquiries remains unverified. Current benchmarks suffer from three critical limitations:

Lack of Disease-Specific Depth: Existing frameworks (e.g., HealthBench) rely on synthetic questions and broad healthcare topics, failing to capture the high-stakes complexity of specific diseases like pancreatic cancer.
Inadequate Safety Metrics: High scores on rubric-based evaluations do not guarantee factual correctness. Models may provide clinically complete answers that contain dangerous hallucinations.
Scalability vs. Rigor Trade-off: Creating rigorous, human-curated evaluation rubrics is time-intensive and costly, limiting the ability to scale evaluations across diverse models and queries.

The authors aim to bridge this gap by creating a benchmark that evaluates LLMs on authentic, de-identified patient questions regarding pancreatic cancer, focusing on clinical completeness, factual accuracy, and the reliability of AI-generated evaluation criteria.

2. Methodology

2.1 Dataset Construction (PanCanBench)

Source: 282 de-identified questions collected from the Pancreatic Cancer Action Network (PanCAN) Patient Services Help Line.
Categorization: Questions were grouped into six categories: Treatment & Clinical Trials, Diagnosis & Screening, Supportive & Palliative Care, Genetics & Risk, Side Effects & Symptom Management, and General.
Rubric Development: A human-in-the-loop pipeline was employed to create 3,130 question-specific evaluation criteria (rubrics).
- Phase 1: Two oncology fellows independently designed rubrics for assigned questions.
- Phase 2: AI models (GPT-4.1, Claude-Sonnet-4) polished rubrics and performed "gut checks" by scoring model pairs to ensure consistency.
- Phase 3: AI semantically merged redundant criteria while retaining unique insights from each fellow.
- Phase 4: A third fellow reviewed and finalized the rubrics.
Rubric Design: Criteria are binary (full credit or zero), written as complete sentences, and include negative points for misleading content.

2.2 Evaluation Framework

The study evaluated 22 LLMs (including proprietary models like GPT-5, o3, Gemini-2.5 Pro, and open-source models like Llama-3.1, Qwen-3) using three distinct evaluation dimensions:

Rubric-Based Completeness (LLM-as-a-Judge):
- Grader: GPT-5 was used as the automated judge to score responses against human-curated rubrics.
- Validation: The LLM judge achieved a Cohen's $\kappa$ of 0.528 against human experts, comparable to the inter-rater reliability between human experts (0.518), validating its use as a scalable proxy.
- Scoring: Final scores were calculated as a percentage of total possible points, accounting for both information inclusion and penalties for errors.
Factual Error Detection (Hallucination Analysis):
- Pipeline: Responses were decomposed into atomic factual claims.
- Verification: Two independent judges (GPT-5 and Gemini-2.5 Pro) with web-search capabilities verified each claim. A claim was labeled an error only if both judges flagged it as incorrect.
- Metric: The percentage of responses containing at least one factual error.
Web Search Integration:
- Evaluated whether enabling web search improved performance.
- Metrics included: Search triggering rate, resource appropriateness (e.g., citing PMIDs vs. blogs), and supportive link percentage (verified via LLM-as-a-judge).

2.3 Comparative Experiment: Human vs. AI Rubrics

A controlled experiment compared scores generated using human-curated rubrics versus AI-generated rubrics (where GPT-5 was tasked with creating rubrics based on the same guidelines given to human fellows).

3. Key Results

3.1 Model Performance on Rubric Completeness

Top Performers: The reasoning-optimized model o3 achieved the highest average score (82.3%), followed by Grok-4 (80.4%) and GPT-5 (78.4%).
Open-Source Performance: Olmo3-32B-think led open-source models with 66.3%.
Variation: Scores ranged widely from 46.5% to 82.3%, indicating significant disparity in clinical completeness.

3.2 Factual Accuracy and Hallucinations

Critical Finding: High rubric scores did not correlate with factual accuracy.
- o3 (highest rubric score) had a hallucination rate of 12.9% (inferred from discussion context) or higher relative to other GPT models.
- Llama-3.1-8B exhibited a massive hallucination rate of 53.8%.
- Gemini-2.5 Pro and GPT-4o had the lowest error rates at 6.0%.
Clinical Impact: Errors were clinically dangerous. For example, Llama-3.1-8B incorrectly defined Stage III pancreatic cancer as metastatic disease (Stage IV), a distinction that drastically alters treatment prognosis and patient psychology.

3.3 Impact of Web Search

No Net Gain: Enabling web search did not statistically improve overall scores for top models (e.g., GPT-5 and Gemini-2.5 Pro showed no significant improvement, $p > 0.05$ ).
Knowledge Crowding Out: Models sometimes omitted critical internal knowledge when search was enabled.
Retrieval Quality: While models triggered searches correctly (>80%), they often retrieved low-quality sources (generic blogs) instead of high-authority peer-reviewed literature required by the rubrics.

3.4 Human vs. AI Rubrics

Score Inflation: Using AI-generated rubrics inflated average scores by 17.9 points compared to human rubrics.
Ranking Stability: Despite the score inflation, the relative ranking of models remained largely stable (18 of 22 models shifted by only one position or less).
Conclusion: AI rubrics are insufficient for absolute scoring but may serve as a scalable proxy for relative model ranking.

4. Key Contributions

PanCanBench Dataset: The first benchmark comprising 282 authentic patient questions and 3,130 expert-designed rubrics specifically for pancreatic oncology.
Human-in-the-Loop Pipeline: A validated methodology for generating consistent, scalable evaluation rubrics that balances human expertise with AI efficiency.
Decoupling Completeness from Factuality: Demonstrated that "completeness" (rubric score) and "factuality" (hallucination rate) are orthogonal metrics; a model can be comprehensive yet dangerously inaccurate.
Web Search Limitations: Provided empirical evidence that current web-search integration in LLMs can degrade performance by "crowding out" internal knowledge and retrieving non-rigorous sources.
AI Rubric Analysis: Quantified the score inflation caused by AI-generated rubrics, highlighting the necessity of human oversight for rigorous safety evaluation.

5. Significance and Implications

Clinical Safety: The study underscores that deploying LLMs for cancer care requires more than just high accuracy on multiple-choice tests. Hallucination rates in open-source models (exceeding 50%) pose severe risks to patient safety, potentially leading to premature abandonment of curative treatments.
Evaluation Standards: The findings suggest that future medical AI benchmarks must explicitly measure factual grounding and source attribution alongside clinical completeness.
Role of Human Expertise: While AI can assist in scaling evaluations, human expertise remains indispensable for creating rigorous, safe evaluation standards, particularly for high-stakes medical domains.
Open-Source Gap: There is an urgent need to develop medically aligned, robust open-source models to ensure equitable access to safe AI-driven health information, as proprietary models often remain behind paywalls.

In conclusion, PanCanBench establishes that while LLMs show promise, significant gaps in factual reliability and information integration persist. The benchmark serves as a scalable, high-agreement proxy for rigorous clinical evaluation, emphasizing that human expertise is non-negotiable for ensuring AI safety in oncology.