Healthy LLMs? Benchmarking LLM Knowledge of UK Government Public Health Information

This paper introduces PubHealthBench, a new benchmark comprising over 8,000 questions derived from UK government guidance, to evaluate LLMs on public health knowledge and finds that while state-of-the-art models excel in multiple-choice tasks, their performance on free-form responses remains limited, highlighting the need for additional safeguards in real-world applications.

The Gist

Imagine you've just hired a new, incredibly smart assistant named "The AI." You want to know if this assistant is ready to help you make important health decisions for your family. You ask, "Do you know the latest rules about flu shots?" or "What should I do if I get a rash?"

This paper is essentially a report card for 24 of these AI assistants, testing their knowledge of the UK Government's public health advice. The researchers from the UK Health Security Agency (UKHSA) built a giant, custom-made exam called PubHealthBench to see how well these AIs perform.

Here is the breakdown of their findings, using some simple analogies:

1. The Exam: A Massive Library Quiz

The researchers didn't just make up random questions. They took over 687 official government health documents (like a massive library of rulebooks) and used a robot to turn them into 8,000+ questions.

• The Multiple Choice Test (MCQA): This is like a standard school exam. The AI is given a question and a list of answers (A, B, C, D, etc.). It just has to pick the right one.
• The Free-Form Test: This is like a conversation. The AI has to explain the answer in its own words, just like a real chatbot would.

2. The Results: The "Smart" vs. The "Chatty"

The Multiple Choice Test: The A-Students

When the AI took the multiple-choice test, the top models (like GPT-4.5 and o1) were brilliant.

• The Score: They got over 90% correct.
• The Comparison: They actually did better than a regular human who was allowed to use Google to look up the answers (but wasn't a medical expert).
• The Analogy: Imagine a student who has memorized the entire encyclopedia. If you ask, "What is the capital of France?" or "What is the rule for X?", they can instantly point to the right page. They are incredibly good at retrieving facts when given a hint (the multiple-choice options).

The Free-Form Test: The "Hallucinating" Storyteller

When the researchers asked the same AIs to explain the answers in their own words (without the multiple-choice hints), the scores dropped significantly.

• The Score: No model scored above 75%.
• The Problem: The AIs started to "hallucinate." This is like a student who knows the facts but gets nervous and starts making up details to sound smart. They might add extra advice that isn't in the official rules, or they might get the timing of a medical intervention slightly wrong.
• The Analogy: Imagine asking that same encyclopedia-student to write an essay. Instead of just stating the facts, they might start rambling, inventing a story about a "new" rule that doesn't exist, or forgetting a crucial detail because they are trying to be too conversational.

3. The "Public" vs. The "Pro"

The researchers noticed something interesting about what the AIs knew best:

• Public Advice: The AIs were great at answering questions meant for regular people (e.g., "When should I wash my hands?").
• Clinical Advice: They struggled more with complex advice meant for doctors and specialists.
• The Analogy: Think of the AI as a tour guide. They are excellent at telling tourists (the public) where the nearest bathroom is or what the opening hours are. But if you ask them for a detailed architectural blueprint of the building (clinical advice), they might get confused or give you a vague answer.

4. The Small vs. The Big Models

There was a huge gap between the "Super AIs" (expensive, proprietary models) and the "Mini AIs" (smaller, open-source models).

• The Big AIs: Like a senior professor with a PhD. They are reliable but still make mistakes when talking freely.
• The Small AIs: Like a bright high schooler. They can do okay on a multiple-choice test, but when asked to write an essay, they are much more likely to make up facts. The gap in performance between the big and small models grew massive in the free-form test.

The Bottom Line: A Warning Label

The paper concludes with a very important message for anyone thinking about using AI for health advice:

"The AI is a great librarian, but a risky doctor."

• Good News: If you use AI to find information (like a search engine), the latest models are very accurate and can help you find the right government guidance quickly.
• Bad News: If you let the AI explain the advice or give you a plan in a chat, it might invent rules or miss crucial details.

The Takeaway: We shouldn't just trust the AI to give us medical advice directly. We need to treat it like a helpful assistant that points you to the official rulebook, but we must always double-check its "free-form" stories with a human expert or the official government website. The AI is getting smarter, but it still needs a safety net.

Technical Summary

Here is a detailed technical summary of the paper "HEALTHY LLMS? BENCHMARKING LLM KNOWLEDGE OF UK GOVERNMENT PUBLIC HEALTH INFORMATION".

1. Problem Statement

As Large Language Models (LLMs) become integrated into real-world workflows, there is a critical need to evaluate their knowledge in high-stakes domains like public health. While medical benchmarks exist (e.g., USMLE), there is a significant gap in evaluating LLMs against public health guidance, which is distinct from clinical medicine as it covers prevention, environmental hazards, and population-level interventions.

Key Challenges Identified:

• Dynamic Nature of Guidance: UK Government public health guidance is frequently revised, making it difficult for static training data to remain current.
• Risk of Hallucination: Inaccurate or outdated advice from LLMs could significantly impact public health decisions and resident safety.
• Lack of Benchmarks: No comprehensive benchmark existed to test LLMs specifically on UK Government public health documents (gov.uk).

2. Methodology: PubHealthBench

The authors introduce PubHealthBench, a new benchmark comprising over 8,000 questions derived from 687 current UK Government guidance documents. The methodology involves an automated pipeline to ensure scalability and reproducibility.

A. Data Collection & Pre-processing

• Source: 1,150 current UK Government guidance documents (HTML and PDF) from gov.uk.
• Extraction:
  • HTML documents were converted to Markdown.
  • PDFs were processed via a two-stage pipeline: raw text extraction followed by GPT-4o-mini (vision) to extract text from images and headers.
• Chunking: Documents were split into 20,488 sections based on Markdown headers, preserving hierarchy for context.

B. Automated Question Generation

• Filtering: An LLM classified chunks to retain only those containing actionable public health recommendations, filtering out operational details. Chunks >2,000 words were excluded.
• Generation: Using Llama-3.3-70B-Instruct, the system generated two Multiple Choice Questions (MCQs) per valid chunk.
  • Format: 1 correct answer + 6 plausible distractors (7 options total).
  • Context: The generator was provided with the target chunk plus surrounding text to ensure context independence.
  • Output: JSON format with Chain-of-Thought (CoT) reasoning.

C. Quality Assurance & Validation

• Automated Error Detection: A second LLM (Llama-3-70B) filtered out questions with potential errors (e.g., ambiguous options, incorrect answers). This reduced the candidate set from ~15,600 to ~14,400.
• Human Review: A random sample of 800 questions (~10%) was manually reviewed by human experts.
  • Findings: Estimated invalid/ambiguous rate was ~5.5%.
  • Upper Bound: Due to the nature of distractors (often having two plausible answers), the theoretical upper bound for human performance was estimated at ~97%.
• Final Dataset: 8,090 MCQs covering 10 topic areas and 352 guidance sub-areas.

D. Evaluation Setup

The benchmark was tested in three modes:

1. PubHealthBench-Full: The complete MCQA dataset.
2. PubHealthBench-Reviewed: A manually verified subset (760 questions) used for cost-effective evaluation of expensive models.
3. PubHealthBench-FreeForm: The same verified subset but without MCQ options, requiring open-ended responses.
   • Judging: A "LLM-as-a-Judge" (GPT-4o-Mini) evaluated free-text answers against the ground truth and source chunks, checking for consistency and hallucinations.

Human Baseline: 5 non-expert humans took a 600-question sample with search engine access (no AI tools), achieving an average score of 88%.

3. Key Contributions

1. PubHealthBench Dataset: The first comprehensive benchmark for UK public health knowledge, featuring 8,000+ MCQs and a free-form evaluation set.
2. Automated Pipeline: A scalable, reproducible pipeline for generating, filtering, and validating synthetic MCQs from government documents.
3. Comprehensive Evaluation: Evaluation of 24 LLMs (including GPT-4.5, o1, Claude-Sonnet-3.7, Llama-3.3, and Gemma-3) across both MCQA and free-form formats.
4. Human Baseline & Upper Bound: Established a human baseline (88%) and a theoretical upper bound (~97%) to contextualize model performance.

4. Key Results

A. MCQA Performance (Multiple Choice)

• Top Performers: The latest proprietary models (GPT-4.5, GPT-4.1, o1) achieved >90% accuracy, outperforming the human baseline (88%) and nearing the theoretical upper bound.
• Open-Weight Models: Mid-sized models (5B–70B parameters) generally scored between 75% and 88%.
• Reasoning Models: "Reasoning" models (o1, o3-Mini) performed similarly to non-reasoning models in the MCQA setting, suggesting the extra compute time offered little benefit for static knowledge retrieval.
• Topic Variance: Models performed best on Climate and Health and Public Guidance (96% for GPT-4.5) but struggled more with Chemicals and Toxicology.

B. Free-Form Performance (Open-Ended)

• Significant Drop: Performance degraded drastically in the free-form setting. No model scored above 75%.
• Best Model: o1 achieved the highest score at 74%, a drop of only 17 percentage points from its MCQA score.
• Worst Performers: Smaller models (e.g., Llama-3.1-8B, Phi-4-4B) saw drops of 40–60 percentage points, scoring as low as 18–19%.
• Error Types: The primary failure modes were omission of key guidance points and contradiction of official advice (hallucinations regarding timing or intervention types).

C. Audience & Domain Analysis

• Public vs. Clinical: Models consistently performed better on guidance intended for the general public compared to clinical or professional guidance. This is significant as the general public is the primary user base for chatbots.
• Model Size Gap: The performance gap between top-tier proprietary models and smaller open-weight models widened significantly in free-form tasks (from ~10-20% in MCQA to >35% in Free-Form).

5. Significance and Implications

• Readiness for Deployment: State-of-the-art (SOTA) LLMs possess a high degree of accurate knowledge regarding UK public health, suggesting they are viable for information retrieval if safeguards are in place.
• The "Free-Form" Risk: While models excel at selecting the right answer from a list (MCQA), they struggle to generate accurate, complete, and non-contradictory free-text responses. This indicates a high risk of hallucination in real-world chatbot deployments.
• Need for Safeguards: The results suggest that for free-form public health queries, Retrieval-Augmented Generation (RAG) or strict grounding mechanisms are essential to prevent the omission or contradiction of critical guidance.
• Future Work: The benchmark provides a foundation for evaluating tool-use, multi-turn interactions, and non-English public health guidance.

Conclusion: The paper concludes that while SOTA LLMs are increasingly accurate sources of public health information, the degradation in performance during free-form generation necessitates additional tools or safeguards before widespread real-world deployment in critical public health scenarios.