💬 NLP

Hallucination Mitigating for Medical Report Generation

This paper proposes KERM, a knowledge-enhanced framework that integrates MedCLIP-based retrieval, a context purification module, and fine-grained reinforcement learning to mitigate hallucinations and improve the clinical accuracy of medical report generation.

Original authors: Ruoqing Zhao, Runze Xia, Piji Li

Published 2026-01-23

📖 4 min read☕ Coffee break read

CC0 1.0

Original authors: Ruoqing Zhao, Runze Xia, Piji Li

Original paper dedicated to the public domain under CC0 1.0 (http://creativecommons.org/publicdomain/zero/1.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a highly intelligent, well-read robot doctor (a Large Vision-Language Model, or LVLM) that looks at X-rays and tries to write a report for a human doctor. This robot is incredibly smart, but it has a bad habit: it likes to make things up.

In the medical world, this is called a "hallucination." It's like the robot seeing a healthy heart but confidently writing, "The patient has a mild heart enlargement," just because it sounds plausible. In real life, this could lead to a patient getting the wrong treatment.

The authors of this paper, Ruoqing Zhao and colleagues, built a new system called KERM to fix this. Think of KERM as a three-step "fact-checking and coaching" system for our robot doctor.

1. The "Librarian" Step (Knowledge Enhancement)

Before the robot looks at the X-ray, the system acts like a super-fast librarian.

The Problem: The robot might not remember every specific medical fact about a rare condition.
The Solution: The system searches a massive, curated library of medical facts (a "Knowledge Corpus") to find sentences that match the X-ray. For example, if the X-ray shows a specific type of lung spot, the librarian finds a sentence from a medical textbook describing exactly that spot.
The "Purification" Filter: Sometimes the librarian brings back too many books, some of which don't fit the specific patient's story (like their history or symptoms). The system has a "Purification Module" that acts like a strict editor, throwing away the irrelevant facts and keeping only the ones that perfectly match the patient's current situation. This ensures the robot starts with the right context.

2. The "Strict Coach" Step (Fine-Grained Rewards)

Once the robot writes a draft report, it doesn't just get a "Good job!" or a "Bad job!" It gets graded on two specific levels by a strict coach (using AI tools like GPT-3.5 and medical classifiers):

Level 1: The Disease Checklist (Disease-Level Reward): The coach checks, "Did you mention the pneumonia? Did you forget the broken bone?" If the robot invents a disease that isn't there, it gets a penalty. If it misses a real one, it gets a penalty. It's like a teacher grading a multiple-choice test for accuracy.
Level 2: The Sentence Flow (Sentence-Level Reward): Even if the facts are right, the sentence might sound weird or unnatural. The coach reads the sentence and asks, "Does this sound like something a real doctor would say? Is it logical?" If the robot writes a sentence that is technically true but sounds weird or out of place, it gets a lower score.

3. The "Training Loop"

The robot learns from these scores. Instead of just guessing, it uses a mathematical method (Reinforcement Learning) to adjust its brain. It learns: "When I see this type of X-ray, I should look up that specific fact from the library, and I should write sentences that get a high score from the coach."

The Results

The authors tested this system on two huge databases of real X-rays and reports (IU-Xray and MIMIC-CXR).

The Outcome: The KERM system wrote reports that were much more accurate than previous methods. It made fewer mistakes (hallucinations) and used medical language that sounded more natural and reliable.
The Analogy: If previous models were like a student who memorized a few facts and guessed the rest, KERM is like a student who has a textbook open, checks their work against a rubric, and gets graded by a strict teacher before handing in the final paper.

Important Limitations (What the paper says)

The authors are honest about what their system can't do yet:

It depends on the library: If the medical library they built is missing a rare fact or has old information, the robot might still get it wrong.
It's not perfect: The "Purification" filter might not catch every tiny nuance of a complex patient history.
It's expensive: Running this system requires powerful computers, which might be hard for smaller hospitals to afford right now.

In short, KERM is a way to stop AI doctors from "making things up" by giving them a reference book to check and a strict teacher to grade their work, resulting in safer and more accurate medical reports.

Technical Summary: Knowledge-Enhanced with Fine-Grained Reinforced Rewards Medical Report Generation (KERM)

Problem Statement

The generation of radiology reports from medical images is a critical yet labor-intensive task for radiologists. While Large Vision-Language Models (LVLMs) have demonstrated impressive capabilities in bridging visual and textual modalities, their application in the medical domain is hindered by a significant susceptibility to "hallucinations." These are instances where models generate plausible-sounding but factually inaccurate claims (e.g., misidentifying a pacemaker as heart enlargement).

Existing mitigation strategies, which often focus on refining training data or adjusting model architectures, have not fully resolved this issue. The authors identify two primary root causes:

Scarcity of High-Quality Annotations: Medical datasets often lack the precise, expert-level annotations required to guide models effectively, leading to the generation of incorrect information.
Long-Tail Distribution: Medical datasets frequently overrepresent common conditions while underrepresenting rare ones, causing model outputs to deviate from expected medical findings for less common cases.

Methodology: The KERM Framework

To address these challenges, the authors propose KERM (Knowledge-Enhanced with Fine-Grained Reinforced Rewards Medical Report Generation). The framework operates on a pretrained LVLM (specifically LLaVA-1.5-7b) and integrates two core modules: Medical Knowledge Enhancement (MKE) and Reward Modeling via Fine-Grained Feedback (RM).

1. Medical Knowledge Enhancement (MKE)

This module aims to ground the LVLM's understanding in external, curated medical facts.

Knowledge Corpus Construction: A corpus of 100,000 medical fact sentences is constructed by parsing region descriptions from the MIMIC-CXR and CheXpert datasets.
Knowledge Retrieval: For a given input image, the system uses MedCLIP (a pretrained multimodal model) to retrieve the top-10 most relevant facts from the corpus based on cosine similarity between image and text embeddings.
Purification Module: To ensure the retrieved knowledge is contextually relevant to the specific patient, a purification module is employed. It constructs a context embedding ( $E_C$ ) derived from the patient's indications and clinical history. The initial retrieved facts are re-ranked based on their similarity to this context vector, and the top-5 most relevant facts are selected to form the final purified knowledge set ( $K'$ ). This set is fed into the LVLM alongside image features to guide report generation.

2. Reward Modeling via Fine-Grained Feedback (RM)

To mitigate hallucinations during training, KERM employs a reinforcement learning strategy with a dual-level reward system.

Disease-Level Reward ( $R_{dis}$ ): The system uses the CheXpert labeling tool to extract 14 medical terminologies from both the generated and reference reports. An F1-score is calculated for each label to assess the alignment of disease predictions with ground truth, balancing precision and recall.
Sentence-Level Reward ( $R_{sen}$ ): Leveraging GPT-3.5, the system evaluates the coherence and plausibility of generated sentences against the ground truth. GPT-3.5 is prompted to score the similarity of sentence pairs on a scale of 0 to 1, penalizing deviations from expected medical findings even if they are not outright incorrect.
Reinforcement Loss: The total reward ( $R_t$ ) for a generated sequence is a weighted combination of $R_{dis}$ and $R_{sen}$ , controlled by a hyperparameter $\alpha$ . This reward guides the optimization of the model via the Reinforce Algorithm, minimizing a combined loss function of cross-entropy and reinforcement loss.

Key Contributions

The paper outlines three primary contributions:

Knowledge-Enhanced Approach: The integration of a curated knowledge corpus sourced from public datasets (MIMIC-CXR, CheXpert) to fortify LVLM inputs. This ensures generated reports are grounded in accurate, relevant medical information, improving the model's ability to produce reliable descriptions.
Fine-Grained Reinforced Reward Modeling: The development of a dual-level reward system (disease-level and sentence-level) that specifically penalizes hallucinatory content. This promotes outputs that align closely with medical norms and mitigates the occurrence of factual errors.
Comprehensive Empirical Validation: Extensive experiments demonstrating that the proposed method outperforms existing state-of-the-art methods on both Natural Language Generation (NLG) metrics and clinical efficacy (CE) metrics.

Experimental Results

The KERM framework was evaluated on two widely used benchmarks: IU-Xray and MIMIC-CXR.

Natural Language Generation (NLG) Metrics: On the IU-Xray dataset, KERM achieved state-of-the-art results across all metrics, including BLEU-4 (0.182), METEOR (0.197), and ROUGE-L (0.388). On MIMIC-CXR, it surpassed existing methods in most NLG metrics, achieving comparable performance to the state-of-the-art in BLEU-4 and METEOR.
Clinical Efficacy (CE) Metrics: KERM significantly outperformed previous methods in F1-score, precision, and recall for disease detection. Notably, while its precision was slightly lower than the KM method, it achieved a superior F1-score (0.415 on MIMIC-CXR), indicating a better balance between identifying true positives and minimizing false positives/negatives.
Ablation Studies: Experiments confirmed that both the MKE and RM modules contribute significantly to performance. The MKE module provided substantial gains by providing detailed contextual information (e.g., lesion position and existence), while the RM module effectively reduced hallucinations. The integration of both modules yielded the best overall performance.
Case Studies: Qualitative analysis (Figure 5) demonstrated that KERM covers nearly all necessary medical terms and abnormalities found in ground-truth reports, whereas the base LVLM frequently missed crucial details or introduced hallucinations.

Significance and Claims

The authors claim that KERM offers a robust solution to the critical challenge of hallucinations in medical report generation. By combining external knowledge retrieval with a purification mechanism and a fine-grained reward system, the framework enhances the visual grounding of LVLMs beyond pretrained baselines.

The paper emphasizes that this approach does not merely improve statistical language metrics but significantly enhances clinical relevance and diagnostic accuracy. The authors assert that their method produces reports that are not only factually accurate but also aligned with the typical patterns observed in medical practice. However, the authors remain modest regarding the scope of their claims, acknowledging limitations such as the dependency on the quality of the knowledge corpus, the potential for the purification module to miss nuanced clinical narratives, and the computational costs associated with training large models. They conclude that while KERM shows promising results, further investigation is required to ensure generalizability across diverse datasets and medical domains.