Visual Alignment of Medical Vision-Language Models for Grounded Radiology Report Generation

1. Problem Statement

The paper addresses the critical challenge of Visual Hallucination in Large Medical Vision-Language Models (Med-VLMs) used for Radiology Report Generation (RRG).

• The Issue: Existing Med-VLMs often generate reports that are clinically plausible but factually incorrect or unsupported by the input image (e.g., reporting "consolidation" when none exists). This occurs because models rely heavily on text-heavy pretraining patterns rather than genuine cross-modal reasoning.
• Limitations of Current Solutions:
  • Preference Data: Methods using Direct Preference Optimization (DPO) require expensive, task-specific curated preference pairs. These often lack clinical context and optimize solely in the textual space, ignoring whether the report aligns with the image.
  • Retrieval-Augmented Generation (RAG): Approaches that retrieve similar reports introduce hallucinations by incorporating findings from other patients' images that do not match the current input. They also require training separate retrieval models.
• Core Question: Can a framework generate clinically accurate reports that are strictly grounded in the visual evidence of the specific input image, without relying on external preference data, retrieval databases, or additional annotations?

2. Methodology: The VALOR Framework

The authors propose VALOR (Visual Alignment of Medical Vision-Language Models for GrOunded Radiology Report Generation), a two-stage multimodal optimization framework. It operates without external preference data or retrieval systems.

Stage 1: Instruction-Following Warmup & Clinically Informed Textual Reasoning

• SFT Warmup: The model undergoes Supervised Fine-Tuning (SFT) on curated data to stabilize the policy and ensure the model respects the standard report structure (Findings and Impression sections).
• Textual Rewards: The model is optimized using a multi-tier reward system:
  • Verifiable Text Rewards ($R_{ver}$): Combines NLP metrics (BLEU, ROUGE, BERTScore) to ensure semantic coherence and token similarity with ground truth.
  • Clinical Rewards ($R_{clin}$): Periodically injected (every $k_{clin}$ steps) to enforce medical accuracy. These are calculated using micro-averaged F1 scores from CheXbert (for disease labels) and RadGraph (for entity triplets).
  • Strategy: By injecting clinical rewards periodically rather than at every step, the model avoids optimization instability while still learning precise medical terminology.

Stage 2: Self-Supervised Visual Reasoning

This is the core innovation of VALOR, designed to enforce visual grounding without external data.

• Frozen Domain Expert: A pre-trained, frozen multi-label domain expert model ($G$) is used to extract feature vectors from the input X-ray ($I$).
• Image-Text Similarity: For every generated candidate report ($o_i$), a text embedding is created. The cosine similarity ($S_i$) between the report's text embedding and the image's visual features is computed.
• Rank-Normalized Advantages: The similarity scores are converted into rank-normalized advantages ($r^{vis}_i$). The candidate report most consistent with the visual features receives the highest reward.
• Composite Reward: A final reward ($R_{vis}$) is computed by mixing the textual reward from Stage 1 and the visual reward. The policy is updated using Group Relative Policy Optimization (GRPO) to maximize this composite reward.
• Key Benefit: This forces the model to distinguish between candidates based on their agreement with the specific visual features of the input X-ray, effectively eliminating hallucinations caused by relying on general medical knowledge alone.

3. Key Contributions

1. Novel Framework (VALOR): A two-stage alignment framework that reduces visual hallucinations and strengthens image-report consistency without requiring external curated preference datasets or retrieval-based methods.
2. Self-Supervised Visual Grounding: Introduction of a mechanism using a frozen domain expert to compute image-text similarity. This converts visual alignment into a rank-normalized advantage signal, allowing the policy to learn which reports are visually grounded without human annotations.
3. Clinically Informed Rewards: Integration of verifiable natural language metrics with periodic clinical metric rewards (CheXbert/RadGraph F1 scores) to ensure precise medical terminology.
4. Comprehensive Evaluation: Extensive experiments on IU-XRay and MIMIC-CXR benchmarks, jointly evaluating generation quality and clinical accuracy—a combination rarely assessed together in prior work.

4. Experimental Results

VALOR was evaluated against baselines (SFT, DPO, GRPO), state-of-the-art (SOTA) post-training methods (MMedPO), and proprietary models (GPT-4o, MedFlamingo, MAIRA-2).

• Performance Gains:
  • Generation Quality: VALOR outperforms MMedPO (SOTA preference method) by approximately 20% in generation quality metrics (BLEU, ROUGE, METEOR).
  • Clinical Accuracy: VALOR achieves a 25% improvement in clinical accuracy metrics (F1, GREEN, RaTEScore) compared to baselines.
  • Zero-Shot Superiority: In zero-shot settings, VALOR surpasses proprietary medical and general multimodal LLMs by an average of 17% in generation quality.
• Specific Metrics (IU-XRay):
  • F1 Score: VALOR (57.2%) vs. MMedPO (49.8%) vs. Base LLaVA-Med (40.6%).
  • GREEN Score: VALOR (0.34) vs. MMedPO (0.26).
• Ablation Studies:
  • Visual Weight ($\lambda_{vis}$): Performance peaks when visual and textual rewards are balanced ($\lambda_{vis} \approx 0.5$). Purely visual or purely textual optimization leads to suboptimal results or training collapse.
  • Periodic Guidance: Periodic injection of clinical rewards ($k_{clin}$) is superior to dense (per-step) injection, which destabilizes training.

5. Significance and Impact

• Solving the "Alignment Gap": VALOR directly addresses the semantic gap between retrieved text and visual evidence by forcing the model to reason about the specific image provided, rather than relying on external knowledge bases.
• Cost-Effective Deployment: By eliminating the need for expensive preference data curation and retrieval model training, VALOR offers a scalable and cost-effective solution for clinical deployment.
• Clinical Reliability: The attention visualizations confirm that VALOR shifts the model's focus to disease-relevant anatomical regions (e.g., lung opacities), reducing false positives and "no abnormality" errors common in baseline models.
• Future Direction: The work charts a path toward self-supervised, image-aligned medical AI systems that prioritize factual accuracy and visual grounding over mere text fluency.