Hallucination in Medical Imaging AI: A Cross-Modality Analytical Framework for Taxonomy, Detection, and Mitigation under Regulatory Constraints

Technical Summary: Hallucination in Medical Imaging AI

Problem Statement
The rapid deployment of Vision-Language Models (VLMs) and AI systems in medical imaging has outpaced the understanding of their failure modes. The most clinically critical failure mode identified is hallucination: outputs that are linguistically or visually coherent but factually unsupported by the input image or patient context. In medical settings, these errors manifest as fabricated anatomical structures, misclassified pathologies, incorrect laterality, invented measurements, or the omission of significant findings. Unlike cosmetic artifacts, these hallucinations directly impact high-stakes clinical decisions, including biopsy selection, cancer staging, and treatment planning.

Current literature suffers from three primary gaps:

1. Lack of Unified Taxonomy: Existing taxonomies are modality-specific (e.g., radiology vs. nuclear medicine) and have not been validated across the full spectrum of imaging (CT, MRI, PET/SPECT, ultrasound, digital pathology).
2. Unverified Model Assumptions: There is no direct empirical comparison between general-purpose foundation models and medical-specialized models regarding hallucination rates, despite the common procurement assumption that specialization reduces hallucination risk.
3. Limited Real-World Validation: Mitigation strategies are largely evaluated in controlled benchmarks, with insufficient evidence regarding their effectiveness in real-world clinical workflows and their alignment with regulatory frameworks like the FDA's Total Product Lifecycle (TPLC).

Methodology
This study employs a structured narrative synthesis rather than a formal meta-analysis, due to the heterogeneity of study designs, benchmark metrics, and outcome measures across the existing literature. The analysis integrates peer-reviewed studies, benchmark datasets, and FDA regulatory guidance (specifically the January 2025 draft guidance and December 2024 Predetermined Change Control Plan [PCCP] finalization).

The analytical framework addresses four dimensions:

1. Taxonomic Classification: Cross-referencing three distinct frameworks (Brooks & Anastasio, DREAM Report, and MediHall) to create a unified cross-modality perspective.
2. Etiological Analysis: Investigating causes of hallucination, including model architecture, training data quality, and inference conditions.
3. Detection Evaluation: Assessing five detection approaches (Uncertainty Quantification, Attention-based analysis, Reconstruction-based methods, Cross-modal verification, and Benchmarking) against criteria of precision, universality, and interpretability.
4. Mitigation Assessment: Evaluating strategies across four intervention layers: architectural constraints, retrieval-augmented grounding, inference-time prompt engineering, and human-in-the-loop safeguards.

Key Contributions and Results

• Unified Taxonomic Framework: The study demonstrates that no single existing taxonomy covers the imaging pipeline comprehensively. A unified framework is proposed by integrating:

  • Brooks & Anastasio: Etiological origin (data failure vs. model confabulation).
  • DREAM Report: Modality-specific physics and clinical risk (e.g., PET/SPECT uptake patterns).
  • MediHall: Clinical severity grading (catastrophic to minor).
    This integration allows for consistent characterization of hallucinations across CT, MRI, PET/SPECT, ultrasound, and digital pathology.

• The General vs. Specialized Model Inversion: Contrary to common assumptions, general-purpose foundation models outperform medical-specialized models on hallucination-specific benchmarks.

  • On the Med-HALT benchmark (2025), general-purpose models achieved a median hallucination-free rate of 76.6%, compared to 51.3% for medical-specialized models (p = 0.012).
  • The authors attribute this to overfitting-induced rigidity in specialized models, where narrow domain priors override visual evidence in ambiguous cases. General-purpose models, trained on diverse data, maintain better uncertainty calibration.
  • Note: This finding applies to language-mediated reasoning tasks; specialized architectures may still hold advantages in purely reconstruction-based tasks (e.g., low-dose CT denoising) where physics-informed biases are critical.

• Detection and Mitigation Effectiveness:

  • Detection: No single method is universally effective. Uncertainty Quantification (UQ) offers high automation but requires retraining; Attention-based analysis aids interpretability but is architecture-dependent; Reconstruction-based methods (e.g., sFRC) are essential for AI-reconstructed MRI/CT; Cross-modal verification is clinically intuitive but data-intensive.
  • Mitigation:
    • Architectural Constraints: Physics-informed constraints (e.g., PI-MoCoNet) improve reconstruction fidelity (approx. 1 dB PSNR improvement) but are limited to the development phase.
    • Prompt Engineering: Chain-of-Thought (CoT) prompting reduces hallucinations in up to 86.4% of evaluated comparisons by forcing explicit intermediate reasoning.
    • Human-in-the-Loop (HITL): Radiologist oversight remains the most decisive intervention, reducing AI false-positive rates by approximately 83.7%. However, this introduces resource costs and risks of automation bias.

• Regulatory Alignment: The study maps all findings to the FDA's TPLC and PCCP frameworks. It emphasizes that hallucination management is a continuous lifecycle obligation, not a pre-deployment checklist. Strategies must support post-market surveillance, generate audit trails, and accommodate continuous refinement through predefined change control processes.

Significance and Claims
The paper claims to provide the first cross-modality analytical framework that unifies taxonomy, etiology, detection, and mitigation under regulatory constraints. Its primary significance lies in challenging the procurement dogma that "specialization equals safety," demonstrating instead that narrow fine-tuning can introduce confabulation.

The authors conclude that:

1. Unified Assessment is Necessary: Clinical deployment requires integrating multiple taxonomic perspectives to avoid blind spots in risk assessment.
2. Benchmarking Must Be Specific: Procurement decisions should rely on hallucination-specific benchmarks rather than general accuracy metrics, as general-purpose models may offer superior robustness against fabrication.
3. Layered Mitigation is Required: No single mitigation strategy is sufficient. Effective deployment requires a combination of architectural constraints (where possible), inference-time reasoning (CoT), and rigorous human oversight, all managed within a regulatory-compliant lifecycle framework.

The study acknowledges limitations, including the reliance on structured narrative synthesis, the scarcity of data for pediatric and rare disease contexts, and the fact that most evidence derives from controlled benchmarks rather than prospective clinical deployments. Future work is directed toward unified cross-modality taxonomies, direct comparative evaluations in reconstruction tasks, and the integration of hallucination monitoring into real-world clinical workflows under FDA-compliant change control.