AI-generated data contamination erodes pathological variability and diagnostic reliability

Technical Summary: AI-Generated Data Contamination Erodes Pathological Variability and Diagnostic Reliability

Problem Statement
The rapid integration of generative artificial intelligence (AI) into clinical workflows has led to the proliferation of synthetic or partially AI-generated content within electronic health records (EHRs). A critical, unexplored risk is the formation of a "self-referential" feedback loop: as future medical AI models are trained on data increasingly populated by prior AI generations, they may inadvertently learn from uncurated synthetic outputs rather than authentic human observations. This phenomenon, analogous to "model collapse" observed in other domains, threatens to erode the pathological variability and diagnostic reliability essential for safe medical practice. Specifically, there is a concern that such cycles could systematically suppress rare but critical clinical findings, amplify demographic biases, and create a false sense of diagnostic confidence, ultimately rendering medical data ecosystems homogenized and clinically useless.

Methodology
The authors conducted a systematic investigation across three clinically central modalities: clinical text generation, vision–language radiology reporting, and medical image synthesis. The study employed a controlled, multi-generational self-referential training framework:

• Experimental Design: Models were initialized from original pre-trained checkpoints (Generation 0) and fine-tuned exclusively on synthetic outputs generated by the preceding generation (Generations 1–4). Crucially, each generation was reset to the original base parameters to isolate data-driven degradation from accumulated model drift.
• Datasets & Models:
  • Clinical Text: Utilized GPT-2 (124M parameters) and Qwen3-8B (8B parameters) on datasets including 216,307 MIMIC-CXR radiology reports, 790 i2b2 clinical notes, 1,000 glaucoma records, and MIMIC-IV discharge summaries.
  • Vision–Language: Employed R2GenGPT (Swin-Transformer + Llama-2) trained on 9,781 chest X-rays, where models viewed real images but were trained exclusively on synthetic reports from the previous generation.
  • Image Synthesis: Used a Denoising Diffusion Probabilistic Model (DDPM) with U-Net architecture on 5,534 chest X-rays to generate synthetic images for subsequent training cycles.
• Evaluation Metrics: The study quantified degradation through linguistic diversity (vocabulary size, Type-Token Ratio, perplexity), clinical knowledge retention (pathophysiological co-occurrence matrices, diagnostic term density), diagnostic safety (sensitivity to critical findings, false reassurance rates), and demographic distribution (age and gender bias).
• Physician Validation: A blinded evaluation by two independent physicians assessed 100 AI-generated discharge summaries across generations, measuring edit distance, content retention, and cognitive burden.
• Mitigation Testing: Three strategies were evaluated: (1) mixing real and synthetic data at varying proportions, (2) scaling synthetic data volume, and (3) quality-aware filtering of synthetic samples.

Key Results
The study demonstrates that unmitigated self-referential training leads to catastrophic degradation across all modalities:

• Linguistic and Structural Collapse: In text generation, vocabulary size plummeted by up to 98.9% (e.g., from ~12,000 to ~200 unique words in radiology reports) within four generations. Reports converged into rigid, formulaic templates, losing the natural variation required for clinical nuance.
• Erosion of Pathological Variability: Rare but critical findings (e.g., pneumothorax, masses, fractures) systematically vanished from synthetic outputs. In radiology reports, the detection rate for life-threatening conditions like pleural effusion dropped from >80% to near-zero. Conversely, models exhibited "false reassurance," tripling the rate of reporting "no acute findings" for images containing critical pathology (reaching 40% in vision–language models).
• Demographic Bias Amplification: Synthetic medical images and text increasingly skewed toward middle-aged male phenotypes. Female representation in synthetic images dropped from 47% to 31% by Generation 4, and age distributions narrowed significantly.
• The Confidence Paradox: A dangerous decoupling emerged where models became increasingly confident in their degraded outputs (low perplexity on synthetic data) while their ability to comprehend authentic clinical text collapsed (perplexity on real text increased 45-fold). Standard performance metrics failed to detect this clinical failure.
• Clinical Utility Loss: Physician evaluation revealed that by Generation 3, AI-generated content required near-complete rewriting (87.6% edit distance), rendering the output clinically useless despite high fluency.
• Mitigation Efficacy:
  • Real-Data Mixing: Mixing authentic human data with synthetic data (specifically at 50–75% real data proportions) was the only strategy that effectively preserved pathological diversity and prevented demographic skew.
  • Volume Scaling: Merely increasing the volume of synthetic data failed to prevent degradation and, in some cases, accelerated the loss of diversity and increased the confidence gap.
  • Quality-Aware Filtering: Filtering synthetic data for quality improved efficiency and preserved some diversity but could not fully substitute for the inclusion of substantial amounts of authentic human data.

Significance and Claims
The paper claims that without policy-mandated human oversight and strict data provenance controls, the deployment of generative AI threatens to degrade the very healthcare data ecosystems it relies upon. The authors establish that:

1. Model Collapse is Intrinsic to Medicine: The erosion of pathological variability is not an architecture-specific failure but a fundamental consequence of self-referential learning in medical contexts, where diagnostic value is concentrated in the "tails" of clinical distributions (rare diseases and atypical presentations).
2. Standard Metrics are Insufficient: Surface-level language quality metrics and standard performance monitoring are blind to this degradation, as models can maintain high fluency and confidence while losing diagnostic accuracy.
3. Real Data is Non-Substitutable: Synthetic data cannot replace authentic human data in training pipelines. Real human data acts as a necessary "biological anchor" to preserve diagnostic integrity.
4. Safety Imperatives: The authors propose minimum safety conditions for medical generative AI, including:
   • Explicit and persistent tagging of AI-generated content in EHRs to prevent silent contamination.
   • Enforcement of minimum proportions of verified human-authored data in retraining pipelines.
   • Mandatory clinician attestation for patient-facing AI documentation, as high fluency does not equate to clinical safety.

The study concludes that the unchecked integration of AI-generated content creates a monoculture optimized for statistical averages at the expense of individual patient safety, necessitating immediate governance interventions to preserve diagnostic reliability.