Impact of Label Noise from Large Language Models Generated Annotations on Evaluation of Diagnostic Model Performance

1. Problem Statement

As Large Language Models (LLMs) are increasingly adopted to extract structured labels from unstructured radiology reports for large-scale AI evaluation, a critical vulnerability has emerged: label noise.

• The Issue: LLMs are not perfect; they introduce errors (false positives and false negatives) when acting as a reference standard.
• The Consequence: When these imperfect LLM-generated labels are used as the "ground truth" to evaluate downstream diagnostic AI models, the errors propagate, leading to biased performance estimates.
• The Gap: There is a lack of systematic understanding regarding how LLM sensitivity/specificity and disease prevalence interact to distort the observed performance (sensitivity and specificity) of diagnostic models. This can lead to the misjudgment of a model's clinical utility, particularly in rare disease scenarios.

2. Methodology

The authors developed a simulation framework to quantify the impact of LLM label noise on model evaluation. The study utilized a synthetic dataset of 10,000 cases and employed both analytical bounds and empirical simulations.

• Experimental Design:
  • Disease Prevalence: Varied across four levels: 10%, 30%, 70%, and 90% (representing rare to common conditions).
  • LLM Performance: Simulated LLM sensitivity and specificity were independently varied from 90% to 100% in 1% increments.
  • Target Model Performance: Simulated diagnostic models with true performance ranging from 90% to 100% sensitivity/specificity.
• Evaluation Metrics:
  • Observed Performance: Calculated by treating LLM-generated labels as the reference standard against the model's predictions.
  • Error Bounds: Analytically calculated Best-Case (maximized agreement) and Worst-Case (minimized agreement) performance bounds.
  • Monte Carlo Simulation: Conducted 5,000 trials per condition to generate empirical uncertainty distributions and validate theoretical bounds.
• Scenarios Tested:
  1. Effect of prevalence on observed performance (fixing model at 100% accuracy).
  2. Impact of LLM quality on performance estimation (fixing model at 95% accuracy).
  3. Impact of varying model accuracy (90% vs. 98%) on error magnitude.

3. Key Contributions

• Quantification of Prevalence-Dependent Bias: The study establishes that the impact of LLM errors is not uniform; it is heavily modulated by disease prevalence.
• Identification of Asymmetric Vulnerability:
  • Low Prevalence (Rare Diseases): Observed Sensitivity is highly vulnerable to LLM Specificity errors. Even small drops in LLM specificity cause massive underestimation of model sensitivity.
  • High Prevalence (Common Diseases): Observed Specificity is highly vulnerable to LLM Sensitivity errors.
• Empirical Validation of Systematic Underestimation: Through Monte Carlo simulations, the authors demonstrated that label noise does not just create random uncertainty; it introduces a systematic downward bias, where estimated performance often falls below the true model performance, even when theoretical error bounds suggest the true value is reachable.
• Strategic Guidance for Prompt Engineering: The findings provide a data-driven basis for "prevalence-aware" prompt engineering, suggesting that optimization goals (sensitivity vs. specificity) must shift based on the target condition's frequency.

4. Key Results

• Low Prevalence (10%):
  • A model with 100% true sensitivity was observed to have only ~53% sensitivity when evaluated against an LLM with 95% specificity (and 100% sensitivity).
  • Mechanism: In rare disease settings, a slight increase in False Positives (due to low LLM specificity) overwhelms the small number of True Positives, drastically skewing the sensitivity calculation.
  • Specificity Stability: Observed specificity remained robust (>98%) even with moderate LLM noise.
• High Prevalence (90%):
  • The pattern inverted. Reduced LLM sensitivity led to a sharp drop in observed specificity (from 100% to ~53%), while sensitivity remained stable.
• Error Magnitude:
  • At 10% prevalence, reducing LLM specificity from 100% to 95% increased the uncertainty in estimated sensitivity by up to 24%.
  • Improving LLM specificity had a far greater impact on reducing estimation error than improving LLM sensitivity in low-prevalence scenarios.
• Model Performance Independence:
  • The pattern of bias (underestimation driven by prevalence and LLM specificity) remained consistent across models with true performance ranging from 90% to 98%. While higher-performing models had slightly narrower error ranges, the direction of the bias was unchanged.

5. Significance and Implications

• Clinical AI Deployment: The study warns that using LLMs for post-deployment evaluation without accounting for label noise can lead to the premature rejection of effective models (due to underestimated sensitivity) or the deployment of unsafe models (due to underestimated specificity).
• Prompt Engineering Strategy:
  • For rare findings, prompts must be optimized to maximize Specificity (minimize false positives), even at the cost of some sensitivity.
  • For common findings, prompts should prioritize Sensitivity.
• Reporting Standards: The authors advocate for reporting prevalence-aware error bounds alongside performance metrics. Clinicians and researchers should interpret LLM-derived evaluation results with caution, acknowledging that the "observed" performance is likely a lower bound of the true performance.
• Future Research: Highlights the need for hybrid evaluation pipelines that combine LLM efficiency with human-in-the-loop verification, particularly for rare conditions where LLM specificity is critical.

In conclusion, this paper provides a rigorous mathematical and empirical demonstration that LLM label noise is a systematic, prevalence-dependent confounder in medical AI evaluation, necessitating a fundamental shift in how reference standards are constructed and interpreted.