Rater choice determines measured fidelity: a multi-model evaluation of large language model raters for AI-generated plain-language biomedical summaries
This study demonstrates that the choice of large language model rater significantly influences measured fidelity and safety compliance in AI-generated biomedical summaries, with different models producing error rates that vary by a factor of four and leading to divergent conclusions about summary quality.