RheumBench: A Specialist-Validated Rubric-Based Benchmark for Evaluating Large Language Models in Rheumatology
RheumBench is the first specialist-validated, rubric-based benchmark for rheumatology that evaluates ten LLMs and RAG systems, revealing that current models exhibit substantial performance gaps, systematic overconfidence, and potentially severe omission errors, thereby underscoring the critical need for human oversight in clinical decision support.