Expert Evaluation of LLM World Models: A High- Superconductivity Case Study
This study evaluates the ability of six LLM-based systems to answer expert-level questions about high-temperature superconductivity using a curated database of 1,726 papers, finding that retrieval-augmented generation (RAG) systems outperform closed models in providing comprehensive, well-supported answers while highlighting both the potential and current limitations of LLMs in specialized scientific domains.