Performance of Five Mainstream Large Language Models on Common Small Cell Lung Cancer Questions: Readability, Patient- Education Suitability, and Information Quality

Imagine a patient newly diagnosed with a serious illness, sitting in a quiet room, trying to understand what is happening to their body. They turn to the internet, hoping to find clear answers about their condition, the treatments available, and what the future might hold. In recent years, many have begun asking these questions to artificial intelligence systems, hoping for immediate, human-like explanations. These systems, known as large language models, are trained on vast amounts of text and can generate fluent, coherent answers to almost any question. However, fluency does not guarantee clarity or accuracy. For a patient facing a complex disease, the difference between a helpful explanation and a confusing jumble of medical terms can be the difference between feeling empowered and feeling lost. This is particularly true for small cell lung cancer, a fast-growing and aggressive form of the disease where treatment plans change quickly and the information a patient needs is dense and technical.

A team of researchers from hospitals in Dalian, China, set out to test how well five of the most popular artificial intelligence systems perform when asked to explain this specific disease to a layperson. They did not just ask if the answers were correct; they asked if the answers were actually useful for a patient trying to learn about their health. The researchers created twenty standard questions covering the most common concerns a patient might have: what causes the disease, what symptoms to watch for, how doctors diagnose it, what the standard treatments are, and how to manage life after treatment. They then fed these same twenty questions into five different AI models: DeepSeek, Doubao, GPT-5, Tongyi Qianwen, and Wenxin Yiyan. This generated one hundred unique responses, which the researchers then analyzed using tools designed to measure two things: how easy the text was to read, and how well it was structured to help a patient understand and act on the information.

The results revealed that not all artificial intelligence is created equal when it comes to patient education. One model, GPT-5, consistently produced answers that experts rated as the highest quality. These responses were well-organized, covered the necessary information clearly, and were deemed the most suitable for patient education. In contrast, another model, Wenxin Yiyan, produced text that was significantly easier to read, using simpler sentence structures and fewer complex words, but its answers were rated lower on overall quality and usefulness. This finding highlights a crucial disconnect: the system that wrote the most readable text did not necessarily write the most helpful text, and the system that wrote the best information did not always write the easiest text to read.

The researchers also discovered that the topic of the question mattered just as much as the AI model used. Questions about symptoms and signs were generally answered in language that was easier to understand, likely because these topics rely on describing feelings and visible changes. However, questions about standardized treatments, staging, and follow-up care consistently resulted in more complex language, regardless of which AI was used. This suggests that the difficulty of the subject matter itself forces the AI to use more technical terms and longer sentences, making it harder to simplify these specific parts of the patient journey.

Perhaps the most important takeaway from the study is that readability and quality are not the same thing. The researchers found only a weak link between how easy a text was to read and how good the information inside it was. A response could be very simple to read but still miss critical details or fail to guide the patient on what to do next. Conversely, a response could be packed with high-quality, accurate information but be written in a way that is difficult for a non-expert to digest. This means that simply picking the AI that writes the "easiest" sentences is not enough to ensure a patient gets the best care.

The study concludes that while these artificial intelligence tools show promise for helping patients understand their health, they cannot yet be trusted to work alone. The best approach involves using these systems as a starting point, but then having a medical professional review the output. The AI can generate the draft, but a human doctor or nurse must check that the information is accurate, that the advice is safe, and that the language is truly appropriate for the specific patient. In the high-stakes world of small cell lung cancer, where treatment decisions are complex and time-sensitive, the human element remains the essential filter between raw data and life-saving understanding.