Evaluating AI Chatbots as Patient Education Tools for Dental Trauma: A Cross-Sectional Comparison of Information Quality and Readability
This cross-sectional study evaluating five AI chatbots for dental trauma education found that while they generated generally well-organized information, significant limitations in source transparency and readability prevent them from replacing professional clinical assessment, positioning them instead as adjuncts requiring human review.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
When a tooth is knocked out, fractured, or loosened, the hours that follow are critical. The outcome often depends on how quickly a person seeks professional help and whether they follow the right steps to preserve the injured tooth before reaching a dentist. In moments of panic, pain, or confusion, many people turn to the internet for immediate answers. They ask questions like, "How should I store a knocked-out tooth?" or "How soon must I get to a hospital?" For decades, the quality of answers found on the web has been a mixed bag, ranging from expert medical advice to dangerous misinformation. Today, a new type of digital helper has emerged: the artificial intelligence chatbot. These are computer programs trained on vast amounts of text that can hold conversations and generate detailed responses to almost any question. While they offer the promise of instant, structured information, a crucial question remains unanswered: Can these machines be trusted to guide someone through a dental emergency, or do they simply sound confident while missing vital details?
A team of researchers from hospitals in Suzhou, China, set out to test the reliability of these digital assistants specifically for dental trauma. They did not simply ask the chatbots random questions; instead, they built a rigorous test based on real-world needs. The team gathered twelve specific questions that patients and caregivers actually ask, drawing from official international dental guidelines, the daily experiences of practicing dentists, and common concerns found on Chinese internet forums. These questions covered everything from how to handle a broken tooth fragment to understanding the long-term effects of an injury on a child's permanent teeth. They then fed these exact same twelve questions to five of the most popular artificial intelligence chatbots available at the time: GPT, Claude, Microsoft Copilot, DeepSeek, and Kimi. This created a total of sixty different responses to evaluate.
To judge the answers, the researchers acted as impartial examiners, using four distinct tools designed to measure different aspects of health information. They looked at how well the answers explained treatment choices and discussed risks, how useful and clear the educational content was for a patient, and whether the information was organized in a helpful way. Perhaps most importantly, they checked for transparency: did the chatbot tell the user where its information came from, who wrote it, and when it was last updated? They also measured how difficult the text was to read, aiming for a level that a person with a sixth-grade education could easily understand. This is a standard recommendation for health materials, ensuring that even those under stress or with limited reading skills can grasp the instructions.
The results revealed a complex picture of both capability and significant limitation. The chatbots were not all the same; some performed better than others depending on what was being measured. One model, Microsoft Copilot, provided the most reliable information regarding treatment choices and risk discussions, while another, DeepSeek, produced the most patient-friendly educational content. However, no single chatbot excelled at everything. A more troubling finding emerged when the researchers checked for sources and dates. Every single chatbot, without exception, failed to provide any citation, author name, or date of publication. They offered confident-sounding advice but gave no way for a user to verify if that advice was current or where it originated. In the world of medical information, this lack of a "receipt" is a major red flag, as guidelines for treating dental injuries change as new science emerges.
The readability of the answers presented another hurdle. Despite the goal of making information simple, the text generated by all five models was too complex for the average person to read easily. The sentences were long, the vocabulary was technical, and the overall reading level was far above the recommended sixth-grade standard. Instead of simple, direct instructions like "put the tooth in milk," the chatbots often produced paragraphs filled with medical jargon and complicated sentence structures. This suggests that while the machines can assemble information, they struggle to translate it into the plain language that a frightened parent or an injured patient needs in an emergency.
The study concludes that these artificial intelligence tools are currently not ready to replace a dentist or a doctor in a crisis. They can generate well-organized information that has some educational value, but their inability to cite sources and their tendency to use difficult language make them unsafe to use on their own. The researchers suggest that these chatbots should only be used as a supplement to professional care, where a human expert can review the information, correct any errors, and explain the details in a way the patient can understand. Until these systems can reliably tell users where their information comes from and speak in clear, simple terms, the best course of action for anyone with a dental injury remains the same: seek immediate professional assessment.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.