Validation asymmetry in generative AI–generated clinical educational content: a scoping review

Imagine a classroom where the teacher is an artificial intelligence capable of writing medical case studies, drawing anatomical diagrams, and simulating conversations with patients in seconds. This technology, known as generative artificial intelligence, is rapidly entering medical schools and training programs for nurses, doctors, and other health professionals. It promises to create endless learning materials that are fluent, plausible, and tailored to specific students. But as these tools become more common, a critical question has emerged: just because the AI can produce a convincing story or a realistic image, does that mean the content is actually safe and fair to use for teaching? The core issue is not whether the machine can speak or draw, but whether the information it generates is accurate, whether it treats all types of people fairly, and whether the methods used to create it are clear enough to be trusted.

A recent comprehensive review of research published between 2020 and 2026 set out to answer exactly this question. The researchers examined 161 studies that looked at how generative AI is used to create educational content for health professions. They were not interested in whether students liked the AI or how fast it worked. Instead, they focused on three specific areas of validation: whether the medical facts were correct, whether the people and groups depicted in the content were represented fairly without harmful stereotypes, and whether the strategies used to fix errors actually worked. The team found a striking imbalance in how these different aspects were checked. While almost every study checked if the medical facts were right, very few checked if the content was fair to different groups of people, and even fewer tested whether the fixes they proposed actually solved the problems.

The researchers discovered that out of the 161 studies they reviewed, 156 of them, or nearly 97 percent, focused on checking representational fidelity. In plain terms, this means they were verifying if the AI's output was medically accurate, if it contained real facts, and if it avoided making up information or "hallucinating" details. This is a vital check, as a medical student learning from a fake diagnosis could be dangerous. However, when the researchers looked for studies that checked for demographic bias—such as whether the AI consistently portrayed certain races, genders, or ages in stereotypical ways or left them out entirely—they found only 21 studies. That is just 13 percent of the total. Among those few studies that did look for bias, 18 of them found that the AI was indeed reproducing stereotypes or creating unbalanced representations. For example, some generated images showed nurses or doctors as belonging to only one gender or race, and some clinical cases linked specific diseases to particular ethnic groups in ways that did not match reality.

This gap in attention is what the authors call a validation asymmetry. It suggests that the field is currently very good at asking, "Is this factually correct?" but is not yet asking, "Is this fair to everyone?" The review also highlighted that the content being tested was mostly text, with a smaller number of studies looking at images or conversations. Surprisingly, none of the studies that tested conversational AI, such as virtual patients that talk to students, explicitly checked for these demographic biases. This does not mean the bias is absent in those conversations; it simply means no one has looked for it yet. The researchers also noted that while many studies proposed ways to fix these errors—such as changing the instructions given to the AI or having human experts review the output—only a small fraction actually tested to see if those fixes worked. In the few cases where they did test the fixes, the results were mixed, with many strategies only partially solving the problem rather than eliminating it.

Another significant finding concerned transparency. For a study to be useful, other scientists need to know exactly how the AI was used so they can repeat the experiment. Yet, in 122 of the 161 studies, the authors did not report the specific settings or parameters used to generate the content. Without knowing these details, it is impossible to know if the results would be the same if the study were repeated today, especially since these AI systems change and update frequently. The review also found that most studies did not report who funded the research or if the researchers had any conflicts of interest, which are standard details needed to trust scientific findings.

The conclusion of this review is not that generative AI is broken, but that the way we check its work is incomplete. The authors argue that for AI-generated content to be used safely in education, it must be validated as a multidimensional resource. This means that before a medical school uses an AI-written case study, they need to verify not just that the diagnosis is correct, but also that the patient described in the story represents a realistic diversity of people, and that any steps taken to ensure fairness were actually proven to work. The current focus on factual accuracy alone is insufficient. As these tools become a standard part of medical training, the standards for their validation must expand to include fairness and transparency, ensuring that the future of healthcare education is built on content that is not only smart, but also just and reliable.