Multicenter Evaluation of the Accuracy and Clinical Reliability of an Artificial Intelligence- Based System for Skeletal Maturity Indicators Staging

Growing children are not just getting taller; their bodies are undergoing a complex internal transformation where bones harden, joints fuse, and growth spurts accelerate and then slow down. For orthodontists, the timing of treatment is everything. Moving teeth or guiding jaw growth works best when the body is in a specific phase of this biological development, a window of opportunity that has little to do with a child's actual age in years and everything to do with their skeletal age. To find this window, doctors have long relied on hand-wrist X-rays, looking at the tiny bones in the hand and wrist to see how far along a child is in their physical maturation. This process, however, requires a human expert to carefully examine the image, identify specific bone shapes, and assign a stage to the patient's development. It is a skill that takes years to master, and like any human task, it can vary from one doctor to another.

As artificial intelligence has begun to assist in many areas of medicine, researchers have asked whether a computer program could learn to read these X-rays with the same care and consistency as a human specialist. A new study published by a team from Yonsei University in South Korea set out to test a commercial software system designed to do exactly this. The researchers wanted to know if this artificial intelligence could reliably determine a child's skeletal maturity stage across different hospitals and for different types of patients, or if it would only work well in a controlled, single-location setting. They gathered a large group of patients from two separate dental hospitals, covering a wide range of ages and growth patterns, to see if the computer could match the judgment of experienced orthodontic specialists.

The study involved reviewing the medical records and hand-wrist X-rays of 595 patients, ranging from six to eighteen years old, who had visited one of two university-affiliated hospitals in South Korea. At these hospitals, the patients had undergone standard orthodontic evaluations, which included taking X-rays of their heads and their hands. The researchers first established a "gold standard" for each patient by having experienced orthodontic specialists review the X-rays and cross-check them with the patients' medical charts. In cases where the specialists disagreed, a third expert was brought in to make the final call. This human-derived stage served as the reference point against which the artificial intelligence was tested. The computer software, which had never seen these specific X-rays before, then analyzed the images on its own to predict the same maturity stage.

The results showed that the artificial intelligence system performed with a high degree of consistency and reliability. When comparing the computer's predictions to the human experts' final decisions, the two matched exactly in 73 percent of all cases. If the researchers allowed for a margin of error of just one stage up or down, the agreement rose to 96 percent. This means that in almost every instance, the computer was either spot-on or off by a very small, clinically insignificant amount. The system proved to be equally effective at both hospitals, despite the fact that the two locations used different X-ray machines and had different patient populations. Statistical analysis confirmed that the hospital location itself did not influence the accuracy of the results, suggesting that the software is robust enough to work in varied clinical environments without needing to be retrained for each new setting.

The study also revealed that the system's accuracy was not uniform across all groups of people. The computer was significantly more accurate when assessing female patients than male patients. Specifically, it matched the human experts exactly 76 percent of the time for girls, compared to 69 percent for boys. Further investigation showed that this difference was driven by the stage of puberty. The system was most accurate for girls who had already started their menstrual cycles, matching the experts 84 percent of the time. For those who had not yet reached that milestone, the accuracy dropped to match that of the boys. This suggests that the skeletal changes occurring after the onset of menstruation are perhaps more distinct or easier for the algorithm to recognize than the changes occurring before it. Interestingly, the type of jaw relationship a patient had—whether their upper and lower jaws were aligned normally, protruding, or receding—did not affect the computer's ability to read the X-rays. The system performed just as well regardless of these skeletal patterns.

When looking at the specific stages of development, the computer excelled at identifying the very beginning and the very end of the growth process. It was nearly perfect at recognizing the earliest stage of development and the final stage where growth is complete. These are critical moments for treatment planning, as knowing when growth has finished is essential for deciding whether to proceed with surgery or continue with braces. The system was less precise in the middle stages, particularly around the peak of the growth spurt, where the exact match rate was lower. However, even in these difficult middle stages, the computer rarely made a large mistake. It almost always guessed a stage that was right next to the correct one, rather than jumping far ahead or falling far behind. This pattern of small, adjacent errors is far less dangerous in a clinical setting than a large, random error, as it keeps the treatment plan within a safe and logical range.

The researchers concluded that this artificial intelligence system is a highly reliable tool that can support orthodontists in their daily practice. By providing a consistent and rapid assessment of skeletal maturity, the software can help doctors determine the optimal timing for treatment with greater confidence. The study demonstrated that the system works well across different hospitals and for a diverse group of patients, with the notable exception that it currently performs slightly better for girls who have passed the onset of menstruation. While the computer is not perfect, its ability to avoid major errors and its high rate of agreement with human experts suggest it is ready to be used as a valuable aid in real-world clinical settings, helping to ensure that orthodontic treatment is delivered at the most effective moment in a child's growth.