When high AUROC does not travel: internal-to-external performance differences in machine-learning models for antimicrobial resistance

In the battle against superbugs, doctors and scientists are increasingly turning to computers to predict which bacteria will resist which antibiotics. These computer programs, built using machine learning, act like highly trained pattern-recognition experts. They scan vast amounts of medical data—such as patient records, lab test results, and genetic sequences—to guess whether a specific infection will survive a standard drug treatment. When these programs are tested in the hospital where they were created, they often look like miracles of modern medicine, correctly identifying resistant bacteria with high accuracy. This success gives researchers and clinicians hope that these tools can soon be deployed globally to guide life-saving treatment decisions.

However, a computer program that learns from one hospital's data does not automatically know how to read the data from another hospital. Just as a person who learns to drive on the quiet streets of a small town might struggle on the chaotic highways of a major city, these models face a hidden challenge when they leave their home environment. The data they encounter in a new country, a different time period, or even a neighboring city can look subtly different due to variations in local bacteria strains, lab equipment, or patient populations. The critical question for the future of medical AI is not just how well these models perform at home, but how much their accuracy drops when they are sent out into the real world to work in new places.

A new study set out to measure exactly this drop in performance. The researcher, Dripto Roy, gathered a collection of published machine-learning studies that had done the hard work of testing their models in two places: first in the hospital where the model was built, and then again in a completely independent hospital or dataset. The goal was simple but vital: to see the difference between the model's confidence at home and its actual performance abroad. By comparing the scores from these paired tests, the study aimed to determine if the high success rates reported in scientific papers are a reliable promise of future success, or if they are often an illusion that fades when the model travels.

The investigation focused on a specific group of studies that reported both an internal score and an external score for the same model and the same prediction task. In total, the researcher analyzed forty-seven distinct comparisons drawn from eight independent research papers. These comparisons covered a wide range of scenarios, including predictions for dangerous bacteria like MRSA and drug-resistant pneumonia, using data from electronic health records, whole-genome sequencing, and clinical lab reports. The researcher calculated the difference between the internal score and the external score for every single comparison to see how much the performance changed.

The results revealed a clear, though nuanced, pattern. In the vast majority of individual comparisons—thirty-six out of forty-seven—the model performed worse when tested on new, external data than it did on the data it was trained on. On average, the drop in accuracy was noticeable, with the external score falling below the internal score by a measurable margin. This confirmed that the "optimism" of seeing a high score in a development setting is a real phenomenon; models do tend to lose some of their sharpness when they leave their home environment.

However, the story becomes more complex when looking at the bigger picture. The researcher realized that simply counting every single comparison as an equal piece of evidence was misleading. Some research papers reported dozens of different model variations or antibiotic targets, all derived from the exact same group of patients and the same data processing steps. If these many results from a single paper were all counted equally, they would overwhelm the results from other papers that only reported one or two comparisons. When the researcher adjusted the analysis to treat each entire research paper as a single unit of evidence, giving every paper an equal voice regardless of how many numbers it contained, the picture changed significantly.

Under this more balanced view, the average drop in performance shrank considerably. While the models still generally performed worse outside their home environment, the gap was much smaller than the initial count suggested. In fact, when looking at the eight studies as a whole, the average difference was so small that it could easily be zero. This means that while some models suffer a significant loss of accuracy when transported, others remain surprisingly robust, and the overall field does not suffer from a single, universal penalty. The size of the drop depends entirely on the specific study, the data used, and how the results are counted.

The study also highlighted what is missing from the current landscape of medical AI. While most of the papers reviewed did attempt to test their models in new settings, very few went further to check if the models were well-calibrated. Calibration is a crucial concept that goes beyond simple accuracy; it asks whether the probability numbers the model outputs actually match the real-world risk. For instance, if a model predicts a patient has a 20% chance of resistance, does that patient actually have resistance about one in five times? The review found that this vital check was rarely reported. Furthermore, very few studies tested their models over time to see if they remained accurate as bacteria evolved, or tested them in a forward-looking, prospective manner where the model predicts outcomes for patients as they arrive in the hospital.

The findings serve as a cautionary tale for how we interpret the success of medical AI. The study argues that we cannot simply take a high accuracy score from a paper and assume it will hold true everywhere. The apparent size of the performance gap is highly sensitive to how we count the evidence. If we count every single model variation in a paper as a separate fact, we exaggerate the problem. If we treat each paper as a single story, the problem looks smaller but still real. The research concludes that there is no simple mathematical fix or universal correction factor that can be applied to all published scores to predict their real-world performance.

Instead, the path forward requires greater transparency and rigor. Researchers need to be explicit about how they split their data to ensure that no information leaks from the testing phase back into the training phase. They must report not just how well a model ranks patients, but how well its probability estimates match reality. Most importantly, they need to validate their models in truly independent environments, reporting the specific numbers of patients and the prevalence of resistance in both the home and the new setting. Until these standards become the norm, the high scores reported in the literature will remain a promise that is not yet fully kept, and the journey from a successful computer model to a reliable tool for doctors will remain a work in progress.