Spatial Transferability of Explainable Machine Learning for Predicting Pre-Treatment Tuberculosis Loss to Follow-up Across West Java Districts

Every year, millions of people around the world are diagnosed with tuberculosis, a serious bacterial infection that attacks the lungs. The medical system has a clear path forward: once a person is diagnosed, they must begin taking medication immediately. However, a dangerous gap exists in this process. Many patients, after receiving their diagnosis, simply never start their treatment. They leave the clinic, lose contact with health workers, or decide not to begin the regimen. This is known as "loss to follow-up." These individuals remain infectious, spreading the disease to their families and communities, while their own health deteriorates rapidly. Health programs often struggle to find the money and staff to support every single patient, so they need a way to identify who is most likely to drop out before they even walk out the door. If doctors could spot these high-risk patients early, they could focus their limited resources on the people who need them most, rather than trying to help everyone equally.

In the Indonesian province of West Java, researchers set out to solve this problem using the vast amounts of data already collected by the national health system. They asked a simple but difficult question: can a computer program, trained on the records of patients from the entire province, accurately predict which specific individuals in a specific town will fail to start treatment? They wanted to know if a single, large-scale model could work everywhere, or if the unique circumstances of each district would break the prediction. To answer this, they turned to a field of computer science called machine learning, which allows computers to find patterns in data without being explicitly programmed with rules. They focused on "explainable" models, which are like transparent boxes where you can see exactly why the computer made a decision, rather than "black boxes" that give an answer without showing their work. This transparency is crucial for doctors, who need to understand the reasoning behind a risk prediction before they act on it.

The team gathered information on more than 174,000 people diagnosed with tuberculosis across twenty-seven different districts and cities in West Java in 2024. They looked at details available at the moment of diagnosis, such as the patient's age, whether they had diabetes, where they lived, and what kind of clinic they visited. Most importantly, they noted whether the patient's HIV status was recorded in the system. About nine percent of these patients had already been lost before starting treatment. The researchers built five different computer models to see which one could best separate the patients who would start treatment from those who would not. They tested everything from simple statistical methods to more complex tree-based systems that make decisions by asking a series of yes-or-no questions.

The results showed that the computer models were surprisingly good at their job. The best model could distinguish between patients who would stay and those who would leave with a high degree of accuracy. What was perhaps more surprising was that all five different types of models performed almost the same. The most complex model did not beat the simpler ones by a wide margin. This suggested that the quality of the data itself, rather than the cleverness of the computer algorithm, was the main factor limiting how well the predictions could work. The data contained strong signals, particularly regarding whether a patient's HIV status had been recorded. Patients whose HIV status was missing from their file were far more likely to be lost to follow-up than those whose status was clearly recorded as negative or positive. This pointed to a deeper issue: the act of recording the HIV test was not just a data entry task, but a sign of how complete and careful the entire medical process was for that patient.

However, when the researchers tested whether this single province-wide model worked equally well in every town, the picture changed. A model that performed well on average across the whole region failed to be reliable in specific districts. In some areas, the model consistently underestimated the risk, predicting that patients would start treatment when they actually did not. In other areas, it overestimated the risk. The researchers discovered two main reasons for these failures. First, in some districts, the way data was recorded was different from the rest of the province. For example, in one district, a much higher percentage of patients had missing HIV records compared to the provincial average. Because the model relied heavily on this information, it struggled to make accurate predictions where the data was incomplete.

The second reason was more mysterious. In certain districts, even when the data looked normal, the model still predicted that patients would start treatment, only to find that many of them did not. This suggested that there were local factors influencing the patients' decisions that the computer could not see. These might include the quality of the local health services, the distance to the clinic, or cultural barriers specific to that town. The researchers found that these local failures did not spread to neighboring towns in a predictable pattern; a town with a bad prediction was not necessarily next to another town with a bad prediction. This meant that a single, one-size-fits-all model could not be trusted to work everywhere without adjustment.

The study concluded that while machine learning is a powerful tool for identifying at-risk patients, it cannot simply be applied blindly across different regions. The researchers found that the number of patients in a district did not determine how well the model worked; even areas with very few patients could be predicted well if the local data was consistent, while large areas could fail if the local conditions were unique. They also found that trying to simplify the list of factors the computer looked at did not improve the results. The most important lesson was that the model needed to be recalibrated for each specific location. By adjusting the model's predictions to match the local reality, health workers could use the computer's insights to create a targeted list of patients who truly needed extra support.

To make this practical for busy clinics, the researchers translated their complex computer model into a simple scoring system. Doctors could calculate a risk score for a patient in seconds using just a few facts: whether their HIV status was recorded, how they were diagnosed, their age, and their treatment history. A patient with a missing HIV record, for instance, would immediately receive a high-risk score, signaling that they needed immediate attention. This approach allows health programs to move away from treating every patient the same way and instead focus their limited resources on the individuals most likely to fall through the cracks. The study proved that with the right data and careful local adjustments, technology can help close the gap between diagnosis and treatment, saving lives in the critical first days after a tuberculosis diagnosis.