Eliminating False-Negative Discharges in Tuberculosis Screening: Multi-Cohort Development and Independent External Stress-Testing of an Auditable Conformal AI Safety Architecture

Tuberculosis remains one of the world's deadliest infectious diseases, claiming over a million lives annually. In many regions where the disease is most common, the primary tool for finding it is a chest X-ray, a quick image that reveals the health of the lungs. However, the people who are best trained to read these images, thoracic radiologists, are often in short supply. This shortage creates a bottleneck where patients wait weeks for a diagnosis, or worse, are sent home without one, continuing to spread the infection. To solve this, scientists have turned to artificial intelligence, hoping to build computer programs that can scan X-rays and flag suspicious cases for human review. The challenge has been that these computer programs often become overconfident. When shown an unusual image or one taken with different equipment, the computer might declare a patient is healthy with absolute certainty, even when they are sick. This "silent false negative" is dangerous because it leads to the immediate discharge of an infectious person without any medical check.

A researcher has developed a new approach to this problem, creating a safety system that prevents computers from making these dangerous mistakes. Instead of relying on a single algorithm to make a final call, they built a framework that acts like a strict gatekeeper. This system checks the quality of the X-ray image before it is even analyzed, ensuring the picture is clear and not distorted by poor scanning or heavy compression. If the image is too blurry or damaged, the system refuses to make a diagnosis and instead sends the case to a human doctor. Furthermore, the system is designed to recognize when it is unsure. Rather than guessing, it admits uncertainty and passes the patient to a clinician. The researcher tested this system on thousands of X-rays from hospitals in India and Pakistan, including images taken in rural clinics with older equipment and images that had been compressed for web viewing. They found that while the computer alone would have mistakenly cleared some sick patients, the addition of these safety checks eliminated those errors entirely. The result is a tool that can run on simple, offline computers in remote clinics, flagging the most urgent cases for immediate testing while ensuring that no one is sent home without a second look from a human professional.

The core of this work addresses a specific weakness in how artificial intelligence learns to see. Deep learning models, which are the brains behind modern image recognition, are trained on vast libraries of medical images. In a controlled setting, these models can achieve near-perfect accuracy. However, when they encounter real-world conditions—such as an X-ray machine that is slightly different from the ones used during training, or an image file that has been shrunk down to save space—their performance can collapse. The researcher discovered that in one set of tests, a standard computer model failed completely when shown images from a Pakistani hospital that had been heavily compressed for the web. The model became so confused by the digital artifacts that it declared every single patient, healthy or sick, to be infected. This highlighted a critical flaw: a model that is too rigid cannot adapt to the messy reality of global healthcare, where equipment varies and data quality is inconsistent.

To fix this, the researcher did not try to force the computer to learn every possible variation of a bad image, a strategy that often makes the model worse at seeing clear, high-quality images. Instead, they built a series of filters around the computer's decision-making process. The first filter is a pre-check that examines the image for signs of damage, such as the blocky distortions caused by heavy file compression or images that are upside down. If an image fails this check, the system stops and does not attempt a diagnosis. The second layer involves a method that measures the computer's confidence. If the computer is not absolutely certain that a patient is healthy, it is programmed to pause and refer the case to a human. This ensures that the system never makes a final decision to discharge a patient unless it is statistically sure, and even then, only if the image quality is perfect.

The researcher tested this multi-layered system on over 16,000 chest X-rays from eight different locations around the world, including hospitals in China, the United States, Belarus, and India. They first trained the computer on a large, diverse set of images and then tested it on completely new data it had never seen before. In the internal tests, the system performed with extremely high accuracy, correctly identifying nearly all cases of tuberculosis and correctly clearing most healthy patients. However, the true test came when they applied the system to independent groups of patients in India and Pakistan. On the high-quality images from a rural district hospital in India, the system correctly identified about 70% of the sick patients on its own, a significant drop from its internal performance but still useful. More importantly, when they applied the safety rules, the system successfully prevented any sick patients from being mistakenly sent home without review. In the Indian cohort, the safety interlocks ensured that zero active tuberculosis cases were discharged without a clinician seeing them first.

The situation was even more dramatic with the images from Pakistan. These images had been compressed so heavily that a standard computer model failed completely, predicting that every single person in the dataset was sick. The researcher showed that trying to retrain the computer to handle these bad images actually made it worse at reading good, clear images. Instead of forcing the computer to adapt, their safety system simply recognized that the Pakistani images were too degraded to trust. It flagged the compression artifacts and routed those cases to human doctors, effectively acting as a shield against the computer's confusion. This demonstrated that the best way to handle poor-quality data is not to make the computer more robust to noise, but to have a system that knows when to say "I don't know" and ask for help.

The study also revealed that the computer was sometimes learning shortcuts. In some cases, the model was able to guess which hospital an X-ray came from just by looking at the image, and it used that clue to guess the diagnosis. For example, it learned that images from a specific hospital in Belarus were almost always sick, while images from a specific database in the US were almost always healthy. This is a dangerous shortcut because if the computer sees an image from a new hospital, it might get the diagnosis wrong. The researcher used a technique to visualize exactly what the computer was looking at, and they found that while the computer did notice these hospital-specific clues, its main focus was still on the actual lung tissue. When they covered up the lungs in the images, the computer's ability to diagnose the disease vanished, proving that it was still looking at the right things, even if it had picked up some extra habits.

The final result is a system that is designed for the real world, where resources are scarce and conditions are unpredictable. The entire software package is small enough to fit on a standard computer without needing a powerful graphics card or an internet connection. It can run in a clinic with no electricity or internet, making decisions in less than a second. The researcher calculated that using this system in a typical clinic could reduce the number of unnecessary lab tests by nearly 80%, saving money and time, while still catching the vast majority of sick patients. Most importantly, the system is built on the principle that it is better to be safe than sorry. By refusing to make a final call on uncertain or poor-quality images, it ensures that no patient is ever sent home with a false sense of security. This approach transforms artificial intelligence from a tool that tries to replace doctors into a tool that supports them, handling the routine work while protecting the most vulnerable patients from being missed.