Development and validation of a machine learning model for community-based tuberculosis screening among persons aged >= 15 years in South Africa and Zambia

This study developed and validated an XGBoost machine learning model using a large harmonized dataset from South Africa and Zambia that significantly outperformed the standard WHO four-symptom screen in predicting tuberculosis risk among adults, although its performance did not yet fully meet WHO 2025 target product profile goals.

The Gist

The Big Problem: Finding a Needle in a Haystack

Imagine Tuberculosis (TB) as a sneaky thief hiding in a massive city (the community). Currently, the police (health workers) only catch these thieves if the thieves walk into the station and confess (passive case finding). But many thieves are too scared to show up, or they don't even know they are sick yet.

To find them, the World Health Organization (WHO) currently uses a simple checklist called the W4SS. It's like a metal detector at an airport that only beeps if you are coughing, have a fever, lost weight, or have night sweats.

• The Flaw: This metal detector is very "noisy." It misses a huge number of thieves who are carrying weapons but aren't making any noise (asymptomatic). It also beeps at innocent people carrying harmless items (false alarms), wasting a lot of time and money.

The New Solution: A Smart, Predictive Radar

This paper is about building a super-smart radar using Artificial Intelligence (Machine Learning) to find these "silent" thieves before they cause trouble.

Instead of just asking "Do you have a cough?", this new radar looks at a much wider picture. It asks:

• How old are you?
• Have you been treated for TB before?
• Do you have a job?
• Do you live with someone who has HIV?
• How long have you had a specific symptom?

Think of it like a weather forecast. A simple forecast might just say, "It's cloudy." But a super-computer weather model looks at humidity, wind speed, barometric pressure, and satellite data to predict a storm with much higher accuracy. This study built a "TB Weather Model" using data from over 169,000 people in South Africa and Zambia.

How They Built the "Brain"

The researchers took data from four massive surveys and fed it into a powerful computer algorithm called XGBoost.

• The Training: They showed the computer 80% of the data, letting it learn the patterns. It learned that, for example, an older person who was unemployed and had been treated for TB before was at much higher risk than a young, employed person with no history.
• The Test: They then gave the computer the remaining 20% of the data (which it had never seen) to see if it could guess correctly.

The Results: A Clear Winner

When they compared the new AI radar to the old "cough checklist" (W4SS), the results were like comparing a smartphone camera to a pinhole camera.

• The Old Checklist (W4SS): It missed about 6 out of 10 sick people. It was like trying to find a needle in a haystack by only looking for needles that were glowing.
• The New AI Model: It caught significantly more sick people. At a setting where it didn't flag too many healthy people, it found 81.5% of the sick individuals, compared to only 38.2% for the old checklist.

The "Magic" Insight: The AI realized that things like age, unemployment, and past TB history were actually stronger clues than just having a cough. It found the "silent" thieves that the old checklist completely ignored.

Why This Matters: The "Two-Step" Strategy

The authors aren't saying this AI app will replace the final medical test (like a chest X-ray or a lab test). Instead, they propose a two-step security line:

1. Step 1 (The AI App): A community health worker uses a smartphone app to ask the AI questions. If the AI says, "Low Risk," the person goes home. No need for expensive tests. This saves money and time.
2. Step 2 (The Heavy Lifting): If the AI says, "High Risk," then the person gets sent for the expensive, high-tech chest X-ray or lab test.

This is like a concert security guard. Instead of patting down every single person in the crowd (expensive and slow), the guard uses a smart scanner to quickly wave through the 90% of people who are clearly safe, and only stops the 10% who look suspicious for a full search.

The Catch (Limitations)

The model is great, but it's not perfect yet.

• It's not quite there: The WHO wants a tool that catches 90% of sick people. This one catches about 81%. It's a huge improvement, but there's still room to grow.
• Missing Data: Sometimes the surveys didn't have answers for everything (like smoking habits or household size), which made the AI's job a little harder.
• False Alarms: Because TB is rare, the AI still flags some healthy people as "sick." This is better than missing the sick ones, but it means some extra testing is still needed.

The Bottom Line

This paper shows that Machine Learning can be a game-changer for fighting TB in poor communities. By using a smartphone app to ask smart questions, we can find the "silent" TB cases that the old methods miss. It turns a blunt instrument (the cough checklist) into a precision tool, helping health workers focus their limited resources on the people who need help the most.

In short: We are upgrading from a "Guess and Check" game to a "Smart Prediction" game to save lives.

Technical Summary

1. Problem Statement

Tuberculosis (TB) remains a leading cause of infectious death, with an estimated 30% of cases undiagnosed annually. Current screening strategies rely heavily on the WHO Four-Symptom Screen (W4SS) (cough, fever, night sweats, weight loss). However, the W4SS suffers from:

• Low Sensitivity: It misses approximately 50% of active TB cases because many individuals are asymptomatic.
• Low Specificity: It generates many false positives, leading to unnecessary diagnostic testing and resource strain.
• Inadequacy for Active Case Finding (ACF): Passive case finding fails to reach the "missing millions" in community settings.

The World Health Organization (WHO) has established 2025 Target Product Profiles (TPP) for a two-step screening algorithm, requiring an initial tool with ≥90% sensitivity and ≥60% specificity. Existing machine learning (ML) models have been limited by narrow variable sets or specific high-risk populations (e.g., HIV-positive individuals) and have not been integrated into accessible tools.

2. Methodology

Study Design & Data Sources

• Type: Secondary data analysis of four large, community-based TB prevalence surveys.
• Locations: South Africa and Zambia (high-burden settings).
• Dataset: A harmonized dataset of 169,813 individuals aged ≥15 years (excluding those currently on TB treatment).
• Data Split: 80% training set ($N=135,854$) and 20% hold-out test set ($N=33,959$), stratified by study and outcome.

Outcome Definition

• A binary composite reference standard was created to avoid incorporation bias:
  • "Possible TB": Positive on any microbiological test (culture/molecular) OR radiographic abnormality (CXR).
  • "Unlikely TB": Negative on all performed assessments.
• Self-reported symptoms were excluded from the outcome to serve as independent predictors.

Model Development

• Algorithm: XGBoost (eXtreme Gradient Boosting) ensemble model.
• Input Variables: 27 harmonized features including demographics, socio-economic indicators, health behaviors, symptom duration, TB history/contact, and HIV status.
• Handling Missing Data: Multiple Imputation by Chained Equations (MICE) was used to generate 20 imputed datasets. Separate XGBoost models were trained on each and ensembled.
• Class Imbalance: Inverse proportional weighting was applied during training to address the low prevalence of TB (~9.7%).
• Hyperparameter Optimization: Bayesian search with cross-validation to maximize balanced accuracy.

Evaluation Metrics

• Discrimination: Area Under the Curve (AUC).
• Performance at Thresholds: Sensitivity/Specificity at the Youden index, 90% sensitivity (WHO TPP), and 60% specificity (WHO TPP).
• Calibration: Brier score, calibration slope, and intercept.
• Clinical Utility: Decision Curve Analysis (DCA) comparing the model against W4SS, "Test All," and "Test None" strategies.
• Interpretability: SHapley Additive exPlanations (SHAP) values to identify feature importance.

3. Key Results

Model Performance

• Discrimination: The XGBoost model achieved an AUC of 79.7% (95% CI: 78.7–80.7), significantly outperforming the W4SS (AUC 57.0%).
• Sensitivity vs. Specificity Trade-offs:
  • At the 60% specificity threshold (meeting one WHO TPP criterion), the model achieved 81.5% sensitivity (95% CI: 77.6–84.9).
  • In contrast, the W4SS achieved only 38.2% sensitivity at a similar specificity level.
  • To meet the 90% sensitivity target, the model required a threshold yielding only 46.2% specificity.
• Predictive Values: The model demonstrated strong Negative Predictive Value (NPV >95% across thresholds), making it effective for ruling out TB. Positive Predictive Value (PPV) remained low (15–23%), consistent with low-prevalence screening scenarios.
• Calibration: The model showed a systematic overestimation of risk (calibration intercept = -2.08), suggesting a need for recalibration before clinical deployment.

Comparison with W4SS

• The ML model provided nearly double the sensitivity of the W4SS at the 60% specificity threshold.
• Decision Curve Analysis (DCA): The XGBoost model demonstrated a superior net benefit across a wide range of threshold probabilities (0.02–0.58) compared to W4SS, "Test All," and "Test None."

Feature Importance (SHAP Analysis)

• Top predictors included: Age, previous TB treatment, number of times treated for TB, unemployment, and chest pain duration.
• Clinical indicators like night sweats, cough, and HIV status also contributed but had lower global impact compared to socio-demographic and historical factors.

4. Key Contributions

1. Large-Scale Validation: Developed and validated a model on a massive, harmonized dataset ($N \approx 170k$) from two distinct high-burden countries, moving beyond small, facility-based cohorts.
2. Superior Performance: Demonstrated that integrating socio-economic and historical data with clinical symptoms via ML significantly outperforms the standard symptom-only screening (W4SS).
3. Accessibility Focus: The model is designed for integration into a smartphone application (mTBScreen), enabling decentralized, low-cost community screening without immediate need for expensive infrastructure like Chest X-rays (CXR).
4. Strategic Positioning: Proposed a two-step screening workflow where the ML app acts as a high-throughput "rule-out" tool to filter low-risk individuals, reserving resource-intensive tests (like CXR with CAD) for high-risk cohorts.

5. Significance and Limitations

Significance

• Bridging the Diagnostic Gap: The model offers a practical solution to identify the "missing millions" of undiagnosed TB cases, particularly asymptomatic individuals who are missed by current symptom screening.
• Resource Optimization: By improving the efficiency of the first screening step, health programs can optimize the use of limited diagnostic resources (CXR, molecular tests) in community settings.
• Scalability: Integration into a smartphone app allows for deployment in remote areas with limited healthcare infrastructure.

Limitations

• TPP Targets Not Fully Met: While superior to W4SS, the model did not simultaneously achieve the WHO TPP targets of 90% sensitivity and 60% specificity.
• Data Heterogeneity: The composite outcome definition varied across the four source surveys (mixing microbiological and radiographic criteria), potentially introducing noise.
• Missing Data: Significant missingness in key variables (e.g., smoking status, household size) across datasets required imputation, which may affect the robustness of certain feature weights (e.g., TB contact history showed low impact due to missingness).
• Generalizability: The model requires external validation in diverse geographic regions and prospective studies to confirm real-world performance.
• Calibration: The systematic overestimation of risk requires recalibration for specific programmatic contexts.

Conclusion
The study presents a promising, accessible ML-based screening tool that significantly improves upon current standard practices for community TB screening. While further refinement and external validation are needed to meet strict WHO targets, the model represents a critical step toward scalable, active case finding strategies in high-burden settings.