Screening for Probable Undiagnosed Hypertension in US Adults Using Interpretable Machine Learning: An NHANES 2017-2018 Study

Technical Summary: Screening for Probable Undiagnosed Hypertension Using Interpretable Machine Learning

Problem Statement
Hypertension remains a critical global health challenge, affecting approximately 1.28 billion adults worldwide, yet fewer than half are aware of their condition. Undiagnosed hypertension leads to preventable cardiovascular morbidity and mortality through silent end-organ damage. While machine learning (ML) has been applied to hypertension risk prediction, most existing models rely on laboratory variables (e.g., lipid panels, renal function) unavailable in resource-constrained or community settings. Furthermore, prior studies often target general risk prediction rather than the specific, clinically urgent task of identifying adults who are already hypertensive but unaware of their diagnosis. This study addresses the gap of developing a screening tool for probable undiagnosed hypertension using only non-invasive, routinely collectable data.

Methodology

• Data Source: The study utilized a cross-sectional design with data from the National Health and Nutrition Examination Survey (NHANES) 2017–2018 cycle.
• Study Population: From an initial 9,254 participants, 5,237 adults (aged ≥18) were included after excluding those with missing blood pressure data, invalid hypertension awareness responses, or age <18.
  • Outcome Definition: "Probable undiagnosed hypertension" was defined as a mean blood pressure ≥130/80 mmHg (per ACC/AHA 2017 guidelines) combined with a self-report of never having been diagnosed by a healthcare professional.
  • Controls: Participants with a prior self-reported diagnosis (n=1,930) were retained as negative controls to distinguish between "undiagnosed" and "normotensive" states.
  • Class Imbalance: The final sample consisted of 1,072 positive cases (20.5%) and 4,165 negative controls.
• Predictor Variables: Eight non-invasive variables were selected to ensure deployability without laboratory access:
  1. Age
  2. Sex
  3. Body Mass Index (BMI)
  4. Waist Circumference
  5. Smoking Status (ever smoked ≥100 cigarettes)
  6. Diabetes Status (doctor-confirmed)
  7. Vigorous Physical Activity (past week)
  8. Weekday Sleep Duration
• Machine Learning Approach:
  • Three classifiers were trained and compared: Logistic Regression (LR), Random Forest (RF), and Extreme Gradient Boosting (XGBoost).
  • Preprocessing: Missing data were imputed (median for continuous, mode for categorical). Categorical variables were binary-encoded. Class imbalance was addressed using class-weighted training (class_weight="balanced" for LR/RF; scale_pos_weight for XGBoost).
  • Validation: The dataset was split into 80% training and 20% test sets using stratified sampling. Stratified 5-fold cross-validation was applied to the training set. Models were evaluated on the held-out test set.
  • Hyperparameters: Default configurations were used to prevent overfitting and isolate algorithmic characteristics, with adjustments only for class imbalance.
• Evaluation Metrics: Primary metric was AUC-ROC. Secondary metrics included sensitivity, specificity, and F1 score, all reported with 95% bootstrap confidence intervals (1,000 iterations). Permutation feature importance was used to interpret the best-performing model.

Key Results

• Model Performance: All three models demonstrated modest but consistent discrimination, with test AUCs ranging from 0.599 to 0.611.
  • Logistic Regression (LR): Achieved the highest test AUC of 0.611 (95% CI: 0.571–0.652). It offered the most balanced clinical profile with a sensitivity of 0.535 and specificity of 0.594.
  • Random Forest (RF): Achieved an AUC of 0.607 but exhibited near-absent sensitivity (0.047) despite high specificity (0.971), rendering it unsuitable for screening at the default threshold.
  • XGBoost: Showed the highest cross-validation AUC (0.621) but the lowest test AUC (0.599), suggesting mild overfitting. It had intermediate sensitivity (0.279) and high specificity (0.804).
• Feature Importance: Permutation analysis on the LR model identified Diabetes Status, Sex, and Age as the most influential predictors.
  • Note on Diabetes: Contrary to biological intuition, diabetes status acted as an inverse predictor of undiagnosed hypertension. The data showed that individuals with diabetes had a lower rate of undiagnosed hypertension (10.1%) compared to non-diabetics (16.4%), likely due to higher healthcare utilization leading to earlier diagnosis.
• Demographics: Participants with probable undiagnosed hypertension were significantly older, more likely to be male, and had larger waist circumferences compared to the negative class.

Significance and Claims
The paper claims that interpretable ML classifiers trained on eight non-invasive variables can achieve modest but consistent discrimination for identifying probable undiagnosed hypertension.

• Feasibility: The study supports the feasibility of laboratory-free community screening, particularly in resource-limited settings (e.g., South Asia) where diagnostic infrastructure is limited.
• Clinical Utility: The authors position the Logistic Regression model not as a diagnostic tool, but as a first-pass screening instrument. With an estimated Negative Predictive Value (NPV) of ~83%, the model is effective at ruling out probable undiagnosed hypertension in negative screens, thereby identifying individuals who warrant confirmatory clinical blood pressure measurement.
• Transparency: By restricting variables to non-invasive measures and using default hyperparameters, the study prioritizes clinical applicability and reproducibility over maximizing accuracy through complex feature engineering or laboratory data.

Limitations and Future Directions
The authors acknowledge several limitations: the cross-sectional design precludes causal inference; the lack of external validation limits generalizability; and the use of self-reported data introduces potential bias. Crucially, the study did not apply NHANES sampling weights to the ML pipeline, meaning prevalence estimates are specific to the cohort rather than the weighted US population. The authors state that external validation in diverse populations and prospective evaluation in community health settings are required before clinical implementation.