Integrating Machine Learning-Based Variable Selection into Heat Vulnerability Index Design

This study demonstrates that integrating machine learning-based variable selection, particularly using Random Forest, significantly improves the accuracy of Heat Vulnerability Index assessments in Chicago by identifying key determinants like poverty, lack of air conditioning, and elderly population proportion, outperforming conventional unsupervised PCA methods in predicting heat-related excess mortality.

The Gist

The Big Picture: Finding the "Hot Spots"

Imagine a city like Chicago is a giant pot of soup. When the weather gets super hot, some parts of the soup boil over (people get sick or die), while other parts stay cool. The goal of this study was to figure out which parts of the city are most likely to boil over and why.

Scientists use a tool called a Heat Vulnerability Index (HVI). Think of this index as a "heat danger map." It assigns a score to every neighborhood to tell city planners, "Hey, this area needs more help (like cooling centers or trees) because it's at high risk."

The Problem: Guessing vs. Knowing

For a long time, scientists made these maps using a "blindfolded" approach. They looked at a list of 10 common risk factors (like poverty, age, or lack of air conditioning) and just mashed them together mathematically to see which ones seemed important. This is like trying to bake a cake by throwing in random ingredients and hoping it tastes good.

The researchers in this paper asked: "What if we don't guess? What if we let the data tell us exactly which ingredients matter?"

They wanted to see if using Machine Learning (smart computer algorithms) could build a better "recipe" for these heat maps than the old, traditional methods.

The Experiment: The Cooking Contest

The researchers set up a contest with five different "chefs" (methods) to see who could create the best Heat Vulnerability Index for Chicago's 77 neighborhoods. They tested their maps against real-life data: actual heat-related deaths from 1993 to 2019.

Here are the five chefs:

1. The Old School Chef (Unsupervised PCA): The traditional method. It looks at all the ingredients and groups them without looking at the final result (deaths). It's like baking a cake without tasting it.
2. The Simple Chef (Linear Regression): Checks if one ingredient (like poverty) goes up when deaths go up. It assumes the relationship is a straight line.
3. The Curvy Chef (Polynomial Regression): Similar to the Simple Chef, but allows for curves (maybe poverty hurts a little at first, then a lot later).
4. The Shrinker Chef (Lasso): A smart algorithm that tries to simplify the recipe by cutting out ingredients that don't seem necessary.
5. The Tree Farmer (Random Forest): A powerful machine learning method that builds thousands of tiny decision trees to find complex patterns. It's like having a team of experts who each look at the problem from a different angle and then vote on the answer.
6. The Booster Chef (XGBoost): Another advanced machine learning method that tries to learn from its mistakes to get better.

The Results: Who Won?

When they compared the maps to the actual death records, the Tree Farmer (Random Forest) won the contest.

• Why? The old methods missed some subtle, complex connections. The Tree Farmer was able to see that certain combinations of factors (like being poor and having no air conditioning) created a danger that was greater than the sum of its parts.
• The Score: The Random Forest map was much better at predicting where the heat deaths actually happened compared to the old "blindfolded" map.

The "Secret Ingredients" (What Matters Most)

Regardless of which chef won, the study found three "secret ingredients" that consistently made a neighborhood dangerous during heatwaves in Chicago:

1. Poverty Rate: If a neighborhood is poor, it's more likely to suffer.
2. No Air Conditioning: If people can't cool their homes, they are in trouble.
3. Age (65+): Older adults are more fragile when it gets hot.

Interestingly, some things people thought were important, like "living alone," didn't show up as a major factor when looking at the whole neighborhood. It turns out that at a community level, having money and AC matters more than whether you live by yourself.

The Takeaway: Stop Using One-Size-Fits-All Maps

The main lesson of this paper is: Don't use the same heat map recipe for every city.

Just because a map worked in New York or Detroit doesn't mean it will work in Chicago. Every city has its own unique "flavor" of risk.

• The Old Way: Use a generic list of rules for everyone.
• The New Way: Use smart computer tools (Machine Learning) to look at your specific city's data, find the specific ingredients that cause trouble there, and build a custom map.

In short: By letting smart computers help pick the right variables, we can draw much more accurate "danger maps." This helps cities save money and, more importantly, save lives by sending help exactly where it's needed most.

Technical Summary

1. Problem Statement

As climate change intensifies extreme heat events, accurate assessment of heat vulnerability is critical for public health interventions. However, traditional Heat Vulnerability Index (HVI) frameworks, such as the widely used Principal Component Analysis (PCA) approach by Reid et al. (2009), rely on unsupervised variable selection. These methods select indicators based on statistical variance without reference to actual health outcomes, potentially failing to capture the specific drivers of heat-related mortality. Furthermore, existing supervised methods often rely on simple linear regression, which may fail to capture complex, non-linear relationships between socioeconomic/demographic factors and health outcomes. There is a need to systematically evaluate how supervised variable selection, particularly using machine learning (ML) algorithms, can improve the predictive performance of HVIs against real-world mortality data.

2. Methodology

The study utilized Chicago, Illinois (77 Community Areas) as a case study, integrating high-resolution environmental, socioeconomic, demographic, and mortality data from 1993 to 2019.

Data Preparation

• Exposure Metric: Daily minimum Heat Index (HI) derived from temperature and humidity. "Heat days" were defined as at least two consecutive days where HI > 70°F (21.1°C).
• Outcome Variable: Heat-related excess mortality (Observed deaths on heat days minus Expected deaths on non-heat days).
• Candidate Indicators: 10 indicators adapted from Reid et al. (2009), including:
  • Demographics: Age >65, Living alone, Race (Black, Hispanic), Education (<High School).
  • Socioeconomic: Poverty rate.
  • Health: Diabetes prevalence.
  • Environment: Lack of green space.
  • Infrastructure: Lack of Air Conditioning (AC).

Comparative Framework

The authors constructed six distinct HVI models, all using PCA for dimensionality reduction but differing in the variable selection step prior to PCA:

1. Unsupervised HVI (Baseline): Uses all 10 pre-selected indicators without outcome-based filtering (Reid et al. method).
2. Supervised HVI (Traditional Statistics):
   • Simple Linear Regression (SLR): Retains indicators with $p < 0.05$.
   • Polynomial Regression (PR): Retains indicators with significant non-linear terms ($p < 0.05$).
3. Supervised HVI (Machine Learning):
   • Lasso Regression: L1 regularization to shrink coefficients to zero.
   • Random Forest (RF): Ensemble tree method using Out-of-Bag (OOB) permutation importance.
   • XGBoost: Gradient boosting using average gain for feature importance.

Evaluation

The performance of the resulting HVIs was validated against community-level heat-related excess mortality using:

• Continuous Metrics: Spearman's rank correlation ($\rho$), Mean Absolute Error (MAE), and Mean Squared Error (MSE).
• Categorical Metrics: Accuracy and F1-score (after binning HVI scores and mortality into 4 vulnerability levels).
• Sensitivity Analysis: Tested robustness against alternative heat definitions (max HI > 110°F), age-standardization of mortality, and lag effects.

3. Key Results

Variable Selection Outcomes

• Robust Indicators: Across almost all methods, Poverty Rate, No AC Access, and Age > 65 were consistently identified as the most critical determinants of heat vulnerability in Chicago.
• Inconsistent Indicators: "Living Alone" and "Hispanic/Latino" population proportion were rarely selected, suggesting they are less robust predictors at the community level in this context compared to structural factors like poverty and cooling access.
• Method Differences:
  • Random Forest (RF) and XGBoost captured non-linear interactions effectively.
  • Lasso excluded "No AC Access" likely due to its correlation with poverty, demonstrating a limitation in handling multicollinearity for specific variables.

Model Performance

• Best Performer: The Random Forest-based HVI achieved the highest performance across all metrics:
  • Spearman Correlation: $\rho = 0.37$ (vs. 0.29 for Unsupervised).
  • Classification: Accuracy = 0.49 and F1-score = 0.51 (vs. 0.32/0.35 for Unsupervised).
  • This represents a ~53% relative increase in accuracy over the baseline unsupervised model.
• Spatial Patterns: All models identified higher vulnerability in Chicago's southern and western communities, but the RF model provided the most accurate spatial alignment with observed mortality clusters.
• Sensitivity: The identification of Poverty, AC access, and Age > 65 remained robust across different heat definitions and age-standardization scenarios.

4. Key Contributions

1. Methodological Advancement: Demonstrates that integrating supervised machine learning variable selection into the traditional PCA-based HVI framework significantly outperforms both unsupervised methods and traditional linear regression approaches.
2. Algorithmic Insight: Identifies Random Forest as the superior selection algorithm for this specific application, likely due to its ability to model non-linear relationships and complex interactions between socioeconomic and environmental factors without overfitting on small sample sizes ($n=77$).
3. Contextual Validation: Confirms that while some indicators (e.g., poverty, AC) are universally important, the specific combination of drivers is highly context-dependent. The study validates that "one-size-fits-all" indicator sets are suboptimal; local outcome-informed selection is necessary.
4. Policy Relevance: Provides a data-driven framework for cities to prioritize resources (cooling centers, housing improvements) based on indicators that statistically correlate with actual mortality, rather than just theoretical vulnerability.

5. Significance and Limitations

• Significance: The study bridges the gap between public health epidemiology and machine learning, offering a reproducible workflow to refine vulnerability assessments. It highlights that Poverty Rate, Lack of AC, and Elderly Population are the primary structural drivers of heat mortality in Chicago.
• Limitations:
  • Sample Size: The analysis is limited to 77 community areas, which constrains the complexity of models that can be trained (e.g., XGBoost underperformed, potentially due to noise sensitivity in small datasets).
  • Indicator Scope: The study was restricted to indicators from the Reid et al. (2009) framework to test the methodology rather than to build a perfect predictive model. Consequently, the correlation coefficients ($\rho \approx 0.37$) suggest room for improvement if more diverse, locally specific indicators (e.g., tree canopy density, building age, social isolation metrics) were included.
  • Generalizability: The "best" method (RF) may vary in different cities with different population structures or data availability.

Conclusion: The paper concludes that while unsupervised HVIs provide a useful starting point, supervised, outcome-informed variable selection—specifically using ensemble machine learning methods like Random Forest—significantly enhances the ability of HVIs to capture heat-related health risks, thereby supporting more equitable and effective climate adaptation strategies.