Integrating Machine Learning Ensembles and Large Language Models for Heart Disease Prediction Using Voting Fusion

This study demonstrates that while traditional machine learning ensembles excel at predicting heart disease from tabular data, integrating them with Large Language Model reasoning through voting fusion yields the highest accuracy (96.62%) and AUC (0.97), establishing a superior hybrid approach for clinical decision support.

The Gist

Imagine you are trying to predict whether a car is about to break down. You have two types of experts you could ask for help:

1. The Veteran Mechanics (Machine Learning Ensembles): These are experts who have studied thousands of specific car parts, oil levels, and engine sounds. They are incredibly good at spotting patterns in numbers and data. If the temperature is 200 degrees and the oil is low, they know exactly what that means.
2. The General Knowledge Professors (Large Language Models): These are experts who have read every book ever written about cars. They understand the story of a breakdown, the history of the brand, and can explain why a car might fail. However, they aren't used to looking at raw spreadsheets of numbers; they prefer reading a story about the car.

The Paper's Big Idea:
The researchers wanted to see if they could combine these two experts to predict heart disease (a "breakdown" of the human body) better than either could alone. They tested this on a dataset of about 1,200 patient records.

Here is the breakdown of their experiment in simple terms:

1. The Veteran Mechanics (Machine Learning)

First, they let the "Veteran Mechanics" (specifically algorithms like Random Forest, XGBoost, and CatBoost) look at the patient data.

• The Result: These models were fantastic. They looked at the numbers (age, blood pressure, cholesterol) and predicted heart disease with 95.78% accuracy.
• The Metaphor: It's like having five different mechanics look at the same engine. If four of them say "the belt is loose," you trust them. By voting together, they became even more reliable.

2. The General Knowledge Professors (LLMs)

Next, they asked the "Professors" (AI chatbots like Gemini, Llama, and others) to look at the same data. They tried two ways of asking:

• Zero-Shot: "Here is a list of numbers. Is this person sick?" (No examples given).
• Few-Shot: "Here are three examples of sick people and three healthy people. Now, look at this new list. Is this person sick?"
• The Result: The Professors were... okay, but not great. They got about 78% accuracy in the best case.
• The Metaphor: Imagine asking a professor who knows everything about cars to guess a breakdown just by looking at a spreadsheet of numbers without any context. They get confused! They are better at reading a story ("The patient feels chest pain and has a history of smoking") than staring at a table of raw numbers.

3. The Grand Experiment: The "Fusion" Team

This is the most exciting part. The researchers didn't just pick one expert; they built a Super-Team.

• They took the predictions from the top 5 Mechanics.
• They took the predictions from the top 5 Professors.
• They created a "Meta-Reasoner" (a smart manager) to listen to both sides.

How it worked:
If the Mechanics said, "High risk!" and the Professors were unsure, the team trusted the Mechanics. But if the data was tricky or "borderline," the Professors used their reasoning skills to help the team make a better call.

• The Result: This hybrid team achieved 96.62% accuracy.
• The Takeaway: While the Professors (LLMs) weren't the best at looking at numbers on their own, they were the perfect "assistant" to the Mechanics. They helped fix the few mistakes the Mechanics made, pushing the accuracy even higher.

Why Does This Matter?

• Heart Disease is Deadly: It's the #1 cause of death globally. We need tools that catch it early.
• The "Best" Tool: The study proves that for looking at medical charts (numbers), Machine Learning is still the king. It's the most reliable way to crunch the data.
• The Future: However, adding an AI "brain" (LLM) on top of the data cruncher makes the system slightly smarter and more trustworthy. It's like having a brilliant doctor (the LLM) double-check the work of a super-fast computer (the ML model).

The Catch (Limitations)

The researchers admit their "test group" was small (only 1,200 people). It's like testing a new car safety feature on a small track before driving it on a highway. They need to test this on millions of patients to be sure it works everywhere. Also, they found that LLMs get confused if you ask them the same question twice in slightly different ways, so they need to be trained to be more consistent.

In a Nutshell

Machine Learning is the best at reading the numbers. Large Language Models are great at understanding the story. When you put them together in a team, they create a super-doctor that is slightly better than either one could be alone, offering a more reliable way to predict heart disease and save lives.

Technical Summary

1. Problem Statement

Cardiovascular disease (CVD) remains the leading cause of global mortality, necessitating early and accurate risk stratification. While traditional Machine Learning (ML) ensembles (e.g., Random Forest, XGBoost) excel at modeling complex, non-linear patterns in structured tabular medical data, they often struggle with model calibration, generalizability, and handling ambiguous or imbalanced clinical cases. Conversely, Large Language Models (LLMs) offer zero-shot and few-shot reasoning capabilities and can interpret unstructured data, but their performance on highly imbalanced numeric datasets is often inconsistent and lower than supervised ML. There is a lack of frameworks that effectively integrate the statistical robustness of ML ensembles with the reasoning capabilities of LLMs for structural heart disease prediction.

2. Methodology

The study proposes a novel Hybrid ML–LLM Fusion Framework utilizing a voting strategy to combine the strengths of both paradigms.

• Dataset:

  • A merged dataset of 1,190 patient records derived from five repositories (Cleveland, Hungarian, Switzerland, Long Beach VA, and Statlog).
  • Features: 11 input features (e.g., Age, Sex, Chest Pain Type, Resting BP, Cholesterol, Max HR) and 1 binary output (Heart Disease: Yes/No).
  • Preprocessing: Categorical encoding, MinMax scaling for continuous variables, and SMOTE (Synthetic Minority Over-sampling Technique) to address class imbalance. The data was split into 60% training, 20% validation, and 20% testing.

• Machine Learning Component:

  • Models Tested: Nine models including Logistic Regression, SVM, Naive Bayes, MLP, and five ensemble methods (CatBoost, Random Forest, XGBoost, LightGBM, Gradient Boosting).
  • Ensemble Strategy: A Voting Classifier was constructed using the top five performing tree-based ensemble models. Both Soft Voting (probability-weighted) and Hard Voting (majority vote) were implemented.

• Large Language Model (LLM) Component:

  • Models Tested: Ten open-source and proprietary LLMs (e.g., Qwen3, LLaMA-4, Gemini 2.5 Flash, Grok) accessed via OpenRouter APIs.
  • Prompting Strategies: Evaluated under Zero-shot (no examples) and Few-shot (with labeled examples) settings.
  • Ensemble Strategy: Similar to the ML component, the top five LLMs were combined using soft and hard voting based on their individual accuracies.

• Hybrid Fusion Pipeline:

  • The core innovation is a meta-reasoning layer (specifically using Gemini 2.5 Flash) that fuses the probabilistic outputs of the ML ensemble and the LLM ensemble.
  • The system weights predictions based on model accuracy to generate a final risk score, aiming to resolve ambiguities where statistical classifiers might be uncertain.

3. Key Contributions

• Novel Fusion Framework: First proposed integration of ML ensembles and LLM reasoning specifically for structural heart disease prediction using a voting fusion mechanism.
• Comprehensive Benchmarking: Extensive evaluation of 9 ML models and 10 LLMs under zero-shot and few-shot conditions, providing a comparative baseline for tabular medical data.
• Demonstration of Synergy: Proves that while LLMs alone underperform on raw tabular data, they provide significant value when used as an auxiliary reasoning layer to refine ML predictions, particularly in uncertain or borderline cases.
• Clinical Decision Support: The hybrid system offers not just a binary prediction but a quantitative risk score and advisory guidance, enhancing clinical transparency.

4. Results

The study compared individual models, pure ensembles, and the hybrid fusion approach:

• Individual ML Models: Tree-based ensembles significantly outperformed others. CatBoost achieved the highest individual accuracy at 92.44%.
• ML Ensemble (Voting): The soft voting ensemble of the top 5 ML models achieved 95.78% accuracy and an AUC of 0.96.
• LLM Performance:
  • Zero-shot: The best LLM ensemble (Soft Voting) achieved 78.9% accuracy (AUC 0.804).
  • Few-shot: Performance dropped to 72.6% accuracy, indicating that few-shot prompting was less effective than zero-shot for this specific tabular task.
• Hybrid ML–LLM Fusion:
  • The proposed fusion framework achieved the highest performance: 96.62% accuracy and an AUC of 0.97.
  • This represents a ~0.84% improvement over the best ML-only ensemble.
  • Error Analysis: The hybrid model showed improved sensitivity in identifying rare disease cases and reducing misclassification on borderline samples compared to ML-only models.

5. Significance and Conclusion

• Validation of Hybrid Approach: The results confirm that while ML ensembles remain the gold standard for structured tabular medical prediction, integrating them with LLMs creates a more robust system. The LLM acts as a "reasoning layer" to handle uncertainty rather than replacing the statistical model.
• Clinical Relevance: The proposed method offers a pathway to more reliable clinical decision-support tools that balance high predictive accuracy with the interpretability and reasoning capabilities of LLMs.
• Limitations & Future Work: The study is limited by the relatively small, harmonized dataset (1,190 records) and a lack of cost-sensitive analysis. Future work will focus on scaling to larger, more diverse datasets and refining the hybrid pipeline for real-world clinical deployment with a focus on safety and equity.

In summary, the paper demonstrates that a Voting Fusion strategy combining high-performance tree-based ensembles with LLM reasoning yields superior predictive capabilities (96.62% accuracy) for heart disease, outperforming both standalone ML and standalone LLM approaches.