Evaluating the Incremental Value of AI-Derived ECG Features Beyond Automated ECG Reports and Clinical Data for 30-Day Mortality: A MIMIC-IV Cohort Study

Every year, millions of people walk into hospitals and have their hearts checked with a simple, painless test called an electrocardiogram, or ECG. This machine records the heart's electrical activity as a squiggly line on a screen, capturing a snapshot of how the heart is beating at that exact moment. For decades, doctors have relied on these lines to spot trouble, looking for specific shapes that signal a heart attack or other serious conditions. Today, computers can read these lines automatically, measuring the height and width of the waves and writing a short report that summarizes what it sees. But as artificial intelligence has grown more powerful, researchers have begun to ask a difficult question: do these new, complex computer models see something that the standard reports and the doctors' own notes are missing? Specifically, can a deep-learning system, trained to recognize subtle patterns in the raw electrical waves, predict who might die within a month better than the tools we already use every day?

A team of researchers set out to answer this by looking at a massive collection of medical records from a hospital in Boston. They focused on over 25,000 adults who had received an ECG while in the hospital. For each person, the researchers gathered three different types of information to see which one was the best predictor of whether the patient would pass away within thirty days. The first type was the raw data itself, fed into a sophisticated artificial intelligence model that had been trained on millions of heartbeats to understand the shape of the waves. The second type was a set of fifteen specific measurements that a researcher could calculate by hand from the same waves, such as the duration of certain bumps and dips in the line. The third type was the actual automated report that a hospital computer would generate for a doctor, along with basic details about the patient like their age, sex, and any known medical history they had before arriving at the hospital.

The researchers compared how well each of these approaches ranked the patients by risk. They found that the artificial intelligence model was indeed better at spotting danger than the simple, hand-calculated measurements. When they looked only at the heart waves, the AI model correctly identified the patients who would die about 7.5 percent more often than the basic measurements did. This gap remained even when they added the patients' age, sex, and medical history to the mix, though the advantage shrank slightly. In this specific comparison, the AI was clearly seeing something the simple measurements were not.

However, the story changed when the researchers compared the AI against the full, automated hospital report. This report is what a doctor actually sees on their computer screen; it includes the basic measurements plus a long list of diagnostic statements and summaries generated by the hospital's own software. When the researchers pitted the advanced AI against this richer, more complete report, the advantage almost disappeared. The AI model did not perform significantly better than the standard automated report, and in some tests, the difference was so small it could have been due to chance. This suggests that the standard hospital report already captures most of the useful information hidden in the heart waves, leaving very little for the new AI to add on top of it.

Perhaps the most surprising discovery was about who gets flagged for review. In a real hospital, doctors cannot look at every single patient's chart; they have to choose which ones to examine more closely. The researchers simulated a scenario where they could only review the top 5 percent of patients ranked as highest risk. They found that the AI model and the standard report picked out almost the same number of patients who would die, but they picked out completely different people. The AI flagged 136 patients who the standard report missed, but it also missed 95 patients that the standard report had caught. This means that even if two tools are equally good at predicting death on average, they might send doctors to look at different groups of people. One tool might save a life by catching a patient the other missed, while the other tool might miss a different patient that the first one caught.

The study also looked at how much of this success came from the AI having been trained on a huge amount of heart data beforehand. They tested whether a brand-new, untrained computer network could do just as well if it was given the same amount of time to learn from the specific patients in this study. They found that when there was plenty of data to learn from, the pre-trained AI and the new, untrained network performed almost identically. The advantage of having been trained on millions of heartbeats beforehand only became clear when the researchers tried to learn from very small amounts of data. This implies that the power of these advanced models might depend heavily on how much data is available to teach them, rather than just the complexity of the model itself.

Finally, the researchers tested whether these findings held up in a completely different setting. They took the models trained on the American hospital data and applied them to a group of patients in Brazil who had never been seen by the American system before. Even in this new population, with a different healthcare system and a longer time horizon of one year instead of thirty days, the AI model still held a small advantage over the basic measurements. However, the gap narrowed significantly when compared to the full automated reports, reinforcing the idea that the standard tools already do a very good job.

The researchers conclude that while artificial intelligence can extract valuable information from heart waves that simple measurements miss, its ability to improve upon the full, automated hospital reports is limited and uncertain. The real value of these new tools may not be in simply predicting who will die, but in how they change which patients doctors choose to look at. Because the AI and the standard reports prioritize different patients, using the AI could mean that some high-risk individuals get attention who would otherwise be overlooked, while others who would have been caught are missed. Before these tools can be used to change how hospitals work, more research is needed to understand exactly which patients benefit from this shift and whether the cost of running these complex systems is worth the difference in who gets saved.