Avoiding Data Leakage in Machine Learning Models for Diabetes and Hypertension Classification Using Routine Occupational Health Screening Data
This study demonstrates that including outcome-defining predictors like fasting blood glucose and blood pressure in machine learning models for diabetes and hypertension classification causes severe data leakage and artificially inflated performance, whereas models excluding these variables using routine occupational screening data achieve moderate but clinically realistic accuracy while highlighting the predictive value of factors such as age, BMI, and psychological distress.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the world of modern medicine, computers are increasingly being asked to look at a patient's data and predict whether they will develop a serious illness like diabetes or high blood pressure. This field, known as machine learning, works by feeding a computer thousands of examples of people who are sick and people who are healthy, along with a long list of measurements like age, weight, and blood test results. The computer then tries to find hidden patterns that connect these measurements to the sickness. If the computer learns well, doctors hope it can spot at-risk individuals early, perhaps during a routine check-up at work, allowing for prevention before a disease takes hold. However, there is a dangerous trap in this process called data leakage. It happens when the computer is accidentally given the answer key while it is still trying to learn. If the computer sees a blood sugar level that is already high and is then asked to predict if the person has diabetes, it is not really predicting anything; it is simply reading the definition of the disease. This creates a false sense of confidence, making the computer seem like a genius when it is actually just using invalid data.
A team of researchers in Iran set out to build a system that could identify diabetes and hypertension among university employees using the data collected during their annual health screenings. They wanted to see if these routine records, which include things like body measurements, blood tests, and even questions about stress and mood, could help spot people at risk. But their most important goal was not just to build a model, but to prove how easily researchers can fool themselves into thinking their models are perfect. They took a large group of employee records and split them into two groups to test their ideas. First, they built a model that included every single piece of information available, including the very blood sugar and blood pressure numbers that doctors use to diagnose the diseases. Unsurprisingly, this model performed almost perfectly, correctly identifying nearly every case. It seemed like a breakthrough.
Then, the researchers did something critical. They built a second model that was strictly forbidden from looking at those specific blood sugar and blood pressure numbers. They forced the computer to rely only on the other clues, such as age, body size, and psychological stress levels, to make its guess. This second model represented what would happen in the real world, where a doctor might want to predict risk before running the final diagnostic test. When they compared the two, the difference was stark. The first model, which had the answer key, claimed to be right about 96 percent of the time. The second model, which had to do the hard work of actual prediction, dropped to being right about 78 percent of the time. This gap of nearly twenty percentage points was not a small error; it was a massive inflation caused entirely by the computer peeking at the answer. The study demonstrated that many previous claims of highly accurate medical prediction might be built on this same shaky foundation, where the computer was simply reading the definition of the disease rather than learning to spot it.
Beyond exposing this methodological flaw, the researchers found that the model without the answer key still held value. Even without the direct blood pressure or sugar readings, the computer could still distinguish between healthy and at-risk employees with a moderate degree of accuracy. More surprisingly, the computer learned that psychological factors were powerful clues. Variables measuring general psychological distress, depression, and anxiety ranked among the top predictors for both conditions, appearing just as important as physical measures like waist circumference or body mass index. This suggests that the mind and body are deeply linked in these diseases, and that a worker's mental state could be a vital early warning sign for physical health problems.
The researchers were careful to note that their findings are a starting point, not a finished solution. The study was conducted on a relatively small group of employees from a single university, and the models have not yet been tested on different groups of people in other places. The performance of the realistic model, while useful for sorting people into risk groups, is not high enough to replace a doctor's diagnosis. The true value of this work lies in its warning to the scientific community: if you want to know if a prediction tool actually works, you must test it without letting it see the very numbers that define the illness. By showing exactly how much performance drops when this rule is followed, the study provides a clear, honest framework for future research. It reminds us that in the race to use artificial intelligence for health, the most important step is ensuring the computer is learning the truth, not just memorizing the test.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.