Interpolated Supervision Can Change What Regression Benchmarks Reward
This paper demonstrates that constructing dense regression labels from sparse temporal observations via interpolation fundamentally alters benchmark rankings and model selection, often reversing performance comparisons without reflecting true clinical efficacy or prediction architecture improvements.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the world of medical monitoring and data science, researchers often rely on a simple assumption: if you want to predict how a patient is feeling every single day, you need a measurement for every single day. But in reality, doctors cannot see a patient every day. They see them once a month, or perhaps once every few weeks, and then they must guess what happened in between. To fill these gaps, data scientists often use a technique called interpolation. Imagine drawing a straight line between two dots on a graph; that line represents the estimated values for the days in the middle. For decades, this method has been treated as a harmless way to create a complete dataset from a sparse one. It allows computers to train on "daily" data even when the only real observations are weekly or monthly. However, a new study suggests that this convenience might be hiding a serious problem. The way we construct these daily estimates can fundamentally change which computer models appear to be the best, potentially leading researchers to choose the wrong tools for the job.
The researchers behind this study set out to investigate a specific question: does the method used to fill in the missing days actually change which prediction model wins a competition? They did not look for new medical discoveries or propose a better way to treat patients. Instead, they acted as auditors, examining the very rules of the game used to judge these models. They focused on a well-known dataset regarding Parkinson's disease, where patients record their voices daily, but their symptom severity is only rated by a doctor on a few specific days. The dataset provided to the public already contained the "daily" ratings, but the researchers knew these were not independent observations; they were mathematically drawn lines connecting the few real doctor visits. The team wanted to see if using these drawn lines as the target for training and testing models would yield different results than if they had used only the real, observed points.
To answer this, the team conducted a series of controlled experiments using not just the Parkinson's data, but also data on air quality and household energy usage. In these other datasets, they knew the true, continuous measurements because they came from sensors that recorded data every moment. This allowed them to create a perfect control group. They took the same computer models and the same input data, but they changed only the target they were trying to predict. In one scenario, the models were judged against the real, dense sensor data. In the other, they were judged against a version of that data where the real points were kept, but the gaps were filled in with straight lines, just like in the Parkinson's dataset. The input data—the voice recordings or the weather conditions—remained exactly the same. The only thing that changed was the definition of the "correct" answer.
The results were striking. When the researchers changed the target from the real sensor data to the interpolated, line-drawn data, the ranking of the models flipped. In the air quality experiments, when the gaps between real data points were large, the order in which the models were ranked changed dramatically. For instance, in one specific setup, nearly half of the pairs of models swapped places; a model that was clearly better when judged against the real data suddenly appeared worse when judged against the drawn lines, and vice versa. This happened even though the models themselves had not changed, and the data they used to make predictions was identical. The study found that the method of constructing the target label was not just a minor detail; it was a powerful force that could reverse the outcome of a scientific comparison.
The researchers also tested what happened when the models were actually trained on these drawn lines rather than just tested against them. They found that training on the smoothed, interpolated data made the models behave differently, often making them more similar to each other and less sensitive to the small, rapid fluctuations that occur in real life. In a separate set of simulations designed to mimic a patient whose symptoms rise and fall in a regular cycle, the team showed that if the "anchor" points used for drawing the lines happened to miss the peaks and valleys of that cycle, the resulting model would systematically fail to see those patterns. The model would learn the smooth line instead of the jagged reality. This confirmed that a model trained on interpolated data might be excellent at predicting the smooth line, but poor at predicting the actual, messy variations of the condition it is supposed to monitor.
Crucially, the study did not claim that interpolation is always wrong or that it should never be used. The author noted that sometimes a smooth trajectory is exactly what is needed. The problem arises when researchers use these smoothed targets to make claims about the real, unobserved daily fluctuations. The study demonstrated that you cannot use a model's success on a smoothed target to prove it is good at detecting daily changes, because the target itself has erased those changes. In the Parkinson's dataset, the researchers could not know the true daily severity of symptoms because no independent daily ratings existed. Therefore, they could not say which model was truly the best at predicting daily symptoms. They could only say that the choice of model depended entirely on whether the researchers cared about the smooth line or the jagged reality.
The final conclusion of the work is a call for greater transparency in how data is prepared. The study showed that the "winner" of a machine learning competition is often determined by how the target labels were constructed, not by the sophistication of the algorithm. If a dataset is built by connecting the dots with straight lines, the models that win will be the ones best at drawing straight lines. If the goal is to understand the complex, fluctuating nature of a disease, relying on a target that has been smoothed out can lead to misleading results. The researchers provided a mathematical check to help others determine if their rankings are stable, but they found that in many cases, the rankings were not stable at all. The work serves as a reminder that in science, the answer you get is often a reflection of the question you asked, and if the question is built on an artificial line, the answer will be an artificial truth.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.