The Claims Floor: Limits of Administrative Claims Data for Dynamic Prediction of Pregnancy Loss and Stillbirth
This study demonstrates that despite their scale, administrative claims databases fail to support dynamic, above-chance prediction of pregnancy loss and stillbirth at clinically actionable gestational time points due to the lack of antenatal predictive signals and coding artifacts, thereby establishing a "claims floor" that contrasts with the higher potential of biophysical models.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Pregnancy is a journey measured in weeks, where the health of a developing baby can shift rapidly. For decades, doctors have relied on static checklists to guess which pregnancies might end in tragedy, such as a miscarriage or a stillbirth. These checklists look at a mother's history at a single moment in time, but they cannot capture the changing reality of a pregnancy as it progresses. In recent years, a new hope emerged: the idea that massive computer databases, which track millions of medical visits and insurance claims, could be used to build a living, breathing prediction tool. The promise was that by watching a mother's medical record update week by week, a computer could spot early warning signs of trouble long before a doctor might notice them, giving families a crucial window to intervene.
Researchers at the University of Nevada, Las Vegas, decided to test this promise using a vast collection of real-world medical records from Virginia. They gathered data on more than 344,000 pregnancies, tracking them from the first signs of pregnancy through to birth or loss. Their goal was to see if these administrative records, which contain codes for diagnoses and procedures, could predict pregnancy loss or stillbirth at specific moments when a doctor could still take action, such as starting medication or increasing monitoring. They built a system that updated its risk assessment every week, trying to distinguish between a healthy birth, an early loss before twenty weeks, and a stillbirth after twenty weeks. They were careful to remove any information that might have been recorded at the exact same time the event happened, ensuring the computer was truly predicting the future and not just reading the present.
The results of this massive experiment were surprising and definitive. When the researchers first looked at the data without strict safeguards, the computer models appeared to be incredibly accurate, seemingly predicting pregnancy loss with high success. However, this was an illusion caused by a subtle flaw in how the data was organized. The medical codes for a loss were often entered into the system during the same week the loss occurred. The computer was not predicting the future; it was simply noticing that a loss was happening right then and reporting it as a prediction. Once the researchers enforced a strict two-week waiting period between the prediction and the event to prevent this "leakage," the accuracy of the models collapsed. The ability to predict early pregnancy loss dropped from a high score to a level barely better than a random guess.
The study then turned to the more tragic outcome of stillbirth, which occurs after twenty weeks of pregnancy. Even with the data flaws fixed, the computer models failed to find any signal in the medical records that could forecast a stillbirth better than chance. The models performed no better than flipping a coin, regardless of how many different types of medical codes they were allowed to examine. In fact, the one code that usually signals a high-risk pregnancy actually made the predictions worse. This code, used when a doctor has already identified a risk and started extra monitoring, seemed to indicate that the pregnancy was under control and likely to continue, effectively hiding the danger from the algorithm. The researchers found that the specific medical details that carry the most weight for predicting these tragedies are often recorded only after the event has already begun or concluded, rather than appearing in the records beforehand.
This work establishes a clear boundary, or a "floor," for what can be learned from insurance claims data. While other studies using ultrasound images and physical measurements of the baby have shown that stillbirth can be predicted with reasonable accuracy, this study proves that administrative records alone cannot reach that same level of insight. The signals needed to foresee these events are simply not present in the daily paperwork of medical visits until it is too late to act. The researchers also uncovered two important lessons for anyone trying to use similar data in the future. First, they showed how easily a model can be fooled by data that leaks from the present into the future, a problem that can be fixed by creating a gap between the prediction time and the event. Second, they demonstrated that a model can appear to be performing terribly or even "worse than useless" due to mathematical quirks in how rare events are counted, rather than because the model is actually broken.
Ultimately, the study concludes that while large databases are excellent for counting how many people have a condition after it happens, they are not yet capable of foreseeing pregnancy loss or stillbirth before it occurs. The information required to make such a prediction exists in the physical world of the womb, captured by ultrasound and Doppler scans, but it does not yet exist in the digital world of insurance claims. For now, the hope of using routine medical records to give doctors a head start on preventing these tragedies remains out of reach, not because the data is too small, but because the necessary clues are not written down until the event is already underway.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.