📊 statistics

A Bayesian Prevalence Incidence Cure model for estimating survival using Electronic Health Records with incomplete baseline diagnoses

This paper proposes a Bayesian Prevalence Incidence Cure (PIC) model, a three-component mixture framework designed to accurately estimate survival, prevalence, and cure proportions from Electronic Health Records with missing baseline diagnoses, demonstrating its superiority over traditional Prevalence-Incidence models through simulations and a real-world application to Diabetic Macular Oedema.

Original authors: Matilda Pitt, Robert J. B. Goudie

Published 2026-02-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Matilda Pitt, Robert J. B. Goudie

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery about a group of patients. You have a massive notebook of medical records (Electronic Health Records, or EHR), but the notebook is messy. Some pages are torn out, some entries are missing, and you don't always know exactly when a patient's condition started or if they will ever get better.

The paper by Pitt and Goudie introduces a new "detective tool" called the PIC Model (Prevalence-Incidence-Cure) to make sense of this messy data. Here is how it works, broken down into simple concepts.

The Three Types of Patients

In many medical studies, researchers usually assume everyone starts "healthy" and waits to see who gets sick. But in the real world, it's more complicated. The authors say you need to sort patients into three different buckets:

The "Already Sick" (Prevalent): These patients already had the problem when they first walked into the clinic. Maybe they didn't have a test done that day, so the doctor didn't know they were sick until the next visit.
The "Newly Sick" (Incident): These patients were healthy at the start, but later developed the problem.
The "Unreachable" (Cured): These are the tricky ones. No matter how long you watch them, they will never get the problem (or never respond to the treatment). They are essentially "immune" or "non-responders."

The Problem with Old Tools

Before this paper, researchers used a tool called the PI Model (Prevalence-Incidence).

The Flaw: The old tool assumed that everyone would eventually get sick if they waited long enough. It didn't have a bucket for the "Unreachable" (Cured) group.
The Result: If you used the old tool on a group where some people would never get sick, the tool would get confused. It would try to force those "immune" people into the "Newly Sick" category, making it look like they would get sick much later than they actually would. This leads to wrong predictions about survival and treatment success.

The New Solution: The PIC Model

The authors built a 3-part mixture model (the PIC model) that acts like a smarter sorting machine.

It handles the missing pages: It uses math to guess whether a patient with a missing initial test was "Already Sick" or "Newly Sick," rather than just guessing randomly.
It handles the "Unreachable": It explicitly calculates the percentage of people who will never experience the event (the "Cure" proportion).
It uses "Expert Intuition" (Bayesian Priors): The model allows researchers to feed in what they already know from past studies (like "we expect the median time to get better to be 10 months"). This acts like a compass to guide the math, especially when there isn't a lot of data.

The Detective Story: Diabetic Macular Oedema (DMO)

To test their new tool, the authors looked at a real-world dataset of 1,964 patients with a specific eye condition called Diabetic Macular Oedema (DMO). The goal was to see how long it took for patients to get their eyesight back to a "good" level.

The Messy Data: Some patients had no eye test at the very start. Some had great vision immediately (they were "Already Sick" with good vision). Some never got better no matter what (the "Unreachable").
The Comparison:
- The Old Tool (PI Model) said: "Everyone will eventually get good vision, but it might take a long time." It predicted that after 4 years, only 8% of people would still have bad vision.
- The New Tool (PIC Model) said: "Wait, about 18% of these people are 'Unreachable'—they will never get good vision with this treatment." It predicted that after 4 years, 18% would still have bad vision.

Why This Matters

The paper shows that the new PIC model is more accurate.

It stops the "False Hope": The old model made it look like the treatment worked better than it did for everyone because it ignored the people who would never respond.
It fixes the "Missing Data" confusion: It correctly identifies who was already doing well at the start, rather than counting them as people who "got better" later.

The Bottom Line

Think of the old model as a weather forecast that assumes it will rain eventually for everyone, even if some people live in a desert. The new PIC Model is a smarter forecast that realizes:

Some people are already wet (Prevalent).
Some people will get wet later (Incident).
Some people live in a desert and will never get wet (Cured).

By acknowledging all three groups, the model gives doctors and patients a much clearer picture of what to expect, especially when the medical records are incomplete.

Technical Summary: A Bayesian Prevalence Incidence Cure Model for Estimating Survival Using Electronic Health Records with Incomplete Baseline Diagnoses

Problem Statement
Retrospective cohorts derived from Electronic Health Records (EHR) offer a cost-effective means to study disease prevalence, incidence, and cure proportions in real-world scenarios. However, EHR data present specific analytical challenges: (1) Missing Baseline Status: Patients often lack a recorded diagnostic test at the time of treatment initiation ( $t=0$ ), making it impossible to distinguish between prevalent cases (those already having the condition) and incident cases (those developing it later). (2) Treatment Non-Response (Cure): In many conditions, a fraction of patients will never experience the event of interest (e.g., achieving a specific visual acuity threshold), rendering standard survival functions that asymptote to zero inappropriate.

Existing Prevalence-Incidence (PI) models address missing baseline data using two-component mixture models but fail when a "cured" fraction exists, as they incorrectly assume all non-cured individuals will eventually experience the event. Conversely, standard Cure Models assume no prevalent cases at baseline. Neither approach simultaneously handles missing baseline diagnoses and a cure fraction, leading to biased estimates of survival probabilities, prevalence, and incidence times.

Methodology
The authors propose a Prevalence-Incidence-Cure (PIC) model, a three-component mixture model designed for interval-censored data with missing baseline status.

Model Structure: The population is partitioned into three latent groups:
- Prevalent Cases: Individuals who have already experienced the event at baseline ( $t < 0$ ).
- Incident Cases: Individuals who are disease-free at baseline but will experience the event ( $0 < t < \infty$ ).
- Cured Cases: Individuals who will never experience the event ( $t = \infty$ ).
Likelihood Construction: The model accounts for four observable data patterns based on diagnostic test timing:
- $c_i=1$ : Positive test at baseline (Prevalent).
- $c_i=2$ : Interval-censored incident cases (Negative at $l_i$ , Positive at $r_i$ ).
- $c_i=3$ : Censored before testing positive (Negative at $l_i$ , no positive test by end of follow-up). This group is a mixture of cured and censored incident cases.
- $c_i=4$ : Missing baseline test but positive at first follow-up. This group is a mixture of prevalent and incident cases.
Parametric Formulation:
- Component Assignment: A multinomial logistic model determines the probabilities of being prevalent ( $\pi_i$ ) or cured ( $\delta_i$ ) as a function of covariates. The probability of being incident is $1 - \pi_i - \delta_i$ .
- Time-to-Event: For incident cases, a Weibull distribution is used to model the survival function $S(t|xi)$, allowing for covariate effects via a proportional hazards parameterization.
Bayesian Inference and Priors: The authors adopt a Bayesian framework using Markov Chain Monte Carlo (MCMC) via JAGS. A key methodological contribution is the specification of informative priors based on expert elicitation or previous literature. Instead of placing priors directly on regression coefficients, the authors place priors on the ratios of probabilities (e.g., ratio of cured to incident) and on the odds ratios of covariates. These are transformed into log-normal priors for the logistic parameters and log-normal priors for Weibull parameters (shape $\alpha$ and median event time). This approach aims to incorporate domain knowledge while maintaining interpretability.

Key Results
The paper validates the model through simulation studies and a real-world application:

Simulation Study: Using a cervical cancer screening scenario, the PIC model was compared against a standard PI model under varying sample sizes ( $n=40, 400, 4000$ ) and prior specifications (informative, vague, misspecified).
- Bias Reduction: The PIC model demonstrated smaller bias in survival probability estimates compared to the PI model, particularly at later time points where the PI model incorrectly forces survival to zero.
- Prior Impact: At small sample sizes ( $n=40$ ), informative priors reduced bias significantly compared to vague priors, though at the cost of wider credible intervals. At larger sample sizes ( $n=400+$ ), vague priors performed comparably or better if the informative prior was slightly misspecified, suggesting the data eventually dominates the prior.
- PI Model Failure: The PI model consistently overestimated prevalence and underestimated long-term survival because it could not account for the cure fraction, leading to a survival function that plateaued at zero rather than the true cure proportion.
Application to Diabetic Macular Oedema (DMO): The model was applied to a dataset of 2,614 eyes from Moorfields Eye Hospital.
- Fit: The PIC model provided a substantially better fit to the non-parametric Turnbull estimate than the PI model, particularly in capturing the plateau in survival probability at 48 months (18% under PIC vs. 8% under PI).
- Estimates: The PIC model estimated that approximately 33% of patients were prevalent at baseline and 18% were non-responders (cured). The median time to achieving "good" eyesight (VA $\ge$ 70) for non-prevalent patients was estimated at 6.44 months, compared to 7.32 months under the PI model.

Significance and Claims
The authors claim that the PIC model offers a necessary extension to existing methods for analyzing EHR data where both missing baseline diagnoses and a cure fraction are present. By explicitly modeling the "cured" state alongside prevalent and incident states, the model prevents the bias inherent in PI models when applied to diseases with non-responders.

The paper emphasizes the utility of the Bayesian approach in incorporating prior knowledge through interpretable ratios, which is particularly valuable in medical settings with limited sample sizes or where previous clinical trial data exists. The authors note that while the model offers limited benefit if the cure proportion is negligible or follow-up is too short to observe the plateau, it is highly relevant for population-wide screening services (e.g., breast, cervical, and diabetic eye screening) and secondary care settings where baseline symptom data is often incomplete.

The authors remain modest regarding the model's scope, acknowledging that it treats eyes as independent (ignoring intra-patient correlation) and models a binary threshold on a continuous scale. They suggest future extensions could include test sensitivity/specificity, non-parametric survival distributions (e.g., Cox models), and time-varying covariates.