📊 statistics

Goodness-of-Fit Checks for Joint Models

This paper introduces a comprehensive Bayesian posterior predictive checks framework implemented in the R package JMbayes2 to assess the goodness-of-fit for joint models of longitudinal and time-to-event data, effectively identifying model misspecifications that standard criteria often overlook.

Original authors: Dimitris Rizopoulos, Jeremy M. G. Taylor, Isabella Kardys

Published 2026-01-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Dimitris Rizopoulos, Jeremy M. G. Taylor, Isabella Kardys

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery about a patient's health. You have two types of clues:

The Longitudinal Clues: A series of check-up notes (like blood pressure or kidney function) taken over time.
The Survival Clue: The moment the patient experiences a major health event (like a heart attack or hospitalization).

Joint Models are like a super-smart detective's notebook that tries to connect these two types of clues. It asks: "Does the way a patient's blood pressure is changing right now tell us when they might have a heart attack?"

For a long time, statisticians had a way to write these notebooks, but they lacked a good way to check if the notebook was actually telling the truth. They had tools to compare two different notebooks and say, "Notebook A is slightly better than Notebook B," but they couldn't say, "Notebook A is completely wrong because it's missing a key piece of the puzzle."

This paper introduces a new set of "Reality Checks" (called Posterior Predictive Checks) to see if the detective's notebook actually matches real life.

The Three Ways to Check the Notebook

The authors suggest three different ways to test the model, depending on who you are looking at:

The "Rearview Mirror" Check (Existing Subjects): You look at a patient you already know. You feed the model their past notes and ask, "If we simulate what should have happened based on your model, does it look like what actually happened?"
The "Blind Prediction" Check (New Subjects): You look at a brand new patient where you only know their age or gender, but you have no medical history yet. You ask, "Based on the general rules of the model, what would a typical person like this look like?" This tests if the model understands the general population, not just the specific people it studied before.
The "Mid-Journey" Check (Dynamic Prediction): Imagine you are halfway through a patient's story. They are still alive, and you have notes up to today. You ask, "Given everything we know right now, does the model predict their future correctly?" This is crucial for doctors who need to update predictions as new data comes in.

How Do They Check? (The Tools)

The paper uses several creative tools to compare the "Simulated World" (what the model thinks happens) with the "Real World" (what actually happened):

The Average Line (Mean): Imagine drawing a line through all the patients' blood pressure readings. Does the model's average line follow the real line? If the real line goes up and the model's line stays flat, the model is missing something.
The Wiggle Room (Variance): Sometimes patients' numbers jump around a lot; other times they are steady. Does the model capture this "wiggle room"? If the real data is chaotic but the model thinks everyone is calm, the model is too rigid.
The Rhythm (Correlation): If a patient's blood pressure spikes today, does the model expect it to be high tomorrow too? The model needs to understand the "rhythm" of the data.
The Link (Concordance): This is the most important part for joint models. It checks if the model correctly connects the change in the notes to the risk of the event. It's like checking if the model correctly guessed that a sudden drop in kidney function leads to a higher chance of hospitalization.

The Real-World Test: The Heart Failure Study

The authors tested these checks on a real study called Bio-SHiFT, which followed heart failure patients. They were tracking two things:

eGFR: A measure of kidney function.
NGAL: A protein in the blood that signals inflammation.

They wanted to see if changes in these numbers predicted heart failure hospitalizations or death.

What they found:

Standard tools (like DIC and WAIC) couldn't tell the difference between a good model and a bad one. They were like a judge who just picks the "neatest looking" notebook without reading the story.
The new Reality Checks were much sharper. They could spot when a model was using the wrong math (like assuming a straight line when the data was actually curving).
They discovered that for kidney function, the model worked better if it looked at the average history of the patient rather than just the current number.
They also found that the protein marker (NGAL) was a much stronger predictor of trouble than the kidney function number, a detail the new checks helped confirm.

The Simulation Test: The "Fake Data" Lab

To prove their checks work, the authors created a "Fake Data" lab. They built a perfect model, generated 300 fake patients, and then tried to trick the system by feeding it broken models (models with wrong math or wrong assumptions).

The Result: The standard tools failed to notice the broken models; they thought the bad models were fine.
The New Checks: The new "Reality Checks" immediately spotted the errors. They could say, "Hey, this model thinks the data is a straight line, but the fake data is actually a curve!"

The Bottom Line

This paper gives statisticians and doctors a new flashlight. Instead of just guessing which model is best, they can now shine a light on the specific parts of the model to see if it fits the data.

The Good News: If a model is wrong, these checks will likely tell you where it is wrong (is it the average? the variability? the connection to the event?).
The Tool: All of this is available in a free software package called JMbayes2, so anyone can use these checks today.

In short, the paper says: "Don't just trust the model because it looks good on paper. Simulate the future, compare it to reality, and make sure your detective notebook is actually solving the mystery."

Problem Statement
Joint models for longitudinal and time-to-event data are widely utilized across disciplines to analyze the association between repeated measurements and survival outcomes. Despite their increasing complexity and adoption, there is a notable absence of systematic tools to assess whether these models adequately fit the data or to identify which specific components may be misspecified. Existing model comparison criteria, such as the Deviance Information Criterion (DIC) and Watanabe-Akaike Information Criterion (WAIC), are primarily designed to select the "best" model among a set of candidates. However, as noted by Zhang et al. (2014) and others, these metrics do not provide assurance that a selected model actually fits the data well, nor do they offer diagnostic insights into which aspects of the joint model (longitudinal, survival, or their association) are incompatible with the observed data. Furthermore, joint models often rely on parametric assumptions that are difficult to evaluate without dedicated goodness-of-fit (GOF) tools.

Methodology
The authors introduce a comprehensive framework of Bayesian posterior predictive checks (PPC) designed to assess the fit of joint models across three distinct evaluation settings:

Existing subjects: Conditioning on observed longitudinal data, event times, and baseline covariates.
New subjects: Assessing fit for individuals with only covariate information (no longitudinal or event data).
Dynamic prediction: Evaluating fit at intermediate follow-up times ( $t_L$ ) for subjects who are event-free up to that point.

The framework distinguishes between "posterior-posterior" checks (sampling random effects conditional on observed subject data) and "posterior-prior" checks (sampling random effects from their prior distribution for generic subjects). Additionally, the authors propose cross-validated checks to mitigate over-optimism by evaluating models on independent test folds.

The GOF assessment targets three specific components:

Longitudinal Component: Fit is evaluated using the empirical cumulative distribution function (eCDF) for global fit. To address specific structural aspects, the authors assess the mean function (via loess curves), the variance function (via standardized residuals from loess curves), and the correlation structure (via sample semi-variograms). To account for informative visiting processes, replicated data are simulated at the same time points as the original observations.
Survival Component: Fit is assessed using the eCDF of simulated event times (adjusted for censoring via Kaplan-Meier or Breslow estimators) and probability integral transforms of subject-specific cumulative distribution functions to check for uniformity.
Association: The link between longitudinal and survival processes is examined using time-dependent concordance statistics ( $C(t)$ ), calculated using the Uno et al. (2011) estimator to account for censoring.

A numerical summary measure, the Mean Integrated Squared Error (MISE), is calculated to quantify the discrepancy between observed and replicated statistics. The methodology is implemented in the R package JMbayes2.

Key Contributions and Results
The paper applies this framework to the Bio-SHiFT heart failure study, investigating the association between renal dysfunction (eGFR and NGAL) and adverse outcomes. The analysis compared linear versus nonlinear longitudinal trajectories and current value versus average value functional forms in the survival submodel.

Bio-SHiFT Application: The results indicated that while DIC, WAIC, and log pseudo marginal likelihood (LPML) did not uniformly agree on the best model, the PPCs revealed distinct fit characteristics. For instance, nonlinear trajectories better described the mean function for specific patients, while linear models captured the correlation structure of eGFR better. The concordance statistic checks suggested that an average/integral functional form provided a better fit for the association between eGFR and the composite event than the current value form.
Simulation Study: A controlled simulation study with 300 subjects demonstrated the sensitivity of the proposed checks. While standard information criteria (DIC, WAIC, LPML) failed to detect misspecification in several scenarios (e.g., misspecified linear trends or incorrect functional forms in the survival submodel), the PPCs successfully identified these issues. Specifically, checks for the mean function and variance structure clearly identified misspecifications in the longitudinal submodel that information criteria missed. The concordance statistic checks were particularly effective at detecting misspecification in the association structure.

Significance and Claims
The authors claim that their proposed framework provides a necessary diagnostic toolset that complements existing model selection criteria. The primary significance lies in the ability to detect model misspecification that standard information criteria fail to identify. By decomposing the assessment into the longitudinal mean, variance, correlation, survival distribution, and the association structure, the framework allows researchers to pinpoint specific areas of model inadequacy. The authors modestly note that while the eCDF is a useful global measure, it is less sensitive to important misspecifications compared to component-specific checks like the mean function or semi-variogram. The methodology is designed to be flexible, allowing for the incorporation of alternative metrics, and is made accessible to the community through the JMbayes2 software package.

The Three Ways to Check the Notebook

How Do They Check? (The Tools)

The Real-World Test: The Heart Failure Study

The Simulation Test: The "Fake Data" Lab

The Bottom Line

More like this