Stable and practical semi-Markov modelling of intermittently-observed data

This paper introduces a practical semi-Markov modeling framework for intermittently observed data by approximating sojourn times with phase-type distributions to enable flexible likelihood calculation, and provides a new R package, "msmbayes," to implement this approach via Bayesian or maximum likelihood estimation.

The Gist

The Big Picture: Tracking Life's Changes

Imagine you are trying to track a person's health over time. You check in on them occasionally—maybe once a year or every few months. You want to know: How long do they stay in a "healthy" state before getting sick? And once they get sick, how long until they recover or pass away?

In statistics, this is called a multi-state model. It's like a map with different rooms (states) and doors (transitions) between them.

The Problem: The "Memory" Trap

Most standard maps assume that the chance of leaving a room depends only on which room you are currently in. This is called the Markov assumption. It's like saying: "If you are in the 'Sick' room, the chance of leaving is 50% tomorrow, regardless of whether you just walked in or have been there for a year."

But in real life, time matters. If you have been sick for a long time, you might be more likely to get better (or worse) than if you just got sick. This is a Semi-Markov model, where the "clock" inside the room matters.

The Catch: Because we only check in on people occasionally (intermittent data), we don't know exactly when they entered a room. We just know they were in Room A in January and Room B in June. We don't know if they got sick in February or May. This makes it incredibly hard to calculate the "clock" inside the room.

The Old Solutions: Too Slow or Too Rigid

Scientists have tried to solve this before, but the tools were either:

1. Too slow: Trying to guess every possible path the person took between check-ins is like trying to count every grain of sand on a beach to find one specific one.
2. Too rigid: Some methods only worked for very simple maps, not the complex ones used in real medicine.
3. Too complicated: Some methods required custom, hard-to-use software that wasn't available to most researchers.

The New Solution: The "Hidden Phase" Trick

The author, Christopher Jackson, introduces a clever new way to solve this using a concept called Phase-Type distributions.

The Analogy: The Hotel with Secret Hallways
Imagine a "Sick" room isn't just one big room. Instead, it's actually a hotel with a long hallway of smaller, hidden rooms (phases) inside it.

• When a person enters the "Sick" state, they enter the first hidden room.
• They move through these hidden rooms one by one.
• The time they spend in each hidden room is simple and predictable (like a standard clock).
• When they finally exit the last hidden room, they leave the "Sick" state.

By stringing these simple hidden rooms together, you can create a complex, realistic "Sick" room where the time spent matters (e.g., you are more likely to leave after passing through 3 hidden rooms than after just 1).

Why this is a game-changer:
Because the movement between these hidden rooms is simple, computers can calculate the math very easily. It turns a complex "Semi-Markov" problem into a standard "Hidden Markov" problem, which computers are already very good at solving.

The Innovation: The "Moment-Matching" Recipe

There was a previous attempt to use this "hidden hallway" idea, but it was like trying to bake a cake by guessing the ingredients. You had to run a massive, slow computer search to figure out how to arrange the hidden rooms to match a specific shape (like a Weibull or Gamma distribution).

This paper introduces a fast, analytic recipe (called Moment-Matching).

• Instead of guessing, the author provides a mathematical formula.
• You tell the computer: "I want the time spent in this state to look like a Gamma distribution with these specific properties."
• The computer instantly calculates exactly how to set up the hidden rooms (the phases) to match that shape perfectly.

It's like having a magic mold that instantly shapes the hidden hallway to fit any specific time pattern you need, without the slow guessing game.

The Tool: msmbayes

The author has packaged this entire method into a new software tool called msmbayes (available in R).

• What it does: It allows researchers to build complex maps of health states, even when data is sparse and irregular.
• Why it's stable: Sometimes, the data is so weak that the computer gets confused and crashes (a problem called "non-identifiability"). This tool uses Bayesian statistics, which is like giving the computer a "hint" based on what we already know from previous studies. This stabilizes the calculation, ensuring it produces a result even when the data is fuzzy.

The Proof: Testing and Real-World Use

The author tested this method in two ways:

1. Simulation: They created fake data where they knew the "true" answer, ran the software, and confirmed it found the correct answer every time.
2. Real Data: They applied it to a study of cognitive function in older adults (the ELSA study). They tracked how people moved between different levels of memory ability and death.
   • The standard method (Markov) assumed the risk of death was constant once you were in a certain memory state.
   • The new method (Semi-Markov) showed that the risk actually changes depending on how long you've been in that state.
   • The results showed that the new method provided a slightly better fit to the data and gave more realistic estimates of how long people stay in different cognitive states.

Summary

This paper builds a new, stable, and easy-to-use software tool that lets scientists model how people move between different life states (like health to sickness) even when they only check in on them occasionally. It does this by breaking complex time patterns into simple "hidden steps" and using a fast mathematical recipe to set them up, making advanced modeling accessible to everyone.

Technical Summary

Technical Summary: Stable and Practical Semi-Markov Modelling of Intermittently-Observed Data

Problem Statement
Multi-state models are standard tools for analyzing categorical variables changing over time, particularly in biomedical contexts where data are "intermittently-observed" (panel data). In such data, the exact times of entry to and exit from states are unknown, as are the specific pathways taken between observation points. Traditionally, these models rely on the Markov assumption, where transition rates are independent of the time spent in the current state. However, this assumption is often unrealistic; in many biological processes, the risk of transition depends on the duration of the current state (sojourn time).

Relaxing the Markov assumption to create semi-Markov models for panel data is computationally challenging. Existing methods face significant hurdles:

1. Integration over latent times: Numerical integration over unobserved transition times becomes computationally prohibitive as the number of unobserved transitions increases.
2. Nonparametric estimators: These are often restricted to acyclic transition structures (no cycles).
3. Simulation-based approaches: Methods like custom MCMC or stochastic EM algorithms exist but lack general software implementation.
4. Phase-type models: While Titman and Sharples proposed using phase-type distributions to express semi-Markov models as Hidden Markov Models (HMMs), allowing for easy likelihood calculation via the forward algorithm, this approach introduces a proliferation of latent parameters. This often leads to poor identifiability, causing optimization algorithms to fail, particularly with sparse intermittent data. Previous attempts to constrain these models (e.g., Titman [28]) relied on intensive numerical optimization and ad-hoc spline choices to approximate standard distributions (Weibull, Gamma), lacking a general, stable software implementation.

Methodology
The paper proposes a new computational framework to make stable semi-Markov modelling accessible for general state structures. The core methodology involves three key components:

1. Phase-Type Approximation via Moment Matching:
   Instead of using numerical optimization to approximate standard distributions, the authors employ a fast analytic procedure. They utilize a specific family of Coxian phase-type distributions (inspired by Bobbio, Horváth, and Telek [5]) defined by a sequence of latent phases.

   • The method matches the first three moments (mean, variance, and third moment) of a target distribution (Gamma or Weibull) to a phase-type distribution.
   • This yields unique closed-form expressions for the phase-type transition intensities as a function of the target distribution's shape and scale parameters.
   • This approach restricts the phase-type family to one with properties similar to simpler distributions, thereby reducing the parameter space and improving identifiability compared to a fully flexible phase-type model.

2. Extension to Covariates:
   The framework extends the approximation to include covariates in two ways:

   • Sojourn Time: Covariates influence the scale parameter of the sojourn distribution via an Accelerated Failure Time (AFT) model. This is implemented by scaling all transition intensities within the latent phase sequence by the same factor.
   • Next-State Probabilities: Covariates influence the probability of transitioning to specific subsequent states using a multinomial logistic regression. This allows the relative risk of different exit destinations to vary with covariates independently of the sojourn time.

3. Software Implementation (msmbayes):
   The method is implemented in a new R package, msmbayes. The package supports:

   • General state-transition structures (including cycles).
   • Both Maximum Likelihood Estimation (via quasi-Newton optimization) and Bayesian estimation (via Hamiltonian Monte Carlo using Stan).
   • The use of prior information to stabilize estimation in cases of weak identifiability, a common issue in multi-state modelling with intermittent data.

Key Results
The paper validates the methodology through simulation-based calibration and a real-world application:

• Simulation-Based Calibration:

  • Accuracy: MCMC samples from the msmbayes package produced posterior distributions that passed simulation-based calibration tests (uniform rank statistics), confirming the correctness of the implementation.
  • Laplace Approximation: The Laplace approximation (posterior mode + Hessian) was found to be substantially faster (approx. 1% of MCMC runtime) but introduced bias in shape parameters and next-state log-odds, particularly underestimating uncertainty. However, it provided a reasonable approximation for Markov models.
  • Stability: In scenarios where Maximum Likelihood Estimation (MLE) failed to converge (e.g., 76% of simulated datasets for a Markov model with weak identifiability), the Bayesian approach successfully produced converged posteriors dominated by the prior, providing a valid characterization of uncertainty where MLE failed.

• Application to Cognitive Function (ELSA Data):

  • The method was applied to the English Longitudinal Study of Ageing (ELSA) to model transitions between cognitive function states and death.
  • A standard Markov model with many parameters was found to be practically unidentifiable, yielding unrealistically large standard errors.
  • The semi-Markov model (using 5-phase Weibull/Gamma approximations) provided a modest improvement in fit (lower penalized log-likelihood) and revealed that the hazard of transition decreases with time since state entry (shape parameters < 1).
  • Covariate effects (age, sex, education) were estimated with greater stability using Bayesian priors derived from external mortality data.

Significance and Claims
The paper claims to provide the first general software implementation for stable semi-Markov modelling of intermittently-observed data with arbitrary transition structures. Its primary contributions are:

1. Accessibility: It transforms semi-Markov modelling from a niche, computationally difficult task into a practical workflow accessible via the msmbayes R package.
2. Stability: By using moment-matching to constrain phase-type distributions to familiar families (Gamma/Weibull), the method mitigates the identifiability issues that plague general phase-type models, particularly when data are sparse.
3. Bayesian Utility: It demonstrates that Bayesian estimation is crucial for stabilizing inference in complex multi-state models where likelihood surfaces are flat or optimization fails.
4. Flexibility: The approach handles general transition structures (including reversible transitions) and covariates affecting both sojourn times and next-state probabilities.

The authors acknowledge limitations, including the computational cost of matrix exponentials for large state spaces (mitigated by using Laplace approximations or fewer phases) and the assumption that next-state probabilities are independent of sojourn time. They note that while the method improves upon the exponential assumption, it remains a parametric approximation, and model checking for such complex, intermittently-observed data remains challenging.