📊 statistics

Adressing Separation: A Firth-corrected Joint Model for Longitudinal and Time-to-event Data with an Application on Dropout from Vocational Training

This paper proposes a Firth-corrected joint model for longitudinal and time-to-event data to address separation issues in survival submodels, demonstrating through simulations and a vocational training application that this approach yields less biased estimates and effectively models both direct and indirect factors influencing dropout risk.

Original authors: Sophie Potts, Viola Deutscher, Elisabeth Bergherr

Published 2026-06-10

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Sophie Potts, Viola Deutscher, Elisabeth Bergherr

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict when a student will quit their vocational training. You have two types of information:

The "Scoreboard": A time-to-event record showing exactly when each student quits.
The "Mood Meter": A long list of weekly surveys where students rate their satisfaction with the training.

In statistics, there is a powerful tool called a Joint Model that tries to combine these two things. It assumes that a student's current mood (satisfaction) directly influences their risk of quitting. It's like saying, "The lower the satisfaction score drops this week, the higher the chance they will quit next week."

However, this tool has a major flaw when the data gets "weird."

The Problem: The "Perfect Predictor" Trap

Imagine you are looking at a specific group of students, say, those working in "Logistics." In your data, zero students in this group quit. Every single one of them stayed.

In the world of statistics, this is called Separation. It's like a teacher who says, "If you wear a red shirt, you will never get a detention." If the data shows no detentions for red shirts, the computer tries to calculate the risk and gets stuck. It thinks, "The risk must be negative infinity!" or "The risk is so low it's impossible to measure!"

The computer's math breaks down. The numbers it spits out become huge, nonsensical, and useless (like a standard error of 4,000 when the number is usually around 1). This happens often with rare events or very small groups.

The Solution: The "Firth-Corrected" Brakes

The authors of this paper, Sophie Potts and her team, invented a fix. They took a technique called Firth's Correction (which was already used in other types of statistics) and adapted it for this Joint Model.

Think of the standard model as a car driving down a hill. When it hits the "Separation" cliff (the group where no one quits), the car accelerates out of control and flies off the edge.

The Firth-Corrected Joint Model adds brakes.

It puts a "penalty" on the math whenever the numbers start getting too crazy.
It forces the estimate to stay within a reasonable, finite range.
Instead of saying "The risk is negative infinity," it says, "Okay, the risk is very low, but let's give you a realistic number based on the rest of the data."

How They Proved It Works

The team ran a massive simulation. They created fake data where they knew the "true" answer.

Without the fix: When they created a "Separation" scenario (a group where no one quit), the standard model gave wild, wrong answers.
With the fix: The corrected model ignored the panic. It gave answers that were very close to the "true" answer, even when the data was messy. It essentially said, "I see no one quit here, but based on how the other groups behave, here is a sensible estimate."

The Real-World Test: German Vocational Training

They tested this new tool on real data from Germany regarding students quitting their apprenticeships.

The Setup: They tracked 4,203 students over time, looking at their satisfaction levels and when they quit.
The Glitch: In the data, there was a specific job sector ("Other commercial services") where zero women quit. This caused the standard model to break, giving a coefficient of -13 with a massive error bar.
The Fix: Using their new Firth-corrected model, they got a stable, sensible number.

What did they learn?

Satisfaction is a moving target: A student's satisfaction isn't just a one-time feeling; it's a journey. As satisfaction drops over time, the risk of quitting goes up.
Direct vs. Indirect Effects: Some factors (like education level) affect quitting directly. Others (like parental income) affect quitting indirectly by first changing how satisfied the student feels. The new model can separate these two paths clearly, whereas old models got confused by the "zero quit" group.

The Bottom Line

This paper doesn't just say "we fixed a math problem." It says: "We built a safety net for statistical models."

When you have rare events or small groups where "nothing happened," the old math breaks. The new Firth-corrected method acts like a stabilizer, allowing researchers to get reliable answers even when the data is sparse or unbalanced. In the case of vocational training, this means we can finally understand the true relationship between how happy a student is and whether they will quit, without the math crashing because one specific group happened to stay.

Technical Summary: Addressing Separation in Joint Models via Firth Correction

Problem Statement
Joint models for longitudinal and time-to-event data are widely used to analyze endogenous time-varying covariates (e.g., biomarkers or satisfaction levels) alongside survival outcomes. However, these models inherit limitations from their survival submodels, specifically the issue of separation. Separation occurs when a categorical covariate perfectly predicts the absence of an event in a specific category (e.g., no dropouts in a specific vocational sector). This leads to monotone likelihood, causing regression coefficients to diverge to infinity or become unstable, rendering standard maximum likelihood estimation (MLE) ineffective. While Firth's correction has been established for binary regression and Cox proportional hazards models to mitigate this bias, it had not been formally integrated into the joint modeling framework, limiting the applicability of joint models in settings with rare events or highly imbalanced covariates.

Methodology
The authors propose a Firth-corrected joint model implemented within a frequentist estimation framework using the Expectation-Maximization (EM) algorithm.

Model Structure: The model consists of two linked submodels:
- A longitudinal submodel (Linear Mixed Model) capturing the trajectory of the endogenous covariate $m_i(t)$ with random intercepts and slopes.
- A survival submodel (Cox-type proportional hazards model) where the hazard function depends on baseline covariates and the current value of the longitudinal process $m_i(t)$ via an association parameter $\alpha$ .
Firth Correction Implementation:
- The authors derive the necessary correction term for the score function of the survival submodel coefficients ( $\theta_{surv}$ ).
- The bias of the MLE is approximated using a second-order Taylor expansion of the score function. The correction term $c(\theta)$ is derived as $c(\theta) = \frac{1}{2} \text{tr}\left[ I(\theta)^{-1} \frac{\partial I(\theta)}{\partial \theta} \right]$ , where $I(\theta)$ is the Fisher Information matrix.
- This term is added to the score function $S(\theta)$ during the M-step of the EM algorithm, specifically for the Newton-Raphson update of the survival coefficients: $\hat{\theta}^{(t+1)}_{surv} = \hat{\theta}^{(t)}_{surv} + [I(\hat{\theta}^{(t)}_{surv})]^{-1} S^*(\hat{\theta}^{(t)}_{surv})$ , where $S^*$ is the corrected score.
- Initialization: To ensure convergence, the authors initialize the exogenous regression coefficients using Penalized Maximum Likelihood estimates from a classical Firth-corrected Cox model (via the coxphf R package).
- Inference: Standard errors are derived using the forward difference approximation of the full Hessian matrix.

Simulation Study Results
A simulation study was conducted to evaluate the performance of the Firth-corrected joint model (FC-JM) against the classical joint model (JM) and a JM with Firth-corrected starting values (JMc).

Bias Reduction: In scenarios with separation, FC-JM yielded significantly less biased estimates for the separating covariate compared to JM. The estimates approached the values observed in non-separation scenarios.
Information Dependence: The performance of the correction is driven by the curvature of the likelihood, which depends on the absolute number of events. FC-JM performed best in low-information settings (small sample sizes, high censoring) where separation is most problematic.
Stability: While FC-JM reduced bias, a small number of outliers were observed in very sparse data settings due to ill-conditioned analytical Hessians, a limitation not present in the standard JM which approximates the Hessian via BFGS.
Other Coefficients: The correction did not significantly alter the estimates of non-separating coefficients, maintaining accuracy across the model.

Real-World Application: Vocational Training Dropout
The method was applied to data from the German National Education Panel Study (NEPS) regarding dropout from vocational training.

Context: The study modeled the trajectory of training satisfaction (longitudinal) and the risk of dropout (time-to-event).
Separation Issue: In the female subsample, the vocational training sector "Other occupations in commercial services" exhibited separation (zero dropout events), causing the classical JM to produce an infinite coefficient and inflated standard error.
Findings:
- The FC-JM provided finite, stable estimates for the separating sector, indicating no significant difference in dropout risk compared to the reference sector.
- The model successfully decomposed effects into direct effects (e.g., education level, field of work compromise directly influencing dropout risk) and indirect effects (mediated through satisfaction).
- Results confirmed that while socioeconomic factors and compromise levels significantly influenced satisfaction, their direct effect on dropout was often non-significant once the satisfaction trajectory was accounted for, whereas education and field compromise retained direct protective or risk-increasing effects.

Significance and Claims
The paper claims that the proposed Firth-corrected estimation routine extends the applicability of joint models to datasets characterized by rare events or highly imbalanced categorical covariates where classical methods fail due to monotone likelihood.

Methodological Contribution: It provides a formal derivation and implementation of Firth's penalty within the joint model EM algorithm, allowing for the estimation of finite, less biased coefficients in separation cases.
Substantive Contribution: In the application to vocational training, the model offers a more rigorous causal framework by explicitly modeling the mediating role of satisfaction. It demonstrates that structural predictors often influence dropout risk indirectly through affective states (satisfaction), a mechanism that is obscured in standard two-stage or static predictor approaches.
Limitations: The authors note that the current implementation is restricted to current-value associations and time-constant exogenous covariates. Furthermore, the reliance on analytical Hessians can lead to numerical instability in extremely sparse data, suggesting a need for future stabilization strategies.

The work is presented as a solution to a specific estimation failure mode, enabling researchers to utilize joint models in contexts previously deemed unsuitable due to separation, thereby improving the robustness of inference in social science and biostatistical applications involving rare events.