💻 computer science

Proper Scoring Rules for Right-Censored Survival Data

This paper proposes a unified framework for proper scoring of right-censored survival data by mapping predictive distributions through the censoring mechanism, which recovers established criteria, extends common scoring rules like CRPS and Brier scores to censored settings, and enables improved multivariate modeling via "censored engression" while avoiding the ranking inconsistencies of existing plug-in weighted methods.

Original authors: Jef Jonkers, Glenn Van Wallendael, Luc Duchateau, Sofie Van Hoecke

Published 2026-06-05

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jef Jonkers, Glenn Van Wallendael, Luc Duchateau, Sofie Van Hoecke

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a weather forecaster trying to predict exactly when a storm will hit a specific town. In a perfect world, you would watch the storm until it arrives, record the exact time, and check if your prediction was right.

But in the real world of survival analysis (predicting when things like equipment failure or patient health events happen), you often don't get to see the full story. This is called right-censoring.

The Problem: The "Mystery Box" of Censoring

Imagine you are watching a race, but some runners leave the track early because the stadium lights go out (they are "censored").

Runner A finishes the race at 10 minutes. You know exactly when they finished.
Runner B is still running when the lights go out at 15 minutes. You know they finished after 15 minutes, but you don't know if they finished at 16, 20, or 100 minutes.

If you try to judge your weather forecast (or race prediction) using standard rules, you run into a trap. If you simply guess "15 minutes" for Runner B because that's when you stopped watching, you are lying about the data. If you try to guess "infinity," you are also wrong. Standard scoring rules get confused because they don't know how to handle these "mystery boxes" where the event time is hidden.

The Solution: "Play the Game by the Same Rules"

The authors of this paper propose a clever, simple idea: Don't try to guess the hidden truth; guess what you would have seen if you were in the same situation as the observer.

Think of it like this:

The Standard Mistake: You have a crystal ball that predicts the true finish time of every runner. You try to compare your crystal ball to the runners who left early. The math breaks because you are comparing a full story to a half-story.
The Paper's Fix: Before you check your crystal ball, you simulate the lights going out on your prediction, too.
- If your crystal ball says "Runner B finishes at 20 minutes," and the lights go out at 15, your "simulated" prediction becomes "Runner B finished after 15."
- Now, you compare your "simulated half-story" with the "actual half-story" you observed.

By forcing your prediction to go through the same "censoring filter" as the real data, you create a fair playing field. You are no longer penalized for not knowing the future; you are only judged on how well you predicted what was observable.

The New Tools: "Censored Scores"

The paper takes several standard ways of grading predictions (like the Brier score or CRPS) and builds these "light-switch" filters into them.

Localized Scores: If you know exactly when the lights went out (e.g., a fixed study end date), you apply the filter once.
Marginalized Scores: If the lights go out at random times (e.g., patients leaving a hospital at different times), you average the scores over all possible times the lights could have gone out, based on the known rules of the hospital.

The authors prove that if you do this, your "oracle" (the perfect predictor) will always get the best grade, even with missing data. They also show that older methods, which try to "fix" the missing data by guessing the probability of the event happening later, can sometimes trick the system and give a bad prediction a better score than a good one.

The New Training Method: "Censored Engression"

The paper doesn't just offer a way to grade predictions; it offers a way to teach computers to make them.

They introduce a method called Censored Engression.

Imagine a robot trying to learn the rules of a game by watching videos where the screen sometimes cuts to black.
Naive Approach: The robot ignores the black screens and tries to learn the full game, which confuses it.
Censored Engression: The robot is trained to predict what the video looks like including the black screens. It learns to generate "fake" videos that also get cut off by black screens at the right moments, and then compares those to the real videos.

This allows the robot to learn complex, multi-variable patterns (like predicting when a patient might have two different types of kidney failure) without needing a simple mathematical formula that might not fit the messy reality.

Real-World Test: The ICU

To prove this works, the authors tested it on data from an Intensive Care Unit (ICU). They tried to predict when patients would develop Acute Kidney Injury (AKI).

The Challenge: Patients leave the ICU (get discharged) or die before they develop AKI. This cuts off the data.
The Result: Their new method, which respects the "cut-off" nature of the data, predicted the timing of kidney failure much better than older methods that ignored the censoring or used "naive" training. It correctly identified that the "lights going out" (discharge) was a crucial part of the story, not just missing data.

Summary

In short, this paper says: When you can't see the whole picture, don't pretend you can. Instead, adjust your predictions to match the "blurry" version of reality you actually have. By doing this, you get fairer grades for your predictions and you can train smarter AI models that understand the limits of what they can see.

Technical Summary: Proper Scoring Rules for Right-Censored Survival Data

Problem Statement

In survival analysis and time-to-event modeling, the primary objective is often to forecast the full conditional event-time distribution $F_X$ given covariates $X$ , rather than just estimating mean survival times or hazard rates. Proper scoring rules provide a rigorous theoretical basis for training and evaluating such probabilistic forecasts, ensuring that the true predictive distribution is optimal in expectation.

However, a fundamental difficulty arises in the presence of right censoring. In these settings, the latent event time $T$ is not fully observed; instead, one observes $Y = \min(T, C)$ and the event indicator $\Delta = \mathbb{1}\{T \le C\}$ , where $C$ is the censoring time. Standard proper scoring rules, designed for uncensored outcomes, cannot be applied directly because the true event time $T$ is missing. Existing approaches often rely on Inverse Probability of Censoring Weighting (IPCW) or plug-in estimates where weights depend on the forecast itself. The paper highlights that forecast-dependent weighting can lead to "ranking reversals," where suboptimal forecasts are incorrectly ranked higher than the oracle forecast, and that standard IPCW implementations may fail to be strictly proper if they do not account for conditional independence correctly.

Methodology

The authors propose a unified framework for defining proper scoring rules under right censoring based on a core principle: score the forecast after applying the same coarsening operation that generated the data.

1. Localization and Marginalization

The framework distinguishes between two scenarios based on the nature of the censoring mechanism:

Localized Scores (Fixed Censoring): For a fixed censoring time $c$ , the predictive distribution $F$ is first mapped to the censored outcome space. The censored law $F^\flat_c$ is defined such that the probability mass on the identifiable region $[0, c]$ remains unchanged, while all mass beyond $c$ is collapsed into a censored state $\ast$ . The proper score is then applied to this induced censored law. This yields a score that is strictly proper for the latent law only on the identifiable region.
Marginalized Scores (Random Censoring): When the censoring time $C$ $C$ is random and only partially observed (e.g., we know $C \ge Y$ $C \geq Y$ if an event is observed), the localized score is averaged over the conditional law of $C$ $C$ given the observed data $(Y, \Delta, X)$ $(Y, Δ, X)$ .
- The resulting marginalized score $\bar{S}$ is defined as:
  $\bar{S}(F; Y, \Delta, X) = \int S^\flat_c(F; Y, \Delta) \, \Pi(dc \mid Y, \Delta, X)$
- Crucially, the weighting distribution $\Pi$ is derived solely from the censoring mechanism (the true or estimated distribution of $C$ ), not from the forecast $F$ being evaluated. This ensures the score remains forecast-independent.

2. Derivation of Specific Scores

The authors demonstrate that this framework recovers familiar criteria and extends them to new ones:

Logarithmic Score: The marginalized version recovers the standard right-censored negative log-likelihood.
CRPS (Continuous Ranked Probability Score): A marginalized version is derived that weights the tail error by the conditional survival function of the censoring time.
Brier Score: The framework yields a marginalized Brier score that, when rescaled, is equivalent to the IPCW Brier score but derived without forecast-dependent weights.
Pinball Loss and Energy Score: New right-censored versions of these scores are derived, suitable for quantile regression and multivariate distribution modeling, respectively.

3. Censored Engression

Extending the principle of localization-by-censoring to training, the authors introduce censored engression. This is a sample-based learning objective for multivariate right-censored survival modeling.

Instead of training a generator $g(x, \epsilon)$ to match latent event times (which are unobserved), the method applies the censoring map $\psi_c$ to both the generated samples and the observed outcomes.
The model is trained to minimize the censored energy score, which compares the distribution of censored generated samples against the observed censored outcomes.
This approach allows for flexible joint modeling of multivariate event times without requiring a tractable likelihood function.

Key Contributions

Unified Framework: A coherent construction of proper scoring rules for right-censored data by censoring the predictive distribution prior to scoring, rather than weighting the score based on the forecast.
Theoretical Guarantees: Proof that the marginalized score is proper under conditional independent censoring and strictly proper on the identifiable region (the region where the censoring mechanism allows the latent distribution to be distinguished).
Concrete Score Derivations: Explicit formulas for right-censored versions of the Logarithmic score, CRPS, Pinball loss, Brier score, and Energy score, clarifying their relationship to existing IPCW-based criteria.
Censored Engression: A novel training objective for multivariate survival modeling that leverages the censored energy score to learn flexible joint distributions from censored data.
Empirical Validation: Demonstration that forecast-independent scores correctly rank the oracle forecast across various censoring regimes, whereas forecast-dependent plug-in weighted scores can exhibit ranking reversals.

Experimental Results

The paper validates the methodology through controlled simulations and a clinical case study:

Forecast Evaluation: In simulations with known data-generating processes (DGPs), the proposed marginalized and localized scores consistently ranked the oracle forecast as the best (Rank 1) across administrative, random independent, and conditionally independent censoring regimes. In contrast, forecast-dependent plug-in versions of existing scores (e.g., from Yanagisawa [3]) frequently failed, ranking the oracle lower than exploit forecasts that concentrated mass in the tail.
Multivariate Learning: In synthetic experiments with multivariate event times (dimensions $k=2$ $k = 2$ to $10$) and a complex mixture log-normal DGP, censored engression substantially outperformed:
- Naive observed engression: Which treats censored times as fully observed, leading to poor predictive distributions.
- Likelihood-based baselines: Including Weibull and Copula models. While these performed better than naive training, they were outperformed by censored engression, particularly in capturing multimodal distributions that lie outside the parametric families of the baselines.
Clinical Case Study (ICU AKI): Applied to predicting the joint time-to-event for creatinine-based and urine-output-based Acute Kidney Injury (AKI) using MIMIC-IV data. Censored engression achieved the lowest localized energy scores and Brier scores compared to naive training and likelihood-based baselines, demonstrating improved performance in modeling joint clinical endpoints under censoring.

Significance and Claims

The paper argues that censored probabilistic evaluation and training must be built on forecast-independent censoring adjustments.

Correcting Evaluation Targets: By basing weights solely on the censoring mechanism, the proposed scores preserve the "oracle ranking" property. This prevents the circularity where a forecast influences its own evaluation weights, which can distort the optimization landscape and lead to ranking reversals.
Flexibility in Modeling: The introduction of censored engression provides a pathway to learn complex, high-dimensional, and multivariate survival distributions without the restrictive assumptions of parametric likelihoods.
Identifiability: The authors modestly note that strict propriety is only achievable on the identifiable region of the latent space. Outside this region (e.g., in the tails beyond the follow-up horizon), no scoring rule based solely on censored data can distinguish between different latent distributions that induce the same observed law.

In summary, the work provides a theoretically sound and practically effective framework for handling right-censored data in probabilistic forecasting, bridging the gap between rigorous scoring rule theory and the practical challenges of survival analysis.