📊 statistics

The super learner for time-to-event outcomes: A tutorial

This paper provides a practical tutorial on implementing super learner methods for time-to-event outcomes, offering an accessible overview of three specific approaches for both discrete and continuous-time data, along with a comparative analysis and R code examples.

Original authors: Ruth H. Keogh, Karla Diaz-Ordaz, Nan van Geloven, Jon Michael Gran, Kamaryn T. Tanner

Published 2026-02-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ruth H. Keogh, Karla Diaz-Ordaz, Nan van Geloven, Jon Michael Gran, Kamaryn T. Tanner

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a doctor trying to predict how long a patient might live after a diagnosis, or perhaps a mechanic trying to guess when a car part will fail. You have a bunch of data: age, weight, blood pressure, mileage, etc. But there's a catch: some patients leave the study early (they move away or drop out), and some cars are still running when you stop looking. This is called censored data.

The big problem is: Which prediction tool is the best?
Should you use a simple formula? A complex machine learning algorithm? A random forest? You don't know for sure until you try them all, and even then, you might get lucky with one specific dataset.

This paper introduces a solution called the Super Learner. Think of it not as a single "super genius" doctor, but as a brilliant committee.

The Core Idea: The "Committee of Experts"

Imagine you are trying to predict the weather.

Expert A is a meteorologist who uses complex physics equations.
Expert B is a local farmer who looks at the clouds and smells the air.
Expert C is a computer program that analyzes satellite data.

If you ask just Expert A, they might be right 80% of the time. If you ask Expert B, maybe 75%. But what if you could combine their opinions? The Super Learner is a method that asks all these experts for their predictions, tests them against past data, and then figures out the perfect recipe for combining their voices.

Sometimes the committee decides, "We should listen 90% to the farmer and 10% to the physicist." Other times, they might say, "The computer program is the star today, so we'll listen to it 100%." The goal is to create a "Super Prediction" that is at least as good as the single best expert in the room, and often much better.

The Three Different Ways to Run the Committee

The paper explains that for time-to-event data (like survival), you can run this committee in three different ways, depending on how you slice the time.

1. The "Stopwatch" Method (Discrete-Time)

The Analogy: Imagine you are watching a race, but you only check the runners every 10 minutes. You ask, "Did anyone cross the finish line in the last 10 minutes?"
How it works: This method chops time into chunks (like 10-minute intervals). It turns the complex problem of "when exactly did they die?" into a simple "Yes/No" question for each chunk.
Pros: It's very flexible. You can use any tool that answers "Yes/No" questions (like simple logic or basic machine learning).
Cons: It's a bit clumsy. If someone dies at minute 11, but you only check at 10 and 20, you lose some precision. It's like trying to measure a liquid with a bucket instead of a graduated cylinder.

2. The "Smooth River" Method (Westling et al.)

The Analogy: Instead of checking every 10 minutes, you watch the river flow continuously. You can see the exact second a leaf hits the waterfall.
How it works: This method doesn't chop time up. It treats time as a smooth, continuous flow. It uses a clever "iterative" process (like a loop) where the committee keeps refining its guess about the "drop-out rate" (censoring) and the "event rate" until they agree perfectly.
Pros: It's very precise and uses all the information available. It handles the "drop-outs" (people leaving the study) very smartly.
Cons: It's computationally heavy and requires more complex math to get the loop right.

3. The "State of Being" Method (Munch & Gerds)

The Analogy: Imagine a video game character. At any moment, the character is in one of three states: Alive and Well, Dead, or Lost/Disconnected.
How it works: This method focuses on the probability of being in each of these three "states" at any given time. It picks the single best combination of experts that minimizes the errors in predicting these states.
Pros: It's very robust and handles complex scenarios (like if a patient dies from a different cause, not just the one you are studying). It's a "non-ensemble" method, meaning it usually picks the single best expert rather than mixing them all, which can be simpler.
Cons: It focuses on "states" rather than just the survival curve, which is a slightly different mathematical angle.

How They Tested It (The "Rotterdam" Race)

The authors tested these three methods using real data from breast cancer patients in Rotterdam.

They split the data: 70% to train the committee, 30% to test it.
They compared the "Super Learner" committees against individual experts (like a standard Cox regression model or a Random Forest).
The Result: The Super Learners generally won. They were more accurate at predicting survival than any single model.
- The "Smooth River" and "State of Being" methods (Continuous-Time) performed slightly better than the "Stopwatch" method (Discrete-Time) because they didn't lose information by chopping time up.
- The "Committee" approach (Ensemble) was very close to the best single expert, but it had the safety net of never doing worse than the best expert.

Why Should You Care?

In the past, if you wanted to build a prediction model, you had to guess which math formula to use. If you guessed wrong, your predictions were bad.

This paper gives you a toolkit to stop guessing. It says: "Don't pick one model. Build a committee of many different models, let them compete, and let the data decide the winning combination."

It's like having a magic compass that automatically finds the best path through a forest of different prediction models, ensuring you don't get lost in the woods of bad statistics. Whether you are a doctor, a data scientist, or just someone interested in how AI makes decisions, the Super Learner is a powerful way to ensure your predictions are as accurate as possible.

1. Problem Statement

The estimation of conditional risks or survival probabilities based on censored time-to-event data is a fundamental task in medical statistics, used for both prediction modeling and causal inference (e.g., estimating treatment effects). A primary challenge in this domain is the inability to know a priori which candidate model (parametric, semi-parametric, or machine learning) will provide the best predictive performance.

Traditional approaches often involve selecting a single model (e.g., Cox proportional hazards) or comparing models ad hoc. The Super Learner (SL) offers a solution by creating an "ensemble" that combines a pre-specified library of candidate learners. The SL is guaranteed to perform asymptotically as well as the best single learner in the library (the "oracle property"). However, while SL methods for continuous or binary outcomes are well-established, the literature for time-to-event outcomes is fragmented, highly technical, and lacks accessible implementation guides. Existing tutorials do not cover survival data, and the specific adaptations for censoring and continuous time are scattered across different papers.

2. Methodology

The paper provides a comprehensive tutorial on three specific implementations of the Super Learner for time-to-event data, outlining the general steps and specific variations for each.

General Super Learner Framework

The SL relies on V-fold cross-validation to estimate the predictive performance of candidate learners and derive an optimal ensemble. The general steps are:

Data Splitting: Divide data into $K$ folds.
Cross-Validation: Fit $p$ candidate learners on training sets ( $D_{-k}$ ) and generate predictions on validation sets ( $D_k$ ).
Loss Calculation: Calculate a loss function (measuring the difference between predicted and true values) for each learner, accounting for censoring.
Ensemble Derivation:
- Non-ensemble (Discrete) SL: Selects the single learner with the lowest cross-validated risk.
- Ensemble (Standard) SL: Fits a regression model (typically non-negative least squares) where the outcome is the observed event status and predictors are the cross-validated predictions from the $p$ learners. Coefficients (weights) are constrained to be non-negative and sum to 1.
Final Estimation: Refit the selected learners (or those with non-zero weights) on the full dataset and combine them using the derived weights.

Specific Implementations Covered

A. Discrete-Time Super Learner (Polley & van der Laan, 2011)

Concept: Discretizes follow-up time into intervals, converting the time-to-event problem into a series of binary outcome problems (did the event occur in interval $t$ ?).
Data Structure: Transforms data into a "long" format (counting process) with one row per person per time interval.
Candidate Learners: Can utilize any binary outcome model (e.g., logistic regression, random forests, GAMs).
Loss Functions:
- $L_{IPCW}$ : Inverse Probability of Censoring Weighted Brier score (targets survival probability directly).
- $L_{L2}$ : Squared error loss on the discrete hazard.
- $L_{loglik}$ : Log-likelihood loss.
Implementation: Uses the SuperLearner R package. Requires manual data discretization.

B. Continuous-Time Super Learner: Westling et al. (2023)

Concept: Operates in continuous time without discretizing the outcome. Targets the entire conditional survival distribution $S(t|X)$ .
Mechanism: Uses a fine time grid to approximate integrals. It employs an iterative approach to simultaneously estimate the survival distribution $S(t|X)$ and the censoring distribution $G(t|X)$ using Super Learners for both.
Loss Function: A weighted integrated Brier score over the interval $(0, \tau)$ .
Ensemble: Derives weights by regressing the "pseudo-outcome" (constructed from the observed data and censoring estimates) against the cross-validated survival predictions from candidate learners.
Implementation: Uses the survSuperLearner R package (GitHub).

C. Joint Survival Super Learner: Munch & Gerds (2025)

Concept: A non-ensemble approach designed for settings that may include competing risks (though applied here to single-event data). It models state occupation probabilities (uncensored/no event, event occurred, censored).
Mechanism: Specifies candidate learners for the cumulative hazards of the event ( $\Lambda$ ) and censoring ( $\Gamma$ ).
Loss Function: An integrated Brier score calculated over all three possible states at each time point.
Ensemble: Unlike the other two, this is a non-ensemble method. It selects the single best pair of candidate learners (one for event, one for censoring) that minimizes the cross-validated loss.
Implementation: Uses the jossl R package (GitHub).

3. Key Contributions

Accessible Tutorial: Bridges the gap between highly technical theoretical papers and practical application, providing a unified view of discrete and continuous-time SL methods.
Implementation Guidance: Details how to implement these methods in R, referencing specific packages (SuperLearner, survSuperLearner, jossl) and providing "by-hand" code to clarify the underlying mechanics.
Comparative Analysis: Systematically compares the three approaches regarding:
- Handling of censoring (pre-specified vs. estimated via SL).
- Discretization requirements (Discrete-time requires it; Continuous-time does not).
- Oracle properties and theoretical guarantees.
- Flexibility in candidate learner selection.
Reproducible Code: Provides a GitHub repository with code to reproduce all analyses and examples.

4. Results (Application to Rotterdam Breast Cancer Data)

The authors applied the methods to the "Rotterdam" dataset (breast cancer recurrence/death) to predict 10-year survival probabilities.

Performance Metrics: Evaluated using Brier score, Scaled Brier score, C-index, and dynamic AUC.
Individual Learners: The Random Survival Forest (RSF) outperformed standard Cox regression and Cox-Lasso.
Discrete-Time SL:
- Performance was sensitive to the level of discretization. Using 100 time periods resulted in worse overall performance (Brier score) compared to 10 periods, likely due to poor baseline hazard modeling with high-dimensional time indicators.
- Discrimination (C-index) was similar across discretization levels.
Continuous-Time SLs:
- Both the Westling et al. (ensemble) and Munch & Gerds (non-ensemble) methods outperformed the discrete-time SLs in terms of overall predictive accuracy (Brier score).
- The Westling et al. ensemble SL achieved performance nearly identical to the best individual learner (RSF), effectively selecting RSF as the dominant component.
- The Munch & Gerds non-ensemble SL selected the RSF as the best pair, yielding results identical to the standalone RSF.
Ensemble Weights: In the continuous-time ensemble, the RSF received the highest weight, with some contribution from Cox models with Lasso or GAMs. The discrete-time ensemble weights varied significantly based on the loss function and discretization level.

5. Significance and Discussion

Practical Utility: The paper demonstrates that Super Learners are viable and effective tools for survival analysis, often matching or exceeding the performance of the best individual machine learning model without requiring the analyst to guess the correct model structure.
Continuous vs. Discrete: While discrete-time SLs offer flexibility in using binary outcome learners, continuous-time methods are generally preferred for survival data as they avoid information loss from discretization and allow for flexible estimation of the censoring distribution.
Causal Inference: The authors highlight that these methods are not just for prediction but are critical components in causal inference frameworks (e.g., Targeted Maximum Likelihood Estimation - TMLE) where nuisance parameters (survival and censoring distributions) must be estimated flexibly.
Future Directions: The paper notes limitations, such as the difficulty of handling left-truncation in some continuous-time methods and the potential for negative risk estimates in pseudo-observation approaches. It suggests that the choice of method should depend on the specific data structure (e.g., presence of competing risks) and computational constraints.

In conclusion, this tutorial demystifies the application of Super Learners for time-to-event data, providing researchers with the theoretical understanding and practical tools to implement robust, oracle-optimal prediction models in the presence of censoring.