🤖 machine learning

RIDE: An Open Dataset and Benchmark for Train Delay Prediction

The paper introduces RIDE, a large-scale open dataset and standardized benchmark for train delay prediction on the Belgian railway network, which facilitates the first comprehensive comparative evaluation showing that graph neural networks outperform non-learning models while enabling detailed analysis across various forecasting regimes.

Original authors: Clément Elliker, Mathis Le Bail, Clément Mantoux, Jesse Read, Sonia Vanier

Published 2026-06-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Clément Elliker, Mathis Le Bail, Clément Mantoux, Jesse Read, Sonia Vanier

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the Belgian railway network as a massive, intricate game of "Follow the Leader," where thousands of trains are constantly moving, stopping, and waiting for each other. Sometimes, a single train gets stuck, and that delay ripples out like a wave, causing a domino effect that makes other trains late too.

The paper introduces RIDE (TRaIn DElay Prediction Dataset and Benchmark), which is essentially a giant, open-source "training gym" for computers to learn how to predict these delays before they happen.

Here is a breakdown of what the authors did, using simple analogies:

1. The Problem: Everyone Was Playing a Different Game

Before this paper, researchers trying to predict train delays were like chefs trying to compare recipes, but everyone was using different ingredients, different ovens, and different tasting spoons. Some looked at one train line, others looked at the whole country; some predicted delays for the next stop, others for the next hour. Because the rules were so different, no one could tell which computer program was actually the best chef.

2. The Solution: A Standardized "Training Camp"

The authors built RIDE, which acts like a standardized training camp with two main parts:

The Raw Ingredients (The Dataset): They gathered a massive amount of data from 2023 to 2025. Imagine a library containing:
- 94.5 million train movement records (every time a train arrived, left, or passed a station).
- 3.6 million complete train journeys.
- 35.7 million weather reports (because rain or snow can slow things down).
- They organized this into layers, like a sandwich:
  - Bronze: The raw, messy data straight from the source.
  - Silver: A cleaned-up, organized version where missing pieces are filled in and the map of the tracks is reconstructed.
  - Gold: The final, ready-to-eat meal specifically prepared for different types of computer models to eat.
The Rules of the Game (The Benchmark): They set strict rules so every computer model has to play the exact same game.
- The Goal: Look at the network at a specific moment (a "snapshot") and predict how late a train will be at its next 15 stops.
- The Test: They trained the models on data from 2023–2024 and tested them on data from 2025. This is like teaching a student last year's math problems and testing them on this year's exam to see if they can actually learn, not just memorize.

3. The Contest: Who Wins?

The authors invited different types of "students" (computer models) to compete:

The Old Schoolers (Non-Learning Models): These are simple, rule-based systems.
- The Translator: Just assumes the train will be as late as it is right now. (Like guessing you'll be late to work because you're already late).
- The Graph-Event: A slightly smarter rule-based system that simulates how trains interact on the tracks.
The Statisticians (Statistical Learning): Models like XGBoost, which are good at finding patterns in tables of numbers.
The Deep Learners (Deep Learning): Complex neural networks like MLP, LSTM (good at sequences), Transformer (good at attention), and GNN (Graph Neural Networks, which understand how the whole network connects).

4. The Results: The Winners

After running the competition, here is what happened:

Learning Beats Rules: The "Deep Learning" and "Statistical" students clearly outperformed the "Old Schoolers." The complex models learned patterns that simple rules missed.
The Champion: The Graph Neural Network (GNN) took the top spot. Think of this model as a student who understands not just the train, but the entire map of the railway and how every train affects every other train.
The Runners-Up: The Transformer and LSTM models were very close behind the GNN. They were almost as good, suggesting that different ways of looking at the data can lead to similar success.
The Surprise: Even the simplest "Translator" model (which just guesses the current delay will continue) was surprisingly hard to beat by a huge margin. It's a tough baseline.

5. The Fine Print: It Depends on the Situation

The paper also looked at when the models did well, not just the average score:

Short-term vs. Long-term: If you want to know what happens in the next 5 minutes, simple models do great. If you want to know what happens 40 minutes from now, the complex GNN model is the best.
Small vs. Big Delays: If a train is just slightly late, simple models handle it well. But if a train is getting really late and the delay is piling up (like a traffic jam), the models that understand how trains interact (like the GNN) are much better at predicting the chaos.

Summary

The paper says: "We built a giant, fair playground (RIDE) with clean data and strict rules. We let different computer models play. The ones that can understand the complex web of connections between trains (Graph Neural Networks) are currently the best at predicting delays, but the gap between the top models is small. This gives us a solid foundation to keep improving how we keep trains running on time."

Technical Summary: RIDE – An Open Dataset and Benchmark for Train Delay Prediction

Problem Statement
Train delay prediction is critical for railway operators and passengers to manage disruptions, improve information systems, and optimize traffic management. However, progress in this field has been hindered by a lack of standardization. Existing research relies on disparate datasets, varying prediction targets (e.g., next-station vs. fixed-time intervals), and inconsistent evaluation protocols. This fragmentation makes it difficult to compare models across studies or draw broad conclusions about the relative efficacy of different modeling approaches, particularly between non-learning, statistical, and deep learning methods.

Methodology: The RIDE Framework
To address these gaps, the authors introduce RIDE (TRaIn DElay Prediction Dataset and Benchmark), a comprehensive open resource built on the Belgian national railway network covering 2023–2025. The framework is structured as a layered data pipeline and a unified benchmark protocol.

Data Pipeline (Layered Release):
- Raw & Bronze: Ingests raw data from Infrabel (railway operations, infrastructure) and Open-Meteo (weather). The Bronze layer standardizes schemas and performs lightweight integrity checks.
- Silver (Intermediate Relational Dataset): A reusable, cleaned relational dataset containing 94.5 million train events, 3.6 million journeys, and 35.7 million weather records. It reconstructs the network topology, infers exact train paths, and aligns weather data to operational points. This layer allows users to construct custom datasets without repeating core cleaning steps.
- Gold (Benchmark Datasets): Model-ready artifacts derived from the Silver layer. It defines a snapshot as the fundamental unit: a timestamped view of the network containing active trains, their history, and future schedules. The Gold release offers two scales (Lite and Standard) and four specific dataset formats tailored to different model families: Tabular (for MLP/XGBoost/Transformer), Sequential (for LSTM), Graph (for GNN), and Graph-Event (for rule-based simulation).
Prediction Task:
- Target: For every active train in a snapshot, predict the delay (in seconds) at the next $n=15$ scheduled events (arrivals, departures, or passages).
- Horizon: This corresponds to an average prediction horizon of approximately 40 minutes.
- Reparameterization: Models predict the residual delay (future delay minus the last known delay) to improve performance, adding the last known delay back at inference.
Evaluation Protocol:
- Metrics: Primary metrics are Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE).
- Breakdowns: To analyze model behavior beyond aggregate scores, the protocol includes breakdowns by prediction horizon (time until the event) and delay-delta bins (change in delay relative to the last known state).
- Splits: Training data is drawn from 2023–2024, and testing from 2025, ensuring a realistic forward-in-time evaluation. A single, fixed test evaluation table ensures all models are evaluated on identical instances.
Models Evaluated:
The benchmark compares three families of approaches:
- Non-Learning: A Translation baseline (predicting the last known delay) and a Graph-event model (deterministic simulation using empirical travel times and precedence constraints).
- Statistical Learning: XGBoost (trained as independent regressors for each future event slot).
- Deep Learning: Multilayer Perceptron (MLP), Long Short-Term Memory (LSTM), Transformer, and Graph Neural Network (GNN).

Key Results
Experiments conducted on the Standard tier of the benchmark reveal the following:

Learning vs. Non-Learning: Learning-based methods clearly outperform non-learning models. The Graph-event model (88.41 MAE) significantly improves upon the Translation baseline (96.65 MAE) but remains behind all learning-based approaches.
Best Performer: The Graph Neural Network (GNN) achieved the best mean performance with an MAE of 73.62 and RMSE of 194.56.
Competitiveness of Architectures: The strongest learning-based models (GNN, Transformer, LSTM, XGBoost) remain relatively close in absolute performance. The Transformer (74.54 MAE) and LSTM (74.62 MAE) follow closely behind the GNN. XGBoost (76.58 MAE) outperformed the MLP (77.20 MAE).
Regime-Specific Strengths:
- Short Horizons (0–5 min): XGBoost performs best, likely due to its strength in handling small, stable delay changes.
- Medium Horizons (5–15 min): LSTM shows strong performance, suggesting sequential modeling captures the dynamics of trains catching up on delays.
- Long Horizons (>15 min) & Large Delays: GNN excels in longer horizons and scenarios with significant delay accumulation, indicating that explicitly modeling network interactions is crucial for capturing propagation effects.
- Delay Recovery: The Graph-event model performs best on large negative delay deltas (strong recovery), consistent with its tendency to predict faster-than-scheduled travel times.

Significance and Claims
The paper claims RIDE as the first resource to provide both a reusable intermediate dataset and a standardized benchmark for train delay prediction at a nationwide scale. Its significance lies in:

Standardization: Establishing a unified protocol for targets, temporal splits, and evaluation metrics, enabling direct comparison across diverse model families.
Comprehensive Baseline: Providing the first comparative evaluation of non-learning, statistical, and deep learning models on a shared, large-scale dataset.
Nuanced Analysis: Moving beyond aggregate accuracy to reveal that different architectures have distinct strengths depending on the forecasting regime (e.g., short-term vs. long-term, delay accumulation vs. recovery).
Future Directions: The authors suggest that while architectural changes alone do not yield dramatic gains among top performers, further progress may depend on better feature design and problem-specific modeling choices. They posit that RIDE serves as a foundation for future research into delay propagation and learning over large-scale transportation systems.