🤖 machine learning

SurF: A Generative Model for Multivariate Irregular Time Series Forecasting

SurF is a novel generative model for multivariate irregular time series that leverages the Time Rescaling Theorem to map event sequences to i.i.d. noise, enabling efficient, scalable, and state-of-the-art forecasting across diverse real-world datasets through multi-dataset pretraining.

Original authors: Mohammad R. Rezaei, Tejas Balaji, Rahul G. Krishnan

Published 2026-05-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Mohammad R. Rezaei, Tejas Balaji, Rahul G. Krishnan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict when the next bus will arrive. In a perfect world, buses come every 10 minutes on the dot. But in the real world, buses are messy: sometimes one comes in 2 minutes, sometimes in 45, and sometimes three come at once. This is what data scientists call an "irregular time series."

Most current AI models are like rigid train schedules; they expect data to arrive at fixed, regular intervals. If you feed them messy, irregular data, they get confused. The paper introduces a new model called SurF (Survival Flow) that is specifically designed to understand this chaos.

Here is how SurF works, explained through simple analogies:

1. The Problem: The "Rubber Band" of Time

Imagine you have a rubber band with events (like earthquakes, tweets, or taxi rides) marked on it.

Old Models: They try to stretch this rubber band to fit a standard ruler. If the gaps between events vary wildly (from seconds to days), the ruler breaks. They try to force the data into a grid, which destroys the unique patterns of the timing.
The SurF Insight: Instead of forcing the data to fit a ruler, SurF stretches the rubber band itself. It uses a mathematical trick called the Time Rescaling Theorem. Think of this as a magical machine that takes your messy, irregular timeline and stretches or compresses it until the gaps between events look perfectly normal and uniform.

2. The Magic Trick: Turning Chaos into "White Noise"

SurF has a special superpower: it can translate any messy event stream into a simple, universal language: random noise.

The Forward Trip (Training): SurF takes a sequence of events (like a string of earthquakes) and runs it through its "stretching machine." It transforms the time gaps until they look like i.i.d. Exp(1).
- In plain English: This is a fancy way of saying it turns the complex timing data into a bag of perfectly random, standard numbers (like rolling a die that always lands on a specific distribution).
- Because every dataset (whether it's earthquakes, stock trades, or tweets) gets transformed into this same bag of random numbers, SurF can learn from all of them at once. It's like learning to speak English by translating every language in the world into the same simple alphabet.
The Reverse Trip (Prediction): This is the genius part. Because the machine is a bijection (a perfect two-way door), SurF can also go backward.
1. It generates a piece of random noise (a fake "Exp(1)" number).
2. It runs that noise through the machine in reverse.
3. The machine un-stretches the rubber band, turning that simple random number back into a realistic prediction of when the next event will happen.

3. Three Different "Stretching Machines"

The paper offers three different ways to build this stretching machine, depending on how complex the data is:

MoE (Mixture of Exponentials): Like a simple set of springs. It's fast and great for data that fades away quickly (like a fading echo).
CSB (Cumulative Softplus Basis): Like a flexible, bendable ruler. It can handle data that has bumps and humps (like a sudden burst of tweets).
GLQ (Gauss-Legendre Quadrature): Like a high-precision laser cutter. It's the most flexible and handles the weirdest, most irregular patterns, but it requires a tiny bit more calculation.

4. The "Foundation Model" Achievement

Usually, to predict earthquakes, you train a model only on earthquake data. To predict tweets, you train a different model on tweet data. They don't talk to each other.

SurF is different. The authors trained one single model on six completely different datasets (Earthquakes, Taxi rides, Amazon reviews, Retweets, StackOverflow questions, and Taobao clicks).

The Result: This single model became a "Foundation Model" for irregular events. When they tested it on a dataset it had never seen before (using a strict "leave-one-out" test), it beat every other specialized model on 5 out of 6 datasets.
Why? Because by learning to translate all these different streams into the same "random noise" language, it learned the universal grammar of how events happen, rather than just memorizing one specific topic.

Summary

Think of SurF as a universal translator for time.

Old AI: "I only understand regular clocks. If your data is messy, I can't help."
SurF: "Give me your messy timeline. I will stretch it into a simple, standard pattern, learn the rules, and then un-stretch it to tell you exactly when the next event will happen."

It doesn't just predict the future; it understands the rhythm of irregular life, from the tremors of the earth to the click of a mouse, all with one brain.

Technical Summary: SurF – A Generative Model for Multivariate Irregular Time Series Forecasting

Problem Statement

Irregularly sampled multivariate event streams (Temporal Point Processes, or TPPs) present a significant challenge for generative modeling. Unlike regularly sampled time series, where data arrives at fixed intervals, event streams feature asynchronous arrivals with inter-event intervals that can vary by orders of magnitude. Existing approaches face two primary bottlenecks:

Tokenization-based models (e.g., TimesFM, Chronos) fail when inter-event intervals vary drastically, as they rely on fixed-rate assumptions.
Neural Temporal Point Processes (TPPs) often rely on maximum likelihood estimation requiring the numerical integration of the intensity function $\lambda(t)$ over the observation window $[0, T]$ . This "window-level numerical quadrature" creates a computational bottleneck that scales with the window size and accumulates error during gradient steps. Furthermore, prior neural TPPs are typically trained per-dataset, lacking the ability to generalize across heterogeneous event streams.

Methodology: SurF (Survival Flow)

SurF reframes the Time Rescaling Theorem (TRT) not merely as an evaluation tool, but as a learnable, bidirectional bijection between event sequences and i.i.d. unit-rate exponential noise.

Core Theoretical Insight

The TRT states that if a cumulative intensity function $\Lambda^*(t) = \int_0^t \lambda^*(s|H_s) ds$ is correct, the rescaled inter-arrival times $\Delta z_i = \Lambda^*(t_i) - \Lambda^*(t_{i-1})$ are i.i.d. $\text{Exp}(1)$ .
SurF introduces Theorem 2, establishing that under mild conditions (continuous intensity with a strictly positive lower bound $\lambda_{min} > 0$ ), $\Lambda^*$ is a $C^1$ bijection with a smooth inverse. This allows the model to:

Forward (Training): Map observed event times to $\text{Exp}(1)$ noise.
Reverse (Sampling): Map $\text{Exp}(1)$ noise back to event times by inverting the learned cumulative intensity.

This bidirectional flow eliminates the need for window-level integration. Instead of modeling $\lambda$ and integrating it, SurF models the cumulative intensity $\Lambda_\theta$ directly and recovers the instantaneous intensity $\lambda_\theta$ via automatic differentiation (or closed-form derivatives).

Architecture and Parameterizations

SurF employs a Transformer-based encoder to process event history, producing a context vector $h_{i-1}$ . The core innovation lies in three distinct parameterizations of the cumulative intensity $\Lambda_\theta(\Delta t | h)$ , each offering different trade-offs between expressiveness and computational cost:

SurF-MoE (Mixture of Exponentials):
- Models intensity as a sum of decaying exponentials.
- Pros: Fully closed-form likelihood; zero numerical quadrature; fastest training.
- Cons: Restricted to monotonically decaying intensities between events.
SurF-CSB (Cumulative Softplus Basis):
- Uses a sum of shifted softplus primitives.
- Pros: Fully closed-form; capable of modeling non-monotonic (e.g., hump-shaped) intensity profiles; universal approximator for positive intensities.
- Cons: Slightly more complex than MoE but still exact.
SurF-GLQ (Gauss-Legendre Quadrature):
- Uses an unconstrained positive MLP for intensity, with $\Lambda_\theta$ approximated via fixed $Q$ -point Gauss-Legendre quadrature per inter-event interval (not over the full window).
- Pros: Highest expressiveness (unconstrained intensity); computational cost is $O(NQ)$ where $Q$ is fixed (e.g., $Q=8$ ) and independent of the sequence length $T$ .
- Key Finding: Empirical error at $Q=8$ is below the gradient noise floor, meaning the quadrature error does not affect training gradients.

Training and Sampling

Loss Function: The objective is the amortized negative log-likelihood derived from the change of variables formula:
$L_{SurF} = \sum_{i=1}^N \Lambda_\theta(\tau_i | h_{i-1}) + \Lambda_\theta(\Delta T | h_N) - \sum_{i=1}^N \log \lambda_\theta(\tau_i | h_{i-1})$
This avoids the $O(T)$ integration cost of prior methods.
Sampling: Events are generated by sampling $z \sim \text{Exp}(1)$ and solving $\Lambda_\theta(\Delta t | h) = z$ for $\Delta t$ using a safeguarded Newton method, which converges globally and quadratically.
Multi-Dataset Pretraining: By mapping all datasets to the same canonical target ( $\text{Exp}(1)$ ), SurF enables a single shared encoder ( $\phi_\theta$ ) and cumulative intensity head ( $\Lambda_\theta$ ) to be trained jointly across heterogeneous datasets. This facilitates zero-shot transfer to unseen datasets.

Key Contributions

Bidirectional TRT as a Flow: The paper explicitly formulates the TRT as a learnable bijection, providing the theoretical conditions (Theorem 2) for invertibility and smoothness, enabling exact likelihood computation and generative sampling.
Efficient Parameterizations: Introduction of three monotone parameterizations (MoE, CSB, GLQ) that scale to long sequences. Notably, SurF-GLQ achieves $T$ -independent computational cost, resolving the numerical quadrature bottleneck of previous neural TPPs.
Cross-Dataset Foundation Model: Demonstration of a single model trained on multiple heterogeneous datasets that achieves state-of-the-art or competitive performance on held-out datasets without fine-tuning, representing an initial step toward foundation models for asynchronous event streams.

Experimental Results

SurF was evaluated on six real-world benchmarks: Taobao, Taxi, Retweet, StackOverflow, Amazon, and Earthquake.

Next-Event Prediction:
- SurF achieved the best reported time RMSE on Earthquake, Retweet, and Taobao.
- On the remaining three datasets, it performed within "trial-level noise" of the strongest specialist baselines.
- Under a strict leave-one-out zero-shot protocol (training on 5 datasets, testing on the 6th), the held-out SurF checkpoint beat every classical and neural-autoregressive baseline on 5/6 datasets (losing only on StackOverflow). It outperformed all baselines on Amazon and Earthquake without any dataset-specific adaptation.
Multi-Horizon Forecasting: SurF demonstrated stability under iterative autoregressive decoding, with RMSE growth being sub-linear and type accuracy remaining stable over 10 future events.
Calibration: Time-rescaling residuals were well-calibrated, with Kolmogorov-Smirnov statistics indicating that the model's predicted distributions closely matched the theoretical $\text{Exp}(1)$ target.
Efficiency: SurF-MoE and SurF-CSB require no numerical integration at training time. SurF-GLQ uses a fixed-cost per-interval rule. Compared to quadrature-based baselines (e.g., NJDTPP), SurF-MoE achieved a $\sim9\times$ training speed-up and $\sim17\times$ inference speed-up.

Significance and Claims

The paper positions SurF as a foundational step toward foundation models for asynchronous event streams. Its primary significance lies in:

Unifying Evaluation and Generation: By treating the TRT as a generative flow, it unifies the evaluation of goodness-of-fit with the mechanism for sampling.
Scalability: It removes the computational barrier of window-level integration, allowing models to handle long sequences efficiently.
Generalization: It provides the first empirical evidence that a learned cumulative intensity can transfer zero-shot across diverse domains (e.g., from social media to seismic data), suggesting that the canonical $\text{Exp}(1)$ space captures universal temporal dynamics independent of the specific dataset's scale or frequency.

The authors acknowledge limitations, such as the requirement for a strictly positive intensity floor (which introduces a negligible bias) and the current corpus size being smaller than language/vision foundation models, identifying scaling as the next key milestone.