📊 statistics

Latent Laplace Diffusion for Irregular Multivariate Time Series

This paper introduces Latent Laplace Diffusion (LLapDiff), a generative framework that addresses the challenges of irregular multivariate time series forecasting by modeling target trajectories in a low-dimensional latent space using Laplace-domain parameterization and stochastic port-Hamiltonian dynamics to enable stable, horizon-wide generation without step-by-step integration.

Original authors: Zinuo You, Jin Zheng, John Cartlidge

Published 2026-06-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zinuo You, Jin Zheng, John Cartlidge

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict the future path of a flock of birds, or the fluctuating temperature in a city, or the stock prices of several companies. In the real world, these things don't happen on a perfect, neat schedule. Sometimes you get a measurement every hour, sometimes every 17 minutes, and sometimes you miss a whole day because a sensor broke.

This is what the paper calls Irregular Multivariate Time Series. It's messy data where the "when" and the "what" are both unpredictable.

The authors of this paper, Zinuo You and colleagues, built a new AI tool called LLapDiff (Latent Laplace Diffusion) to handle this mess. Here is how it works, explained through simple analogies.

The Problem: The "Grid" Trap vs. The "Drift" Trap

Most current AI methods try to fix messy data in one of two ways, and both have problems:

The "Re-gridding" Trap: Imagine trying to fit a jagged, uneven mountain range onto a flat, square grid. You have to force the data into neat boxes (interpolation). If you do this too aggressively, you distort the shape of the mountain. You lose the true timing of events, and the AI gets confused about when things actually happened.
The "Drift" Trap: Other methods try to follow the data continuously, like a hiker walking step-by-step through the forest. But if the forest is full of gaps (missing data), the hiker has to guess where to step next. Over a long journey (a long forecast), these small guesses add up, and the hiker eventually wanders off the path entirely. This is called "drift."

The Solution: LLapDiff

LLapDiff takes a different approach. Instead of trying to walk step-by-step or force the data into a grid, it treats the future as a smooth, invisible melody that needs to be composed.

1. The "Latent Trajectory" (The Invisible Melody)

Instead of predicting every single messy data point directly, LLapDiff first compresses the data into a "low-dimensional latent trajectory."

Analogy: Think of a complex song played by an orchestra. Instead of trying to write down every note for every instrument at every millisecond, the AI learns the "sheet music" or the core melody. It creates a smooth, invisible path that represents the true underlying trend of the data, ignoring the noise and gaps.

2. The "Laplace" Secret (The Stable Blueprint)

This is the paper's biggest innovation. To make sure the AI doesn't "drift" off course, it uses a mathematical concept called the Laplace domain and Stochastic Port-Hamiltonian dynamics.

Analogy: Imagine you are building a bridge. Instead of calculating the stress on every single bolt as you build it (which is slow and prone to error), you use a blueprint that guarantees the bridge is stable by design.
The paper uses "poles" (mathematical anchors) to describe how the data moves. By forcing these poles to be "stable" (like a pendulum that naturally slows down and stops rather than swinging wildly forever), the AI ensures the generated future stays realistic and doesn't explode into nonsense, even over long periods.

3. The "Gap-Aware" Memory (Listening to the Silence)

The AI knows that the gaps in the data are important. If a sensor stops working for a long time, that silence tells a story.

Analogy: Imagine a detective listening to a conversation where someone keeps pausing. The detective doesn't just ignore the silence; they realize the length of the pause changes the meaning of the next sentence.
LLapDiff has a "History Summarizer" that looks at the gaps between data points. It uses a theory called "Renewal Averaging" to understand how these random gaps change the rhythm of the data. It adjusts its internal "melody" to match the specific rhythm of the missing pieces.

4. The "Diffusion" Process (Denoising the Future)

LLapDiff uses a technique called "Diffusion."

Analogy: Imagine a blurry, noisy photo of a landscape. The AI starts with pure static (white noise) and slowly, step-by-step, removes the noise to reveal the clear image underneath.
Unlike other methods that have to calculate the next second, then the next, then the next (which is slow and error-prone), LLapDiff predicts the entire melody at once based on the "sheet music" (the stable poles) it learned. It generates the whole future path in one go, rather than taking tiny, risky steps.

What Can It Do?

According to the paper, this method is excellent at two things:

Long-Horizon Forecasting: It can predict far into the future without losing its way or drifting off, even when the data is very messy.
Filling in the Blanks (Imputation): Because it learns the "melody" of the data, if you ask it, "What was the value at 2:00 PM?" (even if the sensor was broken then), it can play back that part of the melody to give you a very accurate answer. It doesn't need to retrain; it just queries the same model at a different time.

The Bottom Line

The authors tested LLapDiff on real-world data like weather stations, stock markets, and hospital patient monitors. They found that it outperformed existing methods, especially when the data was very irregular or when predicting far into the future. It does this by respecting the physics of time (stability) and the reality of missing data (gaps), rather than trying to force messy reality into a perfect, artificial grid.

Technical Summary: Latent Laplace Diffusion for Irregular Multivariate Time Series

1. Problem Statement

Irregular multivariate time series (IMTS), common in healthcare, climate science, and finance, present a fundamental trade-off for long-horizon forecasting. Existing approaches generally fall into three categories, each with significant limitations:

Discrete-time pipelines: These methods interpolate or re-grid data to a regular structure. While efficient, aggressive re-gridding distorts temporal information and blurs the semantics of missingness under severe irregularity.
Continuous-time models (e.g., Neural ODEs/CDEs): These incorporate timestamps naturally but rely on sequential numerical integration. This increases computational cost and leads to error accumulation (drift) over long horizons, particularly when gaps are large.
Diffusion/Score-based models: While offering strong uncertainty quantification, most existing time-series diffusion methods denoise directly in the observation space. They often lack explicit dynamical structure or stability control, making long-horizon generation fragile under irregular sampling.

The core challenge is to model IMTS in a way that respects physical time and sampling gaps without reducing irregularity to a preprocessing artifact, while ensuring stable, long-horizon trajectory generation without the drift associated with sequential solvers.

2. Methodology: Latent Laplace Diffusion (LLapDiff)

LLapDiff is a conditional generative framework that bridges the gap between discrete and continuous modeling. It avoids sequential numerical integration over physical time by modeling the target as a low-dimensional latent trajectory and performing diffusion in the latent space.

2.1. Latent Trajectory Representation

The method represents irregular target horizons as low-dimensional latent trajectories $z$ on a manifold. A pretrained Variational Autoencoder (VAE) encodes ground-truth targets into this latent space. The diffusion process corrupts and denoises these latent trajectories rather than the raw, sparse observations, avoiding direct denoising over masked data.

2.2. Stability Bias via Stochastic Port-Hamiltonian Dynamics

To guide the reverse diffusion process toward stable dynamics without numerical solvers, the authors draw inspiration from stochastic port-Hamiltonian systems.

Energy Balance: The framework imposes a passivity-based inductive bias where the expected system energy is non-increasing between context updates. This isolates forecastable dynamics from stochastic volatility.
Laplace-Domain Parameterization: Instead of time-stepping, the mean evolution of the latent trajectory is parameterized in the Laplace domain. The system is modeled using stable complex-conjugate poles $(\rho_k, \omega_k)$ , where $\rho_k$ represents decay rates and $\omega_k$ represents oscillation frequencies.
Stable Modal Synthesis: The denoiser predicts these modal parameters (poles and residues) conditioned on the history. The latent trajectory is then synthesized via a closed-form sum of damped sinusoids:
$\hat{z}_0(\tilde{t}_r) = \sum_{k=1}^K e^{-\hat{\rho}_k \tilde{t}_r} \left( \hat{c}_k \cos(\hat{\omega}_k \tilde{t}_r) + \hat{b}_k \sin(\hat{\omega}_k \tilde{t}_r) \right)$
This allows for horizon-wide generation in a single parallelizable pass, bypassing the need for sequential integration.

2.3. Renewal-Averaging and Gap-Aware Conditioning

The authors establish a theoretical link between continuous-time poles and random sampling gaps using renewal-averaging analysis.

Effective Poles: Random gaps map continuous-time poles to effective event-domain poles. The variance of gaps modulates the effective damping and oscillation.
History Summarizer: To adapt learned dynamics to irregular patterns, a "gap-aware history summarizer" aggregates observations, time gaps, and missingness masks. This summarizer conditions the modal predictor, allowing the model to learn continuous-time parameters that remain consistent even under complex, non-renewal sampling patterns.

2.4. Architecture

The model consists of:

History Summarizer ( $S_\phi$ ): Encodes observed history into a token sequence containing port signals, dynamics proxies, and temporal/gap encodings.
Modal Predictor ( $L_\theta$ ): Predicts continuous-time modal parameters (poles and residues) conditioned on the noisy latent state and history summary.
Modal Synthesizer ( $L^+_\theta$ ): Reconstructs the denoised latent trajectory using the predicted parameters via the closed-form Laplace-domain equation.

3. Key Contributions

LLapDiff Framework: A latent diffusion framework for IMTS that models evolution as low-dimensional latent trajectories while preserving timestamp fidelity, avoiding the drift of sequential solvers.
Laplace-Domain Modal Parameterization: A derivation grounded in stochastic port-Hamiltonian dynamics that enables horizon-wide generation with explicit stability control through constrained pole damping ( $\rho_k > 0$ ).
Theoretical Link to Sampling Gaps: An analysis linking continuous-time poles to random sampling gaps via renewal averaging, motivating a gap-aware history conditioning mechanism that improves long-horizon modeling under varying irregularities.

4. Experimental Results

The authors evaluated LLapDiff on seven real-world datasets spanning air quality, healthcare (PhysioNet), meteorology (NOAA), and finance (Crypto, US Equity).

Forecasting Performance: LLapDiff achieved the best distributional metrics (CRPS) and competitive point metrics (MAE/MSE) across most datasets. Gains were most pronounced in long-horizon forecasting (e.g., 168 steps) and on highly irregular datasets (e.g., PhysioNet, NOAA UK).
Imputation: Due to its trajectory-generation nature, the model supports missing-value imputation by querying the same model at historical timestamps. It outperformed baselines like CSDI in probabilistic imputation tasks.
Stability and Robustness:
- Pole Visualization: Learned poles consistently satisfied $\rho_k > 0$ , ensuring exponential decay and stability. The model adapted decay rates based on gap statistics.
- Stress Tests: The model demonstrated graceful degradation under induced missingness. While performance dropped with extreme coverage loss, it remained robust compared to baselines that often drifted or oversmoothed.
- Regime Shifts: The model maintained stability under controlled spectral shifts (frequency and decay changes).
Efficiency: Unlike Neural ODEs or continuous-time Transformers that require sequential solver steps, LLapDiff's inference time remained nearly flat across different forecast horizons. It was significantly faster than TimeGrad (39.8x) and ContiFormer (150.6x) in inference wall-clock time.

5. Significance and Claims

The paper claims that LLapDiff successfully bridges the gap between discrete and continuous time-series modeling by integrating latent diffusion with a stable continuous-time inductive bias.

Solver-Free Generation: By moving the dynamics modeling to the Laplace domain and using stable modal parameterization, the method achieves horizon-wide generation without physical-time stepping, effectively solving the drift problem of sequential solvers.
Unified Forecasting and Imputation: The continuous latent trajectory view allows the same model to perform both forecasting and missing-value imputation simply by querying different timestamps, offering a unified approach to irregular data.
Gap-Aware Adaptation: The renewal-averaging analysis provides a theoretical basis for conditioning on sampling gaps, enabling the model to learn dynamics that are robust to irregular observation patterns.

The authors acknowledge limitations, including a reliance on pretrained latent representations and a locally linear modal approximation. They suggest future work could explore richer nonlinear dynamics and faster reverse sampling. The work is positioned as a step toward more stable and efficient generative modeling for real-world, irregular time series.