Technical Summary: Accelerating Time Series Foundation Models with Speculative Decoding

Problem Statement

Time series foundation models (TSFMs) have emerged as state-of-the-art solutions for operational forecasting tasks (e.g., electricity load, traffic control) due to their ability to forecast unseen series zero-shot with high accuracy. However, their deployment is hindered by inference latency. TSFMs typically operate via patch-autoregressive decoding, where a model generates a sequence of $P$-step patches one by one. To forecast a horizon of $H$ steps, the model must perform $\lceil H/P \rceil$ strictly sequential forward passes. This sequential dependency creates a bottleneck where latency scales linearly with the forecast horizon, contradicting the need for low-latency operational decisions.

While speculative decoding (SD) has successfully accelerated Large Language Models (LLMs) by using a cheap "draft" model to propose tokens that a larger "target" model verifies in parallel, direct transfer to time series is non-trivial. Standard SD relies on discrete vocabularies and token probabilities. TSFMs, however, output continuous patches in $\mathbb{R}^d$, often with heterogeneous heads (point, quantile, flow-matching) that do not expose explicit probability densities. Furthermore, standard SD correction mechanisms (resampling from a residual distribution) are computationally prohibitive in continuous spaces, especially when the draft and target agree frequently.

Methodology

The authors propose a Gaussian Speculative Decoding framework tailored for continuous patch autoregression. The method replaces the discrete ingredients of SD with continuous analogues without modifying the target model's weights or requiring retraining.

1. Gaussian Surrogate Heads

To handle heterogeneous TSFM heads, the framework equips both the target ($p$) and draft ($q$) models with isotropic Gaussian surrogates for their next-patch conditionals:
$$p(x | h) = \mathcal{N}(\mu_p(h), \sigma^2 I_d), \quad q(x | h) = \mathcal{N}(\mu_q(h), \sigma^2 I_d)$$
Here, $\mu_p$ and $\mu_q$ are the point predictions (raw outputs, medians, or flow-matching means) of the respective models. The variance $\sigma^2$ is a fixed, supplied "acceptance temperature," not a learned model output.

2. Acceptance Rule

Instead of comparing token probabilities, the method uses a log-domain Gaussian likelihood-ratio test. For a draft proposal $x_i$ at position $i$, the acceptance statistic $\ell_i$ is defined as:
$$\ell_i = -\frac{\|x_i - \mu_p^{(i)}\|_d^2}{2\sigma^2} + \frac{\|x_i - \mu_q^{(i)}\|_d^2}{2\sigma^2} + \frac{1}{2}\log\frac{\sigma_q^2}{\sigma_p^2}$$
where $\|u\|_d^2 = \frac{1}{d}\|u\|_2^2$ is the per-dimension energy. The proposal is accepted with probability $\alpha_i = \min\{1, e^{\ell_i}\}$.

• Tempering: The $1/d$ scaling prevents the log-ratio from degenerating into a hard 0/1 gate as dimensionality increases, allowing $\sigma$ to act as a tunable acceptance temperature across different model families.
• Mean Propagation: The draft proposes its mean ($x_i = \mu_q^{(i)}$). This avoids injecting sampling noise, which is detrimental under squared-error evaluation.

3. Correction and Fallback

Unlike discrete SD, which resamples from a residual distribution upon rejection (a process that diverges in cost when acceptance is high), this method commits the target's own prediction ($\mu_p^{(i)}$) immediately upon rejection.

• Theoretical Justification: The authors prove that this "fallback" strategy keeps the committed patch within a bounded squared-error radius of the target-only forecast. Resampling is avoided because the cost of exact correction in high-dimensional continuous spaces outweighs the benefits of speculation in the high-acceptance regime.
• Bonus Patch: If all $K$ proposals are accepted, the target's $(K+1)$-th prediction (generated in the same verification pass) is also committed, providing a "free" patch.

4. Single-Pass Verification

The target model performs a single causal forward pass over the concatenated history and the $K$ draft patches $[h, x_1, \dots, x_K]$. This single pass yields predictions for all $K+1$ prefix boundaries, verifying the entire block in parallel.

Key Contributions

1. Framework for Continuous Patches: The paper introduces the first speculative decoding framework operating natively on continuous patch densities, removing the sequential bottleneck of TSFM decoding without modifying the target model.
2. Theoretical Guarantees:
   • Throughput Law: Derives a capped-geometric law for throughput, showing that speedup depends on acceptance rates and the draft-to-target cost ratio. It provides a computable criterion for selecting the optimal block size $K$.
   • Fidelity Bound: Proves that the accelerated output stays within a squared-error radius of the target-only decoding, determined by the acceptance temperature $\sigma$.
   • Predictability: Demonstrates that speedups and failure conditions (e.g., when the draft is too expensive or acceptance is too low) can be predicted before deployment using held-out data.
3. Empirical Validation: Instantiates the framework across five diverse TSFM families (Timer-XL, TimesFM 2.5, Sundial, Time-MoE, TiRex) using drafts that require no specific pretraining (except for a distilled variant of Timer-XL).

Experimental Results

The method was evaluated on standard long-horizon forecasting benchmarks (ETT, Weather, ECL, Traffic, Solar, PEMS).

• Speedup: The framework achieves up to 3.0× inference speedup across the five model families.
• Accuracy: Accelerated forecasts maintain accuracy between the draft and the target. In several cases (e.g., Time-MoE on ETTm1), the speculative decoding was more accurate than the target-only baseline, attributed to reduced error accumulation via periodic re-anchoring to the target's single-step prediction.
• Operating Points: A monotone frontier exists between accuracy and speed controlled by $\sigma$. High acceptance rates (up to 97%) were observed for models with large target-to-draft cost ratios (e.g., Time-MoE).
• Failure Analysis: The paper identifies predictable failure modes:
  • Accuracy Gate: If the draft matches the target's accuracy, speculation offers no benefit.
  • Cost Structure: If the draft is too expensive relative to the target, or if verification cost scales poorly with block size (as seen in TiRex on certain datasets), speedup is lost.

Significance and Claims

The paper claims that speculative decoding is no longer restricted to discrete vocabularies. By rebuilding the acceptance, correction, and verification mechanisms for continuous outputs, the authors transform the horizon-long sequential bottleneck of TSFMs into a parallelism opportunity.

• Training-Free: The approach is largely training-free, reusing existing checkpoints for the target and draft.
• Composable: It composes with kernel-level optimizations (e.g., FlashAttention) and does not require architectural changes to the foundation models.
• Predictable: The method provides a theoretical basis to predict whether speculation will pay off for a given model pair and dataset before deployment, avoiding trial-and-error in production.

The work establishes that for time series forecasting, where outputs are continuous and evaluation is typically based on squared error, a mean-propagating, bounded-deviation approach is superior to exact residual resampling, offering a practical path to accelerating foundation models for real-time operational decision-making.