Stabilizing distribution-free probabilistic forecasts

Technical Summary: Stabilizing Distribution-Free Probabilistic Forecasts

Problem Statement
Multi-step-ahead probabilistic forecasts are frequently updated as new observations become available, a process that typically improves forecast quality due to shorter horizons. However, these updates introduce "forecast instability" (also termed forecast churn, inconsistency, or jumpiness), defined as the variability in forecasts for the same target period generated at different origins. This instability can trigger costly revisions to operational plans (e.g., inventory management, renewable energy scheduling) and erode trust in forecasting systems. While previous research has addressed stability in point forecasting or for Gaussian probabilistic forecasts (where distributional parameters allow for closed-form instability metrics), there is a gap in methods for distribution-free probabilistic forecasting. In non-parametric settings, closed-form expressions for distributional dissimilarity are unavailable, making it difficult to integrate stability directly into the optimization of global neural forecasting models without imposing restrictive distributional assumptions.

Methodology
The authors propose StableSQF, a method that integrates forecast stability directly into the training of distribution-free probabilistic time-series models. The approach consists of two main components:

1. Spline Quantile Function (SQF) Forecaster:

   • Instead of predicting fixed quantiles or parametric distributions, the model directly learns the full conditional quantile function $F^{-1}(\alpha)$ for multiple forecast horizons simultaneously (direct multi-horizon strategy).
   • The quantile function is modeled using linear isotonic regression splines. The spline parameters (intercepts and slopes) are outputs of a deep neural network (based on a doubly residual stacking architecture similar to N-BEATS).
   • This representation ensures the quantile function is monotonically increasing (preventing quantile crossing) and allows for the generation of arbitrary continuous output distributions without predefined distributional assumptions.
   • Training minimizes the average scaled Continuous Ranked Probability Score (CRPS).

2. StableSQF Optimization:

   • To induce stability, the authors augment the loss function with a penalty term based on the $p$-Wasserstein distance ($W_p$) between forecasts for the same target period generated at adjacent origins (e.g., $t$ and $t-1$).
   • Because the model outputs explicit functional forms (splines) rather than sampled distributions, the Wasserstein distance can be computed directly via discretization of the quantile functions.
   • The composite loss function is:
     $$\mathcal{L} = (1-\lambda) \cdot \text{Quality} + \lambda \cdot \text{Instability}$$
     where $\lambda \in [0, 1]$ controls the trade-off.
   • Targeted Stabilization: The method supports quantile-weighted Wasserstein distances. By applying a weight function $v(\alpha)$ to the integral, the penalty can be focused on specific parts of the distribution (e.g., the tails for inventory safety stock or the center for aggregate planning), allowing users to prioritize stability where it matters most for downstream decisions.

Key Contributions

• Distribution-Free Stability: The paper introduces the first method to directly optimize for stability in distribution-free probabilistic forecasts, removing the need for Gaussian assumptions or data transformations.
• Joint Optimization: It demonstrates that stability can be integrated as an explicit objective during model training via a composite loss function, rather than relying solely on post-processing.
• Flexible Targeting: The use of spline-based quantile functions enables the selective stabilization of specific distributional regions (tails vs. center) through quantile-weighted penalties.
• Theoretical Framework: The work extends the concept of "forecast churn" minimization from point forecasts and Gaussian distributions to general probabilistic settings.

Experimental Results
The method was evaluated on two datasets with distinct statistical properties: the M4 monthly dataset (smooth series) and the M5 items dataset (containing intermittent and lumpy demand).

• Quality-Stability Trade-off: The results show that StableSQF can significantly reduce forecast instability (measured by the 1-Wasserstein distance) with only a marginal loss in forecast quality (measured by CRPS). For instance, a 1% increase in CRPS yielded approximately a 30% reduction in instability for M4 and 65% for M5.
• Pareto Efficiency: StableSQF models generally lie on or near the Pareto frontier of quality vs. stability. While post-processing methods (like interpolation of forecasts from different origins) can achieve extreme stability, they often incur higher quality losses than StableSQF for moderate stability levels.
• Targeted Stabilization: Experiments using quantile-weighted penalties confirmed that the method can effectively direct stabilization efforts. Using a tail-focused penalty ($sW^t_1$) significantly reduced tail instability but at the cost of tail forecast quality, while center-focused penalties ($sW^c_1$) balanced stability across the distribution.
• Comparison to Baselines: StableSQF outperformed standard baselines (ETS, seasonal naive, mean) and post-processing stabilized variants in terms of the efficiency of the quality-stability trade-off.

Significance and Claims
The authors claim that their work provides a practical solution to the inherent trade-off between forecast accuracy and stability. They emphasize that while forecast updates generally improve accuracy, the resulting instability can be operationally costly and detrimental to user trust. By enabling the direct optimization of stability within a distribution-free framework, StableSQF allows practitioners to:

1. Control the Trade-off: Explicitly tune the balance between accuracy and stability based on the specific costs of plan revisions versus the benefits of accuracy.
2. Focus Efforts: Prioritize stability in specific parts of the forecast distribution relevant to downstream decision-making (e.g., stabilizing upper quantiles for inventory management).
3. Reduce Complexity: Avoid the need for complex post-processing pipelines by baking stability directly into the model architecture.

The paper concludes that while completely eliminating instability may require post-processing, StableSQF offers a flexible, built-in mechanism to generate inherently more stable forecasts with minimal quality degradation, making it a valuable tool for operational planning in dynamic environments.