Technical Summary: Online Forecast Reconciliation Using Linear Models

Problem Statement

Forecast reconciliation is a technique used to ensure coherence among forecasts of time series that possess hierarchical structures. These structures impose linear constraints (e.g., the sum of sub-regional loads must equal the total regional load). While traditional reconciliation methods (such as Optimal Combination) have been successful in offline settings, they often rely on Ordinary Least Squares (OLS) or Generalized Least Squares (GLS) that require batch processing of historical data.

In practical applications, such as district heating load forecasting, forecasts must be updated regularly as new observations become available. Existing online and adaptive methods for hierarchical forecasting often rely on specific shrinkage estimators (e.g., MinT with Ledoit-Wolf shrinkage) but may lack a unified framework for recursive parameter estimation, uncertainty quantification, and the explicit modeling of error correlations within a multivariate setting. The challenge lies in developing a framework that supports online, adaptive learning for hierarchical time series while providing rigorous uncertainty quantification for both parameters and forecasts.

Methodology

1. Formalization of Hierarchies via Graphs

The authors formalize hierarchical structures as directed trees $H = (Y, E)$, where nodes represent system states and edges define summation constraints.

• Summation Matrix ($S$): Defined based on the reachability of bottom-level nodes (leaves) from top-level nodes. The hierarchy constraints are expressed as $Sy_{bot} = y_{top}$, where $y_{bot}$ and $y_{top}$ are vectors of bottom and top-level states, respectively.
• Reconciliation Goal: The objective is to find a projection matrix $P$ such that reconciled forecasts $\tilde{y}_t = S P \hat{y}_t$ satisfy the hierarchy constraints, where $\hat{y}_t$ represents base forecasts.

2. Multivariate Linear Model with Matrix Normal Distribution

The core of the proposed framework treats forecast reconciliation as a linear model (LM) problem.

• Model Formulation: Instead of assuming independent residuals, the authors utilize a multivariate linear model where residuals follow a Matrix Normal distribution:
  $$Y = X\theta + E, \quad E \sim MN(0, U, V)$$
  Here, $Y$ is the response matrix (bottom-level base forecast errors), $X$ is the design matrix constructed from the difference between top and bottom base forecasts, $\theta$ represents the reconciliation weights, $U$ is the row covariance (temporal correlation), and $V$ is the column covariance (cross-sectional correlation).
• Connection to Møller et al. (2024a): The framework builds upon the result that reconciliation can be stated as an LM in the bottom-level base forecast errors. This allows the application of standard LM theory to hierarchical reconciliation.

3. Regularization and Ridge Regression

Parameter estimation is posed as a ridge regression problem to handle collinearity and ensure stability.

• Objective Function: The authors minimize a weighted Mahalanobis-type norm:
  $$J(\theta) = \|X\theta - Y\|_{P,R}^2 + \|\theta - \theta_0\|_{Q,R}^2$$
  This formulation connects ridge regression, Bayesian estimation (Maximum A Posteriori), and shrinkage methods.
• Shrinkage Interpretation: By choosing specific priors for $\theta_0$ and $Q$, the ridge solution recovers the MinT shrinkage estimator. The authors also explore simpler priors, such as $\theta_0 = 0$, which corresponds to a bottom-up approach in the limit of strong regularization.

4. Recursive Inference for Online Settings

To enable online forecasting, the authors derive recursive update equations inspired by Recursive Least Squares (RLS).

• Assumptions: The method assumes independent residuals over time ($U$ is diagonal) and uses exponential weighting ($P$ is a diagonal matrix with decaying weights $\lambda^{t-s}$).
• Update Equations: The sufficient statistics ($K_t, L_t, H_t$) are updated recursively at each time step $t$:
  $$K_{t+1} = \lambda K_t + x_{t+1}x_{t+1}^\top$$
  $$L_{t+1} = \lambda L_t + x_{t+1}y_{t+1}^\top$$
  $$H_{t+1} = \lambda^2 H_t + u_{t+1}x_{t+1}x_{t+1}^\top$$
  This allows for efficient parameter updates without reprocessing the entire history.
• Uncertainty Quantification: The framework provides the predictive distribution for new observations, including the covariance of the reconciled forecasts, derived from the matrix normal properties.

5. Implementation

The methodology is implemented in the open-source Python package PyOnlineForecast. The implementation treats reconciliation as a composite transformation that:

1. Splits base forecasts into top and bottom levels.
2. Constructs design and response matrices (handling back-shifting for temporal hierarchies).
3. Applies the recursive ridge regression to predict bottom-level forecast errors.
4. Aggregates and shifts predictions to produce coherent reconciled forecasts.

Key Contributions

1. Unified Framework: The paper presents a framework that unifies hierarchical forecast reconciliation with multivariate linear models, utilizing matrix normal distributions to characterize residuals.
2. Online and Adaptive Algorithm: It derives a recursive inference scheme based on ridge regression, enabling efficient online updates of reconciliation parameters and uncertainty estimates.
3. Graph-Theoretic Formalization: Hierarchies are formally defined using graph theory and transitive closure to construct the summation matrix, providing a rigorous structural definition.
4. Open Source Implementation: The authors provide the PyOnlineForecast package, offering a practical tool for implementing these methods, loosely based on the R onlineforecast package.
5. Theoretical Connections: The paper explicitly discusses the connections between ridge regression, Bayesian estimation, and shrinkage estimators (MinT) in the context of hierarchical reconciliation.

Results: District Heating Load Forecasting Case Study

The proposed methodology was demonstrated on a case study involving district heating (DH) load forecasting for three areas in Aarhus, Denmark, using a temporal hierarchy (hourly, 6-hour, 12-hour, and 24-hour aggregations).

• Performance Improvement: Forecast reconciliation consistently improved accuracy across all regions, aggregation levels, and forecast horizons compared to base forecasts. The Relative Reduction in Root Mean Squared Error (RRMSE) was positive in all tested scenarios.
• Update Frequency: The study compared updating parameters every hour (Main Case) versus every 24 hours (Cases 1-3).
  • Hourly Updates: Provided the best predictive performance and stable variance estimates.
  • Daily Updates: Resulted in less stable variance estimates and generally lower accuracy. The authors attribute this to the reduced frequency of parameter adaptation to system changes.
• Regularization Effects:
  • For short horizons, higher regularization (pushing weights toward zero/bottom-up) improved performance, likely because base forecasts for short horizons are already accurate and dynamic.
  • For longer horizons, lower regularization yielded better results, as the reconciliation process effectively leveraged the structure of the hierarchy to correct base forecast errors.
• Uncertainty: The recursive scheme successfully tracked the variance of prediction errors, though the authors note that estimation uncertainty of the variance matrix $V$ itself was not fully propagated into the predictive distribution in the current implementation.

Significance and Claims

The paper claims that its primary significance lies in providing a reference for implementing forecast reconciliation via multivariate linear models in an online setting.

• Practical Utility: The framework addresses the practical need for regular forecast updates in dynamic systems (like district heating) where data arrives sequentially.
• Flexibility: By formulating reconciliation as a regularized linear model, the approach allows for the use of various regularization techniques (e.g., Lasso, Elastic Net) and uncertainty quantification methods beyond standard MinT.
• Modesty: The authors acknowledge limitations, such as the assumption of independent residuals in the recursive derivation and the potential for artificial autocorrelation in temporal hierarchies due to repeated observations. They suggest that future work could explore more general covariance structures (e.g., tensor normal distributions) and more sophisticated variance estimation priors.

The work demonstrates that online hierarchical forecasting is feasible and beneficial, particularly when parameter updates are frequent and regularization is tuned according to the forecast horizon and aggregation level.