Hierarchical Bayes meets hierarchical forecasting: A flexible framework for level-focused forecasts

Technical Summary: Hierarchical Bayes meets Hierarchical Forecasting

Problem Statement
Hierarchical forecasting is essential for decision-making in complex systems where forecasts at lower levels must aggregate coherently to higher levels (e.g., regional demand summing to national totals). Current state-of-the-art methods, such as Minimum Trace (MinT) reconciliation, typically operate as a post-processing step. They generate independent base forecasts and then reconcile them to ensure coherence. This approach has several limitations:

1. Separation of Estimation and Coherence: The requirement for coherence is not incorporated into the underlying parameter estimation, meaning the model does not learn from the hierarchical structure during training.
2. Covariance Estimation Challenges: Probabilistic reconciliation (e.g., extending MinT to distributions) requires estimating the covariance matrix of forecast errors, which is computationally difficult and often unstable for multi-step horizons.
3. Lack of Decision Focus: Existing methods generally treat all levels equally or optimize for global error reduction, failing to account for scenarios where specific levels are more critical to decision outcomes or where strict coherence is less important than accuracy at a specific level.
4. Calibration Issues: Post-hoc reconciliation can force coherence at the expense of forecast calibration, leading to poorly calibrated probabilistic forecasts even if the base model is well-specified.

Methodology
The authors propose a fully Bayesian hierarchical forecasting framework that integrates information sharing, coherence constraints, and decision-focused weighting directly into the model training process. The methodology consists of three core components:

1. Bayesian Hierarchical Priors:
   Instead of estimating parameters for each series independently, the authors employ a hierarchical prior structure where parameters ($\theta_i$) for individual series are drawn from a population distribution defined by hyperparameters ($\phi$). This facilitates partial information pooling, allowing series at the same level and across different levels (parent-child nodes) to share information. This is particularly beneficial for series with noisy or incomplete data, as they are informed by related series.

2. Soft Coherence Penalty:
   Rather than enforcing hard constraints, the framework introduces a soft penalty on incoherence within the likelihood function. The difference between the sum of lower-level series and the corresponding upper-level series ($d_t$) is modeled as a Gaussian distribution $N(0, \sigma^2_{coh})$.

   • The likelihood is multiplied by this penalty term: $p(d|\sigma^2_{coh}I_T)$.
   • The parameter $\sigma_{coh}$ controls the strictness of the penalty. A smaller $\sigma_{coh}$ enforces tighter coherence.
   • This "soft" approach allows the model to balance accuracy against coherence, avoiding unrealistic predictions when the model is misspecified. It eliminates the need to estimate complex multi-step covariance matrices required by MinT.

3. Level Weighting (Power Posteriors):
   To address the varying importance of different levels for decision-making, the authors apply a weighting factor $\lambda_i$ to the likelihood of specific series or levels.

   • By setting $\lambda_i > 1$, the model prioritizes accuracy at specific levels (e.g., the total national level or a specific travel purpose).
   • This is implemented using a power posterior approach, effectively up-weighting the data from the focused level relative to others.
   • This allows the global model to be optimized for specific decision outcomes while maintaining soft coherence constraints.

The full posterior distribution combines the hierarchical prior, the weighted likelihood, and the coherence penalty.

Key Contributions

• Integrated Framework: The paper introduces a unified Bayesian framework that performs parameter estimation, information pooling, and coherence enforcement simultaneously, rather than as a post-hoc correction.
• Soft Coherence: It proposes a distributional constraint that softly penalizes incoherence, offering a flexible trade-off between model fit and aggregation constraints without requiring covariance matrix estimation for multi-step horizons.
• Decision-Aware Focusing: The methodology introduces a mechanism to differentially weight hierarchical levels based on their relevance to decision outcomes, optimizing the hierarchy for accuracy at specific levels of interest.
• Diagnostic Capability: The approach serves as a diagnostic tool; if the model cannot achieve coherence even with tight penalties, it indicates model misspecification rather than just a reconciliation failure.

Results
The framework was evaluated on both simulated data (well-specified and misspecified ARMA/ETS models) and real-world Australian domestic tourism data (a 5-level hierarchy of regions, zones, states, and travel purposes).

• Coherence: The Bayesian hierarchical approach inherently reduced mean and distributional incoherence compared to independent models, even before applying penalties. Applying the soft penalty further reduced incoherence significantly.
• Accuracy (Simulated Data):
  • In well-specified models, independent Bayesian models performed best, as the hierarchical shrinkage slightly biased estimates away from the true parameters. However, the Bayesian approach with coherence penalties outperformed MinT in terms of probabilistic coverage.
  • In misspecified models, the hierarchical approach showed benefits, particularly when weighting levels with better data quality (e.g., bottom levels). Weighting the top level in a misspecified scenario worsened overall performance, highlighting the importance of data/model quality in the focused level.
• Accuracy (Australian Tourism Data):
  • The hierarchical Bayesian models outperformed independent Bayesian models and MinT reconciliation across most metrics (RMSSE, ES, CRPS).
  • Level Weighting: Models weighted toward specific levels (e.g., "Total" or "Purposes") achieved the best performance at those specific levels and often improved performance at neighboring levels compared to unweighted models.
  • Calibration: MinT reconciliation resulted in significant under-coverage (poor calibration) at upper levels. The proposed Bayesian approach maintained better calibration, though level-weighted models showed slight under-coverage at focused levels due to increased confidence, which remained within acceptable bounds (within 5% of the 95% target).
  • Misspecification Detection: The inability of the model to converge with very tight coherence penalties on the tourism data suggested model misspecification (likely due to unmodeled covariates in the ETS model), demonstrating the framework's utility in diagnosing model limitations.

Significance and Claims
The paper claims that its primary significance lies in shifting hierarchical forecasting from a post-processing reconciliation task to an integrated modeling process. By embedding coherence and decision priorities directly into the Bayesian parameter estimation:

1. It improves information sharing both within and across hierarchical levels, leading to better parameter estimates, especially in data-scarce or noisy scenarios.
2. It provides flexible control over the trade-off between accuracy and coherence, allowing forecasters to prioritize specific decision-relevant levels without rigidly forcing coherence that degrades forecast quality.
3. It avoids the computational and statistical pitfalls of covariance matrix estimation for multi-step probabilistic reconciliation.
4. It offers a diagnostic perspective: if a soft penalty cannot achieve coherence, it signals that the underlying model is misspecified, prompting a re-evaluation of the model choice rather than forcing a suboptimal reconciliation.

The authors conclude that while their approach excels in balancing accuracy and coherence, it is most effective when the underlying model is reasonably well-specified or when focusing on levels with high-quality data. They acknowledge that in small sample sizes, the weighting approach may lead to some posterior miscalibration, but argue that the gains in focused predictive performance outweigh this cost.