Evaluation of Policy-Relevant Forecasting Advantage of Machine and Deep Learning Models over Classical Econometrics: Evidence from Ghanaian Headline Inflation

Technical Summary: Evaluation of Policy-Relevant Forecasting in Ghana

Problem Statement
This study addresses a critical question for central banks operating inflation-targeting regimes in small, open, commodity-exposed economies: do modern machine learning (ML) and deep learning (DL) models offer a genuine forecasting advantage over classical econometric benchmarks for headline inflation? While the literature often presumes that greater model flexibility translates to superior predictive accuracy, this assumption has not been rigorously tested for Ghana's headline inflation under strict, leakage-free conditions that span a genuine structural break. The specific context is Ghana's 2022–2023 inflation surge, where year-on-year inflation rose from approximately 10% to a peak exceeding 50%, driven by currency depreciation, fiscal pressures, and global commodity shocks. The challenge lies in forecasting through such extreme volatility, a scenario where accurate predictions are most vital for monetary policy yet where models are most severely tested.

Methodology
The authors constructed a strictly one-step-ahead forecasting matrix using 280 months of data (January 2000–April 2023) from the Bank of Ghana, covering 24 core macro-monetary variables including interest rates, monetary aggregates, and international commodity prices.

• Data Integrity and Leakage Control: A rigorous feature-engineering protocol was employed to ensure no data leakage. Every predictor in the feature matrix reflects information available strictly one month prior to the forecast target ($t-1$). This includes lagged levels/log-levels of variables, rolling means of the target computed over $t-3$ to $t-1$, and calendar months. No contemporaneous regressors or look-ahead information were used.
• Chronological Split: The data was split chronologically without shuffling into three non-overlapping windows:
  • Training (200 months): February 2001–September 2017. Used for model fitting and hyperparameter tuning via five-fold walk-forward cross-validation.
  • Validation (24 months): October 2017–September 2019. Used solely to compare tuned candidates on unseen data.
  • Test (42 months): October 2019–April 2023. Touched exactly once after model selection. This window was deliberately chosen to include the 2022–2023 inflation surge, serving as a "stress test" rather than a calm holdout.
• Candidate Models: Nine models were evaluated:
  1. Baselines: Random Walk and Seasonal Naive.
  2. Classical Benchmark: ARIMA(3,1,0), selected via Akaike Information Criterion (AIC) and re-estimated one-step-ahead.
  3. Regularized Linear Models: Ridge and Lasso regression.
  4. Tree Ensembles: Random Forest and XGBoost.
  5. Kernel Method: Support Vector Regression (SVR) with a radial basis function kernel.
  6. Deep Learning: A shallow feed-forward neural network (MLP) with a single hidden layer. (Note: More complex architectures like LSTMs were deliberately excluded due to the modest sample size of 200 training months, which would likely lead to overfitting).
• Evaluation Metrics: Performance was ranked primarily by Root-Mean-Squared Error (RMSE) on the test set, supplemented by Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), and $R^2$. Statistical significance of differences was assessed using Diebold–Mariano tests.

Key Results
Contrary to the presumption that flexible models dominate, the study found that the classical ARIMA(3,1,0) benchmark achieved the lowest test-set RMSE (2.41 percentage points), narrowly outperforming the Random Walk (2.65) and Lasso (3.41).

• Performance Hierarchy: Every nonlinear machine learning model (Ridge, Random Forest, XGBoost, SVR, and the neural network) underperformed the simple baselines out-of-sample. The Seasonal Naive baseline performed the worst (RMSE 15.08), reflecting the lack of simple 12-month seasonality in the surge period.
• Statistical Significance: Diebold–Mariano tests revealed that ARIMA's advantage over the Random Walk and Lasso was not statistically significant ($p > 0.05$). However, ARIMA's advantage over all other nonlinear models was statistically significant.
• Overfitting Diagnostics: A stark contrast was observed between training and test performance for flexible models. For instance, XGBoost achieved a near-perfect training RMSE (0.03) but degraded to a test RMSE of 9.04. Similarly, Random Forest showed a large gap (0.83 to 7.58). This indicates that these models memorized training-period idiosyncrasies rather than learning generalizable structures.
• Structural Break Handling: During the 2022–2023 inflation surge, the ARIMA and Random Walk models tracked the actual inflation path most closely, lagging only slightly behind the sharp rise. In contrast, machine learning models drifted further away, often underestimating the peak (e.g., Lasso predicted 45.7% vs. the actual 54.1% in December 2022).

Key Contributions

1. Methodological Rigor: The study establishes a leakage-free, one-step-ahead forecasting design for Ghanaian inflation that strictly excludes contemporaneous information and utilizes walk-forward validation, addressing a gap in previous literature that often relied on in-sample decomposition or favorable holdout periods.
2. Adversarial Stress Testing: By evaluating models specifically across the 2022–2023 inflation surge, the study provides evidence on how models perform during a genuine structural break, rather than during calm economic periods.
3. Empirical Evidence on Model Complexity: The paper provides robust evidence that for highly persistent series like headline inflation in a small sample context (200 training months), simpler, constrained models (ARIMA, Random Walk) degrade more gracefully under stress than high-capacity, flexible models which are prone to overfitting.

Significance and Claims
The authors claim this is a genuine, policy-relevant finding rather than a specification failure. They argue that headline inflation is highly persistent, and models that best exploit this persistence (ARIMA, Random Walk) outperform those with the greatest functional flexibility when facing a structural break.

• Policy Implication: For the Bank of Ghana and similar institutions, the study suggests that operational forecasting should not default to the most sophisticated available method. A well-specified, transparent, and low-cost ARIMA model remains a superior primary choice for one-step-ahead headline inflation forecasting compared to the tested ML alternatives.
• Role of Machine Learning: The authors do not claim ML has no role but suggest it should be used as an auxiliary cross-check or ensemble input rather than a primary forecast generator. They posit that ML might add value if applied to richer, higher-frequency feature sets, regime-aware architectures, or hybrid models that combine classical trend components with ML residuals.
• Limitations: The authors modestly qualify their conclusions, noting that the sample size is modest for deep learning, the feature set is limited to published statistics (excluding survey expectations or textual data), and the evaluation focused on point forecasts rather than density or interval calibration. They acknowledge that their findings apply specifically to the tested shallow neural network and may not generalize to all deep architectures or different inflation components (e.g., food or core inflation).