Machine Learning and the Random Walk Puzzle: Forecasting the CAD/USD Exchange Rate with Expanding Window Evaluation and SHAP Interpretability

Technical Summary: Machine Learning and the Random Walk Puzzle in USD/CAD Forecasting

Problem Statement
This study addresses the enduring "Meese-Rogoff puzzle" in international finance: the empirical difficulty of forecasting exchange rates out-of-sample using structural macroeconomic models, which consistently fail to outperform a naïve random walk benchmark. Specifically, the paper investigates whether modern machine learning (ML) ensemble methods can overcome this limitation in forecasting the monthly USD/CAD exchange rate. The authors focus on the USD/CAD pair due to its economic significance for Canada and its status as a commodity-linked currency, hypothesizing that its structural features (e.g., correlation with oil prices) might offer predictable dynamics absent in purely financial currencies.

Methodology
The research employs a rigorous, leakage-free evaluation framework using daily data from the Bank of Canada (January 2017 – May 2026), resampled to 113 monthly observations.

• Data Preprocessing: The daily series was aggregated to monthly means. The dataset was split into a training period (January 2017 – November 2024, $T_0=95$) and a strict out-of-sample test period (December 2024 – May 2026, $H=18$).
• Feature Engineering: To prevent data leakage, all predictors for the ML models were constructed exclusively from information available prior to the forecast origin ($\mathcal{F}_{t-1}$). Features included:
  • Lagged values ($y_{t-1}$ to $y_{t-12}$).
  • Rolling statistics (means and standard deviations) over 3, 6, and 12-month windows.
  • Expanding mean (historical average).
  • First differences and cyclical encodings (sine/cosine) for seasonality.
• Models Evaluated:
  • Benchmarks: Naïve Random Walk (RW) without drift and Exponential Smoothing with Holt-Winters seasonality (ETS).
  • Machine Learning: Linear Regression, Random Forest (RF), Gradient Boosting, XGBoost, and AdaBoost.
• Evaluation Protocol:
  • Expanding Window: Models were re-estimated at each step using an expanding training set to mirror real-time forecasting conditions.
  • Metrics: Performance was assessed using MAE, RMSE, MAPE, SMAPE, and Theil's U statistic.
  • Statistical Testing: The Diebold-Mariano (DM) test was used to determine if forecast errors differed significantly from the random walk benchmark.
  • Interpretability: SHAP (Shapley Additive explanations) values were applied to the best-performing ML model to decompose feature contributions.
• Structural Analysis: The PELT algorithm was used to detect structural breaks, identifying four significant regime shifts corresponding to major macroeconomic events (2018 trade war, 2020 pandemic recovery, 2022 rate hikes, 2024 rate cuts).

Key Results

1. Dominance of the Random Walk: The naïve random walk (equivalent to ARIMA(0,1,0)) remained the most robust benchmark. It achieved the lowest RMSE (0.0158) and MAPE (0.92%) among all models.
2. Limited ML Outperformance:
   • Linear Regression was the only model to statistically outperform the random walk (DM statistic = 3.0585, $p = 0.0071$). The authors attribute this to the model's ability to approximate the random walk with low variance under near-unit-root conditions.
   • Ensemble Models: All complex ML models (Random Forest, Gradient Boosting, XGBoost, AdaBoost) yielded negative DM statistics, indicating they were marginally worse than the random walk, though not always statistically significant at the 5% level.
   • ETS Performance: The traditional ETS model performed the worst, significantly underperforming the random walk, likely due to spurious seasonal adjustments in a series lacking strong seasonality.
3. Feature Importance (SHAP): SHAP analysis of the Random Forest model revealed that short-term lags ($lag\_1$, $lag\_2$) and recent rolling means dominated predictions. $lag\_1$ accounted for ~21% of feature importance. Long-horizon lags and seasonal features contributed minimally, confirming the series behaves largely as a random walk with short-term momentum.
4. Forecast Accuracy: Despite failing to beat the random walk statistically, all ML models achieved "High Accuracy" (MAPE < 10%) and outperformed the ETS benchmark. Random Forest achieved the lowest MAPE (1.17%) among non-random-walk models.

Significance and Claims
The paper claims to provide a direct, statistically rigorous answer to the Meese-Rogoff puzzle for the USD/CAD pair within the specified sample period. Its primary contributions are:

• Reinforcement of the Puzzle: The study confirms that even with advanced ensemble methods and strict expanding-window protocols, exchange rates in developed economies remain largely unpredictable beyond the random walk using only endogenous historical data.
• Methodological Rigor: By utilizing an expanding-window framework and the Diebold-Mariano test, the study avoids common pitfalls of data leakage and overfitting often found in ML forecasting literature.
• Interpretability: The application of SHAP values addresses the "black box" critique of ML in economics, demonstrating that the models are learning economically sensible, near-random-walk behaviors rather than spurious patterns.
• Practical Nuance: While ML models did not beat the random walk, they outperformed traditional statistical models (ETS). The authors suggest that for practitioners, Linear Regression and Random Forest (under expanding windows) may serve as useful conditional forecasting tools, particularly when compared to classical benchmarks, though they caution against interpreting these gains as reliable trading rules.

The authors conclude that the benefits of ML in exchange rate forecasting are likely episodic and model-dependent, heavily constrained by the near-random-walk nature of the data, and that future gains may require incorporating macroeconomic fundamentals (e.g., interest rate differentials, commodity prices) rather than relying solely on historical price data.