Technical Summary: Yield Curve Forecasting using Machine Learning and Econometrics

Problem Statement

The paper addresses the ongoing debate regarding the efficacy of machine learning (ML) and deep learning (DL) methods compared to traditional econometric and time-series approaches for forecasting financial time series. Specifically, the authors focus on the U.S. Treasury yield curve, a critical indicator for bond markets (which exceed equity markets in size) and a predictor of economic recessions. While ML has revolutionized fields like NLP and computer vision, its dominance in time-series forecasting remains disputed. Previous literature, such as the M4 competition, has shown that pure ML models often underperform statistical hybrids or naive methods. Furthermore, there is a lack of comprehensive literature comparing these methods on daily yield curve data, particularly regarding the handling of nonstationary inputs and the performance of modern foundation models and transformers.

Methodology

The study utilizes daily U.S. Treasury constant maturity yields for six tenors (1, 2, 5, 7, 10, and 30 years) from the FRED database, spanning 47 years (1977–2024). This dataset includes six distinct recession periods.

Experimental Setup:

• Forecasting Horizons: The models are evaluated on 15, 20, and 60 business day forecast horizons.
• Validation Strategies: Two cross-validation (CV) setups are employed to prevent data leakage:
  1. Sliding Window: A fixed window of 1,044 business days (approx. 4 years) slides forward by 5 days.
  2. Expanding Window: The initial window is 1,044 days and expands by 5 days for each subsequent iteration.
• Hyperparameter Time Blocks: To account for structural economic shifts, the data is divided into contiguous time blocks (e.g., Block 0: 1977–1986) where hyperparameters are selected once per block and applied to all windows within that block.
• Stationarity: The study explicitly tests both stationary (differenced) and nonstationary (raw) input data to determine which is more appropriate for DL algorithms.
• Metrics: Performance is measured using Root Mean Squared Error (RMSE) and Mean Absolute Percentage Error (MAPE).

Models Evaluated:

• Naive Benchmarks: Mean Forecast, Naive Forecast (Random Walk), and Naive Seasonal Forecast.
• Traditional Econometrics: ARIMA (with auto-selection), DHR-ARIMA (ARIMA with Fourier terms), VAR, VECM, and diff+VAR.
• Classical Machine Learning: Random Forests, XGBoost, and LightGBM (LGBM).
• Deep Learning:
  • Recurrent Neural Networks (RNNs)
  • Transformers: Vanilla Transformer, Informer, PatchTST, Temporal Fusion Transformer (TFT).
  • Foundation Models: TimeGPT (pre-trained, zero-shot), NBEATS, NHITS, DeepAR.

Key Contributions

The paper distinguishes itself through four primary contributions:

1. Comprehensive Comparison: It evaluates a wide spectrum of models, including recent foundation models (TimeGPT) and various transformer architectures, against traditional econometric methods on yield curve data.
2. Data Frequency: Unlike much of the existing literature that relies on monthly data, this study utilizes high-frequency daily data.
3. Stationarity Analysis: It systematically investigates whether deep learning models perform better with stationary or nonstationary inputs, a topic with sparse prior research.
4. Temporal Robustness: The study analyzes performance across multiple forecast horizons and distinct economic time blocks to assess model stability over time.

Results

The empirical analysis yields the following findings:

• Dominance of Traditional Methods: Overall, the Naive Forecast and ARIMA models (both sliding and expanding window configurations) outperform all other methods across the full 47-year sample. The differences in MAPE between the best ARIMA variants and the Naive Forecast are negligible (often <0.01%).
• Machine Learning Performance:
  • Among classical ML methods, LGBM with an expanding window configuration performs the best, though it still fails to surpass ARIMA or the Naive benchmark.
  • Among deep learning methods, TimeGPT (a foundation model) achieves the best performance, followed by RNNs with sliding stationary inputs.
• Stationarity vs. Nonstationarity: There is no universal pattern; the optimal input type depends on the specific model. For instance, RNNs performed better with stationary sliding inputs, while PatchTST and TFT performed better with nonstationary expanding inputs.
• Temporal Variability: While traditional methods dominate the aggregate average, specific deep learning models (e.g., PatchTST and TFT with expanding nonstationary data) outperformed benchmarks in the most recent time block (Block 8, 2017–2023), though the margins were small (e.g., 0.03%–0.05%).
• Forecast Horizons: The superiority of the Naive and ARIMA methods held consistent across 20 and 60-day forecast horizons.

Significance and Claims

The paper concludes that for U.S. Treasury yield curve forecasting, traditional econometric methods (specifically ARIMA) and naive benchmarks remain the most effective approaches.

The authors claim that deep learning models, including advanced transformers and foundation models, generally fail to outperform these simpler methods on this specific task. They suggest that the daily yield curve process is close to a martingale (staying stable day-to-day with occasional sharp jumps), a characteristic well-captured by naive and linear models. The underperformance of deep learning is attributed to potential overfitting and the difficulty of learning from limited, noisy financial data compared to the massive corpora used for foundation models in other domains.

The study emphasizes that while deep learning shows promise in specific time blocks or with specific configurations, the simplicity, robustness, and cost-effectiveness of traditional econometric methods make them the preferred choice for this application. The paper does not claim that ML is useless for finance but rather that, for yield curve forecasting, it has not yet demonstrated a consistent advantage over established statistical baselines.