A Comparative Analysis of Machine Learning Models for Long and Short-Term Forecasting of the Egyptian Stock Market: A Focus on EGX30

Technical Summary: A Comparative Analysis of Machine Learning Models for Long and Short-Term Forecasting of the Egyptian Stock Market

Problem Statement
The Egyptian stock market, specifically the EGX30 index, represents a dynamic yet under-researched emerging financial arena. While extensive literature exists on global stock markets, there is a critical gap in understanding the specific trends and dynamics of developing markets like Egypt. Investors in this region often face challenges due to a lack of technical analysis expertise and the inherent risks of high-return markets. The primary objective of this study is to bridge this gap by developing and evaluating precise predictive models tailored to the EGX30. The research addresses the need for robust tools that can forecast both short-term (up to one week) and long-term (up to two months) market trends to aid in risk management and investment strategy formulation.

Methodology
The study employs a comprehensive comparative framework utilizing historical EGX30 data. The methodology is structured around three core components:

1. Data Preprocessing and Feature Engineering:

   • The dataset was segmented into temporal features (day of week, month, year, etc.), though correlation analysis revealed these alone were insufficient due to low correlation with stock prices.
   • To capture historical context and temporal dependencies, the study introduced rolling window lag features.
   • Technical Indicators were engineered, including Simple Moving Averages (SMA), Exponential Moving Averages (EMA), Bollinger Bands, Relative Strength Index (RSI), and Commodity Channel Index (CCI).
   • A lag function was applied to historical data to enable the expectation of trends over multiple days, respecting the inherent temporal structure of financial data.

2. Model Selection:
   The research evaluated a diverse array of seven machine learning algorithms and two deep learning architectures:

   • Traditional ML: K-Nearest Neighbours (KNN), Decision Tree Regressor, Random Forest, Extra Trees, Adaptive Boosting (AdaBoost), Light Gradient Boosted Machine (LightGBM), and Extreme Gradient Boosting (XGBoost).
   • Deep Learning: Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU).
   • Ensemble Approach: A blending technique was applied to the top-performing models to create an ensemble predictor, particularly for long-term horizons.

3. Evaluation Framework:

   • Time Horizons: Models were tested on four distinct prediction intervals: one-day, one-week, one-month, and two-months.
   • Metrics: Performance was quantified using Root Mean Squared Error (RMSE), Mean Absolute Percentage Error (MAPE), and the Coefficient of Determination ($R^2$).
   • Optimization: Deep learning models utilized the Adam optimizer for efficient convergence.

Key Results
The experimental results revealed distinct performance characteristics depending on the prediction horizon:

• One-Day Predictions: XGBoost emerged as the superior model, achieving the highest accuracy with an RMSE of 0.0163 and an $R^2$ of 0.993. LightGBM and Random Forest also demonstrated robust performance, though XGBoost was marginally more effective.
• One-Week Predictions: The GRU model outperformed all others, achieving the lowest RMSE (0.0406) and the highest $R^2$ (0.961). Random Forest and Extra Trees also performed well, but GRU's ability to capture seasonality and intricate relationships proved decisive.
• Long-Term Predictions (One and Two Months):
  • GRU continued to demonstrate superior performance, maintaining the best RMSE and $R^2$ scores across both the one-month and two-month horizons.
  • KNN yielded a surprising result, significantly outperforming its short-term results to become a top-tier performer in long-term forecasts (ranking third in one-month and second in two-month predictions).
  • Ensemble Technique: The ensemble model, which blended the best-performing individual models, achieved the most significant improvements in long-term forecasting. Notably, in the two-month prediction scenario, the ensemble model's RMSE was approximately five times lower than that of the standalone GRU model, highlighting the efficacy of ensemble methods for extended horizons.

Significance and Contributions
The study claims several significant contributions to the field of financial forecasting in emerging markets:

1. Context-Specific Insights: It provides the first detailed comparative analysis of ML and DL models specifically for the EGX30, offering actionable insights for investors navigating the unique dynamics of the Egyptian market.
2. Horizon-Dependent Model Selection: The research establishes that no single model is universally optimal. It demonstrates that XGBoost is best suited for high-frequency, one-day predictions, while GRU and Ensemble techniques are essential for capturing the complexities of weekly and monthly trends.
3. Re-evaluation of KNN: The study challenges the conventional view of KNN as a purely short-term or simple algorithm, revealing its unexpected and enduring relevance for long-term trend prediction in this specific market context.
4. Ensemble Superiority: The paper highlights that while individual deep learning models are powerful, ensemble blending significantly enhances robustness and accuracy, particularly for long-term forecasts where individual models struggle with higher error rates.

Limitations and Future Directions
The authors acknowledge that fine-tuning numerous models requires substantial computational resources, which can limit the depth of hyperparameter optimization. Future work is proposed to include:

• Integration of Transformer architectures, which have shown promise in time-series prediction.
• Incorporation of Sentiment Analysis derived from global/local business calendars, news, and social media posts by public figures to enrich the predictive features.

In conclusion, this study validates the utility of advanced machine learning and deep learning techniques for the Egyptian stock market, emphasizing that model selection must be aligned with the specific time horizon of the investment strategy.