Statistical and Machine Learning Models for Stock Price and Return Forecasting: An Out-of-Sample Comparison
This study demonstrates that in out-of-sample forecasting of S&P 500 prices and returns, complex machine learning models do not consistently outperform simple benchmarks like the random walk or zero-return assumption, highlighting that increased model complexity does not guarantee improved short-horizon financial predictions.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Predicting the future of the stock market is one of the most persistent challenges in finance, not because the data is hidden, but because the nature of what is being predicted changes the rules of the game. When analysts look at the price of a stock index, they are watching a value that tends to keep moving in the same direction, building on its own history like a heavy train that is hard to stop. When they look at returns, which measure the daily percentage gain or loss, they are watching a value that fluctuates around a stable average, behaving more like a jittery leaf in the wind. For decades, economists have debated whether complex mathematical formulas or modern computer algorithms can outsmart the market. The central question is whether these sophisticated tools can actually find a pattern that simple, common-sense guesses cannot.
A recent study from the University of Nevada, Reno, tackles this question by treating these two types of predictions as entirely separate problems. The researcher, Smaila Amoanu, set out to see if advanced computer models could beat two very simple benchmarks: a "random walk" for predicting the next day's price, and a "zero return" guess for predicting the next day's percentage change. The random walk idea is straightforward: the best guess for tomorrow's price is simply today's price, because the market moves so much that yesterday's news is already baked in. The zero return guess is equally simple: the best guess for tomorrow's percentage change is zero, assuming the market will neither go up nor down significantly. To test this, the study used eleven years of daily data from the S&P 500, a major index tracking five hundred large American companies, and reserved the final year, 2025, as a completely unseen test to see how well the models performed on new ground, while also using the years 2022 through 2025 for an annually re-estimated robustness analysis.
The study compared four different approaches. On the statistical side, it used ARIMA, a classic method that looks for patterns in how prices change over time, and ARMA–GARCH, which adds a layer to account for the fact that market volatility tends to cluster in groups of calm and stormy days. On the machine learning side, it tested Random Forest and XGBoost, two powerful algorithms that build many decision trees to find complex, non-linear relationships in the data. The goal was not to build a perfect trading system, but to see if these methods could consistently beat the simple benchmarks when tested on data they had never seen before.
The results for predicting the actual price level were revealing. The complex machine learning models struggled significantly when the market moved into new territory. During the test year, the S&P 500 rose higher than it had ever been in the eleven years of data used to train the models. When the index climbed past the highest price the computers had ever seen, the machine learning models hit a ceiling. They could not guess that the price would go higher because they had no examples of such high numbers in their training. Their predictions flattened out, stuck near the top of the old range, while the actual price continued to climb. In contrast, the simple random walk and the statistical ARIMA model tracked the rising price much more closely. The ARIMA model did produce slightly smaller errors than the random walk, but the difference was so small that it could easily be due to chance rather than a real advantage.
When the focus shifted to predicting daily returns, the picture became even more complicated. Here, the goal was to guess the percentage change, not the absolute price. The study found that no single model consistently beat the simple zero-return benchmark. In some years, one model might have the lowest error, and in the next year, a different model would take the lead. Sometimes the machine learning models were better at guessing the direction of the move, but they did not get the size of the move right enough to be considered a reliable improvement over just guessing zero. The statistical models that accounted for changing volatility performed well in some metrics but failed to show a stable, long-term advantage over the simple guess.
The study concludes that the difficulty of forecasting depends entirely on what you are trying to predict. For the price level, the market is so persistent that the simplest guess—assuming tomorrow looks like today—is incredibly hard to beat, and complex models can fail badly if the market moves outside the range of their past experience. For returns, the signal is so weak that even the most flexible computer algorithms cannot find a pattern that reliably outperforms a guess of zero. The research suggests that in financial forecasting, adding more complexity does not automatically lead to better predictions. Instead, the most reliable approach is to test models against simple, honest benchmarks and to recognize that when the market moves into new, uncharted territory, the most sophisticated tools may not know what to do.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.