Forecasting accuracy to portfolio value: Model risk and regime dependence in an emerging equity market
This study analyzes Moscow Exchange equities to demonstrate that while CatBoost offers the best statistical forecast accuracy, PatchTST delivers superior portfolio performance, and no model selection or regime-switching strategy can reliably overcome the fundamental disconnect between forecasting precision and cross-sectional ranking ability in emerging markets.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Investors who manage large pools of money face a constant, quiet dilemma: they must guess which stocks will rise and which will fall, but they can never be sure if their guessing method is the right one. This is not just a matter of bad luck; it is a fundamental problem of uncertainty. On one level, an investor worries that their estimate of a stock's future return is wrong. On a deeper level, they worry that the entire mathematical framework they are using to make that estimate might be flawed. In stable times, simple rules often work well, but when the economic environment shifts—when inflation spikes, markets become volatile, or prices crash—those same rules can fail. The question becomes whether a more complex, flexible method can adapt to these changing conditions better than a straightforward one, and whether a method that is statistically accurate at predicting numbers actually leads to better investment results.
A team of researchers set out to test these questions in the Russian stock market, a place where economic conditions have shifted dramatically over the last decade. They gathered data on nearly three hundred companies traded on the Moscow Exchange, covering a period from late 2016 through 2025. This timeframe included years of steady inflation, the global shock of the pandemic, and a period of intense geopolitical tension and monetary tightening. The researchers wanted to see if the relationship between a company's characteristics and its future returns changed depending on whether the market was calm or stressed. To do this, they built a controlled experiment where eight different forecasting methods competed against each other. These methods ranged from simple linear equations, which assume a steady relationship between factors and returns, to sophisticated machine learning algorithms capable of detecting complex, non-linear patterns and adapting to new data.
The researchers fed all eight models the exact same information: data on company size, value, momentum, and liquidity, combined with indicators of market stress like high inflation and sharp price drops. They then asked each model to predict the returns of stocks over the next three months. Crucially, they did not let the models compete in a free-for-all; instead, they forced every model to use the same rules for building a portfolio. They selected the top twenty-five stocks with the highest predicted returns, capped the amount of money invested in any single stock, and managed risk in an identical way. This ensured that any difference in the final results came purely from the quality of the forecasts, not from how the money was allocated.
The results revealed a surprising disconnect between statistical accuracy and real-world value. One of the most advanced machine learning models, known as CatBoost, produced the smallest average errors when predicting returns. It was the most precise at guessing the numbers. However, when the researchers looked at whether the models could correctly rank the best stocks against the worst, all eight methods performed poorly. None of them could consistently identify the top performers better than random chance. This finding suggests that even the most sophisticated algorithms struggle to find a reliable signal in the noise of an emerging market, especially when that market is undergoing rapid structural changes.
More importantly, the study found that the model with the best statistical accuracy did not produce the best investment portfolio. A different model, based on a sequence-learning architecture called PatchTST, had relatively poor overall accuracy in its predictions. Yet, when its forecasts were used to build a portfolio, it delivered risk-adjusted returns that were competitive with the best-performing strategies and suffered smaller losses during market downturns. This outcome highlights a critical lesson: a model does not need to be perfect at predicting every number to be useful for investing. It only needs to be good enough at identifying the specific stocks that will perform well, even if it gets the rest of the market wrong.
The researchers also tested whether it was possible to switch between models depending on the current market conditions, such as using one model during high inflation and another during market crashes. They created a rule that would automatically select the best-performing model based on recent results or the current state of the economy. While this switching rule produced the highest nominal returns in their backtest, the advantage was not statistically reliable. The results were so close to those of fixed strategies that the difference could easily be due to chance. Furthermore, because the rule was designed after looking at the data, it cannot be considered a proven strategy for the future. The study suggests that while knowing the market is stressed is useful for understanding when forecasts might be unreliable, it does not provide a clear, automatic signal for which mathematical tool to use.
Ultimately, the paper concludes that there is no single "best" model for predicting stock returns in an emerging market. The most flexible and complex models do not automatically outperform simpler ones, and the model that wins in a statistical contest is not necessarily the one that builds the most profitable portfolio. The uncertainty investors face is twofold: they must deal with the inherent difficulty of predicting the future, and they must also accept that they cannot know for certain which method of prediction is the right one for the moment. In this environment, the most robust approach may not be to search for a perfect forecasting machine, but to recognize that all models carry risk and that the value of a model lies not in its average accuracy, but in its ability to guide sound investment decisions under pressure.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.