Optimization of dynamic crypto portfolios via deep reinforcement learning against supervised learning under transaction costs
This paper demonstrates that a Deep Reinforcement Learning approach using PPO significantly outperforms supervised learning models like LSTM, Random Forest, and XGBoost in optimizing a dynamic cryptocurrency portfolio under transaction costs, achieving superior risk-adjusted returns and market neutrality by prioritizing trading efficiency over directional accuracy.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
The world of digital currency operates on a scale of movement that defies traditional financial intuition. Unlike the relatively steady markets of stocks or bonds, the cryptocurrency landscape is defined by extreme volatility, where prices can swing wildly in a single day, and the rules governing those prices change without warning. For decades, investors have relied on mathematical models designed to balance risk and reward, but these classic tools often fail when applied to assets that do not behave like normal economic goods. They assume that market patterns repeat themselves and that relationships between different assets remain constant, assumptions that crumble in the chaotic environment of digital finance. To navigate this turbulence, researchers have turned to artificial intelligence, hoping to build systems that can learn from the noise rather than trying to predict the future with perfect precision. The core challenge is not just finding a profitable trade, but doing so without losing money to the hidden costs of trading itself, such as fees that eat away at profits every time a portfolio is adjusted.
In a recent study, a team of researchers from universities in Morocco tackled this specific problem by testing a new approach to managing a portfolio of ten major cryptocurrencies. They set out to see if a type of artificial intelligence known as deep reinforcement learning could outperform established machine learning methods that have been the standard for years. The researchers created a digital environment where an artificial agent had to make daily decisions about how to split its money among ten different coins, including Bitcoin, Ethereum, and others, while also keeping a portion in a safe, zero-risk asset similar to cash. Crucially, every time the agent decided to move money from one coin to another, it had to pay a small fee, mimicking the real-world cost of trading. The goal was to see which method could grow the portfolio's value the most while surviving the inevitable market crashes and avoiding the trap of over-trading.
The team compared their new approach against three well-known supervised learning models, which work by first trying to predict the future price of an asset and then deciding what to buy based on that prediction. These traditional models included Long Short-Term Memory networks, which are good at remembering past patterns; Random Forests, which use many decision trees to find answers; and XGBoost, a powerful tool for spotting complex relationships in data. The researchers found that these traditional methods struggled significantly when transaction fees were introduced. Because these models focus on predicting the exact direction of a price, even a tiny error in their prediction caused them to trade too frequently. This constant buying and selling, driven by slight miscalculations, meant that the fees quickly ate up any potential profit, leaving the portfolios flat or even in the red. One of the traditional models actually lost money, failing to cover the cost of the trades it made.
In contrast, the deep reinforcement learning agent, which was trained using a method called Proximal Policy Optimization, took a fundamentally different approach. Instead of trying to guess the next price, this agent learned directly from the consequences of its actions. It was rewarded for making money but penalized heavily for moving its money around unnecessarily. This taught the agent a vital lesson: it was better to hold steady than to trade unless the expected gain was large enough to justify the fee. When the market turned dangerous, this agent learned to retreat into its safe cash position, protecting its capital from the worst of the downturns. When the market was calm, it stayed invested to capture growth. This strategy allowed it to navigate the volatile period from 2021 to 2024 with remarkable efficiency.
The results of the experiment were stark. The deep reinforcement learning agent achieved an annual return of 37.22 percent, far outpacing the other methods. While the traditional models managed returns ranging from a modest 6.75 percent to a negative 1.48 percent, the new agent not only grew the portfolio but also managed risk effectively. It suffered a maximum loss of 28.74 percent during the worst market dips, which was less severe than the losses experienced by the other strategies. Perhaps most importantly, the agent proved to be market neutral, meaning its success did not depend on the overall market going up or down. It generated a specific value known as alpha, which represents the skill of the strategy itself, independent of the general market trend. The traditional models, by comparison, were heavily tied to the market's direction and failed to generate this extra value, often destroying wealth rather than creating it.
This study suggests that the future of managing volatile digital assets may lie in systems that understand the cost of action, not just the promise of reward. By embedding the reality of transaction fees directly into the learning process, the researchers created an agent that behaves more like a prudent human investor than a reactive machine. It learned that patience and discipline are often more profitable than constant activity. While the study was conducted using historical data and simulations, the findings offer a compelling argument for moving away from simple prediction models toward systems that optimize for the entire trading experience, including the friction of the market itself. The success of this approach highlights that in a chaotic financial world, the ability to know when not to trade is just as valuable as knowing when to buy.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.